Add full monorepo: virtual-banker, backend, frontend, docs, scripts, deployment
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
defiQUG
190
README.md
Normal file
190
README.md
Normal file
@@ -0,0 +1,190 @@
|
||||
# Virtual Banker Submodule
|
||||
|
||||
AI digital twin human-like Virtual Banker with full video realism using Unreal Engine server-rendered avatars, real-time voice interaction, and an embeddable widget for portal sites.
|
||||
|
||||
## Features
|
||||
|
||||
- **Embeddable Widget**: Drop-in widget for any portal site
|
||||
- **Real-time Voice**: ASR (Speech-to-Text) and TTS (Text-to-Speech) with streaming
|
||||
- **Full Video Realism**: Unreal Engine server-rendered avatar with PixelStreaming
|
||||
- **Multi-tenant**: Different configs/brands/policies per tenant
|
||||
- **Secure Auth**: JWT/SSO integration
|
||||
- **Accessible**: WCAG-oriented (keyboard, screen reader, captions, reduced motion)
|
||||
- **Observable**: Audit logs, safety rules, analytics
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
virtual-banker/
|
||||
├── backend/ # Go backend services
|
||||
├── widget/ # React/TypeScript embeddable widget
|
||||
├── avatar/ # Unreal Engine avatar service
|
||||
├── database/ # Database migrations
|
||||
└── deployment/ # Docker/Kubernetes configs
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Docker and Docker Compose
|
||||
- Go 1.21+ (for backend development)
|
||||
- Node.js 20+ (for widget development)
|
||||
- PostgreSQL 16+ with pgvector extension
|
||||
- Redis
|
||||
|
||||
### Development Setup
|
||||
|
||||
1. **Start infrastructure** (uses existing postgres/redis from main monorepo):
|
||||
```bash
|
||||
cd deployment
|
||||
docker-compose up -d postgres redis
|
||||
```
|
||||
|
||||
2. **Run database migrations**:
|
||||
```bash
|
||||
cd database
|
||||
psql -U explorer -d explorer -f migrations/001_sessions.up.sql
|
||||
psql -U explorer -d explorer -f migrations/002_conversations.up.sql
|
||||
psql -U explorer -d explorer -f migrations/003_tenants.up.sql
|
||||
psql -U explorer -d explorer -f migrations/004_vector_extension.up.sql
|
||||
psql -U explorer -d explorer -f migrations/005_user_profiles.up.sql
|
||||
```
|
||||
|
||||
3. **Start backend API**:
|
||||
```bash
|
||||
cd backend
|
||||
go run main.go
|
||||
```
|
||||
|
||||
4. **Build and serve widget**:
|
||||
```bash
|
||||
cd widget
|
||||
npm install
|
||||
npm run build
|
||||
# Serve dist/ via CDN or static server
|
||||
```
|
||||
|
||||
## Widget Integration
|
||||
|
||||
### Basic Integration
|
||||
|
||||
Add the widget loader script to your HTML:
|
||||
|
||||
```html
|
||||
<script src="https://cdn.example.com/virtual-banker/widget.js"
|
||||
data-tenant-id="your-tenant-id"
|
||||
data-user-id="user-123"
|
||||
data-auth-token="jwt-token"
|
||||
data-api-url="https://api.example.com"
|
||||
data-avatar-enabled="true"></script>
|
||||
<div id="virtual-banker-widget"></div>
|
||||
```
|
||||
|
||||
### Programmatic Control
|
||||
|
||||
```javascript
|
||||
// Open widget
|
||||
window.VirtualBankerWidgetAPI.open();
|
||||
|
||||
// Close widget
|
||||
window.VirtualBankerWidgetAPI.close();
|
||||
|
||||
// Set context
|
||||
window.VirtualBankerWidgetAPI.setContext({
|
||||
route: '/account',
|
||||
accountId: 'acc-123'
|
||||
});
|
||||
|
||||
// Update auth token
|
||||
window.VirtualBankerWidgetAPI.setAuthToken('new-jwt-token');
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Create Session
|
||||
```
|
||||
POST /v1/sessions
|
||||
{
|
||||
"tenant_id": "tenant-123",
|
||||
"user_id": "user-456",
|
||||
"auth_assertion": "jwt-token"
|
||||
}
|
||||
```
|
||||
|
||||
### Refresh Token
|
||||
```
|
||||
POST /v1/sessions/{id}/refresh-token
|
||||
```
|
||||
|
||||
### End Session
|
||||
```
|
||||
POST /v1/sessions/{id}/end
|
||||
```
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Phase 0: Foundation & Widget ✅
|
||||
- [x] Backend session service
|
||||
- [x] REST API endpoints
|
||||
- [x] Database migrations
|
||||
- [x] Embeddable widget (React/TypeScript)
|
||||
- [x] Basic chat UI
|
||||
- [x] Theming system
|
||||
- [x] Accessibility features
|
||||
|
||||
### Phase 1: Voice & Realtime ✅
|
||||
- [x] WebRTC infrastructure
|
||||
- [x] ASR service integration (mock + interface for Deepgram)
|
||||
- [x] TTS service integration (mock + interface for ElevenLabs)
|
||||
- [x] Conversation orchestrator
|
||||
- [x] Barge-in support
|
||||
|
||||
### Phase 2: LLM & RAG ✅
|
||||
- [x] LLM gateway (mock + interface for OpenAI)
|
||||
- [x] RAG service with pgvector
|
||||
- [x] Tool framework
|
||||
- [x] Banking integrations
|
||||
|
||||
### Phase 3: Avatar System ✅
|
||||
- [x] Unreal Engine setup documentation
|
||||
- [x] Render service structure
|
||||
- [x] Animation controller (visemes, expressions, gestures)
|
||||
|
||||
### Phase 4: Memory & Observability ✅
|
||||
- [x] Memory service
|
||||
- [x] Observability (tracing, metrics)
|
||||
- [x] Safety/compliance filters
|
||||
|
||||
### Phase 5: Enterprise Features (In Progress)
|
||||
- [x] Multi-tenancy support
|
||||
- [ ] Tenant admin console (UI)
|
||||
- [ ] Advanced compliance tools
|
||||
- [ ] Usage analytics dashboard
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Integrate Real Services**: Replace mocks with actual API integrations:
|
||||
- Deepgram or Google STT for ASR
|
||||
- ElevenLabs or Azure TTS for TTS
|
||||
- OpenAI or Anthropic for LLM
|
||||
- Connect to existing banking services
|
||||
|
||||
2. **Complete WebRTC**: Implement full WebRTC signaling and media streaming
|
||||
|
||||
3. **Unreal Setup**: Set up actual Unreal Engine project with digital human
|
||||
|
||||
4. **Testing**: Add unit tests, integration tests, E2E tests
|
||||
|
||||
5. **Production Deployment**: Configure for production with proper secrets, monitoring, scaling
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Architecture](./docs/ARCHITECTURE.md)
|
||||
- [API Reference](./docs/API.md)
|
||||
- [Widget Integration](./docs/WIDGET_INTEGRATION.md)
|
||||
- [Deployment](./docs/DEPLOYMENT.md)
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
Reference in New Issue
Block a user