Apply Composer changes: comprehensive API updates, migrations, middleware, and infrastructure improvements

- Add comprehensive database migrations (001-024) for schema evolution
- Enhance API schema with expanded type definitions and resolvers
- Add new middleware: audit logging, rate limiting, MFA enforcement, security, tenant auth
- Implement new services: AI optimization, billing, blockchain, compliance, marketplace
- Add adapter layer for cloud integrations (Cloudflare, Kubernetes, Proxmox, storage)
- Update Crossplane provider with enhanced VM management capabilities
- Add comprehensive test suite for API endpoints and services
- Update frontend components with improved GraphQL subscriptions and real-time updates
- Enhance security configurations and headers (CSP, CORS, etc.)
- Update documentation and configuration files
- Add new CI/CD workflows and validation scripts
- Implement design system improvements and UI enhancements
This commit is contained in:
defiQUG
2025-12-12 18:01:35 -08:00
parent e01131efaf
commit 9daf1fd378
968 changed files with 160890 additions and 1092 deletions

View File

@@ -0,0 +1,178 @@
# Azure Migration Guide
Guide for migrating from Microsoft Azure to Sankofa Phoenix.
## Overview
Sankofa Phoenix provides a superior alternative to Azure with:
- **Sovereign infrastructure**: Own your hardware
- **Better features**: More flexible, granular, better UX
- **No vendor lock-in**: Open source, self-hosted
- **Lower costs**: No per-user licensing
- **Superior billing**: Per-second granularity vs hourly
## Migration Strategy
### Phase 1: Assessment
1. **Inventory Azure resources**
- List all VMs, storage, networks
- Document configurations
- Identify dependencies
2. **Map to Sankofa equivalents**
- Azure VMs → Phoenix VMs
- Azure Storage → Phoenix Storage (Ceph/MinIO)
- Azure Networking → Phoenix Networking
- Azure AD → Keycloak (Phoenix Identity Spine)
3. **Identify migration blockers**
- Azure-specific services
- Proprietary features
- Licensing dependencies
### Phase 2: Identity Migration
#### From Azure AD to Keycloak
1. **Export users from Azure AD**
```powershell
Get-AzureADUser | Export-Csv users.csv
```
2. **Import to Keycloak**
- Use Keycloak import API
- Map Azure AD groups to Keycloak groups
- Preserve user attributes
3. **Configure SSO**
- Set up SAML/OIDC provider in Keycloak
- Update applications to use Keycloak
- Test authentication flows
### Phase 3: Resource Migration
#### VMs
1. **Export VM configurations**
- Document VM specs
- Export disk images
- Note network configurations
2. **Create in Sankofa**
```graphql
mutation {
createResource(input: {
name: "vm-name"
type: VM
siteId: "site-id"
metadata: {
vcpu: 4
memory: 8192
disk: 100
}
}) {
id
status
}
}
```
#### Storage
1. **Export data from Azure Storage**
- Use Azure CLI or SDK
- Download blobs/files
2. **Import to Phoenix Storage**
- Upload to MinIO/Ceph
- Preserve directory structure
- Update application configs
#### Networking
1. **Document network topology**
- VNets, subnets, NSGs
- Load balancers
- VPN connections
2. **Recreate in Phoenix**
- Use Proxmox networking
- Configure VLANs
- Set up Cloudflare tunnels
### Phase 4: Application Migration
1. **Update API endpoints**
- Change from Azure APIs to Phoenix APIs
- Update authentication
- Test functionality
2. **Update configurations**
- Environment variables
- Connection strings
- Service endpoints
3. **Deploy to Phoenix**
- Use GitOps (ArgoCD)
- Deploy via Crossplane
- Verify functionality
## Feature Mapping
| Azure Service | Sankofa Phoenix Equivalent | Notes |
|---------------|----------------------------|-------|
| Azure AD | Keycloak | Sovereign, self-hosted |
| Azure VMs | Phoenix VMs (Proxmox) | More flexible |
| Azure Storage | Phoenix Storage (Ceph/MinIO) | S3-compatible |
| Azure Networking | Phoenix Networking | Cloudflare tunnels |
| Azure Cost Management | Phoenix Billing | Per-second vs hourly |
| Azure RBAC | Phoenix Permissions | More granular |
| Azure Monitor | Prometheus/Grafana | Open source |
## Benefits of Migration
1. **Cost Savings**
- No per-user licensing
- Per-second billing (vs hourly)
- Reserved capacity discounts
2. **Better Control**
- Own your infrastructure
- No vendor lock-in
- Custom configurations
3. **Superior Features**
- Per-second billing
- Real-time cost tracking
- ML-based forecasting
- Blockchain-backed billing
4. **Sovereignty**
- Complete data control
- No Microsoft dependencies
- Self-hosted everything
## Migration Checklist
- [ ] Inventory Azure resources
- [ ] Map to Sankofa equivalents
- [ ] Export users from Azure AD
- [ ] Import to Keycloak
- [ ] Configure SSO
- [ ] Export VM configurations
- [ ] Create VMs in Sankofa
- [ ] Migrate storage data
- [ ] Recreate network topology
- [ ] Update applications
- [ ] Test functionality
- [ ] Cut over traffic
- [ ] Decommission Azure resources
## Support
For migration assistance, contact:
- Documentation: [docs/tenants/](./)
- Support: support@sankofa.nexus

View File

@@ -0,0 +1,254 @@
# Billing Guide - Superior to Azure Cost Management
Comprehensive billing and cost management guide for Sankofa Phoenix.
## Overview
Sankofa Phoenix billing system provides features superior to Azure Cost Management:
- **Per-second granularity** (vs Azure's hourly)
- **Real-time cost tracking** (better than Azure)
- **ML-based forecasting** (predictive analytics)
- **Blockchain-backed billing** (immutable audit trail)
- **Multi-currency support**
- **Custom pricing models**
## Usage Tracking
### Per-Second Granularity
Unlike Azure's hourly billing, Sankofa tracks usage per-second:
```graphql
query {
usage(
tenantId: "tenant-id"
timeRange: {
start: "2024-01-01T00:00:00Z"
end: "2024-01-01T23:59:59Z"
}
granularity: SECOND
) {
totalCost
currency
byResource {
resourceId
resourceName
cost
quantity
}
}
}
```
### Granularity Options
- **SECOND**: Per-second tracking (most detailed)
- **MINUTE**: Per-minute aggregation
- **HOUR**: Per-hour aggregation (Azure equivalent)
- **DAY**: Daily aggregation
## Cost Breakdown
Get detailed cost breakdown with flexible grouping:
```graphql
query {
costBreakdown(
tenantId: "tenant-id"
groupBy: ["resource", "service", "region"]
) {
total
byResource {
resourceId
resourceName
cost
percentage
}
byService {
service
cost
percentage
}
byRegion {
region
cost
percentage
}
allocations {
key
value
cost
percentage
}
}
}
```
## Cost Forecasting
ML-based cost forecasting (better than Azure):
```graphql
query {
costForecast(
tenantId: "tenant-id"
timeframe: "30D"
confidence: 0.8
) {
currentCost
predictedCost
confidence
trend
recommendations
}
}
```
## Cost Optimization
Get optimization recommendations:
```graphql
query {
costOptimization(tenantId: "tenant-id") {
id
type
title
description
potentialSavings
impact
effort
}
}
```
## Budgets
Set and manage budgets (more granular than Azure):
```graphql
mutation {
setBudget(
tenantId: "tenant-id"
budget: {
name: "Monthly Budget"
amount: 10000
currency: "USD"
period: MONTHLY
startDate: "2024-01-01T00:00:00Z"
alertThresholds: [0.5, 0.75, 0.9]
filters: {
resourceType: "VM"
region: "us-west"
}
}
) {
id
name
currentSpend
remaining
}
}
```
## Billing Alerts
Configure alerts (more flexible than Azure):
```graphql
mutation {
createBillingAlert(
tenantId: "tenant-id"
alert: {
name: "High Spending Alert"
alertType: THRESHOLD
threshold: 5000
condition: {
operator: "greater_than"
value: 5000
}
notificationChannels: ["email", "slack"]
enabled: true
}
) {
id
name
enabled
}
}
```
## Invoices
View and manage invoices:
```graphql
query {
invoices(
tenantId: "tenant-id"
filter: {
status: PENDING
startDate: "2024-01-01T00:00:00Z"
endDate: "2024-01-31T23:59:59Z"
}
) {
id
invoiceNumber
total
status
dueDate
lineItems {
description
quantity
unitPrice
total
}
}
}
```
## Reserved Capacity
Manage reserved instances (more flexible than Azure):
Reserved capacity is tracked in the database and automatically applied to usage calculations.
## Discounts
Custom discount rules per tenant:
Discounts can be configured with:
- Percentage-based
- Fixed amount
- Volume discounts
- Tier-based discounts
## Blockchain Integration
All billing records are optionally recorded on blockchain for:
- Immutable audit trail
- Multi-party verification
- Transparent cost breakdown
- Automated settlement
## Comparison to Azure Cost Management
| Feature | Azure | Sankofa Phoenix |
|---------|-------|-----------------|
| Billing Granularity | Hourly | Per-second |
| Real-time Tracking | Limited | Full real-time |
| Cost Forecasting | Basic | ML-based |
| Optimization | Manual | Automated recommendations |
| Blockchain | No | Yes (unique) |
| Multi-currency | Limited | Full support |
| Custom Pricing | Limited | Per-tenant models |
## Best Practices
1. **Set budgets early**: Configure budgets before deployment
2. **Use alerts**: Set up alerts for cost thresholds
3. **Review regularly**: Check cost breakdown weekly
4. **Optimize continuously**: Use optimization recommendations
5. **Monitor trends**: Use forecasting to plan capacity

View File

@@ -0,0 +1,198 @@
# Identity Setup Guide - Keycloak Configuration
Guide for setting up sovereign identity management using Keycloak (NO Azure dependencies).
## Overview
Sankofa Phoenix uses Keycloak for identity management, providing a sovereign alternative to Azure AD with superior features:
- **Self-hosted**: No Microsoft dependencies
- **Multi-realm support**: One realm per tenant for complete isolation
- **Flexible authentication**: Custom flows, multiple providers
- **Fine-grained permissions**: Beyond Azure RBAC
- **Blockchain integration**: Optional blockchain-based identity verification
## Keycloak Installation
### Docker Compose
Keycloak is included in `docker-compose.yml`:
```yaml
keycloak:
image: quay.io/keycloak/keycloak:latest
environment:
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: admin
KC_DB: postgres
KC_DB_URL_HOST: postgres
KC_DB_URL_DATABASE: keycloak
KC_DB_USERNAME: postgres
KC_DB_PASSWORD: postgres
KC_HTTP_ENABLED: "true"
ports:
- "8080:8080"
command: start-dev
```
### Kubernetes Deployment
See `gitops/apps/keycloak/` for Kubernetes manifests.
## Configuration
### Environment Variables
```env
KEYCLOAK_URL=https://keycloak.sankofa.nexus
KEYCLOAK_REALM=master
KEYCLOAK_CLIENT_ID=sankofa-api
KEYCLOAK_CLIENT_SECRET=your-client-secret
KEYCLOAK_MULTI_REALM=true
```
### Multi-Realm Support
Enable multi-realm support for tenant isolation:
```env
KEYCLOAK_MULTI_REALM=true
```
When enabled, each tenant gets its own Keycloak realm automatically.
## Client Configuration
### API Client
1. Log in to Keycloak admin console
2. Create new client:
- Client ID: `sankofa-api`
- Client Protocol: `openid-connect`
- Access Type: `confidential`
- Valid Redirect URIs: `*`
- Service Accounts Enabled: `ON`
3. Get client secret from Credentials tab
### Portal Client
1. Create new client:
- Client ID: `portal-client`
- Client Protocol: `openid-connect`
- Access Type: `confidential`
- Valid Redirect URIs: `http://localhost:3000/*`
- Web Origins: `http://localhost:3000`
## Identity Providers
### SAML Provider
Configure SAML provider for enterprise SSO:
1. Go to Identity Providers
2. Add SAML provider
3. Configure:
- Alias: `saml-enterprise`
- Display Name: `Enterprise SAML`
- First Broker Login Flow: `first broker login`
- Post Broker Login Flow: `browser`
### OIDC Provider
Configure OIDC provider for federated identity:
1. Go to Identity Providers
2. Add OpenID Connect provider
3. Configure:
- Alias: `oidc-provider`
- Provider URL: `https://provider.example.com`
- Client ID: `client-id`
- Client Secret: `client-secret`
### LDAP Provider
Configure LDAP for directory integration:
1. Go to User Federation
2. Add LDAP provider
3. Configure connection settings
4. Sync users and groups
## Custom Authentication Flows
Create custom authentication flows (more flexible than Azure AD):
1. Go to Authentication
2. Create new flow
3. Add execution providers:
- Username/Password form
- OTP form
- Blockchain verification (custom)
- Conditional OTP
## Blockchain Identity Verification
Enable blockchain-based identity verification (unique to Phoenix):
1. Set `BLOCKCHAIN_IDENTITY_ENABLED=true`
2. Configure blockchain RPC URL
3. Identity verification is performed on-chain
4. Immutable identity records
## Tenant Realm Creation
When a tenant is created, a Keycloak realm is automatically created if `KEYCLOAK_MULTI_REALM=true`:
- Realm name: Tenant ID
- Display name: Tenant name
- Enabled: True
- Users can authenticate to tenant-specific realm
## User Management
### Adding Users to Tenant
Users are added to tenant via the API:
```graphql
mutation {
addUserToTenant(
tenantId: "tenant-id"
userId: "user-id"
role: TENANT_ADMIN
)
}
```
### Roles and Permissions
Tenant roles are more granular than Azure RBAC:
- **TENANT_OWNER**: Full control
- **TENANT_ADMIN**: Administrative access
- **TENANT_USER**: Standard access
- **TENANT_VIEWER**: Read-only
- **TENANT_BILLING_ADMIN**: Billing management
Additional permissions can be set via JSON for fine-grained control.
## Comparison to Azure AD
| Feature | Azure AD | Keycloak (Phoenix) |
|---------|----------|-------------------|
| Self-hosted | No | Yes |
| Multi-realm | Limited | Full support |
| Custom flows | Limited | Full support |
| Permissions | RBAC only | RBAC + JSON |
| Blockchain | No | Yes |
| Cost | Per-user licensing | Open source |
## Best Practices
1. **Use strong passwords**: Enforce password policies
2. **Enable MFA**: Require multi-factor authentication
3. **Regular audits**: Review user access regularly
4. **Monitor logs**: Track authentication attempts
5. **Backup realms**: Export realm configurations regularly

View File

@@ -0,0 +1,223 @@
# Tenant Management Guide
Comprehensive guide for managing multi-tenant operations in Sankofa Phoenix - Superior to Azure.
## Overview
Sankofa Phoenix provides advanced multi-tenancy capabilities that exceed Azure's offerings:
- **More flexible than Azure**: Custom domains per tenant, cross-tenant resource sharing
- **Finer-grained permissions**: Beyond Azure RBAC with JSON-based permissions
- **Better isolation**: Database-level RLS policies, network isolation options
- **Sovereign identity**: Keycloak-based (no Azure dependencies)
## Tenant Tiers
### FREE
- Basic resource quotas
- Community support
- Standard SLA
### STANDARD
- Increased resource quotas
- Email support
- Enhanced SLA
### ENTERPRISE
- Custom resource quotas
- Priority support
- Custom SLA
- Dedicated resources available
### SOVEREIGN
- Complete isolation
- Dedicated infrastructure
- Custom compliance requirements
- Sovereign data residency
## Creating Tenants
### Via GraphQL API
```graphql
mutation {
createTenant(input: {
name: "acme-corp"
domain: "acme.sankofa.nexus"
tier: ENTERPRISE
quotaLimits: {
compute: { vcpu: 100, memory: 512, instances: 50 }
storage: { total: 10000, perInstance: 500 }
network: { bandwidth: 1000, egress: 5000 }
}
metadata: {
contact: "admin@acme.com"
region: "us-west"
}
}) {
id
name
billingAccountId
status
}
}
```
### Via API Service
```typescript
import { tenantService } from './services/tenant'
const tenant = await tenantService.createTenant({
name: 'acme-corp',
domain: 'acme.sankofa.nexus',
tier: 'ENTERPRISE',
quotaLimits: {
compute: { vcpu: 100, memory: 512, instances: 50 },
storage: { total: 10000, perInstance: 500 },
network: { bandwidth: 1000, egress: 5000 },
},
})
```
## Managing Tenant Users
### Adding Users to Tenant
```graphql
mutation {
addUserToTenant(
tenantId: "tenant-id"
userId: "user-id"
role: TENANT_ADMIN
permissions: {
canCreateVMs: true
canDeleteVMs: false
maxVMs: 10
}
)
}
```
### Tenant Roles
- **TENANT_OWNER**: Full control over tenant
- **TENANT_ADMIN**: Administrative access (more granular than Azure)
- **TENANT_USER**: Standard user access
- **TENANT_VIEWER**: Read-only access
- **TENANT_BILLING_ADMIN**: Billing management access
## Tenant Quotas
Quotas are more flexible than Azure:
```graphql
mutation {
updateTenantQuotas(
tenantId: "tenant-id"
quotas: {
compute: {
vcpu: 200
memory: 1024
instances: 100
}
storage: {
total: 20000
perInstance: 1000
}
network: {
bandwidth: 2000
egress: 10000
}
custom: {
gpu: 4
specialFeatures: ["ai-inference"]
}
}
) {
id
quotaLimits
}
}
```
## Custom Domains
Each tenant can have a custom domain (better than Azure):
```graphql
mutation {
updateTenant(
id: "tenant-id"
input: {
domain: "acme.sankofa.nexus"
}
) {
id
domain
}
}
```
## Cross-Tenant Resource Sharing
Share resources across tenants (more flexible than Azure):
```graphql
mutation {
shareResourceAcrossTenants(
resourceId: "resource-id"
targetTenants: ["tenant-1", "tenant-2"]
)
}
```
## Tenant Status Management
### Suspend Tenant
```graphql
mutation {
suspendTenant(
id: "tenant-id"
reason: "Payment overdue"
)
}
```
### Activate Tenant
```graphql
mutation {
activateTenant(id: "tenant-id")
}
```
## Keycloak Integration
Each tenant can have its own Keycloak realm for complete isolation:
1. Set `KEYCLOAK_MULTI_REALM=true`
2. Realm is automatically created when tenant is created
3. Users authenticate to tenant-specific realm
4. Complete isolation between tenants
## Best Practices
1. **Use appropriate tier**: Match tier to tenant needs
2. **Set realistic quotas**: Prevent resource exhaustion
3. **Monitor usage**: Use billing alerts to track spending
4. **Regular audits**: Review tenant access and permissions
5. **Backup policies**: Configure per-tenant backup policies
## Comparison to Azure
| Feature | Azure | Sankofa Phoenix |
|---------|-------|-----------------|
| Custom Domains | Limited | Full support per tenant |
| Permissions | RBAC only | RBAC + JSON permissions |
| Resource Sharing | Limited | Cross-tenant sharing |
| Billing Granularity | Hourly | Per-second |
| Identity | Azure AD only | Keycloak (sovereign) |
| Isolation | Logical | Logical + optional physical |