Sankofa/docs/phoenix/FAQ.md

Phoenix Operating Model - Frequently Asked Questions

Common questions and answers about Phoenix operating model

This document provides answers to frequently asked questions about the Phoenix operating model, helping users understand concepts, resolve common issues, and implement best practices.

---

General Questions

Q1: What is the Phoenix Operating Model?

A: The Phoenix Operating Model is an enterprise-grade operating model for cloud services that separates commercial governance, technical tenancy, and content/devops control into five orthogonal control planes:

1. Commercial Plane - Who pays (Client/Billing Profile)
2. Tenancy Plane - Who owns domains & identity (Tenant)
3. Subscription Plane - What is provisioned (Subscription)
4. Environment Plane - Where workloads run (Environment)
5. Content & DevOps Plane - What is built, governed, and deployed (Enterprise → Portfolio → Product → Application → Component)

Each plane operates independently but references each other through IDs, enabling clean separation of concerns while maintaining interoperability.

See: Operating Model

---

Q2: How is Phoenix different from Azure or AWS?

A: Phoenix offers several key advantages:

1. Superior Multi-Tenancy: Finer-grained control than Azure/AWS
2. Superior Billing: Per-second granularity vs Azure's hourly
3. Sovereign Identity: Keycloak-based, no Azure/AWS dependencies
4. Multi-Region Native: Built for international/multi-national deployments
5. Decentralized Architecture: Supports distributed sovereignty
6. Landing Zone Patterns: Sovereign cloud deployments per region
7. Hard Data Residency: Enforced data residency per region
8. Air-Gapped Support: Native support for classified workloads

See: Cloud Provider Mapping

---

Q3: What is a Landing Zone?

A: A Landing Zone is a sovereign cloud deployment per region/nation that provides:

• Complete regional control over infrastructure and data
• Regional data residency enforcement
• Regional compliance and audit capabilities
• Network isolation with controlled cross-region connectivity
• Identity federation with regional control

Landing zones enable sovereign governments to maintain complete regional autonomy while enabling coordination across regions.

See: Multi-Region Landing Zones

---

Entity Model Questions

Q4: What is the difference between Client and Tenant?

A:

• Client (Billing Profile): Represents the legal entity that contracts with Phoenix for cloud services. It is the financial and contractual boundary for billing and invoicing. A Client can own multiple Tenants.

• Tenant: Represents the identity and domain boundary. It is the security blast-radius boundary and owns all identity, domain, and security configuration. A Tenant cannot span multiple Clients.

Key Rule: A Client can own multiple Tenants, but a Tenant cannot span multiple Clients.

See: Operating Model - Commercial Plane and Tenancy Plane

---

Q5: How do Subscriptions relate to Tenants and Clients?

A:

• Subscriptions live inside a Tenant (one Tenant → many Subscriptions)
• Subscriptions are mapped to one Client billing profile (via the Tenant's Client)
• Subscriptions define what services are available, quotas, limits, and policy packs

Key Rules:

• Subscriptions live inside a Tenant
• Subscriptions are mapped to one Client billing profile
• Billing aggregates at Client level, not directly tied to Subscriptions

See: Operating Model - Subscription Plane

---

Q6: What are the different Environment Types?

A: Phoenix supports 8 environment types:

Standard Environments:

• DEV - Development
• INT - Integration testing
• UAT - User acceptance testing
• STAGING - Pre-production validation
• PROD - Production

Specialized Environments:

• REGULATED - Regulated workloads (HIPAA, PCI-DSS, etc.)
• SOVEREIGN - Sovereign workloads with data residency
• AIR-GAPPED - Classified workloads with no external connectivity

See: Operating Model - Environment Plane

---

Multi-Region Questions

Q7: How does multi-region deployment work?

A: Phoenix supports multi-region deployments through:

1. Landing Zones: Sovereign cloud deployment per region/nation
2. Multi-Region Tenants: Tenants that span multiple regions with regional data residency
3. Cross-Region Connectivity: Controlled connectivity between regions
4. Federated Identity: Identity federation across regions
5. Coordinated Governance: Governance policies that span regions

See: Multi-Region Landing Zones

---

Q8: How is data residency enforced?

A: Phoenix enforces data residency at multiple levels:

1. Hard Enforcement: Data cannot leave region (enforced at storage, network, and application layers)
2. Soft Enforcement: Data preferred in region, warnings if outside
3. Advisory: Recommendations for data placement

Data residency is configured per Tenant and enforced per Landing Zone.

See: Multi-Region Landing Zones - Regional Data Residency

---

Q9: What is decentralized architecture?

A: Decentralized architecture means:

• Distributed Control Planes: Control planes deployed per region
• Federated Governance: Governance policies federated across regions
• Regional Autonomy: Regional control with coordination
• No Single Point of Control: No centralized control plane

This enables sovereign governments to maintain complete regional control while enabling coordination.

See: Operating Model - Decentralized Architecture

---

Identity and Access Questions

Q10: How does identity management work?

A: Phoenix uses Keycloak for identity management:

• One Tenant = One Keycloak Realm: Each tenant gets its own Keycloak realm
• Sovereign Identity: No Azure/AWS dependencies
• Federated Identity: Can federate with Azure AD, Okta, etc.
• Multi-Region Identity: Federated identity across regions

See: Operating Model - Tenancy Plane and Identity Setup

---

Q11: How does RBAC work across planes?

A: RBAC is scoped per plane:

• Commercial Plane: Finance Admin, Billing Viewer, Cost Center Owner
• Tenancy Plane: Tenant Owner, Security Admin, Identity Admin, Compliance Officer
• Subscription Plane: Subscription Owner, Platform Admin, Service Operator, Auditor
• Environment Plane: Environment Owner, Release Manager, Operator, Observer
• Content & DevOps Plane: Enterprise Architect, Portfolio Lead, Product Owner, Dev Lead, Contributor, Reviewer, Release Approver

Key Rule: No role crosses planes by default. Cross-plane access requires explicit delegation.

See: Operating Model - Hierarchical Access Model

---

Billing Questions

Q12: How does billing work in the new model?

A: Billing operates at the Client (Billing Profile) level:

• Client aggregates billing from all associated Tenants and Subscriptions
• Subscriptions track costs per subscription
• Billing is never tied directly to environments or repos
• Cost Centers enable chargeback to internal departments

Key Rule: Billing is never tied directly to environments or repos.

See: Operating Model - Commercial Plane and Billing Guide

---

Q13: How does billing compare to Azure?

A: Phoenix billing is superior to Azure:

• Granularity: Per-second vs Azure's hourly
• Real-Time Tracking: Full real-time vs Azure's limited
• Cost Forecasting: ML-based vs Azure's basic
• Optimization: Automated recommendations vs Azure's manual
• Blockchain: Optional blockchain billing vs Azure's none
• Multi-Currency: Full support vs Azure's limited

See: Cloud Provider Mapping - Feature Comparison

---

Content & DevOps Questions

Q14: How does the Content & DevOps plane work?

A: The Content & DevOps plane is separate from billing and tenancy:

• Enterprise Content Hierarchy: Enterprise → Portfolio → Product → Application → Component
• Git Integration: Repositories mapped to Applications
• CI/CD Integration: Pipelines with policy gates
• Policy-Driven Promotion: Automated promotion with approval workflows

Critical Principle: Git never directly deploys to PROD without environment + subscription authorization.

See: Operating Model - Content & DevOps Plane

---

Q15: How does promotion flow work?

A: Promotion flow is policy-driven:

1. Code Commit → CI (Test, Scan) → Artifact Registry
2. Environment Promotion (Policy-Driven)
3. Subscription Deployment

Policy Rules:

• DEV → INT → UAT: Automated if tests pass
• UAT → STAGING: Requires approval
• STAGING → PROD: Requires multiple approvals and compliance checks

See: Operating Model - Promotion Flow

---

Migration Questions

Q16: How do I migrate from the current tenant-based model?

A: Migration involves:

1. Create Client Structure: Group existing tenants by billing entity
2. Restructure Tenants: Update tenants with new attributes
3. Create Subscriptions: Map tenant resources to subscriptions
4. Create Environments: Map resources to environments
5. Content & DevOps Migration: Create content hierarchy and update CI/CD

See: Migration Guide - From Existing Model

---

Q17: How do I migrate from Azure to Phoenix?

A: Migration from Azure involves:

1. Assessment: Inventory Azure resources and map to Phoenix model
2. Setup: Create Phoenix Client, Tenants, Subscriptions
3. Identity Migration: Export Azure AD users, import to Keycloak
4. Resource Migration: Export Azure resources, convert and import to Phoenix
5. Application Migration: Migrate applications to Phoenix
6. Cutover: Final validation, cutover, decommission Azure

See: Migration Guide - From Azure

---

Q18: How long does migration take?

A: Migration timeline depends on scale:

• Small-Scale (< 100 resources): 1-3 months
• Medium-Scale (100-1000 resources): 3-6 months
• Large-Scale (> 1000 resources): 6-12 months
• Sovereign/Air-Gapped: 6-18 months (additional complexity)

See: Migration Guide - Timeline Estimates

---

Compliance Questions

Q19: What compliance standards are supported?

A: Phoenix supports:

• ISO: ISO 27001, ISO 27017, ISO 27018
• SOC: SOC 2, SOC 3
• Healthcare: HIPAA
• Financial: PCI-DSS
• Privacy: GDPR, CCPA
• Government: FedRAMP, ITAR
• Custom: Government-specific standards

Compliance profiles are configured per Tenant and enforced per Landing Zone.

See: Operating Model - Compliance

---

Q20: How does air-gapped deployment work?

A: Air-gapped deployment provides:

• Complete Network Isolation: No external connectivity
• No Cross-Region Connectivity: Complete isolation per region
• Local Identity Only: Independent Keycloak realm
• Local Governance Only: Independent governance
• AIR-GAPPED Environment Type: Specialized environment type

See: Multi-Region Landing Zones - Air-Gapped

---

Technical Questions

Q21: What APIs are available?

A: Phoenix provides APIs for all five control planes:

• Commercial Plane API: Client and billing operations
• Tenancy Plane API: Tenant and identity operations
• Subscription Plane API: Subscription and quota operations
• Environment Plane API: Environment and deployment operations
• Content & DevOps Plane API: Content and Git operations

APIs support both GraphQL (primary) and REST (alternative) interfaces.

See: API Specification

---

Q22: How do I integrate with existing infrastructure?

A: Phoenix integrates with:

• Proxmox: Environment → Proxmox resource pool mapping
• Kubernetes: Environment → Kubernetes namespace mapping
• Cloudflare: Tenant → Cloudflare Access Policy mapping
• Keycloak: Tenant → Keycloak realm (1:1)
• ArgoCD: Application → ArgoCD Application mapping
• Crossplane: Subscription → Crossplane Composite Resource mapping

See: Operating Model - Integration

---

Q23: What is the MVP scope?

A: MVP includes:

• All five control planes (core functionality)
• Client, Tenant, Subscription, Environment entities
• Keycloak integration (1:1 Tenant to Realm)
• Basic infrastructure integration (Proxmox, Kubernetes)
• Basic CI/CD integration
• Policy-driven promotion
• Basic multi-region support
• Basic compliance support

See: MVP Control Plane

---

Best Practices

Q24: What are best practices for landing zone design?

A: Best practices:

1. Start with Standard Pattern: Begin with standard sovereign landing zone
2. Plan for Growth: Design landing zones to scale
3. Regional Autonomy: Ensure regional autonomy while enabling coordination
4. Data Residency: Enforce data residency from the start
5. Compliance First: Design compliance into landing zones

See: Multi-Region Landing Zones - Best Practices

---

Q25: What are best practices for promotion flows?

A: Best practices:

1. Policy-Driven: Use policy-driven promotion, not manual
2. Approval Workflows: Require approval for PROD deployments
3. Validation: Validate policies before promotion
4. Audit Logging: Log all promotion activities
5. Rollback: Plan for rollback procedures

See: Operating Model - Promotion Flow

---

Troubleshooting

Q26: Tenant creation fails. What do I do?

A: Troubleshooting steps:

1. Check Keycloak connectivity
2. Verify Keycloak admin access
3. Check tenant creation logs
4. Verify input data
5. Retry with verbose logging

See: Operational Runbooks - Troubleshooting

---

Q27: Promotion fails. What do I do?

A: Troubleshooting steps:

1. Check promotion status
2. Review policy validation results
3. Check approval status
4. Review deployment logs
5. Verify environment configuration

See: Operational Runbooks - Troubleshooting

---

Q28: Billing aggregation fails. What do I do?

A: Troubleshooting steps:

1. Check billing aggregation job status
2. Verify subscription cost tracking
3. Check billing service logs
4. Trigger manual aggregation
5. Verify data integrity

See: Operational Runbooks - Troubleshooting

---

References

• Operating Model - Complete operating model
• Architecture Diagrams - Visual diagrams
• Cloud Provider Mapping - Azure/AWS comparison
• Migration Guide - Migration guides
• API Specification - API reference
• Implementation Examples - Code examples
• Operational Runbooks - Operational procedures

---

Last Updated: 2025-01-09
Version: 1.0
Status: Complete FAQ