Sankofa/docs/phoenix/FAQ.md

# Phoenix Operating Model - Frequently Asked Questions

**Common questions and answers about Phoenix operating model**

This document provides answers to frequently asked questions about the Phoenix operating model, helping users understand concepts, resolve common issues, and implement best practices.

---

## General Questions

### Q1: What is the Phoenix Operating Model?

**A:** The Phoenix Operating Model is an enterprise-grade operating model for cloud services that separates commercial governance, technical tenancy, and content/devops control into **five orthogonal control planes**:

1. **Commercial Plane** - Who pays (Client/Billing Profile)
2. **Tenancy Plane** - Who owns domains & identity (Tenant)
3. **Subscription Plane** - What is provisioned (Subscription)
4. **Environment Plane** - Where workloads run (Environment)
5. **Content & DevOps Plane** - What is built, governed, and deployed (Enterprise → Portfolio → Product → Application → Component)

Each plane operates independently but references each other through IDs, enabling clean separation of concerns while maintaining interoperability.

**See:** [Operating Model](./OPERATING_MODEL.md)

---

### Q2: How is Phoenix different from Azure or AWS?

**A:** Phoenix offers several key advantages:

1. **Superior Multi-Tenancy**: Finer-grained control than Azure/AWS
2. **Superior Billing**: Per-second granularity vs Azure's hourly
3. **Sovereign Identity**: Keycloak-based, no Azure/AWS dependencies
4. **Multi-Region Native**: Built for international/multi-national deployments
5. **Decentralized Architecture**: Supports distributed sovereignty
6. **Landing Zone Patterns**: Sovereign cloud deployments per region
7. **Hard Data Residency**: Enforced data residency per region
8. **Air-Gapped Support**: Native support for classified workloads

**See:** [Cloud Provider Mapping](./CLOUD_PROVIDER_MAPPING.md)

---

### Q3: What is a Landing Zone?

**A:** A Landing Zone is a **sovereign cloud deployment per region/nation** that provides:

- Complete regional control over infrastructure and data
- Regional data residency enforcement
- Regional compliance and audit capabilities
- Network isolation with controlled cross-region connectivity
- Identity federation with regional control

Landing zones enable sovereign governments to maintain complete regional autonomy while enabling coordination across regions.

**See:** [Multi-Region Landing Zones](./MULTI_REGION_LANDING_ZONES.md)

---

## Entity Model Questions

### Q4: What is the difference between Client and Tenant?

**A:** 

- **Client (Billing Profile)**: Represents the legal entity that contracts with Phoenix for cloud services. It is the **financial and contractual boundary** for billing and invoicing. A Client can own multiple Tenants.

- **Tenant**: Represents the **identity and domain boundary**. It is the **security blast-radius boundary** and owns all identity, domain, and security configuration. A Tenant cannot span multiple Clients.

**Key Rule:** A Client can own multiple Tenants, but a Tenant cannot span multiple Clients.

**See:** [Operating Model - Commercial Plane](./OPERATING_MODEL.md#i-commercial-plane--clients-billing-profiles) and [Tenancy Plane](./OPERATING_MODEL.md#ii-tenancy-plane--tenants-domains)

---

### Q5: How do Subscriptions relate to Tenants and Clients?

**A:** 

- **Subscriptions** live inside a **Tenant** (one Tenant → many Subscriptions)
- **Subscriptions** are mapped to one **Client** billing profile (via the Tenant's Client)
- **Subscriptions** define what services are available, quotas, limits, and policy packs

**Key Rules:**
- Subscriptions live inside a Tenant
- Subscriptions are mapped to one Client billing profile
- Billing aggregates at Client level, not directly tied to Subscriptions

**See:** [Operating Model - Subscription Plane](./OPERATING_MODEL.md#iii-subscription-plane--subscriptions)

---

### Q6: What are the different Environment Types?

**A:** Phoenix supports 8 environment types:

**Standard Environments:**
- **DEV** - Development
- **INT** - Integration testing
- **UAT** - User acceptance testing
- **STAGING** - Pre-production validation
- **PROD** - Production

**Specialized Environments:**
- **REGULATED** - Regulated workloads (HIPAA, PCI-DSS, etc.)
- **SOVEREIGN** - Sovereign workloads with data residency
- **AIR-GAPPED** - Classified workloads with no external connectivity

**See:** [Operating Model - Environment Plane](./OPERATING_MODEL.md#iv-environment-plane--environments)

---

## Multi-Region Questions

### Q7: How does multi-region deployment work?

**A:** Phoenix supports multi-region deployments through:

1. **Landing Zones**: Sovereign cloud deployment per region/nation
2. **Multi-Region Tenants**: Tenants that span multiple regions with regional data residency
3. **Cross-Region Connectivity**: Controlled connectivity between regions
4. **Federated Identity**: Identity federation across regions
5. **Coordinated Governance**: Governance policies that span regions

**See:** [Multi-Region Landing Zones](./MULTI_REGION_LANDING_ZONES.md)

---

### Q8: How is data residency enforced?

**A:** Phoenix enforces data residency at multiple levels:

1. **Hard Enforcement**: Data cannot leave region (enforced at storage, network, and application layers)
2. **Soft Enforcement**: Data preferred in region, warnings if outside
3. **Advisory**: Recommendations for data placement

Data residency is configured per Tenant and enforced per Landing Zone.

**See:** [Multi-Region Landing Zones - Regional Data Residency](./MULTI_REGION_LANDING_ZONES.md#v-regional-data-residency)

---

### Q9: What is decentralized architecture?

**A:** Decentralized architecture means:

- **Distributed Control Planes**: Control planes deployed per region
- **Federated Governance**: Governance policies federated across regions
- **Regional Autonomy**: Regional control with coordination
- **No Single Point of Control**: No centralized control plane

This enables sovereign governments to maintain complete regional control while enabling coordination.

**See:** [Operating Model - Decentralized Architecture](./OPERATING_MODEL.md#ix-decentralized-architecture)

---

## Identity and Access Questions

### Q10: How does identity management work?

**A:** Phoenix uses Keycloak for identity management:

- **One Tenant = One Keycloak Realm**: Each tenant gets its own Keycloak realm
- **Sovereign Identity**: No Azure/AWS dependencies
- **Federated Identity**: Can federate with Azure AD, Okta, etc.
- **Multi-Region Identity**: Federated identity across regions

**See:** [Operating Model - Tenancy Plane](./OPERATING_MODEL.md#ii-tenancy-plane--tenants-domains) and [Identity Setup](../tenants/IDENTITY_SETUP.md)

---

### Q11: How does RBAC work across planes?

**A:** RBAC is scoped per plane:

- **Commercial Plane**: Finance Admin, Billing Viewer, Cost Center Owner
- **Tenancy Plane**: Tenant Owner, Security Admin, Identity Admin, Compliance Officer
- **Subscription Plane**: Subscription Owner, Platform Admin, Service Operator, Auditor
- **Environment Plane**: Environment Owner, Release Manager, Operator, Observer
- **Content & DevOps Plane**: Enterprise Architect, Portfolio Lead, Product Owner, Dev Lead, Contributor, Reviewer, Release Approver

**Key Rule:** No role crosses planes by default. Cross-plane access requires explicit delegation.

**See:** [Operating Model - Hierarchical Access Model](./OPERATING_MODEL.md#vi-hierarchical-access-model-rbac)

---

## Billing Questions

### Q12: How does billing work in the new model?

**A:** Billing operates at the **Client (Billing Profile)** level:

- **Client** aggregates billing from all associated Tenants and Subscriptions
- **Subscriptions** track costs per subscription
- **Billing** is never tied directly to environments or repos
- **Cost Centers** enable chargeback to internal departments

**Key Rule:** Billing is never tied directly to environments or repos.

**See:** [Operating Model - Commercial Plane](./OPERATING_MODEL.md#i-commercial-plane--clients-billing-profiles) and [Billing Guide](../tenants/BILLING_GUIDE.md)

---

### Q13: How does billing compare to Azure?

**A:** Phoenix billing is superior to Azure:

- **Granularity**: Per-second vs Azure's hourly
- **Real-Time Tracking**: Full real-time vs Azure's limited
- **Cost Forecasting**: ML-based vs Azure's basic
- **Optimization**: Automated recommendations vs Azure's manual
- **Blockchain**: Optional blockchain billing vs Azure's none
- **Multi-Currency**: Full support vs Azure's limited

**See:** [Cloud Provider Mapping - Feature Comparison](./CLOUD_PROVIDER_MAPPING.md#vi-feature-comparison-matrix)

---

## Content & DevOps Questions

### Q14: How does the Content & DevOps plane work?

**A:** The Content & DevOps plane is **separate from billing and tenancy**:

- **Enterprise Content Hierarchy**: Enterprise → Portfolio → Product → Application → Component
- **Git Integration**: Repositories mapped to Applications
- **CI/CD Integration**: Pipelines with policy gates
- **Policy-Driven Promotion**: Automated promotion with approval workflows

**Critical Principle:** Git never directly deploys to PROD without environment + subscription authorization.

**See:** [Operating Model - Content & DevOps Plane](./OPERATING_MODEL.md#v-content--devops-plane-separate-but-integrated)

---

### Q15: How does promotion flow work?

**A:** Promotion flow is **policy-driven**:

1. **Code Commit** → CI (Test, Scan) → Artifact Registry
2. **Environment Promotion** (Policy-Driven)
3. **Subscription Deployment**

**Policy Rules:**
- DEV → INT → UAT: Automated if tests pass
- UAT → STAGING: Requires approval
- STAGING → PROD: Requires multiple approvals and compliance checks

**See:** [Operating Model - Promotion Flow](./OPERATING_MODEL.md#b-git--devops-integration-model)

---

## Migration Questions

### Q16: How do I migrate from the current tenant-based model?

**A:** Migration involves:

1. **Create Client Structure**: Group existing tenants by billing entity
2. **Restructure Tenants**: Update tenants with new attributes
3. **Create Subscriptions**: Map tenant resources to subscriptions
4. **Create Environments**: Map resources to environments
5. **Content & DevOps Migration**: Create content hierarchy and update CI/CD

**See:** [Migration Guide - From Existing Model](./MIGRATION_GUIDE.md#i-migration-from-existing-phoenix-model)

---

### Q17: How do I migrate from Azure to Phoenix?

**A:** Migration from Azure involves:

1. **Assessment**: Inventory Azure resources and map to Phoenix model
2. **Setup**: Create Phoenix Client, Tenants, Subscriptions
3. **Identity Migration**: Export Azure AD users, import to Keycloak
4. **Resource Migration**: Export Azure resources, convert and import to Phoenix
5. **Application Migration**: Migrate applications to Phoenix
6. **Cutover**: Final validation, cutover, decommission Azure

**See:** [Migration Guide - From Azure](./MIGRATION_GUIDE.md#ii-migration-from-azure)

---

### Q18: How long does migration take?

**A:** Migration timeline depends on scale:

- **Small-Scale** (< 100 resources): 1-3 months
- **Medium-Scale** (100-1000 resources): 3-6 months
- **Large-Scale** (> 1000 resources): 6-12 months
- **Sovereign/Air-Gapped**: 6-18 months (additional complexity)

**See:** [Migration Guide - Timeline Estimates](./MIGRATION_GUIDE.md#vii-timeline-estimates)

---

## Compliance Questions

### Q19: What compliance standards are supported?

**A:** Phoenix supports:

- **ISO**: ISO 27001, ISO 27017, ISO 27018
- **SOC**: SOC 2, SOC 3
- **Healthcare**: HIPAA
- **Financial**: PCI-DSS
- **Privacy**: GDPR, CCPA
- **Government**: FedRAMP, ITAR
- **Custom**: Government-specific standards

Compliance profiles are configured per Tenant and enforced per Landing Zone.

**See:** [Operating Model - Compliance](./OPERATING_MODEL.md#ii-tenancy-plane--tenants-domains)

---

### Q20: How does air-gapped deployment work?

**A:** Air-gapped deployment provides:

- **Complete Network Isolation**: No external connectivity
- **No Cross-Region Connectivity**: Complete isolation per region
- **Local Identity Only**: Independent Keycloak realm
- **Local Governance Only**: Independent governance
- **AIR-GAPPED Environment Type**: Specialized environment type

**See:** [Multi-Region Landing Zones - Air-Gapped](./MULTI_REGION_LANDING_ZONES.md#pattern-2-air-gapped-landing-zone)

---

## Technical Questions

### Q21: What APIs are available?

**A:** Phoenix provides APIs for all five control planes:

- **Commercial Plane API**: Client and billing operations
- **Tenancy Plane API**: Tenant and identity operations
- **Subscription Plane API**: Subscription and quota operations
- **Environment Plane API**: Environment and deployment operations
- **Content & DevOps Plane API**: Content and Git operations

APIs support both GraphQL (primary) and REST (alternative) interfaces.

**See:** [API Specification](./API_SPECIFICATION.md)

---

### Q22: How do I integrate with existing infrastructure?

**A:** Phoenix integrates with:

- **Proxmox**: Environment → Proxmox resource pool mapping
- **Kubernetes**: Environment → Kubernetes namespace mapping
- **Cloudflare**: Tenant → Cloudflare Access Policy mapping
- **Keycloak**: Tenant → Keycloak realm (1:1)
- **ArgoCD**: Application → ArgoCD Application mapping
- **Crossplane**: Subscription → Crossplane Composite Resource mapping

**See:** [Operating Model - Integration](./OPERATING_MODEL.md#x-integration-with-existing-infrastructure)

---

### Q23: What is the MVP scope?

**A:** MVP includes:

- All five control planes (core functionality)
- Client, Tenant, Subscription, Environment entities
- Keycloak integration (1:1 Tenant to Realm)
- Basic infrastructure integration (Proxmox, Kubernetes)
- Basic CI/CD integration
- Policy-driven promotion
- Basic multi-region support
- Basic compliance support

**See:** [MVP Control Plane](./MVP_CONTROL_PLANE.md)

---

## Best Practices

### Q24: What are best practices for landing zone design?

**A:** Best practices:

1. **Start with Standard Pattern**: Begin with standard sovereign landing zone
2. **Plan for Growth**: Design landing zones to scale
3. **Regional Autonomy**: Ensure regional autonomy while enabling coordination
4. **Data Residency**: Enforce data residency from the start
5. **Compliance First**: Design compliance into landing zones

**See:** [Multi-Region Landing Zones - Best Practices](./MULTI_REGION_LANDING_ZONES.md#xiv-best-practices)

---

### Q25: What are best practices for promotion flows?

**A:** Best practices:

1. **Policy-Driven**: Use policy-driven promotion, not manual
2. **Approval Workflows**: Require approval for PROD deployments
3. **Validation**: Validate policies before promotion
4. **Audit Logging**: Log all promotion activities
5. **Rollback**: Plan for rollback procedures

**See:** [Operating Model - Promotion Flow](./OPERATING_MODEL.md#b-git--devops-integration-model)

---

## Troubleshooting

### Q26: Tenant creation fails. What do I do?

**A:** Troubleshooting steps:

1. Check Keycloak connectivity
2. Verify Keycloak admin access
3. Check tenant creation logs
4. Verify input data
5. Retry with verbose logging

**See:** [Operational Runbooks - Troubleshooting](./OPERATIONAL_RUNBOOKS.md#issue-1-tenant-creation-fails)

---

### Q27: Promotion fails. What do I do?

**A:** Troubleshooting steps:

1. Check promotion status
2. Review policy validation results
3. Check approval status
4. Review deployment logs
5. Verify environment configuration

**See:** [Operational Runbooks - Troubleshooting](./OPERATIONAL_RUNBOOKS.md#issue-2-promotion-fails)

---

### Q28: Billing aggregation fails. What do I do?

**A:** Troubleshooting steps:

1. Check billing aggregation job status
2. Verify subscription cost tracking
3. Check billing service logs
4. Trigger manual aggregation
5. Verify data integrity

**See:** [Operational Runbooks - Troubleshooting](./OPERATIONAL_RUNBOOKS.md#issue-3-billing-aggregation-fails)

---

## References

- **[Operating Model](./OPERATING_MODEL.md)** - Complete operating model
- **[Architecture Diagrams](./OPERATING_MODEL_DIAGRAMS.md)** - Visual diagrams
- **[Cloud Provider Mapping](./CLOUD_PROVIDER_MAPPING.md)** - Azure/AWS comparison
- **[Migration Guide](./MIGRATION_GUIDE.md)** - Migration guides
- **[API Specification](./API_SPECIFICATION.md)** - API reference
- **[Implementation Examples](./IMPLEMENTATION_EXAMPLES.md)** - Code examples
- **[Operational Runbooks](./OPERATIONAL_RUNBOOKS.md)** - Operational procedures

---

**Last Updated**: 2025-01-09  
**Version**: 1.0  
**Status**: Complete FAQ