Files

defiQUG 8649ad4124 feat: implement naming convention, deployment automation, and infrastructure updates

- Add comprehensive naming convention (provider-region-resource-env-purpose)
- Implement Terraform locals for centralized naming
- Update all Terraform resources to use new naming convention
- Create deployment automation framework (18 phase scripts)
- Add Azure setup scripts (provider registration, quota checks)
- Update deployment scripts config with naming functions
- Create complete deployment documentation (guide, steps, quick reference)
- Add frontend portal implementations (public and internal)
- Add UI component library (18 components)
- Enhance Entra VerifiedID integration with file utilities
- Add API client package for all services
- Create comprehensive documentation (naming, deployment, next steps)

Infrastructure:
- Resource groups, storage accounts with new naming
- Terraform configuration updates
- Outputs with naming convention examples

Deployment:
- Automated deployment scripts for all 15 phases
- State management and logging
- Error handling and validation

Documentation:
- Naming convention guide and implementation summary
- Complete deployment guide (296 steps)
- Next steps and quick start guides
- Azure prerequisites and setup completion docs

Note: ESLint warnings present - will be addressed in follow-up commit

2025-11-12 08:22:51 -08:00

16 KiB

Raw Blame History

Deployment Steps Summary - Ordered by Execution Sequence

Last Updated: 2025-01-27
Purpose: Complete list of all deployment steps grouped by execution order

Overview

This document lists all deployment steps in the exact order they must be executed. Steps are grouped into phases that can be executed sequentially, with some phases able to run in parallel (noted below).

Total Phases: 15
Estimated Total Time: 8-12 weeks (with parallelization)

Phase 1: Prerequisites ⚙️

Duration: 1-2 days
Can Run In Parallel: No
Dependencies: None

1.1 Development Environment

Install Node.js >= 18.0.0
Install pnpm >= 8.0.0
Install Azure CLI
Install Terraform >= 1.5.0
Install kubectl
Install Docker (for local dev)
Clone repository
Initialize git submodules
Install dependencies (pnpm install)
Build all packages (pnpm build)

1.2 Azure Account

Create Azure subscription (if needed)
Login to Azure CLI (az login)
Set active subscription
Verify permissions (Contributor/Owner role)

1.3 Local Services (Optional)

Start Docker Compose services (PostgreSQL, Redis, OpenSearch)

Phase 2: Azure Infrastructure Setup 🏗️

Duration: 4-6 weeks
Can Run In Parallel: Yes (with Phase 3)
Dependencies: Phase 1

2.1 Azure Subscription Preparation

Run ./infra/scripts/azure-setup.sh
Run ./infra/scripts/azure-register-providers.sh
Run ./infra/scripts/azure-check-quotas.sh
Review quota reports
Verify all 13 resource providers registered

2.2 Terraform Infrastructure

Navigate to infra/terraform
Run terraform init
Create Terraform state storage (resource group, storage account, container)
Configure remote state backend in versions.tf
Re-initialize Terraform with terraform init -migrate-state
Run terraform plan
Deploy resource groups
Deploy storage accounts
Deploy AKS cluster (configuration to be added)
Deploy Azure Database for PostgreSQL (configuration to be added)
Deploy Azure Key Vault (configuration to be added)
Deploy Azure Container Registry (configuration to be added)
Deploy Virtual Network (configuration to be added)
Deploy Application Gateway/Load Balancer (configuration to be added)

2.3 Kubernetes Configuration

Get AKS credentials (az aks get-credentials)
Verify cluster access (kubectl get nodes)
Configure Azure CNI networking
Install External Secrets Operator
Configure Azure Key Vault Provider for Secrets Store CSI
Attach ACR to AKS (az aks update --attach-acr)
Enable Azure Monitor for Containers
Configure Log Analytics workspace

Phase 3: Entra ID Configuration 🔐

Duration: 1-2 days
Can Run In Parallel: Yes (with Phase 2)
Dependencies: Phase 1

3.1 Azure AD App Registration

Create App Registration in Azure Portal
Note Application (client) ID
Note Directory (tenant) ID
Configure API permissions (Verifiable Credentials Service)
Grant admin consent for permissions
Create client secret
Save client secret securely (only shown once)
Configure redirect URIs for portals
Configure logout URLs

3.2 Microsoft Entra VerifiedID

Enable Verified ID service in Azure Portal
Wait for service activation
Create credential manifest
Define credential type
Define claims schema
Note Manifest ID
Verify Issuer DID format
Test DID resolution

3.3 Azure Logic Apps (Optional)

Create Logic App workflows (eIDAS, VC issuance, document processing)
Note workflow URLs
Generate access keys OR configure managed identity
Grant necessary permissions
Test workflow triggers

Phase 4: Database & Storage Setup 💾

Duration: 1-2 days
Dependencies: Phase 2

4.1 PostgreSQL

Create databases (dev, stage, prod)
Create database users
Grant privileges
Configure firewall rules for AKS
Test database connection

4.2 Storage Accounts

Verify storage accounts created
Create container: intake-documents
Create container: dataroom-deals
Create container: credentials
Configure managed identity access
Configure CORS (if needed)
Enable versioning and soft delete

4.3 Redis Cache (If using Azure Cache)

Create Azure Cache for Redis (Terraform to be added)
Configure firewall rules
Set up access keys
Test connection

4.4 OpenSearch (If using managed service)

Create managed OpenSearch cluster (Terraform to be added)
Configure access
Set up indices
Test connection

Phase 5: Container Registry Setup 📦

Duration: 1 day
Dependencies: Phase 2

5.1 Azure Container Registry

Verify ACR created
Enable admin user (or configure managed identity)
Get ACR credentials
Attach ACR to AKS (az aks update --attach-acr)
Test ACR access from AKS

Phase 6: Application Build & Package 🔨

Duration: 2-4 hours
Dependencies: Phase 1, Phase 5

6.1 Build Packages

Build shared packages (pnpm build)
Build @the-order/ui
Build @the-order/auth
Build @the-order/api-client
Build @the-order/database
Build @the-order/storage
Build @the-order/crypto
Build @the-order/schemas

6.2 Build Frontend Apps

Build portal-public
Build portal-internal

6.3 Build Backend Services

Build @the-order/identity
Build @the-order/intake
Build @the-order/finance
Build @the-order/dataroom

6.4 Create Docker Images

Create services/identity/Dockerfile (to be created)
Create services/intake/Dockerfile (to be created)
Create services/finance/Dockerfile (to be created)
Create services/dataroom/Dockerfile (to be created)
Create apps/portal-public/Dockerfile (to be created)
Create apps/portal-internal/Dockerfile (to be created)
Login to ACR (az acr login)
Build and push identity image
Build and push intake image
Build and push finance image
Build and push dataroom image
Build and push portal-public image
Build and push portal-internal image
Sign all images with Cosign (security best practice)

Phase 7: Database Migrations 🗄️

Duration: 1-2 hours
Dependencies: Phase 4, Phase 6

7.1 Run Migrations

Set DATABASE_URL for dev environment
Run migrations for dev (pnpm --filter @the-order/database migrate up)
Verify schema created (check tables)
Set DATABASE_URL for staging environment
Run migrations for staging
Verify schema created
Set DATABASE_URL for production environment
Run migrations for production
Verify schema created
Run seed scripts (if needed)

Phase 8: Secrets Configuration 🔒

Duration: 2-4 hours
Dependencies: Phase 2, Phase 3

8.1 Store Secrets in Key Vault

Store database-url-dev in Key Vault
Store database-url-stage in Key Vault
Store database-url-prod in Key Vault
Store entra-tenant-id in Key Vault
Store entra-client-id in Key Vault
Store entra-client-secret in Key Vault
Store entra-credential-manifest-id in Key Vault
Store storage-account-name in Key Vault
Store jwt-secret in Key Vault
Store kms-key-id in Key Vault
Store payment-gateway-api-key in Key Vault
Store ocr-service-api-key in Key Vault
Store eidas-api-key in Key Vault
Store other service-specific secrets

8.2 Configure External Secrets Operator

Create SecretStore for Azure Key Vault (YAML to be created)
Create ExternalSecret resources (YAML to be created)
Apply SecretStore configuration
Apply ExternalSecret configuration
Verify secrets synced to Kubernetes

Phase 9: Infrastructure Services Deployment 🛠️

Duration: 1-2 days
Dependencies: Phase 2, Phase 8

9.1 External Secrets Operator

Install External Secrets Operator
Wait for operator to be ready
Verify SecretStore working

9.2 Monitoring Stack

Add Prometheus Helm repository
Install Prometheus stack
Configure Grafana
Deploy OpenTelemetry Collector
Configure exporters
Set up trace collection

9.3 Logging Stack

Deploy OpenSearch (if not using managed service)
Configure Fluent Bit/Fluentd
Configure log forwarding
Set up log retention policies

Phase 10: Backend Services Deployment 🚀

Duration: 2-4 days
Dependencies: Phase 6, Phase 7, Phase 8, Phase 9

10.1 Create Kubernetes Manifests

Create infra/k8s/base/identity/deployment.yaml (to be created)
Create infra/k8s/base/identity/service.yaml (to be created)
Create infra/k8s/base/intake/deployment.yaml (to be created)
Create infra/k8s/base/intake/service.yaml (to be created)
Create infra/k8s/base/finance/deployment.yaml (to be created)
Create infra/k8s/base/finance/service.yaml (to be created)
Create infra/k8s/base/dataroom/deployment.yaml (to be created)
Create infra/k8s/base/dataroom/service.yaml (to be created)

10.2 Deploy Identity Service

Apply Identity Service manifests
Verify pods running
Check logs
Test health endpoint
Verify service accessible

10.3 Deploy Intake Service

Apply Intake Service manifests
Verify pods running
Check logs
Test health endpoint

10.4 Deploy Finance Service

Apply Finance Service manifests
Verify pods running
Check logs
Test health endpoint

10.5 Deploy Dataroom Service

Apply Dataroom Service manifests
Verify pods running
Check logs
Test health endpoint

10.6 Verify Service Communication

Test internal service-to-service communication
Verify service discovery working

Phase 11: Frontend Applications Deployment 🎨

Duration: 1-2 days
Dependencies: Phase 6, Phase 10

11.1 Portal Public

Create infra/k8s/base/portal-public/deployment.yaml (to be created)
Create infra/k8s/base/portal-public/service.yaml (to be created)
Create infra/k8s/base/portal-public/ingress.yaml (to be created)
Apply Portal Public manifests
Verify pods running
Check logs
Test application in browser

11.2 Portal Internal

Create infra/k8s/base/portal-internal/deployment.yaml (to be created)
Create infra/k8s/base/portal-internal/service.yaml (to be created)
Create infra/k8s/base/portal-internal/ingress.yaml (to be created)
Apply Portal Internal manifests
Verify pods running
Check logs
Test application in browser

Phase 12: Networking & Gateways 🌐

Duration: 2-3 days
Dependencies: Phase 10, Phase 11

12.1 Configure Ingress

Deploy NGINX Ingress Controller (if not using Application Gateway)
Create Ingress resources (YAML to be created)
Apply Ingress configuration
Verify ingress rules

12.2 Configure Application Gateway (If using)

Create backend pools
Configure routing rules
Configure SSL termination
Set up health probes

12.3 Configure DNS

Create DNS record for api.theorder.org
Create DNS record for portal.theorder.org
Create DNS record for admin.theorder.org
Verify DNS resolution

12.4 Configure SSL/TLS

Install cert-manager (if using Let's Encrypt)
Create ClusterIssuer
Configure certificate requests
Verify certificates issued
Test HTTPS access

12.5 Configure WAF

Set up OWASP rules
Configure custom rules
Set up rate limiting
Configure IP allow/deny lists

Phase 13: Monitoring & Observability 📊

Duration: 2-3 days
Dependencies: Phase 9, Phase 10, Phase 11

13.1 Application Insights

Create Application Insights resource
Add instrumentation keys to services
Configure custom metrics
Set up alerts

13.2 Log Analytics

Create Log Analytics workspace
Set up container insights
Configure log forwarding
Set up log queries

13.3 Set Up Alerts

Create alert rule for high error rate
Create alert rule for high latency
Create alert rule for resource usage
Configure email notifications
Configure webhook actions
Set up PagerDuty integration (if needed)

13.4 Configure Dashboards

Create Grafana dashboard for service health
Create Grafana dashboard for performance metrics
Create Grafana dashboard for business metrics
Create Grafana dashboard for error tracking
Create Azure custom dashboards
Configure shared dashboards
Set up access permissions

Phase 14: Testing & Validation ✅

Duration: 3-5 days
Dependencies: All previous phases

14.1 Health Checks

Verify all pods running
Check all service endpoints
Verify all health endpoints responding
Check service logs for errors

14.2 Integration Testing

Test Identity Service API endpoints
Test Intake Service API endpoints
Test Finance Service API endpoints
Test Dataroom Service API endpoints
Test Portal Public application
Test Portal Internal application
Test authentication flow
Test API integration from frontend

14.3 End-to-End Testing

Test user registration flow
Test application submission flow
Test credential issuance flow
Test payment processing flow
Test document upload flow
Test complete user journeys

14.4 Performance Testing

Run load tests (k6, Apache Bench, or JMeter)
Verify response times acceptable
Verify throughput meets requirements
Verify resource usage within limits
Optimize based on results

14.5 Security Testing

Run Trivy security scan
Check for exposed secrets
Verify network policies configured
Verify RBAC properly set up
Verify TLS/SSL working
Verify authentication required
Test authorization controls

Phase 15: Production Hardening 🔒

Duration: 2-3 days
Dependencies: Phase 14

15.1 Production Configuration

Update replica counts for production
Configure resource limits and requests
Configure liveness probes
Configure readiness probes
Set up horizontal pod autoscaling
Configure pod disruption budgets

15.2 Backup Configuration

Configure database backups
Configure storage backups
Enable blob versioning
Configure retention policies
Set up geo-replication (if needed)
Test backup restore procedures

15.3 Disaster Recovery

Document backup procedures
Test restore procedures
Set up automated backups
Configure multi-region deployment (if needed)
Configure DNS failover
Test disaster recovery procedures

15.4 Documentation

Update deployment documentation
Document all configuration
Create operational runbooks
Document troubleshooting steps
Create incident response procedures
Document escalation procedures

Summary Statistics

Total Steps: 296
Phases: 15
Estimated Duration: 8-12 weeks
Critical Path: Phases 1 → 2 → 4 → 6 → 7 → 8 → 10 → 11 → 12 → 14 → 15
Can Run in Parallel: Phases 2 & 3

Quick Status Tracking

✅ Completed Phases

Phase 1: Prerequisites
Phase 2: Azure Infrastructure Setup
Phase 3: Entra ID Configuration
Phase 4: Database & Storage Setup
Phase 5: Container Registry Setup
Phase 6: Application Build & Package
Phase 7: Database Migrations
Phase 8: Secrets Configuration
Phase 9: Infrastructure Services Deployment
Phase 10: Backend Services Deployment
Phase 11: Frontend Applications Deployment
Phase 12: Networking & Gateways
Phase 13: Monitoring & Observability
Phase 14: Testing & Validation
Phase 15: Production Hardening

Next Steps After Deployment

Monitor: Watch logs and metrics for first 24-48 hours
Optimize: Adjust resource allocations based on actual usage
Document: Update runbooks with lessons learned
Train: Train operations team on new infrastructure
Iterate: Plan next deployment cycle improvements

See DEPLOYMENT_GUIDE.md for detailed instructions for each step.
See DEPLOYMENT_QUICK_REFERENCE.md for quick command reference.

16 KiB Raw Blame History