c39465c2bd
Some checks failed
Test / test (push) Has been cancelled
Co-authored-by: Cursor <cursoragent@cursor.com>
221 lines
12 KiB
Markdown
221 lines
12 KiB
Markdown
# Architecture Overview
|
|
|
|
## System Architecture
|
|
|
|
This document describes the complete architecture of the Proxmox VE → Azure Arc → Hybrid Cloud Stack implementation.
|
|
|
|
## High-Level Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Azure Portal │
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
|
│ │ Azure Arc │ │ Azure Policy │ │ Azure Monitor │ │
|
|
│ │ Servers │ │ │ │ │ │
|
|
│ └──────────────┘ └──────────────┘ └──────────────┘ │
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
|
│ │ Arc K8s │ │ GitOps │ │ Defender │ │
|
|
│ │ │ │ (Flux) │ │ for Cloud │ │
|
|
│ └──────────────┘ └──────────────┘ └──────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
│ Azure Arc Connection
|
|
│
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ On-Premises Infrastructure │
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ Proxmox VE Cluster (2 Nodes) │ │
|
|
│ │ ┌──────────────┐ ┌──────────────┐ │ │
|
|
│ │ │ PVE Node 1 │◄────────────►│ PVE Node 2 │ │ │
|
|
│ │ │ │ Cluster │ │ │ │
|
|
│ │ │ Azure Arc │ Network │ Azure Arc │ │ │
|
|
│ │ │ Agent │ │ Agent │ │ │
|
|
│ │ └──────────────┘ └──────────────┘ │ │
|
|
│ │ │ │ │ │
|
|
│ │ └───────────┬───────────────┘ │ │
|
|
│ │ │ │ │
|
|
│ │ ┌──────▼──────┐ │ │
|
|
│ │ │ NFS Storage │ │ │
|
|
│ │ │ (Shared) │ │ │
|
|
│ │ └─────────────┘ │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ Proxmox VMs │ │
|
|
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
|
|
│ │ │ K3s VM │ │ Git Server │ │ Other VMs │ │ │
|
|
│ │ │ │ │ (Gitea/ │ │ │ │ │
|
|
│ │ │ Azure Arc │ │ GitLab) │ │ Azure Arc │ │ │
|
|
│ │ │ K8s │ │ │ │ Agents │ │ │
|
|
│ │ │ Resource │ │ │ │ │ │ │
|
|
│ │ │ Bridge │ │ │ │ │ │ │
|
|
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ Kubernetes Cluster (K3s) │ │
|
|
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
|
|
│ │ │ Ingress │ │ Cert- │ │ GitOps │ │ │
|
|
│ │ │ Controller │ │ Manager │ │ (Flux) │ │ │
|
|
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
|
|
│ │ │ │
|
|
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
|
|
│ │ │ Besu │ │ Firefly │ │ Chainlink │ │ │
|
|
│ │ │ (Ethereum) │ │ (Middleware)│ │ CCIP │ │ │
|
|
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
|
|
│ │ │ │
|
|
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
|
|
│ │ │ Blockscout │ │ Cacti │ │ NGINX │ │ │
|
|
│ │ │ (Explorer) │ │ (Monitoring) │ │ Proxy │ │ │
|
|
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Component Details
|
|
|
|
### 1. Proxmox VE Cluster
|
|
|
|
**Purpose**: Hypervisor layer providing virtualization and high availability
|
|
|
|
**Components**:
|
|
- 2 Proxmox nodes in cluster configuration
|
|
- Shared NFS storage for VM data
|
|
- Linux bridge networking (vmbr0)
|
|
- Corosync for cluster communication
|
|
|
|
**Features**:
|
|
- High availability (HA) for VMs
|
|
- Live migration between nodes
|
|
- Centralized management via web UI
|
|
- Azure Arc integration for portal visibility
|
|
|
|
### 2. Azure Arc Integration
|
|
|
|
**Purpose**: Extend Azure management capabilities to on-premises infrastructure
|
|
|
|
**Components**:
|
|
- **Azure Connected Machine Agent**: Installed on Proxmox hosts and VMs
|
|
- **Azure Arc Kubernetes**: K3s cluster onboarded to Azure Arc
|
|
- **Resource Bridge**: Custom Kubernetes-based bridge for VM lifecycle control
|
|
- **GitOps Extension**: Flux-based GitOps for declarative deployments
|
|
|
|
**Capabilities**:
|
|
- VM visibility in Azure Portal
|
|
- Azure Policy enforcement
|
|
- Azure Update Manager
|
|
- Defender for Cloud
|
|
- Azure Monitor integration
|
|
- GitOps-based deployments
|
|
|
|
### 3. Kubernetes (K3s)
|
|
|
|
**Purpose**: Container orchestration platform for HC Stack services
|
|
|
|
**Components**:
|
|
- K3s lightweight Kubernetes distribution
|
|
- NGINX Ingress Controller
|
|
- Cert-Manager for TLS certificates
|
|
- Flux GitOps operator
|
|
|
|
**Namespaces**:
|
|
- `hc-stack`: Core infrastructure
|
|
- `blockchain`: Blockchain services (Besu, Firefly, Chainlink)
|
|
- `monitoring`: Monitoring tools (Cacti)
|
|
- `ingress-nginx`: Ingress controller
|
|
- `cert-manager`: Certificate management
|
|
|
|
### 4. Hybrid Cloud Stack Services
|
|
|
|
#### Hyperledger Besu
|
|
- Ethereum client for blockchain operations
|
|
- RPC endpoints (HTTP/WebSocket)
|
|
- P2P networking
|
|
- Metrics and monitoring
|
|
|
|
#### Hyperledger Firefly
|
|
- Blockchain middleware and API layer
|
|
- Multi-party system support
|
|
- Token and asset management
|
|
- Event streaming
|
|
|
|
#### Chainlink CCIP
|
|
- Cross-chain interoperability protocol
|
|
- Oracle services
|
|
- Secure cross-chain messaging
|
|
|
|
#### Blockscout
|
|
- Blockchain explorer
|
|
- Transaction and block visualization
|
|
- Contract verification
|
|
- Analytics dashboard
|
|
|
|
#### Cacti
|
|
- Network monitoring and graphing
|
|
- Performance metrics
|
|
- Alerting capabilities
|
|
|
|
#### NGINX Proxy
|
|
- Reverse proxy for all services
|
|
- Load balancing
|
|
- SSL termination
|
|
|
|
### 5. Private Git/DevOps
|
|
|
|
**Options**:
|
|
- **Gitea**: Lightweight Git server (recommended for small deployments)
|
|
- **GitLab CE**: Full-featured DevOps platform
|
|
- **Azure DevOps**: Self-hosted agents for Azure DevOps pipelines
|
|
|
|
**Purpose**:
|
|
- Version control for infrastructure and applications
|
|
- CI/CD pipeline execution
|
|
- GitOps repository for Kubernetes deployments
|
|
|
|
## Data Flow
|
|
|
|
1. **Infrastructure Management**:
|
|
- Terraform → Proxmox API → VM Creation
|
|
- Azure Arc Agent → Azure Portal → Visibility & Management
|
|
|
|
2. **Application Deployment**:
|
|
- Git Repository → Flux GitOps → Kubernetes API → Pod Deployment
|
|
- Azure Arc GitOps → Flux → Kubernetes → Application Updates
|
|
|
|
3. **Monitoring & Observability**:
|
|
- Services → Metrics → Azure Monitor / Cacti
|
|
- Logs → Azure Log Analytics / Local Storage
|
|
|
|
## Security Architecture
|
|
|
|
- **Network Isolation**: Separate networks for management, storage, and application traffic
|
|
- **Azure Arc Security**: Managed identities and RBAC
|
|
- **Kubernetes Security**: RBAC, network policies, pod security policies
|
|
- **TLS/SSL**: Cert-Manager for automatic certificate management
|
|
- **Secrets Management**: Kubernetes secrets (consider Azure Key Vault integration)
|
|
|
|
## High Availability
|
|
|
|
- **Proxmox Cluster**: 2-node cluster with shared storage
|
|
- **VM HA**: Automatic failover for VMs
|
|
- **Kubernetes**: Multiple replicas for stateless services
|
|
- **Storage**: NFS shared storage for persistent data
|
|
- **Load Balancing**: NGINX Ingress for service distribution
|
|
|
|
## Scalability
|
|
|
|
- **Horizontal Scaling**: Add more Proxmox nodes to cluster
|
|
- **Kubernetes Scaling**: Add worker nodes to K3s cluster
|
|
- **Application Scaling**: Kubernetes HPA for automatic scaling
|
|
- **Storage Scaling**: Expand NFS storage as needed
|
|
|
|
## Integration Points
|
|
|
|
1. **Azure Portal**: Full visibility and management
|
|
2. **Git Repository**: Source of truth for infrastructure and applications
|
|
3. **Kubernetes API**: Application deployment and management
|
|
4. **Proxmox API**: VM lifecycle management
|
|
5. **Monitoring Systems**: Metrics and alerting
|
|
|