d-bis/proxmox
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4eead3e53fecdb2b7e629bb9f56b22ff92c61d2e
proxmox/docs/02-architecture/PROXMOX_CLUSTER_ARCHITECTURE.md
defiQUG cb47cce074 Complete markdown files cleanup and organization 
- Organized 252 files across project
- Root directory: 187 → 2 files (98.9% reduction)
- Moved configuration guides to docs/04-configuration/
- Moved troubleshooting guides to docs/09-troubleshooting/
- Moved quick start guides to docs/01-getting-started/
- Moved reports to reports/ directory
- Archived temporary files
- Generated comprehensive reports and documentation
- Created maintenance scripts and guides

All files organized according to established standards.
2026-01-06 01:46:25 -08:00

251 lines
6.7 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Proxmox Cluster Architecture
**Last Updated:** 2025-01-20
**Document Version:** 1.0
**Status:** Active Documentation
---
## Overview
This document describes the Proxmox cluster architecture, including node configuration, storage setup, network bridges, and VM/container distribution.
---
## Cluster Architecture Diagram
```mermaid
graph TB
Cluster[Proxmox Cluster<br/>Name: h]
ML110[ML110 Management Node<br/>192.168.11.10<br/>6 cores, 125GB RAM]
R6301[R630-01<br/>192.168.11.11<br/>32 cores, 503GB RAM]
R6302[R630-02<br/>192.168.11.12<br/>32 cores, 503GB RAM]
R6303[R630-03<br/>192.168.11.13<br/>32 cores, 512GB RAM]
R6304[R630-04<br/>192.168.11.14<br/>32 cores, 512GB RAM]
Cluster --> ML110
Cluster --> R6301
Cluster --> R6302
Cluster --> R6303
Cluster --> R6304
ML110 --> Storage1[local: 94GB<br/>local-lvm: 813GB]
R6301 --> Storage2[local: 536GB<br/>local-lvm: Available]
R6302 --> Storage3[local: Available<br/>local-lvm: Available]
R6303 --> Storage4[Storage: Available]
R6304 --> Storage5[Storage: Available]
ML110 --> Bridge1[vmbr0<br/>VLAN-aware]
R6301 --> Bridge2[vmbr0<br/>VLAN-aware]
R6302 --> Bridge3[vmbr0<br/>VLAN-aware]
R6303 --> Bridge4[vmbr0<br/>VLAN-aware]
R6304 --> Bridge5[vmbr0<br/>VLAN-aware]
```
---
## Cluster Nodes
### Node Summary
| Hostname | IP Address | CPU | RAM | Storage | VMs/Containers | Status |
|----------|------------|-----|-----|---------|----------------|--------|
| ml110 | 192.168.11.10 | 6 cores @ 1.60GHz | 125GB | local (94GB), local-lvm (813GB) | 34 | ✅ Active |
| r630-01 | 192.168.11.11 | 32 cores @ 2.40GHz | 503GB | local (536GB), local-lvm (available) | 0 | ✅ Active |
| r630-02 | 192.168.11.12 | 32 cores @ 2.40GHz | 503GB | local (available), local-lvm (available) | 0 | ✅ Active |
| r630-03 | 192.168.11.13 | 32 cores | 512GB | Available | 0 | ✅ Active |
| r630-04 | 192.168.11.14 | 32 cores | 512GB | Available | 0 | ✅ Active |
---
## Storage Configuration
### Storage Types
**local (Directory Storage):**
- Type: Directory-based storage
- Used for: ISO images, container templates, backups
- Location: `/var/lib/vz`
**local-lvm (LVM Thin Storage):**
- Type: LVM thin provisioning
- Used for: VM/container disk images
- Benefits: Thin provisioning, snapshots, efficient space usage
### Storage by Node
**ml110:**
- `local`: 94GB total, 7.4GB used (7.87%)
- `local-lvm`: 813GB total, 214GB used (26.29%)
- Status: ✅ Active and operational
**r630-01:**
- `local`: 536GB total, 0% used
- `local-lvm`: Available (needs activation)
- Status: ⏳ Storage available, ready for use
**r630-02:**
- `local`: Available
- `local-lvm`: Available (needs activation)
- Status: ⏳ Storage available, ready for use
**r630-03/r630-04:**
- Storage: Available
- Status: ⏳ Ready for configuration
---
## Network Configuration
### Network Bridge (vmbr0)
**All nodes use VLAN-aware bridge:**
```bash
# Bridge configuration (all nodes)
auto vmbr0
iface vmbr0 inet static
address 192.168.11.<HOST_IP>/24
gateway 192.168.11.1
bridge-ports <PHYSICAL_INTERFACE>
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 11 110 111 112 120 121 130 132 133 134 140 141 150 160 200 201 202 203
```
**Bridge Features:**
- **VLAN-aware:** Supports multiple VLANs on single bridge
- **Native VLAN:** 11 (MGMT-LAN)
- **Tagged VLANs:** All service VLANs (110-203)
- **802.1Q Trunking:** Enabled for VLAN support
---
## VM/Container Distribution
### Current Distribution
**ml110 (192.168.11.10):**
- **Total:** 34 containers/VMs
- **Services:** All current services running here
- **Breakdown:**
- Besu validators: 5 (VMIDs 1000-1004)
- Besu sentries: 4 (VMIDs 1500-1503)
- Besu RPC: 3+ (VMIDs 2500-2502+)
- Blockscout: 1 (VMID 5000)
- DBIS services: Multiple
- Other services: Various
**r630-01, r630-02, r630-03, r630-04:**
- **Total:** 0 containers/VMs
- **Status:** Ready for VM migration/deployment
---
## High Availability
### Current Setup
- **Cluster Name:** "h"
- **HA Mode:** Active/Standby (manual)
- **Quorum:** 3+ nodes required for quorum
- **Storage:** Local storage (not shared)
### HA Considerations
**Current Limitations:**
- No shared storage (each node has local storage)
- Manual VM migration required
- No automatic failover
**Future Enhancements:**
- Consider shared storage (NFS, Ceph, etc.) for true HA
- Implement automatic VM migration
- Configure HA groups for critical services
---
## Resource Allocation
### CPU Resources
| Node | CPU Cores | CPU Usage | Available |
|------|-----------|-----------|-----------|
| ml110 | 6 @ 1.60GHz | High | Limited |
| r630-01 | 32 @ 2.40GHz | Low | Excellent |
| r630-02 | 32 @ 2.40GHz | Low | Excellent |
| r630-03 | 32 cores | Low | Excellent |
| r630-04 | 32 cores | Low | Excellent |
### Memory Resources
| Node | Total RAM | Used | Available | Usage % |
|------|-----------|------|-----------|---------|
| ml110 | 125GB | 94GB | 31GB | 75% ⚠️ |
| r630-01 | 503GB | ~5GB | ~498GB | 1% ✅ |
| r630-02 | 503GB | ~5GB | ~498GB | 1% ✅ |
| r630-03 | 512GB | Low | High | Low ✅ |
| r630-04 | 512GB | Low | High | Low ✅ |
---
## Storage Recommendations
### For R630 Nodes
**Boot Drives (2×600GB):**
- **Recommended:** ZFS mirror or hardware RAID1
- **Purpose:** Proxmox OS and boot files
- **Benefits:** Redundancy, data integrity
**Data SSDs (6×250GB):**
- **Option 1:** ZFS striped mirrors (3 pairs)
- Capacity: ~750GB usable
- Performance: High
- Redundancy: Good
- **Option 2:** ZFS RAIDZ1 (5 drives + 1 parity)
- Capacity: ~1.25TB usable
- Performance: Good
- Redundancy: Single drive failure tolerance
- **Option 3:** ZFS RAIDZ2 (4 drives + 2 parity)
- Capacity: ~1TB usable
- Performance: Good
- Redundancy: Dual drive failure tolerance
---
## Network Recommendations
### VLAN Configuration
**All Proxmox hosts should:**
- Use VLAN-aware bridge (vmbr0)
- Support all 19 VLANs
- Maintain native VLAN 11 for management
- Enable 802.1Q trunking on physical interfaces
### Network Performance
- **Link Speed:** Ensure 1Gbps or higher for trunk ports
- **Jumbo Frames:** Consider enabling if supported
- **Bonding:** Consider link aggregation for redundancy
---
## Related Documentation
- **[NETWORK_ARCHITECTURE.md](NETWORK_ARCHITECTURE.md)** ⭐⭐⭐ - Network architecture with VLAN plan
- **[PHYSICAL_HARDWARE_INVENTORY.md](PHYSICAL_HARDWARE_INVENTORY.md)** ⭐⭐⭐ - Physical hardware inventory
- **[PROXMOX_COMPREHENSIVE_REVIEW.md](PROXMOX_COMPREHENSIVE_REVIEW.md)** ⭐⭐ - Comprehensive Proxmox review
- **[ORCHESTRATION_DEPLOYMENT_GUIDE.md](ORCHESTRATION_DEPLOYMENT_GUIDE.md)** ⭐⭐⭐ - Deployment orchestration
---
**Last Updated:** 2025-01-20
**Document Version:** 1.0
**Review Cycle:** Quarterly
Reference in New Issue View Git Blame Copy Permalink