# Proxmox VE Management

Comprehensive management tools and integrations for Proxmox VE virtualization infrastructure.

## Overview

This directory contains management components for Proxmox VE clusters deployed across Sankofa Phoenix edge sites. It complements the existing Crossplane provider (`crossplane-provider-proxmox/`) with additional tooling for operations, monitoring, and automation.

## Components

### API Client (`api/`)

Proxmox API client utilities and helpers for:
- Cluster operations
- Storage management
- Network configuration
- Backup operations
- Node management

### Terraform (`terraform/`)

Terraform modules for:
- Proxmox cluster provisioning
- Storage pool configuration
- Network bridge setup
- Resource pool management

### Ansible (`ansible/`)

Ansible roles and playbooks for:
- Cluster deployment
- Node configuration
- Storage setup
- Network configuration
- Monitoring agent installation

### Scripts (`scripts/`)

Management scripts for:
- Cluster health checks
- Backup automation
- Disaster recovery
- Performance tuning
- Maintenance operations

## Integration with Crossplane Provider

The Proxmox management components work alongside the Crossplane provider:

- **Crossplane Provider**: Declarative VM management via Kubernetes
- **Management Tools**: Operational tasks, monitoring, and automation
- **API Client**: Direct Proxmox API access for advanced operations

## Usage

### Cluster Setup

```bash
# Setup a new Proxmox cluster
./scripts/setup-cluster.sh \
  --site us-east-1 \
  --nodes pve1,pve2,pve3 \
  --storage local-lvm \
  --network vmbr0
```

### Storage Management

```bash
# Add storage pool
./scripts/add-storage.sh \
  --pool ceph-storage \
  --type ceph \
  --nodes pve1,pve2,pve3
```

### Network Configuration

```bash
# Configure network bridge
./scripts/configure-network.sh \
  --bridge vmbr1 \
  --vlan 100 \
  --nodes pve1,pve2,pve3
```

### Ansible Deployment

```bash
# Deploy Proxmox configuration
cd ansible
ansible-playbook -i inventory.yml site-deployment.yml \
  -e site=us-east-1 \
  -e nodes="pve1,pve2,pve3"
```

### Terraform

```bash
# Provision Proxmox infrastructure
cd terraform
terraform init
terraform plan -var="site=us-east-1"
terraform apply
```

## Configuration

### Site Configuration

Each Proxmox site requires configuration:

```yaml
site: us-east-1
nodes:
  - name: pve1
    ip: 10.1.0.10
    role: master
  - name: pve2
    ip: 10.1.0.11
    role: worker
  - name: pve3
    ip: 10.1.0.12
    role: worker
storage:
  pools:
    - name: local-lvm
      type: lvm
    - name: ceph-storage
      type: ceph
networks:
  bridges:
    - name: vmbr0
      type: bridge
      vlan: untagged
    - name: vmbr1
      type: bridge
      vlan: 100
```

### API Authentication

Proxmox API authentication via tokens:

```bash
# Create API token
export PROXMOX_API_URL=https://pve1.sankofa.nexus:8006
export PROXMOX_API_TOKEN=root@pam!token-name=abc123def456
```

## Monitoring

Proxmox monitoring integrates with the Prometheus stack:

- **pve_exporter**: Prometheus metrics exporter
- **Grafana Dashboards**: Pre-built dashboards for Proxmox
- **Alerts**: Alert rules for cluster health

See [Monitoring](../monitoring/README.md) for details.

## Backup and Recovery

### Automated Backups

```bash
# Configure backup schedule
./scripts/configure-backups.sh \
  --schedule "0 2 * * *" \
  --retention 30 \
  --storage backup-storage
```

### Disaster Recovery

```bash
# Restore from backup
./scripts/restore-backup.sh \
  --backup backup-20240101 \
  --target pve1
```

## Multi-Site Management

For managing multiple Proxmox sites:

```bash
# List all sites
./scripts/list-sites.sh

# Get site status
./scripts/site-status.sh --site us-east-1

# Sync configuration across sites
./scripts/sync-config.sh --sites us-east-1,eu-west-1
```

## Security

- API tokens with least privilege
- TLS/SSL for all API communications
- Network isolation via VLANs
- Regular security updates
- Audit logging

## Troubleshooting

### Common Issues

**Cluster split-brain:**
```bash
./scripts/fix-split-brain.sh --site us-east-1
```

**Storage issues:**
```bash
./scripts/diagnose-storage.sh --pool local-lvm
```

**Network connectivity:**
```bash
./scripts/test-network.sh --node pve1
```

## Related Documentation

- [Crossplane Provider](../../crossplane-provider-proxmox/README.md)
- [System Architecture](../../docs/system_architecture.md)
- [Deployment Scripts](../../scripts/README.md)