Sankofa/docs/VM_CREATION_PROCEDURE.md

VM Creation Procedure - Complete Guide

Last Updated: 2025-12-11
Status: ✅ Complete with Guest Agent Configuration

---

Overview

This document provides step-by-step procedures for creating VMs in the Sankofa Phoenix infrastructure using Crossplane and Proxmox. All procedures ensure proper QEMU Guest Agent configuration.

---

Prerequisites

1. Crossplane Provider Installed

   • Provider configured and connected to Proxmox
   • Provider config secret created

2. Proxmox Access

   • Valid credentials in .env file
   • Network access to Proxmox nodes

3. Kubernetes Access

   • kubectl configured
   • Access to target namespace

---

Method 1: Using Production Templates (Recommended)

Step 1: Choose Template

Available templates in examples/production/:

• basic-vm.yaml - 2 CPU, 4Gi RAM, 50Gi disk
• medium-vm.yaml - 4 CPU, 8Gi RAM, 100Gi disk
• large-vm.yaml - 8 CPU, 16Gi RAM, 200Gi disk

Step 2: Customize Template

Required changes:

• metadata.name - Unique VM name
• spec.forProvider.name - VM name in Proxmox
• spec.forProvider.node - Proxmox node (ml110-01 or r630-01)
• spec.forProvider.site - Site identifier
• userData.users[0].ssh_authorized_keys - Your SSH public key

Example:

apiVersion: proxmox.sankofa.nexus/v1alpha1
kind: ProxmoxVM
metadata:
  name: my-vm-001
  namespace: default
spec:
  forProvider:
    node: "ml110-01"
    name: "my-vm-001"
    cpu: 2
    memory: "4Gi"
    disk: "50Gi"
    # ... rest of config
    userData: |
      #cloud-config
      users:
        - name: admin
          ssh_authorized_keys:
            - ssh-rsa YOUR_PUBLIC_KEY_HERE
      # ... rest of cloud-init

Step 3: Apply Template

kubectl apply -f examples/production/basic-vm.yaml

Step 4: Monitor Creation

# Watch VM status
kubectl get proxmoxvm -w

# Check events
kubectl describe proxmoxvm <vm-name>

# Check logs (if provider logs available)
kubectl logs -n crossplane-system -l app=crossplane-provider-proxmox

Step 5: Verify Guest Agent

Wait for cloud-init (1-2 minutes), then:

# On Proxmox node
ssh root@<proxmox-node>

# Get VMID
VMID=$(qm list | grep <vm-name> | awk '{print $1}')

# Check guest agent
qm config $VMID | grep agent
qm guest exec $VMID -- systemctl status qemu-guest-agent

---

Method 2: Using Crossplane Examples

Step 1: Use Example Template

# Copy example
cp crossplane-provider-proxmox/examples/vm-example.yaml my-vm.yaml

# Edit and customize
vim my-vm.yaml

Step 2: Apply

kubectl apply -f my-vm.yaml

Step 3: Verify

Same as Method 1, Step 5.

---

Method 3: Using GitOps Templates

Step 1: Use Template

Templates in gitops/templates/vm/:

• ubuntu-22.04.yaml
• ubuntu-20.04.yaml
• debian-12.yaml

Step 2: Render with Values

Create values file:

name: my-vm
namespace: default
node: ml110-01
cpu: 2
memory: 4Gi
disk: 50Gi
site: us-sfvalley

Render template:

# Using helm or similar tool
helm template my-vm gitops/templates/vm/ubuntu-22.04.yaml -f values.yaml

Step 3: Apply

kubectl apply -f rendered-template.yaml

---

Guest Agent Configuration

Automatic Configuration

✅ All templates include:

• qemu-guest-agent package in cloud-init
• Service enablement and startup
• Verification with retry logic
• Error handling and automatic installation

✅ Crossplane provider automatically:

• Sets agent: 1 in Proxmox VM config
• Enables guest agent communication channel

Manual Verification

After VM creation (wait 1-2 minutes for cloud-init):

# On Proxmox node
VMID=<vm-id>

# Check Proxmox config
qm config $VMID | grep agent
# Expected: agent: 1

# Check package
qm guest exec $VMID -- dpkg -l | grep qemu-guest-agent
# Expected: Package listed as installed

# Check service
qm guest exec $VMID -- systemctl status qemu-guest-agent
# Expected: active (running)

Manual Fix (if needed)

If guest agent not working:

# 1. Enable in Proxmox
qm set $VMID --agent 1

# 2. Install in guest (via console/SSH)
apt-get update
apt-get install -y qemu-guest-agent
systemctl enable qemu-guest-agent
systemctl start qemu-guest-agent

# 3. Restart VM
qm shutdown $VMID  # Graceful
# OR
qm stop $VMID && qm start $VMID  # Force

---

Post-Creation Checklist

• VM created successfully
• VM is running (qm status <VMID>)
• Guest agent enabled in Proxmox (agent: 1)
• Guest agent package installed
• Guest agent service running
• Cloud-init completed (check /var/log/cloud-init-output.log)
• SSH access working
• Network connectivity verified
• Time synchronization working (NTP)
• Security updates configured

---

Troubleshooting

VM Not Created

Check:

# Provider status
kubectl get providerconfig

# VM resource status
kubectl describe proxmoxvm <vm-name>

# Provider logs
kubectl logs -n crossplane-system -l app=crossplane-provider-proxmox

Common issues:

• Provider not connected
• Invalid credentials
• Resource quota exceeded
• Network connectivity issues

VM Created But Not Starting

Check:

# VM status
qm status <VMID>

# VM config
qm config <VMID>

# Boot order
qm config <VMID> | grep boot

# Disk
qm config <VMID> | grep disk

Common issues:

• Missing boot disk
• Incorrect boot order
• Disk not imported
• Network configuration issues

Guest Agent Not Working

See: docs/GUEST_AGENT_COMPLETE_PROCEDURE.md

Quick fix:

# Enable in Proxmox
qm set <VMID> --agent 1

# Install in guest
qm guest exec <VMID> -- apt-get install -y qemu-guest-agent
qm guest exec <VMID> -- systemctl start qemu-guest-agent

# Restart VM
qm shutdown <VMID>

---

Best Practices

1. Always Use Templates

• Use production templates from examples/production/
• Templates include all required configurations
• Ensures consistency across VMs

2. Include Guest Agent

• All templates include guest agent configuration
• Verify after creation
• Monitor service status

3. Use Proper Naming

• Follow naming conventions
• Include environment/tenant identifiers
• Use descriptive names

4. Configure SSH Keys

• Always include SSH public keys in cloud-init
• Use ssh_authorized_keys in userData
• Disable password authentication

5. Monitor Resources

• Check resource quotas before creation
• Monitor disk usage
• Set appropriate resource limits

6. Document Exceptions

• Document any custom configurations
• Note any deviations from templates
• Record troubleshooting steps

---

Related Documents

• docs/GUEST_AGENT_COMPLETE_PROCEDURE.md - Guest agent setup
• docs/VM_100_GUEST_AGENT_FIXED.md - Specific VM troubleshooting
• examples/production/ - Production templates
• crossplane-provider-proxmox/examples/ - Provider examples

---

Quick Reference

Create VM:

kubectl apply -f examples/production/basic-vm.yaml

Check status:

kubectl get proxmoxvm
qm list

Verify guest agent:

qm config <VMID> | grep agent
qm guest exec <VMID> -- systemctl status qemu-guest-agent

Access VM:

# Get IP
qm guest exec <VMID> -- hostname -I

# SSH
ssh admin@<vm-ip>

---

Last Updated: 2025-12-11