loc_az_hci/docs/template-improvements.md

# Template 9000 Improvement Recommendations

## Current State

The template VM 9000 (`ubuntu-24.04-cloud-init`) is a basic Ubuntu 24.04 cloud image with:
- ✅ Cloud-init configured with SSH keys
- ✅ DHCP IP configuration
- ✅ QEMU Guest Agent enabled in VM config (but **not installed in guest OS**)
- ✅ Basic Ubuntu 24.04 cloud image

## Recommended Improvements

### 🔴 Critical (High Priority)

#### 1. **Pre-install QEMU Guest Agent in Template**
**Why:** Currently, QEMU Guest Agent is enabled in VM config but not installed in the guest OS. This means every cloned VM needs manual installation.

**How:** Boot the template VM, install QGA, then convert back to template:
```bash
# Boot template VM 9000
qm start 9000

# SSH into it and install QGA
ssh ubuntu@<template-ip>
sudo apt-get update
sudo apt-get install -y qemu-guest-agent
sudo systemctl enable qemu-guest-agent
sudo systemctl start qemu-guest-agent

# Stop and convert back to template
qm stop 9000
qm template 9000
```

**Benefit:** All cloned VMs will have QGA ready immediately, enabling IP discovery from first boot.

#### 2. **Pre-install Essential Utilities**
**Why:** Every VM needs these tools, installing them in template saves time.

**Packages to add:**
- `jq` - JSON parsing (needed for guest-agent IP discovery)
- `curl`, `wget` - HTTP clients
- `git` - Version control
- `vim` or `nano` - Text editors
- `net-tools` - Network utilities (ifconfig, netstat)
- `htop` - Process monitor
- `unattended-upgrades` - Automatic security updates
- `apt-transport-https` - HTTPS apt support
- `ca-certificates` - SSL certificates

**Benefit:** Faster VM provisioning, consistent tooling across all VMs.

### 🟡 Important (Medium Priority)

#### 3. **Configure Automatic Security Updates**
**Why:** Keep all VMs secure with minimal manual intervention.

**Configuration:**
```bash
sudo apt-get install -y unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades
# Or configure via /etc/apt/apt.conf.d/50unattended-upgrades
```

**Benefit:** Automatic security patches, reduced maintenance overhead.

#### 4. **Set Timezone and Locale**
**Why:** Consistent timezone across all VMs, proper locale for logs.

**Configuration:**
```bash
sudo timedatectl set-timezone UTC
sudo locale-gen en_US.UTF-8
sudo update-locale LANG=en_US.UTF-8
```

**Benefit:** Consistent timestamps, proper character encoding.

#### 5. **SSH Hardening**
**Why:** Improve security posture from template.

**Configuration:**
```bash
# Edit /etc/ssh/sshd_config
sudo sed -i 's/#PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
sudo sed -i 's/#PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo sed -i 's/#PubkeyAuthentication.*/PubkeyAuthentication yes/' /etc/ssh/sshd_config
sudo systemctl restart sshd
```

**Benefit:** Better security defaults, reduces attack surface.

#### 6. **Configure Log Rotation**
**Why:** Prevent disk space issues from log growth.

**Configuration:**
```bash
# Ensure logrotate is configured properly
sudo logrotate -f /etc/logrotate.conf
```

**Benefit:** Prevents disk full issues from logs.

### 🟢 Nice to Have (Low Priority)

#### 7. **Pre-configure Firewall (UFW)**
**Why:** Enable firewall but don't block anything by default (let VMs configure as needed).

**Configuration:**
```bash
sudo apt-get install -y ufw
sudo ufw --force enable
# Don't add rules - let each VM configure as needed
```

**Benefit:** Firewall ready but not blocking, each VM can configure rules.

#### 8. **Add Cloud-init User Data Template**
**Why:** Allow per-VM customization via cloud-init user-data.

**Create:** `/etc/cloud/cloud.cfg.d/99-custom.cfg` with common settings:
```yaml
# Example cloud-init user-data template
# This can be overridden per-VM via Proxmox cicustom parameter
users:
  - default
  - name: ubuntu
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash

# Common packages to install
package_update: true
package_upgrade: true
packages:
  - jq
  - curl
  - wget
  - git
  - vim
  - htop

# Timezone
timezone: UTC

# SSH configuration
ssh_pwauth: false
disable_root: true
```

**Benefit:** Flexible per-VM customization while maintaining base template.

#### 9. **Pre-configure Swap (Optional)**
**Why:** Some VMs may benefit from swap, but it's better to configure per-VM.

**Recommendation:** Don't add swap to template - configure per-VM based on workload.

#### 10. **Add Monitoring Agent Support (Optional)**
**Why:** If you plan to use monitoring agents (Prometheus node exporter, etc.), pre-install in template.

**Configuration:**
```bash
# Example: Prometheus node exporter
# Only if all VMs will use it
```

**Benefit:** Consistent monitoring across all VMs.

#### 11. **Optimize Disk Image**
**Why:** Reduce template size and improve clone speed.

**Actions:**
```bash
# After installing packages, clean up
sudo apt-get autoremove -y
sudo apt-get autoclean
sudo rm -rf /tmp/*
sudo rm -rf /var/tmp/*
sudo truncate -s 0 /var/log/*.log
sudo journalctl --vacuum-time=1d
```

**Benefit:** Smaller template, faster clones.

#### 12. **Add EFI Boot Support (Already Present)**
**Status:** ✅ Already configured with `--bios ovmf --efidisk0`

**Benefit:** Secure boot support, modern boot standard.

## Implementation Script

Create a script to apply all improvements to template 9000:

**File:** `scripts/infrastructure/improve-template-9000.sh`

This script would:
1. Boot template VM 9000
2. Wait for SSH access
3. Install all recommended packages
4. Configure system settings (timezone, locale, SSH, etc.)
5. Install QEMU Guest Agent
6. Clean up disk
7. Stop VM and convert back to template

## Priority Order

1. **First:** Pre-install QEMU Guest Agent (#1) - Critical for automation
2. **Second:** Pre-install essential utilities (#2) - Saves time on every VM
3. **Third:** Configure automatic security updates (#3) - Security best practice
4. **Fourth:** Set timezone/locale (#4) - Consistency
5. **Fifth:** SSH hardening (#5) - Security
6. **Sixth:** Log rotation (#6) - Prevent issues
7. **Seventh:** Everything else - Nice to have

## Template Update Process

When updating the template:

1. **Clone template to temporary VM:**
   ```bash
   qm clone 9000 9999 --name template-update
   ```

2. **Boot and update:**
   ```bash
   qm start 9999
   # Wait for boot, then SSH and apply changes
   ```

3. **Test the updated template:**
   ```bash
   # Clone to test VM
   qm clone 9999 9998 --name template-test
   qm start 9998
   # Verify everything works
   ```

4. **Replace original template:**
   ```bash
   qm stop 9999
   qm template 9999
   qm destroy 9000
   qm set 9999 --vmid 9000
   ```

## Notes

- **Don't install Docker in template** - Different VMs may need different Docker versions/configurations
- **Don't install service-specific software** - Keep template generic
- **Do install common utilities** - Things every VM needs
- **Do configure security defaults** - Better security posture from start
- **Do document changes** - Keep a changelog of template updates

## Template Versioning

Consider adding version metadata to template:
- Add a file `/etc/template-version` with version number and date
- Update this file each time template is improved
- Scripts can check this to verify template version

Example:
```bash
echo "template-9000-v1.1.0-$(date +%Y%m%d)" > /etc/template-version
```