smom-dbis-138/docs/deployment/VM_DEPLOYMENT.md

# VM Deployment Guide

## Overview

This guide describes how to deploy the Besu network on Azure Virtual Machines (VMs) or Virtual Machine Scale Sets (VMSS) with Docker Engine, as an alternative to AKS deployment.

## Architecture

### Deployment Options

1. **Individual VMs**: Separate VMs for each node (validators, sentries, RPC)
2. **VM Scale Sets**: Auto-scaling VM groups for each node type
3. **Multi-Region**: Deploy nodes across multiple Azure regions for high availability

### Node Types

- **Validators**: Private subnets, no public IPs, IBFT2 consensus
- **Sentries**: Public-facing P2P nodes, peer to validators and sentries
- **RPC Nodes**: Public HTTPS JSON-RPC, no P2P, read-only

## Prerequisites

- Azure CLI installed and configured
- Terraform >= 1.0
- SSH key pair for VM access
- Azure subscription with appropriate permissions
- Resource group created

## Quick Start

### 1. Generate SSH Key (if not exists)

```bash
ssh-keygen -t rsa -b 4096 -C "besu-vm-deployment"
```

### 2. Set Environment Variables

```bash
export SSH_PUBLIC_KEY=$(cat ~/.ssh/id_rsa.pub)
export AZURE_SUBSCRIPTION_ID="your-subscription-id"
export RESOURCE_GROUP_NAME="defi-oracle-mainnet-rg"
export CLUSTER_NAME="defi-oracle-aks"
```

### 3. Configure Terraform Variables

```bash
# Copy example variables file
cp terraform/terraform.tfvars.vm.example terraform/terraform.tfvars.vm

# Edit terraform.tfvars.vm with your values
# Set vm_deployment_enabled = true
# Set ssh_public_key = "$(cat ~/.ssh/id_rsa.pub)"
```

### 4. Deploy Infrastructure

```bash
# Initialize Terraform
cd terraform
terraform init

# Plan deployment (VM deployment)
terraform plan -var-file=terraform.tfvars.vm -var="vm_deployment_enabled=true"

# Apply deployment
terraform apply -var-file=terraform.tfvars.vm -var="vm_deployment_enabled=true"
```

### 5. Alternative: Use Deployment Script

```bash
# Use the deployment script
./scripts/vm-deployment/deploy-vm-network.sh
```

### 4. Setup VMs

After VMs are created, they will be automatically configured via cloud-init. To manually setup:

```bash
# SSH into VM
ssh besuadmin@<vm-public-ip>

# Run setup script
sudo /opt/besu/setup.sh
```

### 5. Verify Deployment

```bash
# Check VM status
az vm list --resource-group $RESOURCE_GROUP_NAME --show-details

# Check Besu container status
ssh besuadmin@<vm-ip> "docker ps"

# Check Besu logs
ssh besuadmin@<vm-ip> "docker logs besu-validator-0"
```

## Manual VM Setup

### 1. Create VM

```bash
# Create resource group
az group create --name $RESOURCE_GROUP_NAME --location eastus

# Create VM
az vm create \
  --resource-group $RESOURCE_GROUP_NAME \
  --name besu-validator-0 \
  --image Ubuntu2204 \
  --size Standard_D4s_v3 \
  --admin-username besuadmin \
  --ssh-key-values ~/.ssh/id_rsa.pub \
  --vnet-name besu-vnet \
  --subnet validators-subnet \
  --nsg besu-validator-nsg
```

### 2. Setup VM

```bash
# Copy setup script to VM
scp scripts/vm-deployment/setup-vm.sh besuadmin@<vm-ip>:~

# SSH into VM
ssh besuadmin@<vm-ip>

# Run setup script
sudo bash setup-vm.sh validator 0
```

### 3. Configure Besu

```bash
# Copy configuration files
scp config/genesis.json besuadmin@<vm-ip>:~/genesis.json
scp config/validators/besu-config.toml besuadmin@<vm-ip>:~/besu-config.toml

# Copy validator keys
scp keys/validator-0/* besuadmin@<vm-ip>:~/keys/
```

### 4. Start Besu

```bash
# SSH into VM
ssh besuadmin@<vm-ip>

# Start Besu container
cd /opt/besu
docker compose up -d

# Check status
docker ps
docker logs besu-validator-0
```

## VM Scale Sets Deployment

### Deploy VM Scale Set

```bash
# Update terraform.tfvars.vm
use_vmss = true

# Apply Terraform
terraform apply -var-file=terraform.tfvars.vm
```

### Scale VM Scale Set

```bash
# Scale validators
az vmss scale \
  --resource-group $RESOURCE_GROUP_NAME \
  --name besu-validator-vmss \
  --new-capacity 4

# Scale RPC nodes
az vmss scale \
  --resource-group $RESOURCE_GROUP_NAME \
  --name besu-rpc-vmss \
  --new-capacity 5
```

## Multi-Region Deployment

### Deploy to Multiple Regions

```bash
# Update terraform.tfvars.vm
vm_regions = ["eastus", "westus", "westeurope", "southeastasia"]

# Apply Terraform
terraform apply -var-file=terraform.tfvars.vm
```

### Configure Cross-Region Peering

```bash
# Create VNet peering between regions
az network vnet peering create \
  --resource-group $RESOURCE_GROUP_NAME \
  --name eastus-to-westus \
  --vnet-name besu-vnet-eastus \
  --remote-vnet besu-vnet-westus \
  --allow-vnet-access
```

## Monitoring

### View VM Metrics

```bash
# View VM metrics
az monitor metrics list \
  --resource /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Compute/virtualMachines/besu-validator-0 \
  --metric "Percentage CPU" \
  --start-time 2024-01-01T00:00:00Z
```

### View Besu Logs

```bash
# SSH into VM
ssh besuadmin@<vm-ip>

# View logs
docker logs -f besu-validator-0

# View logs from file
tail -f /opt/besu/logs/besu.log
```

### View Metrics

```bash
# Check metrics endpoint
curl http://<vm-ip>:9545/metrics
```

## Backup and Recovery

### Backup Chaindata

```bash
# SSH into VM
ssh besuadmin@<vm-ip>

# Stop Besu
docker compose down

# Backup data
tar -czf besu-data-backup-$(date +%Y%m%d).tar.gz /opt/besu/data

# Upload to Azure Storage
az storage blob upload \
  --account-name $STORAGE_ACCOUNT_NAME \
  --container-name backups \
  --name besu-data-backup-$(date +%Y%m%d).tar.gz \
  --file besu-data-backup-$(date +%Y%m%d).tar.gz

# Restart Besu
docker compose up -d
```

### Restore Chaindata

```bash
# Download backup
az storage blob download \
  --account-name $STORAGE_ACCOUNT_NAME \
  --container-name backups \
  --name besu-data-backup-20240101.tar.gz \
  --file besu-data-backup-20240101.tar.gz

# Stop Besu
docker compose down

# Restore data
tar -xzf besu-data-backup-20240101.tar.gz -C /

# Restart Besu
docker compose up -d
```

## Troubleshooting

### VM Not Accessible

```bash
# Check VM status
az vm show --resource-group $RESOURCE_GROUP_NAME --name besu-validator-0 --show-details

# Check NSG rules
az network nsg rule list --resource-group $RESOURCE_GROUP_NAME --nsg-name besu-validator-nsg

# Restart VM
az vm restart --resource-group $RESOURCE_GROUP_NAME --name besu-validator-0
```

### Besu Container Not Starting

```bash
# SSH into VM
ssh besuadmin@<vm-ip>

# Check container logs
docker logs besu-validator-0

# Check systemd service
systemctl status besu.service

# Check Docker
docker ps -a
systemctl status docker
```

### Network Issues

```bash
# Check network connectivity
ping <validator-ip>

# Check P2P port
telnet <sentry-ip> 30303

# Check RPC port
curl http://<rpc-ip>:8545
```

## Cost Optimization

### Use Spot VMs

```bash
# Create VM with spot pricing
az vm create \
  --resource-group $RESOURCE_GROUP_NAME \
  --name besu-validator-0 \
  --priority Spot \
  --max-price -1 \
  --eviction-policy Deallocate
```

### Use Reserved Instances

```bash
# Purchase reserved instance
az vm reservation create \
  --resource-group $RESOURCE_GROUP_NAME \
  --reserved-resource-type VirtualMachines \
  --billing-scope /subscriptions/$SUBSCRIPTION_ID \
  --term P1Y \
  --quantity 1 \
  --sku Standard_D4s_v3
```

## Security

### Network Security

- Use Network Security Groups (NSGs) to restrict access
- Use private subnets for validators
- Use public IPs only for sentries and RPC nodes
- Implement firewall rules

### Key Management

- Use Azure Key Vault for validator keys
- Use Managed Identity for Key Vault access
- Rotate keys regularly
- Backup keys securely

### Access Control

- Use SSH keys instead of passwords
- Disable root login
- Use Azure AD for VM access
- Implement just-in-time access

## Comparison: AKS vs VM Deployment

### AKS Deployment

**Pros**:
- Kubernetes orchestration
- Auto-scaling
- Service discovery
- Rolling updates
- Resource management

**Cons**:
- More complex setup
- Higher cost (control plane)
- Requires Kubernetes expertise

### VM Deployment

**Pros**:
- Simpler setup
- Lower cost (no control plane)
- Full control over VMs
- Easy to understand
- Direct Docker access

**Cons**:
- Manual scaling
- Manual updates
- No service discovery
- More manual configuration

## Recommendations

1. **Use AKS for production**: Better orchestration and management
2. **Use VMs for development**: Simpler and cheaper
3. **Use VMSS for auto-scaling**: Better than individual VMs
4. **Multi-region deployment**: High availability and disaster recovery
5. **Use Managed Disks**: Better performance and reliability

## Troubleshooting

See [VM Deployment Troubleshooting Guide](VM_DEPLOYMENT_TROUBLESHOOTING.md) for common issues and solutions.

## Checklist

See [VM Deployment Checklist](VM_DEPLOYMENT_CHECKLIST.md) for a comprehensive deployment checklist.

## References

- [Azure VM Documentation](https://docs.microsoft.com/azure/virtual-machines/)
- [Azure VMSS Documentation](https://docs.microsoft.com/azure/virtual-machine-scale-sets/)
- [Docker Documentation](https://docs.docker.com/)
- [Besu Documentation](https://besu.hyperledger.org/)
- [Cloud-init Documentation](https://cloudinit.readthedocs.io/)