smom-dbis-138/docs/deployment/VM_DEPLOYMENT.md

VM Deployment Guide

Overview

This guide describes how to deploy the Besu network on Azure Virtual Machines (VMs) or Virtual Machine Scale Sets (VMSS) with Docker Engine, as an alternative to AKS deployment.

Architecture

Deployment Options

1. Individual VMs: Separate VMs for each node (validators, sentries, RPC)
2. VM Scale Sets: Auto-scaling VM groups for each node type
3. Multi-Region: Deploy nodes across multiple Azure regions for high availability

Node Types

• Validators: Private subnets, no public IPs, IBFT2 consensus
• Sentries: Public-facing P2P nodes, peer to validators and sentries
• RPC Nodes: Public HTTPS JSON-RPC, no P2P, read-only

Prerequisites

• Azure CLI installed and configured
• Terraform >= 1.0
• SSH key pair for VM access
• Azure subscription with appropriate permissions
• Resource group created

Quick Start

1. Generate SSH Key (if not exists)

ssh-keygen -t rsa -b 4096 -C "besu-vm-deployment"

2. Set Environment Variables

export SSH_PUBLIC_KEY=$(cat ~/.ssh/id_rsa.pub)
export AZURE_SUBSCRIPTION_ID="your-subscription-id"
export RESOURCE_GROUP_NAME="defi-oracle-mainnet-rg"
export CLUSTER_NAME="defi-oracle-aks"

3. Configure Terraform Variables

# Copy example variables file
cp terraform/terraform.tfvars.vm.example terraform/terraform.tfvars.vm

# Edit terraform.tfvars.vm with your values
# Set vm_deployment_enabled = true
# Set ssh_public_key = "$(cat ~/.ssh/id_rsa.pub)"

4. Deploy Infrastructure

# Initialize Terraform
cd terraform
terraform init

# Plan deployment (VM deployment)
terraform plan -var-file=terraform.tfvars.vm -var="vm_deployment_enabled=true"

# Apply deployment
terraform apply -var-file=terraform.tfvars.vm -var="vm_deployment_enabled=true"

5. Alternative: Use Deployment Script

# Use the deployment script
./scripts/vm-deployment/deploy-vm-network.sh

4. Setup VMs

After VMs are created, they will be automatically configured via cloud-init. To manually setup:

# SSH into VM
ssh besuadmin@<vm-public-ip>

# Run setup script
sudo /opt/besu/setup.sh

5. Verify Deployment

# Check VM status
az vm list --resource-group $RESOURCE_GROUP_NAME --show-details

# Check Besu container status
ssh besuadmin@<vm-ip> "docker ps"

# Check Besu logs
ssh besuadmin@<vm-ip> "docker logs besu-validator-0"

Manual VM Setup

1. Create VM

# Create resource group
az group create --name $RESOURCE_GROUP_NAME --location eastus

# Create VM
az vm create \
  --resource-group $RESOURCE_GROUP_NAME \
  --name besu-validator-0 \
  --image Ubuntu2204 \
  --size Standard_D4s_v3 \
  --admin-username besuadmin \
  --ssh-key-values ~/.ssh/id_rsa.pub \
  --vnet-name besu-vnet \
  --subnet validators-subnet \
  --nsg besu-validator-nsg

2. Setup VM

# Copy setup script to VM
scp scripts/vm-deployment/setup-vm.sh besuadmin@<vm-ip>:~

# SSH into VM
ssh besuadmin@<vm-ip>

# Run setup script
sudo bash setup-vm.sh validator 0

3. Configure Besu

# Copy configuration files
scp config/genesis.json besuadmin@<vm-ip>:~/genesis.json
scp config/validators/besu-config.toml besuadmin@<vm-ip>:~/besu-config.toml

# Copy validator keys
scp keys/validator-0/* besuadmin@<vm-ip>:~/keys/

4. Start Besu

# SSH into VM
ssh besuadmin@<vm-ip>

# Start Besu container
cd /opt/besu
docker compose up -d

# Check status
docker ps
docker logs besu-validator-0

VM Scale Sets Deployment

Deploy VM Scale Set

# Update terraform.tfvars.vm
use_vmss = true

# Apply Terraform
terraform apply -var-file=terraform.tfvars.vm

Scale VM Scale Set

# Scale validators
az vmss scale \
  --resource-group $RESOURCE_GROUP_NAME \
  --name besu-validator-vmss \
  --new-capacity 4

# Scale RPC nodes
az vmss scale \
  --resource-group $RESOURCE_GROUP_NAME \
  --name besu-rpc-vmss \
  --new-capacity 5

Multi-Region Deployment

Deploy to Multiple Regions

# Update terraform.tfvars.vm
vm_regions = ["eastus", "westus", "westeurope", "southeastasia"]

# Apply Terraform
terraform apply -var-file=terraform.tfvars.vm

Configure Cross-Region Peering

# Create VNet peering between regions
az network vnet peering create \
  --resource-group $RESOURCE_GROUP_NAME \
  --name eastus-to-westus \
  --vnet-name besu-vnet-eastus \
  --remote-vnet besu-vnet-westus \
  --allow-vnet-access

Monitoring

View VM Metrics

# View VM metrics
az monitor metrics list \
  --resource /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Compute/virtualMachines/besu-validator-0 \
  --metric "Percentage CPU" \
  --start-time 2024-01-01T00:00:00Z

View Besu Logs

# SSH into VM
ssh besuadmin@<vm-ip>

# View logs
docker logs -f besu-validator-0

# View logs from file
tail -f /opt/besu/logs/besu.log

View Metrics

# Check metrics endpoint
curl http://<vm-ip>:9545/metrics

Backup and Recovery

Backup Chaindata

# SSH into VM
ssh besuadmin@<vm-ip>

# Stop Besu
docker compose down

# Backup data
tar -czf besu-data-backup-$(date +%Y%m%d).tar.gz /opt/besu/data

# Upload to Azure Storage
az storage blob upload \
  --account-name $STORAGE_ACCOUNT_NAME \
  --container-name backups \
  --name besu-data-backup-$(date +%Y%m%d).tar.gz \
  --file besu-data-backup-$(date +%Y%m%d).tar.gz

# Restart Besu
docker compose up -d

Restore Chaindata

# Download backup
az storage blob download \
  --account-name $STORAGE_ACCOUNT_NAME \
  --container-name backups \
  --name besu-data-backup-20240101.tar.gz \
  --file besu-data-backup-20240101.tar.gz

# Stop Besu
docker compose down

# Restore data
tar -xzf besu-data-backup-20240101.tar.gz -C /

# Restart Besu
docker compose up -d

Troubleshooting

VM Not Accessible

# Check VM status
az vm show --resource-group $RESOURCE_GROUP_NAME --name besu-validator-0 --show-details

# Check NSG rules
az network nsg rule list --resource-group $RESOURCE_GROUP_NAME --nsg-name besu-validator-nsg

# Restart VM
az vm restart --resource-group $RESOURCE_GROUP_NAME --name besu-validator-0

Besu Container Not Starting

# SSH into VM
ssh besuadmin@<vm-ip>

# Check container logs
docker logs besu-validator-0

# Check systemd service
systemctl status besu.service

# Check Docker
docker ps -a
systemctl status docker

Network Issues

# Check network connectivity
ping <validator-ip>

# Check P2P port
telnet <sentry-ip> 30303

# Check RPC port
curl http://<rpc-ip>:8545

Cost Optimization

Use Spot VMs

# Create VM with spot pricing
az vm create \
  --resource-group $RESOURCE_GROUP_NAME \
  --name besu-validator-0 \
  --priority Spot \
  --max-price -1 \
  --eviction-policy Deallocate

Use Reserved Instances

# Purchase reserved instance
az vm reservation create \
  --resource-group $RESOURCE_GROUP_NAME \
  --reserved-resource-type VirtualMachines \
  --billing-scope /subscriptions/$SUBSCRIPTION_ID \
  --term P1Y \
  --quantity 1 \
  --sku Standard_D4s_v3

Security

Network Security

• Use Network Security Groups (NSGs) to restrict access
• Use private subnets for validators
• Use public IPs only for sentries and RPC nodes
• Implement firewall rules

Key Management

• Use Azure Key Vault for validator keys
• Use Managed Identity for Key Vault access
• Rotate keys regularly
• Backup keys securely

Access Control

• Use SSH keys instead of passwords
• Disable root login
• Use Azure AD for VM access
• Implement just-in-time access

Comparison: AKS vs VM Deployment

AKS Deployment

Pros:

• Kubernetes orchestration
• Auto-scaling
• Service discovery
• Rolling updates
• Resource management

Cons:

• More complex setup
• Higher cost (control plane)
• Requires Kubernetes expertise

VM Deployment

Pros:

• Simpler setup
• Lower cost (no control plane)
• Full control over VMs
• Easy to understand
• Direct Docker access

Cons:

• Manual scaling
• Manual updates
• No service discovery
• More manual configuration

Recommendations

1. Use AKS for production: Better orchestration and management
2. Use VMs for development: Simpler and cheaper
3. Use VMSS for auto-scaling: Better than individual VMs
4. Multi-region deployment: High availability and disaster recovery
5. Use Managed Disks: Better performance and reliability

Troubleshooting

See VM Deployment Troubleshooting Guide for common issues and solutions.

Checklist

See VM Deployment Checklist for a comprehensive deployment checklist.

References

• Azure VM Documentation
• Azure VMSS Documentation
• Docker Documentation
• Besu Documentation
• Cloud-init Documentation