# VM Deployment Guide ## Overview This guide describes how to deploy the Besu network on Azure Virtual Machines (VMs) or Virtual Machine Scale Sets (VMSS) with Docker Engine, as an alternative to AKS deployment. ## Architecture ### Deployment Options 1. **Individual VMs**: Separate VMs for each node (validators, sentries, RPC) 2. **VM Scale Sets**: Auto-scaling VM groups for each node type 3. **Multi-Region**: Deploy nodes across multiple Azure regions for high availability ### Node Types - **Validators**: Private subnets, no public IPs, IBFT2 consensus - **Sentries**: Public-facing P2P nodes, peer to validators and sentries - **RPC Nodes**: Public HTTPS JSON-RPC, no P2P, read-only ## Prerequisites - Azure CLI installed and configured - Terraform >= 1.0 - SSH key pair for VM access - Azure subscription with appropriate permissions - Resource group created ## Quick Start ### 1. Generate SSH Key (if not exists) ```bash ssh-keygen -t rsa -b 4096 -C "besu-vm-deployment" ``` ### 2. Set Environment Variables ```bash export SSH_PUBLIC_KEY=$(cat ~/.ssh/id_rsa.pub) export AZURE_SUBSCRIPTION_ID="your-subscription-id" export RESOURCE_GROUP_NAME="defi-oracle-mainnet-rg" export CLUSTER_NAME="defi-oracle-aks" ``` ### 3. Configure Terraform Variables ```bash # Copy example variables file cp terraform/terraform.tfvars.vm.example terraform/terraform.tfvars.vm # Edit terraform.tfvars.vm with your values # Set vm_deployment_enabled = true # Set ssh_public_key = "$(cat ~/.ssh/id_rsa.pub)" ``` ### 4. Deploy Infrastructure ```bash # Initialize Terraform cd terraform terraform init # Plan deployment (VM deployment) terraform plan -var-file=terraform.tfvars.vm -var="vm_deployment_enabled=true" # Apply deployment terraform apply -var-file=terraform.tfvars.vm -var="vm_deployment_enabled=true" ``` ### 5. Alternative: Use Deployment Script ```bash # Use the deployment script ./scripts/vm-deployment/deploy-vm-network.sh ``` ### 4. Setup VMs After VMs are created, they will be automatically configured via cloud-init. To manually setup: ```bash # SSH into VM ssh besuadmin@ # Run setup script sudo /opt/besu/setup.sh ``` ### 5. Verify Deployment ```bash # Check VM status az vm list --resource-group $RESOURCE_GROUP_NAME --show-details # Check Besu container status ssh besuadmin@ "docker ps" # Check Besu logs ssh besuadmin@ "docker logs besu-validator-0" ``` ## Manual VM Setup ### 1. Create VM ```bash # Create resource group az group create --name $RESOURCE_GROUP_NAME --location eastus # Create VM az vm create \ --resource-group $RESOURCE_GROUP_NAME \ --name besu-validator-0 \ --image Ubuntu2204 \ --size Standard_D4s_v3 \ --admin-username besuadmin \ --ssh-key-values ~/.ssh/id_rsa.pub \ --vnet-name besu-vnet \ --subnet validators-subnet \ --nsg besu-validator-nsg ``` ### 2. Setup VM ```bash # Copy setup script to VM scp scripts/vm-deployment/setup-vm.sh besuadmin@:~ # SSH into VM ssh besuadmin@ # Run setup script sudo bash setup-vm.sh validator 0 ``` ### 3. Configure Besu ```bash # Copy configuration files scp config/genesis.json besuadmin@:~/genesis.json scp config/validators/besu-config.toml besuadmin@:~/besu-config.toml # Copy validator keys scp keys/validator-0/* besuadmin@:~/keys/ ``` ### 4. Start Besu ```bash # SSH into VM ssh besuadmin@ # Start Besu container cd /opt/besu docker compose up -d # Check status docker ps docker logs besu-validator-0 ``` ## VM Scale Sets Deployment ### Deploy VM Scale Set ```bash # Update terraform.tfvars.vm use_vmss = true # Apply Terraform terraform apply -var-file=terraform.tfvars.vm ``` ### Scale VM Scale Set ```bash # Scale validators az vmss scale \ --resource-group $RESOURCE_GROUP_NAME \ --name besu-validator-vmss \ --new-capacity 4 # Scale RPC nodes az vmss scale \ --resource-group $RESOURCE_GROUP_NAME \ --name besu-rpc-vmss \ --new-capacity 5 ``` ## Multi-Region Deployment ### Deploy to Multiple Regions ```bash # Update terraform.tfvars.vm vm_regions = ["eastus", "westus", "westeurope", "southeastasia"] # Apply Terraform terraform apply -var-file=terraform.tfvars.vm ``` ### Configure Cross-Region Peering ```bash # Create VNet peering between regions az network vnet peering create \ --resource-group $RESOURCE_GROUP_NAME \ --name eastus-to-westus \ --vnet-name besu-vnet-eastus \ --remote-vnet besu-vnet-westus \ --allow-vnet-access ``` ## Monitoring ### View VM Metrics ```bash # View VM metrics az monitor metrics list \ --resource /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Compute/virtualMachines/besu-validator-0 \ --metric "Percentage CPU" \ --start-time 2024-01-01T00:00:00Z ``` ### View Besu Logs ```bash # SSH into VM ssh besuadmin@ # View logs docker logs -f besu-validator-0 # View logs from file tail -f /opt/besu/logs/besu.log ``` ### View Metrics ```bash # Check metrics endpoint curl http://:9545/metrics ``` ## Backup and Recovery ### Backup Chaindata ```bash # SSH into VM ssh besuadmin@ # Stop Besu docker compose down # Backup data tar -czf besu-data-backup-$(date +%Y%m%d).tar.gz /opt/besu/data # Upload to Azure Storage az storage blob upload \ --account-name $STORAGE_ACCOUNT_NAME \ --container-name backups \ --name besu-data-backup-$(date +%Y%m%d).tar.gz \ --file besu-data-backup-$(date +%Y%m%d).tar.gz # Restart Besu docker compose up -d ``` ### Restore Chaindata ```bash # Download backup az storage blob download \ --account-name $STORAGE_ACCOUNT_NAME \ --container-name backups \ --name besu-data-backup-20240101.tar.gz \ --file besu-data-backup-20240101.tar.gz # Stop Besu docker compose down # Restore data tar -xzf besu-data-backup-20240101.tar.gz -C / # Restart Besu docker compose up -d ``` ## Troubleshooting ### VM Not Accessible ```bash # Check VM status az vm show --resource-group $RESOURCE_GROUP_NAME --name besu-validator-0 --show-details # Check NSG rules az network nsg rule list --resource-group $RESOURCE_GROUP_NAME --nsg-name besu-validator-nsg # Restart VM az vm restart --resource-group $RESOURCE_GROUP_NAME --name besu-validator-0 ``` ### Besu Container Not Starting ```bash # SSH into VM ssh besuadmin@ # Check container logs docker logs besu-validator-0 # Check systemd service systemctl status besu.service # Check Docker docker ps -a systemctl status docker ``` ### Network Issues ```bash # Check network connectivity ping # Check P2P port telnet 30303 # Check RPC port curl http://:8545 ``` ## Cost Optimization ### Use Spot VMs ```bash # Create VM with spot pricing az vm create \ --resource-group $RESOURCE_GROUP_NAME \ --name besu-validator-0 \ --priority Spot \ --max-price -1 \ --eviction-policy Deallocate ``` ### Use Reserved Instances ```bash # Purchase reserved instance az vm reservation create \ --resource-group $RESOURCE_GROUP_NAME \ --reserved-resource-type VirtualMachines \ --billing-scope /subscriptions/$SUBSCRIPTION_ID \ --term P1Y \ --quantity 1 \ --sku Standard_D4s_v3 ``` ## Security ### Network Security - Use Network Security Groups (NSGs) to restrict access - Use private subnets for validators - Use public IPs only for sentries and RPC nodes - Implement firewall rules ### Key Management - Use Azure Key Vault for validator keys - Use Managed Identity for Key Vault access - Rotate keys regularly - Backup keys securely ### Access Control - Use SSH keys instead of passwords - Disable root login - Use Azure AD for VM access - Implement just-in-time access ## Comparison: AKS vs VM Deployment ### AKS Deployment **Pros**: - Kubernetes orchestration - Auto-scaling - Service discovery - Rolling updates - Resource management **Cons**: - More complex setup - Higher cost (control plane) - Requires Kubernetes expertise ### VM Deployment **Pros**: - Simpler setup - Lower cost (no control plane) - Full control over VMs - Easy to understand - Direct Docker access **Cons**: - Manual scaling - Manual updates - No service discovery - More manual configuration ## Recommendations 1. **Use AKS for production**: Better orchestration and management 2. **Use VMs for development**: Simpler and cheaper 3. **Use VMSS for auto-scaling**: Better than individual VMs 4. **Multi-region deployment**: High availability and disaster recovery 5. **Use Managed Disks**: Better performance and reliability ## Troubleshooting See [VM Deployment Troubleshooting Guide](VM_DEPLOYMENT_TROUBLESHOOTING.md) for common issues and solutions. ## Checklist See [VM Deployment Checklist](VM_DEPLOYMENT_CHECKLIST.md) for a comprehensive deployment checklist. ## References - [Azure VM Documentation](https://docs.microsoft.com/azure/virtual-machines/) - [Azure VMSS Documentation](https://docs.microsoft.com/azure/virtual-machine-scale-sets/) - [Docker Documentation](https://docs.docker.com/) - [Besu Documentation](https://besu.hyperledger.org/) - [Cloud-init Documentation](https://cloudinit.readthedocs.io/)