# RPC-01 (VMID 2500) Troubleshooting Guide **Last Updated:** 2026-01-31 **Document Version:** 1.0 **Status:** Active Documentation --- **Container**: besu-rpc-1 **VMID**: 2500 **IP Address**: 192.168.11.250 **Expected Ports**: 8545 (HTTP), 8546 (WS), 30303 (P2P), 9545 (Metrics) --- ## ๐Ÿ” Quick Diagnostic Run the automated troubleshooting script: ```bash cd /home/intlc/projects/proxmox ./scripts/troubleshoot-rpc-2500.sh ``` --- ## ๐Ÿ“‹ Common Issues & Solutions ### Issue 1: Container Not Running **Symptoms**: - `pct status 2500` shows "stopped" - Cannot connect to container **Solution**: ```bash # Start container pct start 2500 # Check why it stopped pct config 2500 pct logs 2500 ``` --- ### Issue 2: Service Not Active **Symptoms**: - Container running but service inactive - `systemctl status besu-rpc.service` shows failed/stopped **Diagnosis**: ```bash # Check service status pct exec 2500 -- systemctl status besu-rpc.service # Check recent logs pct exec 2500 -- journalctl -u besu-rpc.service -n 50 --no-pager ``` **Common Causes**: #### A. Configuration File Missing **Error**: `Unable to read TOML configuration, file not found` **Solution**: ```bash # Check if config exists pct exec 2500 -- ls -la /etc/besu/config-rpc.toml # If missing, copy from template pct push 2500 /path/to/config-rpc.toml /etc/besu/config-rpc.toml ``` #### B. Deprecated Configuration Options **Error**: `Unknown options in TOML configuration file` **Solution**: Remove deprecated options from config: - `log-destination` - `max-remote-initiated-connections` - `trie-logs-enabled` - `accounts-enabled` - `database-path` - `rpc-http-host-allowlist` **Fix**: ```bash # Edit config file pct exec 2500 -- nano /etc/besu/config-rpc.toml # Or use sed to remove deprecated options pct exec 2500 -- sed -i '/^log-destination/d' /etc/besu/config-rpc.toml pct exec 2500 -- sed -i '/^max-remote-initiated-connections/d' /etc/besu/config-rpc.toml pct exec 2500 -- sed -i '/^trie-logs-enabled/d' /etc/besu/config-rpc.toml pct exec 2500 -- sed -i '/^accounts-enabled/d' /etc/besu/config-rpc.toml pct exec 2500 -- sed -i '/^database-path/d' /etc/besu/config-rpc.toml pct exec 2500 -- sed -i '/^rpc-http-host-allowlist/d' /etc/besu/config-rpc.toml # Restart service pct exec 2500 -- systemctl restart besu-rpc.service ``` #### C. RPC Not Enabled **Error**: Service starts but RPC endpoint not accessible **Solution**: ```bash # Check if RPC is enabled pct exec 2500 -- grep "rpc-http-enabled" /etc/besu/config-rpc.toml # Enable if disabled pct exec 2500 -- sed -i 's/rpc-http-enabled=false/rpc-http-enabled=true/' /etc/besu/config-rpc.toml # Restart service pct exec 2500 -- systemctl restart besu-rpc.service ``` --- ### Issue 3: RPC Endpoint Not Responding **Symptoms**: - Service is active - Ports not listening - Cannot connect to RPC **Diagnosis**: ```bash # Check if ports are listening pct exec 2500 -- ss -tlnp | grep -E "8545|8546" # Test RPC endpoint pct exec 2500 -- curl -X POST http://localhost:8545 \ -H "Content-Type: application/json" \ -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' ``` **Solutions**: #### A. Check RPC Configuration ```bash # Verify RPC is enabled and configured correctly pct exec 2500 -- grep -E "rpc-http|rpc-ws" /etc/besu/config-rpc.toml ``` Expected: ```toml rpc-http-enabled=true rpc-http-host="0.0.0.0" rpc-http-port=8545 rpc-ws-enabled=true rpc-ws-host="0.0.0.0" rpc-ws-port=8546 ``` #### B. Check Firewall ```bash # Check if firewall is blocking pct exec 2500 -- iptables -L -n | grep -E "8545|8546" # If needed, allow ports pct exec 2500 -- iptables -A INPUT -p tcp --dport 8545 -j ACCEPT pct exec 2500 -- iptables -A INPUT -p tcp --dport 8546 -j ACCEPT ``` #### C. Check Host Allowlist ```bash # Check allowlist configuration pct exec 2500 -- grep "rpc-http-host-allowlist" /etc/besu/config-rpc.toml # If too restrictive, update to allow all or specific IPs pct exec 2500 -- sed -i 's/rpc-http-host-allowlist=.*/rpc-http-host-allowlist=["*"]/' /etc/besu/config-rpc.toml ``` --- ### Issue 4: Network Configuration **Symptoms**: - Wrong IP address - Cannot reach container from network **Diagnosis**: ```bash # Check IP address pct exec 2500 -- ip addr show eth0 # Check Proxmox config pct config 2500 | grep net0 ``` **Solution**: ```bash # Update IP in Proxmox config (if needed) pct set 2500 -net0 name=eth0,bridge=vmbr0,ip=192.168.11.250/24,gw=192.168.11.1 # Restart container pct stop 2500 pct start 2500 ``` --- ### Issue 5: Missing Required Files **Symptoms**: - Service fails to start - Errors about missing genesis or static nodes **Diagnosis**: ```bash # Check required files pct exec 2500 -- ls -la /genesis/genesis.json pct exec 2500 -- ls -la /genesis/static-nodes.json pct exec 2500 -- ls -la /permissions/permissions-nodes.toml ``` **Solution**: ```bash # Copy files from source project # (Adjust paths as needed) pct push 2500 /path/to/genesis.json /genesis/genesis.json pct push 2500 /path/to/static-nodes.json /genesis/static-nodes.json pct push 2500 /path/to/permissions-nodes.toml /permissions/permissions-nodes.toml # Set correct ownership pct exec 2500 -- chown -R besu:besu /genesis /permissions # Restart service pct exec 2500 -- systemctl restart besu-rpc.service ``` --- ### Issue 6: Database/Storage Issues **Symptoms**: - Service starts but crashes - Errors about database corruption - Disk space issues **Diagnosis**: ```bash # Check disk space pct exec 2500 -- df -h # Check database directory pct exec 2500 -- ls -la /data/besu/database/ # Check for corruption errors in logs pct exec 2500 -- journalctl -u besu-rpc.service | grep -i "database\|corrupt" ``` **Solution**: ```bash # If database is corrupted, may need to resync # (WARNING: This will delete local blockchain data) pct exec 2500 -- systemctl stop besu-rpc.service pct exec 2500 -- rm -rf /data/besu/database/* pct exec 2500 -- systemctl start besu-rpc.service ``` --- ## ๐Ÿ”ง Manual Diagnostic Commands ### Check Service Status ```bash pct exec 2500 -- systemctl status besu-rpc.service ``` ### View Service Logs ```bash # Real-time logs pct exec 2500 -- journalctl -u besu-rpc.service -f # Last 100 lines pct exec 2500 -- journalctl -u besu-rpc.service -n 100 --no-pager # Errors only pct exec 2500 -- journalctl -u besu-rpc.service | grep -iE "error|fail|exception" ``` ### Check Configuration ```bash # View config file pct exec 2500 -- cat /etc/besu/config-rpc.toml # Validate config syntax pct exec 2500 -- besu --config-file=/etc/besu/config-rpc.toml --help 2>&1 | head -20 ``` ### Test RPC Endpoint ```bash # From container pct exec 2500 -- curl -X POST http://localhost:8545 \ -H "Content-Type: application/json" \ -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' # From host (if accessible) curl -X POST http://192.168.11.250:8545 \ -H "Content-Type: application/json" \ -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' ``` ### Check Process ```bash # Check if Besu process is running pct exec 2500 -- ps aux | grep besu # Check process details pct exec 2500 -- ps aux | grep besu | head -1 ``` ### Check Network Connectivity ```bash # Check IP pct exec 2500 -- ip addr show # Test connectivity to other nodes pct exec 2500 -- ping -c 3 192.168.11.100 # Validator pct exec 2500 -- ping -c 3 192.168.11.150 # Sentry ``` --- ## ๐Ÿ”„ Restart Procedures ### Soft Restart (Service Only) ```bash pct exec 2500 -- systemctl restart besu-rpc.service ``` ### Hard Restart (Container) ```bash pct stop 2500 sleep 5 pct start 2500 ``` ### Full Restart (With Config Reload) ```bash # Stop service pct exec 2500 -- systemctl stop besu-rpc.service # Verify config pct exec 2500 -- cat /etc/besu/config-rpc.toml # Start service pct exec 2500 -- systemctl start besu-rpc.service # Check status pct exec 2500 -- systemctl status besu-rpc.service ``` --- ## ๐Ÿ“Š Expected Configuration ### Configuration File Location - **Path**: `/etc/besu/config-rpc.toml` - **Type**: Core RPC node configuration ### Key Settings ```toml # Network network-id=138 p2p-host="0.0.0.0" p2p-port=30303 # RPC HTTP rpc-http-enabled=true rpc-http-host="0.0.0.0" rpc-http-port=8545 rpc-http-api=["ETH","NET","WEB3"] rpc-http-cors-origins=["*"] # RPC WebSocket rpc-ws-enabled=true rpc-ws-host="0.0.0.0" rpc-ws-port=8546 rpc-ws-api=["ETH","NET","WEB3"] rpc-ws-origins=["*"] # Metrics metrics-enabled=true metrics-port=9545 metrics-host="0.0.0.0" # Data data-path="/data/besu" genesis-file="/genesis/genesis.json" static-nodes-file="/genesis/static-nodes.json" permissions-nodes-config-file="/permissions/permissions-nodes.toml" ``` --- ## โœ… Verification Checklist After troubleshooting, verify: - [ ] Container is running - [ ] Service is active - [ ] IP address is 192.168.11.250 - [ ] Port 8545 is listening - [ ] Port 8546 is listening - [ ] Port 30303 is listening (P2P) - [ ] Port 9545 is listening (Metrics) - [ ] RPC endpoint responds to `eth_blockNumber` - [ ] No errors in recent logs - [ ] Configuration file is valid - [ ] All required files exist --- ## ๐Ÿ“š Related Documentation - [Besu Configuration Guide](/docs/01-getting-started/README.md) - [RPC Node Types Architecture](../05-network/RPC_NODE_TYPES_ARCHITECTURE.md) - [Network Troubleshooting](TROUBLESHOOTING_FAQ.md) - [BESU_CONFIGURATION_GUIDE.md](../04-configuration/BESU_CONFIGURATION_GUIDE.md) ยท [CHAIN138_BESU_CONFIGURATION.md](../06-besu/CHAIN138_BESU_CONFIGURATION.md) --- **Last Updated**: $(date)