proxmox/docs/archive/historical/ETHEREUM_MAINNET_BLOCKING_ISSUE.md

Ethereum Mainnet Configuration - Persistent Blocking Issue

Date: $(date)
Status: ⚠️ PERSISTENT MEMPOOL BLOCK - DEEPER INVESTIGATION NEEDED

---

🔍 Issue Summary

Ethereum Mainnet configuration is persistently blocked by a stuck transaction in the mempool, even after:

• ✅ Restarting all validators (1000-1004)
• ✅ Restarting all RPC nodes (2500-2502)
• ✅ Restarting all sentries
• ✅ Using extremely high gas prices (up to 100,000 gwei)
• ✅ Trying nonce skipping (rejected by Besu)

Error: Replacement transaction underpriced (error code -32000)

---

📊 Current Status

Network Configuration: 6/7 Networks ✅

Network	Status
BSC	✅ CONFIGURED
Polygon	✅ CONFIGURED
Avalanche	✅ CONFIGURED
Base	✅ CONFIGURED
Arbitrum	✅ CONFIGURED
Optimism	✅ CONFIGURED
Ethereum Mainnet	❌ BLOCKED

Test Results

• Total Tests: 33
• Passed: 24
• Failed: 1 (Ethereum Mainnet)
• Success Rate: 72.72%

---

🔍 Root Cause Analysis

Possible Causes

1. Transaction Persisted in Database

   • Besu may have persisted the transaction in its database
   • Restarting services doesn't clear persisted transactions
   • Requires database-level clearing

2. Transaction in Validator Mempool

   • Transaction may be in a validator's mempool, not RPC
   • Validators don't expose RPC, so can't query directly
   • May need to check validator logs

3. Transaction Already Mined but Failed

   • Transaction with nonce 23 may have been mined but failed
   • Nonce stuck because transaction failed
   • Need to check transaction receipt

4. Besu Configuration Issue

   • Minimum gas price may be preventing replacement
   • Transaction pool settings may be blocking
   • Account permissioning may be blocking

---

🔧 Investigation Steps Needed

Step 1: Check Transaction History

# Check if transaction with nonce 23 was mined
cast tx --rpc-url http://192.168.11.250:8545 <tx_hash>
cast receipt --rpc-url http://192.168.11.250:8545 <tx_hash>

Step 2: Check Validator Logs

On ML110, check validator logs for transaction rejection:

# On ML110
for vmid in 1000 1001 1002 1003 1004; do
    echo "=== VMID $vmid ==="
    pct exec $vmid -- journalctl -u besu-validator -n 100 | grep -iE "transaction|reject|nonce|23"
done

Step 3: Check Besu Database

The transaction may be persisted. Check if we can clear it:

# On ML110, check Besu data directory
pct exec 2500 -- ls -la /data/besu/

Step 4: Try Different Account

If mempool persists, try using a different account:

# Generate new account
cast wallet new

# Fund with ETH
# Then use for configuration

---

💡 Alternative Solutions

Solution 1: Clear Besu Database (Nuclear Option)

WARNING: This will require re-syncing the node

# On ML110
pct exec 2500 -- systemctl stop besu-rpc
pct exec 2500 -- rm -rf /data/besu/caches/
pct exec 2500 -- systemctl start besu-rpc

Solution 2: Use Different Deployer Account

Create a new account and use it for configuration:

# Create new account
NEW_ACCOUNT=$(cast wallet new)
echo "New account: $NEW_ACCOUNT"

# Fund with ETH
# Use for configuration

Solution 3: Check if Transaction Actually Succeeded

The transaction may have succeeded but the nonce didn't advance:

# Check if Ethereum Mainnet is actually configured
cast call 0x89dd12025bfCD38A168455A44B400e913ED33BE2 \
  "destinations(uint64)" \
  5009297550715157269 \
  --rpc-url http://192.168.11.250:8545

---

📋 Next Steps

1. Investigate transaction history - Check if nonce 23 transaction was mined
2. Check validator logs - Look for transaction rejection reasons
3. Try different account - Use a new account to bypass stuck transaction
4. Check Besu configuration - Verify minimum gas price and pool settings
5. Consider database clearing - If all else fails, clear Besu caches

---

🎯 Priority

HIGH - Ethereum Mainnet is the only network not configured. All other networks (6/7) are working correctly.

---

Last Updated: $(date)
Status: ⚠️ REQUIRES DEEPER INVESTIGATION