proxmox/docs/archive/historical/FLUSH_ALL_STUCK_TRANSACTIONS.md

Flush All Stuck Transactions - Complete Guide

Date: $(date)
Status: ✅ SCRIPT READY

---

🎯 Objective

Flush all stuck transactions from all Besu nodes using multiple methods.

---

📋 Available Methods

The script flush-all-stuck-transactions.sh tries three methods in order:

Method 1: TXPOOL API Clear (Non-Invasive)

• Uses txpool_besuClear or txpool_clear RPC methods
• Requires: TXPOOL API enabled on RPC node (VMID 2500 has this enabled)
• Status: ⚠️ Methods not available in this Besu version
• Impact: None - just clears in-memory transaction pool

Method 2: Restart All Services (Recommended)

• Restarts all Besu services (validators, sentries, RPC nodes)
• Requires: Proxmox host access (pct command)
• Impact: Services restart, clears in-memory transaction pools
• Duration: ~30 seconds

Method 3: Clear Database Files (Most Aggressive)

• Stops services, clears transaction pool database files, restarts
• Requires: Proxmox host access and confirmation
• Impact: All pending transactions lost, requires re-sync
• Duration: ~2-3 minutes

---

🚀 Quick Start

Step 1: Copy Scripts to Proxmox Host

From your local machine, copy the scripts to the Proxmox host:

cd /home/intlc/projects/proxmox
./scripts/copy-flush-scripts-to-proxmox.sh

This will copy:

• flush-all-mempools-proxmox.sh
• flush-all-stuck-transactions.sh
• clear-transaction-pool-database.sh

Step 2: Run on Proxmox Host

SSH to the Proxmox host and run:

ssh root@192.168.11.10  # or your Proxmox host IP
cd /home/intlc/projects/proxmox
./scripts/flush-all-stuck-transactions.sh

Or run directly via SSH:

ssh root@192.168.11.10 "cd /home/intlc/projects/proxmox && ./scripts/flush-all-stuck-transactions.sh"

The script will:

1. Try Method 1 (TXPOOL API) - usually fails as methods aren't available
2. Automatically try Method 2 (restart services)
3. Offer Method 3 (database clear) if needed

---

📋 Nodes Affected

Validators (5 nodes)

• VMID 1000-1004
• Service: besu-validator.service

Sentries (4 nodes)

• VMID 1500-1503
• Service: besu-sentry.service

RPC Nodes (3 nodes)

• VMID 2500-2502
• Service: besu-rpc.service

Total: 12 nodes

---

🔧 Manual Methods

Option 1: Simple Restart (Quick)

cd /home/intlc/projects/proxmox
./scripts/flush-all-mempools-proxmox.sh

This just restarts all services - simplest and fastest method.

Option 2: Database Clear (Aggressive)

cd /home/intlc/projects/proxmox
./scripts/clear-transaction-pool-database.sh

This stops services, clears transaction pool database files, then restarts.

---

✅ Verification

After flushing, verify transactions are cleared:

# Check transaction pool via RPC (if TXPOOL enabled)
curl -X POST http://192.168.11.250:8545 \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"txpool_besuTransactions","params":[],"id":1}' | jq '.result | length'

# Should return 0 or empty result if pool is clear

---

⚠️ Important Notes

1. Transaction Retention: If transactions persist after restart, they may be stored in blockchain state (not just mempool)
2. Network Sync: After restart, nodes will re-sync with peers - transactions may reappear
3. Nonce State: If nonce is stuck, it's part of blockchain state, not just transaction pool
4. Best Practice: Use Method 2 (restart) first - it's the safest and most effective

---

🔍 Troubleshooting

Transactions Persist After Restart

If transactions persist after restarting services:

1. Check if transaction was already mined

   cast tx <tx_hash> --rpc-url http://192.168.11.250:8545
   

2. Check blockchain state (nonce)

   cast nonce <address> --rpc-url http://192.168.11.250:8545
   

3. Transaction may be in blockchain state, not mempool

   • Nonce 23 means 23 transactions have been sent
   • This is blockchain state, not just transaction pool
   • Cannot be cleared by flushing mempool

Solution for Blockchain State Issues

If nonce is stuck due to blockchain state:

• Use a different account
• Wait for transaction to be processed or expire
• Accept the nonce and continue with next nonce

---

📚 Related Documentation

• Flush Mempools Instructions
• Transaction Pool Clear Results
• Blockchain Database Clear Results
• Ethereum Mainnet Blocking Issue

---

Last Updated: $(date)