proxmox/docs/runbooks/BRIDGE_OPERATIONS_RUNBOOK.md

Bridge Operations Runbook

Last Updated: 2026-01-31
Document Version: 1.0
Status: Active Documentation

---

Purpose: Step-by-step procedures for common bridge operations

---

🔄 Bridge Transfer Procedure

Standard Transfer

1. Check System Status

   bash scripts/health-check.sh
   

2. Verify Balances

   bash scripts/check-bridge-status.sh
   

3. Estimate Fees

   bash scripts/fee-management.sh estimate
   

4. Execute Transfer

   bash scripts/bridge-with-dynamic-gas.sh weth9 1.0 bsc
   

5. Monitor Transfer

   bash scripts/monitor-bridge-transfers.sh <transaction_hash>
   

---

🚨 Emergency Procedures

Pause Bridge Operations

1. Access Bridge Contract

   cast send <BRIDGE_ADDRESS> "pause()" --rpc-url $RPC_URL --private-key $PRIVATE_KEY
   

2. Verify Pause

   cast call <BRIDGE_ADDRESS> "paused()" --rpc-url $RPC_URL
   

3. Resume Operations

   cast send <BRIDGE_ADDRESS> "unpause()" --rpc-url $RPC_URL --private-key $PRIVATE_KEY
   

---

🔧 Troubleshooting

Transaction Stuck

1. Check Transaction Status

   cast tx <transaction_hash> --rpc-url $RPC_URL
   

2. Retry with Higher Gas

   bash scripts/retry-failed-transactions.sh
   

3. Check Nonce

   cast nonce <address> --rpc-url $RPC_URL
   

Low Balance

1. Check Balances

   bash scripts/check-bridge-status.sh
   

2. Add Funds

   • Transfer ETH to deployer address
   • Transfer LINK tokens if needed

RPC Issues

1. Test RPC Connectivity

   cast block-number --rpc-url $RPC_URL
   

2. Check RPC Health

   bash scripts/health-check.sh
   

---

📊 Monitoring

Daily Checks

# Run automated monitoring
bash scripts/automated-monitoring.sh

# Generate daily report
bash scripts/generate-bridge-report.sh daily

Weekly Review

# Generate weekly report
bash scripts/generate-bridge-report.sh weekly

# Run test suite
bash scripts/test-suite.sh all

---

🌐 Chain138 / ALL Mainnet → Tezos USDtz

Bridge Provider Status Matrix

Provider	Chains	Status
CCIP	138 ↔ 1	active
AlltraAdapter	651940 ↔ 1	active
Wrap Protocol	1 → Tezos	active
Hop Protocol	ETH ↔ L2	active (explorer aggregator)

Stuck Tx / Failure Recovery

1. Check audit trail: GET /v1/audit/:executionId for hop status.
2. Identify failed hop: Compare chain_id and step_type in audit vs route_plan.
3. CCIP 138→1: Check CCIP explorer for message status; resubmit if expired.
4. ETH→Tezos: Verify Wrap Protocol dashboard; check Tezos RPC health.
5. Quote drift: Re-run POST /v1/routes/chain138-to-usdtz with async_quotes: true for fresh quotes.

Monitoring

• Quote-to-execution drift: compare min_amount_out in route vs final delivery.
• Bridge completion time: track submitted → confirmed per hop.

---

Last Updated: 2026-02-01