3f76bc9507
Some checks failed
Deploy to Phoenix / deploy (push) Has been cancelled
- MASTER_INDEX: Last Updated 2026-03-06; status 59/59 contracts; add NEXT_STEPS_LIST, CONTRACT_NEXT_STEPS_LIST - docs/README, NEXT_STEPS_INDEX, 06-besu/MASTER_INDEX: Last Updated 2026-03-06 - Contract check script: 59 addresses (PMM, vault/reserve, CompliantFiatTokens); canonical CCIP/router - New docs: EXECUTION_CHECKLIST, NEXT_STEPS_LIST, DOTENV_AUDIT, ADDITIONAL_PATHS, deployer gas runbook, WEMIX_ACQUISITION_TABLED, etc. - Config: deployer-gas-routes, cro-wemix-swap-routes, routing-registry, token-mapping - Scripts: check-contracts-on-chain-138, check-pmm-pool-balances-chain138, deployer-gas-auto-route, acquire-cro-and-wemix-gas - Operator rule: operator-lan-access-check.mdc Made-with: Cursor
59 lines
1.8 KiB
Markdown
59 lines
1.8 KiB
Markdown
# DBIS PostgreSQL HA Pair Runbook
|
|
|
|
## Topology
|
|
- Primary writer: `192.168.11.105` (CT `10100`)
|
|
- Replica reader: `192.168.11.106` (CT `10101`)
|
|
- Proxmox host: `192.168.11.11`
|
|
- Database: `dbis_core`
|
|
|
|
## Why this matters
|
|
- Writes must go to exactly one primary.
|
|
- Reads/indexing should prefer replica.
|
|
- If both nodes are writable, split-brain risk is high.
|
|
|
|
## 1) Convert to true HA (streaming replica)
|
|
Use once to enforce primary/replica:
|
|
|
|
```bash
|
|
cd /home/intlc/projects/proxmox
|
|
REPL_PASSWORD='<replication-password>' \
|
|
scripts/database/convert-dbis-postgres-to-ha.sh
|
|
```
|
|
|
|
## 2) Verify HA health
|
|
|
|
```bash
|
|
cd /home/intlc/projects/proxmox
|
|
DB_PASSWORD='<db-password>' \
|
|
scripts/database/check-dbis-postgres-ha.sh
|
|
```
|
|
|
|
Expected:
|
|
- `192.168.11.105`: `pg_is_in_recovery = f`
|
|
- `192.168.11.106`: `pg_is_in_recovery = t`
|
|
- Replica write test fails with read-only transaction error.
|
|
|
|
## 3) Correct connection policy
|
|
- Application writes: `192.168.11.105:5432`
|
|
- Read-only APIs/indexers: `192.168.11.106:5432`
|
|
- Admin checks:
|
|
- Primary replication status:
|
|
- `sudo -u postgres psql -Atc "select client_addr,state,sync_state from pg_stat_replication;"`
|
|
- Replica recovery status:
|
|
- `sudo -u postgres psql -Atc "select pg_is_in_recovery();"`
|
|
|
|
## 4) Explorer label/indexing publishing
|
|
- Publish labels/tags to primary only (`192.168.11.105`).
|
|
- Replica receives updates via WAL streaming automatically.
|
|
- Do not run dual-write mode when streaming replication is healthy.
|
|
|
|
## 5) Optional hardening
|
|
- Add VIP (keepalived/HAProxy) for stable writer endpoint.
|
|
- Enable synchronous replication if zero-data-loss failover is required.
|
|
- Add scheduled health checks + failover playbook.
|
|
|
|
## 6) Notes on rsync
|
|
- Do not use rsync for live PostgreSQL data directories.
|
|
- Use streaming replication for live sync.
|
|
- Use rsync only for ancillary files/runbooks/backups, not active PG data.
|