7ac74f432b
- Institutional / JVMTM / reserve-provenance / GRU transport + standards JSON - Validation and verify scripts (Blockscout labels, x402, GRU preflight, P1 local path) - Wormhole wiring in AGENTS, MCP_SETUP, MASTER_INDEX, 04-configuration README - Meta docs, integration gaps, live verification log, architecture updates - CI validate-config workflow updates Operator/LAN items, submodule working trees, and public token-aggregation edge routes remain follow-up (see TODOS_CONSOLIDATED P1). Made-with: Cursor
8.0 KiB
8.0 KiB
Proxmox VE — Operational deployment template
Last Updated: 2026-03-25
Status: Active — ties hypervisors, LAN/WAN, cluster peering, Chain 138 Besu tiers, NPMplus ingress, FQDNs, and deployment gates into one place.
Machine-readable: config/proxmox-operational-template.json (sync when you change VMIDs/IPs/FQDNs).
Authoritative detail (do not drift):
- VMID, port, status tables:
docs/04-configuration/ALL_VMIDS_ENDPOINTS.md - Shell/env single source:
config/ip-addresses.conf - Edge, port forwards, four NPMplus picture:
docs/11-references/NETWORK_CONFIGURATION_MASTER.md - Contract deploy order / gates:
docs/03-deployment/DEPLOYMENT_ORDER_OF_OPERATIONS.md
1. Proxmox VE hosts (management)
| Hostname | MGMT IP | Proxmox UI | Cluster | Role (target) |
|---|---|---|---|---|
| ml110 | 192.168.11.10 | https://192.168.11.10:8006 | h (legacy) | Planned WAN aggregator (OPNsense/pfSense); migrate CT/VM off before repurpose |
| r630-01 | 192.168.11.11 | https://192.168.11.11:8006 | h | Primary: Chain 138 RPC/CCIP-adjacent workloads, Sankofa Phoenix stack, much of DBIS |
| r630-02 | 192.168.11.12 | https://192.168.11.12:8006 | h | Firefly, MIM4U, Mifos LXC, extra NPMplus instances, supporting infra |
LAN: 192.168.11.0/24, gateway 192.168.11.1 (UDM Pro), VLAN 11. Extended node IP plan (r630-03 …): config/ip-addresses.conf comments.
2. Cluster peering (Corosync / quorum)
| Item | Value / note |
|---|---|
| Cluster name | h (verify live: pvecm status) |
| Ring | Typically same L2/L3 as MGMT — 192.168.11.0/24 |
| UDP ports | 5405–5412 between all nodes (+ SSH 22, API 8006 TCP) |
| Quorum | Odd node count preferred; during ml110 removal use 2-node awareness (risk window) or add qdevice |
Cluster and UDM: docs/04-configuration/UDM_PRO_PROXMOX_CLUSTER.md. Live inventory: docs/04-configuration/ALL_VMIDS_ENDPOINTS.md, config/proxmox-operational-template.json.
3. Chain 138 Besu — peering model (summary)
| Layer | VMID range (typical) | IPv4 pattern | P2P |
|---|---|---|---|
| Validators | 1000–1004 | 192.168.11.100–104 | 30303 — to sentries, not raw public |
| Sentries | 1500–1506 | .150–.154, .213–.214 | Boundary / fan-out |
| Core RPC (deploy) | 2101 | 192.168.11.211 | 8545/8546 + 30303 |
| Core RPC (Nathan core-2) | 2102 | 192.168.11.212 | NPMplus 10235 / tunnel |
| Public RPC | 2201 | 192.168.11.221 | Frontends / bridge / read-mostly |
| Named RPC | 2303–2308 | .233–.238 | Partner-dedicated |
| ThirdWeb stack | 2400–2403 | .240–.243 | Includes translator/nginx on 2400 |
Canonical roles and adjacency rules: docs/02-architecture/CHAIN138_CANONICAL_NETWORK_ROLES_VALIDATORS_SENTRY_AND_RPC.md.
4. NPMplus and public ingress
| VMID | Internal IP(s) | Public IP (typical) | Purpose |
|---|---|---|---|
| 10233 | 192.168.11.166 / .167 | 76.53.10.36 | Main d-bis.org, explorer, Option B RPC, MIM4U |
| 10234 | 192.168.11.168 | 76.53.10.37 | Secondary HA (confirm running) |
| 10235 | 192.168.11.169 | 76.53.10.38 (alt 76.53.10.42) | rpc-core-2, Alltra, HYBX |
| 10236 | 192.168.11.170 | 76.53.10.40 | Dev / Codespaces tunnel, Gitea, Proxmox admin |
| 10237 | 192.168.11.171 | (tunnel/Mifos) | mifos.d-bis.org → VMID 5800 |
UDM Pro forwards 80 / 443 (and 81 where documented) to the matching internal IP. Detail: docs/04-configuration/NPMPLUS_FOUR_INSTANCES_MASTER.md.
5. FQDN → backend (high level)
Use the full table in ALL_VMIDS_ENDPOINTS (“NPMplus Endpoint Configuration Reference”). Critical correctness checks:
- explorer.d-bis.org → VMID 5000, 192.168.11.140 (not Sankofa IPs).
- sankofa.nexus / phoenix.sankofa.nexus → VMID 7801 / 7800 at .51:3000 / .50:4000.
- rpc-http-prv / rpc-ws-prv → 2101 (.211); rpc-http-pub / rpc-ws-pub → 2201 (.221).
- rpc.public-0138.defi-oracle.io → 2400 192.168.11.240:443 (update NPM if still pointing at decommissioned IPs).
the-order.sankofa.nexus: NPMplus → order HAProxy 10210 @ 192.168.11.39:80 (proxies to Sankofa portal 192.168.11.51:3000). See scripts/deployment/provision-order-haproxy-10210.sh.
5.1 Order stack (live VMIDs, r630-01 unless noted)
| VMID | Hostname | IP | Role (short) |
|---|---|---|---|
| 10030 | order-identity | 192.168.11.40 | Identity |
| 10040 | order-intake | 192.168.11.41 | Intake |
| 10050 | order-finance | 192.168.11.49 | Finance |
| 10060 | order-dataroom | 192.168.11.42 | Dataroom |
| 10070 | order-legal | 192.168.11.87 | Legal — moved off .54 2026-03-25 (IP_ORDER_LEGAL); .54 is only VMID 7804 gov-portals |
| 10080 | order-eresidency | 192.168.11.43 | eResidency |
| 10090 | order-portal-public | 192.168.11.36 | Public portal |
| 10091 | order-portal-internal | 192.168.11.35 | Internal portal |
| 10092 | order-mcp-legal | 192.168.11.94 | MCP legal — moved off .37 2026-03-29 to avoid MIM4U conflict (IP_ORDER_MCP_LEGAL) |
| 10200 | order-prometheus | 192.168.11.46 | Metrics |
| 10201 | order-grafana | 192.168.11.47 | Dashboards |
| 10202 | order-opensearch | 192.168.11.48 | Search |
| 10210 | order-haproxy | 192.168.11.39 | Edge / HAProxy |
Redis: ORDER_REDIS_IP = 192.168.11.38 in ip-addresses.conf — bind to live VMID via pct list / audit script.
6. Deployment requirements (cross-domain)
6.1 Platform (Proxmox / network)
- All cluster nodes quorate; storage sufficient for CT/VM disks (local-lvm / future Ceph per master plan).
- vmbr0 VLAN-aware; each workload IP unique on 192.168.11.0/24 (see ALL_VMIDS conflict section).
- UDM Pro routes and port-forwards match NETWORK_CONFIGURATION_MASTER.
- NPMplus proxy host rows match ALL_VMIDS (no Blockscout IP on Sankofa hostnames).
6.2 Chain 138 (contracts / ops)
- Core RPC 2101 reachable:
http://192.168.11.211:8545for deploy only (not public RPC). smom-dbis-138/.env:PRIVATE_KEY,RPC_URL_138, nonce discipline — DEPLOYMENT_ORDER_OF_OPERATIONS Phase 0.- Optional:
./scripts/deployment/preflight-chain138-deploy.shbefore any broadcast.
6.3 Application / operator
- Repo
.env+smom-dbis-138/.envfor operator scripts (scripts/lib/load-project-env.sh). - Blockscout / verify / NPM backup scripts per OPERATOR_READY_CHECKLIST when doing release ops.
7. Maintaining this template
- Change ALL_VMIDS_ENDPOINTS and/or ip-addresses.conf first (operator truth).
- Update
config/proxmox-operational-template.jsonso automation (future CMDB, checks) stays aligned. - Run
./scripts/validation/validate-config-files.sh(includes JSON shape check for the template). - Live diff (read-only, SSH): from repo root on a host with SSH to Proxmox nodes:
bash scripts/verify/audit-proxmox-operational-template.sh. Compares template VMIDs topct/qmlists on ML110 + R630s (overridePROXMOX_HOSTSif needed).
8. Related runbooks
| Topic | Doc |
|---|---|
| Operational runbooks index | OPERATIONAL_RUNBOOKS.md |
| Phoenix / Sankofa deploy | PHOENIX_DEPLOYMENT_RUNBOOK.md |
| NPMplus health | docs/04-configuration/NPMPLUS_QUICK_REF.md |
| 13-node / HA roadmap | docs/02-architecture/R630_13_NODE_DOD_HA_MASTER_PLAN.md |