d-bis/proxmox
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
dc8abb06e2bbd33e30803ca59d284ead8ea970c6
proxmox/docs/04-configuration/DNS_NPMPLUS_VM_COMPREHENSIVE_ARCHITECTURE.md
defiQUG 4f383490a3 docs(A): sync high-value runbooks for The Order (10210 HAProxy) 
- SANKOFA_CUTOVER_PLAN: live backends table, fix TBDs, historical step labels
- SANKOFA_THE_ORDER_CHECKLIST: replace with done + bypass + pointers
- DNS comprehensive + streamlined tables: the-order row and sankofa zone live
- E2E Cloudflare runbook: the-order backend column

Made-with: Cursor
2026-03-27 15:24:54 -07:00

382 lines
17 KiB
Markdown
Raw Blame History

# DNS → NPMplus → VM Comprehensive Architecture Table
**Last Updated:** 2026-03-27
**Document Version:** 1.1
**Status:** Active Documentation
---
**Date**: 2026-01-20
**Status**: Complete Architecture Reference
**Purpose**: Streamlined DNS, SSL, and traffic routing documentation
**Related Documentation**:
- **HA Setup**: `docs/04-configuration/NPMPLUS_HA_SETUP_GUIDE.md` - High Availability setup guide
- **Backup/Restore**: `docs/04-configuration/NPMPLUS_BACKUP_RESTORE.md` - Backup and restore procedures
- **Verification**: `docs/04-configuration/INGRESS_VERIFICATION_RUNBOOK.md` - Verification procedures
- **Risks**: `docs/04-configuration/INGRESS_RISKS_AND_HARDENING.md` - Risk assessment and hardening
---
## Architecture Overview
```
Internet
↓
Cloudflare DNS (A Records → 76.53.10.36)
↓
UDM Pro Port Forwarding (76.53.10.36:80/443 → 192.168.11.166:80/443)
↓
NPMplus (VMID 10233: 192.168.11.166) - SSL Termination & Routing
↓
Backend VMs (Various IPs) - Services with/without Nginx
```
---
## Complete Service Mapping Table
### Primary Table: Cloudflare DNS → NPMplus → VM Routing
| Domain | Cloudflare DNS | NPMplus Config | Backend VM | Traffic Flow |
|--------|---------------|----------------|------------|--------------|
| | **DNS Type** | **Target IP** | **Proxy** | **SSL Cert ID** | **Proxy Host ID** | **Backend Target** | **VMID** | **IP** | **Hostname** | **Host** | **Service** | **Has Nginx** | **Internal Port** | **NPMplus→VM** |
|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|
| **d-bis.org Zone** |
| `explorer.d-bis.org` | A | 76.53.10.36 | DNS Only | 49 | 8 | `192.168.11.140:4000` (direct) | 5000 | 192.168.11.140 | blockscout-1 | r630-02 | Blockscout Explorer | ✅ Yes | 80, 4000 | HTTP → 4000 |
| `rpc-http-pub.d-bis.org` | A | 76.53.10.36 | DNS Only | 53 | 10 | `192.168.11.221:8545` | 2201 | 192.168.11.221 | besu-rpc-public-1 | ml110 | Besu RPC HTTP | ❌ No | 8545 | HTTP → 8545 |
| `rpc-ws-pub.d-bis.org` | A | 76.53.10.36 | DNS Only | 55 | 11 | `192.168.11.221:8546` | 2201 | 192.168.11.221 | besu-rpc-public-1 | ml110 | Besu RPC WebSocket | ❌ No | 8546 | WS → 8546 |
| `rpc-http-prv.d-bis.org` | A | 76.53.10.36 | DNS Only | 52 | 12 | `192.168.11.211:8545` | 2101 | 192.168.11.211 | besu-rpc-core-1 | ml110 | Besu RPC HTTP (Private) | ❌ No | 8545 | HTTP → 8545 |
| `rpc-ws-prv.d-bis.org` | A | 76.53.10.36 | DNS Only | 54 | 13 | `192.168.11.211:8546` | 2101 | 192.168.11.211 | besu-rpc-core-1 | ml110 | Besu RPC WebSocket (Private) | ❌ No | 8546 | WS → 8546 |
| `dbis-admin.d-bis.org` | A | 76.53.10.36 | DNS Only | 46 | 14 | `192.168.11.130:80` | 10130 | 192.168.11.130 | dbis-frontend | r630-01 | DBIS Admin Frontend | ✅ Yes | 80 | HTTP → 80 |
| `dbis-api.d-bis.org` | A | 76.53.10.36 | DNS Only | 48 | 15 | `192.168.11.155:3000` | 10150 | 192.168.11.155 | dbis-api-primary | r630-01 | DBIS API Primary | ❌ No | 3000 | HTTP → 3000 |
| `dbis-api-2.d-bis.org` | A | 76.53.10.36 | DNS Only | 47 | 16 | `192.168.11.156:3000` | 10151 | 192.168.11.156 | dbis-api-secondary | r630-01 | DBIS API Secondary | ❌ No | 3000 | HTTP → 3000 |
| `secure.d-bis.org` | A | 76.53.10.36 | DNS Only | 58 | 17 | `192.168.11.130:80` | 10130 | 192.168.11.130 | dbis-frontend | r630-01 | DBIS Secure Portal | ✅ Yes | 80 | HTTP → 80 |
| **mim4u.org Zone** |
| `mim4u.org` | A | 76.53.10.36 | DNS Only | 50 | 17 | `192.168.11.37:80` | 7810 | 192.168.11.37 | mim-web-1 | r630-02 | MIM4U Main Site | ✅ Yes | 80 | HTTP → 80 |
| `www.mim4u.org` | A | 76.53.10.36 | DNS Only | 50 | 17 (same) | `192.168.11.37:80` | 7810 | 192.168.11.37 | mim-web-1 | r630-02 | MIM4U Main Site | ✅ Yes | 80 | HTTP → 80 |
| `secure.mim4u.org` | A | 76.53.10.36 | DNS Only | 59 | 19 | `192.168.11.37:80` | 7810 | 192.168.11.37 | mim-web-1 | r630-02 | MIM4U Secure Portal | ✅ Yes | 80 | HTTP → 80 |
| `training.mim4u.org` | A | 76.53.10.36 | DNS Only | 61 | 20 | `192.168.11.37:80` | 7810 | 192.168.11.37 | mim-web-1 | r630-02 | MIM4U Training Portal | ✅ Yes | 80 | HTTP → 80 |
| **sankofa.nexus Zone** (see [ALL_VMIDS_ENDPOINTS.md](ALL_VMIDS_ENDPOINTS.md) — do not point these to explorer/192.168.11.140) |
| `sankofa.nexus` | A | 76.53.10.36 | DNS Only | 57 | 21 | `192.168.11.51:3000` | 7801 | 192.168.11.51 | sankofa-portal-1 | r630-01 | Sankofa Portal | ❌ No | 3000 | HTTP → 3000 |
| `www.sankofa.nexus` | A | 76.53.10.36 | DNS Only | 64 | 22 | `192.168.11.51:3000` | 7801 | 192.168.11.51 | sankofa-portal-1 | r630-01 | Sankofa Portal | ❌ No | 3000 | HTTP → 3000 |
| `phoenix.sankofa.nexus` | A | 76.53.10.36 | DNS Only | 51 | 23 | `192.168.11.50:4000` | 7800 | 192.168.11.50 | sankofa-api-1 | r630-01 | Phoenix API | ❌ No | 4000 | HTTP → 4000 |
| `www.phoenix.sankofa.nexus` | A | 76.53.10.36 | DNS Only | 63 | 24 | `192.168.11.50:4000` | 7800 | 192.168.11.50 | sankofa-api-1 | r630-01 | Phoenix API | ❌ No | 4000 | HTTP → 4000 |
| `the-order.sankofa.nexus` | A | 76.53.10.36 | DNS Only | 60 | 25 | `192.168.11.39:80` | 10210 | 192.168.11.39 | order-haproxy | r630-01 | The Order (HAProxy→portal) | ❌ No | 80 | HTTP → 80 → `.51:3000` |
| **defi-oracle.io Zone** |
| `rpc.public-0138.defi-oracle.io` | A | 76.53.10.36 | DNS Only | 56 | 26 | `192.168.11.240:443` | 2400 | 192.168.11.240 | thirdweb-rpc-1 | ml110 | ThirdWeb RPC | ✅ Yes | 443 | HTTPS → 443 |
**Legend:**
- ✅ = Configured and working
- ❌ = Not applicable
- ⚠️ = Requires attention / Not deployed
- TBD = To Be Determined
**Notes:**
1. **Sankofa/Phoenix domains** must route to VMID 7801 (192.168.11.51:3000) and VMID 7800 (192.168.11.50:4000) respectively — **not** to Blockscout (192.168.11.140). See [ALL_VMIDS_ENDPOINTS.md](ALL_VMIDS_ENDPOINTS.md) and [RPC_ENDPOINTS_MASTER.md](RPC_ENDPOINTS_MASTER.md). If NPMplus currently points these to .140, update proxy hosts to the correct IP:port.
2. **NPMplus** terminates SSL and proxies HTTP to backend VMs (except ThirdWeb RPC which uses HTTPS).
3. **VMID 7810** has nginx running on port 80 serving MIM4U sites.
4. **VMID 5000** has nginx on port 80 that proxies `/api/*` to port 4000 (Blockscout API).
5. **VMID 2400** has nginx on port 443 serving ThirdWeb RPC with SSL.
---
## Detailed VM Service Configuration
### VMs with Nginx Web Server
| VMID | IP | Hostname | Host | Status | Nginx Version | Config Location | Purpose | Public Domains |
|------|----|----------|------|--------|--------------|-----------------|---------|----------------|
| 5000 | 192.168.11.140 | blockscout-1 | r630-02 | ✅ Running | 1.18.0+ | `/etc/nginx/sites-available/blockscout` | Blockscout Explorer | `explorer.d-bis.org` |
| 7810 | 192.168.11.37 | mim-web-1 | r630-02 | ✅ Running | 1.18.0 | `/etc/nginx/sites-available/mim4u` | MIM4U Web App | `mim4u.org`, `www.mim4u.org`, `secure.mim4u.org`, `training.mim4u.org` |
| 10130 | 192.168.11.130 | dbis-frontend | r630-01 | ✅ Running | TBD | TBD | DBIS Admin Frontend | `dbis-admin.d-bis.org`, `secure.d-bis.org` |
| 2400 | 192.168.11.240 | thirdweb-rpc-1 | ml110 | ✅ Running | TBD | TBD | ThirdWeb RPC (HTTPS) | `rpc.public-0138.defi-oracle.io` |
### VMs without Nginx (Direct Service Access)
| VMID | IP | Hostname | Host | Status | Service | Port | Protocol | Public Domains |
|------|----|----------|------|--------|---------|------|----------|----------------|
| 2101 | 192.168.11.211 | besu-rpc-core-1 | ml110 | ✅ Running | Besu RPC | 8545/8546 | HTTP/WS | `rpc-http-prv.d-bis.org`, `rpc-ws-prv.d-bis.org` |
| 2201 | 192.168.11.221 | besu-rpc-public-1 | ml110 | ✅ Running | Besu RPC | 8545/8546 | HTTP/WS | `rpc-http-pub.d-bis.org`, `rpc-ws-pub.d-bis.org` |
| 10150 | 192.168.11.155 | dbis-api-primary | r630-01 | ✅ Running | Node.js API | 3000 | HTTP | `dbis-api.d-bis.org` |
| 10151 | 192.168.11.156 | dbis-api-secondary | r630-01 | ✅ Running | Node.js API | 3000 | HTTP | `dbis-api-2.d-bis.org` |
---
## NPMplus Configuration Details
### NPMplus Container Information
#### Primary NPMplus (10233)
| Property | Value |
|----------|-------|
| **VMID** | 10233 |
| **Host** | r630-01 (192.168.11.11) |
| **Internal IP (eth0)** | 192.168.11.166 |
| **Internal IP (eth1)** | 192.168.11.167 |
| **Management UI** | `https://192.168.11.166:81` |
| **Public IP** | 76.53.10.36 |
| **Public Ports** | 80 (HTTP), 443 (HTTPS) |
| **Status** | ✅ Running |
#### NPMplus Alltra/HYBX (10235)
| Property | Value |
|----------|-------|
| **VMID** | 10235 |
| **Host** | r630-01 (192.168.11.11) |
| **Internal IP** | 192.168.11.169 |
| **Management UI** | `https://192.168.11.169:81` |
| **Port forward** | 76.53.10.38:80/81/443 → 192.168.11.169 |
| **Designated public IP** | 76.53.10.42 |
| **Tunnel target** | https://192.168.11.169:443 (Option B) |
| **Backends** | Alltra + HYBX Sentries, RPC, Cacti, Firefly, Fabric, Indy |
| **Status** | ⏳ To be deployed |
| **Reference** | [NPMPLUS_ALLTRA_HYBX_MASTER_PLAN.md](NPMPLUS_ALLTRA_HYBX_MASTER_PLAN.md) |
### SSL Certificate Management
| Cert ID | Domains | Provider | Expires | Auto-Renewal |
|---------|---------|----------|---------|--------------|
| 46 | `dbis-admin.d-bis.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 47 | `dbis-api-2.d-bis.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 48 | `dbis-api.d-bis.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 49 | `explorer.d-bis.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 50 | `mim4u.org`, `www.mim4u.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 51 | `phoenix.sankofa.nexus` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 52 | `rpc-http-prv.d-bis.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 53 | `rpc-http-pub.d-bis.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 54 | `rpc-ws-prv.d-bis.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 55 | `rpc-ws-pub.d-bis.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 56 | `rpc.public-0138.defi-oracle.io` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 57 | `sankofa.nexus` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 58 | `secure.d-bis.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 59 | `secure.mim4u.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 60 | `the-order.sankofa.nexus` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 61 | `training.mim4u.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 62 | `www.mim4u.org` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 63 | `www.phoenix.sankofa.nexus` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
| 64 | `www.sankofa.nexus` | Let's Encrypt | 2026-04-16 | ✅ Enabled |
**Total Certificates**: 19 active SSL certificates
**Certificate Storage**: `/data/tls/certbot/live/npm-XX/`
---
## Port Forwarding Configuration (UDM Pro)
### Public to Internal Port Mapping
| Public IP:Port | Internal IP:Port | Protocol | Service | Status |
|----------------|------------------|----------|---------|--------|
| 76.53.10.36:443 | 192.168.11.166:443 | TCP | NPMplus HTTPS | ✅ Active |
| 76.53.10.36:80 | 192.168.11.166:80 | TCP | NPMplus HTTP | ✅ Active |
**Router**: UDM Pro
**Forwarding Rule**: Port forwarding configured in UDM Pro firewall rules
---
## Cloudflare DNS Records Summary
### DNS Record Statistics
| Zone | Total Records | A Records | CNAME Records | Proxied | DNS Only |
|------|---------------|-----------|---------------|---------|----------|
| d-bis.org | 9 | 9 | 0 | 0 | 9 |
| mim4u.org | 4 | 4 | 0 | 0 | 4 |
| sankofa.nexus | 5 | 5 | 0 | 0 | 5 |
| defi-oracle.io | 1 | 1 | 0 | 0 | 1 |
| **TOTAL** | **19** | **19** | **0** | **0** | **19** |
**Note**: All DNS records use "DNS Only" mode (gray cloud) to bypass Cloudflare proxy and route directly to NPMplus at 76.53.10.36. SSL termination is handled by NPMplus using Let's Encrypt certificates.
---
## Service Types and Protocols
### Web Services (HTTP/HTTPS)
| Service Type | Domain Example | Port | Protocol | Backend Type |
|--------------|----------------|------|----------|--------------|
| Web Application | `mim4u.org` | 80 | HTTP | Nginx |
| Admin Portal | `dbis-admin.d-bis.org` | 80 | HTTP | Nginx |
| API Service | `dbis-api.d-bis.org` | 3000 | HTTP | Node.js |
| Blockchain Explorer | `explorer.d-bis.org` | 80/4000 | HTTP | Nginx + Blockscout |
### RPC Services (JSON-RPC over HTTP/WebSocket)
| Service Type | Domain Example | Port | Protocol | Backend Type |
|--------------|----------------|------|----------|--------------|
| RPC HTTP | `rpc-http-pub.d-bis.org` | 8545 | HTTP | Besu |
| RPC WebSocket | `rpc-ws-pub.d-bis.org` | 8546 | WebSocket | Besu |
| RPC HTTPS | `rpc.public-0138.defi-oracle.io` | 443 | HTTPS | Nginx + Besu |
---
## Traffic Flow Examples
### Example 1: MIM4U Main Site
```
User Request: https://mim4u.org
↓
DNS Resolution: mim4u.org → 76.53.10.36
↓
UDM Pro: Port Forward 76.53.10.36:443 → 192.168.11.166:443
↓
NPMplus (192.168.11.166:443):
├─ SSL Termination (Cert ID: 50)
├─ Hostname: mim4u.org
├─ Proxy Host ID: 17
└─ Proxy Pass: http://192.168.11.37:80
↓
nginx on VMID 7810 (192.168.11.37:80):
├─ Server Name: mim4u.org
├─ Root: /var/www/html
└─ Response → User (HTTPS)
```
### Example 2: DBIS API
```
User Request: https://dbis-api.d-bis.org
↓
DNS Resolution: dbis-api.d-bis.org → 76.53.10.36
↓
UDM Pro: Port Forward 76.53.10.36:443 → 192.168.11.166:443
↓
NPMplus (192.168.11.166:443):
├─ SSL Termination (Cert ID: 48)
├─ Hostname: dbis-api.d-bis.org
├─ Proxy Host ID: 15
└─ Proxy Pass: http://192.168.11.155:3000
↓
Node.js API on VMID 10150 (192.168.11.155:3000):
├─ Service: DBIS API Primary
└─ Response → User (HTTPS)
```
### Example 3: RPC Endpoint (ThirdWeb)
```
User Request: https://rpc.public-0138.defi-oracle.io
↓
DNS Resolution: rpc.public-0138.defi-oracle.io → 76.53.10.36
↓
UDM Pro: Port Forward 76.53.10.36:443 → 192.168.11.166:443
↓
NPMplus (192.168.11.166:443):
├─ SSL Termination (Cert ID: 56)
├─ Hostname: rpc.public-0138.defi-oracle.io
├─ Proxy Host ID: 26
└─ Proxy Pass: https://192.168.11.240:443
↓
nginx on VMID 2400 (192.168.11.240:443):
├─ SSL Termination (Internal)
├─ Backend: Besu RPC + Translator
└─ Response → User (HTTPS)
```
---
## Issues and Action Items
### ✅ Sankofa/Phoenix routing (authoritative)
**Source of truth:** [ALL_VMIDS_ENDPOINTS.md](ALL_VMIDS_ENDPOINTS.md), [RPC_ENDPOINTS_MASTER.md](RPC_ENDPOINTS_MASTER.md). Sankofa and Phoenix services **are deployed**. Correct NPMplus backend targets:
| Domain | Correct backend | Wrong (do not use) |
|--------|------------------|---------------------|
| `sankofa.nexus`, `www.sankofa.nexus` | 192.168.11.51:3000 (VMID 7801) | 192.168.11.140 |
| `phoenix.sankofa.nexus`, `www.phoenix.sankofa.nexus` | 192.168.11.50:4000 (VMID 7800) | 192.168.11.140 |
| `the-order.sankofa.nexus`, `www.the-order.sankofa.nexus` | 192.168.11.39:80 (10210 HAProxy → .51:3000); www → 301 apex | 192.168.11.140 |
**Action:** If any Sankofa/Phoenix proxy host in NPMplus points to 192.168.11.140 (Blockscout), update it to the correct IP:port above. Only `explorer.d-bis.org` should point to 192.168.11.140.
### 📋 Recommended Improvements
1. **Documentation**
- ✅ This comprehensive table created
- ⚠️ Add nginx config file paths for all VMs with nginx
- ⚠️ Document custom nginx configurations
2. **Monitoring**
- Set up certificate expiration alerts
- Monitor backend VM health
- Track DNS resolution status
3. **Security**
- All SSL certificates auto-renewing ✅
- HSTS enabled on all domains ✅
- Security headers configured ✅
---
## Quick Reference Commands
### Test DNS Resolution
```bash
dig +short mim4u.org
dig +short explorer.d-bis.org
dig +short rpc-http-pub.d-bis.org
```
### Test SSL Certificates
```bash
curl -vI https://mim4u.org 2>&1 | grep -E "(certificate|SSL|TLS)"
curl -vI https://explorer.d-bis.org 2>&1 | grep -E "(certificate|SSL|TLS)"
```
### Test Backend Services
```bash
# Test Blockscout
curl -I http://192.168.11.140:80
# Test MIM4U
curl -I http://192.168.11.37:80
# Test DBIS API
curl -I http://192.168.11.155:3000
# Test RPC
curl -X POST http://192.168.11.221:8545 \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}'
```
### Check NPMplus Status
```bash
# From Proxmox host
ssh root@192.168.11.11 "pct exec 10233 -- docker ps --filter 'name=npmplus'"
# Check NPMplus logs
ssh root@192.168.11.11 "pct exec 10233 -- docker logs npmplus --tail 50"
```
### Check VM Status
```bash
# Check specific VM
ssh root@192.168.11.12 "pct status 7810"
# Check nginx status on VM
ssh root@192.168.11.12 "pct exec 7810 -- systemctl status nginx"
```
---
## Related Documentation
- **VMID Endpoints**: `docs/04-configuration/ALL_VMIDS_ENDPOINTS.md`
- **NPMplus Setup**: `docs/04-configuration/NPMPLUS_COMPLETE_SETUP_SUMMARY.md`
- **NPMplus Service Mapping**: `docs/04-configuration/NPMPLUS_SERVICE_MAPPING_COMPLETE.md`
- **MIM4U DNS Config**: `reports/VMID_7810_DNS_NPMPLUS_CONFIGURATION.md`
- **Cloudflare DNS**: `docs/04-configuration/cloudflare/CLOUDFLARE_DNS_SPECIFIC_SERVICES.md`
---
**Last Updated**: 2026-01-20
**Maintained By**: Infrastructure Team
**Status**: ✅ Complete Architecture Reference
Reference in New Issue View Git Blame Copy Permalink