proxmox/docs/04-configuration/OMADA_HARDWARE_CONFIGURATION_REVIEW.md

# Omada Hardware & Configuration Review

**Last Updated:** 2026-01-31  
**Document Version:** 1.0  
**Status:** Active Documentation

---

**Review Date:** 2025-01-20  
**Reviewer:** Infrastructure Team  
**Status:** Comprehensive Review

---

## Executive Summary

This document provides a comprehensive review of all Omada hardware and configuration in the environment. The review covers:

- **Hardware Inventory**: 2× ER605 routers, 3× ES216G switches
- **Controller Configuration**: Omada Controller on ml110 (192.168.11.8)
- **Network Architecture**: Current flat LAN (192.168.11.0/24) with planned VLAN migration
- **API Integration**: Omada API library and MCP server configured
- **Configuration Status**: Partial deployment (Phase 0 complete, Phase 1+ pending)

---

## 1. Hardware Inventory

### 1.1 Routers

#### ER605-A (Primary Edge Router)

**Status:** ✅ Configured (Phase 0 Complete)

**Configuration:**
- **WAN1 (ER605):** Replaced by UDM Pro.
- **UDM Pro (edge):** 76.53.10.34. Port forwarding: 76.53.10.36:80/443 → 192.168.11.167:80/443 (NPMplus LXC). Proxmox hosts: 192.168.11.10–12. NPMplus has .166 and .167; only .167 in UDM Pro.

- **WAN2 (Failover):**
  - ISP: ISP #2 (to be configured)
  - Failover Mode: Pending configuration
  - Priority: Lower than WAN1 (planned)

- **LAN:**
  - Connection: Trunk to ES216G-1 (core switch)
  - Current Network: 192.168.11.0/24 (flat LAN)
  - Planned: VLAN-aware trunk with 16+ VLANs

**Role:** Active edge router, NAT pools, inter-VLAN routing

**Configuration Status:**
- ✅ WAN1 configured with Block #1
- ⏳ WAN2 failover configuration pending
- ⏳ VLAN interfaces creation pending (16 VLANs planned)
- ⏳ Role-based egress NAT pools pending (Blocks #2-6)

#### ER605-B (Standby Edge Router)

**Status:** ⏳ Pending Configuration

**Planned Configuration:**
- **WAN1:** ISP #2 (alternate/standby)
- **WAN2:** Optional (if available)
- **LAN:** Trunk to ES216G-1 (core switch)

**Role Decision Required:**
- **Option A:** Standby edge router (failover only)
- **Option B:** Dedicated sovereign edge (separate policy domain)

**Note:** ER605 does not support full stateful HA. This is **active/standby operational redundancy**, not automatic session-preserving HA.

**Configuration Status:**
- ⏳ Physical deployment status unknown
- ⏳ Configuration not started
- ⏳ Role decision pending

---

### 1.2 Switches

#### ES216G-1 (Core Switch)

**Status:** ⏳ Configuration Pending

**Planned Role:** Core / uplinks / trunks

**Configuration Requirements:**
- Trunk ports to ES216G-2 and ES216G-3
- Trunk port to ER605-A (LAN)
- VLAN trunking support for all VLANs (11, 110-112, 120-121, 130-134, 140-141, 150, 160, 200-203)
- Native VLAN: 11 (MGMT-LAN)

**Configuration Status:**
- ⏳ Trunk ports configuration pending
- ⏳ VLAN configuration pending
- ⏳ Physical deployment status unknown

#### ES216G-2 (Compute Rack Aggregation)

**Status:** ⏳ Configuration Pending

**Planned Role:** Compute rack aggregation

**Configuration Requirements:**
- Trunk ports to R630 compute nodes (4×)
- Trunk port to ML110 (management node)
- Trunk port to ES216G-1 (core)
- VLAN trunking support for all VLANs
- Native VLAN: 11 (MGMT-LAN)

**Configuration Status:**
- ⏳ Trunk ports configuration pending
- ⏳ VLAN configuration pending
- ⏳ Physical deployment status unknown

#### ES216G-3 (Management & Out-of-Band)

**Status:** ⏳ Configuration Pending

**Planned Role:** Management + out-of-band / staging

**Configuration Requirements:**
- Management access ports (untagged VLAN 11)
- Staging ports (untagged VLAN 11 or tagged staging VLAN)
- Trunk port to ES216G-1 (core)
- VLAN trunking support
- Native VLAN: 11 (MGMT-LAN)

**Configuration Status:**
- ⏳ Configuration pending
- ⏳ Physical deployment status unknown

---

### 1.3 Omada Controller

**Location:** ML110 Gen9 (Bootstrap & Management node)  
**IP Address:** `192.168.11.8:8043` (actual) / `192.168.11.10` (documented)  
**Status:** ✅ Operational

**Note:** There is a discrepancy between documented IP (192.168.11.10) and configured IP (192.168.11.8). The actual controller is accessible at 192.168.11.8:8043.

**Configuration:**
- **Base URL:** `https://192.168.11.8:8043`
- **SSL Verification:** Disabled (OMADA_VERIFY_SSL=false)
- **Site ID:** `090862bebcb1997bb263eea9364957fe`
- **API Credentials:** Configured (Client ID/Secret)

**API Configuration:**
- **Client ID:** `273615420c01452a8a2fd2e00a177eda`
- **Client Secret:** `8d3dc336675e4b04ad9c1614a5b939cc`
- **Authentication Note:** See `OMADA_AUTH_NOTE.md` for authentication method details

**Features:**
- ✅ Open API enabled
- ✅ API credentials configured
- ⏳ Device adoption status unknown (needs verification)
- ⏳ Device management status unknown (needs verification)

---

## 2. Network Architecture

### 2.1 Current State (Flat LAN)

**Network:** 192.168.11.0/24  
**Gateway:** 192.168.11.1 (ER605-A)  
**DHCP:** Configured (if applicable)  
**Status:** ✅ Operational (Phase 0)

**Current Services:**
- 12 Besu containers (validators, sentries, RPC nodes)
- All services on flat LAN (192.168.11.0/24)
- No VLAN segmentation

### 2.2 Planned State (VLAN-based)

**Migration Status:** ⏳ Pending (Phase 1)

**VLAN Plan:** 16+ VLANs planned

#### Key VLANs:

| VLAN ID | VLAN Name | Subnet | Gateway | Purpose | Status |
|--------:|-----------|--------|---------|---------|--------|
| 11 | MGMT-LAN | 192.168.11.0/24 | 192.168.11.1 | Proxmox mgmt, switches mgmt | ⏳ Pending |
| 110 | BESU-VAL | 10.110.0.0/24 | 10.110.0.1 | Validator-only network | ⏳ Pending |
| 111 | BESU-SEN | 10.111.0.0/24 | 10.111.0.1 | Sentry mesh | ⏳ Pending |
| 112 | BESU-RPC | 10.112.0.0/24 | 10.112.0.1 | RPC / gateway tier | ⏳ Pending |
| 120 | BLOCKSCOUT | 10.120.0.0/24 | 10.120.0.1 | Explorer + DB | ⏳ Pending |
| 121 | CACTI | 10.121.0.0/24 | 10.121.0.1 | Interop middleware | ⏳ Pending |
| 130 | CCIP-OPS | 10.130.0.0/24 | 10.130.0.1 | Ops/admin | ⏳ Pending |
| 132 | CCIP-COMMIT | 10.132.0.0/24 | 10.132.0.1 | Commit-role DON | ⏳ Pending |
| 133 | CCIP-EXEC | 10.133.0.0/24 | 10.133.0.1 | Execute-role DON | ⏳ Pending |
| 134 | CCIP-RMN | 10.134.0.0/24 | 10.134.0.1 | Risk management network | ⏳ Pending |
| 140 | FABRIC | 10.140.0.0/24 | 10.140.0.1 | Fabric | ⏳ Pending |
| 141 | FIREFLY | 10.141.0.0/24 | 10.141.0.1 | FireFly | ⏳ Pending |
| 150 | INDY | 10.150.0.0/24 | 10.150.0.1 | Identity | ⏳ Pending |
| 160 | SANKOFA-SVC | 10.160.0.0/22 | 10.160.0.1 | Service layer | ⏳ Pending |
| 200 | PHX-SOV-SMOM | 10.200.0.0/20 | 10.200.0.1 | Sovereign tenant | ⏳ Pending |
| 201 | PHX-SOV-ICCC | 10.201.0.0/20 | 10.201.0.1 | Sovereign tenant | ⏳ Pending |
| 202 | PHX-SOV-DBIS | 10.202.0.0/20 | 10.202.0.1 | Sovereign tenant | ⏳ Pending |
| 203 | PHX-SOV-AR | 10.203.0.0/20 | 10.203.0.1 | Sovereign tenant | ⏳ Pending |

**Migration Requirements:**
- Configure VLAN interfaces on ER605-A for all VLANs
- Configure trunk ports on all ES216G switches
- Enable VLAN-aware bridge on Proxmox hosts
- Migrate services from flat LAN to appropriate VLANs

---

## 3. Public IP Blocks & NAT Configuration

### 3.1 Public IP Block #1 (Configured)

**Network:** 76.53.10.32/28  
**Gateway:** 76.53.10.33  
**Usable Range:** 76.53.10.33–76.53.10.46  
**Broadcast:** 76.53.10.47  
**UDM Pro (edge):** 76.53.10.34 (replaced ER605). Port forward: 76.53.10.36:80/443 → 192.168.11.167:80/443.  
**Status:** ✅ Active

**Usage:**
- ER605-A WAN1 interface
- Break-glass emergency VIPs (planned)
  - 76.53.10.35: Emergency SSH/Jumpbox (planned)
  - 76.53.10.36: Emergency Besu RPC (planned)
  - 76.53.10.37: Emergency FireFly (planned)
  - 76.53.10.38: Sankofa/Phoenix/PanTel VIP (planned)
  - 76.53.10.39: Indy DID endpoints (planned)

### 3.2 Public IP Blocks #2-6 (Pending)

**Status:** ⏳ To Be Configured (when assigned)

| Block | Network | Gateway | Designated Use | NAT Pool Target | Status |
|-------|---------|---------|----------------|-----------------|--------|
| #2 | `<PUBLIC_BLOCK_2>/28` | `<GW2>` | CCIP Commit egress NAT pool | 10.132.0.0/24 (VLAN 132) | ⏳ Pending |
| #3 | `<PUBLIC_BLOCK_3>/28` | `<GW3>` | CCIP Execute egress NAT pool | 10.133.0.0/24 (VLAN 133) | ⏳ Pending |
| #4 | `<PUBLIC_BLOCK_4>/28` | `<GW4>` | RMN egress NAT pool | 10.134.0.0/24 (VLAN 134) | ⏳ Pending |
| #5 | `<PUBLIC_BLOCK_5>/28` | `<GW5>` | Sankofa/Phoenix/PanTel service egress | 10.160.0.0/22 (VLAN 160) | ⏳ Pending |
| #6 | `<PUBLIC_BLOCK_6>/28` | `<GW6>` | Sovereign Cloud Band tenant egress | 10.200.0.0/20-10.203.0.0/20 (VLANs 200-203) | ⏳ Pending |

**Configuration Requirements:**
- Configure outbound NAT pools on ER605-A
- Map each private subnet to its designated public IP block
- Enable PAT (Port Address Translation)
- Configure firewall rules for egress traffic
- Document IP allowlisting requirements

---

## 4. API Integration & Automation

### 4.1 Omada API Library

**Location:** `/home/intlc/projects/proxmox/omada-api/`  
**Status:** ✅ Implemented

**Features:**
- TypeScript library for Omada Controller REST API
- OAuth2 authentication with automatic token refresh
- Support for all Omada devices (ER605, ES216G, EAP)
- Device management (list, configure, reboot, adopt)
- Network configuration (VLANs, DHCP, routing)
- Firewall and NAT rule management
- Switch port configuration and PoE management
- Router WAN/LAN configuration

### 4.2 MCP Server

**Location:** `/home/intlc/projects/proxmox/mcp-omada/`  
**Status:** ✅ Implemented

**Features:**
- Model Context Protocol server for Omada devices
- Claude Desktop integration
- Available tools:
  - `omada_list_devices` - List all devices
  - `omada_get_device` - Get device details
  - `omada_list_vlans` - List VLAN configurations
  - `omada_get_vlan` - Get VLAN details
  - `omada_reboot_device` - Reboot a device
  - `omada_get_device_statistics` - Get device statistics
  - `omada_list_firewall_rules` - List firewall rules
  - `omada_get_switch_ports` - Get switch port configuration
  - `omada_get_router_wan` - Get router WAN configuration
  - `omada_list_sites` - List all sites

**Configuration:**
- Environment variables loaded from `~/.env`
- Base URL: `https://192.168.11.8:8043`
- Client ID: Configured
- Client Secret: Configured
- Site ID: `090862bebcb1997bb263eea9364957fe`
- SSL Verification: Disabled

**Connection Status:** ⚠️ Cannot connect to controller (network issue or controller offline)

### 4.3 Test Script

**Location:** `/home/intlc/projects/proxmox/test-omada-connection.js`  
**Status:** ✅ Implemented

**Purpose:** Test Omada API connection and authentication

**Last Test Result:** ❌ Failed (Network error: Failed to connect)

**Possible Causes:**
- Controller not accessible from current environment
- Network connectivity issue
- Firewall blocking connection
- Controller service offline

---

## 5. Configuration Issues & Discrepancies

### 5.1 IP Address Discrepancy

**Issue:** Omada Controller IP mismatch

- **Documented:** 192.168.11.10 (ML110 management IP)
- **Actual Configuration:** 192.168.11.8:8043

**Impact:**
- API connections may fail if using documented IP
- Documentation inconsistency

**Recommendation:**
- Verify actual controller IP and update documentation
- Clarify if controller runs on different host or if IP changed
- Update all references in documentation

### 5.2 Authentication Method

**Issue:** Authentication method confusion

**Documented:** OAuth Client Credentials mode  
**Actual:** May require admin username/password (see `OMADA_AUTH_NOTE.md`)

**Note:** The Omada Controller API `/api/v2/login` endpoint may require admin username/password, not OAuth Client ID/Secret.

**Recommendation:**
- Verify actual authentication method required
- Update code or configuration accordingly
- Document correct authentication approach

### 5.3 Device Adoption Status

**Issue:** Unknown device adoption status

**Status:** Not verified

**Questions:**
- Are ER605-A and ER605-B adopted in Omada Controller?
- Are ES216G-1, ES216G-2, and ES216G-3 adopted?
- What is the actual device inventory?

**Recommendation:**
- Query Omada Controller to list all adopted devices
- Verify device names, IPs, firmware versions
- Document actual hardware inventory
- Verify device connectivity and status

### 5.4 Configuration Completeness

**Issue:** Many configurations are planned but not implemented

**Missing Configurations:**
- ER605-A: VLAN interfaces (16+ VLANs)
- ER605-A: WAN2 failover configuration
- ER605-A: Role-based egress NAT pools (Blocks #2-6)
- ER605-B: Complete configuration
- ES216G switches: Trunk port configuration
- ES216G switches: VLAN configuration
- Proxmox: VLAN-aware bridge configuration
- Services: VLAN migration from flat LAN

**Recommendation:**
- Prioritize Phase 1 (VLAN Enablement)
- Create detailed implementation checklist
- Execute configurations in logical order
- Verify each step before proceeding

---

## 6. Deployment Status Summary

### Phase 0 — Foundation ✅

- [x] ER605 replaced by UDM Pro (76.53.10.34); port forward 76.53.10.36:80/443 → 192.168.11.167
- [x] Proxmox mgmt accessible
- [x] Basic containers deployed
- [x] Omada Controller operational
- [x] API integration code implemented

### Phase 1 — VLAN Enablement ⏳

- [ ] ES216G trunk ports configured
- [ ] VLAN-aware bridge enabled on Proxmox
- [ ] VLAN interfaces created on ER605-A
- [ ] Services migrated to VLANs
- [ ] VLAN routing verified

### Phase 2 — Observability ⏳

- [ ] Monitoring stack deployed
- [ ] Grafana published via Cloudflare Access
- [ ] Alerts configured
- [ ] Device monitoring enabled

### Phase 3 — CCIP Fleet ⏳

- [ ] CCIP Ops/Admin deployed
- [ ] 16 commit nodes deployed
- [ ] 16 execute nodes deployed
- [ ] 7 RMN nodes deployed
- [ ] NAT pools configured (Blocks #2-4)

### Phase 4 — Sovereign Tenants ⏳

- [ ] Sovereign VLANs configured
- [ ] Tenant isolation enforced
- [ ] Access control configured
- [ ] NAT pools configured (Block #6)

---

## 7. Recommendations

### 7.1 Immediate Actions (This Week)

1. **Verify Device Inventory**
   - Connect to Omada Controller web interface
   - Document all adopted devices (routers, switches, APs)
   - Verify device names, IPs, firmware versions
   - Check device connectivity status

2. **Resolve IP Discrepancy**
   - Verify actual Omada Controller IP (192.168.11.8 vs 192.168.11.10)
   - Update documentation with correct IP
   - Verify API connectivity from management host

3. **Fix API Authentication**
   - Verify required authentication method (OAuth vs admin credentials)
   - Update code/configuration accordingly
   - Test API connection successfully

4. **Document Current Configuration**
   - Export ER605-A configuration
   - Document actual VLAN configuration (if any)
   - Document actual switch configuration (if any)
   - Create baseline configuration document

### 7.2 Short-term Actions (This Month)

1. **Complete ER605-A Configuration**
   - Configure WAN2 failover
   - Create VLAN interfaces for all planned VLANs
   - Configure DHCP for each VLAN (if needed)
   - Test inter-VLAN routing

2. **Configure ES216G Switches**
   - Configure trunk ports (802.1Q)
   - Configure VLANs on switches
   - Verify VLAN tagging
   - Test connectivity between switches

3. **Enable VLAN-aware Bridge on Proxmox**
   - Configure vmbr0 for VLAN-aware mode
   - Test VLAN tagging on container interfaces
   - Verify connectivity to ER605 VLAN interfaces

4. **Begin VLAN Migration**
   - Migrate one service VLAN as pilot
   - Verify routing and connectivity
   - Migrate remaining services systematically

### 7.3 Medium-term Actions (This Quarter)

1. **Configure NAT Pools**
   - Obtain public IP blocks #2-6
   - Configure role-based egress NAT pools
   - Test allowlisting functionality
   - Document IP usage per role

2. **Configure ER605-B**
   - Decide on role (standby vs dedicated sovereign edge)
   - Configure according to chosen role
   - Test failover (if standby)

3. **Implement Monitoring**
   - Deploy monitoring stack
   - Configure device monitoring
   - Set up alerts for device failures
   - Create dashboards for network status

4. **Complete CCIP Fleet Deployment**
   - Deploy all CCIP nodes
   - Configure NAT pools for CCIP VLANs
   - Verify connectivity and routing

---

## 8. Configuration Files Reference

### 8.1 Environment Configuration

**Location:** `~/.env`

```bash
OMADA_CONTROLLER_URL=https://192.168.11.8:8043
OMADA_API_KEY=273615420c01452a8a2fd2e00a177eda
OMADA_API_SECRET=8d3dc336675e4b04ad9c1614a5b939cc
OMADA_SITE_ID=090862bebcb1997bb263eea9364957fe
OMADA_VERIFY_SSL=false
```

### 8.2 Documentation Files

- **Network Architecture:** `docs/02-architecture/NETWORK_ARCHITECTURE.md`
- **ER605 Configuration Guide:** `docs/04-configuration/ER605_ROUTER_CONFIGURATION.md`
- **Omada API Setup:** `docs/04-configuration/OMADA_API_SETUP.md`
- **Deployment Status:** `docs/03-deployment/DEPLOYMENT_STATUS_CONSOLIDATED.md`
- **Authentication Notes:** `OMADA_AUTH_NOTE.md`

### 8.3 Code Locations

- **Omada API Library:** `omada-api/`
- **MCP Server:** `mcp-omada/`
- **Test Script:** `test-omada-connection.js`

---

## 9. Verification Checklist

Use this checklist to verify current configuration:

### Hardware Verification

- [ ] ER605-A is adopted in Omada Controller
- [ ] UDM Pro port forward: 76.53.10.36:80/443 → 192.168.11.167:80/443 (NPMplus)
- [ ] ER605-A can reach internet via WAN1
- [ ] ER605-B is adopted (if deployed)
- [ ] ES216G-1 is adopted and accessible
- [ ] ES216G-2 is adopted and accessible
- [ ] ES216G-3 is adopted and accessible
- [ ] All switches are manageable via Omada Controller

### Network Verification

- [ ] Current flat LAN (192.168.11.0/24) is operational
- [ ] Gateway (192.168.11.1) is reachable
- [ ] DNS resolution works
- [ ] Inter-VLAN routing works (if VLANs configured)
- [ ] Switch trunk ports are configured correctly

### API Verification

- [ ] Omada Controller API is accessible
- [ ] API authentication works
- [ ] Can list devices via API
- [ ] Can query device details via API
- [ ] Can list VLANs via API
- [ ] MCP server can connect and function

### Configuration Verification

- [ ] ER605-A configuration matches documentation
- [ ] VLAN interfaces exist (if VLANs configured)
- [ ] Switch VLANs match router VLANs
- [ ] Proxmox VLAN-aware bridge is configured (if VLANs configured)
- [ ] NAT pools are configured (if public blocks assigned)

---

## 10. Next Steps

1. **Verify actual hardware inventory** by querying Omada Controller
2. **Resolve IP discrepancy** and update documentation
3. **Fix API connectivity** and authentication
4. **Create detailed implementation plan** for Phase 1 (VLAN Enablement)
5. **Execute Phase 1** systematically with verification at each step
6. **Document actual configuration** as implementation progresses

---

**Document Status:** Complete (Initial Review)  
**Maintained By:** Infrastructure Team  
**Review Cycle:** Monthly  
**Last Updated:** 2025-01-20