d-bis/proxmox
4
0
Fork 0
Code Issues 1 Pull Requests 9 Actions Packages Projects Releases Wiki Activity
Files
f0f0a4bb844eefe6c19e9568c3af9b2d387e5a2e
proxmox/docs/06-besu/BESU_NODES_FILE_REFERENCE.md
defiQUG fbda1b4beb
Some checks failed
Deploy to Phoenix / deploy (push) Has been cancelled
Details
docs: Ledger Live integration, contract deploy learnings, NEXT_STEPS updates 
- ADD_CHAIN138_TO_LEDGER_LIVE: Ledger form done; public code review repo bis-innovations/LedgerLive; init/push commands
- CONTRACT_DEPLOYMENT_RUNBOOK: Chain 138 gas price 1 gwei, 36-addr check, TransactionMirror workaround
- CONTRACT_*: AddressMapper, MirrorManager deployed 2026-02-12; 36-address on-chain check
- NEXT_STEPS_FOR_YOU: Ledger done; steps completable now (no LAN); run-completable-tasks-from-anywhere
- MASTER_INDEX, OPERATOR_OPTIONAL, SMART_CONTRACTS_INVENTORY_SIMPLE: updates
- LEDGER_BLOCKCHAIN_INTEGRATION_COMPLETE: bis-innovations/LedgerLive reference

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-12 15:46:57 -08:00

371 lines
22 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Besu Nodes File Reference
**Last Updated:** 2026-02-08
**Document Version:** 1.2
**Status:** Active Documentation
**See also:** [MASTER_DOCS_AND_NODE_LISTS_REVIEW.md](MASTER_DOCS_AND_NODE_LISTS_REVIEW.md) — review of master documentation, static-nodes.json, and permissions-nodes.toml. **Node lists (single source):** `config/besu-node-lists/`; deploy to **all 32 Besu nodes** (validators 1000–1004, sentries 1500–1508, RPC 2101, 2102, 2201, 2301, 2303–2306, 2400–2403, 2500–2505) with `scripts/deploy-besu-node-lists-to-all.sh`. Verify presence and checksum: `scripts/verify/verify-static-permissions-on-all-besu-nodes.sh --checksum`. Restart Besu to reload lists: `scripts/besu/restart-besu-reload-node-lists.sh` (recognizes `besu-validator`, `besu-sentry`, `besu-rpc`, and generic `besu.service` used by 1505–1508, 2500–2505). IP source of truth: `config/ip-addresses.conf`, [NETWORK_CONFIGURATION_MASTER.md](../11-references/NETWORK_CONFIGURATION_MASTER.md).
---
This document provides a comprehensive reference table mapping all Besu nodes to their container IDs, IP addresses, and the files required for each node type.
## Network Topology
This deployment follows a **production-grade validator ↔ sentry architecture** that isolates consensus from public networking and provides DDoS protection.
### Validator ↔ Sentry Topology (Logical Diagram)
```text
┌──────────────────────────┐
│ External / │
│ Internal Peers │
│ (Other Networks / │
│ RPC Consumers) │
└────────────┬─────────────┘
│
P2P (30303) │
▼
┌─────────────────────────────────────────────────┐
│ SENTRY LAYER │
│ (Public-facing, peer-heavy, no consensus) │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────┐ │
│ │ besu-sentry │ │ besu-sentry │ │ besu- │ │
│ │ -2 │ │ -3 │ │ sentry- │ │
│ │192.168.11.150 (DHCP)│ │192.168.11.151 (DHCP)│ │ 4 │ │
│ └──────┬──────┘ └──────┬──────┘ └────┬────┘ │
│ │ │ │ │
│ └─────────┬───────┴───────┬───────┘ │
└───────────────────┼───────────────┼────────────┘
│ │
Restricted P2P (30303) – static only
│ │
▼ ▼
┌─────────────────────────────────────────────────────────────────┐
│ VALIDATOR LAYER │
│ (Private, consensus-only, no public peering) │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌────────────┐│
│ │ besu- │ │ besu- │ │ besu- │ │ besu- ││
│ │ validator-1 │ │ validator-2 │ │ validator-3 │ │ validator- ││
│ │192.168.11.100 (DHCP)│ │192.168.11.101 (DHCP)│ │192.168.11.102 (DHCP)│ │ 4 ││
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬─────┘│
│ │ │ │ │ │
│ └────────────── QBFT / IBFT2 Consensus ───────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
▲
│
Internal access only
│
┌──────────────────────────────────────────┐
│ RPC LAYER │
│ (Read / Write, No P2P) │
│ │
│ Production RPC: 2101 .211 (core), 2201 .221 (public), 2301 .232 (private) │
│ HTTP 8545 / WS 8546 │
└──────────────────────────────────────────┘
```
### Topology Design Principles
#### 1. **Validators are Never Exposed**
- ❌ No public P2P connections
- ❌ No RPC endpoints exposed
- ✅ Only peer with **known sentry nodes** (via `static-nodes.json`)
- ✅ Appear in `genesis.json` validator set (if using static validators)
- ✅ Validator keys remain private and secure
#### 2. **Sentry Nodes Absorb Network Risk**
- ✅ Handle peer discovery and gossip
- ✅ Accept external connections
- ✅ Can be replaced or scaled **without touching consensus**
- ❌ Do **not** sign blocks (not validators)
- ✅ First line of defense against DDoS
#### 3. **RPC Nodes are Isolated**
- ✅ Serve dApps, indexers, and operational tooling
- ✅ Provide HTTP JSON-RPC (port 8545) and WebSocket (port 8546)
- ❌ Never participate in consensus
- ✅ Can peer with sentries or validators (internal only)
- ✅ Stateless and horizontally scalable
### Static Peering Rules
The topology enforces the following peering configuration:
| Node Type | `static-nodes.json` Contains | Purpose |
|------------|------------------------------------------------|--------------------------------------------|
| **Validators** | Sentries + other validators | Connect to network via sentries |
| **Sentries** | Validators + other sentries | Relay messages to/from validators |
| **RPC Nodes** | Sentries or validators (optional) | Internal access to network state |
### Why This Topology Is Production-Grade
✅ **DDoS-Resistant**: Validators are not publicly accessible
✅ **Security**: Validator keys never exposed to public network
✅ **Fault Isolation**: Sentry failures don't affect consensus
✅ **Easy Validator Rotation**: Replace validators without network disruption
✅ **Auditable Consensus Boundary**: Clear separation of concerns
✅ **Matches Besu / ConsenSys Best Practice**: Industry-standard architecture
## Container Information
| VMID | Hostname | IP Address | Node Type | Service Name |
|------|--------------------|---------------|-----------|-----------------------|
| 1000 | besu-validator-1 | 192.168.11.100 (DHCP) | Validator | besu-validator |
| 1001 | besu-validator-2 | 192.168.11.101 (DHCP) | Validator | besu-validator |
| 1002 | besu-validator-3 | 192.168.11.102 (DHCP) | Validator | besu-validator |
| 1003 | besu-validator-4 | 192.168.11.103 (DHCP) | Validator | besu-validator |
| 1004 | besu-validator-5 | 192.168.11.104 (DHCP) | Validator | besu-validator |
| 1500 | besu-sentry-1 | 192.168.11.150 (DHCP) | Sentry | besu-sentry |
| 1501 | besu-sentry-2 | 192.168.11.151 (DHCP) | Sentry | besu-sentry |
| 1502 | besu-sentry-3 | 192.168.11.152 (DHCP) | Sentry | besu-sentry |
| 1503 | besu-sentry-4 | 192.168.11.153 (DHCP) | Sentry | besu-sentry |
| 1504 | besu-sentry-5 / besu-sentry-ali | 192.168.11.154 | Sentry | besu-sentry |
| 2500 | besu-rpc-alltra-1 | 192.168.11.172 | ALLTRA RPC | besu-rpc (see NPMPLUS_ALLTRA_HYBX) |
| 2501 | besu-rpc-alltra-2 | 192.168.11.173 | ALLTRA RPC | besu-rpc |
| 2502 | besu-rpc-alltra-3 | 192.168.11.174 | ALLTRA RPC | besu-rpc |
### Production RPC nodes (current VMID → IP)
These are the **production** RPC nodes in use. Scripts and configs use `config/ip-addresses.conf` and [NETWORK_CONFIGURATION_MASTER.md](../11-references/NETWORK_CONFIGURATION_MASTER.md) as the IP source of truth.
| VMID | Hostname | IP Address | Role |
|------|----------|------------|------|
| 2101 | besu-rpc-core-1 | 192.168.11.211 | Admin, contract deployment (RPC_CORE_1) |
| 2102 | besu-rpc-core-2 | 192.168.11.212 | Nathan RPC, SFValley2 tunnel (RPC_CORE_2) |
| 2201 | besu-rpc-public-1 | 192.168.11.221 | Bridge, monitoring, public-facing (RPC_PUBLIC_1) |
| 2301 | besu-rpc-private-1 | 192.168.11.232 | Private RPC (RPC_PRIVATE_1) |
| 2303 | (RPC) | 192.168.11.233 | RPC |
| 2400 | (ThirdWeb primary) | 192.168.11.240 | RPC_THIRDWEB_PRIMARY |
| 2401 | (ThirdWeb) | 192.168.11.241 | RPC_THIRDWEB_1 |
| 2402 | besu-rpc-thirdweb-0x8a-2 | 192.168.11.242 | RPC_THIRDWEB_2 |
| 2403 | besu-rpc-thirdweb-0x8a-3 | 192.168.11.243 | RPC_THIRDWEB_3 |
## Required Files by Node Type
### Files Generated by Quorum Genesis Tool
The Quorum Genesis Tool typically generates the following files that are shared across all nodes:
#### Network-Wide Files (Same for All Nodes)
| File | Location | Description | Generated By |
|-----------------------------|-----------------------|------------------------------------------------|-----------------------|
| `genesis.json` | `/etc/besu/` | Network genesis block configuration (QBFT settings, but **no validators** - uses dynamic validator management) | Quorum Genesis Tool |
| `static-nodes.json` | `/etc/besu/` | List of static peer nodes (validators) | Quorum Genesis Tool |
| `permissions-nodes.toml` | `/etc/besu/` | Node allowlist (permissioned network) | Quorum Genesis Tool |
| `permissions-accounts.toml` | `/etc/besu/` | Account allowlist (if using account permissioning) | Quorum Genesis Tool |
### Files Generated by Besu (Per-Node)
#### Validator Nodes (1000-1004)
| File | Location | Description | Generated By |
|-----------------------------|-----------------------|------------------------------------------------|-----------------------|
| `config-validator.toml` | `/etc/besu/` | Besu configuration file (references validator key directory) | Deployment Script |
| `nodekey` | `/data/besu/` | Node private key (P2P identity) | Besu (first run) |
| `nodekey.pub` | `/data/besu/` | Node public key | Derived from nodekey |
| `validator-keys/` | `/keys/validators/` | Validator signing keys (QBFT/IBFT). Contains `address.txt` with validator address (NOT in genesis) | Quorum Genesis Tool |
| `database/` | `/data/besu/database/`| Blockchain database | Besu (runtime) |
**Note**: Validator addresses are stored in `/keys/validators/validator-{N}/address.txt`, not in the genesis file. The genesis file uses dynamic validator management via validator contract.
#### Sentry Nodes (1500-1504)
| File | Location | Description | Generated By |
|-----------------------------|-----------------------|------------------------------------------------|-----------------------|
| `config-sentry.toml` | `/etc/besu/` | Besu configuration file | Deployment Script |
| `nodekey` | `/data/besu/` | Node private key (P2P identity) | Besu (first run) |
| `nodekey.pub` | `/data/besu/` | Node public key | Derived from nodekey |
| `database/` | `/data/besu/database/`| Blockchain database | Besu (runtime) |
#### RPC Nodes — Production Chain 138 (2101, 2201, 2301, …)
For **admin, bridge, and deployment** use **2101** (192.168.11.211, RPC_CORE_1). See "Production RPC nodes" table above. VMIDs 2500–2502 are **ALLTRA** RPC at .172–.174, not .250–.252.
#### RPC Nodes (2500-2502 — ALLTRA)
| File | Location | Description | Generated By |
|-----------------------------|-----------------------|------------------------------------------------|-----------------------|
| `config-rpc-{type}.toml` | `/etc/besu/` | Besu configuration file (type-specific) | Deployment Script |
| `nodekey` | `/data/besu/` | Node private key (P2P identity) | Besu (first run) |
| `nodekey.pub` | `/data/besu/` | Node public key | Derived from nodekey |
| `database/` | `/data/besu/database/`| Blockchain database | Besu (runtime) |
## Complete File Reference Table
### Validator Nodes (1000-1004)
| VMID | IP Address | Required Files |
|------|---------------|-----------------------------------------------------------------------------------------------------------------|
| 1000 | 192.168.11.100 (DHCP) | `genesis.json`, `static-nodes.json`, `permissions-nodes.toml`, `permissions-accounts.toml`, `config-validator.toml`, `nodekey`, `validator-keys/` |
| 1001 | 192.168.11.101 (DHCP) | `genesis.json`, `static-nodes.json`, `permissions-nodes.toml`, `permissions-accounts.toml`, `config-validator.toml`, `nodekey`, `validator-keys/` |
| 1002 | 192.168.11.102 (DHCP) | `genesis.json`, `static-nodes.json`, `permissions-nodes.toml`, `permissions-accounts.toml`, `config-validator.toml`, `nodekey`, `validator-keys/` |
| 1003 | 192.168.11.103 (DHCP) | `genesis.json`, `static-nodes.json`, `permissions-nodes.toml`, `permissions-accounts.toml`, `config-validator.toml`, `nodekey`, `validator-keys/` |
| 1004 | 192.168.11.104 (DHCP) | `genesis.json`, `static-nodes.json`, `permissions-nodes.toml`, `permissions-accounts.toml`, `config-validator.toml`, `nodekey`, `validator-keys/` |
### Sentry Nodes (1500-1504)
| VMID | IP Address | Required Files |
|------|---------------|-----------------------------------------------------------------------------------------------------------------|
| 1500 | 192.168.11.150 (DHCP) | `genesis.json`, `static-nodes.json`, `permissions-nodes.toml`, `config-sentry.toml`, `nodekey` |
| 1501 | 192.168.11.151 (DHCP) | `genesis.json`, `static-nodes.json`, `permissions-nodes.toml`, `config-sentry.toml`, `nodekey` |
| 1502 | 192.168.11.152 (DHCP) | `genesis.json`, `static-nodes.json`, `permissions-nodes.toml`, `config-sentry.toml`, `nodekey` |
| 1503 | 192.168.11.153 (DHCP) | `genesis.json`, `static-nodes.json`, `permissions-nodes.toml`, `config-sentry.toml`, `nodekey` |
| 1504 | 192.168.11.154 | `genesis.json`, `static-nodes.json`, `permissions-nodes.toml`, `config-sentry.toml`, `nodekey` |
### RPC Nodes (2500-2502 — ALLTRA)
| VMID | IP Address | Node Type | Required Files |
|------|------------|-----------|-----------------------------------------------------------------------------------------------------------------|
| 2500 | 192.168.11.172 | **ALLTRA RPC** | `genesis.json`, `static-nodes.json`, `permissions-nodes.toml`, config, `nodekey` |
| 2501 | 192.168.11.173 | **ALLTRA RPC** | (same) |
| 2502 | 192.168.11.174 | **ALLTRA RPC** | (same) |
**Note:** For **Chain 138 admin/deployment** use **2101** (192.168.11.211, RPC_CORE_1). See "Production RPC nodes" table and [NPMPLUS_ALLTRA_HYBX_MASTER_PLAN.md](../04-configuration/NPMPLUS_ALLTRA_HYBX_MASTER_PLAN.md) for 2500-2502 IPs.
## File Locations Summary
### Configuration Directory: `/etc/besu/`
All configuration files are stored here:
- `genesis.json`
- `static-nodes.json`
- `permissions-nodes.toml`
- `permissions-accounts.toml` (validators only)
- `config-validator.toml` (validators)
- `config-sentry.toml` (sentries)
- `config-rpc-public.toml` (RPC nodes)
### Data Directory: `/data/besu/`
Runtime data and node keys:
- `nodekey` - Node private key (generated by Besu)
- `database/` - Blockchain database (created by Besu)
### Keys Directory: `/keys/validators/`
Validator signing keys (validators only):
- `validator-1/` - Validator 1 keys
- `validator-2/` - Validator 2 keys
- `validator-3/` - Validator 3 keys
- `validator-4/` - Validator 4 keys
- `validator-5/` - Validator 5 keys
## File Generation Sources
### Quorum Genesis Tool Generates:
1. **genesis.json** - Network genesis block with QBFT/IBFT configuration
2. **static-nodes.json** - List of validator enode URLs
3. **permissions-nodes.toml** - Node allowlist (can be JSON or TOML)
4. **permissions-accounts.toml** - Account allowlist (optional, for account permissioning)
5. **validator-keys/** - Validator signing keys (one directory per validator)
### Besu Generates:
1. **nodekey** - Automatically generated on first startup (if not provided)
2. **database/** - Blockchain database (created during sync)
### Deployment Scripts Generate:
1. **config-validator.toml** - Validator configuration
2. **config-sentry.toml** - Sentry configuration
3. **config-rpc-{type}.toml** - RPC node configuration (type-specific):
- `config-rpc-core.toml` - Core RPC (VMID 2500)
- `config-rpc-perm.toml` - Permissioned RPC (VMID 2501)
- `config-rpc-public.toml` - Public RPC (VMID 2502)
## Enode URL Format
Each node's enode URL is derived from:
- **Node ID**: 128 hex characters from `nodekey` (public key)
- **IP Address**: Container IP address
- **Port**: Default P2P port 30303
Format: `enode://<128-char-node-id>@<ip-address>:30303`
Example: `enode://889ba317e10114a035ef82248a26125fbc00b1cd65fb29a2106584dddd025aa3dda14657bc423e5e8bf7d91a9858e85a@192.168.11.100 (DHCP):30303`
## Validator Configuration in Genesis File
**Answer: No, validators do NOT appear in the genesis file.**
This network uses **dynamic validator management** via a validator contract. The QBFT configuration in `genesis.json` contains:
```json
"qbft": {
"blockperiodseconds": 2,
"epochlength": 30000,
"requesttimeoutseconds": 10
}
```
**Note**: There is no `validators` array in the `qbft` section of the genesis file.
### Validator Storage
Instead of being defined in the genesis file, validator addresses are:
1. **Stored in validator key directories**: `/keys/validators/validator-{N}/address.txt`
2. **Managed dynamically** via the validator contract during runtime
3. **Referenced in configuration files**: Each validator node references its key directory in `config-validator.toml`
This approach allows for:
- Dynamic addition/removal of validators without a hard fork
- Runtime validator set changes via smart contract
- More flexible validator management
### Validator Key Directory Structure
Each validator has a directory at `/keys/validators/validator-{N}/` containing:
- `key.pem` - Private key (PEM format)
- `pubkey.pem` - Public key (PEM format)
- `address.txt` - Validator address (hex format)
- `key.priv` - Private key (raw format)
## Network Configuration
- **Network ID**: 138
- **Consensus**: QBFT (Quorum Byzantine Fault Tolerance) with dynamic validators
- **P2P Port**: 30303 (all nodes)
- **RPC Port**: 8545 (RPC nodes only, validators have RPC disabled)
- **WebSocket Port**: 8546 (RPC nodes only)
- **Metrics Port**: 9545 (all nodes)
## File Permissions
All Besu files should be owned by the `besu` user:
```bash
chown -R besu:besu /etc/besu/
chown -R besu:besu /data/besu/
chown -R besu:besu /keys/validators/
```
## Quick Reference
### Check File Existence on Container
```bash
pct exec <vmid> -- ls -la /etc/besu/
pct exec <vmid> -- ls -la /data/besu/
pct exec <vmid> -- ls -la /keys/validators/ # validators only
```
### View Configuration
```bash
pct exec <vmid> -- cat /etc/besu/config-validator.toml # validators
pct exec <vmid> -- cat /etc/besu/config-sentry.toml # sentries
pct exec <vmid> -- cat /etc/besu/config-rpc-core.toml # Core RPC (2500)
pct exec <vmid> -- cat /etc/besu/config-rpc-perm.toml # Permissioned RPC (2501)
pct exec <vmid> -- cat /etc/besu/config-rpc-public.toml # Public RPC (2502)
```
### View Genesis
```bash
pct exec <vmid> -- cat /etc/besu/genesis.json
```
### View Node Allowlist
```bash
pct exec <vmid> -- cat /etc/besu/permissions-nodes.toml
pct exec <vmid> -- cat /etc/besu/static-nodes.json
```
Reference in New Issue View Git Blame Copy Permalink