proxmox/docs/04-configuration/PORT_4000_INVESTIGATION_REPORT.md

# Port 4000 Configuration Investigation Report

**Last Updated:** 2026-01-31  
**Document Version:** 1.0  
**Status:** Active Documentation

---

**Date**: 2026-01-18  
**Domain**: `explorer.d-bis.org`  
**Status**: ⚠️ **Configuration Intentional - BUT NOT YET IMPLEMENTED**

---

## Executive Summary

The port 4000 configuration for `explorer.d-bis.org` is **intentional and correct**. It implements a **direct route** to Blockscout, bypassing the nginx proxy layer to fix 502 Bad Gateway errors and improve performance.

### Current Configuration Status

- ✅ **Configuration Files**: All updated to use port 4000
- ✅ **Documentation**: Fully documented
- ❌ **Blockscout Service**: **NOT accessible on port 4000** (diagnostic confirmed)
- ❌ **Port 4000**: NOT listening (service may not be running or not configured)
- ⚠️ **NPMplus**: Should remain on port 80 until Blockscout is fixed

### 🔴 Critical Finding

**Diagnostic Result (2026-01-18):**
- ❌ Port 4000 is NOT accessible from network
- ❌ Port 4000 is NOT listening on VMID 5000
- ❌ Blockscout does not respond on localhost:4000
- ❌ Blockscout service may not be running or not bound to port 4000

**Conclusion**: The port 4000 configuration is **planned/intentional** but **not yet implemented**. Blockscout needs to be configured to listen on port 4000 before the direct route can work.

---

## Why Port 4000?

### The Problem

**Old Architecture (Port 80):**
```
Internet → NPMplus → 192.168.11.140:80 (nginx) → 127.0.0.1:4000 (Blockscout)
```

This architecture had issues:
- ❌ nginx proxy layer added latency
- ❌ 502 Bad Gateway errors
- ❌ Additional point of failure
- ❌ More complex routing

### The Solution

**New Architecture (Port 4000 - Direct Route):**
```
Internet → NPMplus → 192.168.11.140:4000 (Blockscout directly)
```

Benefits:
- ✅ Removes nginx proxy layer (one less hop)
- ✅ Reduces latency
- ✅ Fewer points of failure
- ✅ Simpler architecture
- ✅ Should fix 502 errors

---

## Configuration Details

### NPMplus Configuration

**Domain**: `explorer.d-bis.org`  
**Target**: `http://192.168.11.140:4000`  
**Scheme**: `http`  
**WebSocket**: Not enabled  
**Status**: Configured in scripts, needs verification in running system

### Blockscout Service (VMID 5000)

**IP**: 192.168.11.140  
**Port**: 4000  
**Service**: Blockscout blockchain explorer  
**Network Binding**: Must be accessible on `0.0.0.0:4000` (not just `127.0.0.1:4000`)

---

## Files That Reference Port 4000

### ✅ Configuration Scripts (Updated)

1. **`scripts/nginx-proxy-manager/configure-npmplus-domains.js`**
   - Line 35: `explorer.d-bis.org → http://192.168.11.140:4000`

2. **`scripts/nginx-proxy-manager/configure-ssl-all-domains.js`**
   - Updated to port 4000

3. **`scripts/nginx-proxy-manager/configure-ssl-api.js`**
   - Updated to port 4000

### ✅ Documentation (Updated)

1. **`docs/04-configuration/RPC_ENDPOINTS_MASTER.md`**
   - Port: 4000
   - Note: "Direct Route - bypasses nginx"

2. **`docs/04-configuration/ALL_VMIDS_ENDPOINTS.md`**
   - Port: 4000
   - Note: "Direct Route"

3. **`docs/04-configuration/DIRECT_BLOCKSCOUT_ROUTE_UPDATE.md`**
   - Complete documentation of the change

### ✅ Diagnostic Scripts

1. **`scripts/verify-blockscout-port-4000.sh`**
   - Verifies Blockscout accessibility on port 4000

2. **`scripts/apply-direct-blockscout-route.sh`**
   - Applies the direct route configuration to NPMplus

3. **`scripts/diagnose-explorer-502-error.sh`**
   - Comprehensive diagnostic tool

---

## Comparison with Endpoints Export

### From `endpoints-export.json`

The endpoints export shows Blockscout on **port 80**, not port 4000:

```json
{
  "vmid": "5000",
  "ip": "192.168.11.140",
  "hostname": "blockscout-1",
  "service": "Web",
  "protocol": "http",
  "port": "80",
  "domain": "explorer.d-bis.org",
  "status": "Running",
  "purpose": "Blockchain explorer"
}
```

### Discrepancy Explanation

**This is expected** because:

1. **Port 80**: Blockscout still has nginx running on port 80 (for other services/local access)
2. **Port 4000**: Blockscout service itself runs on port 4000
3. **Direct Route**: NPMplus is configured to bypass nginx and go directly to port 4000

The endpoints JSON may not reflect the direct route configuration because:
- It captures the service as configured on the VM (which has both ports)
- The NPMplus routing configuration is separate from the VM endpoint configuration
- Port 80 may still be documented for backward compatibility

---

## Verification Checklist

### ✅ Step 1: Verify Blockscout Accessibility

Check if Blockscout is accessible on port 4000:

```bash
curl -I http://192.168.11.140:4000/api/v2/stats
```

**Expected**: HTTP 200  
**If fails**: Blockscout may only be listening on localhost or not running

### ✅ Step 2: Check NPMplus Configuration

Verify NPMplus is routing to port 4000:

**Option A: Check via Web UI**
1. Log into NPMplus: `https://192.168.0.166:81`
2. Navigate to Proxy Hosts
3. Find `explorer.d-bis.org`
4. Verify Forward Port is `4000` (not `80`)

**Option B: Check via API/Database**
```bash
# From NPMplus container (VMID 10233)
docker exec npmplus node -e "
const Database = require('better-sqlite3');
const db = new Database('/data/npmplus/database.sqlite', { readonly: true });
const host = db.prepare('SELECT * FROM proxy_host WHERE domain_names LIKE \"%explorer.d-bis.org%\"').get();
console.log(JSON.stringify(host, null, 2));
db.close();
"
```

### ✅ Step 3: Test Public Domain

Test the public domain endpoint:

```bash
curl -I https://explorer.d-bis.org/api/v2/stats
```

**Expected**: HTTP 200 (not 502)  
**If 502**: NPMplus is still routing to port 80 or Blockscout isn't accessible on 4000

### ✅ Step 4: Check Blockscout Binding

Verify Blockscout is listening on the network interface (not just localhost):

```bash
# From Proxmox host
pct exec 5000 -- ss -tlnp | grep :4000
```

**Expected**: Should show `0.0.0.0:4000` or `192.168.11.140:4000`  
**If shows `127.0.0.1:4000`**: Blockscout needs to be reconfigured

---

## Prerequisites for Direct Route

Before the direct route works, ensure:

1. **Blockscout Service Running**
   ```bash
   pct exec 5000 -- systemctl status blockscout.service
   ```

2. **Blockscout Listening on Network Interface**
   ```bash
   pct exec 5000 -- ss -tlnp | grep :4000
   # Should show 0.0.0.0:4000, not 127.0.0.1:4000
   ```

3. **Network Accessibility**
   ```bash
   curl -I http://192.168.11.140:4000/api/v2/stats
   # Should return HTTP 200
   ```

4. **NPMplus Configuration Updated**
   - Forward Port set to `4000` (not `80`)

---

## Action Items

### Immediate Actions

1. **Verify NPMplus Configuration**
   - Check if `explorer.d-bis.org` is routing to port 4000
   - If still on port 80, update it

2. **Verify Blockscout Accessibility**
   - Test: `curl -I http://192.168.11.140:4000/api/v2/stats`
   - Should return HTTP 200

3. **Test Public Domain**
   - Test: `curl -I https://explorer.d-bis.org/api/v2/stats`
   - Should return HTTP 200 (not 502)

### If Blockscout Not Accessible on Port 4000

**Check Blockscout binding**:
```bash
pct exec 5000 -- ss -tlnp | grep :4000
```

**If only listening on localhost**, configure Blockscout to listen on all interfaces:
- Update Docker port binding: `0.0.0.0:4000:4000` (not `127.0.0.1:4000:4000`)
- Or update systemd service to bind to `0.0.0.0:4000`

### Update Endpoints Export

Consider updating `get-all-endpoints.sh` to reflect the direct route:
- Add a note about port 4000 for `explorer.d-bis.org`
- Or include both ports (80 and 4000) with notes

---

## Architecture Diagram

### Old Architecture (Port 80)

```
┌─────────┐     ┌──────────┐     ┌──────────┐     ┌─────────────┐
│ Internet│────▶│ NPMplus  │────▶│ nginx:80 │────▶│Blockscout:  │
│         │     │          │     │          │     │  4000       │
└─────────┘     └──────────┘     └──────────┘     └─────────────┘
                                    (localhost)
                                   127.0.0.1:4000
```

**Issues**: Extra hop, 502 errors

### New Architecture (Port 4000 - Direct)

```
┌─────────┐     ┌──────────┐                       ┌─────────────┐
│ Internet│────▶│ NPMplus  │──────────────────────▶│Blockscout:  │
│         │     │          │     Direct Route      │  4000       │
└─────────┘     └──────────┘     (0.0.0.0:4000)    └─────────────┘
```

**Benefits**: Direct connection, faster, more reliable

---

## Conclusion

The port 4000 configuration is **planned and intentional**, but **not yet fully implemented**. 

### 🔴 Critical Finding

**Diagnostic Result (2026-01-18):**
- ❌ Port 4000 is **NOT accessible** from network (`curl` failed)
- ❌ Port 4000 is **NOT listening** on VMID 5000
- ❌ Blockscout does **NOT respond** on localhost:4000
- ❌ Blockscout service may not be running or not configured for port 4000

### Current Situation

The configuration files and documentation all reference port 4000 as the target, which is the **correct plan** for a direct route architecture. However:

**The service is not yet accessible on port 4000**, which means:

1. **NPMplus Configuration**: Should NOT be updated to port 4000 yet (would cause 502 errors)
2. **Current Route**: Still needs to use port 80 (via nginx) until Blockscout is fixed
3. **Implementation Required**: Blockscout must be configured to listen on port 4000 first

### Implementation Steps (In Order)

1. **Fix Blockscout Service** (REQUIRED FIRST)
   - Start Blockscout service if not running
   - Configure to listen on `0.0.0.0:4000` (network accessible, not just localhost)
   - For Docker: Update port binding to `0.0.0.0:4000:4000`
   - For systemd: Update service environment variables
   - Verify: `curl -I http://192.168.11.140:4000/api/v2/stats` returns HTTP 200

2. **Then Update NPMplus** (ONLY AFTER STEP 1)
   - Only after Blockscout is accessible on port 4000
   - Update `explorer.d-bis.org` Forward Port: 80 → 4000
   - Verify: `curl -I https://explorer.d-bis.org/api/v2/stats` returns HTTP 200

### Benefits (Once Implemented)

✅ Bypasses nginx for improved performance  
✅ Reduces latency by removing a proxy layer  
✅ Fixes 502 Bad Gateway errors  
✅ Simplifies the architecture

### Diagnostic Tools

Run the diagnostic script to check current status:
```bash
./scripts/diagnose-blockscout-port-4000.sh
```

---

## Related Documentation

- [DIRECT_BLOCKSCOUT_ROUTE_UPDATE.md](./DIRECT_BLOCKSCOUT_ROUTE_UPDATE.md) - Complete update guide
- [APPLY_DIRECT_ROUTE_MANUAL.md](./APPLY_DIRECT_ROUTE_MANUAL.md) - Manual application instructions
- [RPC_ENDPOINTS_MASTER.md](./RPC_ENDPOINTS_MASTER.md) - Endpoint master reference
- [ALL_VMIDS_ENDPOINTS.md](./ALL_VMIDS_ENDPOINTS.md) - All VMID endpoints

---

**Last Updated**: 2026-01-18  
**Investigated By**: AI Assistant  
**Status**: Configuration intentional - verification recommended