Files
Sankofa/docs/infrastructure/CEPH_SETUP_ISSUE.md
defiQUG 33d50fb91e
Some checks failed
API CI / API Lint (push) Successful in 47s
API CI / API Type Check (push) Failing after 47s
API CI / API Test (push) Successful in 1m0s
API CI / API Build (push) Failing after 50s
API CI / Build Docker Image (push) Has been skipped
Build Crossplane Provider / build (push) Failing after 5m51s
CD Pipeline / Deploy to Staging (push) Failing after 29s
CI Pipeline / Lint and Type Check (push) Failing after 36s
CI Pipeline / Build (push) Has been skipped
CI Pipeline / Test Backend (push) Failing after 1m33s
CI Pipeline / Test Frontend (push) Failing after 30s
CI Pipeline / Security Scan (push) Failing after 1m16s
Crossplane Provider CI / Go Test (push) Failing after 3m23s
Crossplane Provider CI / Go Lint (push) Failing after 7m27s
Crossplane Provider CI / Go Build (push) Failing after 3m27s
Deploy to Staging / Deploy to Staging (push) Failing after 30s
Portal CI / Portal Lint (push) Failing after 21s
Portal CI / Portal Type Check (push) Failing after 21s
Portal CI / Portal Test (push) Failing after 21s
Portal CI / Portal Build (push) Failing after 22s
Test Suite / frontend-tests (push) Failing after 30s
Test Suite / api-tests (push) Failing after 49s
Test Suite / blockchain-tests (push) Failing after 30s
Type Check / type-check (map[directory:. name:root]) (push) Failing after 23s
Type Check / type-check (map[directory:api name:api]) (push) Failing after 21s
Type Check / type-check (map[directory:portal name:portal]) (push) Failing after 19s
Validate Configuration Files / validate (push) Failing after 1m52s
CD Pipeline / Deploy to Production (push) Has been skipped
chore: consolidate local WIP (repo cleanup 20260707)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-07 09:41:34 -07:00

1.7 KiB

Ceph Setup Issue - Configuration Mismatch

Date: 2025-12-15
Issue: Cannot add r630-01 monitor due to configuration mismatch

Problem

When attempting to add r630-01 as a Ceph monitor, we discovered:

  1. Different FSIDs:

    • ML110-01 cluster: f601f0e2-cd09-402f-9e15-4b1c9a7a7b25 (from earlier status)
    • R630-01 config: 021d20d3-446f-42cb-a219-5e01213b7b2d (from current /etc/pve/ceph.conf)
  2. Network Configuration Mismatch:

    • ML110-01: 192.168.11.0/24
    • R630-01: 192.168.11.11/24 (incorrect - should be 192.168.11.0/24)
  3. Monitor Connection Issues:

    • ML110-01 monitor is running but commands timeout
    • Cannot get monmap from ml110-01
    • R630-01 cannot connect to cluster

Root Cause

The /etc/pve/ceph.conf on r630-01 appears to have a different/incorrect configuration, possibly from a previous cluster setup or misconfiguration. The Proxmox cluster filesystem should sync this, but there may be a conflict.

Solution Options

Option 1: Sync Configuration from ML110-01

Since Proxmox cluster filesystem should sync /etc/pve/ceph.conf, we need to ensure:

  1. ML110-01 has the correct configuration
  2. Configuration syncs to r630-01
  3. Then add r630-01 monitor

Option 2: Reinitialize Ceph Cluster

If the cluster on ml110-01 is not operational or has issues:

  1. Stop Ceph on ml110-01
  2. Reinitialize with proper configuration
  3. Add r630-01 monitor
  4. Add OSDs

Option 3: Use Proxmox Web UI

The Proxmox web interface may handle the configuration and monitor addition more reliably.

Current Status

  • ML110-01: Monitor running but cluster commands timeout
  • R630-01: Cannot connect to cluster, configuration mismatch
  • Need to resolve configuration before proceeding