51 lines
1.7 KiB
Markdown
51 lines
1.7 KiB
Markdown
|
# Ceph Setup Issue - Configuration Mismatch
|
||
|
|
|
||
|
|
**Date**: 2025-12-15
|
||
|
|
**Issue**: Cannot add r630-01 monitor due to configuration mismatch
|
||
|
|
|
||
|
|
## Problem
|
||
|
|
|
||
|
|
When attempting to add r630-01 as a Ceph monitor, we discovered:
|
||
|
|
|
||
|
|
1. **Different FSIDs**:
|
||
|
|
- ML110-01 cluster: `f601f0e2-cd09-402f-9e15-4b1c9a7a7b25` (from earlier status)
|
||
|
|
- R630-01 config: `021d20d3-446f-42cb-a219-5e01213b7b2d` (from current /etc/pve/ceph.conf)
|
||
|
|
|
||
|
|
2. **Network Configuration Mismatch**:
|
||
|
|
- ML110-01: `192.168.11.0/24`
|
||
|
|
- R630-01: `192.168.11.11/24` (incorrect - should be 192.168.11.0/24)
|
||
|
|
|
||
|
|
3. **Monitor Connection Issues**:
|
||
|
|
- ML110-01 monitor is running but commands timeout
|
||
|
|
- Cannot get monmap from ml110-01
|
||
|
|
- R630-01 cannot connect to cluster
|
||
|
|
|
||
|
|
## Root Cause
|
||
|
|
|
||
|
|
The `/etc/pve/ceph.conf` on r630-01 appears to have a different/incorrect configuration, possibly from a previous cluster setup or misconfiguration. The Proxmox cluster filesystem should sync this, but there may be a conflict.
|
||
|
|
|
||
|
|
## Solution Options
|
||
|
|
|
||
|
|
### Option 1: Sync Configuration from ML110-01
|
||
|
|
Since Proxmox cluster filesystem should sync `/etc/pve/ceph.conf`, we need to ensure:
|
||
|
|
1. ML110-01 has the correct configuration
|
||
|
|
2. Configuration syncs to r630-01
|
||
|
|
3. Then add r630-01 monitor
|
||
|
|
|
||
|
|
### Option 2: Reinitialize Ceph Cluster
|
||
|
|
If the cluster on ml110-01 is not operational or has issues:
|
||
|
|
1. Stop Ceph on ml110-01
|
||
|
|
2. Reinitialize with proper configuration
|
||
|
|
3. Add r630-01 monitor
|
||
|
|
4. Add OSDs
|
||
|
|
|
||
|
|
### Option 3: Use Proxmox Web UI
|
||
|
|
The Proxmox web interface may handle the configuration and monitor addition more reliably.
|
||
|
|
|
||
|
|
## Current Status
|
||
|
|
|
||
|
|
- ML110-01: Monitor running but cluster commands timeout
|
||
|
|
- R630-01: Cannot connect to cluster, configuration mismatch
|
||
|
|
- Need to resolve configuration before proceeding
|
||
|
|
|