Sankofa/docs/ceph/DRIVE_VERIFICATION_RESULTS.md

# Drive Verification Results - R630-01

**Date**: 2025-12-13  
**Status**: 🔍 **6x 250GB DRIVES NOT DETECTED BY OS**

---

## Verification Results

### Disks Detected by OS
- **sda**: 279.4 GB (300GB) - Proxmox installation (LVM)
- **sdb**: 279.4 GB (300GB) - Ceph OSD

### Expected Disks
- **6x 250GB drives** - **NOT DETECTED** ❌

---

## Analysis

The 6x 250GB drives are **not visible to the operating system**. This means they are likely:

1. **In PERC Controller but Unconfigured** (Most Likely)
   - Drives are physically installed
   - Detected by PERC controller
   - Not configured as virtual disks
   - Not exposed to OS

2. **In RAID Array**
   - Drives are part of a RAID virtual disk
   - Not exposed as individual disks
   - Need to reconfigure to expose individually

3. **Not Physically Installed** (Less Likely)
   - User confirmed 8 total bays with drives
   - But OS only sees 2 drives
   - May need physical verification

---

## Current Ceph Status

- **OSD Count**: 2 (insufficient for 3-way replication)
- **Health**: HEALTH_WARN with multiple issues:
  - TOO_FEW_OSDS: OSD count 2 < 3 required
  - Too many PGs per OSD (288 > max 250)
  - Degraded data redundancy
  - Slow operations

---

## Next Steps

### Immediate: Check PERC Controller

The drives are likely in the PERC controller but not exposed. Check:

1. **Install Dell OpenManage** (if not installed):
   ```bash
   apt-get install -y srvadmin-all
   systemctl start dsm_sa_datamgrd
   ```

2. **Check Physical Disks in PERC**:
   ```bash
   omreport storage pdisk controller=0
   ```

3. **Check Virtual Disks**:
   ```bash
   omreport storage vdisk controller=0
   ```

4. **Check Controller Status**:
   ```bash
   omreport storage controller
   ```

### If Drives Found in PERC

**Option A: Configure as Individual Virtual Disks** (Recommended for Ceph)
- Create one virtual disk per physical disk
- Expose each 250GB drive individually
- Use for Ceph OSDs

**Option B: Use PERC HBA Mode** (If Supported)
- Configure PERC in HBA (Host Bus Adapter) mode
- Passes drives directly to OS
- Best for Ceph (no RAID overhead)

### If Drives Not Found in PERC

1. **Physical Verification**:
   - Check drive bay LEDs
   - Verify drives are seated properly
   - Check backplane connections

2. **Controller Issues**:
   - Check PERC controller status
   - May need firmware update
   - May need controller replacement

---

## Commands to Run on R630-01

```bash
# 1. Check for PERC controller
lspci | grep -iE "raid|perc|lsi|megaraid"

# 2. Check kernel messages
dmesg | grep -iE "sd[a-z]|disk|perc" | tail -20

# 3. Check SCSI devices
cat /proc/scsi/scsi

# 4. Check if OpenManage available
which omreport
which perccli

# 5. If OpenManage installed
omreport storage pdisk controller=0
omreport storage vdisk controller=0
```

---

## Expected PERC Output

If drives are in PERC, you should see:

```
Physical Disk Information
-------------------------
ID    Status    State    Name    Size
0:0:0 Online    Ready    ...     300 GB
0:0:1 Online    Ready    ...     300 GB
0:0:2 Online    Ready    ...     250 GB  ← Should see 6 of these
0:0:3 Online    Ready    ...     250 GB
0:0:4 Online    Ready    ...     250 GB
0:0:5 Online    Ready    ...     250 GB
0:0:6 Online    Ready    ...     250 GB
0:0:7 Online    Ready    ...     250 GB
```

---

## Summary

### Current State
- ✅ 2x 300GB drives detected and in use
- ❌ 6x 250GB drives **NOT detected by OS**
- 🔍 Need to check PERC controller configuration

### Action Required
1. **Check PERC controller** for 6x 250GB drives
2. **Configure drives** to expose to OS (if found)
3. **Create Ceph OSD** on one of the 250GB drives
4. **Fix Ceph health** issues

### Priority
🔴 **URGENT** - Need to expose 250GB drives to create third OSD

---

**Last Updated**: 2025-12-13  
**Status**: 🔍 **CHECK PERC CONTROLLER FOR 250GB DRIVES**