Sankofa/docs/ceph/WIPE_ALL_DRIVES.md

# Wipe All 6x 250GB Drives

**Date**: 2025-12-13  
**Status**: ✅ **READY TO EXECUTE**

---

## Overview

Wipe all 6x 250GB drives (sdc, sdd, sde, sdf, sdg, sdh) to prepare them for Ceph OSDs.

**WARNING**: This will destroy ALL data on these drives!

---

## Option 1: Wipe Only (Then Create OSDs Manually)

### Script: `wipe-all-250gb-drives.sh`

This script will:
1. Remove drives from ubuntu-vg
2. Wipe all 6 drives
3. Leave them ready for OSD creation

**Usage**:
```bash
# Copy to R630-01
scp scripts/wipe-all-250gb-drives.sh root@192.168.11.11:/tmp/

# SSH and run
ssh root@192.168.11.11
bash /tmp/wipe-all-250gb-drives.sh
```

**After wiping**, create OSDs manually:
```bash
# Create third OSD (minimum needed)
ceph-volume lvm create --data /dev/sdc

# Or create OSDs on all drives (for better performance)
ceph-volume lvm create --data /dev/sdc
ceph-volume lvm create --data /dev/sdd
ceph-volume lvm create --data /dev/sde
ceph-volume lvm create --data /dev/sdf
ceph-volume lvm create --data /dev/sdg
ceph-volume lvm create --data /dev/sdh
```

---

## Option 2: Wipe AND Create OSDs (Automated)

### Script: `wipe-and-create-osds.sh`

This script will:
1. Remove drives from ubuntu-vg
2. Wipe all 6 drives
3. Create Ceph OSDs on all 6 drives automatically

**Usage**:
```bash
# Copy to R630-01
scp scripts/wipe-and-create-osds.sh root@192.168.11.11:/tmp/

# SSH and run
ssh root@192.168.11.11
bash /tmp/wipe-and-create-osds.sh
```

**This is the recommended option** - it does everything in one go!

---

## Manual Method

If you prefer to do it manually:

```bash
# 1. Remove from ubuntu-vg
umount /dev/ubuntu-vg/* 2>/dev/null || true
vgchange -a n ubuntu-vg
for drive in sdc sdd sde sdf sdg sdh; do
    pvremove /dev/${drive}3 -y -ff 2>/dev/null || true
done

# 2. Wipe all drives
for drive in sdc sdd sde sdf sdg sdh; do
    umount /dev/${drive}* 2>/dev/null || true
    wipefs -a /dev/$drive
    dd if=/dev/zero of=/dev/$drive bs=1M count=100
done

# 3. Create OSDs
for drive in sdc sdd sde sdf sdg sdh; do
    ceph-volume lvm create --data /dev/$drive
done

# 4. Verify
ceph osd tree
ceph health
```

---

## Expected Results

### Before
- 2 OSDs (insufficient)
- HEALTH_WARN: TOO_FEW_OSDS

### After (with all 6 OSDs)
- **8 OSDs total** (2 existing + 6 new)
- Excellent redundancy and performance
- HEALTH_OK or much improved
- Can handle many more VMs

### After (with just 1 OSD)
- **3 OSDs total** (minimum for 3-way replication)
- Fixes TOO_FEW_OSDS error
- HEALTH_OK or improved
- VMs can be created

---

## Recommendations

### Minimum (Fix Current Issue)
- Create OSD on **1 drive** (e.g., sdc)
- Fixes TOO_FEW_OSDS error
- Allows VM creation

### Recommended (Better Performance)
- Create OSDs on **all 6 drives**
- Much better performance
- Better data distribution
- More redundancy

### Best Practice
- Use all available drives for OSDs
- Better utilization of hardware
- Improved Ceph performance

---

## Safety Checklist

Before running:

- [ ] **Backup important data** (if ubuntu-vg contains data)
- [ ] **Verify ubuntu-vg is not critical** for system operation
- [ ] **Confirm all 6 drives** are the ones you want to wipe
- [ ] **Have recovery plan** if something goes wrong
- [ ] **Type 'yes' to confirm** when script asks

---

## Troubleshooting

### Issue: pvremove fails

**Solution**:
```bash
# Force remove
vgchange -a n ubuntu-vg
pvremove /dev/sdX3 -y -ff
```

### Issue: wipefs fails

**Solution**:
```bash
# Unmount first
umount /dev/sdX* 2>/dev/null || true
# Then wipe
wipefs -a /dev/sdX
```

### Issue: OSD creation fails

**Solution**:
```bash
# Make sure drive is completely wiped
wipefs -a /dev/sdX
dd if=/dev/zero of=/dev/sdX bs=1M count=100

# Check Ceph status
ceph health
ceph osd tree

# Try again
ceph-volume lvm create --data /dev/sdX
```

---

## Summary

### Current State
- 6x 250GB drives in ubuntu-vg
- Need to be wiped and prepared for Ceph

### Action
- Run `wipe-and-create-osds.sh` (recommended)
- Or run `wipe-all-250gb-drives.sh` then create OSDs manually

### Expected Result
- 8 OSDs total (if all 6 created)
- Or 3 OSDs minimum (if just 1 created)
- Ceph health improves significantly
- VMs can be created with ceph-fs storage

---

**Last Updated**: 2025-12-13  
**Status**: ✅ **READY TO EXECUTE**