Sankofa/docs/ceph/FREE_DRIVE_FROM_UBUNTU_VG.md

# Free Drive from ubuntu-vg for Ceph OSD

**Date**: 2025-12-13  
**Status**: 🔍 **250GB DRIVES FOUND IN ubuntu-vg VOLUME GROUP**

---

## Current Situation

The verification revealed:
- ✅ **6x 250GB drives ARE detected** (sdc, sde, sdh, and others)
- ⚠️ **They're in `ubuntu-vg` volume group** (100% allocated)
- ✅ **User confirmed**: 2 drives need to be formatted
- 🎯 **Goal**: Free up one drive for third Ceph OSD

---

## Step-by-Step: Free Drive from ubuntu-vg

### Step 1: Check Current Status

Run the check script on R630-01:

```bash
bash /tmp/check-ubuntu-vg-and-prepare-osd.sh
```

Or manually:

```bash
# Check ubuntu-vg status
vgs ubuntu-vg
lvs ubuntu-vg
pvs | grep ubuntu-vg

# Check what's mounted
mount | grep ubuntu-vg
df -h | grep ubuntu-vg
```

### Step 2: Identify Drive to Free

From the screenshot, drives in ubuntu-vg include:
- `/dev/sdc3` (247.90 GB)
- `/dev/sde3` (247.90 GB)
- `/dev/sdh3` (247.90 GB)
- At least one more (partially visible)

**Choose one drive** that's safe to remove (preferably one that's not critical).

### Step 3: Unmount and Remove from ubuntu-vg

**WARNING**: This will destroy data on the logical volume!

```bash
# 1. Unmount any mounted logical volumes (if any)
umount /dev/ubuntu-vg/* 2>/dev/null || true

# 2. Deactivate the volume group
vgchange -a n ubuntu-vg

# 3. Remove the physical volume from ubuntu-vg
# Replace sdX with your chosen drive (e.g., sdc, sde, sdh)
pvremove /dev/sdX3

# 4. If you want to remove the entire ubuntu-vg (if not needed):
# vgremove ubuntu-vg
```

### Step 4: Wipe the Drive

**WARNING**: This destroys all data on the drive!

```bash
# Wipe filesystem signatures
wipefs -a /dev/sdX

# Optionally, zero out the beginning (for clean slate)
dd if=/dev/zero of=/dev/sdX bs=1M count=100
```

### Step 5: Create Ceph OSD

```bash
# Create Ceph OSD on the freed drive
# Replace sdX with your drive (e.g., sdc, sde, sdh)
ceph-volume lvm create --data /dev/sdX
```

### Step 6: Verify OSD Created

```bash
# Check OSD tree (should show 3 OSDs now)
ceph osd tree

# Check Ceph health (should improve)
ceph health
ceph health detail
```

---

## Alternative: If ubuntu-vg is Needed

If `ubuntu-vg` contains important data:

### Option 1: Backup and Remove One Drive

1. **Backup data** from ubuntu-vg
2. **Reduce ubuntu-vg** to remove one drive
3. **Use freed drive** for Ceph OSD

### Option 2: Use Different Drive

If 2 drives need formatting (as user mentioned):
- Use one of those 2 drives for Ceph OSD
- Format the other as needed

### Option 3: Shrink ubuntu-vg

1. **Shrink logical volume** in ubuntu-vg
2. **Remove one physical volume** from ubuntu-vg
3. **Use freed drive** for Ceph OSD

---

## Quick Commands Reference

```bash
# Check ubuntu-vg
vgs ubuntu-vg
lvs ubuntu-vg
pvs | grep ubuntu-vg

# Check what's using it
mount | grep ubuntu-vg
lsof | grep ubuntu-vg

# Remove drive from ubuntu-vg (WARNING: Destroys data!)
vgchange -a n ubuntu-vg
pvremove /dev/sdX3

# Wipe drive
wipefs -a /dev/sdX

# Create Ceph OSD
ceph-volume lvm create --data /dev/sdX

# Verify
ceph osd tree
ceph health
```

---

## Safety Checklist

Before removing a drive from ubuntu-vg:

- [ ] **Backup any important data** (if ubuntu-vg contains data)
- [ ] **Verify ubuntu-vg is not critical** for system operation
- [ ] **Check if logical volumes are mounted** (unmount first)
- [ ] **Confirm which drive to remove** (choose carefully)
- [ ] **Have recovery plan** if something goes wrong

---

## Expected Results

### Before
- 2 OSDs (insufficient for 3-way replication)
- HEALTH_WARN: TOO_FEW_OSDS

### After
- 3 OSDs (sufficient for 3-way replication)
- HEALTH_OK or improved (TOO_FEW_OSDS warning gone)
- VMs can be created with ceph-fs storage

---

## Summary

### Current State
- ✅ 6x 250GB drives found in ubuntu-vg
- ⚠️ All drives 100% allocated
- 🎯 Need to free one drive for Ceph OSD

### Action Required
1. Check ubuntu-vg status
2. Identify safe drive to remove
3. Remove from ubuntu-vg (if safe)
4. Wipe and create Ceph OSD
5. Verify Ceph health improves

---

**Last Updated**: 2025-12-13  
**Status**: 🔍 **READY TO FREE DRIVE FROM ubuntu-vg**