cb47cce074
- Organized 252 files across project - Root directory: 187 → 2 files (98.9% reduction) - Moved configuration guides to docs/04-configuration/ - Moved troubleshooting guides to docs/09-troubleshooting/ - Moved quick start guides to docs/01-getting-started/ - Moved reports to reports/ directory - Archived temporary files - Generated comprehensive reports and documentation - Created maintenance scripts and guides All files organized according to established standards.
200 lines
5.1 KiB
Markdown
200 lines
5.1 KiB
Markdown
# Storage Migration Fix - Implementation Summary
|
|
|
|
**Date**: $(date)
|
|
**Status**: ✅ Scripts Created and Ready
|
|
|
|
## Problem
|
|
|
|
Container migrations failing with "migration aborted" errors:
|
|
- Containers: 1504, 2503, 2504, 6201
|
|
- Target: pve node
|
|
- Error: Migration aborted for all containers
|
|
|
|
## Solution Implemented
|
|
|
|
### Scripts Created
|
|
|
|
1. **`scripts/diagnose-and-fix-migration-storage.sh`**
|
|
- Comprehensive diagnostic tool
|
|
- Checks all nodes (ml110, pve, pve2)
|
|
- Finds container locations
|
|
- Checks storage configuration
|
|
- Automatically fixes issues
|
|
- Performs migrations
|
|
|
|
2. **`scripts/fix-migration-storage.sh`**
|
|
- Focused storage fix tool
|
|
- Ensures `thin1` storage is available on target nodes
|
|
- Creates thin pools if needed
|
|
- Configures storage for Proxmox
|
|
|
|
### Documentation Created
|
|
|
|
1. **`docs/MIGRATION_STORAGE_FIX.md`**
|
|
- Complete guide for fixing migration issues
|
|
- Step-by-step instructions
|
|
- Troubleshooting section
|
|
- Multiple migration options
|
|
|
|
2. **`docs/MIGRATION_QUICK_REFERENCE.md`**
|
|
- Quick reference for common tasks
|
|
- Fast commands for immediate fixes
|
|
|
|
## Usage
|
|
|
|
### Quick Start (Recommended)
|
|
|
|
```bash
|
|
cd /home/intlc/projects/proxmox
|
|
./scripts/diagnose-and-fix-migration-storage.sh
|
|
```
|
|
|
|
This will automatically:
|
|
1. Diagnose all nodes
|
|
2. Find container locations
|
|
3. Fix storage issues
|
|
4. Migrate containers to pve
|
|
|
|
### Step-by-Step
|
|
|
|
If you prefer to fix storage first:
|
|
|
|
```bash
|
|
# Step 1: Fix storage configuration
|
|
./scripts/fix-migration-storage.sh
|
|
|
|
# Step 2: Run migration
|
|
./scripts/diagnose-and-fix-migration-storage.sh
|
|
```
|
|
|
|
## Dependencies
|
|
|
|
The scripts require:
|
|
- `sshpass` - For password-based SSH authentication
|
|
- `jq` - For JSON parsing (usually pre-installed on Proxmox)
|
|
- `bash` - Version 4.0 or higher
|
|
|
|
### Install Dependencies (if needed)
|
|
|
|
```bash
|
|
# On Debian/Ubuntu
|
|
sudo apt-get update
|
|
sudo apt-get install -y sshpass jq
|
|
|
|
# On Proxmox (usually already installed)
|
|
# Check with: which sshpass jq
|
|
```
|
|
|
|
## Container Information
|
|
|
|
| VMID | Hostname | Current Location | Target Location |
|
|
|------|-----------|------------------|-----------------|
|
|
| 1504 | besu-sentry-ali | ml110 | pve |
|
|
| 2503 | besu-rpc-ali-0x8a | ml110 | pve |
|
|
| 2504 | besu-rpc-ali-0x1 | ml110 | pve |
|
|
| 6201 | firefly-ali-1 | ml110 | pve |
|
|
|
|
## Node Information
|
|
|
|
| Node | IP Address | Password | Purpose |
|
|
|------|------------|----------|---------|
|
|
| ml110 | 192.168.11.10 | L@kers2010 | Source node |
|
|
| pve | 192.168.11.11 | password | Target node |
|
|
| pve2 | 192.168.11.12 | password | Alternative target |
|
|
|
|
## Expected Behavior
|
|
|
|
### Diagnostic Phase
|
|
- Checks connectivity to all nodes
|
|
- Lists storage status on each node
|
|
- Shows volume groups
|
|
- Finds container locations
|
|
- Displays container storage information
|
|
|
|
### Fix Phase
|
|
- Ensures `thin1` storage exists on target nodes
|
|
- Creates thin pools if needed
|
|
- Configures storage for Proxmox
|
|
- Verifies storage is active
|
|
|
|
### Migration Phase
|
|
- Stops containers if running
|
|
- Migrates containers to target node
|
|
- Waits for migration to complete
|
|
- Verifies final locations
|
|
|
|
## Troubleshooting
|
|
|
|
### Script Fails to Connect
|
|
|
|
**Issue**: Cannot connect to nodes
|
|
**Solution**:
|
|
- Verify network connectivity
|
|
- Check IP addresses are correct
|
|
- Verify passwords in script match actual passwords
|
|
|
|
### Storage Creation Fails
|
|
|
|
**Issue**: Cannot create thin pools
|
|
**Solution**:
|
|
- Check available disk space: `vgs`
|
|
- Verify volume groups exist
|
|
- Check LVM is properly configured
|
|
|
|
### Migration Still Fails
|
|
|
|
**Issue**: Containers still fail to migrate after storage fix
|
|
**Solution**:
|
|
- Try manual migration (see `MIGRATION_STORAGE_FIX.md`)
|
|
- Use `local` storage instead of `thin1`
|
|
- Consider backup/restore method
|
|
|
|
## Verification
|
|
|
|
After running scripts, verify success:
|
|
|
|
```bash
|
|
# Check container locations
|
|
ssh root@192.168.11.10 "for vmid in 1504 2503 2504 6201; do echo -n \"\$vmid: \"; pvesh get /nodes/pve/lxc/\$vmid/status/current 2>/dev/null | jq -r '.status' && echo \" (pve)\" || echo \"not on pve\"; done"
|
|
|
|
# Check storage
|
|
ssh root@192.168.11.11 "pvesm status | grep thin1"
|
|
```
|
|
|
|
## Next Steps
|
|
|
|
1. ✅ Run diagnostic script to understand current state
|
|
2. ✅ Fix storage if needed
|
|
3. ✅ Migrate containers
|
|
4. ⏳ Verify all containers are on pve
|
|
5. ⏳ Start containers if needed
|
|
6. ⏳ Verify container functionality
|
|
7. ⏳ Update documentation
|
|
|
|
## Files Modified/Created
|
|
|
|
### Scripts
|
|
- ✅ `scripts/diagnose-and-fix-migration-storage.sh` (new)
|
|
- ✅ `scripts/fix-migration-storage.sh` (new)
|
|
|
|
### Documentation
|
|
- ✅ `docs/MIGRATION_STORAGE_FIX.md` (new)
|
|
- ✅ `docs/MIGRATION_QUICK_REFERENCE.md` (new)
|
|
- ✅ `docs/STORAGE_MIGRATION_FIX_SUMMARY.md` (this file)
|
|
|
|
## Notes
|
|
|
|
- Scripts use non-interactive mode by default (set `NON_INTERACTIVE=1` or run without TTY)
|
|
- All scripts include error handling and logging
|
|
- Scripts can be run multiple times safely (idempotent)
|
|
- Storage fix script will not destroy existing data
|
|
- Migration script will stop containers before migration
|
|
|
|
## Support
|
|
|
|
For detailed troubleshooting, see:
|
|
- `docs/MIGRATION_STORAGE_FIX.md` - Complete guide
|
|
- `docs/MIGRATION_QUICK_REFERENCE.md` - Quick commands
|
|
- `docs/STORAGE_MIGRATION_ISSUE.md` - Original issue documentation
|
|
|