Files
proxmox/docs/archive/deployment-reports/STORAGE_CLEANUP_COMPLETE.md
defiQUG fbda1b4beb
Some checks failed
Deploy to Phoenix / deploy (push) Has been cancelled
docs: Ledger Live integration, contract deploy learnings, NEXT_STEPS updates
- ADD_CHAIN138_TO_LEDGER_LIVE: Ledger form done; public code review repo bis-innovations/LedgerLive; init/push commands
- CONTRACT_DEPLOYMENT_RUNBOOK: Chain 138 gas price 1 gwei, 36-addr check, TransactionMirror workaround
- CONTRACT_*: AddressMapper, MirrorManager deployed 2026-02-12; 36-address on-chain check
- NEXT_STEPS_FOR_YOU: Ledger done; steps completable now (no LAN); run-completable-tasks-from-anywhere
- MASTER_INDEX, OPERATOR_OPTIONAL, SMART_CONTRACTS_INVENTORY_SIMPLE: updates
- LEDGER_BLOCKCHAIN_INTEGRATION_COMPLETE: bis-innovations/LedgerLive reference

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-12 15:46:57 -08:00

2.6 KiB

Storage Cleanup Complete

Date: 2026-01-18
Action: Cleaned up systemd journal logs in Blockscout container (VMID 5000)
Status: SUCCESS


Cleanup Results

Disk Usage Before Cleanup:

  • Used: 7.4GB (4%)
  • Available: 179GB
  • Journal Size: ~4.0GB

Disk Usage After Cleanup:

  • Used: 3.6GB (2%)
  • Available: 183GB
  • Journal Size: <100MB (limited by SystemMaxUse)
  • Freed Space: 3.8GB

Actions Completed

  1. Cleaned up archived journal logs

    • Deleted 31 archived journal files (each 128MB)
    • Freed 3.8GB of space
    • Kept only active journal (limited to 100MB)
  2. Configured journal limits

    • Added SystemMaxUse=100M to /etc/systemd/journald.conf
    • Restarted systemd-journald service
    • Prevents future journal bloat

Storage Pool Impact

thin2 Pool Status:

  • Before: 88.78% used (210.5 GB / 237.1 GB)
  • Expected After: ~86.9% used (~206.7 GB / 237.1 GB)
  • Freed: ~3.8GB
  • Status: ⚠️ IMPROVED - Still monitoring recommended

Remaining Space:

  • Before: 26.6 GB available
  • After: ~30.4 GB available
  • Improvement: +3.8GB freed

Verification

To verify cleanup and monitor future usage:

# Check journal size
ssh root@192.168.11.12 'pct exec 5000 -- journalctl --disk-usage'

# Check disk usage
ssh root@192.168.11.12 'pct exec 5000 -- df -h /'

# Check storage pool
ssh root@192.168.11.12 'pvesm status | grep thin2'

Future Maintenance

Automated Cleanup (Optional)

To prevent future journal bloat, you can set up a cron job:

# Add to crontab on Proxmox host
# Clean up journal logs weekly (keep last 100MB)
0 2 * * 0 pct exec 5000 -- journalctl --vacuum-size=100M > /dev/null 2>&1

Monitor Journal Size

Check periodically:

ssh root@192.168.11.12 'pct exec 5000 -- journalctl --disk-usage'

Should show: < 100MB (configured limit)


Summary

Cleanup Successful

  • Freed 3.8GB of space
  • Configured journal limits to prevent future issues
  • Improved thin2 pool availability from 26.6GB to ~30.4GB

⚠️ Note: thin2 pool is still at ~87% usage, but the cleanup has provided more breathing room. Continue monitoring and consider additional cleanup if needed.


Next Steps (Optional)

  1. Monitor thin2 pool usage over the next few days
  2. Review other containers on thin2 for potential cleanup
  3. Consider redistributing containers to other pools if needed
  4. Set up alerts for storage pool usage > 85%

Status: Cleanup complete, monitoring recommended