86 lines
2.2 KiB
Markdown
86 lines
2.2 KiB
Markdown
|
# VM Deployment Monitoring Summary
|
||
|
|
|
||
|
|
**Date**: 2025-12-13
|
||
|
|
**Status**: 🟡 **MONITORING IN PROGRESS - AUTHENTICATION ISSUE DETECTED**
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Current Status
|
||
|
|
|
||
|
|
### Provider
|
||
|
|
- **Status**: Running (1/1 Ready)
|
||
|
|
- **Image**: crossplane-provider-proxmox:latest
|
||
|
|
- **Token Auth**: Detected in logs ("Using token authentication")
|
||
|
|
- **Issue**: Still getting "invalid PVE ticket" errors
|
||
|
|
|
||
|
|
### VMs
|
||
|
|
- **Total**: 30
|
||
|
|
- **Deployed in K8s**: 30
|
||
|
|
- **Created on Proxmox**: 0 (blocked by authentication errors)
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Issue Identified
|
||
|
|
|
||
|
|
### Authentication Errors
|
||
|
|
**Symptom**: "401 permission denied - invalid PVE ticket"
|
||
|
|
**Frequency**: All node health checks failing
|
||
|
|
**Impact**: VM creation blocked - cannot proceed past health check
|
||
|
|
|
||
|
|
### Root Cause Analysis
|
||
|
|
1. Token authentication is being detected in logs ✅
|
||
|
|
2. Token format appears correct: `tokenid=token` ✅
|
||
|
|
3. Cookie header implementation is correct ✅
|
||
|
|
4. But API calls still fail with "invalid PVE ticket" ❌
|
||
|
|
|
||
|
|
### Possible Causes
|
||
|
|
1. Token might be expired or invalid
|
||
|
|
2. Token permissions might be insufficient
|
||
|
|
3. Token format might need adjustment
|
||
|
|
4. Provider might not be using latest code (unlikely - pod restarted)
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Monitoring Commands
|
||
|
|
|
||
|
|
### Real-Time Monitoring
|
||
|
|
```bash
|
||
|
|
# Watch all VMs
|
||
|
|
kubectl get proxmoxvm -A -w
|
||
|
|
|
||
|
|
# Continuous monitor
|
||
|
|
./scripts/continuous-monitor.sh 30
|
||
|
|
|
||
|
|
# View provider logs
|
||
|
|
kubectl logs -n crossplane-system -l app=crossplane-provider-proxmox -f
|
||
|
|
```
|
||
|
|
|
||
|
|
### Check Authentication Status
|
||
|
|
```bash
|
||
|
|
# Count authentication errors
|
||
|
|
kubectl logs -n crossplane-system -l app=crossplane-provider-proxmox --since=5m | grep -i "invalid PVE ticket" | wc -l
|
||
|
|
|
||
|
|
# Check token authentication usage
|
||
|
|
kubectl logs -n crossplane-system -l app=crossplane-provider-proxmox --since=5m | grep "Using token authentication" | wc -l
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Next Steps
|
||
|
|
|
||
|
|
1. ⚠️ **URGENT**: Fix token authentication issue
|
||
|
|
- Verify token is valid and not expired
|
||
|
|
- Check token permissions
|
||
|
|
- Test token manually with curl
|
||
|
|
- Verify token format is correct
|
||
|
|
|
||
|
|
2. Monitor VM creation once authentication is fixed
|
||
|
|
3. Verify VMs are being created on Proxmox
|
||
|
|
4. Post-deployment verification
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
**Last Updated**: 2025-12-13
|
||
|
|
**Status**: 🟡 **MONITORING - AUTHENTICATION ISSUE BLOCKING DEPLOYMENT**
|
||
|
|
|