97 lines
2.2 KiB
Markdown
97 lines
2.2 KiB
Markdown
|
# Authentication Error Fix - "invalid PVE ticket"
|
||
|
|
|
||
|
|
**Date**: 2025-12-13
|
||
|
|
**Status**: 🔧 **FIXING**
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Issue
|
||
|
|
|
||
|
|
### Symptom
|
||
|
|
- **Error**: "401 permission denied - invalid PVE ticket"
|
||
|
|
- **Frequency**: Intermittent to continuous
|
||
|
|
- **Impact**: **BLOCKS ALL VM CREATION**
|
||
|
|
- **Affected**: Both ML110-01 and R630-01 nodes
|
||
|
|
|
||
|
|
### Error Details
|
||
|
|
```
|
||
|
|
GET /nodes/r630-01/status failed: 401 permission denied - invalid PVE ticket
|
||
|
|
node r630-01 is not reachable or unhealthy
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Root Cause Analysis
|
||
|
|
|
||
|
|
### Previous Fix Applied
|
||
|
|
- Changed from `req.AddCookie()` to `req.Header.Set("Cookie", ...)`
|
||
|
|
- Reason: `AddCookie()` automatically URL-encodes values, breaking token format
|
||
|
|
|
||
|
|
### Current Status
|
||
|
|
- Code fix is in place ✅
|
||
|
|
- Provider rebuilt and restarted ✅
|
||
|
|
- But errors still occurring ⚠️
|
||
|
|
|
||
|
|
### Possible Causes
|
||
|
|
1. **Provider not using latest code** - Pod may be using old image
|
||
|
|
2. **Token format issue** - Token may not be in correct format
|
||
|
|
3. **Token validity** - Token may be expired or invalid
|
||
|
|
4. **Cookie header format** - May need different format
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Investigation Steps
|
||
|
|
|
||
|
|
### 1. Verify Token Validity
|
||
|
|
- Test token manually with curl
|
||
|
|
- Verify token works with both endpoints
|
||
|
|
- Check token format matches Proxmox expectations
|
||
|
|
|
||
|
|
### 2. Verify Code Fix
|
||
|
|
- Check if Cookie header fix is in code
|
||
|
|
- Verify provider is using latest image
|
||
|
|
- Ensure provider pod restarted with new image
|
||
|
|
|
||
|
|
### 3. Test Authentication
|
||
|
|
- Test token with exact format from controller
|
||
|
|
- Verify Cookie header is set correctly
|
||
|
|
- Check HTTP request format
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Fix Applied
|
||
|
|
|
||
|
|
### Code Changes
|
||
|
|
- **File**: `crossplane-provider-proxmox/pkg/proxmox/http_client.go`
|
||
|
|
- **Change**: `req.AddCookie()` → `req.Header.Set("Cookie", fmt.Sprintf("PVEAuthCookie=%s", c.token))`
|
||
|
|
|
||
|
|
### Provider Rebuild
|
||
|
|
- Rebuilt provider image
|
||
|
|
- Loaded into kind cluster
|
||
|
|
- Restarted provider pod
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Verification
|
||
|
|
|
||
|
|
### After Fix
|
||
|
|
- Check authentication errors (should be 0)
|
||
|
|
- Verify node health checks pass
|
||
|
|
- Monitor VM creation progress
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Next Steps
|
||
|
|
|
||
|
|
1. ✅ Verify token is valid
|
||
|
|
2. ✅ Test token manually
|
||
|
|
3. ✅ Rebuild provider with latest code
|
||
|
|
4. ✅ Restart provider pod
|
||
|
|
5. ⏳ Monitor for resolution
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
**Last Updated**: 2025-12-13
|
||
|
|
**Status**: 🔧 **FIXING - VERIFYING RESULTS**
|
||
|
|
|