proxmox/docs/04-configuration/SECRET_USAGE_PATTERNS.md

# Secret Usage Patterns Documentation

**Last Updated:** 2026-01-31  
**Document Version:** 1.0  
**Status:** Active Documentation

---

**Date:** 2025-01-27  
**Status:** 📋 Documentation Complete  
**Purpose:** Document how secrets are currently used across the codebase

---

## Overview

This document tracks how secrets are accessed and used throughout the codebase, helping identify all locations that need to be updated during HSM Key Vault migration.

---

## Secret Access Patterns

### 1. Direct File Reading

#### Pattern: Reading from .env files
```bash
# Shell scripts
source .env
export $(cat .env | xargs)

# Node.js
require('dotenv').config()
process.env.PRIVATE_KEY

# Python
from dotenv import load_dotenv
load_dotenv()
os.getenv('PRIVATE_KEY')
```

**Locations:**
- `scripts/*.sh` - Multiple shell scripts
- `smom-dbis-138/scripts/*.ts` - TypeScript deployment scripts
- `services/*/` - Service applications

**Migration:** Replace with Vault API calls or Vault Agent

---

### 2. Hardcoded in Scripts

#### Pattern: Secrets directly in code
```bash
# Example from scripts
NPM_PASSWORD="ce8219e321e1cd97bd590fb792d3caeb7e2e3b94ca7e20124acaf253f911ff72"
CLOUDFLARE_API_TOKEN="JSEO_sruWB6lf1id77gtI7HOLVdhkhaR2goPEJIk"
```

**Locations:**
- `scripts/create-npmplus-proxy.sh`
- `scripts/fix-certbot-dns-propagation.sh`
- `scripts/install-shared-tunnel-token.sh`
- `scripts/nginx-proxy-manager/*.sh`

**Migration:** Replace with Vault secret retrieval

---

### 3. Environment Variable Injection

#### Pattern: Using environment variables
```bash
# Scripts
PRIVATE_KEY="${PRIVATE_KEY:-default_value}"
CLOUDFLARE_TOKEN="${CLOUDFLARE_API_TOKEN:-}"

# Applications
const privateKey = process.env.PRIVATE_KEY;
const apiToken = process.env.CLOUDFLARE_API_TOKEN;
```

**Locations:**
- All deployment scripts
- Service applications
- Frontend build processes

**Migration:** Vault Agent can inject as environment variables

---

### 4. Configuration Files

#### Pattern: Secrets in config files
```yaml
# docker-compose.yml
environment:
  - PRIVATE_KEY=${PRIVATE_KEY}
  - DATABASE_URL=${DATABASE_URL}

# Kubernetes secrets
apiVersion: v1
kind: Secret
data:
  private-key: <base64>
```

**Locations:**
- `docker-compose/*.yml`
- Kubernetes manifests (if any)
- Terraform configurations

**Migration:** Use Vault Kubernetes integration or external secrets operator

---

## Service-Specific Patterns

### Blockchain Services

**Services:**
- `smom-dbis-138/`
- `no_five/`
- `237-combo/`

**Secrets Used:**
- `PRIVATE_KEY` - For contract deployment and transactions
- `RPC_URL` - Blockchain RPC endpoint
- Contract addresses (less sensitive)

**Access Pattern:**
```typescript
// Foundry scripts
const privateKey = process.env.PRIVATE_KEY;
const deployer = new ethers.Wallet(privateKey, provider);

// Hardhat scripts
const accounts = await ethers.getSigners();
const deployer = accounts[0];
```

**Migration Strategy:**
- Store private key in HSM (never export)
- Use Vault Agent to inject as env var
- Or use Vault API with short-lived tokens

---

### Cloudflare Integration

**Services:**
- DNS automation scripts
- SSL certificate management
- Tunnel configuration

**Secrets Used:**
- `CLOUDFLARE_API_TOKEN` - API access
- `CLOUDFLARE_TUNNEL_TOKEN` - Tunnel authentication
- `CLOUDFLARE_ORIGIN_CA_KEY` - Origin CA

**Access Pattern:**
```bash
# Shell scripts
curl -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
     https://api.cloudflare.com/client/v4/zones

# Python scripts
import requests
headers = {"Authorization": f"Bearer {os.getenv('CLOUDFLARE_API_TOKEN')}"}
```

**Migration Strategy:**
- Store tokens in Vault
- Use Vault Agent for scripts
- Rotate tokens quarterly

---

### Database Services

**Services:**
- `dbis_core/`
- `explorer-monorepo/`

**Secrets Used:**
- `DATABASE_URL` - Connection string with password
- `POSTGRES_PASSWORD` - Database password
- `DB_USER` - Database username

**Access Pattern:**
```javascript
// Node.js
const db = new Client({
  connectionString: process.env.DATABASE_URL
});

// Python
import psycopg2
conn = psycopg2.connect(os.getenv('DATABASE_URL'))
```

**Migration Strategy:**
- Store connection string in Vault
- Or store components separately (user, password, host)
- Use Vault database secrets engine for dynamic credentials

---

### Infrastructure Services

**Services:**
- Nginx Proxy Manager (NPMplus)
- UniFi Controller
- Omada Controller

**Secrets Used:**
- `NPM_PASSWORD` - NPM admin password
- `NPM_EMAIL` - NPM admin email
- `UNIFI_API_KEY` - UniFi API key
- `UNIFI_PASSWORD` - UniFi password
- `OMADA_API_KEY` - Omada API key

**Access Pattern:**
```bash
# NPM API
curl -X POST "$NPM_URL/api/tokens" \
  -H "Content-Type: application/json" \
  -d "{\"identity\":\"$NPM_EMAIL\",\"secret\":\"$NPM_PASSWORD\"}"

# UniFi API
curl -X POST "$UNIFI_URL/api/login" \
  -d "{\"username\":\"$UNIFI_USER\",\"password\":\"$UNIFI_PASSWORD\"}"
```

**Migration Strategy:**
- Store credentials in Vault
- Use Vault Agent for automation scripts
- Implement credential rotation

---

## Application Integration Points

### Frontend Applications

**Services:**
- `frontend-dapp/`
- `dbis_core/frontend/`

**Secrets Used:**
- `VITE_ETHERSCAN_API_KEY` - Public API key (less sensitive)
- `VITE_WALLETCONNECT_PROJECT_ID` - Public identifier

**Access Pattern:**
```typescript
// Vite environment variables (public)
const apiKey = import.meta.env.VITE_ETHERSCAN_API_KEY;
```

**Note:** Vite variables prefixed with `VITE_` are exposed to the browser. Only use for public API keys.

**Migration Strategy:**
- Keep public keys in .env (less sensitive)
- Or use Vault for consistency
- Never expose private keys to frontend

---

### Backend Services

**Services:**
- `services/relay/`
- `services/state-anchoring-service/`
- `services/transaction-mirroring-service/`

**Secrets Used:**
- `PRIVATE_KEY` - For blockchain operations
- `DATABASE_URL` - Database connection
- `JWT_SECRET` - Token signing

**Access Pattern:**
```typescript
// Node.js services
import dotenv from 'dotenv';
dotenv.config();

const privateKey = process.env.PRIVATE_KEY;
const dbUrl = process.env.DATABASE_URL;
```

**Migration Strategy:**
- Use Vault Agent for automatic injection
- Or Vault API with service account authentication
- Implement secret rotation

---

## Migration Checklist by Pattern

### Direct File Reading
- [ ] Identify all `source .env` or `load_dotenv()` calls
- [ ] Replace with Vault Agent or API calls
- [ ] Test secret retrieval
- [ ] Update documentation

### Hardcoded Secrets
- [ ] Find all hardcoded secrets in scripts
- [ ] Move to Vault
- [ ] Update scripts to retrieve from Vault
- [ ] Remove hardcoded values

### Environment Variables
- [ ] Identify all `process.env.*` or `$VAR` usage
- [ ] Configure Vault Agent templates
- [ ] Test environment injection
- [ ] Verify application functionality

### Configuration Files
- [ ] Review docker-compose.yml files
- [ ] Review Kubernetes manifests
- [ ] Update to use Vault secrets
- [ ] Test deployment

---

## Vault Integration Patterns

### Pattern 1: Vault Agent (Recommended for Applications)

**Use Case:** Long-running services that need secrets

```hcl
# vault-agent.hcl
template {
  source      = "/etc/secrets/.env.tpl"
  destination = "/etc/secrets/.env"
  perms       = 0600
}
```

**Template:**
```bash
PRIVATE_KEY={{ with secret "secret/data/blockchain/private-keys/deployer" }}{{ .Data.data.private_key }}{{ end }}
```

---

### Pattern 2: Vault API (For Scripts)

**Use Case:** One-time scripts, automation

```bash
#!/bin/bash
PRIVATE_KEY=$(vault kv get -field=private_key secret/blockchain/private-keys/deployer)
CLOUDFLARE_TOKEN=$(vault kv get -field=token secret/cloudflare/api-tokens/main)

# Use secrets
cast send ... --private-key "$PRIVATE_KEY"
```

---

### Pattern 3: Vault CLI with Caching

**Use Case:** Development, local scripts

```bash
# Authenticate once
vault auth -method=userpass username=dev

# Use cached token
export PRIVATE_KEY=$(vault kv get -field=private_key secret/blockchain/private-keys/deployer)
```

---

### Pattern 4: Kubernetes Secrets Operator

**Use Case:** Kubernetes deployments

```yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: blockchain-secrets
spec:
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: blockchain-secrets
  data:
    - secretKey: private-key
      remoteRef:
        key: secret/data/blockchain/private-keys/deployer
        property: private_key
```

---

## Testing Strategy

### Pre-Migration Testing
1. Document current secret usage
2. Identify all access points
3. Test Vault connectivity
4. Create test secrets in Vault

### Migration Testing
1. Migrate one secret at a time
2. Test application functionality
3. Verify no hardcoded fallbacks
4. Check logs for errors

### Post-Migration Testing
1. Verify all secrets in Vault
2. Test secret rotation
3. Verify access controls
4. Security audit

---

## Related Documentation

- [Master Secrets Inventory](MASTER_SECRETS_INVENTORY.md)
- [Secrets Migration Summary](SECRETS_MIGRATION_SUMMARY.md)
- [Secrets Quick Reference](SECRETS_QUICK_REFERENCE.md)

---

**Last Updated:** 2025-01-27  
**Status:** 📋 Documentation Complete  
**Next Review:** During migration implementation