Closes the 'JWT hygiene' gap identified by the review:
- 24h TTL was used for every track, including Track 4 operator sessions
carrying operator.write.* permissions.
- Tokens had no server-side revocation path; rotating JWT_SECRET was
the only way to invalidate a session, which would punt every user.
- Tokens carried no jti, so individual revocation was impossible even
with a revocations table.
Changes:
Migration 0016_jwt_revocations (up + down):
- CREATE TABLE jwt_revocations (jti PK, address, track,
token_expires_at, revoked_at, reason) plus indexes on address and
token_expires_at. Append-only; idempotent on duplicate jti.
backend/auth/wallet_auth.go:
- tokenTTLs map: track 1 = 12h, 2 = 8h, 3 = 4h, 4 = 60m. tokenTTLFor
returns the ceiling; default is 12h for unknown tracks.
- generateJWT now embeds a 128-bit random jti (hex-encoded) and uses
the per-track TTL instead of a hardcoded 24h.
- parseJWT: shared signature-verification + claim-extraction helper
used by ValidateJWT and RefreshJWT. Returns address, track, jti, exp.
- jtiFromToken: parses jti from an already-trusted token without a
second crypto roundtrip.
- isJTIRevoked: EXISTS query against jwt_revocations, returning
ErrJWTRevocationStorageMissing when the table is absent (migration
not run yet) so callers can surface a 503 rather than silently
treating every token as valid.
- RevokeJWT(ctx, token, reason): records the jti; idempotent via
ON CONFLICT (jti) DO NOTHING. Refuses legacy tokens without jti.
- RefreshJWT(ctx, token): validates, revokes the old token (reason
'refresh'), and mints a new token with fresh jti + fresh TTL. Same
(address, track) as the inbound token, same permissions set.
- ValidateJWT now consults jwt_revocations when a DB is configured;
returns ErrJWTRevoked for revoked tokens.
backend/api/rest/auth_refresh.go (new):
- POST /api/v1/auth/refresh handler: expects 'Authorization: Bearer
<jwt>'; returns WalletAuthResponse with the new token. Maps
ErrJWTRevoked to 401 token_revoked and ErrWalletAuthStorageNotInitialized
to 503.
- POST /api/v1/auth/logout handler: same header contract, idempotent,
returns {status: ok}. Returns 503 when the revocations table
isn't present so ops know migration 0016 hasn't run.
- Both handlers reuse the existing extractBearerToken helper from
auth.go so parsing is consistent with the rest of the access layer.
backend/api/rest/routes.go:
- Registered /api/v1/auth/refresh and /api/v1/auth/logout.
Tests:
- TestTokenTTLForTrack4IsShort: track 4 TTL <= 1h.
- TestTokenTTLForTrack1Track2Track3AreReasonable: bounded at 12h.
- TestGeneratedJWTCarriesJTIClaim: jti is present, 128 bits / 32 hex.
- TestGeneratedJWTExpIsTrackAppropriate: exp matches tokenTTLFor per
track within a couple-second tolerance.
- TestRevokeJWTWithoutDBReturnsError: a WalletAuth with nil db must
refuse to revoke rather than silently pretending it worked.
- All pre-existing wallet_auth tests still pass.
Also fixes a small SA4006/SA4017 regression in mission_control.go that
PR #5 introduced by shadowing the outer err with json.Unmarshal's err
return. Reworked to uerr so the outer err and the RPC fallback still
function as intended.
Verification:
go build ./... clean
go vet ./... clean
go test ./auth/... PASS (including new tests)
go test ./api/rest/... PASS
staticcheck ./auth/... ./api/rest/... clean on SA4006/SA4017/SA1029
Advances completion criterion 3 (JWT hygiene): 'Track 4 sessions TTL
<= 1h; server-side revocation list (keyed on jti) enforced on every
token validation; refresh endpoint rotates the token in place so the
short TTL is usable in practice; logout endpoint revokes immediately.'
The Chain 138 RPC access product catalog (core-rpc / alltra-rpc /
thirdweb-rpc, each with VMID + HTTP/WS URL + tier + billing model + use
cases + management features) used to be a hardcoded 50-line Go literal
in api/rest/auth.go. The review flagged this as the biggest source of
'magic constants in source' in the backend: changing a partner URL, a
VMID, or a billing model required a Go recompile, and the internal
192.168.11.x CIDR endpoints were baked into the binary.
This PR moves the catalog to backend/config/rpc_products.yaml and adds
a lazy loader so every call site reads from the YAML on first use.
New files:
backend/config/rpc_products.yaml source of truth
backend/api/rest/rpc_products_config.go loader + fallback defaults
backend/api/rest/rpc_products_config_test.go unit tests
Loader path-resolution order (first hit wins):
1. $RPC_PRODUCTS_PATH (absolute or cwd-relative)
2. $EXPLORER_BACKEND_DIR/config/rpc_products.yaml
3. <cwd>/backend/config/rpc_products.yaml
4. <cwd>/config/rpc_products.yaml
5. compiled-in defaultRPCAccessProducts fallback (logs a WARNING)
Validation on load:
- every product must have a non-empty slug,
- every product must have a non-empty http_url,
- slugs must be unique across the catalog.
A malformed YAML causes a WARNING + fallback to defaults, never a
silent empty product list.
Call-site changes in auth.go:
- 'var rpcAccessProducts []accessProduct' (literal) -> func
rpcAccessProducts() []accessProduct (forwards to the lazy loader).
- Both existing consumers (/api/v1/access/products handler at line
~369 and findAccessProduct() at line ~627) now call the function.
Zero other behavioural changes; the JSON shape of the response is
byte-identical.
Tests added:
- TestLoadRPCAccessProductsFromRepoDefault: confirms the shipped
YAML loads, produces >=3 products, and contains the 3 expected
slugs with non-empty http_url.
- TestLoadRPCAccessProductsRejectsDuplicateSlug.
- TestLoadRPCAccessProductsRejectsMissingHTTPURL.
Verification:
go build ./... clean
go vet ./... clean
go test ./api/rest/ PASS (new + existing)
go mod tidy pulled yaml.v3 from indirect to direct
Advances completion criterion 7 (no magic constants): 'Chain 138
access products / VMIDs / provider URLs live in a YAML that operators
can change without a rebuild; internal CIDRs are no longer required
to be present in source.'
Decomposes backend/api/rest/ai.go (which the review flagged at 1180 lines
and which was the largest file in the repo by a wide margin) into six
purpose-built files inside the same package, so no import paths change
for any caller and *Server receivers keep working:
ai.go 198 handlers + feature flags + exported AI* DTOs
ai_context.go 381 buildAIContext + indexed-DB queries
(stats / tx / address / block) + regex patterns +
extractBlockReference
ai_routes.go 139 queryAIRoutes + filterAIRouteMatches +
routeMatchesQuery + normalizeHexString
ai_docs.go 136 loadAIDocSnippets + findAIWorkspaceRoot +
scanDocForTerms + buildDocSearchTerms
ai_xai.go 267 xAI / OpenAI request/response types +
normalizeAIMessages + latestUserMessage +
callXAIChatCompletions + parseXAIError +
extractOutputText
ai_helpers.go 112 pure-function utilities (firstRegexMatch,
compactStringMap, compactAnyMap, stringValue,
stringSliceValue, uniqueStrings, clipString,
fileExists)
ai_runtime.go (rate limiter + metrics + audit log) is unchanged.
This is a pure move: no logic changes, no new public API, no changes to
HTTP routes. Each file carries only the imports it actually uses so
goimports is clean on every file individually. Every exported symbol
retained its original spelling so callers (routes.go, server.go, and
the AI e2e tests) keep compiling without edits.
Verification:
go build ./... clean
go vet ./... clean
go test ./api/rest/... PASS
staticcheck ./... clean on the SA* correctness family
Advances completion criterion 6 (backend maintainability): 'no single
Go file exceeds a few hundred lines; AI/LLM plumbing is separated from
HTTP handlers; context-building is separated from upstream calls.'
.github/workflows/ci.yml:
- Go version: 1.22 -> 1.23.4 (matches go.mod's 'go 1.23.0' declaration).
- Split into four jobs with explicit names:
* test-backend: go vet + go build + go test
* scan-backend: staticcheck + govulncheck (installed from pinned tags)
* test-frontend: npm ci + eslint + tsc --noEmit + next build
* gitleaks: full-history secret scan on every PR
- Branches triggered: master + main + develop (master is the repo
default; the previous workflow only triggered on main/develop and
would never have run on the repo's actual PRs).
- actions/checkout@v4, actions/setup-go@v5, actions/setup-node@v4.
- Concurrency group cancels stale runs on the same ref.
- Node and Go caches enabled for faster CI.
.gitleaks.toml (new):
- Extends gitleaks defaults.
- Custom rule 'explorer-legacy-db-password-L@ker' keeps the historical
password pattern L@kers?\$?2010 wedged in the detection set even
after rotation, so any re-introduction (via copy-paste from old
branches, stale docs, etc.) fails CI.
- Allowlists docs/SECURITY.md and CHANGELOG.md where the string is
cited in rotation context.
backend/staticcheck.conf (new):
- Enables the full SA* correctness set.
- Temporarily disables ST1000/1003/1005/1020/1021/1022, U1000, S1016,
S1031. These are stylistic/cosmetic checks; the project has a long
tail of pre-existing hits there that would bloat every PR. Each is
commented so the disable can be reverted in a dedicated cleanup.
Legit correctness issues surfaced by staticcheck and fixed in this PR:
- backend/analytics/token_distribution.go: 'best-effort MV refresh'
block no longer dereferences a shadowed 'err'; scope-tight 'if err :='
used for the subsequent QueryRow.
- backend/api/rest/middleware.go: compressionMiddleware() was parsing
Accept-Encoding and doing nothing with it. Now it's a literal
pass-through with a TODO comment pointing at gorilla/handlers.
- backend/api/rest/mission_control.go: shadowed 'err' from
json.Unmarshal was assigned to an ignored outer binding via
fmt.Errorf; replaced with a scoped 'if uerr :=' that lets the RPC
fallback run as intended.
- backend/indexer/traces/tracer.go: best-effort CREATE TABLE no longer
discards the error implicitly.
- backend/indexer/track2/block_indexer.go: 'latestBlock - uint64(i) >= 0'
was a tautology on uint64. Replaced with an explicit
'if uint64(i) > latestBlock { break }' guard so operators running
count=1000 against a shallow chain don't underflow.
- backend/tracing/tracer.go: introduces a local ctxKey type and two
constants so WithValue calls stop tripping SA1029.
Verification:
- go build ./... clean.
- go vet ./... clean.
- go test ./... all existing tests PASS.
- staticcheck ./... clean except for the SA1029 hits in
api/middleware/auth.go and api/track4/operator_scripts_test.go,
which are resolved by PR #4 once it merges to master.
Advances completion criterion 4 (CI in good health).
backend/api/middleware/context.go (new):
- Introduces an unexported ctxKey type and three constants
(ctxKeyUserAddress, ctxKeyUserTrack, ctxKeyAuthenticated) that
replace the bare string keys 'user_address', 'user_track', and
'authenticated'. Bare strings trigger go vet's SA1029 and collide
with keys from any other package that happens to share the name.
- Helpers: ContextWithAuth, UserAddress, UserTrack, IsAuthenticated.
- Sentinel: ErrMissingAuthorization replaces the misuse of
http.ErrMissingFile as an auth-missing signal. (http.ErrMissingFile
belongs to multipart form parsing and was semantically wrong.)
backend/api/middleware/auth.go:
- RequireAuth, OptionalAuth, RequireTrack now all read/write via the
helpers; no more string literals for context keys in this file.
- extractAuth returns ErrMissingAuthorization instead of
http.ErrMissingFile.
- Dropped now-unused 'context' import.
backend/api/track4/operator_scripts.go, backend/api/track4/endpoints.go,
backend/api/rest/features.go:
- Read user address / track via middleware.UserAddress() and
middleware.UserTrack() instead of a raw context lookup with a bare
string key.
- Import 'github.com/explorer/backend/api/middleware'.
backend/api/track4/operator_scripts_test.go:
- Four test fixtures updated to seed the request context through
middleware.ContextWithAuth (track 4, authenticated) instead of
context.WithValue with a bare 'user_address' string. This is the
load-bearing change that proves typed keys are required: a bare
string key no longer wakes up the middleware helpers.
backend/api/middleware/context_test.go (new):
- Round-trip test for ContextWithAuth + UserAddress + UserTrack +
IsAuthenticated.
- Defaults: UserTrack=1, UserAddress="", IsAuthenticated=false on a
bare context.
- TestContextKeyIsolation: an outside caller that inserts
'user_address' as a bare string key must NOT be visible to
UserAddress; proves the type discipline.
- ErrMissingAuthorization sentinel smoke test.
Verification:
- go build ./... clean.
- go vet ./... clean (removes SA1029 on the old bare keys).
- go test ./api/middleware/... ./api/track4/... ./api/rest/... PASS.
Advances completion criterion 3 (Auth correctness).
backend/api/rest/server.go:
- NewServer() now delegates to loadJWTSecret(), which:
- Rejects JWT_SECRET < 32 bytes (log.Fatal).
- Requires JWT_SECRET when APP_ENV=production or GO_ENV=production.
- Generates a 32-byte crypto/rand ephemeral secret in dev only.
- Treats rand.Read failure as fatal (removes the prior time-based
fallback that was deterministic and forgeable).
- Default Content-Security-Policy rewritten:
- Drops 'unsafe-inline' and 'unsafe-eval'.
- Drops private CIDRs (192.168.11.221:854[5|6]).
- Adds frame-ancestors 'none', base-uri 'self', form-action 'self'.
- CSP_HEADER is required in production; fatal if unset there.
backend/api/rest/server_security_test.go (new):
- Covers the three loadJWTSecret() paths (valid, whitespace-trimmed,
ephemeral in dev).
- Covers isProductionEnv() across APP_ENV / GO_ENV combinations.
- Asserts defaultDevCSP contains no unsafe directives or private CIDRs
and includes the frame-ancestors / base-uri / form-action directives.
scripts/*.sh:
- Removed '***REDACTED-LEGACY-PW***' default value from SSH_PASSWORD / NEW_PASSWORD in
7 helper scripts. Each script now fails with exit 2 and points to
docs/SECURITY.md if the password isn't supplied via env or argv.
EXECUTE_DEPLOYMENT.sh, EXECUTE_NOW.sh:
- Replaced hardcoded DB_PASSWORD='***REDACTED-LEGACY-PW***' with a ':?' guard that
aborts with a clear error if DB_PASSWORD (and, for EXECUTE_DEPLOYMENT,
RPC_URL) is not exported. Other env vars keep sensible non-secret
defaults via ${VAR:-default}.
README.md:
- Removed the hardcoded Database Password / RPC URL lines. Replaced with
an env-variable reference table pointing at docs/SECURITY.md and
docs/DATABASE_CONNECTION_GUIDE.md.
docs/DEPLOYMENT.md:
- Replaced 'PASSWORD: SSH password (default: ***REDACTED-LEGACY-PW***)' with a
required-no-default contract and a link to docs/SECURITY.md.
docs/SECURITY.md (new):
- Full secret inventory keyed to the env variable name and the file that
consumes it.
- Five-step rotation checklist covering the Postgres role, the Proxmox
VM SSH password, JWT_SECRET, vendor API keys, and a gitleaks-based
history audit.
- Explicit note that merging secret-scrub PRs does NOT invalidate
already-leaked credentials; rotation is the operator's responsibility.
Verification:
- go build ./... + go vet ./... pass clean.
- Targeted tests (LoadJWTSecret*, IsProduction*, DefaultDevCSP*) pass.
Advances completion criterion 2 (Secrets & config hardened). Residual
leakage from START_HERE.md / LETSENCRYPT_CONFIGURATION_GUIDE.md is
handled by PR #2 (doc consolidation), which deletes those files.
- Remove committed Go binaries:
backend/bin/api-server (~18 MB)
backend/cmd (~18 MB)
backend/api/rest/cmd/api-server (~18 MB)
- Remove scratch / build output dirs from the repo:
out/, cache/, test-results/
- Extend .gitignore to cover these paths plus playwright-report/
and coverage/ so they don't drift back in.
Total artifact weight removed: ~54 MB of binaries + small scratch files.
- Introduced a new Diagnostics struct to capture transaction visibility state and activity state.
- Updated BuildSnapshot function to return diagnostics alongside snapshot, completeness, and sampling.
- Enhanced test cases to validate the new diagnostics data.
- Updated frontend components to utilize the new diagnostics information for improved user feedback on freshness context.
This change improves the observability of transaction activity and enhances the user experience by providing clearer insights into the freshness of data.
- Updated branding from "SolaceScanScout" to "Solace" across various files including deployment scripts, API responses, and documentation.
- Changed default base URL for Playwright tests and updated security headers to reflect the new branding.
- Enhanced README and API documentation to include new authentication endpoints and product access details.
This refactor aligns the project branding and improves clarity in the API documentation.
