**Completion status (in-repo):** All libs are present under `backend/libs/` and `frontend/libs/`. Backend is wired to use **go-pgconfig** (API + indexer), **go-rpc-gateway** (Track 1). Frontend is wired to use **frontend-api-client** (services/api/client) and **frontend-ui-primitives** (all pages using Card, Table, Address). CI uses `submodules: recursive`; README documents clone with submodules. To publish as separate repos, copy each lib to its own repo and add as submodule.
**Review and test:** Backend handlers that need the DB use `requireDB(w)`; without a DB they return 503. Tests run with a nil DB and accept 200/503/404 as appropriate. Run backend tests: `go test ./...` in `backend/`. Frontend build: `npm run build` in `frontend/` (ESLint uses root `.eslintrc.cjs` and frontend `"root": true` in `.eslintrc.json`). E2E: `npm run e2e` from repo root (Playwright, default base URL https://explorer.d-bis.org; set `EXPLORER_URL` for local).
---
**Goal:** Extract reusable, non–Explorer-specific components into their own repositories under `/home/intlc/projects`, and link them back to this monorepo via **git submodules** using best practices.
**Scope:** Components that are generic (EVM/chain-agnostic, multi-tenant, or generic UI/infra) are candidates. Explorer-specific logic (Chain 138 routes, Blockscout integration, explorer branding) stays in `explorer-monorepo`.
| **Docs** | — | All current docs are explorer/deployment-specific |
---
## 2. Proposed New Repositories (under `/home/intlc/projects`)
All new repos live as siblings of `proxmox/explorer-monorepo`, e.g.:
-`/home/intlc/projects/go-bridge-aggregator`
-`/home/intlc/projects/go-chain-adapters`
-`/home/intlc/projects/go-pgconfig`
-`/home/intlc/projects/go-tiered-auth`
-`/home/intlc/projects/go-http-middleware`
-`/home/intlc/projects/go-logging`
-`/home/intlc/projects/go-rpc-gateway`
-`/home/intlc/projects/frontend-ui-primitives`
-`/home/intlc/projects/deployment-common`
Relationship to monorepo: **submodules** inside `explorer-monorepo` (e.g. `backend/vendor/go-bridge-aggregator` or `libs/go-bridge-aggregator`), so the monorepo stays the single checkout for development and CI.
- **Reusability:** Provider interface, `BridgeRequest`/`BridgeQuote`/`BridgeStep`, `Aggregator`, `GetBestQuote`. Chain IDs and integrator names should be config (env or struct), not hardcoded 138.
- **Suggested layout:** `go.mod` (module e.g. `github.com/yourorg/go-bridge-aggregator`), `provider.go`, `aggregator.go`, `providers/` (one file per provider). Dependencies: minimal (HTTP client, context).
- **Explorer coupling:** Explorer will depend on this module via Go module (or submodule + replace). CCIP “138 <-> 1” and integrator strings become config in explorer.
- **Explorer coupling:** Explorer `api/rest/cmd/main.go` and indexer use DB config; import this module.
---
### 3.4 `go-tiered-auth`
- **Purpose:** Tiered (track) access: wallet-based auth (nonce + JWT), optional “tier” level, and feature access by tier.
- **Source:** `backend/auth/` (auth.go, roles.go, wallet_auth.go), `backend/featureflags/flags.go`, and the auth/track parts of `backend/api/middleware/auth.go`.
- **Reusability:** Wallet nonce + JWT and “tier” (track) are generic; feature names in `featureflags` can be default map, overridable by consumer. Middleware uses `auth` + `featureflags`; can live in this repo or in `go-http-middleware` with dependency on this repo.
- **Suggested layout:** `go.mod`, `auth/` (wallet_auth, nonce storage interface), `featureflags/` (HasAccess, GetEnabledFeatures, configurable feature map), `middleware/` (RequireAuth, RequireTier, OptionalAuth). DB tables (e.g. `wallet_nonces`, `operator_roles`) stay defined in explorer migrations; this repo only needs interfaces (e.g. NonceStore, TierResolver).
- **Explorer coupling:** Explorer implements storage (existing DB); middleware and WalletAuth stay in explorer but can call into this lib for validation and JWT. Alternatively, move full WalletAuth + middleware here and pass DB and feature map from explorer.
---
### 3.5 `go-http-middleware`
- **Purpose:** Generic HTTP middleware: security headers (CSP, X-Frame-Options, etc.), CORS, compression, logging. No explorer URLs.
- **Source:** `backend/api/middleware/security.go` (generic headers only; CSP `connect-src` etc. are explorer-specific and should be supplied by explorer or config).
- **Reusability:** Security headers and CORS are generic; CSP should be a parameter or callback.
- **Suggested layout:** `go.mod`, `security.go` (AddSecurityHeaders with configurable CSP), `cors.go` if extracted, `logging.go` if request logging is extracted. Explorer passes CSP string when wiring middleware.
- **Explorer coupling:** Explorer server uses this middleware with explorer-specific CSP.
- **Suggested layout:** `go.mod`, `logger.go`. Optional: integrate with a standard logger interface (e.g. slog) for compatibility.
- **Explorer coupling:** Optional; explorer can keep using it or switch to stdlib/slog and deprecate this in monorepo.
---
### 3.7 `go-rpc-gateway`
- **Purpose:** Generic RPC gateway: in-memory (and optionally Redis) cache, in-memory (and optionally Redis) rate limiter, and a generic HTTP proxy to an upstream RPC URL. No route definitions or explorer semantics.
- **Suggested layout:** `go.mod`, `cache/` (memory + Redis), `ratelimit/` (memory + Redis), `gateway.go` (proxy to single upstream). Explorer track1 endpoints call this gateway and format responses (e.g. chain_id 138).
- **Explorer coupling:** Explorer imports gateway + cache + ratelimit; track1 endpoints remain in explorer and use this package.
---
### 3.8 `frontend-ui-primitives`
- **Purpose:** Reusable React UI components: Button, Card, Table. Optionally a minimal Address (truncate + copy) without explorer-specific links.
- **Suggested layout:** npm package (e.g. `@yourorg/ui-primitives`). Structure: `src/Button.tsx`, `Card.tsx`, `Table.tsx`, `Address.tsx`; export from `index.ts`; build with tsup or similar; peer deps: `react`, `clsx`. Tailwind: consumer app includes Tailwind and uses same class names, or package ships minimal CSS.
- **Explorer coupling:** Explorer frontend adds dependency (npm package or git submodule + workspace). If submodule: e.g. `frontend/libs/ui-primitives` and in package.json `"@yourorg/ui-primitives": "file:./libs/frontend-ui-primitives"`.
---
### 3.9 `frontend-api-client` (optional)
- **Purpose:** Generic API client: axios instance, interceptors, optional API key from storage, typed `ApiResponse<T>` and error shape.
- **Source:** `frontend/src/services/api/client.ts` (and possibly a minimal `types.ts` for ApiResponse/ApiError).
- **Reusability:** No explorer endpoints; base URL and headers are config.
- **Suggested layout:** npm package exporting `createApiClient(baseURL, options?)` and types. Explorer uses `createApiClient(process.env.NEXT_PUBLIC_API_URL)` and keeps explorer-specific API modules (blocks, addresses, transactions) in monorepo.
- **Explorer coupling:** Explorer depends on this package; explorer-specific services import client from package and define endpoints locally.
---
### 3.10 `deployment-common`
- **Purpose:** Reusable deployment and ops snippets: nginx (generic reverse proxy, SSL, `location /api/` template), systemd unit templates, Cloudflare Tunnel sample config, fail2ban jail/config, and small generic “verify” scripts (e.g. curl health, check port).
- **Source:** `deployment/`– extract generic parts of `setup-nginx.sh`, `systemd/*.service`, `cloudflare/tunnel-config.yml`, `fail2ban/`. Omit explorer-specific env (VMID 5000, Blockscout, explorer.d-bis.org) or make them placeholders.
- **Reusability:** No explorer branding or VMIDs; only patterns (proxy to backend, systemd, tunnel, fail2ban).
- **Suggested layout:** `nginx/`, `systemd/`, `cloudflare/`, `fail2ban/`, `scripts/` (e.g. verify-http-endpoint.sh). README with usage and variable placeholders.
- **Explorer coupling:** Explorer’s `deployment/` can include this repo as submodule (e.g. `deployment/common`) and copy or symlink templates, or reference them from docs.
---
## 4. Submodule Strategy (Best Practices)
- **Location of submodules:** Prefer a single directory for “vendored” or “lib” submodules to avoid clutter, e.g.:
- **Option A:** `explorer-monorepo/libs/` with one submodule per repo (`libs/go-bridge-aggregator`, `libs/frontend-ui-primitives`, etc.).
- **Option B:** Backend libs under `backend/libs/` and frontend under `frontend/libs/` (so Go and Node resolve paths naturally).
- **Recommendation:** Use **Option B** for simpler Go/Node resolution:
- **Publishing:** When a reusable repo is stable, publish it (Go: tag and push to GitHub; npm: publish to npm or private registry). Explorer can then depend on versioned releases and only use submodules for active development or private forks.
---
## 5. Phased Extraction Order
To minimize breakage and respect dependencies:
1.**Phase 1 – No dependency on other extracted pieces**
-`go-rpc-gateway` (cache + rate limit + gateway; no dependency on auth)
-`go-http-middleware` (security headers only)
-`go-tiered-auth` (auth + featureflags + optional middleware; depends on DB interface, so explorer keeps DB)
-`frontend-api-client` (optional)
3.**Phase 3 – Integration in explorer**
- Replace internal packages with submodule or published module references.
- Update explorer `backend/go.mod` and `frontend/package.json`.
- Keep explorer-specific code (routes, indexer, Blockscout, SPA) in monorepo; ensure tests and CI pass.
---
## 6. What Stays in Explorer Monorepo
- **Backend:** All REST route handlers, track* endpoint logic, indexer (listener + processor + backfill), Blockscout/etherscan compatibility, explorer-specific config (chain_id 138, RPC URLs), and migrations (schema stays here; libs use interfaces or config).
- **Frontend:** All pages and views, `public/index.html` SPA, explorer API service modules (blocks, transactions, addresses), Next.js app and deployment config.
- **Deployment:** All explorer- and VMID 5000–specific scripts (`fix-502-blockscout.sh`, `complete-explorer-api-access.sh`, legacy `deploy-frontend-to-vmid5000.sh`, etc.), and nginx/config that reference explorer.d-bis.org and Blockscout.
- **Use a consistent base URL** (SSH or HTTPS) for all submodules so clones work without reconfiguring.
- **Prefer `branch = main`** (or your default branch) so `git submodule update --remote` pulls the right branch; for releases you can pin by committing the submodule at a specific tag.
- **Clone explorer-monorepo with submodules (first time):**
This plan keeps Explorer-specific behavior in the monorepo while moving generic building blocks into separate, reusable repos linked via submodules and optional published modules.
