proxmox/dbis_chain_138_technical_master_plan.md

# DBIS Chain 138 Technical Master Plan

## Purpose
This document is the governance and execution baseline for DBIS Chain 138 infrastructure. It is intentionally grounded in repo-backed and operator-verified reality, so it can be used for audits, deployment planning, and readiness decisions without confusing `currently deployed`, `under validation`, and `future-state` work.

The objective is to move from architecture theory to a production-grade sovereign deployment program that is evidence-based, phased, and operationally auditable.

---

# SECTION 1 — MASTER OBJECTIVES

## Primary objectives

1. Inventory currently installed stack components and host placement.
2. Validate actual service readiness, not just declared architecture.
3. Standardize Proxmox VE deployment topology and preferred workload placement.
4. Assign infrastructure ownership across ecosystem entities once governance is finalized.
5. Define production-grade deployment and verification workflows.
6. Track the gap between today’s footprint and sovereign target-state architecture.
7. Produce auditable artifacts that operators can regenerate and maintain.

---

# SECTION 2 — CURRENT STACK STATUS

## Deployed now

- Hyperledger Besu (QBFT, Chain 138)
- Hyperledger Fabric containers and VMIDs are allocated
- Hyperledger Indy containers and VMIDs are allocated
- Hyperledger FireFly primary container footprint exists
- Blockscout / explorer stack
- Hyperledger Caliper hook and performance guidance (documentation only)

## Partially deployed / under validation

- Hyperledger FireFly:
  - primary `6200` is restored as a minimal local FireFly API footprint
  - secondary `6201` is present in inventory but currently behaves like a retired / standby shell with no valid deployment payload
- Hyperledger Fabric:
  - `6000`, `6001`, `6002` are present in inventory but are now intentionally stopped as reserved placeholders
  - current app-level verification did not show active Fabric peer / orderer workloads or meaningful Fabric payloads inside those CTs
- Hyperledger Indy:
  - `6400`, `6401`, `6402` are present in inventory but are now intentionally stopped as reserved placeholders
  - current app-level verification did not show active Indy node listeners or meaningful Indy payloads inside those CTs

## Planned / aspirational

- Hyperledger Aries as a proven deployed service tier
- Hyperledger AnonCreds as an operationally verified deployed layer
- Hyperledger Ursa as a required runtime dependency
- Hyperledger Quilt
- Hyperledger Avalon
- Hyperledger Cacti as a proven live interoperability layer
- Full multi-region sovereignized Proxmox with Ceph-backed storage and segmented production VLANs

---

# SECTION 3 — CURRENT ENVIRONMENT DISCOVERY

## Canonical discovery artifacts

The source-of-truth discovery path for current state is:

- [docs/02-architecture/DBIS_NODE_ROLE_MATRIX.md](docs/02-architecture/DBIS_NODE_ROLE_MATRIX.md)
- [docs/03-deployment/PHASE1_DISCOVERY_RUNBOOK.md](docs/03-deployment/PHASE1_DISCOVERY_RUNBOOK.md)
- [docs/03-deployment/DBIS_HYPERLEDGER_RUNTIME_STATUS.md](docs/03-deployment/DBIS_HYPERLEDGER_RUNTIME_STATUS.md)
- [scripts/verify/run-phase1-discovery.sh](scripts/verify/run-phase1-discovery.sh)
- [config/proxmox-operational-template.json](config/proxmox-operational-template.json)
- [docs/04-configuration/ALL_VMIDS_ENDPOINTS.md](docs/04-configuration/ALL_VMIDS_ENDPOINTS.md)
- [docs/02-architecture/PHYSICAL_HARDWARE_INVENTORY.md](docs/02-architecture/PHYSICAL_HARDWARE_INVENTORY.md)

## Discovery scope

Reality mapping must validate:

1. Proxmox hosts and cluster health
2. VMID / CT inventory versus template JSON
3. Besu validators, sentries, and RPC tiers
4. Explorer and public RPC availability
5. Hyperledger CT presence and app-level readiness where possible
6. Storage topology and current backing stores
7. Network topology and current LAN / VLAN reality
8. ML110 role reality versus migration plan

## Required outputs

Every discovery run should produce:

- Infrastructure inventory report
- Service state map
- Dependency context
- Critical failure summary

The markdown report is evidence capture; the script exit code is the pass/fail signal.

---

# SECTION 4 — PROXMOX VE DEPLOYMENT DESIGN

## Current state

- Current cluster footprint is smaller than the target sovereign model.
- Current storage is primarily local ZFS / LVM-based, not Ceph-backed distributed storage.
- Current workload placement is represented as `preferred host` in the planning template, not guaranteed live placement.

## Target model

- Multi-node Proxmox VE cluster with stable quorum
- HA-aware workload placement
- Dedicated roles for core compute, RPC exposure, identity/workflow DLT, ingress, and future storage tiers

## Current interpretation rule

This plan must not describe the target sovereignized Proxmox model as already achieved. All references to HA, Ceph, dedicated storage nodes, or dedicated network nodes are roadmap items unless Phase 1 evidence proves they are already active.

---

# SECTION 5 — NETWORK ARCHITECTURE

## Current network reality

- Primary active management / services LAN is `192.168.11.0/24`
- Public ingress is fronted through NPMplus / edge services
- RPC exposure is already tiered across core, public, private, named, and thirdweb-facing nodes

## Target network layers

1. Management network
2. Storage replication network
3. Blockchain validator / P2P network
4. Identity / workflow DLT network
5. Public access / DMZ network
6. Validator-only restricted paths

## Status

- Public access and RPC role separation exist in practice.
- Full sovereign segmentation with dedicated VLANs and zero-trust internal routing remains roadmap work.

---

# SECTION 6 — ENTITY ASSIGNMENT MODEL

## Governance model

The entity-assignment model remains valid as a target governance structure:

- DBIS Core Authority
- Central Banks
- International Financial Institutions
- Regional Operators

## Current status

- Entity ownership for many deployed workloads is still `TBD` in the operational matrix.
- Until governance assigns final owners, operator documentation must keep those fields explicitly marked as `TBD` rather than inventing ownership.

The executable placement artifact is:

- [docs/02-architecture/DBIS_NODE_ROLE_MATRIX.md](docs/02-architecture/DBIS_NODE_ROLE_MATRIX.md)

---

# SECTION 7 — VM AND CONTAINER DESIGN

## Current status by workload family

### Deployed now

- Settlement / Besu VM family
- Explorer / observability family
- Ingress / proxy family
- Application and DBIS-support workloads

### Partially deployed / under validation

- Workflow VM / CT family for FireFly
- Institutional VM / CT family for Fabric
- Identity VM / CT family for Indy

### Planned / aspirational

- Identity VM template that includes proven Aries + AnonCreds runtime
- Interoperability VM template for true Hyperledger Cacti usage

## Implementation rule

Template language in this plan must map to actual repo artifacts and actual VMIDs, not hypothetical inventory.

---

# SECTION 8 — STORAGE ARCHITECTURE

## Current state

- Current guest storage is backed by local Proxmox storage pools.
- Ceph-backed distributed storage is not yet an achieved platform baseline.

## Target state

- Ceph or equivalent distributed storage tier
- Snapshot-aware backup strategy by workload class
- Archive and audit retention policy

## Roadmap artifact

- [docs/02-architecture/DBIS_PHASE2_PROXMOX_SOVEREIGNIZATION_ROADMAP.md](docs/02-architecture/DBIS_PHASE2_PROXMOX_SOVEREIGNIZATION_ROADMAP.md)

---

# SECTION 9 — SECURITY ARCHITECTURE

## Current baseline

- Chain 138 validator, sentry, and RPC tiering exists as an operational pattern.
- Public RPC capability validation is now script-backed.
- Explorer and wallet metadata are now explicitly documented and verifiable.

## Target-state controls

- HSM-backed key management
- stronger secrets segregation
- certificate hierarchy and operator MFA
- formalized tier-to-tier firewall policy

## Status

These remain partially implemented and must not be represented as fully complete without separate evidence.

---

# SECTION 10 — GOVERNANCE ARCHITECTURE

## Target

- validator governance across multiple entities
- admission control
- key rotation
- emergency controls

## Current state

- Chain 138 validator topology exists
- final multi-entity validator governance assignment is still pending

This section remains a target architecture section, not a statement of fully executed governance.

---

# SECTION 11 — FIREFLY WORKFLOW ARCHITECTURE

## Current state

- FireFly primary footprint exists and now exposes a local API again.
- Current restored `6200` configuration is a minimal local gateway profile for stability and API availability.
- Full multiparty FireFly workflow behavior across blockchain, shared storage, and data exchange is not yet evidenced as healthy in the current container deployment.

## Program objective

Use FireFly as the workflow layer only after:

1. primary and secondary footprint are clearly defined
2. connector/plugin model is explicit
3. upstream blockchain and shared-storage dependencies are validated

## Current execution artifacts

- [docs/03-deployment/DBIS_RTGS_FIRST_SLICE_ARCHITECTURE.md](docs/03-deployment/DBIS_RTGS_FIRST_SLICE_ARCHITECTURE.md)
- [docs/03-deployment/DBIS_RTGS_FIRST_SLICE_DEPLOYMENT_CHECKLIST.md](docs/03-deployment/DBIS_RTGS_FIRST_SLICE_DEPLOYMENT_CHECKLIST.md)

## 11.1 Depository / CSD architecture

### Current state

- A dedicated depository / central securities depository runtime is not currently evidenced as deployed in this environment.
- The depository role is still implied inside broader settlement, securities, and custody discussions rather than frozen as a first-class production component.
- The canonical production checklist row is:
  - [Depository / CSD layer](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)

### Target role

- maintain the authoritative asset register for in-scope instruments
- define issuance, transfer, pledge, and lien semantics
- provide the settlement-touch point between asset ownership and RTGS finality

### Required integrations

- OMNL / Fineract participant and account model
- custody and safekeeping lifecycle
- Chain 138 settlement and evidence path where on-ledger finality is in scope
- external statements, reconciliation, and regulatory evidence outputs

### Current gaps

- No frozen decision yet on whether the depository role is on-ledger, off-ledger, or hybrid.
- No participant-to-asset-register relationship is yet frozen for custody, pledge, and transfer scenarios.

### Execution artifacts

- [docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)
- [docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md](docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md)
- [docs/03-deployment/DBIS_RTGS_DEPOSITORY_AND_CUSTODY_OPERATING_MODEL.md](docs/03-deployment/DBIS_RTGS_DEPOSITORY_AND_CUSTODY_OPERATING_MODEL.md)

### System flow

```mermaid
flowchart LR
    OMNL["OMNL / Fineract"] -->|"participant + account context"| CSD["Depository / CSD"]
    CSD -->|"asset ownership + settlement touch"| RTGS["RTGS Orchestrator"]
    RTGS -->|"cash settlement leg"| BANK["Bank / Correspondent Rail"]
    RTGS -->|"optional finality evidence"| CHAIN["Chain 138 Settlement"]
    CSD -->|"holdings + entitlements"| CUSTODY["Custody / Safekeeping"]
    CUSTODY -->|"statements + evidence"| EVIDENCE["Audit / Reconciliation Package"]
```

### Contract — Depository asset-register and settlement-touch

- Owning subsystem: Depository / CSD layer
- Required integrations: participant model, custody model, settlement orchestration, reconciliation/evidence
- Canonical business object or event: asset position, transfer instruction, pledge/release instruction
- Reconciliation / evidence requirement: holdings register must reconcile to settlement state and custody reporting
- Production completion condition: one canonical asset flow proves issuance/transfer/settlement-touch behavior end to end

## 11.2 Global custodian architecture

### Current state

- No explicit global custodian runtime or operating model is currently evidenced as active in the repo-backed deployment state.
- Custodian responsibilities are currently implied through correspondent-bank and safekeeping language, not frozen as one production role.
- The canonical production checklist row is:
  - [Global custodian layer](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)

### Target role

- manage safekeeping accounts and sub-custody relationships
- coordinate global bank, correspondent, and asset-servicing obligations
- provide statement, confirmation, and reconciliation surfaces for institutional holdings

### Required integrations

- depository / CSD role
- correspondent and global-bank messaging lanes
- custody / safekeeping / asset-servicing lifecycle
- OMNL and RTGS reconciliation packages

### Current gaps

- No frozen custody account structure or reporting model exists yet.
- Corporate-action, entitlement, and asset-servicing obligations are not yet mapped into the RTGS program.

### Execution artifacts

- [docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)
- [docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md](docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md)
- [docs/03-deployment/DBIS_RTGS_DEPOSITORY_AND_CUSTODY_OPERATING_MODEL.md](docs/03-deployment/DBIS_RTGS_DEPOSITORY_AND_CUSTODY_OPERATING_MODEL.md)

### Contract — Global custodian account, reporting, and reconciliation

- Owning subsystem: Global custodian layer
- Required integrations: correspondent/global-bank path, depository role, custody operations, evidence package
- Canonical business object or event: custody account statement, holdings advice, settlement confirmation
- Reconciliation / evidence requirement: custodian statements must reconcile to OMNL and settlement state
- Production completion condition: one canonical custody flow includes account structure, reporting, and reconciliation outputs

## 11.3 FX pricing and dealing architecture

### Current state

- FX pricing, valuation, and revaluation requirements are documented, but no single production pricing/dealing engine contract is yet frozen.
- Existing materials prove the need for FX handling, not a finalized runtime ownership model.
- The canonical production checklist row is:
  - [FX pricing / dealing engine](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)

### Target role

- own quote generation or ingestion
- apply spread and pricing policy
- lock rates, value dates, and booking terms
- feed OMNL, treasury, and settlement services with the approved FX terms

### Required integrations

- treasury policy and limits
- participant / office / GL model
- `server-funds-sidecar` and `off-ledger-2-on-ledger-sidecar`
- reconciliation and evidence path

### Current gaps

- No frozen source hierarchy yet for rates, triangulation, and overrides.
- No canonical quote lifecycle is yet mapped from request to booking to reconciliation.

### Execution artifacts

- [docs/03-deployment/DBIS_RTGS_FX_TRANSACTION_CATALOG.md](docs/03-deployment/DBIS_RTGS_FX_TRANSACTION_CATALOG.md)
- [docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)
- [docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md](docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md)
- [docs/03-deployment/DBIS_RTGS_FX_AND_LIQUIDITY_OPERATING_MODEL.md](docs/03-deployment/DBIS_RTGS_FX_AND_LIQUIDITY_OPERATING_MODEL.md)

### Sequence diagram

```mermaid
sequenceDiagram
    participant Client as Initiating System
    participant ORCH as RTGS Orchestrator
    participant FX as FX Pricing / Dealing Engine
    participant TREASURY as Treasury / Funds
    participant OMNL as OMNL / Fineract
    participant SETTLE as Settlement Service

    Client->>ORCH: FX-backed payment request
    ORCH->>FX: Quote request with currencies, amount, value date
    FX-->>ORCH: Locked quote, spread, rate source, expiry
    ORCH->>TREASURY: Liquidity and approval check
    TREASURY-->>ORCH: Funding approval / rejection
    ORCH->>OMNL: Post booked FX and settlement journals
    OMNL-->>ORCH: Accounting confirmation
    ORCH->>SETTLE: Trigger settlement leg with FX references
    SETTLE-->>ORCH: Settlement reference and finality state
```

### Contract — FX quote, pricing, and booking

- Owning subsystem: FX pricing / dealing engine
- Required integrations: treasury, OMNL, sidecars, settlement, reconciliation
- Canonical business object or event: FX quote, booked FX instruction, revaluation event
- Reconciliation / evidence requirement: rate source, booked rate, and realized/unrealized P&L must reconcile
- Production completion condition: one canonical FX transaction completes with frozen inputs, accounting, and reconciliation

## 11.4 Liquidity pooling and aggregation architecture

### Current state

- Liquidity and prefunding checks are documented, but no explicit pooling/aggregation engine is yet modeled as a first-class production component.
- Liquidity sourcing is currently spread across treasury, correspondent, and optional on-chain discussions.
- The canonical production checklist row is:
  - [Liquidity pooling and aggregation engine](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)

### Target role

- evaluate available liquidity sources
- apply prioritization and eligibility policy
- allocate funding across internal and external sources
- expose operator controls for override, hold, and audit

### Required integrations

- treasury account model
- reserve policy
- bank and correspondent source adapters
- optional on-chain liquidity and settlement lanes

### Current gaps

- No source-priority model is yet frozen.
- No operator control model is yet defined for overrides, holds, or emergency liquidity routing.

### Execution artifacts

- [docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)
- [docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md](docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md)
- [docs/03-deployment/DBIS_RTGS_FX_AND_LIQUIDITY_OPERATING_MODEL.md](docs/03-deployment/DBIS_RTGS_FX_AND_LIQUIDITY_OPERATING_MODEL.md)

### Flowchart

```mermaid
flowchart LR
    REQUEST["Funding Request"] --> ENGINE["Liquidity Pooling / Aggregation Engine"]
    ENGINE --> INTERNAL["Internal Treasury Pool"]
    ENGINE --> BANKLINES["Bank Credit / Liquidity Lines"]
    ENGINE --> CORR["Correspondent / Global Bank Sources"]
    ENGINE --> ONCHAIN["Optional On-Chain Liquidity"]
    INTERNAL --> DECISION["Funding Decision"]
    BANKLINES --> DECISION
    CORR --> DECISION
    ONCHAIN --> DECISION
    DECISION --> ORCH["RTGS Orchestrator"]
    ORCH --> OMNL["OMNL / Fineract"]
```

### Contract — Liquidity-engine source selection and allocation

- Owning subsystem: Liquidity pooling and aggregation engine
- Required integrations: treasury policy, source adapters, RTGS orchestrator, OMNL
- Canonical business object or event: funding request, allocation decision, liquidity hold/release
- Reconciliation / evidence requirement: chosen source and allocation rationale must be reconstructible
- Production completion condition: one canonical funding decision path is documented and validated

## 11.5 Liquidity source adapter model

### Current state

- Source classes are referenced in treasury and correspondent-bank materials, but no canonical adapter model is yet frozen for each source family.
- The canonical production checklist row is:
  - [Liquidity source adapters](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)

### Target role

- normalize access to internal treasury pools, bank lines, correspondent banks, and optional on-chain liquidity
- hide transport/auth differences behind one adapter family
- return funding availability, hold, release, and confirmation events into the liquidity engine

### Required integrations

- liquidity pooling and aggregation engine
- correspondent-bank and global-bank rails
- treasury controls and operator policies
- optional Chain 138 or sidecar/provider adapters

### Current gaps

- No adapter catalog yet exists for source families.
- No required minimum adapter contract is yet documented.

### Execution artifacts

- [docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)
- [docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md](docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md)
- [docs/03-deployment/DBIS_RTGS_FX_AND_LIQUIDITY_OPERATING_MODEL.md](docs/03-deployment/DBIS_RTGS_FX_AND_LIQUIDITY_OPERATING_MODEL.md)

### Contract — Liquidity source adapter

- Owning subsystem: Treasury / integrations layer
- Required integrations: liquidity engine, bank/correspondent paths, treasury controls
- Canonical business object or event: liquidity quote, hold confirmation, release confirmation, failure reason
- Reconciliation / evidence requirement: source selection and adapter result must be linked to the settlement package
- Production completion condition: each in-scope source class has a defined adapter contract and mandatory sources are validated

## 11.6 Custody / safekeeping / asset servicing architecture

### Current state

- Custody and safekeeping obligations are referenced implicitly in correspondent-bank, securities, and evidence discussions, but not yet frozen as one canonical lifecycle.
- The canonical production checklist row is:
  - [Custody / safekeeping / asset servicing flow](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)

### Target role

- manage safekeeping, transfer, entitlement, and servicing lifecycles
- bind depository positions, custodian reporting, and settlement state into one auditable trail
- produce holdings, statements, and servicing evidence for institutional participants

### Required integrations

- depository / CSD layer
- global custodian layer
- OMNL participant and account model
- RTGS settlement and evidence package

### Current gaps

- No canonical custody lifecycle is yet frozen.
- Corporate-action, entitlement, and servicing events are not yet mapped into reconciliation artifacts.

### Execution artifacts

- [docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md](docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md)
- [docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md](docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md)
- [docs/03-deployment/DBIS_RTGS_DEPOSITORY_AND_CUSTODY_OPERATING_MODEL.md](docs/03-deployment/DBIS_RTGS_DEPOSITORY_AND_CUSTODY_OPERATING_MODEL.md)

### Sequence and state view

```mermaid
sequenceDiagram
    participant DEP as Depository / CSD
    participant CUST as Custodian
    participant ORCH as RTGS Orchestrator
    participant OMNL as OMNL / Fineract
    participant EVIDENCE as Evidence Package

    DEP->>CUST: Position / entitlement update
    CUST->>ORCH: Safekeeping or servicing instruction
    ORCH->>OMNL: Accounting impact or fee posting
    OMNL-->>ORCH: Posting confirmation
    ORCH->>EVIDENCE: Reconciliation and servicing references
    EVIDENCE-->>CUST: Statement / audit package references
```

```mermaid
stateDiagram-v2
    [*] --> Registered
    Registered --> Safekept
    Safekept --> Transferred
    Safekept --> Serviced
    Transferred --> Reconciled
    Serviced --> Reconciled
    Reconciled --> Reported
    Reported --> [*]
```

### Contract — Custody, safekeeping, and asset-servicing lifecycle

- Owning subsystem: Custody operations / product architecture layer
- Required integrations: depository, custodian, OMNL, evidence package
- Canonical business object or event: custody instruction, holdings statement, servicing event
- Reconciliation / evidence requirement: holdings, statements, and servicing events must reconcile to settlement and participant records
- Production completion condition: one end-to-end custody lifecycle is documented and validated with reconciliation/evidence output

---

# SECTION 12 — CROSS-CHAIN INTEROPERABILITY DESIGN

## Current state

- CCIP relay and Chain 138 cross-chain infrastructure exist in the broader stack.
- Hyperledger Cacti is not currently proven as the live interoperability engine for DBIS in this environment.

## Planning rule

This plan must refer to Cacti as `future / optional` until a deployed and validated Cacti environment is evidenced in discovery artifacts.

---

# SECTION 13 — DEVSECOPS PIPELINE

## Required execution model

1. Source control
2. Build / validation
3. Security and config review
4. Service verification
5. Deployment
6. Monitoring and readiness evidence

## Repo-backed implementation

- discovery scripts
- RPC health checks
- route / explorer verification
- operator runbooks
- submodule hygiene and deployment docs

The pipeline is partially implemented via scripts and runbooks; it is not yet a single unified CI/CD system for every DBIS workload.

---

# SECTION 14 — PERFORMANCE VALIDATION

## Current state

- Hyperledger Caliper is not vendored in this repo.
- A documented performance hook exists instead of a committed benchmark harness.

## Canonical artifact

- [docs/03-deployment/CALIPER_CHAIN138_PERF_HOOK.md](docs/03-deployment/CALIPER_CHAIN138_PERF_HOOK.md)

## Interpretation rule

Performance benchmarking is planned and documented, but not yet a routine automated readiness gate.

---

# SECTION 15 — MONITORING AND OBSERVABILITY

## Deployed now

- Explorer / Blockscout
- Besu RPC health verification
- operational checks and route verification scripts

## Partially deployed / under validation

- Hyperledger-side service health beyond CT status
- unified status reporting for the broader DLT stack

---

# SECTION 16 — DISASTER RECOVERY DESIGN

## Target state

- RPO / RTO by workload tier
- cross-site replication
- cold / standby recovery paths

## Current state

DR remains a program requirement, not a fully evidenced completed deployment capability.

---

# SECTION 17 — PRODUCTION DEPLOYMENT WORKFLOW

## Phase 1 — Reality mapping

Canonical implementation:

- [scripts/verify/run-phase1-discovery.sh](scripts/verify/run-phase1-discovery.sh)
- [docs/03-deployment/PHASE1_DISCOVERY_RUNBOOK.md](docs/03-deployment/PHASE1_DISCOVERY_RUNBOOK.md)

## Phase 2 — Sovereignization roadmap

Canonical implementation:

- [docs/02-architecture/DBIS_PHASE2_PROXMOX_SOVEREIGNIZATION_ROADMAP.md](docs/02-architecture/DBIS_PHASE2_PROXMOX_SOVEREIGNIZATION_ROADMAP.md)

## Phase 3 — Liveness and production-simulation wrapper

Canonical implementation:

- [scripts/verify/run-dbis-phase3-e2e-simulation.sh](scripts/verify/run-dbis-phase3-e2e-simulation.sh)
- [docs/03-deployment/DBIS_PHASE3_E2E_PRODUCTION_SIMULATION_RUNBOOK.md](docs/03-deployment/DBIS_PHASE3_E2E_PRODUCTION_SIMULATION_RUNBOOK.md)

---

# SECTION 18 — END-TO-END PRODUCTION FLOW

## Reference flow

1. Identity issued
2. Credential verified
3. Workflow triggered
4. Settlement executed
5. Cross-chain sync
6. Compliance recorded
7. Final settlement confirmed

## Current interpretation

This is the target business flow. Current automation verifies only selected infrastructure slices of that flow:

- Besu liveness
- optional FireFly HTTP
- operator-guided manual follow-ups for Indy / Fabric / CCIP
- the currently recommended narrow RTGS first slice documented in:
  - [docs/03-deployment/DBIS_RTGS_FIRST_SLICE_ARCHITECTURE.md](docs/03-deployment/DBIS_RTGS_FIRST_SLICE_ARCHITECTURE.md)
  - [docs/03-deployment/DBIS_RTGS_FIRST_SLICE_DEPLOYMENT_CHECKLIST.md](docs/03-deployment/DBIS_RTGS_FIRST_SLICE_DEPLOYMENT_CHECKLIST.md)

It must not be represented as fully automated end-to-end execution today.

---

# SECTION 19 — EXECUTION DIRECTIVES

Cursor / operators should execute the following in order:

1. Run Phase 1 discovery and review the critical failure summary.
2. Reconcile node-role matrix conflicts, especially duplicate IP planning entries.
3. Validate live Hyperledger CTs at the app layer, not only CT status.
4. Track sovereignization gaps in the Phase 2 roadmap.
5. Run the Phase 3 liveness wrapper and manual follow-ups.
6. Produce or refresh readiness evidence.

These directives must map to repo scripts and docs, not hypothetical tooling.

---

# SECTION 20 — EXPECTED DELIVERABLES

The executable deliverables in this repository are:

1. Infrastructure inventory report
2. Node role assignment map
3. Phase 2 sovereignization roadmap
4. Phase 3 liveness simulation runbook
5. Caliper performance hook
6. Operator readiness checklist

Separate security compliance and benchmark reports remain future deliverables unless explicitly generated.

---

# SECTION 21 — CURRENT GAPS

## Infrastructure gaps

- FireFly secondary `6201` is currently stopped and should be treated as retired / standby until intentionally rebuilt.
- Fabric CTs are present in inventory, but current app-level verification did not prove active Fabric peer or orderer services and did not show meaningful Fabric payloads; they are now intentionally stopped as reserved placeholders.
- Indy CTs are present in inventory, but current app-level verification did not prove active Indy validator listeners and did not show meaningful Indy payloads; they are now intentionally stopped as reserved placeholders.
- The current per-node app-level evidence table is maintained in [docs/03-deployment/DBIS_HYPERLEDGER_RUNTIME_STATUS.md](docs/03-deployment/DBIS_HYPERLEDGER_RUNTIME_STATUS.md).

## Platform gaps

- Ceph-backed distributed storage is still roadmap work.
- Full VLAN / sovereign network segmentation is still roadmap work.
- Final entity ownership assignments remain incomplete.
- The selected first-slice HYBX sidecars are now deployed internally on Proxmox VE and healthy at the runtime level.
- The `mifos-fineract-sidecar` lane has now completed at least one authenticated live OMNL / Fineract transfer posting, but the broader participant model, Chain 138 settlement leg, reconciliation/evidence output, and the `server-funds-sidecar` / `off-ledger-2-on-ledger-sidecar` business flows are still not frozen end to end.

## Planning gaps

- Future-state architecture items must remain clearly labeled as planned, not deployed.

---

# SECTION 22 — IMPLEMENTATION ARTIFACTS

Executable counterparts in this repository:

| Deliverable | Location |
|-------------|----------|
| Node Role Matrix | `docs/02-architecture/DBIS_NODE_ROLE_MATRIX.md` |
| Phase 1 discovery | `scripts/verify/run-phase1-discovery.sh`, `docs/03-deployment/PHASE1_DISCOVERY_RUNBOOK.md`, `reports/phase1-discovery/` |
| Phase 2 roadmap | `docs/02-architecture/DBIS_PHASE2_PROXMOX_SOVEREIGNIZATION_ROADMAP.md` |
| Phase 3 liveness wrapper | `scripts/verify/run-dbis-phase3-e2e-simulation.sh`, `docs/03-deployment/DBIS_PHASE3_E2E_PRODUCTION_SIMULATION_RUNBOOK.md` |
| Production gate | `docs/03-deployment/DBIS_PHASES_1_TO_3_PRODUCTION_GATE.md` |
| RTGS canonical production checklist | `docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md` |
| RTGS FX transaction catalog | `docs/03-deployment/DBIS_RTGS_FX_TRANSACTION_CATALOG.md` |
| RTGS depository and custody operating model | `docs/03-deployment/DBIS_RTGS_DEPOSITORY_AND_CUSTODY_OPERATING_MODEL.md` |
| RTGS FX and liquidity operating model | `docs/03-deployment/DBIS_RTGS_FX_AND_LIQUIDITY_OPERATING_MODEL.md` |
| RTGS control-plane deployment checklist | `docs/03-deployment/DBIS_RTGS_CONTROL_PLANE_DEPLOYMENT_CHECKLIST.md` |
| RTGS control-plane deployment scripts | `scripts/deployment/create-dbis-rtgs-control-plane-lxcs.sh`, `scripts/deployment/deploy-dbis-rtgs-control-plane.sh`, `scripts/verify/check-dbis-rtgs-control-plane.sh` |
| RTGS later-phase sidecars deployment checklist | `docs/03-deployment/DBIS_RTGS_LATER_PHASE_SIDECARS_DEPLOYMENT_CHECKLIST.md` |
| RTGS later-phase sidecars deployment scripts | `scripts/deployment/create-dbis-rtgs-later-phase-sidecar-lxcs.sh`, `scripts/deployment/deploy-dbis-rtgs-later-phase-sidecars.sh`, `scripts/verify/check-dbis-rtgs-later-phase-sidecars.sh` |
| Indonesia / BNI E2E integration blueprint | `docs/03-deployment/DBIS_OMNL_INDONESIA_BNI_E2E_INTEGRATION_BLUEPRINT.md` |
| RTGS first-slice architecture | `docs/03-deployment/DBIS_RTGS_FIRST_SLICE_ARCHITECTURE.md` |
| RTGS first-slice deployment checklist | `docs/03-deployment/DBIS_RTGS_FIRST_SLICE_DEPLOYMENT_CHECKLIST.md` |
| Caliper hook | `docs/03-deployment/CALIPER_CHAIN138_PERF_HOOK.md`, `scripts/verify/print-caliper-chain138-stub.sh` |
| Operator readiness checklist | `docs/00-meta/OPERATOR_READY_CHECKLIST.md` section 10 |