# SolaceScanScout Tasks — Completed Implementation

**Date:** 2026-02-09  
**Reference:** [SOLACESCANSCOUT_DEEP_DIVE_FIXES_AND_TIMING.md](SOLACESCANSCOUT_DEEP_DIVE_FIXES_AND_TIMING.md), Todos expl-1–expl-12.

---

## Implemented (code & runbooks)

| Task | What was done |
|------|----------------|
| **expl-1** Harden explorer check | `daily-weekly-checks.sh`: [135] now **FAILs** when Blockscout API unreachable (unless `EXPLORER_FAIL_WHEN_UNREACHABLE=0`). Tries direct then public URL. |
| **expl-2** Indexer lag check | [135b] added: compares RPC `eth_blockNumber` to Blockscout `total_blocks`; **FAIL** if lag > `EXPLORER_INDEXER_LAG_THRESHOLD` (default 500). |
| **expl-3** Weekly thin pool check | [138a] added in weekly mode: SSH to r630-02, `pvesm status \| grep thin`; **WARN** at 85%, **FAIL** at 100%. |
| **expl-4** Optional alerting | Metric file written: `logs/maintenance-checks.metric` (`maintenance_checks_failed`, `maintenance_checks_timestamp`). See `scripts/maintenance/README.md` for email/Slack/Prometheus examples. |
| **expl-5** RPC decommission checklist | OPERATIONAL_RUNBOOKS: new § "When decommissioning or changing RPC nodes" and § "After NPMplus/DNS change", "After frontend/Blockscout deploy", "Before/after Blockscout version change". |
| **expl-8, 9, 10, 11, 12** Procedures | Same runbook sections + NEXT_STEPS_OPERATOR quick command index entries. |

---

## One-time / when needed (operator)

| Task | Where to do it |
|------|----------------|
| **expl-6** Request SSL for explorer.d-bis.org | NPMplus UI https://192.168.11.167:81 → SSL Certificates → Let's Encrypt for explorer.d-bis.org. See NEXT_STEPS_OPERATOR § Explorer SSL. |
| **expl-7** Migrate VM 5000 to thin5 when thin1 >85% | BLOCKSCOUT_FIX_RUNBOOK § "Fix: Migrate VM 5000 to thin5". Weekly check [138a] will warn/fail when thin pool is high. |

---

## Fix indexer lag (2026-02-09)

Restart Blockscout to resume indexing when lag is high:

```bash
bash scripts/fix-explorer-indexer-lag.sh
```

Run from a host that can SSH to r630-02 (192.168.11.12), or from the Proxmox host. After a restart the indexer catches up within minutes; lag dropped from 1013 to 32 in one run.

**Lag cron (check-and-fix every 6 hours):**

```bash
bash scripts/maintenance/schedule-explorer-lag-cron.sh --install
```

Runs `check-and-fix-explorer-lag.sh` at 0:00, 6:00, 12:00, 18:00; if lag > 500 it runs `fix-explorer-indexer-lag.sh`. Log: `logs/explorer-lag-fix.log`. Use `--show` to print the cron line, `--remove` to uninstall.

---

## Env / tuning

- `EXPLORER_FAIL_WHEN_UNREACHABLE=0` — restores legacy SKIP when explorer unreachable (e.g. cron off-LAN).
- `EXPLORER_INDEXER_LAG_THRESHOLD=500` — override indexer lag threshold (blocks).
- `MAINTENANCE_METRIC_FILE` — set empty to disable metric file.

---

## Cron

Re-install to pick up script changes:

```bash
bash scripts/maintenance/schedule-daily-weekly-cron.sh --install
```

Daily (08:00): [135], [135b], [136].  
Weekly (Sun 09:00): [137], [138a], reminder [138] for explorer logs.