> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sequency.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Terrain backend notes

# Terrain — Backend Notes for UI Wiring

Ground truth for the Strategy Lab / Terrain Workbench wiring. Authoritative for the
**backend** (this repo: ClickHouse + the Python API). Frontend routing and theme are
covered by the app-repo wiring gap analysis and are out of scope here.

Last verified against code 2026-06-24.

***

## 0. Naming

**Terrain** is Sequency's regime classification & detection system. It replaces the
internal working terms "vNext" / "regime\_v2" in all product/UI naming.

* **Terrain v1** = the 2×3 grid.
* **Terrain v2** = the multi-axis read.

Internal backend symbols (`regime_v2`, `RegimeSnapshotV2`, `regime_vnext/`,
`regime_archive`) keep their names for now; "Terrain" is the product-facing name.

**Product name → code symbol map:**

| Product (Terrain)     | Code / internal                                                                                         |
| --------------------- | ------------------------------------------------------------------------------------------------------- |
| Terrain               | the regime classification & detection system (`regime_*`, `regime_archive`)                             |
| Terrain v1 (2×3 grid) | `daily_regime_labels.regime_grid`, `tag_trades_with_regime`                                             |
| Terrain v2 (3-axis)   | `regime_v2` / `RegimeSnapshotV2` / `tag_trades_with_regime_v2_axes`; prereg id `regime-vnext-prereg-v1` |

**Current canon — cite these, not the `docs/superpowers/specs/_deprecated/` set:**

* Program charter — `docs/superpowers/specs/2026-06-11-regime-engine-program-charter.md`
* Terrain v2 design — `docs/superpowers/specs/2026-06-21-regime-vnext-design.md` *(currently local-only — not yet committed to `main`)*
* Locked prereg + evidence — `docs/superpowers/specs/2026-06-22-regime-vnext-prereg-reset.md`
* Production-policy design — `docs/superpowers/specs/2026-06-22-regime-vnext-production-policy-readiness-design.md`
* SP1/SP2/SP3 designs; `docs/api/screener-regime-intraday.md`; `docs/runbooks/regime-vnext-rd.md`

***

## 1. The taxonomies + EXACT enums (stop guessing these)

### Terrain v1 — the 2×3 grid

Direction × Volatility. Persisted **daily** in `daily_regime_labels.regime_grid`.

* Direction: `BULL` | `SIDEWAYS` | `BEAR`  (note: `SIDEWAYS`, not "NEUTRAL", in the label string)
* Volatility: `LOW_VOL` | `HIGH_VOL`
* Cell string: e.g. `BULL_LOW_VOL`, `BEAR_HIGH_VOL` (6 cells)

### Terrain v2 — the 3-axis read (backtester-derivable subset)

* `risk_state`: `stress` | `elevated` | `calm` | `normal`
* `volatility_change`: `expanding` | `compressing` | `stable`  ← **NOT `rising`**
* `directional_pressure`: `upside_extended` | `balanced` | `downside_extended`  ← **no `persistent_trend`**
* Cell string: `"{risk_state}|{volatility_change}|{directional_pressure}"`
  — e.g. `calm|stable|upside_extended`, `normal|stable|balanced`

> **Important:** the `RegimeSnapshotV2` model defines **6** axes (the above 3 plus
> `breadth_state`, `correlation_state`, `intraday_structure`). Those three have **no
> point-in-time source** in `regime_archive`, so the backtester / historical path
> produces only the **3 axes** above. Do not render the other three for any
> historical/backtest view — they don't exist there.

***

## 2. Data provenance + pins (corrections to the handoff)

* **Terrain v2 axes are NOT persisted daily.** Only the 2×3 grid is stored daily
  (`daily_regime_labels`). Terrain v2 is **derived on-the-fly** from `regime_archive`
  via `services/regime_vnext/comparison.py` / `backtester/regime_v2.py`
  (`derive_regime_v2_axes_from_snapshot`). This is the biggest hidden cost of the
  Workbench, and it is the recommended slice-1 path.
* **Pins:** `logic_version = 'sp1-v1'` (the invented `'rl-2026.06.2'` does not exist).
  `variant`:
  * `'emitted'` — point-in-time, look-ahead-free. **PIT already exists**; the limit is
    data **volume** (forward accrual), not schema.
  * `'corrected'` — descriptive / full-history backfill. Use this for descriptive
    displays and backtest attribution.
* A persisted daily Terrain-v2 table (full backfill) is **NOT** required for slice 1,
  and is **NOT** blocked by the regime-program governance charter — that charter
  governs *alpha certification* (promoting regime-conditioning to a live-money gate),
  not a descriptive display table. Treat backfill as a future performance
  optimization, not a prerequisite.

***

## 3. Terrain v2 ↔ v1 grid projection (for the UI)

Both derive from the same `regime_archive` substrate (`slope_pctile`, `vol_pctile`,
`shock`), so v2 maps onto the familiar grid:

| Terrain v2 axis        | → grid axis | mapping                                                               |
| ---------------------- | ----------- | --------------------------------------------------------------------- |
| `directional_pressure` | Direction   | `upside_extended`→BULL, `balanced`→SIDEWAYS, `downside_extended`→BEAR |
| `risk_state`           | Volatility  | `{stress, elevated}`→HIGH\_VOL, `{calm, normal}`→LOW\_VOL             |
| `volatility_change`    | — none —    | the new axis the grid can't represent                                 |

v1 and v2 are **computed independently** (v1 = persisted grid; v2 = derived), so they
*agree* but are not a strict per-row mechanical projection. The UI should read **v1 for
the grid headline** and **v2 for the axis drill-down** — both are stored (§4.2), so no
client-side projection is needed.

***

## 4. Two distinct backend surfaces — do NOT conflate them

### 4.1 Terrain analysis (market-level) — NET-NEW, still to build

Purpose: current/historical **market** regime + per-regime market attribution
(the Terrain Workbench).

* \~70% of the math exists offline in `services/regime_vnext/comparison.py` (reads
  `regime_archive` + daily rollups, derives the v2 axes, computes per-regime
  attribution). It has **no HTTP route yet** — that route is the critical path.
* **Route naming:** use **`/api/graph/v1/terrain/...`**. The singular
  `/api/graph/v1/regime` prefix is taken (`regime.py` + `regime_v2.py`); a plural
  `/regimes` would work but `/terrain` both avoids the collision *and* adopts the
  product name. `regime_vnext.py` exists but is currently **unmounted**.
* **Slice-1 sourcing:** derive on-the-fly (§2). `variant='corrected'` for full history,
  `variant='emitted'` for honest PIT.

### 4.2 Strategy effectiveness across Terrain — ALREADY SHIPPED (PR #1165 + #1166)

Purpose: **how a strategy performed across regimes** (the heatmap). **You do not need
to build this aggregation — it already exists and accrues automatically.**

* **Table:** `backtest_regime_effectiveness` (ClickHouse, DB `sequency`). Written on
  **every** backtest completion (best-effort; never blocks a run), for both the
  rule-driven and cross-sectional paths.

* Stores **both** taxonomies per (strategy, cell): rows with
  `regime_taxonomy='terrain_v1'` (grid cells) **and** `regime_taxonomy='terrain_v2'`
  (3-axis cells).

* **Schema:**

  | column             | type                   | notes                                                                                                 |
  | ------------------ | ---------------------- | ----------------------------------------------------------------------------------------------------- |
  | `run_id`           | String                 | backtest run id                                                                                       |
  | `user_id`          | String                 |                                                                                                       |
  | `strategy_id`      | LowCardinality(String) | empty on assessment-harness runs (run\_id encodes the strategy); populated on the regular `/run` path |
  | `strategy_version` | UInt32                 |                                                                                                       |
  | `strategy_kind`    | LowCardinality(String) | e.g. `rule_driven`                                                                                    |
  | `regime_taxonomy`  | LowCardinality(String) | `terrain_v1` \| `terrain_v2`                                                                          |
  | `regime_cell`      | LowCardinality(String) | grid cell (v1) or 3-axis tuple (v2)                                                                   |
  | `trade_count`      | UInt32                 |                                                                                                       |
  | `win_rate`         | Float32                | 0–1                                                                                                   |
  | `avg_pnl_pct`      | Float32                | mean per-trade %                                                                                      |
  | `net_pnl_dollars`  | Float64                |                                                                                                       |
  | `profit_factor`    | Float32                | `999.0` sentinel when a cell has no losing trades                                                     |
  | `per_trade_sharpe` | Float32                | mean/stdev of per-trade pnl\_pct; `0.0` if undefined (single-trade cell)                              |
  | `expectancy_pct`   | Float32                | == `avg_pnl_pct`                                                                                      |
  | `computed_at`      | DateTime               | DEFAULT now()                                                                                         |

  Engine: `ReplacingMergeTree(computed_at)`, `ORDER BY (strategy_id, regime_taxonomy, regime_cell, run_id)`.

* **UI usage:** read `terrain_v1` rows for the 6-cell grid heatmap headline; read
  `terrain_v2` rows for the axis drill-down. Query "strategy X across all runs in cell Y"
  directly off this table.

* Migration: `clickhouse/058_backtest_regime_effectiveness.sql`. Accumulator:
  `api/app/services/backtester/regime_effectiveness.py`.

***

## 5. Quick reference for the wiring agent

* It's **Terrain** (v1 grid / v2 3-axis), not "vNext". Name the design-system module
  `lib/terrain.ts` and the tokens accordingly.
* Use the exact enums in §1 — `compressing|stable|expanding`, no `rising`, no
  `persistent_trend`.
* Pins: `logic_version='sp1-v1'`; `variant` `emitted` (PIT) / `corrected` (descriptive).
* New work = the **market-level** Terrain analysis route (§4.1). Strategy-across-Terrain
  data is **done** (§4.2) — don't rebuild it.
* Theme + frontend routing (`/api/v4`, `/market/pulse`, providers.tsx default): see the
  app-repo gap analysis; those are not backend concerns.