Skip to main content

​
Pattern Events API

Read-only API over the deterministic pattern corpus (sequency.pattern_events) backing the Pattern Intelligence UI and Track P’s lifecycle/scenario surfaces. Base path /api/graph/v1/pattern-events (api/app/routes/pattern_events.py). Auth as in the README, plus: the whole router is gated behind the patterns capability (Capability.PATTERNS, app/core/pattern_events_auth.py’s require_pattern_reader). A JWT holder resolves through the ordinary DB-authoritative capability path. A service principal that AuthMiddleware has already validated (request.state.internal_caller) may also read the corpus, but only with an X-Sequency-Agent-ID header naming the calling agent — the chart-sessions precedent, not a blanket internal-secret bypass; a bare valid secret with no agent id is refused. That bridge is read-only by construction: it never populates request.state.user, so the one mutating route below (PUT /scenario-policies) carries its own additional Capability.ADMIN_PANEL dependency and stays operator-only — an agent principal hitting it gets a 401 (require_capability sees no user), not a grant. (data#1657, child of data#1536.) Every event-row query reads FROM sequency.pattern_events FINAL (a ReplacingMergeTree — FINAL is required to force dedup at read) with server-side {name:Type} parameters only, and every one of those reads scopes the merge with do_not_merge_across_partitions_select_final = 1 — safe on this table for a structural reason, and unsafe on the two Track P tables. See “FINAL merge scoping” before copying it anywhere. Under FINAL a count() cannot use part metadata, so its cost tracks the corpus rather than the result set — which is why /search’s former per-request total was the binding constraint on how far pattern_events can grow (AC-VOL stage two R-5/R-6). The rule enforced here, stated as what the code checks: paging answers “is there more?” by over-fetching one row, never by counting; an exact total is opt-in and capped; and an opt-in total is admitted only when the request carries a filter that prunes the scan — symbols, or a date_from whose value falls inside a bounded lookback. See GET /search.

​
Endpoints (orientation)

​
GET /meta

/meta returns the exact per-grain and per-pattern corpus counts used for interval enablement, plus registry provenance, corpus generation, and the time the aggregate was generated. It does not aggregate pattern_events FINAL on a user’s request. At the current corpus size, the former three cold aggregates cost several seconds each and intermittently exceeded the API query budget (data#2316). Migration 108 seeds pattern_events_meta_snapshots with one exact aggregate. After every clean non-dry-run pattern backfill or nightly incremental, the authoritative writer recomputes and appends the next snapshot. API workers read only the newest small row, so a cold process and a warm process see the same shared result. A failed refresh is logged and leaves the prior snapshot intact; its generated_at and corpus_generation remain visible. A malformed persisted snapshot fails explicitly with 503; it is never turned into zero counts. The legacy full-corpus loader remains only as bootstrap/development compatibility when the snapshot table is genuinely empty, including a genuinely empty local corpus. corpus_registry_versions and corpus_registry_hashes disclose mixed historical registry stamps. registry remains the newest corpus writer stamp; the definitions endpoint is the authority for the currently served registry. The GET /event/{event_id} and supersession reads still preserve FINAL and exact identity semantics. Migration 109 materializes bloom indices for event_id and dedup_parent, allowing ClickHouse to skip unrelated granules without changing the result (data#1903). It does not create a second event table or weaken deduplication.

​
GET /definitions

/definitions is the authoritative discovery surface for the currently activated pattern vocabulary. It serves all 62 pattern_registry_v8 definitions from the generated registry artifact, including definitions with zero materialized corpus rows. Each definition includes its canonical ID, version, family, direction, grains, min_bars, and level capability. The response carries two different pins on purpose:
  • registry identifies the generated definition set currently served by the worker;
  • corpus identifies the registry stamp on the latest materialized corpus snapshot and includes matches_registry.
A mismatch is an observable rollout state, not permission to hide new IDs or manufacture rows. Consumers discover vocabulary here, then use /search for events. /meta remains the row-derived count/enablement surface.

​
Point-in-time reads (as_of)

GET /search, GET /symbol/{symbol}, and GET /symbol/{symbol}/insights all accept an optional as_of query param (data#1655): an ISO-8601 datetime (bare date or full timestamp; naive input is treated as UTC — a bare as_of=YYYY-MM-DD resolves to 00:00:00Z on that date, so it excludes every event confirmed later that same day; pass an explicit time if you mean “through the end of” a date). When present, every one of those reads adds confirm_bar_time <= as_of to its WHERE clause — a parameterized DateTime64(3) bind, never interpolated — on top of the existing FINAL + suppressed = 0 semantics, which are unchanged. What as_of actually filters, precisely: it restricts the result to events whose confirm_bar_time is at or before T, evaluated against the corpus as it stands today — not a bitemporal read of the corpus as it stood at T. It mirrors the guard the product side already applies internally (app/services/registry_patterns.py’s _ACTIVE_PATTERNS_SQL, which binds confirm_bar_time <= {as_of:DateTime64(3)} the same way), and that guard has the identical scope, not a stronger one. The nightly re-emission caveat. The pattern-events pipeline re-emits trailing days nightly (backfill corrections, dedup resolution). as_of filters on confirm_bar_time — WHEN a pattern is claimed to have confirmed — which is stable under this filter, but every OTHER column (quality, grade, suppressed, dedup_parent, geometry) reads back in whatever form the corpus holds today, regardless of as_of. A row confirmed 2026-06-10 but corrected by a 2026-06-20 re-emission is visible to as_of=2026-06-15 in its POST-correction form — the read cannot reconstruct what that row looked like on 2026-06-15 itself. as_of answers “which events had confirmed by T”, not “what did the corpus say about those events as of T”; conflating the two is the mistake this filter must not be used to make. This is therefore not sufficient, alone, for a lookahead-safe backtest. A backtest that needs the second guarantee — the exact field values a strategy would have observed at T, not just which rows existed by T — must read the sealed pattern-events tape via the research freeze (ADR-0007, “Backtests consume the sealed pattern corpus via the research freeze; they never recompute detections”; ratified 2026-08-08 — docs/adr/0007-backtester-pattern-corpus-sealed-tape.md), which remains the authoritative PIT mechanism for that use case. Closing the bitemporal gap for a live HTTP read directly — an inserted_at axis alongside confirm_bar_time — is tracked separately (data#1707). Research replay and agent tooling that only need “which events had confirmed by T” (not their exact historical field values) are exactly what as_of as shipped here answers. as_of layers on top of date_from/date_to, not in place of them: a caller can combine all three, e.g. “events confirmed in June, as known at end-of-June” (date_from=2026-06-01&date_to=2026-06-30&as_of=2026-06-30T23:59:59Z).

​
GET /search

Params: pattern_ids, families, direction, grade_floor, grains, date_from, date_to, as_of, symbols, include_suppressed, sort (detected | grade | quality | symbol), limit (1-500, default 100), offset (>= 0), and include_total (default false). An unrecognized filter value is 422, never a silent empty page. Response: {"events": [Event], "total", "total_capped", "total_cap", "has_more", "limit", "offset", "query_ms", "registry"}.

​
Paging: has_more, not total (data#1599)

This endpoint used to run an uncached, unlimited count() … FINAL on every request, sequentially after the row query and under the same 8s budget. count() cannot use part metadata under FINAL — it has to materialise the dedup — so the cost scaled with the corpus, not with the result set, and the default view (no symbols filter, i.e. no prefix of the sort key) made it a full-table count. That one query is what placed an operational ceiling on how far pattern_events may grow (AC-VOL stage two R-5/R-6). The default response now carries has_more instead: a boolean computed by over-fetching a single row (limit + 1), the same mechanism /symbol uses. “Is there another page?” is what a pager actually needs, and it costs one row rather than a corpus traversal. offset paging is unchanged.

​
total is opt-in, nullable, and bounded twice

total is null unless you pass include_total=true. Null means not computed — never zero. A client that renders total ?? 0 will print “0 results” over a full page of events; read has_more (and, if you asked for one, total_capped) instead. With include_total=true:
  • the count runs over a subquery capped at total_cap (50,000) rows, so the aggregate itself can never be unbounded;
  • total_capped: true means total is a floor (“at least this many”), not an exact count;
  • the request must carry a filter that bounds the scan — either symbols, or a date_from within the last 400 days — or it is refused with 422.
That last rule is not redundant with the cap, and the reason is easy to get wrong: a LIMIT inside the subquery bounds the rows aggregated, not the rows scanned. It short-circuits only once at least total_cap rows match. A selective predicate on a non-prefix column — pattern_ids=gapexh with no symbol and no lower date bound — matches far fewer rows than the cap and therefore still traverses the whole corpus under FINAL to prove it. The two admissible bounds prune by different mechanisms, which is why only one of them has a value limit: date_from’s value is checked because presence bounds nothing: ?include_total=true&date_from=1970-01-01&pattern_ids=gapbrk prunes zero partitions and, measured cold on prod, read 56,301,859 rows / 5.56 s — the entire corpus, 70% of the 8s query budget. The same query with a recent date_from read 376,813 rows in 0.21 s. 400 days spans at most 14 monthly partitions (≈23% of the corpus) and still clears a trailing-year total. An ancient date_from is allowed alongside symbols, since the symbol prefix already bounds the read. date_to/as_of never qualify: they prune the newest partitions, which on a corpus whose mass sits in the past prunes almost nothing.

​
FINAL merge scoping

Every pattern_events read on this router carries SETTINGS do_not_merge_across_partitions_select_final = 1. It is safe on this table specifically, for a structural reason rather than a writer convention: migration 081 partitions on toYYYYMM(confirm_bar_time) while confirm_bar_time is itself in the ORDER BY, so two rows sharing a full sort key necessarily share a partition and the cross-partition merge can never have anything to do. The Track P tables (083_pattern_lifecycle_events, 084_trading_scenarios) partition on state_time, which is absent from their ORDER BY — the same setting there would return duplicate transitions, silently. Never widen it to those tables.

​
Latency metric

Both queries are timed into the Prometheus histogram pattern_events_query_duration_seconds, labelled query = search_rows | search_count | symbol_rows. search_count is observed only when a count actually runs, so the series stays a distribution of real count latencies rather than one diluted by zeros. This is the series AC-VOL R-6’s tripwire reads — p95 of the count ≤ 4,000 ms, half the route’s 8,000 ms _QUERY_TIMEOUT_MS; both values are exact bucket edges.

​
Cursor pagination + expand[] on /search (#1845 P1-a/b/c)

sort=confirm_bar_time is the canonical recency sort (detected is a deprecated alias with identical order — the tuple is confirm_bar_time DESC, event_id DESC, direction DESC, total per migration 081’s collision rationale). Cursor (canonical sort only; other sorts page with offset):
Rules (all fail-closed 422): the cursor is opaque and fingerprint-bound — the filters and sort may not change mid-scan; cursor and offset are mutually exclusive; malformed or foreign cursors are rejected. Pages are detection-only, not snapshot-consistent: the nightly re-emits trailing days and the corp-action sweep restates history. Compare corpus_generation across pages — unequal proves the corpus moved (restart the scan or move to an ADR-0007 freeze); equal does NOT prove it didn’t (the watermark is cached, <=300s stale). Snapshot-stable reads are the freezes’ job, never this surface. expand[] (Stripe bracket form): anchors, levels, provenance, evidence, relations. Unknown values are a 422. With expand[]=levels, "levels": null means the pattern maps no levels (see /definitions for which and why); key absence means “not expanded”, never “no data”. direction accepts 1/-1 and bullish/bearish — anything the API emits round-trips as a filter.

​
GET /symbol/

DEPRECATED (#1845 ruling §7.3): now sugar over /search’s filter grammar with a frozen wire shape; responses carry "deprecated": true; removed once the app-consumer migration is verified (condition-gated per the 2026-08-16 amendment; tracked on #1889 — no calendar date). Unknown query params are a 422 pointing at /search (AC-4). next_cursor tokens are now 3-part (millis:event_id:direction — total order per E3); legacy 2-part tokens remain accepted for in-flight scans.
Params:
  • symbol — path, required.
  • grain — query, default "daily". Must be one of 1min, 5min, 15min, 30min, 60min, daily (CANONICAL_GRAINS); an unknown grain is 422.
  • date_from, date_to — query, optional ISO date strings; filter confirm_bar_time when present. date_from has a default (see windowing below) — omitting it does NOT mean “no lower bound.”
  • as_of — query, optional ISO-8601 datetime. See “Point-in-time reads” above.
  • limit — query, 1-2000, default 2000. offset — query, >= 0, default 0.
  • cursor — query, optional keyset token (data#1600). See “Truncation” below. Mutually exclusive with a non-zero offset (422).
  • include_suppressed — query, default false.
Windowing & pagination (data#1656): when date_from is omitted, the scan defaults to a trailing 90-day window anchored to the MIN of whichever of as_of/date_to are present, else now — never the unbounded full-history corpus, and never a fixed priority order between as_of and date_to. The agent bench’s original t8-all-drawn-levels task proved the failure mode the default itself exists to prevent: listing drawn levels from the full undated corpus on a liquid symbol exhausted an 8k-token budget on roughly 300 prices (332 on NVDA/daily when measured). That task has since been rescoped to the 3 most recently confirmed events (t8-recent-drawn-levels, data#1708); the finding about the unbounded read stands. The min-of-bounds rule exists to prevent a narrower failure: as_of and date_to both constrain confirm_bar_time, so anchoring to either one unconditionally over the other reproduces a guaranteed-empty window (date_from > date_to, returned as a quiet 200) whenever that param names an earlier point than the one anchored to — a bare date_to in the past breaks under an unconditional “now” anchor, and as_of later than date_to (e.g. a research replay combining a recent as_of with an old date_to) breaks the same way under an unconditional as_of anchor. Taking the min of whatever bounds are present is never later than the tightest explicit constraint, regardless of which params accompany it. The resolved window is always echoed back in the response (date_from/date_to) so a caller can see what was actually scanned, whether it came from the default or an explicit param. Within that window, results page most-recent-first: the underlying query orders by confirm_bar_time DESC, event_id DESC before applying limit/offset, then re-sorts the page by the exact reverse key (confirm_bar_time, event_id ascending) for consumers (the chart lane/overlay expect ascending). The event_id tiebreak on both sides is load-bearing, not decorative: ClickHouse doesn’t guarantee a subquery’s row order survives an outer re-sort, so a coarser outer sort (on confirm_bar_time alone) would leave rows tied on that column in an unspecified relative order — which matters because the query over-fetches one surplus row to compute has_more (below) and trims it by position. offset=0 is always the newest limit events in the window; increasing offset walks backward in time. has_more (boolean) tells a caller whether more matching events exist beyond the current page within the resolved window — TRUE means the page was truncated, FALSE means this page is everything the window has to offer (not necessarily everything in the full corpus; a narrower date_from/date_to can still exclude older or newer matching events entirely — check the echoed-back window if that distinction matters). It is computed by over-fetching one extra row (limit + 1) server-side rather than a separate count() query. That’s a deliberate choice, not an oversight: an exact total here would carry a corpus-scaled cost for a question (“is there more?”) that a boolean already answers. /search reached the same conclusion when its own unbounded count() was removed (#1599) — where an exact total survives at all, it is opt-in, capped, and requires a filter that demonstrably prunes the scan. Do not retrofit one onto this endpoint.

​
Truncation is disclosed, not silent (data#1600)

The limit cap tops out at 2000 and is not raised — raising it buys a slower query and the identical silent failure one density doubling later. What changed is that its effect is now visible. Three fields say so:
  • truncated — the window holds more than this page. Same value as has_more, named for the question a chart asks (“am I showing a complete picture?”) rather than the one a pager asks. Both ship, so existing has_more consumers need no change. truncated: false means “everything in the effective window”, not “everything that exists” — when date_from was defaulted, that window is the trailing 90 days, so paging backwards with next_cursor reaches the default floor and reports false while older events remain in the corpus. Read the echoed-back date_from if that distinction matters, and pass an explicit date_from to widen it.
  • oldest_confirm_bar_time — the confirm_bar_time of the oldest row in this response, i.e. where the window was cut, so a caller can disclose a bounded span honestly instead of implying a complete one. null only on an empty page, where there is no cut to report.
  • next_cursor — a keyset token for the next page backwards in time, or null when there is nothing older. A missing token means “no next page”; there is no token that returns an empty page.
Why this matters more as the corpus grows: at higher event density the same 2000 rows cover a proportionally shorter span, so the undisclosed gap widens on its own. #1485 fixed which 2000 rows are lost (the oldest, not the newest — an ascending cap froze dense grains at ancient history and the chart’s key-moments layer went permanently stale); it did not make their loss visible. Paging backwards. Echo next_cursor back verbatim as cursor. The token is <epoch_millis>:<event_id> — the exact two-column key the feed already sorts on, in the column’s own millisecond resolution, and URL-safe with no escaping. A malformed token is 422, never ignored: a silently-dropped cursor would restart the caller at the newest page while it believed it was paging backwards, looping forever over the same rows while appearing to progress. cursor supersedes offset and the two cannot be combined (422). A cursor resumes strictly older than the row it encodes, so pages cannot overlap or skip when new events land at the head of the feed between requests — which offset cannot promise, since an insert at the head shifts every later page by one.

​
GET /symbol//insights

The chart-annotation path: projects the same rows /symbol/{symbol} returns (same query, same suppression semantics, same registry stamp) into CT-01 SemanticChartObjects (kind="insight") so charts-core renders confirmed pattern structure through its insight projection instead of a bespoke overlay. This is a view, never a detector — every anchor is a coordinate the registry already wrote; nothing here fits a line, infers a level, or derives a price (api/app/services/pattern_insight_projection.py). This is the replacement for the deleted app-side analyze proxy: chart geometry now comes from this corpus-backed endpoint, never from an LLM. Params: same as /symbol/{symbol} above — grain, date_from/date_to (with the same 90-day default window, data#1656), as_of (data#1655), limit/offset/cursor (data#1600) — since this endpoint is a view over that query. The one difference: no include_suppressed switch; suppressed rows (dedup losers) are never projected, by design — a view must not re-surface a precedence loser. Response: {"symbol", "grain", "objects": [SemanticChartObject], "registry", "limit", "offset", "has_more", "truncated", "oldest_confirm_bar_time", "next_cursor", "date_from", "date_to"}. Everything after objects/registry mirrors /symbol/{symbol}’s windowing/pagination envelope verbatim (data#1656, data#1600) — see that section for has_more/truncated semantics, why the signal is a boolean rather than an exact count, and how to page backwards. The truncation disclosure matters more on this endpoint than on the raw feed, not less: this is the surface that would otherwise present a bounded window to a chart as a complete one. Each object’s own scope sub-object (part of the CT-01 InsightScope contract, not a request param) always carries symbol, interval (=grain), from_time, to_time — required by charts-core’s semanticGraph validation, computed per-event from the anchor window (or the confirm bar, for an event with no drawable geometry). confirm_time — a top-level field on every object (data#1648), an ISO-8601 string mirroring the source event’s confirm_bar_time. It exists because scope.to_time is not a recency signal despite looking like one: to_time is the max anchor-pivot time (or the confirm bar, for a geometry-less event), and that is not monotonic with confirm order whenever confirm_lag_bars > 0 — a pattern’s anchors can sit well before its own confirmation bar. Verified empirically on a real NVDA matchhl payload: an object confirmed earlier had a later scope.to_time than one confirmed after it, so picking “the most recent X” by max to_time silently returns the wrong object (a real value, for the wrong event) — the exact failure an LLM benchmark reproduced against this endpoint (Task 11, docs/superpowers/specs/oq-pde-001-results/bench-insights.md). objects stays in the same ascending-by-confirm_bar_time order /symbol/{symbol} returns, so list position is also a valid recency ordering; confirm_time means a caller no longer has to rely on either that ordering or the misleading scope.to_time proxy. confirm_time sits as a sibling of semantics, not a key inside it — also load-bearing, for the mirror-image reason claim.levels is nested inside claim below: charts-core’s annotationFromObject reads a fixed allow-list of semantics keys and silently drops anything outside it, so a new semantics.confirm_time would vanish at the renderer. A top-level object field has no such allow-list — SemanticChartObject is generated with extra="allow" (app/generated/chart_contracts.py), so an additional field here round-trips through the wire contract untouched, and the app’s typed consumer (which reads whole objects) sees it without a contract regeneration. semantics.claim.levels — the structural levels (resistance/support/ invalidation) shipped this week (data#1585). Per-pattern lookup only, never computed:
  • Shape: {resistance?: number, support?: number, invalidation?: number}. There is no target — a target requires projecting a measured move, which this module deliberately does not do (data#1584).
  • Values are verbatim corpus field values — a named price already written by the detector behind the pinned registry hash (e.g. nbar → trigger_high/trigger_low, nr7id → entry_long/entry_short), looked up by name, never derived. Six patterns currently have a defensible horizontal band mapping (nbar, matchhl, spkldg, two_b, nr7id, cuphandle — the last is support-only). The other registered patterns may project detector-authored pivots, regions, or confirmation markers, or remain semantic-only; /definitions reports levels=null plus the specific absence reason instead of treating missing level geometry as missing data. See the _LEVEL_BANDS and _NO_LEVELS_REASONS tables in pattern_insight_projection.py.
  • nr7id never carries invalidation, even though it emits both resistance and support — unlike cuphandle’s empty band key, this is a data-driven suppression (_NO_INVALIDATION in pattern_insight_projection.py), not an absent band. nr7id’s direction is candle-bias only (the compression bar’s own color), not a committed trade side — the pattern’s real directional resolution is the ORB-band breakout, and until that resolves both entry_long and entry_short are live triggers, neither is anyone’s stop. Drawing an invalidation off the candle’s color would assert a direction the detector explicitly disclaims (ratification pack epic #1654 item B2, sequencyhq/sequency-data#1638).
  • two_b’s penetration_high/penetration_low guarantee is one-sided, per direction — never rely on “outside the range.” A short two_b guarantees penetration_high > new_extreme_price; a long guarantees new_extreme_price > penetration_low (the literal definition of penetration). The other side of the interval is architecturally free — new_extreme_price sits inside [penetration_low, penetration_high] in the common case, since a 2B failure is a marginal poke that fails, not a clean break of both sides. A consumer that filters or scores on “new_extreme_price is outside the penetration range” will be wrong most of the time; only the one-sided, per-direction inequality above is a real invariant (ratification pack epic #1654 item D, sequencyhq/sequency-data#1640).
  • Nested inside claim, not a sibling of it — load-bearing. charts-core’s annotationFromObject maps a fixed set of semantics keys (insight_kind, insight_status, claim, evidence, uncertainty, invalidation, presentation); a sibling semantics.levels would be silently dropped at the renderer and look identical to a pattern having no band. claim is cloned wholesale, so nesting survives the contract’s own reader.
An event with no drawable corpus geometry degrades explicitly (semantics.degraded = true) rather than fabricating anchors. This includes the semantic-only donchian_break projection and historical contextual events that predate their confirmation-marker coordinate. A registered pattern with no corpus row produces no object at all; the API never synthesizes an event to exercise presentation. Charts-core’s persistence tests pin semantics.degraded through ChartSession hydration, the semantic graph, agent projection, and accessibility copy. The generic annotation-kind contract handles detector-authored trendlines, levels, regions (including four-anchor pipe), and confirmation markers without a per-pattern chart allow-list.