Skip to main content

sequency-data — Agent Instructions

Shared delivery policy

All AI agents working in this repository must read and follow docs/agent-delivery-discipline.md. It is the vendor-neutral source for active packet scope, vertical delivery, token discipline, verification economy, and the five delivery states. Repository-specific rules below add to that policy.

Governed delegated implementation

Delegated Sequency implementation, inventory, architecture, security, and diff review must use the discoverable .agents/skills/sol-luna-promotion skill and its one-use task packet. Sol retains orchestration, exact-diff review, promotion, deployment, and acceptance ownership; the skill does not replace repository tests, CI, or security review. GitHub: sequencyhq/sequency-data Parent product context: ../CLAUDE.md, ../AGENTS.md, operator ../THREATMODEL.md. This file is the repository-local guide for backend, data plane, agents, and deploy.

Mission

Own market data ingestion, scoring, patterns, options, debates, backtests, lifecycle/scenario pipelines, and the FastAPI graph API that powers sequency-app and sequency-charts consumers.

Layout

DSOM contract & geometric-detection invariants (2026-08-05)

Standing rules from this week’s deterministic geometric-detection work — read before touching pattern/setup/signal/proposal/execution surfaces:
  1. DSOM is the governing contract. Read docs/superpowers/specs/2026-08-04-decision-support-object-model.md before touching any of the five objects it defines: Observation → Setup → Signal → Proposal → Execution.
  2. OA-13: Setup identity is the lifecycle_id dual-key; event_id is the confirmation fact and is NULL on prospects. OA-14: the word “revision” is banned — three named refs replace it (see DSOM). OA-15: “Evidence Envelope” is the semantic name for the existing Decision Lens wire object; chart_contracts.Observation remains the documented legacy wire name (no breaking rename).
  3. No LLM-derived price, ever. Chart geometry/levels come from the corpus-backed /pattern-events/symbol/{symbol}/insights endpoint (api/app/services/pattern_insight_projection.py); this is a view, never a detector. The legacy app-side analyze proxy is deleted — do not reintroduce a path where an LLM produces a drawn/actionable price (security-review-verified invariant).
  4. Pattern-catalogue work is governed by epic #1654 (production-readiness spec docs/superpowers/specs/2026-08-07-geometric-detection-production-readiness.md). Class A construction is AUTHORIZED in tranches under the ratified classification pass (2026-08-06-class-a-adj2-classification-pass.md) + OQ-PDE-002 discipline. The _shape substrate EXISTS (#1634 shipped 2026-08-08: migration 090_shape_facts.sql + pkg/patterns/shapes.go) and TRANCHE 1 HAS BEGUN: doji is the first registered shape emitter (pkg/patterns/detect_shapes.go), so the substrate is no longer inert. Registering an emitter still persists nothing in prod — the nightly stays gated on the operator’s /opt/sequency/shape-facts-nightly-enabled marker, created only after a supervised 5y backfill + dual-build gate. OQ-PDE-002 v0.3 is EFFECTIVE for tranche 1: its §8.4 declares the cases for the 8 constructible geometry-only shapes, §8.5 the one-detector-per-PR mechanics (each shape lands with its own OQ cases, its A-SH-09 band declared from an INDEPENDENT reference implementation before the emitter is written, and its §5 execution rows). Contextual shapes are gated on AC-CTX (#1588); kicking on AC-VOL stage two. Classes B–G need their own ratified classification passes first.
  5. Screener filters fail closed. Build requests with registry filter ids; an unsupported filter leaf returns 422 unsupported_filter, it is never silently dropped.
(The lint/deploy-verification rule this work also established is folded into Verification commands below, not repeated here.)

Data ownership (do not invent schemas)

  • ClickHouse — OHLCV, scores, signals, indicators, options, lifecycle/outcomes
  • FalkorDB (trading_knowledge) — graph market context (1-hop store; not a full KG replacement)
  • PostgreSQL — users, debates, configs, risk profiles, relational state
  • Redis Streams — realtime bus (same Redis as FalkorDB module)
  • Qdrant — vectors for analogs / historical setups
Column/type mismatches between writers and readers are production incidents. Prefer verifying CH column types and Go scanner types against live schema files.

Deploy & migrations

  1. Work on a clean worktree if data/ root is dirty/stale (common).
  2. Commit + push to main (or PR → merge with gates).
  3. CI: tests → cross-compile Go → package → upload compute → migrations → restart → health.
  4. Never SSH-apply migrations or SCP binaries.
  5. Migration runner: scripts/apply-db-migrations.sh (CH + PG, schema_migrations tracking).
  6. Post-deploy:
    ssh [email protected] "curl -s -o /dev/null -w '%{http_code}' http://localhost:8081/health"
Workflow: .github/workflows/deploy-worker.yml.
Env on server: /opt/sequency/.env (mode 600). systemd EnvironmentFile= overrides Environment=; deploy Stage 2 overlays preserved server .env over package env — server wins.

Verification commands (confirm in CI if scripts change)

Ruff in scratch trees: always use api/pyproject.toml (length 100). Default ruff 88 fails CI.

Auth & internal secrets

  • User routes: Supabase JWT; AUTH_ENFORCEMENT_MODE=strict in prod.
  • Service-to-service: X-Internal-Secret / INTERNAL_SECRET_<SERVICE> — hard 403, no fallback.
  • Per-service secrets do not fall back to a global secret.
  • Read ../THREATMODEL.md before new trust boundaries or /security-review.

Options & market data landmines

  • MSE owns OPRA WebSocket options quotes; OI from Trading API (daily).
  • Do not reintroduce Alpaca REST polling for options steady-state.
  • Microstructure / analysis should read CH (MSE-populated), not call Alpaca from random jobs.
  • market_data_daily includes after-hours; date column is often day not bucket.

Python preferences

  • Prefer Polars + numpy; no pandas in new code (numpy stats ≠ pandas).
  • IndicatorBank stays scalar-Python where established.
  • asyncpg: JSONB may arrive as strings; pool bound to creating loop; careful with TIMESTAMPTZ vs naive dt.

ClickHouse landmines

  • Never alias an aggregate to a column name a sibling aggregate reads → ILLEGAL_AGGREGATION (data#1472 / #1345).
  • User settings belong in users.d/, not config.d/ (wrong placement → crash loop).
  • CH FINAL + capture_source='live' semantics matter for lifecycle parity monitors.

FalkorDB landmines

  • Never RETURN {map} — use aliased returns.
  • openCypher dialect quirks; Go type coercion on query results is a recurring bug class.

Scoring ownership

  • MSE = realtime scores/signals writer
  • pattern-detector = structural / pattern side
  • Registry / ontology: see app docs documentation/ontology/SCORING_REGISTRY.md when touching scores
  • Confluence v5 field names are easy to typo (entry_confluence_net_v5 vs entry_net_confluence_v5) — grep before inventing

Git / transport

  • Local data/ is often dirty and behind remote. Verify with gh, not only git status.
  • Prefer worktrees: ~/projects/sequency/.worktrees/data-<topic>
  • Helpers for stuck transport may exist as .git/gh_push*.py — do not force-push main
  • Org ruleset: security-review-gate required on main

Active backend programs (re-check memory)

  • Track P lifecycle/scenarios — wire to prod on merge; monitors not data-accrual gates
  • Intraday alpha Phase G shipped; RF2 calendar seal
  • Footprint order-flow inert behind FOOTPRINT_SYMBOLS
  • Backtestable confluence / strategy-run look-ahead issues — treat as capital-safety

Spec discipline

When editing specs under docs/:
  1. Verify every named symbol against current code.
  2. Read full surrounding blocks before delete/replace guidance.
  3. Echo-sweep the doc after every correction.
  4. Spec example responses may be slimmer than shipped structs — project full fields when building UI/API.

Issues

gh issue create -R sequencyhq/sequency-data
Cross-cutting work defaults to this repo.