Use case · model & strategy research

Turn one-off experiments into a compounding research loop.

Research is iterative, but agent sessions are amnesiac: every new chat re-explores dead ends, re-asks settled questions, and re-runs experiments you already paid for. PriorStates closes the loop with two habits — recall before you work, record as you finish — so the thing you learned in iteration N is waiting for you in iteration N+1, in a brand-new session, and for your teammates.

▶ Open the runnable demo ⬇ Download PriorStates

The loop, in five moves

Most people only ever do steps 2–3. PriorStates is steps 1, 4, 5 — the brackets that make the work accumulate instead of evaporate.

  1. 1

    Recall first

    Before touching the model, ask what's already known. A journal_search on the topic surfaces the prior losers, decisions and gotchas; a semantic memory_search recalls your standing preferences. The mistakes you won't repeat are free.

  2. 2

    Hypothesize

    Recall shapes the plan. With the dead ends ruled out, form one specific, mechanism-driven change to test — not a blind sweep.

  3. 3

    Run & measure

    Train and evaluate, or run the sim. The step ends with a number you can stand behind — PR-AUC on a held-out split, Sharpe on an out-of-sample window.

  4. 4

    Record the finding

    The iteration isn't done when the number prints — it's done when the next session can find what you learned. One journal_add (winner / loser / decision / gotcha) plus a memory_add for any durable preference.

  5. 5

    Recall again

    Next session re-runs step 1 — and now your conclusion is in the results. Iteration N+1 begins where N ended. That accumulation is the product.

Watch the loop close

Same search, two sessions apart. The recall that started today's work already carries everything earlier sessions learned — and tomorrow it carries today's win too.

Step 1 — recall before you start. Three prior findings appear before you write a line of code:

$ journal_search topic=churn-model

 outcome   title                                            TL;DR
 ───────   ──────────────────────────────────────────────  ─────────────────────────────
 gotcha    tenure_days leaks the label via a churn join     computed to account-close date
 decision  Optimize PR-AUC, not accuracy (4% positives)     accuracy moved the wrong way
 loser     Blind 64-cell grid search was noise              lift vanished on a fresh split

You now know — for free — not to grid-search, to optimize PR-AUC, and to distrust tenure_days. Three mistakes you won't make this iteration.

Step 5 — record, then recall again. You added one hypothesis-driven feature, measured +0.029 PR-AUC, and journaled it. The same search now returns four:

$ journal_search topic=churn-model

 outcome   title                                            TL;DR
 ───────   ──────────────────────────────────────────────  ─────────────────────────────
 gotcha    tenure_days leaks the label via a churn join     computed to account-close date
 decision  Optimize PR-AUC, not accuracy (4% positives)     accuracy moved the wrong way
 loser     Blind 64-cell grid search was noise              lift vanished on a fresh split
 winner    events_recent + lighter regularization          PR-AUC 0.612 → 0.641  (+0.029)

The list grew. A fresh session — you tomorrow, a teammate, a different agent — starts from the win instead of a blank page.

Four kinds of finding worth keeping

Each entry is a plain Markdown file on your disk, semantically searchable, with a one-line TL;DR and the headline number.

🏆

Winner

What worked, with the metric. “events_recent + lighter regularization → +0.029 PR-AUC.” Promote it; build the next iteration on it.

🪦

Loser

The dead end, so no one re-runs it. “Blind grid search was inside the noise band.” The single most expensive thing to rediscover.

⚖️

Decision

A standing choice the team already made. “Optimize PR-AUC, not accuracy.” Stops every new session from re-litigating it.

⚠️

Gotcha

A trap that bit you once. “tenure_days leaks the label.” Catches the failure before it ships, not after.

Same loop, any research

The example above tunes a classifier, but the five moves don't care what the number is. On a quant desk, swap AUC for Sharpe and a held-out split for an out-of-sample window — the loop is identical.

On a trading desk, the same findings read:

 loser     Loosening cancels to fix maker OTR sacrificed edge proportionally
 decision  Fix order-to-trade ratio by suppressing placement, not loosening cancels
 gotcha    Prediction r² is not a maker-PnL proxy — always validate with a sim
 winner    blockbbo + keep-deep cancel: 49k → 2.9k adds/day AND +0.46 Sharpe

Recall before a new sim, record the result with its number, and the next session skips the dead ends — whether the unit is dollars, Sharpe, latency, or accuracy.

You don't run any of this by hand

Once installed, your agents do it on their own.

priorstates init gives a project its own memory and journal, and wires a short protocol block into Claude Code, Codex, Gemini, Copilot and Cursor: recall before you work, record durable findings as you go. The agent calls journal_search / journal_add the same way it decides to read a file — no background process, nothing captured silently, every entry a file you can read, edit or delete.

Want to see it close in 60 seconds? The runnable demo seeds a few "previous sessions," then walks the whole loop in one file — run it with:

priorstates mdlab run model-research-loop.mdlab.md

No VSCode or extension needed — the runner ships with PriorStates. Prefer the cockpit's ▶ Run button? Start it with priorstates cockpit --allow-write (Run executes code on the host, so it's opt-in).

It's in the repo at docs/research-demo/model-research-loop.mdlab.md — runnable prose, code and results in one Markdown file.

Give your research a memory.

Free, open source, and yours to run.

⬇ Download PriorStates ▶ Open the runnable demo