All docs

Docs › MTS Lab

Notebooks and editors

Research freely in notebooks, edit in JupyterLab or VS Code, and turn what works into engine code with a check that the two agree.

Notebooks in a project

Notebooks live in notebooks/ (+ notebook starts one). The project page shows a notebook read-only with its outputs; buttons open it in JupyterLab or VS Code, make it a step (Run as step — every run is then a recorded experiment keeping the executed copy and what it records), or ask the agent to convert it. Git stores notebooks without their outputs, so commits and diffs show code; running a notebook is not a change.

Inside a notebook import mtslab first: it puts the project's own modules (import config) and its libraries' Python (import qlib) on the path. mtslab.record, chart and table work as in any step.

Data in pandas, as the engine sees it

from mtslab import data
ev = data.events("ll_day1")               # every event of a day: ts, port, type, bid, ask, …, px, qty, side
bn = data.book(ev, ["BTCUSDT", "binance"]) # that instrument's top of book at every event
fires = data.fires(ev, 1000)               # where a 1 s TimeSampler fires
rows = data.print_rows(ev, 1000)           # where the engine writes a dataset row
data.ema3(x, halflife)                     # the engine's average

events() merges files in time order, keeps the workspace's instruments and drops BBOs that do not change the top — exactly what components see. Written on these helpers, a feature lines up with the engine's values row for row.

JupyterLab

One JupyterLab per workspace, started on the workspace's Python and rooted at the workspace (port 8931; "jupyter_port" in ~/MTS Lab/settings.json changes it). The MTS Lab (Python) kernel has mtslab and the workspace set. When the workspace is a server's, its JupyterLab is forwarded over the same ssh connection, so it opens on your laptop.

VS Code

The whole workspace is one VS Code workspace: the VS Code button opens ~/MTS Lab — projects, libraries and notebooks as one tree, with the Python paths, .env, C++ include paths and git repositories set up, and two tasks: MTS Lab: open this file in MTS Lab and MTS Lab: run this project's steps as an experiment. VS Code ↗ on a file opens that file in the VS Code window that has the workspace — over Remote-SSH when the workspace is on a server.

Convert a notebook to the engine

Convert… on a notebook puts the request in the agent's box. In the conversation the agent:

  1. reads the notebook and says what maps to the engine and what stays exploration;
  2. moves the feature code into an importable file (features.py, or a library's python/) written on mtslab.data, and makes the notebook import it — one definition;
  3. builds the engine version: existing classes, an Expr, or a new C++ class (in your library when other projects could use it), declared in vargen.py with the same names;
  4. adds a parity step, runs it when you ask, and fixes until it passes; then records the conversion (conversions.json; the notebook shows converted, or notebook changed since after an edit).

Nothing runs until you ask and nothing is committed until you say the change is settled.

Parity

from mtslab import parity
res = parity.check(research_frame, engine_values_csvs, rtol=1e-6)   # per feature: rows, readiness, max diff, corr

It records a table and a chart of the worst feature in the experiment and fails the step when any feature differs. In the leadlag template the pandas features and the engine's values agree on every row to about 1e-10 — so a conversion that passes parity carries the research over exactly.