Docs › MTS Lab

Projects and experiments

A project is a folder of code under git; an experiment is one recorded run of its steps. This page covers both, and the mtslab Python library your scripts use.

• A project
• Steps
• The mtslab library
• Experiments
• Reuse and rerun
• Compare
• Changes and commits
• Where steps run

A project

Each project is ~/MTS Lab/projects/<name>, a git repository you can also open in VS Code or a terminal. What you write is code: Python for the research steps, C++ for strategies and components, JSON for settings. out/ holds what runs write and is never versioned.

leadlag/
  project.json        the steps (below)
  config.py           settings shared by the steps
  vargen.py fit.py sim.py parity.py
  features.py         research features (pandas), imported by the notebook and by parity
  strategies/*.cpp    strategy plugins        variables/*.cpp   variable plugins
  notebooks/*.ipynb   free-form research
  out/exp/<id>/       one folder per experiment

Steps — project.json

{
 "description": "…",
 "libs": {"qlib": "main"},
 "steps": [
  {"name": "vargen", "run": "{python} vargen.py", "doc": "…"},
  {"name": "parity", "run": "{python} parity.py", "needs": ["pandas", "numpy"]},
  {"name": "notebook", "run": "{python} -m mtslab nb run notebooks/leadlag_research.ipynb", "needs": ["nbclient", "nbformat", "ipykernel"]}
 ],
 "requires": {"instruments": ["BTCUSDT.binance"], "data": {"generator": "leadlag", "files": ["ll_day1.cdata"]}}
}

• run is a shell command run in the project's snapshot; {python} is the workspace's Python (click the Python label in the project header to pick your own — a virtualenv, a conda env).
• needs lists Python modules the step imports; when the chosen Python lacks them the step starts unticked with an Install link.
• libs: the libraries the project uses, by version. requires: instruments and generated data the project expects in the workspace (added when missing).

The mtslab library

Your scripts import mtslab (it is on the path in every step, notebook and terminal of the Lab). The essentials:

from mtslab import Cfg, engine, values, fit, out, inp, record, chart, table, series

cfg = Cfg()                                        # instruments from the workspace
mid = cfg.pricing_model("mid_bn", "MidPx", port=["BTCUSDT", "binance"])
s = cfg.sampler("ts_1s", "TimeSampler", msecs=1000, halflife=10)
cfg.variable("trend_bn_10s", "Trend", pm=mid, sampler=s)
cfg.check()                                        # the engine's verdict, or its error message

paths = engine.make_dataset(cfg, engine.data_files("ll_day*.cdata"), out("dataset"))   # values per day
runs = engine.backtest_days(cfg_with_strategy, files, out("model"))                    # one backtest per day
record(r2_oos=0.71, rows=10797)                    # numbers: the experiment's columns
chart("daily P&L", "bar", {"model": [["d1", 3.2]]}, x="day", y="P&L")
x = inp("vargen", "dataset", "features.json")      # an earlier step's output

mtslab.data reads tapes into pandas exactly as the engine replays them; mtslab.parity compares research features with engine values; mtslab.simana turns backtests into daily statistics and charts; mtslab.cdata writes and checks data files. Ask the agent, or run python -m mtslab describe for the engine's classes.

Experiments

Pressing Run experiment (or asking the agent to run) makes an experiment:

• a snapshot of the code as it is — uncommitted and new files included — kept as a hidden git commit, so the run is exactly reproducible and two experiments' code can be diffed;
• the libraries' commits, the engine version, the Python and the data files' hashes;
• per step: the log, the numbers (record), charts and tables, and the outputs — including, for datasets and backtests, each day's generated cfg, engine log and values or fills, browsable day by day;
• a title, an optional hypothesis, and the verdict you give it.

Reuse and rerun

Untick an early step and the experiment reuses its outputs from an earlier experiment (the latest where it succeeded, or one you pick) — refit without regenerating the dataset.

Rerun on an experiment runs its exact snapshot again — same code, libraries, steps, arguments and computer — as a new experiment: all steps, or from a step on reusing the steps before it (after installing a missing package, or fixing data, without touching the code).

Compare

Tick two experiments and press Compare: the code diff between their snapshots (libraries included), the steps and how each was obtained, every metric side by side with the difference, and charts of the same title overlaid.

Changes and commits

Saves — yours in the page, the agent's, edits in VS Code — go to the working copy and show as uncommitted changes with their diffs (History). Commit when the change is settled with Review & commit; a commit after agent edits carries a co-author line. Discard any file's changes, compare a file with an earlier version, revert a commit. Experiments do not need commits: the snapshot keeps the code either way.

Where steps run

Run on picks the computer: this one, or a server you add with servers…:

• ssh — a Linux server you reach with ssh keys; the engine and the code are copied there, outputs stay there and are listed and read on demand (the small files come back).
• Slurm — the steps run as an sbatch job (partition, CPUs, memory, time, account).
• Kubernetes / OpenShift — a Job with an image you choose, the work folder on a PVC or a host path.

Each server has its own data folder; MTSLAB_JOBS sets how many days run at once. See Servers and workspaces.