A container of verification tools for trading systems, one app per venue and component. Each tool is a venue simulator armed with the faults that matter — lost packets, bursts, 2,600-message gaps, terminates, rejects — plus an oracle the code under test cannot influence, and a report that says pass or fail per expectation with the evidence lines. It runs on your host.

Shipped: 23 tools across 11 venues — CME, CFE, BrokerTec, Nasdaq, Nasdaq BX, Nasdaq PSX, NYSE Arca, NYSE, NYSE American, NYSE National, NYSE Chicago, Cboe BZX, Cboe EDGX, Cboe BYX, Cboe EDGA, IEX, MEMX, MIAX Pearl, the CTA and UTP SIPs, Coinbase, Kraken, OKX, Binance, Hyperliquid, dYdX v4 and Injective — a feed-handler and an order-gateway verifier per venue, and drop-copy verifiers for CME and BrokerTec. The launcher is the catalogue, with the reference results per case; planned tools show there muted.

A coding agent will write an MDP 3.0 feed handler or an iLink 3 gateway in an afternoon, and the tests to go with it. The tests share the code's assumptions: when the same author — human or model — reads the spec once and writes both, a misread spec is tested as if it were right. The errors are correlated, and more tests of the same kind do not fix correlation. What fixes it is a different author, a different code base, and an oracle the code under test cannot share.

The oracle is what the verdict is measured against: the venue's own log of what it withheld and served; a truth log of the book the venue actually published after every packet, asserted equal to its internal ladder. The provenance is what the tool was built from, what it found first, and the reference results with their date. Our own systems passing a tool is the reference run, not the guarantee. Silence is measured, not punished: a handler that goes quiet under loss is reported as quiet, never as right.

Every tool was run first against our own feed handlers and gateways — code that had traded, and in CME's case passed AutoCert+. None of these had shown on the live venues. Each is described in its venue section.

• CME — a gateway that had passed AutoCert+ never detected a gap signalled by Sequence.NextSeqNo in a quiet period. §
• BrokerTec — after a snapshot recovery the feed handler pruned its cached packets at one floor per channel instead of per instrument: in market-by-price it re-applied packets a newer instrument's snapshot already held (a duplicated best level for the next ~40 packets), in market-by-order it dropped packets an older instrument still needed (orders missing from its book until they next changed). Never seen in production, where the trader re-requests a snapshot. §
• Nasdaq — the gateway forwarded cancels for orders its own rate limit had rejected; OUCH answers neither, so only the venue could tell. §
• NYSE · Arca — the handler published a partial book after a late join; the gateway reset its sequence on every reconnect, so nothing missed while disconnected was ever recovered. §
• Cboe — the gateway kept alive a New Order the venue never received, and later cancelled a ClOrdID the venue never knew. §
• Coinbase — a 15 s dead session went unnoticed (the receive timeout never fires under a blocking read); a concurrent read and write on one TLS object; acknowledgements ~100 ms late from records parked in a memory BIO. §
• Kraken — the handler logged the book checksum and never checked it, so a lost update left a stale touch for the rest of the run; the gateway skipped the executions snapshot; every REST call took 40–80 ms (Nagle met delayed ACK). §
• Binance — the handler died with SIGPIPE at its first resync and applied a diff on a stale snapshot; the gateway never reconciled after the stream came back, so fills in the gap were lost. §
• OKX — the handler applied every update blindly (no prevSeqId chain); the gateway booked an order as cancelled on the REST accept when it had already filled. §
• Hyperliquid — the handler pinged only on the back of inbound frames; through a quiet market its own watchdog reconnected it twice for nothing. §

mts-toolbox-0.20.tar.gz (82 MB) · sha256 · README. --net host is required (the venue is multicast; your gateway connects to a port), so Docker Desktop on macOS / Windows will not do — use a Linux box or VM. Your code runs on the same host, or on another machine on the LAN with the venue interface set (below).

If reference prints ## md · clean — PASS, the host is fine. The volume keeps run data between commands; -v $PWD:/in lets you submit files from the current directory. Venue interface: multicast goes out on 127.0.0.1 by default (your code joins the groups on loopback). For a client on another machine, run every command with -e CME_IFACE=<this host's LAN IP> and have the client join on its own LAN interface.

What the venue gives you — MDP 3.0, three instruments, depth 10

What you give the tool — the witness

A file your handler writes as it runs — nothing is entered by hand. One JSON line per instrument the packet touched, appended at every event boundary (about 30 lines a second at this venue's rate; a 40 s case is ~1,300 lines), with the book it holds at that moment. You submit the file once:

• seq — the packet sequence number of the incremental whose processing this boundary ends; the venue's truth is keyed by the same number.
• bid / ask — [price, quantity] best first, up to 10 levels.
• A book just rebuilt from the snapshot channel carries "src": "snapshot"; write nothing while you are recovering and publishing nothing.
• Full contract: mtsx contract cme-md-verifier.

How to produce it — 15 lines in your handler

Your handler already has the moment: the end of an incremental packet (EndOfEvent, or simply "packet applied"). There, for each instrument the packet touched, write one line with the packet's MsgSeqNum and that instrument's book. Two helpers do the formatting, and a complete example handler shows the whole shape:

If you cannot edit the handler, a sidecar that reads wherever it publishes its book (shared memory, a socket, a log) and calls the same line() works too; with the MCP server, your coding agent writes this adapter.

Procedure — one case

1. Start the venue for the case in one terminal; it prints the endpoints and runs until Ctrl-C: mtsx venue cme-md-verifier clean
2. Start your handler, pointed at the groups above, writing witness.jsonl as above. Let it run the case's length (40–90 s, shown by mtsx list); stop your handler.
3. Ctrl-C the venue, then submit and judge: mtsx submit cme-md-verifier clean witness /in/witness.jsonl · mtsx report cme-md-verifier clean
4. Read the report — one row per expectation with the evidence. Fix, repeat.

Order of cases: clean first (your handler joins a running session and must build its book from the snapshot channel), then drop_single, gap_burst, duplicates, reorder, and last drop_sustained (a broken channel: quiet is allowed, wrong is not). Your handler's log is optional: mtsx submit cme-md-verifier <case> handler_log /in/md.log.

Market-by-order — mbo_clean, mbo_gap_burst, mbo_duplicates, mbo_reorder

The same venue with the orders behind every level: the 46 carries NoOrderIDEntries (OrderID, MDOrderPriority, MDDisplayQty, ReferenceID → the MBP entry of the order's side and price, OrderUpdateAction New / Update / Delete), and the MBO snapshot channel 239.203.56.3:17927 carries SnapshotFullRefreshOrderBook53 per instrument per second — recover from that channel in these cases. Add your order-level book to each witness line, "orders": {"bid": [[order_id, price, qty, priority], …], "ask": […]}, best price first then priority; the report compares [id, price, qty] per side as a set at the same sequence, and still checks your levels (they must be the aggregation of your orders). The Python example does it with --mbo: mtsx example cme-md-verifier mbo_clean.

Drive the feed yourself — the manual case

The scripted cases run on a timer. When you are debugging you want to decide when things happen: start the venue with no faults, watch your handler settle, inject one fault at a time, mark the phases. The venue has a control channel for that (mtsx ctl, or venue_control over MCP); Ctrl-C, submit and report as usual, and the report adds a per-phase table.

What the venue gives you

A Globex-shaped iLink 3 acceptor (FIXP / SBE) on tcp://<host>:44821, plus the MDP 3.0 incremental feed on 239.203.56.1:17925 for your trader's market data. Any access key and firm are accepted; PartyDetails pre-registration is accepted; instruments as above (ES segment 54, ZN 82, CL 80).

What you give the tool — order flow

The session cases need a client that keeps sending orders while its acks are withheld: the >2,500-message gap needs ~2,600 outbound messages from the venue, so your trader must send steadily (our reference sends IOC limit orders far from the touch at 15/s; a market maker that waits for acks stalls and the gap never forms). Your gateway's log is optional but adds the client-side checks (mtsx contract cme-oe-verifier lists the lines it looks for).

Procedure — one case

1. Start the venue for the case with its duration: mtsx venue cme-oe-verifier gap_gt_2500 150
2. Connect your gateway to :44821, start your order flow, keep it running for the whole duration. The venue plays the scenario (withholds, terminates, rejects, stops keepalives) on its own schedule.
3. When the time is up the venue stops and prints the report, judged from its own log. Optionally add your gateway log and re-run: mtsx submit cme-oe-verifier gap_gt_2500 client_log /in/gw.log · mtsx report cme-oe-verifier gap_gt_2500

Order of cases: clean, rejects, keepalive_lapse, terminate_reconnect, multiple_gaps, realtime_during_resend, gap_gt_2500; then the order cases.

The order cases — order_flow, partial_fills, cancel_fill_race, replace_rejects, unsolicited_cancels

These judge your gateway's order state, not the session. The venue keeps an order truth (every execution report it sent, with the order's state after it) and you submit your order witness — orders.jsonl, one JSON line per order, the last line per ClOrdID counts:

Your order keeper already holds this — dump it at the end of the run — or write it from your gateway's log: examples/order_witness/gw_witness.py does that for BTS2's log and is the shape your coding agent writes for yours. Use an order flow with the whole lifecycle (resting orders that fill, a cancel/replace, cancels). Then mtsx submit cme-oe-verifier partial_fills orders /in/orders.jsonl · mtsx report cme-oe-verifier partial_fills. The checks: every venue order known; status and cum agree; fills booked in full; replaces under the new ClOrdID with CumQty carried over; partially filled orders with the venue's cum; an order whose cancel lost the race to a fill filled, not cancelled; a rejected replace leaving the original standing; an exchange-cancelled order cancelled; and from the venue's side alone, no cancel or replace for an order it closed more than 2 s earlier (the in-flight race is counted separately and allowed).

Drive the acceptor yourself — the manual case

As for the feed: mtsx venue cme-oe-verifier manual, connect your gateway and order flow, then from another terminal (mtsx ctl cme-oe-verifier manual …) or the GUI:

What the venue gives you

A Drop Copy 4.0 (MSGW) server on tcp://<host>:44803: FIX 4.2, any SenderCompID / access key / HMAC signature accepted, the Logon answered with your HeartBtInt, sequence numbers kept across connections, ResendRequest served from a store (PossDupFlag=Y replay, SequenceReset-GapFill for admin messages), and the venue's own ResendRequest when your side is ahead. The stream is the encapsulated 35=n with the embedded 35=8 (New, Cancel, Fill, Reject) for the venue's own order flow across ESZ6 / ZNZ6 / CLZ6 at 5 orders/s — and for anything your gateway sends on the iLink 3 port :44821.

What you give the tool

Nothing but a logged-on client: the venue sees every session message you send — Logon, ResendRequest and its range, GapFill, TestRequest, the Logout confirmation, the new Logon — and judges from that. Your client's log is optional (mtsx contract cme-dc-verifier lists the lines it looks for).

Procedure — one case

1. Start the venue for the case with its duration: mtsx venue cme-dc-verifier gap_resend 40
2. Log your client on to :44803 and leave it running for the whole duration. The venue plays the scenario (withholds, logs you out, goes silent, asks you for a resend) on its own schedule.
3. When the time is up the venue stops and prints the report. Optionally add your log and re-run: mtsx submit cme-dc-verifier gap_resend client_log /in/dc.log · mtsx report cme-dc-verifier gap_resend

Order of cases: clean, bow_logon (log on with ResetSeqNumFlag=Y), midweek_logon (the venue is ahead: ResendRequest, accept the GapFill), gap_resend, multiple_gaps, realtime_during_resend, venue_resend_request (answer with a GapFill), venue_logout (confirm, log on again with the right sequence, recover what you missed), heartbeat_lapse (TestRequest after 2 × HeartBtInt, reconnect when it goes unanswered).

Drive it yourself — the manual case

BrokerTec's US Treasury markets run on CME Globex: the same MDP 3.0, iLink 3 and Drop Copy 4.0 the CME tools verify, on the fixed-income shape — security definitions as MDInstrumentDefinitionFixedIncome57, prices in decimal on the venue's tick ladder (1/256 for the 2y and 5y, 1/128 for the 10y, 1/64 for the 30y), quantity in $1mm lots, instruments keyed by ISIN on the order side. The three tools run the CME procedures above against a venue in that shape; the reference clients are the desk's own BrokerTec handler, gateway and drop-copy client — the code that traded on the venue. Case references are to BrokerTec's own AutoCert+ suites (the venue certifies market data, order entry and drop copy separately).

Feed handler — btec-md-verifier

The witness is the CME contract keyed by the packet sequence, prices as decimals (99.53125, not 99-17). Order of cases: clean (join a running session: definitions and the book from the snapshot channel — MD case 1 / 3), drop_single, gap_burst (recovery — MD case 4), duplicates, reorder, drop_sustained; then mbo_clean, mbo_gap_burst (Premium MBOFD book management and recovery from the order-book snapshot — MD cases 3 / 4), mbo_duplicates, mbo_reorder; manual with the feed's controls on udp://127.0.0.1:14495. Not played here: channel reset (MD case 5), the statistics messages (6), TCP replay recovery (7) and workup states. The first run found two recovery defects in the reference handler (see what it found) — the same fixes its CME twin had received a week earlier; both fixed.

Order gateway — btec-oe-verifier

A Globex-shaped iLink 3 acceptor (FIXP / SBE) on tcp://<host>:44784 — any access key, firm and session; PartyDetails pre-registration accepted — with the four actives above (ISINs USBENCH002Y1, USBENCH005Y2, USBENCH010Y3, USBENCH030Y4; MarketSegmentID 40) and the MDP 3.0 feed for your trader's market data; the acceptor fills against the feed's touch. The procedure and the order witness are the CME gateway's (cl = ClOrdID, quantity in lots, price as the venue quotes it); examples/order_witness/gw_witness.py reads BTS2's iLink 3 log for either venue.

Not played here: FAK / minimum-quantity / display-quantity orders (cases 12, 15, 16), Order Mass Action (18) and Order Status requests (19) — the acceptor answers them but the report does not yet judge them — self-match prevention (21) and the Chicago fractional book.

Drop copy — btec-dc-verifier

The Drop Copy 4.0 (MSGW) server on tcp://<host>:44785, FIX 4.2, the encapsulated 35=n stream of the venue's own order flow on the four actives plus anything your gateway sends on :44784. The procedure is the CME drop copy's; the cases carry BrokerTec's numbering: clean (logon and the encapsulated New / Fill reports — cases 1, 8), bow_logon (2), midweek_logon (3), gap_resend (5, respond to resend), venue_resend_request (6, bi-directional), realtime_during_resend (7), multiple_gaps, venue_logout, heartbeat_lapse; manual on udp://127.0.0.1:14497. Not played here: the Order Mass Action Report (case 4), mid-week key rotation (9) and logon with failover (10).

The same two procedures on the Nasdaq protocols, against one venue (one order book behind both sides).

BX and PSX. Nasdaq's three exchanges run the same TotalView-ITCH 5.0 over MoldUDP64 and OUCH 4.2 over SoupBinTCP; what differs is the endpoints and the session. bx-* and psx-* are the one Nasdaq source built for each exchange (bx_*, psx_*: its own feed type and session), venue and reference clients alike, on each exchange's own bench endpoints: BX multicast 233.54.20.121:26587 · retransmission 26588 · OUCH 27120 · session MTSBX0001 · control 26590 / 26591; PSX 233.54.21.121:26597 · 26598 · OUCH 27130 · session MTSPS0001 · control 26600 / 26601. The procedure, the witness and the cases are Nasdaq's.

Feed handler — nsdq-md-verifier

The witness is the CME shape keyed by the MoldUDP64 sequence of the message just applied, secid = locate, prices as decimals — one line per symbol whose top-10 changed, after applying the message:

Order of cases: clean (the late join), drop_single, gap_burst, duplicates, reorder, drop_sustained (a bad line: keep recovering, never publish a wrong book); manual with pause · resume · drop N · dup N · reorder N · rate MS · mark · stats on udp://127.0.0.1:26580. If your handler publishes its book over a shared-memory or socket contract, start the witness writer before the handler so a late-join recovery is not missed.

Order gateway — nsdq-oe-verifier

OUCH 4.2 over SoupBinTCP 3.0 on tcp://<host>:27100: any user / password; the login's requested sequence number is honoured — the venue keeps the sequenced stream per user and replays from it, which is what the session cases test:

The venue also counts cancels for tokens it never saw or closed more than 2 s earlier (stale_cancels): OUCH answers neither, so only the venue can tell you — the first run caught the reference gateway forwarding cancels for orders its own rate limit had rejected. examples/order_witness/nsdq_gw_witness.py builds orders.jsonl from BTS2's gateway log.

One venue per exchange (the same Pillar code), one order book behind both sides. NYSE endpoints below; Arca's are the same shape on 233.54.13.121:26787 · recovery .122:26788 · request tcp 26789 · FIX 27110 · control 26790 / 26791, TargetCompID ARCA.

NYSE American, National and Chicago. All five Pillar markets run the same XDP Integrated Feed and Pillar FIX; what differs is the endpoints, the symbol universe and the CompIDs. amex-*, nsx-* and chx-* are the one Pillar source built for each market (amex_*, nsx_*, chx_*: its own XDP market id, exchange code and CompID), venue and reference clients alike, on each market's own bench endpoints: American XDP 233.54.22.121:26937 · recovery .122:26938 · request tcp 26939 · FIX 27220 (TargetCompID AMEX) · control 26940 / 26941; National 233.54.23.121:26947 · 26948 · 26949 · FIX 27230 (NSX) · 26950 / 26951; Chicago 233.54.24.121:26957 · 26958 · 26959 · FIX 27240 (CHX) · 26960 / 26961. The procedure, the witness and the cases are NYSE's.

Feed handler — nyse-md-verifier

The witness is the CME shape keyed by the XDP stream sequence of the message just applied, secid = the symbol index; a book rebuilt from a refresh goes under the refresh packet's sequence minus one, and nothing should be published for a symbol before its refresh has landed — the first run caught the reference handler publishing a partial book after a late join.

Order of cases: clean (the late join), drop_single, gap_burst, duplicates, reorder, drop_sustained; manual with pause · resume · drop N · dup N · reorder N · rate MS · mark · stats on udp://127.0.0.1:26980.

Order gateway — nyse-oe-verifier

Pillar FIX (FIX 4.2) on tcp://<host>:27210, TargetCompID NYSE, any SenderCompID. The venue keeps every outbound message per SenderCompID and serves ResendRequest with PossDupFlag=Y replay and SequenceReset-GapFill; a Logon without ResetSeqNumFlag continues the sequence stream — which is what the session cases test:

The first run found the reference gateway resetting its sequence on every reconnect — nothing missed while disconnected was ever recovered — and forwarding cancels for orders its own rate limit had rejected; both fixed. examples/order_witness/pillar_gw_witness.py builds orders.jsonl from BTS2's gateway log.

One Cboe source built twice: BZX for equities (AAPL · MSFT · SPY, shares), the Cboe Futures Exchange for VIX futures (VXZ6 · VXF7 on the 0.05 tick, contracts, with the PITCH Futures Instrument Definition and Settlement messages). One order book behind both sides. BZX endpoints below; CFE's are the same shape on 233.54.17.121:26887 · gap group .122:26888 · gap/spin server tcp 26889 · BOE 27410 (session sub id 0003) · control 26890 / 26891.

The four equity exchanges. Cboe's BZX, EDGX, BYX and EDGA run the same Multicast PITCH and BOE specifications; what differs is the endpoints, the matching-unit numbers and the session identity. The tools follow suit: one Cboe source built once per exchange (bzx_*, edgx_*, byx_*, edga_*: its own feed type, session sub id and order-id space), venue and reference clients alike, on each exchange's own bench endpoints. The procedure, the witness and the cases below are the same for all four; only the endpoints change:

Feed handler — bzx-md-verifier (and edgx- / byx- / edga-md-verifier)

The witness is the CME shape keyed by the unit sequence of the PITCH message just applied — the packet header's sequence plus the message's index in the packet; after a spin, one line per symbol at the spin's sequence.

Order of cases: clean (the late join by spin), drop_single, gap_burst, duplicates, reorder, drop_sustained; manual with pause · resume · drop N · dup N · reorder N · rate MS · mark · stats on udp://127.0.0.1:26880.

Order gateway — bzx-oe-verifier (and edgx- / byx- / edga-oe-verifier)

BOE v2 on tcp://<host>:27310, session sub id 0001, any username / password. The venue keeps every sequenced message per session and matching unit; the Login Request's Unit Sequences group names the last sequence you received, the venue replays from there and sends Replay Complete — which is what the session cases test:

The first run found the reference gateway keeping alive a New Order the venue never received, so the trader later cancelled a ClOrdID the venue never knew — fixed (reported lost on re-login; a cancel for a ClOrdID never sent is answered locally). examples/order_witness/boe_gw_witness.py builds orders.jsonl from BTS2's bzx_gw / cfe_gw log.

IEX's own protocols, built from the exchange's current documents (IEX-TP v1.25, DEEP v1.08, DEEP SNAP v1.6, IEX FIX v3.11): a price-level depth feed over IEX-TP multicast with the transport's Gap Fill Request for holes and DEEP SNAP (TCP) for the late join, and FIX 4.2 order entry. One venue (iex_me) behind both sides. The first run of the bench found no defect in the reference handler, one in the bench's own witness binary (it predated the IEX feed type and silently dropped every frame) and one in the venue simulator (a reconnected SenderCompID could not cancel its pre-disconnect orders) — both fixed.

Feed handler — iex-md-verifier

The witness is keyed by the IEX-TP sequence of the Price Level Update that completed the event (Event Flags = 1), secid = locate, prices as decimals — one line per symbol whose top-10 changed; after a DEEP SNAP rebuild write the books at the snapshot's sequence.

Order of cases: clean (the late join from DEEP SNAP), drop_single (one segment in 150 lost: a Gap Fill Request must close it), gap_burst (20 segments), duplicates, reorder (A/B line shapes: by sequence number, not re-applied), drop_sustained (a segment lost every 10 for the whole run: keep recovering, never publish a wrong book); manual with pause · resume · drop N · dup N · reorder N · rate MS · mark · stats on udp://127.0.0.1:25490.

Order gateway — iex-oe-verifier

IEX FIX v3.11 (FIX 4.2) on tcp://<host>:28510: Logon with any SenderCompID, TargetCompID IEXG, 108 HeartBtInt required, 141 ResetSeqNumFlag honoured; New Order Single D (18=i, 40 1/2, 59 0/3/4), Order Cancel Request F, Cancel/Replace G; Execution Report 8 / Order Cancel Reject 9. The venue keeps a replayable stream per SenderCompID and serves Resend Requests with PossDupFlag and Sequence Reset-GapFill — the session cases test exactly that. There is no cancel-on-disconnect in the specification (it is a port attribute), so a dropped session keeps its resting orders:

examples/order_witness/iex_gw_witness.py builds orders.jsonl from BTS2's gateway log.

MEMX's own protocols — MEMOIR Depth v2.0 over MEMX-UDP v1.1 in, MEMO SBE v2.0 over MEMX-TCP v1.2 out, with the June 2026 errata — against one venue (memx_me). Provenance: the documents used are the ones MEMX publishes through LTSE (the Long-Term Stock Exchange runs on MEMX technology; they carry "used under license"); MEMX's own editions sit behind its member portal and were not available, so venue-configured values (session id, user, MPID, timings, the price grid) are bench choices, and two table typos were resolved by the documents' own hex dumps. The first run found two defects: the reference handler's gap grace timer started from the last applied message rather than from gap detection, so a two-second outage made it skip ahead and discard its own replay; and the simulator armed a heartbeat lapse before the client held any sequence. Both fixed; no gateway defect.

Feed handler — memx-md-verifier

The witness is keyed by the MEMX-UDP sequence of the MEMOIR message just applied, secid = SecurityID, prices as decimals (the mantissa divided by 1e6) — one line per security whose top-10 changed:

Order of cases: clean (the late join, replayed from 1), drop_single, gap_burst (20 datagrams: hold the grace from the moment the gap is seen), duplicates, reorder, drop_sustained; manual with pause · resume · drop N · dup N · reorder N · rate MS · mark · stats on udp://127.0.0.1:25590.

Order gateway — memx-oe-verifier

MEMO SBE v2.0 over MEMX-TCP v1.2 in Stream mode on tcp://<host>:28610: Login Request with token type P user:password (any), Start of Session (session 1), a Stream Request from your next expected sequence is honoured and replayed; NewOrderSingle / OrderCancelRequest / OrderCancelReplaceRequest / MassCancelRequest as Unsequenced Messages, execution reports as Sequenced Messages. Unlike OUCH, a cancel that finds nothing is answered: OrderCancelReject TooLateToCancel inside the in-flight window, UnknownOrigOrder otherwise. Cancel-on-disconnect is on (CancelReason 13; the reports wait on your stream and replay on re-login):

examples/order_witness/memx_gw_witness.py builds orders.jsonl from BTS2's gateway log.

MIAX's own protocols, built from the exchange's current documents — Depth of Market feed 1.3.d over MACH 1.2e in, MEO 2.7.b over ESesM 1.0.a out, with the Port Attributes and Liquidity Indicator Codes documents — against one venue (miax_me). The first run found two defects in the reference handler (its retransmission client judged a gap fill or refresh at the TCP close before parsing the packets already received — the interface sends the data, GoodBye, then disconnects — so every recovery "failed"; and the gap counter ticked once per packet behind a hole instead of once per hole) and two in the simulator's bench bookkeeping. All fixed; no gateway defect.

Feed handler — miax-md-verifier

The witness is keyed by the MACH sequence of the DoM message just applied, secid = Symbol ID, prices as decimals (Prc6U divided out) — one line per symbol whose top-10 changed; a book rebuilt from the Order Book Refresh is written under the refresh's sequence.

Order of cases: clean (the late join from the Order Book Refresh), drop_single, gap_burst, duplicates, reorder, drop_sustained (keep recovering over the retransmission interface, never publish a wrong book); manual with pause · resume · drop N · dup N · reorder N · rate MS · mark · stats on udp://127.0.0.1:25390.

Order gateway — miax-oe-verifier

MEO 2.7.b over ESesM 1.0.a on tcp://<host>:28710: login l (version 1.0, any username / computer ID, application protocol MEO, one matching engine), the requested sequence per matching engine is honoured — replay then Synchronization Complete; N1 New Order / M1 Modify / CO Cancel; NR / MR / CR responses (sequenced only when successful), O1 / P1 / E1 / XN / MN notifications, SN system state and SU symbol updates on a fresh stream. The bench session elects Auto Cancel on Disconnect = Cancel All Open Orders (port attribute 7; the venue default is Do Not Cancel):

examples/order_witness/miax_gw_witness.py builds orders.jsonl from BTS2's gateway log.

The other SIP, on the Pillar output: built from the CTA plan's own documents — CQS and CTS Pillar Output v2.11b, the Common IP Multicast Distribution Network specification, the Retransmission and Snapshot User Guide v1.8 and the CQS Pillar Snapshot v3.1 — against one simulator (cta_me) that plays both lines, seven participants (Nasdaq's CTA code is T, not Q), the FINRA ADF and the TRFs, the request server and the retransmission group. The first run found five defects, all in the simulator or the handler's fine print: a block numbered from the wrong message, a band change applied one republished quote at a time so the NBBO briefly sat outside the band, a republished quote losing its original time priority, a tie unbreakable after a snapshot (the handler now adopts the SIP's ranking), and stale quotes crossing the touch. All fixed.

What the witness holds: the same as for UTP — every participant's protected quote aggregated by price, level 1 = the National BBO (Appendix F rules: sizes floored to the round lot, the NBBO appendage cross-checked).

Cases: the six transport cases per line (clean, drop_single, gap_burst, drop_sustained, duplicates, reorder) and the five tape cases: luld_bands (bands republished while the market trends; every band logged, the NBBO inside the band in force, the SIP's republished quotes taken), halt_resume (the listing market halts JPM, every participant clears, price indications during the halt are seen and ignored, then the resume), off_exchange_prints (ADF / TRF prints as kind 1, never moving the NBBO), trade_corrections (each T/X and T/C logged with its reference number), mwcb (M/L on both lines, every symbol halted and resumed); manual with pause · resume · drop N · dup N · reorder N · rate MS · halt SYM · resume SYM · band SYM · mwcb LEVEL · trf N · cancel N · correct N · mark · stats on udp://127.0.0.1:25290.

A third kind of tool: tape. The SIP has no order entry; the question is whether your consolidated-feed handler holds the National BBO, the LULD bands, the halts and every off-exchange print the way the plan defines them. Built from the UTP plan's own documents — the Data Feed Services Specification v4.1 (UQDF quotes and UTDF trades in the binary output over MoldUDP64) and UTP Snap-Shot v4.0 over SoupBinTCP — against one simulator (utp_me) that plays seven participants, the FINRA ADF and the TRFs. The first run found a real defect in the desk's trader, not in the handler: a started strategy quoting a symbol whose exchange has no venue adapter dereferenced a null symbol trader and crashed bts_trader — fixed the same day with a local reject.

What the witness holds. Per symbol the SIP-eligible protected quote of every participant, aggregated by price: level 1 of each side is the National BBO (price and the total size at it), levels 2+ the other participants' prices. The truth is the simulator's consolidated book plus the NBBO it put in the appendage; both must match at every packet boundary.

Cases: the six transport cases (clean late join from the Snap-Shot, drop_single, gap_burst, drop_sustained, duplicates, reorder) and five tape cases judged from your handler's log against the truth: luld_bands (every band logged, the NBBO inside the band in force), halt_resume (a Cross SRO halt: status H, then Q, then T; quotes during the halt leave the NBBO blank), off_exchange_prints (ADF and TRF prints published as kind 1, as many as the tape carried, never moving the NBBO), trade_corrections (each cancel / error and correction logged with the original trade id), mwcb (a level-1 breach: the status message, every symbol halted MWC1 and resumed MWCQ); manual with pause · resume · drop N · dup N · reorder N · rate MS · halt SYM · resume SYM · band SYM · mwcb LEVEL · trf N · cancel N · correct N · mark · stats on udp://127.0.0.1:25190.

The first crypto venue. One simulator written from Coinbase's published WebSocket-feed and FIX 4.2 order-entry documentation serves both sides behind one order book (BTC-USD, ETH-USD; a random walk in basis points, synthetic takers; your resting orders show in the level2 book). No sequence on level2, no retransmission, no FIX resend — this venue's recovery story is reconnect and resubscribe, and the tools time it.

Feed handler — coinbase-md-verifier

The witness is the CME shape keyed by the l2update's time as UTC ns: one line per l2update after all of its changes, nothing for the snapshot itself (the first l2update after it is judged); secid is yours, the report joins on symbol + time. The reference witness (cts_witness) rebuilds the book from the handler's shared-memory ring exactly as a strategy would.

Order of cases: clean, disconnect (the venue closes the socket at 20 s: reconnect, resubscribe, right again within 5 s), silence (a dead socket for 40 s — no frames, not even heartbeats: detect it, reconnect, right again within 45 s), burst (one l2update of 400 changes), churn (20 ms batches); manual with pause · resume · ws_disconnect · ws_silence MS · burst N · rate MS · mark · stats on udp://127.0.0.1:27602.

Order gateway — coinbase-oe-verifier

FIX 4.2 over TLS on tcp://<host>:27601 (a self-signed certificate — the reference client does not verify the peer): Logon with 141=Y, a 96 signature (accepted as given) and 8013=Y; a UUID ClOrdID; ExecutionReports 150=0/1/2/4/8; OrderCancelReject; TestRequest. A ResendRequest gets a session Reject — there is no resend on this venue, which is what the session cases are about:

The first run found three things in the reference gateway: a 15 s dead session went unnoticed (its SO_RCVTIMEO guard never fires under asio's blocking read — now a TestRequest after 1.5× and a reconnect after 2.5× HeartBtInt); a bad record mac from a concurrent SSL_read / SSL_write on one SSL object; and acknowledgements arriving ~100 ms late (TLS records parked in asio's memory BIO, invisible to poll). None of them had shown on the live venue at 30 s heartbeats and one order a second. examples/order_witness/coinbase_gw_witness.py builds orders.jsonl from coinbase_gw's log.

One simulator written from Kraken's published WebSocket v2 and REST documentation: the v2 server (public channels and the private executions channel on one socket) and HTTPS REST with a self-signed certificate (Time · GetWebSocketsToken · AddOrder · CancelOrder · QueryOrders; API-Key / API-Sign accepted as given), one order book per pair (BTC/USD tick 0.1, ETH/USD tick 0.01).

Feed handler — kraken-md-verifier

The witness is the CME shape keyed by the update's timestamp as UTC ns: one line per update after applying and truncating; nothing for the snapshot itself. Order of cases: clean, disconnect, silence, lost_update (one update per pair is not sent — it carries a jump of the touch — and only the next checksum says so: notice it, resubscribe, right again within 5 s), burst, churn; manual with pause · resume · ws_disconnect · ws_silence MS · lost_update · burst N · rate MS · mark · stats on udp://127.0.0.1:27702. The first run found the reference handler logging the checksum and never checking it: a lost update left a stale touch for the rest of the run — fixed.

Order gateway — kraken-oe-verifier

Signed REST on https://127.0.0.1:27701 (turn certificate verification off for the bench): AddOrder → txid; CancelOrder by txid → count, or EOrder:Unknown order; QueryOrders for an order's fate. A GetWebSocketsToken token opens the executions channel: a snapshot of your open orders (with cum_qty), then new / trade / canceled updates with cl_ord_id.

The first run found two things in the reference gateway: it skipped the executions snapshot, so anything executed while the socket was down was never booked; and every REST call took 40–80 ms (httplib writes headers and body separately; Nagle met delayed ACK — TCP_NODELAY, 1 ms now), and the stale quotes that latency produced had been showing up as post-only refusals. examples/order_witness/kraken_gw_witness.py builds orders.jsonl from kraken_gw's log.

One simulator written from Binance's published WebSocket-streams, WebSocket-API, user-data-stream and REST documentation (the spot docs as of 2026): the combined stream and the WebSocket API on one plain-ws port, HTTPS REST with a self-signed certificate (the API key and signature accepted as given), one order book per symbol (BTCUSDT, ETHUSDT, tick 0.01) where every level change gets an update id, a ping frame every 20 s. Until 0.9 this pair was the Binance.US edition with the listenKey user stream — Binance deprecated that stream 2025-04-07 and removed it from the spot documentation 2025-10-24.

Feed handler — binance-md-verifier

The witness is the CME shape keyed by the update's final update id u, one line per applied update. Order of cases: clean, drop_depth (one diff per symbol not sent — see the gap, re-snapshot, right again within 5 s), stale_snapshot (/depth answers a cached snapshot ~4 s behind the stream for 6 s — fetch again until it is not), disconnect, silence, burst; manual with pause · resume · ws_disconnect · ws_silence MS · drop_depth · stale_snapshot MS · burst N · rate MS · mark · stats on udp://127.0.0.1:27802. The first run found the reference handler dying with SIGPIPE at its first resync (the venue had closed the idle REST keep-alive socket — no CTS process ignored the signal; fixed in all of them) and applying the triggering event on top of a stale snapshot without re-checking continuity.

Order gateway — binance-oe-verifier

order.place / order.cancel on ws://127.0.0.1:27800/ws-api/v3 (or signed REST on 27801). The user data stream is a subscription on the WebSocket API connection: userDataStream.subscribe after an Ed25519 session.logon, or userDataStream.subscribe.signature (apiKey + timestamp + signature, any key type) → a subscriptionId; executionReports (x NEW / TRADE / CANCELED / EXPIRED, z the cumulative quantity) then arrive on that connection as {"subscriptionId", "event"}; eventStreamTerminated ends a subscription. Binance replays nothing: what executed while the subscription was down is found only by asking.

The first runs found the reference gateway never reconciling after the stream came back (fills in the gap were lost for good), reconnecting with a stale credential, and letting a cancel race the order.place response — all fixed (reconcile on every subscription, a fresh subscription per connect and after a termination, cancels parked until the acknowledgement). examples/order_witness/binance_gw_witness.py builds orders.jsonl from binance_gw's log.

One simulator written from OKX's published v5 WebSocket and REST documentation: /ws/v5/public and /ws/v5/private on one plain-ws port, HTTPS REST with a self-signed certificate (the OK-ACCESS-* headers accepted as given), one order book per instrument (BTC-USDT tick 0.1, ETH-USDT tick 0.01; levels as [px, sz, "0", n] strings), the literal ping / pong, the 30 s rule.

Feed handler — okx-md-verifier

The witness is the CME shape keyed by the update's seqId, at depth 20. Order of cases: clean, disconnect, silence (no pong either — your read timeout has to notice), lost_update (one update per instrument withheld: see the gap, resubscribe, right again within 5 s), seq_reset (apply it — it is not a gap), quiet_market (65 s: the heartbeat arrives, ping or be closed under the 30 s rule, stay on one connection), churn (20 ms); manual with pause · resume · ws_disconnect · ws_silence MS · lost_update · seq_reset · quiet MS · burst N · rate MS · mark · stats on udp://127.0.0.1:28002. The first run found the reference handler applying every update blindly: a lost update left a stale touch in its book for the rest of the run and nothing noticed. It now chains prevSeqId, resubscribes on a gap, skips the heartbeat and applies a reset.

Order gateway — okx-oe-verifier

Signed REST order entry on https://127.0.0.1:28001 (POST /api/v5/trade/order, cancel-order; {"code":"0","data":[{ordId, clOrdId, sCode:"0"}]} or code "1" with sCode / sMsg per order); login then the orders channel on ws://127.0.0.1:28000/ws/v5/private, which pushes nothing on subscribe. A cancel's sCode 0 means the request was accepted — not that the order is cancelled: the result comes on the channel (or from GET /api/v5/trade/order).

The first run found the reference gateway publishing CANCELED on the REST accept (an order that filled first was booked as cancelled) and never reconciling after the private socket came back — both fixed (the channel decides the cancel; reconcile on every subscribe). examples/order_witness/okx_gw_witness.py builds orders.jsonl from okx_gw's log.

One simulator written from Hyperliquid's published WebSocket API documentation: one plain-ws endpoint with subscribe / unsubscribe / ping / post, the l2Book, trades, bbo and activeAssetCtx channels, the error channel, and the venue's rule that a connection it has not received a message from in 60 s is closed. One order book per coin (BTC tick 1, ETH tick 0.1; prices and sizes as strings, 20 levels per side).

Feed handler — hyperliquid-md-verifier

The witness is the CME shape keyed by the snapshot's time (ms × 1 000 000), one line per snapshot, the first after a subscribe included, at depth 20. Order of cases: clean, disconnect, silence (no heartbeat channel exists — your read timeout has to notice), quiet_market (the market stops for 75 s: stay on one connection, keep pinging, be right when it moves again), thin_book (the ask side thins to 3 levels, then to none — ghost levels show), churn (20 ms); manual with pause · resume · ws_disconnect · ws_silence MS · quiet MS · thin MS · rate MS · mark · stats on udp://127.0.0.1:27902. The first run found the reference handler pinging only when a frame arrived: through the quiet market its own 30 s read watchdog reconnected it twice for nothing (each fresh connection restarting the venue's idle clock — with a longer watchdog the venue would have closed it instead). It now pings every 20 s on a timer, and the pong feeds the watchdog.

Order gateway — hyperliquid-oe-verifier

Hyperliquid has no session and no API key: every POST https://127.0.0.1:27901/exchange request is a signed L1 action and the venue identifies the account by recovering the signer — keccak256(msgpack(action) ‖ nonce ‖ 0x00) under an EIP-712 Agent (domain Exchange / 1 / chainId 1337), secp256k1 with RFC 6979. The bench venue recovers signers with the same code, so a wrong byte anywhere in what was signed answers the docs' L1 error: User or API Wallet 0x… does not exist. — this is the one bench that cannot accept a signature as given. The nonce rules apply (strictly larger than the smallest of the 100 highest per signer, never reused, inside the window).

The reference gateway was built for this tool: until 0.10 it was a scaffold that shelled out to a Python signer per order and had no fill path. It now signs natively — byte-identical to the official SDK (tests/hl_sign_equivalence.py in CTS diffs the msgpack bytes, the action hash and r / s / v against the SDK on the same key, action and nonce) — keeps nonces strictly increasing, books fills from both channels against a watermark and reconciles by cloid on every subscription. examples/order_witness/hyperliquid_gw_witness.py builds orders.jsonl from its log.

One simulator written from the indexer's own source (dydxprotocol/v4-chain, indexer/services/socks: the message builders and their exact texts, message_id bumped on every message sent, the 2-per-second subscribe limit per channel + id and its 1008 close, the 30 s ping / 10 s pong heartbeat) and the docs' orderbook pages. BTC-USD (tick 1) and ETH-USD (tick 0.1).

Feed handler — dydx-md-verifier

The witness is the CME shape keyed by the message's message_id, at depth 20. Order of cases: clean, disconnect, silence, drop_message (one channel_data per market withheld, its message_id consumed — see the gap, resubscribe, right again within 5 s), cross (for 8 s the levels that took the touch come now and the deletions of the levels they crossed one message later — 88 crossed updates in the reference run, 0 wrong books for a handler that uncrosses, 88 for one that does not), burst, churn; manual with pause · resume · ws_disconnect · ws_silence MS · drop_message · cross MS · burst N · rate MS · mark · stats on udp://127.0.0.1:28102. The first run found the reference handler never checking message_id: a lost frame left a stale level in its book for the rest of the run and nothing noticed. It now expects every message_id in sequence and resubscribes on a gap. The order gateway is planned (a Cosmos SDK transaction: MsgPlaceOrder with goodTilBlock ≤ height + 20, native signing as for Hyperliquid).

Order gateway — dydx-oe-verifier

Order entry on dYdX v4 is a Cosmos SDK transaction: MsgPlaceOrder / MsgCancelOrder in a TxBody, an AuthInfo with the secp256k1 pubkey and SIGN_MODE_DIRECT, a signature over sha256(SignDoc{body, auth_info, chain_id, account_number}), broadcast as base64 TxRaw to POST https://127.0.0.1:28101/cosmos/tx/v1beta1/txs. The venue decodes it, verifies the signature against the pubkey in its AuthInfo and requires the derived address (bech32 dydx, ripemd160(sha256(pubkey))) to own the order — a wrong byte anywhere in what was signed is signature verification failed. The account (number, sequence) comes from GET /cosmos/auth/v1beta1/accounts/<address>; the units from GET /v4/perpetualMarkets.

The reference gateway was built for this tool: until 0.12 it was a scaffold that shelled out to a Python signer per order and had no fill path. It now builds and signs the transaction natively — byte-identical to the official dydx-v4-client (tests/dydx_tx_equivalence.py in CTS is the proof) — and books fills from the indexer against a per-order watermark.

One simulator written from the chain's public interfaces as a feed handler meets them: the Tendermint RPC WebSocket (JSON-RPC 2.0 subscribe with tm.event='NewBlock', 5 subscriptions per client), the gRPC-gateway LCD (the exchange module's orderbook query as chain Dec strings, with the Grpc-Metadata-X-Cosmos-Block-Height header every answer carries) and the indexer trades listing (every fill twice: the taker and the maker record). A block every 700 ms; BTC/USDC PERP and ETH/USDC PERP.

Feed handler — injective-md-verifier

The witness is the CME shape keyed by the served height, in human units, at depth 20. Order of cases: clean, disconnect (the block stream closes; the chain goes on), stream_silence (NewBlock stops for 20 s while blocks keep coming: poll on the timer, key by the served height — a handler that waits for the announcement freezes), lcd_lag (the node is one block behind for 8 s: its header says so), lcd_down (503 for 6 s), halt (no blocks for 12 s: no phantom heights), churn (200 ms blocks); manual with pause · resume · ws_disconnect · ws_silence MS · lcd_lag MS · lcd_down MS · halt MS · block MS · mark · stats on udp://127.0.0.1:28202. The first run found two things in the reference handler: it keyed the book by the announced height and never read the LCD's height header (a lagging node gave a stale book a fresh height), and when the block stream died its timer fallback polled but never published (the announced height had not changed, so the poll was skipped) — the book froze. Both fixed. The order gateway is not planned yet: Injective's fills come from indexer gRPC streams that the reference stack has no client for.

Order gateway — injective-oe-verifier

Order entry on Injective is a Cosmos SDK transaction — MsgCreateDerivativeLimitOrder / MsgCancelDerivativeOrder (injective.exchange.v2, human-readable units × 10¹⁸ on the wire) — signed with an Ethereum-style key: the pubkey travels as /injective.crypto.v1beta1.ethsecp256k1.PubKey, the signature is secp256k1 over keccak256(SignDoc), the address is bech32 inj of the Ethereum address, the subaccount that address plus a 12-byte index; broadcast as base64 TxRaw to POST https://127.0.0.1:28201/cosmos/tx/v1beta1/txs. The venue verifies the signature, requires the signer to own the subaccount, the sequence to be the mempool's next (committed + the transactions already accepted this block — account sequence mismatch, expected N, sdk 32 otherwise) and the fee to cover gas × 160000000 inj.

The reference gateway was built for this tool: until 0.15 it was a scaffold that shelled out to a Python signer per order and had no fill path. It now builds and signs the transaction natively — byte-identical to the official injective-py, the RFC 6979 nonce on HMAC-keccak256 included (tests/inj_tx_equivalence.py in CTS is the proof) — follows every transaction up with the tx query, and books fills from the chain stream against a per-order watermark.

TRACE is not a venue: FINRA members report every corporate-bond trade and FINRA disseminates the reports on the Bond Trade Dissemination Service. The tool is built from FINRA's public BTDS 2.1 (MOLD/UDP 64) specification, effective 2024-10-21, carried by Nasdaq's MoldUDP64 1.00 — both public downloads, which is why this bench is in the catalogue where the credit venues' dealer lanes are not. One simulator (trace_ds) disseminates the tape byte for byte and keeps the issues' high / low / last by the specification's sale-condition matrix; its truth log is the oracle. The first run found the venue's end-of-session packets going out as heartbeats (the packet builder rewrote the count on send) — fixed; the reference reader was right.

The witness is one line per message your reader applied — the fields as decoded and the issue's day state after it, kept per Section 8.3.8 (move the fields the Change Indicator names to the trade's price, or to the Summary's on a cancel and a correction). The report joins on CUSIP + trade id; seq is welcome, not needed.

The reference reader is the credit demo's trace_md (rebuilt to the specification from a to-shape prototype for this tool); trace_witness writes the witness from its shared-memory channel.

Every handler and gateway is wired differently, so the one piece a tool needs from you — your book at each boundary, your gateway's log — is different every time. Register the toolbox in your agent, then ask it: "Verify my MDP 3.0 handler with the mts-toolbox tools."

• list_toolscatalogue, oracle, provenance
• get_contractwhat your code must provide
• start_venueendpoints + run dir
• venue_controlpause · drop N · withhold N · mark …
• submit_inputwitness lines / your log
• stop_venue · reportthe verdict with evidence

Open an app: pick the case, Start venue, watch the live counters, Drive it (one button per control command plus a free command box and the command log), Your side (upload the witness or your log), Report this case (the verdict table and, with marks, the phase table), or Run our reference client. It is your host's gallery — results are your runs. The CLI and the GUI can drive a venue the other started.

Some venues publish their protocol; some hand it to onboarded firms under an agreement. Every tool in the download is built from a published specification. A bench built from a specification held under agreement — the Tradeweb dealer-to-client lane — is not in the download and not described here: no cases, no endpoints, no fields. It exists, and it is delivered as a service to firms that are themselves onboarded with the venue and hold the same specification under their own agreement, in an engagement: the bench runs on your host or ours, your gateway and feed handler are the code under test, the report is the deliverable. Write to service@priorstates.com with the venue and your onboarding contact.

One row per expectation: the check name, pass or fail, what happened in numbers, and up to three evidence lines from the venue's log or the comparison; a per-phase table when you used marks. Files live in the volume at /data/toolbox/<tool>/<case>/ (report.md, report.json, the venue's logs and truth, your inputs, control.log). mtsx report <tool> all writes a combined report; mtsx serve (add -p 8790:8790) shows a local gallery with your results. The results on this site are our reference runs; yours stay on your host.

Free for evaluation and non-commercial use. Commercial licences, certification packs (the full case sets, spec-version tracking), and CME pre-certification or onboarding engagements — by the person who built and certified the components — through priorstates.com/mts. Built by PriorStates.

No warranty, no liability for trading losses. The software, the venue simulators, the reference clients, the cases, the reports and the published reference results are provided as is, without warranty of any kind. A verdict is evidence about your software against a simulated venue — not an exchange certification, not a guarantee of production behaviour, not trading or financial advice. The author is not liable for any loss arising from the use of the software or its results, including any trading loss, erroneous or duplicated orders, missed executions, or exchange penalties; you are solely responsible for testing, certifying and operating your trading systems. Full terms: the licence (all rights reserved; evaluation and non-commercial use permitted; commercial use by licence from the author — the same text ships as LICENSE.md in the download).