Usage and configuration

Everything tunable lives in config/config.toml. Nothing operational is hardcoded, and the parser enforces exactly what the comments promise — types, enumerated choices, bounds — failing with a message that names the offending key, so a session cannot start from a configuration it could not honor.

Credentials

Keys come from .env or the process environment, never from the configuration file. Copy the template and fill it in:

cp .env.example .env      # then set ALPACA_API_KEY_ID and ALPACA_SECRET_KEY
chmod 600 .env

Paper-account keys work for market data; point [alpaca] trading_base at the paper endpoint, as the shipped configuration does. load_credentials! warns if the file is group- or world-readable.

Entry points

Every script activates and instantiates the environment itself, so a fresh clone needs no preparation, and each accepts an alternative config path as its first argument.

julia --threads=auto scripts/stream.jl   # live capture
julia scripts/backfill.jl                # historical download over the configured window
julia scripts/compact.jl  data/raw/<session>_part001.jsonl
julia scripts/replay.jl   data/raw/<session>_part001.jsonl
julia scripts/visualize.jl data/raw/<session>_part001.jsonl
julia scripts/monitor.jl                 # read-only dashboard for a running session

From the REPL the same operations are run_stream, run_backfill, compact_raw and replay_source.

[provider]

On the free tier delayed_sip is the scientifically stronger choice: it is the complete consolidated tape, and a fixed 15-minute offset is irrelevant to any analysis that is not trading on it.

[stream]

The watchdog exists because HTTP.jl 1.x WebSockets have no read idle timeout, so a silently dead TCP connection would otherwise block the read loop indefinitely.

[storage]

[limits]

These are the safety margins. They are the sole source of truth for the cutoffs — no threshold is hardcoded elsewhere in the pipeline.

[replay]

[backfill]

The sentinels keep a committed configuration from going stale. They count calendar days, not trading days: a window may land on a weekend or a holiday and return nothing, which the session report will show as an empty capture rather than an error.

[quality]

non_price_conditions lists, per tape, the sale-condition codes that disqualify a print from a price path. Tapes "A" and "B" are CTA-processed, "C" is UTP-processed, "O" is the OTC tape; the same character means different things across them, which is why the lists are separate. The shipped defaults are the provider's own published lists.

This is used only by price_forming, filter_price_forming and the n_price_forming column of session_report. Capture is never filtered — see Replay & Analysis Interfaces for why the distinction belongs to the analysis rather than to the recording.

A print carrying several conditions is disqualified by any one of them, and a print on a tape with no list is kept. The set lands in the session sidecar with the rest of the configuration, so a result stays attributable to the eligibility rule that produced it.

To decode the codes themselves, fetch the provider's own glossary with condition_map rather than assuming a mapping.

[monitor]

Attach-mode terminal dashboard, read-only and opt-in.

[logging]

File logs are always flushed and are sanitized of terminal control sequences, so a killed session still leaves a readable tail.

[alpaca]

Endpoint roots, overridable so the suite can point the whole pipeline at an in-process mock server.