Usage and configuration

Everything tunable lives in config/config.toml. Nothing operational is hardcoded, and the parser enforces exactly what the comments promise — types, enumerated choices, bounds — failing with a message that names the offending key, so a session cannot start from a configuration it could not honor.

Credentials

Keys come from the file named by credentials.env_file — .env beside the repository root in the shipped configuration — or from the process environment, never from the configuration file itself. Copy the template and fill it in:

cp .env.example .env      # then set ALPACA_API_KEY_ID and ALPACA_SECRET_KEY
chmod 600 .env

Paper-account keys work for market data; point [alpaca] trading_base at the paper endpoint, as the shipped configuration does. load_credentials! warns if the file is group- or world-readable.

Entry points

Every script activates and instantiates the scripts environment itself (scripts/Project.toml: the package by path, plus CairoMakie and UnicodePlots for the figures and the dashboard plots), so a fresh clone needs no preparation, and each accepts an alternative config path as its first argument.

julia --threads=auto scripts/stream.jl   # live capture
julia scripts/backfill.jl                # historical download over the configured window
julia scripts/compact.jl  data/raw/<session>_part001.jsonl
julia scripts/replay.jl   data/raw/<session>_part001.jsonl
julia scripts/visualize.jl data/raw/<session>_part001.jsonl
julia scripts/monitor.jl                 # read-only dashboard for a running session

From the REPL the same operations are run_stream, run_backfill, compact_raw and replay_source. The figure functions need using CairoMakie beside the package, and monitor_raw draws its plots once using UnicodePlots has run; the package itself depends on neither.

[provider]

On the free tier delayed_sip is the scientifically stronger choice: it is the complete consolidated tape, and a fixed 15-minute offset is irrelevant to any analysis that is not trading on it.

[credentials]

Relative paths are resolved against the directory of the configuration file, so a configuration can be copied into any project and keeps working. load_config() without an argument reads the repository's own config/config.toml, and is refused in an installed copy of the package, where that file is a read-only template to copy (MarketTickStreamer.DEFAULT_CONFIG).

[stream]

The watchdog exists because HTTP.jl 1.x WebSockets have no read idle timeout, so a silently dead TCP connection would otherwise block the read loop indefinitely.

[storage]

[limits]

These are the safety margins. They are the sole source of truth for the cutoffs — no threshold is hardcoded elsewhere in the pipeline.

[replay]

[backfill]

The sentinels keep a committed configuration from going stale. They count calendar days, not trading days: a window may land on a weekend or a holiday and return nothing, which the session report will show as an empty capture rather than an error.

[rest]

[quality]

non_price_conditions lists, per tape, the sale-condition codes that disqualify a print from a price path. Tapes "A" and "B" are CTA-processed, "C" is UTP-processed, "O" is the OTC tape; the same character means different things across them, which is why the lists are separate. The shipped defaults are the provider's own published lists.

This is used only by price_forming, filter_price_forming and the n_price_forming column of session_report. Capture is never filtered — see Replay & Analysis Interfaces for why the distinction belongs to the analysis rather than to the recording.

A print carrying several conditions is disqualified by any one of them, and a print on a tape with no list is kept. The set lands in the session sidecar with the rest of the configuration, so a result stays attributable to the eligibility rule that produced it.

To decode the codes themselves, fetch the provider's own glossary with condition_map rather than assuming a mapping.

[monitor]

Attach-mode terminal dashboard, read-only and opt-in.

[logging]

File logs are always flushed and are sanitized of terminal control sequences, so a killed session still leaves a readable tail.

[alpaca]

Endpoint roots, overridable so the suite can point the whole pipeline at an in-process mock server.