Export

Export

HDF5 product export: every consumer-facing product of a run — event logs, metrics profile, batch-state timeline, batch epochs, point-wise masks, metrology tables, markers, component events — in one self-describing file with the run's provenance as attributes, for analysis pipelines that read HDF5 rather than a directory of CSV files. The CSV products remain the primary interface; the export is a derived, regenerable view of them (export_hdf5).

export_hdf5(run_dir::String; path = joinpath(run_dir, "products.h5")) -> String

Writes the run's products into one HDF5 file (default <run_dir>/products.h5, an existing file is rotated to products#k.h5): the tabular products of TABLE_PRODUCTS as column datasets, the batch-state timeline (write_mask_timeline!), the point-wise masks (write_pointwise_masks!), and the provenance attributes (write_provenance!). Times are seconds since the run's start_sim_time with ISO-8601 twins. Returns the file path.

FORMAT_VERSION

Layout version of the exported file, stored as the root attribute format_version; incremented when a group or dataset changes meaning.

TABLE_PRODUCTS

(group, relative path) pairs of the tabular CSV products written by export_hdf5 through write_table!; absent files are skipped.

ensure_group(parent, path::String)

The group at the slash-separated path below parent, creating the missing levels.

write_column!(g::HDF5.Group, name::String, values::AbstractVector, epoch::DateTime)

One CSV column as HDF5 datasets under g: a DateTime column becomes name (Float64 seconds since epoch, NaN for missing) plus name_iso (ISO-8601 strings); booleans become UInt8 (0/1, the sentinel 0xff for missing); integers stay Int64 unless a value is missing (then Float64 with NaN); other reals become Float64 with NaN for missing; everything else is written as strings with "" for missing.

write_table!(file::HDF5.File, group::String, df::DataFrame, epoch::DateTime, source::String)

A CSV product as one HDF5 group: one dataset per column (write_column!), the attributes source (the relative CSV path) and rows; an empty table yields the group with its attributes only.

write_mask_timeline!(file::HDF5.File, run_dir::String, epoch::DateTime)

masks/telemetry_mask_timeline.csv as masks/timeline: states — the Int8 state matrix laid out so that C-order readers (h5py, NumPy) index states[snapshot, batch] while Julia reads its transpose (batch, snapshot) — with batch_id, the snapshot instants (SimTime, SimTime_iso), and the state-code attribute. Nothing is written when the timeline is absent.

write_pointwise_masks!(file::HDF5.File, run_dir::String)

Every masks/pointwise_mask_*.csv expansion as masks/pointwise/<stem>/Ground_Available (Int8 0/1 per sample, sample k at index k).

write_provenance!(file::HDF5.File, run_dir::String, cfg::AbstractDict, epoch::DateTime)

Root attributes: format_version, run_id, start_sim_time, speed_up, exported_at, the platform fingerprint of the run snapshot ([provenance.platform]: hostname, package version, git commit, Julia version, …), and config_snapshot — the run's configuration as TOML text.