Output Artefacts

Purpose

This page defines what the YAML runner writes, where files are written, and which config flags control each artefact.

Related pages:

Run directory and layout semantics

Run directory precedence:

  1. CLI --run-dir
  2. outputs.run_dir
  3. Timestamped folder under outputs.root_dir

Layout behaviour:

  • outputs.layout: staged (default) writes files under numbered stage folders.
  • outputs.layout: flat writes all files directly under the run directory.

Stage folders used by the runner:

  • 00_run_metadata
  • 10_pre_run
  • 20_model_fit
  • 30_post_run
  • 40_diagnostics
  • 50_model_selection
  • 60_scenario_analysis (only when scenario_analysis.enabled: true)
  • 80_optimisation (only when optimisation or allocation output is written)

artifact_schema_version: 4 identifies the current layout and corrected score/interval semantics. Schema 3 used the same stage layout but earlier score/interval meanings. Schema-v1 run directories retain 60_optimisation/, and schema-v2 run directories may retain the former empty 70_forecast/ stage; the runner does not rename historical results.

Command behaviour

validate

  • validate uses dry_run = TRUE.
  • If no run directory is resolved, no artefacts are written.
  • If a run directory is resolved (--run-dir or outputs.run_dir), config.original.yaml is written.
  • If a run directory is resolved (--run-dir or outputs.run_dir), config.resolved.yaml is written.
  • If a run directory is resolved (--run-dir or outputs.run_dir), config.compiled.yaml is written.
  • If a managed holiday country filter is active and a run directory is resolved, holiday_calendar.filtered.csv is materialised under 10_pre_run/.
  • If a run directory is resolved and outputs.save_session_info_txt: true, session_info.txt is written.
  • forecast.enabled: true is rejected before a run directory is created.

run

  • run writes the full artefact set subject to config toggles and runtime conditions.

Fixed-effects run boundary

Fixed-effects (model.type: fe) runs use a dedicated artefact contract. They fit the existing coefficient-only MCMC estimator, then bypass the generic post-fit and decision-layer paths. The only opt-in exception is seen-unit time-series validation.

With the tracked config/fe_panel.yaml defaults, a completed staged run writes:

  • 00_run_metadata/artifact_schema.yaml
  • 00_run_metadata/config.original.yaml
  • 00_run_metadata/config.resolved.yaml
  • 00_run_metadata/config.compiled.yaml
  • 00_run_metadata/run_status.yaml
  • 00_run_metadata/session_info.txt
  • 20_model_fit/model.rds
  • 30_post_run/posterior_summary.csv
  • 40_diagnostics/chain_diagnostics.txt
  • 40_diagnostics/diagnostics_summary.txt
  • 40_diagnostics/within_design.csv
  • 40_diagnostics/contrast_residuals.csv
  • 40_diagnostics/contrast_ppc.csv
  • 40_diagnostics/level_diagnostics.csv

The relevant outputs.* flags may remove optional files from this inventory. Setting outputs.save_posterior_rds: true adds 20_model_fit/posterior.rds. Managed holidays may add the existing holiday provenance files under 10_pre_run/.

The two RDS files have different purposes. model.rds is the full fitted same-environment analysis object; it preserves the fitted RStan state and retained FE metadata for reload under the compatible R, RStan, and package environment. posterior.rds is the optional compact extracted posterior table; it is not an executable fitted model and does not replace model.rds.

FE CSV column contracts are:

File Columns
posterior_summary.csv response_scale, parameter, parameter_role, mean, median, sd, p2_5, p25, p75, p97_5
within_design.csv contrast_id, contrast_label, unit, contrast_index, term, design_value, predictor_scale, contrast_basis
contrast_residuals.csv contrast_id, contrast_label, unit, contrast_index, observed_within, fitted_mean_within, residual_mean_within, response_scale, contrast_basis
contrast_ppc.csv contrast_id, contrast_label, unit, contrast_index, observed_within, predictive_mean_within, predictive_sd_within, predictive_p2_5_within, predictive_p50_within, predictive_p97_5_within, response_scale, contrast_basis
level_diagnostics.csv unit, date, diagnostic_scope, response_scale, observed, fitted, residual, and posterior-predictive summaries (*_mean, *_p2_5, *_p50, *_p97_5)

posterior_summary.csv contains slopes and noise_sd; it does not contain a unit intercept, RMSE, or SMAPE. The three contrast tables use the deterministic orthonormal Helmert basis retained by the fitted model. Their values and labels are basis-dependent diagnostics, not row-level observations, level-scale fitted values, or predictions. Preserve contrast_label and contrast_basis when comparing or joining these files.

level_diagnostics.csv is separate from the contrast tables. It uses the approved conditional retained-unit intercept reconstruction and is labelled conditional_level: its fitted rows contribute to the reconstructed intercept. It is therefore an in-sample conditional diagnostic, not held-out validation.

diagnostics_summary.txt records factual design metadata and includes:

generic_diagnostics_report: not_run
publish_gate: disabled
qualification_status: not_assessed

Therefore, a completed FE run makes no diagnostic pass, publishability, or production-qualification claim. No generic level-scale fitted, observed, residual, diagnostics, decomposition, scenario, optimisation, PSIS model-selection, forecast, or deployment artefacts are written. When the explicit FE seen-unit time-series validation is enabled, the scoped 50_model_selection/tscv_* artefacts are written instead.

Artefact contract by stage

00_run_metadata

File Controlled by Written when Notes
config.original.yaml always run dir materialised Raw YAML text from the input config.
config.resolved.yaml always run dir materialised Authored config after defaults, path resolution, and v2 schema validation.
config.compiled.yaml always run dir materialised Internal compiled runner config after the friendly YAML is translated into the downstream runtime shape.
artifact_schema.yaml always run dir materialised Machine-readable runner artifact contract marker. Includes artifact_schema_version, layout (staged or flat), and fitted model_comparison provenance when available. Missing provenance prevents automatic ranking.
run_status.yaml best-effort run dir materialised Machine-readable terminal run outcome. The runner attempts to write it for dry runs, fit failures after metadata creation, successful completions, scenario-analysis failures, diagnostics publish-gate failures, and post-fit artefact-write failures. Severe file-system failures can still prevent the file from being created.
session_info.txt outputs.save_session_info_txt flag is true Includes dsambayesplus version, artifact schema version, config schema version, model/fit metadata, and sessionInfo().

10_pre_run

File Controlled by Written when Notes
transform_assumptions.txt outputs.save_transform_assumptions_txt flag is true Written even if transform sensitivity scenarios are disabled.
transform_sensitivity_summary.csv outputs.save_transform_sensitivity_summary_csv sensitivity object exists with rows Requires transforms.sensitivity.enabled: true and successful scenario execution.
transform_sensitivity_parameters.csv outputs.save_transform_sensitivity_parameters_csv sensitivity object exists with rows Parameter means/SD by scenario.
dropped_groups.csv none groups dropped by pooling.min_waves filter Written only when sparse groups are excluded.
holiday_calendar.filtered.csv none managed holidays enabled with a country filter Materialised filtered holiday calendar consumed by config.compiled.yaml.
holiday_feature_manifest.csv none managed holidays enabled and features generated Documents generated holiday terms and active-week counts.
design_matrix_manifest.csv outputs.save_design_matrix_manifest_csv flag is true and manifest non-empty Per-term design metadata.
data_dictionary.csv outputs.save_data_dictionary_csv flag is true and dictionary table non-empty Merges inline YAML metadata and optional CSV dictionary metadata.
spec_summary.csv outputs.save_spec_summary_csv flag is true and table available Single-row model/spec summary.
vif_report.csv outputs.save_vif_report_csv flag is true and predictors available VIF diagnostics for non-intercept predictors.

20_model_fit

File Controlled by Written when Notes
model.rds outputs.save_model_rds flag is true Fitted model object.
deployment_model.rds outputs.save_deployment_model_rds flag is true and the fitted model is either model.type: blm, model.type: pooled with fit.method: mcmc, or hierarchical model.type: re/cre with fit.method: mcmc Compact deployment artifact for explicit predict(newdata = ...) and explicit-data decomposition. It is additive to model.rds and does not replace the full analysis object. Pooled deployment artifacts retain authored-term scoring behaviour without shipping runtime dimension_map state. Hierarchical deployment artifacts are seen-groups-only; explicit prediction and decomposition data must include raw grouping columns, and decomposition also requires the response source column(s).
posterior.rds outputs.save_posterior_rds flag is true and MCMC fit Raw posterior object for MCMC runs only.
fit_metrics_by_group.csv implicit fitted summary is computed Written when any of save_fitted_csv, save_fit_png, save_residuals_csv, save_diagnostics_png is true.
fit_timeseries.png outputs.save_fit_png flag is true and ggplot2 installed Observed vs fitted over time on the model response scale, with a subtitle that states the model form (levels or semilog), the displayed scale, fit metrics including Classical R^2 (posterior mean), and monthly date labels when date is a true Date.
fit_scatter.png outputs.save_fit_png flag is true and ggplot2 installed Observed vs fitted scatter on the model response scale, with a subtitle that states the model form (levels or semilog) and the displayed scale.
posterior_forest.png none posterior draws available and ggplot2 installed Posterior coefficient forest plot; skipped for optimise/MAP runs.
prior_posterior.png none posterior draws available, model has priors, and ggplot2 installed Prior-versus-posterior comparison plot; skipped for optimise/MAP runs.

30_post_run

File Controlled by Written when Notes
observed.csv outputs.save_observed_csv flag is true Observed response on model response scale.
observed_kpi.csv outputs.save_observed_csv flag is true and response scale is log KPI-scale observed values (exp) with conversion_method = point_exp.
fitted.csv outputs.save_fitted_csv flag is true Fitted summaries on model response scale.
fitted_kpi.csv outputs.save_fitted_csv flag is true and response scale is log KPI-scale fitted summaries (exp).
posterior_summary.csv outputs.save_posterior_summary_csv flag is true and MCMC fit Posterior summaries for coefficients and scalar diagnostics.
decomp_predictor_impact.csv outputs.save_decomp_csv flag is true and runner linear term-contribution tables are supported Predictor-level design column × posterior mean coefficient table. This is not the mapping/reference-point result returned by decomp(). Unsupported models produce a skip in artifact_status.csv.
decomp_timeseries.csv outputs.save_decomp_csv flag is true and runner linear term-contribution tables are supported Long-format linear contribution-by-date table. Unsupported models produce a skip in artifact_status.csv.
decomp_predictor_impact.png outputs.save_decomp_png runner contribution tables are supported and ggplot2 is installed Predictor-impact linear contribution plot.
decomp_timeseries.png outputs.save_decomp_png runner contribution tables are supported and ggplot2 is installed Media linear-contribution time-series plot.

Active v1.3 pipeline note:

  • 30_post_run/ emits observed and fitted summaries, posterior summaries, and runner linear term-contribution artefacts when their toggles are enabled.
  • Runner contribution artefacts fail closed for hierarchical models, models with offsets, and models fitted with probabilistic media transforms. Use the native interactive decomp() contract where that model class is supported.
  • When decomposition is unavailable, the runner records deterministic skip rows in 40_diagnostics/artifact_status.csv rather than silently dropping the contract entries.

40_diagnostics

File Controlled by Written when Notes
chain_diagnostics.txt outputs.save_chain_diagnostics_txt flag is true and MCMC fit Chain diagnostics text output.
diagnostics_report.csv outputs.save_diagnostics_report_csv flag is true and diagnostics object exists One row per diagnostic check.
diagnostics_summary.txt outputs.save_diagnostics_summary_txt flag is true and diagnostics object exists Counts by status and overall status.
estimator_checks.csv none the fitted model retains typed estimator checks Stable estimator check IDs, severity, status, metric, threshold, and recovery action. Structural failure aborts before fitting, so a completed run records checks that reached pass or advisory warning.
row_reconciliation.csv none exact-frame reconciliation metadata exists Input, retained, and excluded counts plus excluded source-row IDs and reason codes. It never contains response values.
cre_estimability_summary.csv none the fitted CRE model retains its preparation-time estimability report Combined between-design rank and residual degrees of freedom plus separately labelled unweighted and group-size-weighted condition/VIF summaries. Each row records the ordered design terms, preparation policy, runner diagnostics mode, and diagnostic thresholds used.
cre_estimability_groups.csv none the fitted CRE model retains its preparation-time estimability report Selected CRE grouping keys, prefixed with group_var_, and retained observation counts. It contains no response values.
cre_estimability_vif.csv none the fitted CRE model retains its preparation-time estimability report Per-term VIF values for unweighted and group-size-weighted between designs. When fewer than two eligible predictors exist, each eligible predictor has an explicit not-applicable row.
cre_estimability_variation.csv none the fitted CRE model retains its preparation-time estimability report Within-, between-, and total-variation evidence for each CRE variable, including exact-zero and near-zero flags and the active warning threshold.
cre_estimability_singular_values.csv none the fitted CRE model retains its preparation-time estimability report Singular values, ranks, and tolerances for the combined between design and both centred predictor information designs.
hierarchy_support_summary.csv none a fitted RE or CRE model retains its preparation-time hierarchy support report One row per random-effects block with group-size distribution, covariance dimension and groups-per-parameter evidence, advisory flags, and the active policy. Covariance support is reported without a threshold.
hierarchy_support_groups.csv none a fitted RE or CRE model retains its preparation-time hierarchy support report Retained grouping keys, prefixed with group_var_, and observation counts for every random-effects block. It contains no response values because the response is structurally rejected from random-effect terms and grouping keys.
hierarchy_support_rank.csv none a fitted RE or CRE model retains its preparation-time hierarchy support report Per-group random-effects design rank, residual degrees of freedom, singular-value bounds, and the active rank tolerance.
hierarchy_support_variation.csv none a fitted RE or CRE model retains its preparation-time hierarchy support report Within-, between-, and total-variation evidence for every random slope, including exact-zero and near-zero flags.
artifact_status.csv none an artefact is skipped, warned, or errors Per-artifact status log; absent when no status event occurs.
residuals.csv outputs.save_residuals_csv flag is true and fitted summary is computed Residual table on response scale.
residuals_timeseries.png outputs.save_diagnostics_png flag is true and ggplot2 installed Residuals over time.
residuals_vs_fitted.png outputs.save_diagnostics_png flag is true and ggplot2 installed Residuals vs fitted.
residuals_hist.png outputs.save_diagnostics_png flag is true and ggplot2 installed Residual histogram.
residuals_acf.png outputs.save_diagnostics_png flag is true and ggplot2 installed Residual autocorrelation plot.
residual_diagnostics.csv none diagnostics residual checks available Ljung-Box / ACF check outputs.
residuals_latent.csv none diagnostics latent residuals available Latent residual series from diagnostics object.
residuals_latent_acf.png outputs.save_diagnostics_png latent residuals available and ggplot2 installed Latent residual ACF plot.
ppc.png none posterior predictive plot available and ggplot2 installed Posterior predictive check plot; skipped for optimise/MAP runs.
boundary_hits.csv none boundary-hit table available Boundary-hit rates per parameter.
boundary_hits.png outputs.save_diagnostics_png boundary-hit table available and ggplot2 installed Boundary-hit visualisation.
within_variation.csv none within-variation table available Within-variation diagnostics for hierarchical terms.
within_variation.png outputs.save_diagnostics_png within-variation table available and ggplot2 installed Within-variation visualisation.
predictor_risk_register.csv outputs.save_predictor_risk_register_csv flag is true and table non-empty Ranked risk register combining VIF, within-variation, boundary hits, and slow-moving flags.

50_model_selection

File Controlled by Written when Notes
loo_summary.csv outputs.save_model_selection_csv flag is true, diagnostics.model_selection.enabled: true, and diagnostics report exists May be full PSIS-LOO summary or a stub row with skip reason. A successful summary records the conditional-exchangeability assumption and directs time-ordered selection to blocked or leave-future-out CV.
loo_pointwise.csv outputs.save_model_selection_pointwise_csv flag is true, diagnostics report exists, and pointwise PSIS-LOO is available Optional pointwise LOO diagnostics.
pareto_k.png outputs.save_diagnostics_png pointwise PSIS-LOO available and ggplot2 installed Pareto-k diagnostic plot.
elpd_influence.png outputs.save_diagnostics_png pointwise PSIS-LOO available and ggplot2 installed Pointwise ELPD influence plot.
tscv_folds.csv diagnostics.time_series_selection.enabled time-series selection enabled and folds produced Fold windows plus the active TSCV policy (method, horizon_weeks, stride_weeks, min_train_weeks, gap_weeks) and fold-level runtime/status metadata. FE rejects a fold before fitting if its panel support is invalid.
tscv_summary.csv diagnostics.time_series_selection.enabled time-series selection enabled Written for success, skipped, or error outcomes; the overall row is ok only when every scheduled fold succeeds and records n_folds and n_ok_folds. Each row also carries the active TSCV policy fields.
tscv_pointwise.csv diagnostics.time_series_selection.enabled + diagnostics.time_series_selection.save_pointwise enabled and pointwise rows available Optional pointwise holdout log predictive densities. FE rows include unit, date, and lpd in original holdout order.
tscv_elpd_by_fold.png diagnostics.time_series_selection.save_png + outputs.save_diagnostics_png enabled and ggplot2 installed ELPD-by-fold chart.

60_scenario_analysis

File Controlled by Written when Notes
scenario_response_summary.csv scenario_analysis.enabled scenario analysis succeeds Row-level posterior summaries for scenario, reference, and scenario - reference.
scenario_aggregate_summary.csv scenario_analysis.enabled scenario analysis succeeds Draw-wise totals aggregated before summarisation, optionally by scenario_analysis.aggregate_by.
scenario_metadata.yaml scenario_analysis.enabled scenario analysis succeeds Estimand, scale, interval, source paths, carry-over initialisation, and the explicit causal_effect: false limitation.
scenario_aggregate_draws.csv scenario_analysis.save_draws scenario analysis succeeds and flag is true Draw-level aggregate totals and differences. Disabled by default because this file can be large.

The runner emits model-implied fitted-response contrasts. These are not causal effects unless the model design and external assumptions justify that claim.

80_optimisation

File Controlled by Written when Notes
optimisation_runs.csv none fit.method: optimise All optimisation starts, including objective value and return code when available.
optimisation_best.csv none fit.method: optimise The selected MAP optimum: highest optimiser objective when available, otherwise lowest RMSE.
budget_summary.csv outputs.save_allocator_csv allocation enabled and flag is true Scenario-level optimisation summary.
budget_allocation.csv outputs.save_allocator_csv allocation enabled and flag is true Recommended allocation by channel.
budget_diagnostics.csv outputs.save_allocator_csv allocation enabled and flag is true Candidate and objective diagnostics.
budget_response_curves.csv outputs.save_allocator_csv allocation enabled and flag is true Response-curve payload.
budget_response_points.csv outputs.save_allocator_csv allocation enabled and flag is true Key plotted points for response curves.
budget_roi_cpa.csv outputs.save_allocator_csv allocation enabled and flag is true ROI/CPA panel payload (depends on KPI type).
budget_impact.csv outputs.save_allocator_csv allocation enabled and flag is true Allocation impact payload.
budget_response_curves.png outputs.save_allocator_png allocation enabled, flag is true, and ggplot2 installed Response curves plot.
budget_roi_cpa.png outputs.save_allocator_png allocation enabled, flag is true, and ggplot2 installed ROI/CPA panel plot.
budget_impact.png outputs.save_allocator_png allocation enabled, flag is true, and ggplot2 installed Allocation impact plot.
budget_optimisation.json outputs.save_allocator_json allocation enabled, flag is true, and jsonlite installed Combined JSON payload (summary, allocation, diagnostics, plot_data).

Deployment artifact note:

  • deployment_model.rds lives under 20_model_fit/; it is not a future-data prediction output.
  • It is a compact packaging artifact for deployment consumers, not a signal that the runner now generates future-data forecasts or scenarios.

Response scale semantics (*_kpi.csv vs base files)

Base files (observed.csv, fitted.csv) are always on the model response scale:

  • identity response: KPI units
  • log response: log(KPI)

KPI-scale files are written only for log-response models:

  • observed_kpi.csv
  • fitted_kpi.csv

Conversion metadata:

  • observed_kpi.csv uses conversion_method = point_exp.
  • fitted_kpi.csv uses conversion_method = lognormal_mean by default for log-response fitted values.
  • fitted_kpi.csv uses conversion_method = point_exp only when the median back-transform is explicitly requested.

Fitted intervals and model-selection densities (schema 4)

For BLM, pooled, RE and CRE runs, fitted.csv contains:

Columns Meaning
mean, p2_5, p50, p97_5 Mean and quantiles of conditional-mean draws (yhat)
interval_type, interval_level conditional_mean, 0.95
predictive_p2_5, predictive_p50, predictive_p97_5 Replicate-observation quantiles from y_rep
predictive_interval_status available or unavailable; unavailable quantiles are NA

Both interval sets retain the fitted row order and use the unstandardised model response scale. MAP retains mean, with all quantiles NA. fitted_kpi.csv labels its conditional-mean interval but does not export predictive quantiles. The fit ribbon is the conditional-mean credible interval; it is absent for MAP.

fit_metrics_by_group.csv replaces pi95_coverage with posterior_predictive_coverage_95. This is observed inclusion in the predictive interval, by group and overall, and is NA without predictive endpoints. It is an in-sample diagnostic, not a calibration or estimator qualification result.

Successful loo_summary.csv rows contain likelihood_scale = unstandardised_model_response. For scaled fits, log likelihoods include -log(s_y) per observation. Log-response scores remain on the log-response scale. Only compare scores for identical response transformations and observation sets. Schema 3 and earlier score CSVs require regeneration; do not mix them with corrected scores. Incorrectly centred no-intercept models require refitting first. Historical files are not modified.

loo_summary.csv also records r_eff_method = chain_relative_eff. PSIS uses chronological post-warmup draws and original chain membership to estimate relative efficiency. If diagnostics.model_selection.max_draws truncates the posterior, the runner retains balanced prefixes from all chains, possibly fewer draws than the limit. A budget below six draws per chain produces a skipped diagnostic; it does not silently assume independent draws.

artifact_schema.yaml records model_comparison with response expression, response scale, observation count, density units and a SHA-256 fingerprint of ordered raw model-response values and date/group labels. Current fit preparation retains those values before numerical scaling. The optional digest package is required to produce the fingerprint. Missing metadata or incompatible candidate data leaves LOO and TSCV ranks and deltas unavailable. TSCV comparisons also require matching policies and realised windows. See Compare Runs for status meanings and legacy-fit limits.

Runner sMAPE uses smape(observed, fitted) / 100, on the 0–2 scale. Zero/zero pairs contribute zero and remain in the observation count. This shares the public function’s near-zero denominator convention.

Resolved diagnostic policy and terminal report

40_diagnostics/diagnostic_gates.resolved.yaml records the complete profile, source path, content hash and algorithm resolved before fitting. It is written with run metadata, including materialised validation runs. The fitted diagnostic report retains the same policy snapshot. The authored profile is validated even when diagnostic collection is disabled.

diagnostics_report.csv adds gate_key, category, label, comparison, warn_threshold, fail_threshold, gate_profile, gate_hash, entity, scope and recovery where applicable. Missing fail_threshold means failure is disabled for a finite value; it does not turn invalid numerical evidence into a pass. Hard structural/availability checks retain their existing rules.

diagnostic_vif.csv contains per-predictor population-design VIF and, when available, status, warning/failure limits, design scope and policy hash. It complements the existing vif_report.csv and separate CRE design reports. diagnostics_summary.txt includes the detailed grouped report printed in the terminal. Gate-controlled VIF plot colours and Pareto-k influence flags use the same resolved policy. See Diagnostic Gates.

Diagnostics status semantics

diagnostics_report.csv status values:

  • pass: check passed configured thresholds
  • warn: check breached warning threshold
  • fail: check breached fail threshold
  • skipped: check not applicable or intentionally skipped

Overall status logic:

  • fail if any check is fail
  • warn if no fails and at least one warn
  • skipped when all checks are skipped or no checks exist
  • pass otherwise

diagnostics_summary.txt reports:

  • overall_status
  • counts for pass, warn, fail, skipped

Quick verification commands

List produced files for a run:

latest_run="$(ls -td results/* | head -n 1)"
find "$latest_run" -type f | sort

Inspect key diagnostics files:

latest_run="$(ls -td results/* | head -n 1)"
head -n 20 "$latest_run/40_diagnostics/diagnostics_report.csv"
head -n 20 "$latest_run/40_diagnostics/diagnostics_summary.txt"

Pre-fit geographic panel evidence

When panel diagnostics are enabled, 10_pre_run/ also contains panel_gates.resolved.yaml, panel_design_checks.csv, panel_design_terms.csv, panel_design_dates.csv and (when available) panel_design_design.csv. These files are written before the runner enforces its panel structural-rank gate, including in validation mode when an output directory exists. See panel design diagnostics for definitions and availability limits.