Usage¶

The snippets below use the package's simulated mixed-frequency real-time data. Each example creates a fresh NowcastData and RealTimeModel so its forecasts do not affect the next example.

import forecast_evaluation as fe
import forecast_realtime as rt

sample_data = rt.generate_synthetic_data(
    N=2,
    first_period="2015-01-31",
    endpoint="2024-12-31",
)

Lags¶

The linear, tree and neural models support autoregressive (y_lags) and distributed (X_lags) lags, supplied at forecast time:

forecast_data = fe.NowcastData(outturns_data=sample_data.copy())
rt_model = rt.RealTimeModel(
    data=forecast_data,
    models=rt.models.ForecastRidge(cv=5, scale=True),
)
rt_model.forecast(
    y_variables=["quarterly_1"],
    X_variables=["quarterly_2"],
    data_transformation={"quarterly_1": "pop", "quarterly_2": "pop"},
    steps=2,
    y_lags=4,  # append y_{t-1} … y_{t-4}
    X_lags={"quarterly_2": 2},  # an int applies the same count to every regressor
    X_imputation="last",
    first_vintage="2024-01-31",
    last_vintage="2024-06-30",
)

y_lags=k appends _y_lag1 … _y_lagk; X_lags appends col_lag1 … col_lagk per regressor.

Regularised models leave retained target lags unpenalised by default. Set penalise_ar=True to shrink them. cv accepts an integer or a scikit-learn-compatible splitter; to choose a penalty manually, set cv=None and provide a fixed alpha on the model constructor.

Outlier dummies¶

Pass dummies to add one-off point dummies (value 1 on a single date, 0 elsewhere) for outliers such as the COVID quarter. Supply either a list of dates or a {name: date} mapping; the same argument works on ForecastModel.fit(...) and RealTimeModel.forecast(...).

forecast_data = fe.NowcastData(outturns_data=sample_data.copy())
rt_model = rt.RealTimeModel(
    data=forecast_data,
    models=rt.models.ForecastOLS(label="OLS"),
)
rt_model.forecast(
    y_variables=["quarterly_1"],
    data_transformation={"quarterly_1": "pop"},
    steps=2,
    dummies=["2020-06-30"],  # or {"outlier": "2020-06-30"}
    first_vintage="2024-01-31",
    last_vintage="2024-06-30",
)

Dummies are rebuilt from the DatetimeIndex at both fit and forecast time (no imputation), follow formula selection, and appear as ordinary components in the decomposition. Regularised models always leave them unpenalised. FWL CV scores held-out design rows in original target units, not recursive multi-step backtests; choose a splitter that matches the forecasting task. See dummies_strategy.md and models.md.

Regressor imputation¶

Regressors are often ragged — columns end at different dates and/or fall short of the forecast horizon. Set X_imputation to fill those gaps at both fit and forecast time:

forecast_data = fe.NowcastData(outturns_data=sample_data.copy())
rt_model = rt.RealTimeModel(
    data=forecast_data,
    models=rt.models.ForecastRidge(cv=5, scale=True),
)
rt_model.forecast(
    y_variables=["quarterly_1"],
    X_variables=["quarterly_2"],
    data_transformation={"quarterly_1": "pop", "quarterly_2": "pop"},
    steps=2,
    X_imputation="last",
    first_vintage="2024-01-31",
    last_vintage="2024-06-30",
)  # None | "zero" | "last" | "mean" | "ar1_t"

Columns containing no observed values are rejected when imputation is enabled. Provide at least one observed value for each regressor you want to estimate.

X_imputation is applied only when the model's _needs_ragged_edge_imputation class attribute is True. This is the default for ForecastModel subclasses, so RealTimeModel applies the selected strategy to their ragged-edge X data. When a model sets _needs_ragged_edge_imputation = False, RealTimeModel does not apply X_imputation; the model is responsible for handling its own ragged edge. Models that determine publication availability and forecast dates from raw X data themselves, such as the MIDAS family, use this setting. The flag does not enable imputation unless X_imputation is also supplied.

Direct model results¶

For a fitted model, forecast() returns a validated, DataFrame-compatible ForecastResult. Point results are long tables with a RangeIndex and the columns date, variable, and value. Quantile results add quantile. The .forecast property returns the same payload as an ordinary long DataFrame; forecast_origin and decomposition remain on the original result. Point and quantile forecasts may use custom dates. Slices and copies return ordinary DataFrames without result metadata.

Use an explicit pivot when downstream code needs a point matrix:

y = (
    sample_data.loc[
        (sample_data["vintage_date"] == sample_data["vintage_date"].max())
        & (sample_data["variable"] == "quarterly_1")
        & (sample_data["metric"] == "levels")
    ]
    .set_index("date")[["value"]]
    .rename(columns={"value": "quarterly_1"})
)
model = rt.models.ForecastOLS().fit(y)
point_result = model.forecast(steps=4)
point_matrix = point_result.pivot(
    index="date",
    columns="variable",
    values="value",
)

Data transformations¶

data_transformation maps each variable to the space the model is estimated in. Forecasts are returned in that space and automatically back-transformed to levels where possible (reconstruct_levels=True by default).

Models receive the transformed inputs described by the call-level data_transformation or by a model-specific mapping. Forecast target values use the same metric as their transformed target input, so no separate output metric argument is needed:

transformed_model = rt.models.ForecastOLS(
    data_transformation={"quarterly_1": "diff"},
)

Transformation frequency is inferred independently from each raw y/X column's dates. The forecast horizon frequency is also inferred from the selected target variables; pass step_frequency only when those variables have mixed or ambiguous frequencies. It does not control input transformations. If a raw column has an ambiguous frequency, provide it through the resolved input_frequencies mapping passed to the model.

Density forecasts¶

Pass the keyword-only quantiles argument to a fitted model's forecast() call:

density = model.forecast(steps=4, quantiles=[0.9, 0.1, 0.5])

quantiles=False (the default) returns the long point result. True uses the default probabilities (0.16, 0.5, 0.84). A supplied sequence must contain distinct, finite probabilities strictly between 0 and 1; the package sorts the sequence before forecasting. Density mode returns one long, DataFrame-compatible ForecastResult with the columns date, variable, quantile, and value. It contains only the requested quantile rows, in the model's native forecast metric, and does not include a point forecast.

For realtime forecasts, pass the same argument to RealTimeModel.forecast():

forecast_data = fe.NowcastData(outturns_data=sample_data.copy())
rt_model = rt.RealTimeModel(
    data=forecast_data,
    models=rt.models.ForecastOLS(),
)
rt_model.forecast(
    y_variables=["quarterly_1"],
    data_transformation={"quarterly_1": "levels"},
    steps=2,
    quantiles=True,
    first_vintage="2024-01-31",
    last_vintage="2024-06-30",
)
quantiles = rt_model.quantiles

Density mode stores only native-metric rows in rt_model.quantiles. Rows include date, variable, quantile, value, metric, source, frequency, vintage_date, and forecast_horizon. It sets reconstruct_levels=False internally, even when the argument is True. It does not add point rows to data.forecasts or change rows already there. decomp=True is not supported with quantiles.

Built-in density support is currently available for ForecastOLS and ForecastBVAR. Models without quantile support, including penalised regressions and tree models, reject the request. The package does not expose predictive draws, derive growth or level quantiles from marginal quantiles, or ingest these rows through forecast-evaluation.

News decomposition¶

Set decomp=True to attribute each forecast revision to news (newly released data), reestimation (parameter changes from refitting), and interaction (the residual cross-term). Results are stored on rt_model.decompositions, separately from the forecasts. Decomposition requires the model to implement _forecast_decomp(); models without that method return None.

forecast_data = fe.NowcastData(outturns_data=sample_data.copy())
rt_model = rt.RealTimeModel(
    data=forecast_data,
    models=rt.models.ForecastRidge(cv=5, scale=True),
)
rt_model.forecast(
    y_variables=["quarterly_1"],
    X_variables=["quarterly_2"],
    data_transformation={"quarterly_1": "pop", "quarterly_2": "pop"},
    steps=2,
    decomp=True,
    X_imputation="last",
    first_vintage="2024-01-31",
    last_vintage="2024-06-30",
)
print(rt_model.decompositions)

See forecasting_strategy.md for the full methodology.

Model-owned source conditioning¶

Use the model's conditioning argument when models in one run need different sources or durations. The resolver handles each model's policy independently. periods counts from the first forecast period, so it is positive and inclusive in ordinary language (periods=3 means periods 1 to 3). The older run-level y_steps_ahead and X_steps_ahead arguments keep their zero-based inclusive convention (0 means one period).

The precedence rules are deliberately strict: conditioning=None inherits the run fallback, a non-empty mapping replaces it completely, and conditioning={} disables external conditioning. None and {} are also distinct in the legacy source and horizon mappings. Treat a missing source label as invalid. When a known source has no path for a variable or vintage, preserve the existing missing-path behaviour.

Only models that explicitly opt in to target conditioning can accept explicit future y constraints. Recursive regression and tree models without a supporting root reject unsupported-y conditioning rather than ignoring it. Direct model calls do not consult conditioning policies: they use the explicitly supplied frames.

For the tree-specific routing rules and the ForecastContext.y_published migration, see ForecastTree and Adding a New Model.

Parallel execution¶

The vintage loop can run in parallel across models and vintage batches:

if __name__ == "__main__":
    forecast_data = fe.NowcastData(outturns_data=sample_data.copy())
    rt_model = rt.RealTimeModel(
        data=forecast_data,
        models=rt.models.ForecastOLS(label="OLS"),
    )
    rt_model.forecast(
        y_variables=["quarterly_1"],
        data_transformation={"quarterly_1": "levels"},
        steps=2,
        y_lags=4,
        parallel=True,
        max_workers=2,
        first_vintage="2024-01-31",
        last_vintage="2024-06-30",
    )

With parallel=True, ForecastTree callable transforms and model instances must be pickleable for ProcessPoolExecutor; module-level callables are the usual choice. Sequential mode (parallel=False) also supports local callables.