Skip to content

Weather Data Sampling

This page explains the simulation-side weather sampling concepts implemented in rencal.simulation.weather_data and rencal.simulation.intermittent_bucketer.

RenCal needs future hourly weather-conditioned generation streams that are:

  • statistically plausible,
  • seasonally aware,
  • compatible with calibrated plant behavior.

Rather than naive random draws, the simulation layer samples historical periods with bucketing and distribution controls.

HistoricalMetadata describes a historical NPY artifact and its temporal limits:

  • artifact basename,
  • path resolver,
  • left/right time bounds,
  • column list,
  • optional block size (hours_per_block) for histogram acceleration.

This descriptor can also be built from a loader-provided manifest via HistoricalMetadata.from_manifest(...), keeping simulation metadata aligned with calibrated artifact metadata.

WeatherData.random_sample(...) returns a VersionedColumnsNDArray, not a plain array.

This wrapper keeps track of:

  • canonical columns from the base sampled data,
  • appended transformed versions for columns that have one or more desired-average overrides.

Use:

WeatherData is the main sampler object.

It uses metadata + optional desired-average overrides and then samples historical periods for a requested future window.

Constructor semantics that matter in practice:

  • desired_averages is a mapping: dict[int, list[float]], where each key is a canonical column index and each value is one or more target averages.
  • If desired averages are provided, at least one of historical_data or prefix_histograms must be provided at construction (otherwise optimization cannot be computed).
  • ignore_zeros=True activates solar-style behavior: histogram building and resampling operate only on non-zero hours, while zeros are preserved.

Key behaviors include:

  • process-scope cache by NPY basename,
  • bucket-aware random draw selection,
  • optional zero handling mode (ignore_zeros) for solar-like series,
  • histogram/inverse-CDF style resampling utilities,
  • version-aware output for duplicated target averages per stream.

IntermittentBucketer applies seasonality-aware bucketing with a single category.

  • Date windows are assigned to monthly buckets.
  • Draws are selected from historical ranges that match bucket context.
  • Resulting samples preserve seasonal structure better than unconstrained random shuffling.

For a deeper explanation of bucket design and draw mechanics, see Bucketing Strategy.

WeatherData supports two conceptual modes:

  • ignore_zeros = False: all values contribute to histograms and resampling (typical wind behavior).
  • ignore_zeros = True: zero-valued hours are preserved; only non-zero values are resampled (typical solar behavior where nighttime zeros are structural).

This avoids distorting physical patterns (for example, introducing solar output at night).

When ignore_zeros=True, desired averages are internally rescaled from all-hours to non-zero-hours before optimization, so transformed daytime values still target the intended whole-series behavior.

WeatherData.random_sample(...) works as follows:

  1. load/mmap historical NPY via process-scope cache (_HIST_CACHE),
  2. extend right-hand horizon with BucketedData.adjust_forecasted_horizon(...) so draw-period chunking can fully cover the request,
  3. call the inherited Bucketer.random_sample(...) to get historical block start dates,
  4. slice historical arrays in draw_period * 24 hour chunks and concatenate,
  5. trim to exactly requested hour count,
  6. append transformed columns (if any) via BucketedData.append_resampled_columns(...),
  7. return VersionedColumnsNDArray.

When desired averages are present, the constructor precomputes old/new distributions:

How it connects to loader/downloader layers

Section titled “How it connects to loader/downloader layers”

Simulation sampling depends on upstream artifacts:

  1. downloader produces local weather/generation artifacts,
  2. loader validates calibrated weather artifacts + manifests,
  3. simulation reads those validated historical artifacts for draw/resample operations.

See: