solarforecastarbiter.metrics.preprocessing.process_forecast_observations

solarforecastarbiter.metrics.preprocessing.process_forecast_observations(forecast_observations, filters, forecast_fill_method, start, end, data, timezone, costs=())[source]

Convert ForecastObservations into ProcessedForecastObservations applying any filters and resampling to align forecast and observation.

Parameters:
  • forecast_observations (list of solarforecastarbiter.datamodel.ForecastObservation, solarforecastarbiter.datamodel.ForecastAggregate) – Pairs to process
  • filters (list of solarforecastarbiter.datamodel.BaseFilter) – Filters to apply to each pair.
  • forecast_fill_method (str) – Indicates what process to use for handling missing forecasts. Currently supports : ‘drop’, ‘forward’, and bool or numeric value.
  • start (pandas.Timestamp) – Start date and time for assessing forecast performance.
  • end (pandas.Timestamp) – End date and time for assessing forecast performance.
  • data (dict) – Dict with keys that are the Forecast/Observation/Aggregate object and values that are the corresponding pandas.Series/DataFrame for the object. Keys must also include all Forecast objects assigned to the reference_forecast attributes of the forecast_observations.
  • timezone (str) – Timezone that data should be converted to
  • costs (tuple of solarforecastarbiter.datamodel.Cost) – Costs that are referenced by any pairs. Pairs and costs are matched by the Cost name.
Returns:

list of ProcessedForecastObservation

Notes

The logic is as follows.

For each forecast, observation pair in forecast_observations:

  1. Remove observation data points with quality_flag in filters. Remaining observation series is discontinuous.
  2. Fill missing forecast data points according to forecast_fill_method.
  3. Fill missing reference forecast data points according to forecast_fill_method.
  4. Resample observations to match forecast intervals. A minimum of 10% of the observation intervals within a forecast interval must be valid (not flagged or previously missing) else the resampled observation is NaN.
  5. Drop remaining NaN observation and forecast values.
  6. Align observations to match forecast times. Observation times for which there is not a matching forecast time are dropped.
  7. Create ProcessedForecastObservation with resampled, aligned data and metadata.