solarforecastarbiter.metrics.preprocessing.filter_resample

solarforecastarbiter.metrics.preprocessing.filter_resample(fx_obs: Union[solarforecastarbiter.datamodel.ForecastObservation, solarforecastarbiter.datamodel.ForecastAggregate], fx_data: Union[pandas.core.series.Series, pandas.core.frame.DataFrame], obs_data: pandas.core.frame.DataFrame, quality_flags: Tuple[solarforecastarbiter.datamodel.QualityFlagFilter, ...], outages: Tuple[solarforecastarbiter.datamodel.TimePeriod, ...] = ()) → Tuple[Union[pandas.core.series.Series, pandas.core.frame.DataFrame], pandas.core.series.Series, List[solarforecastarbiter.datamodel.ValidationResult]][source]

Filter and resample the observation to the forecast interval length.

Parameters:
  • fx_obs (solarforecastarbiter.datamodel.ForecastObservation, solarforecastarbiter.datamodel.ForecastAggregate) – Pair of forecast and observation.
  • fx_data (pandas.Series or pandas.DataFrame) – Timeseries data of the forecast.
  • obs_data (pandas.DataFrame) – Timeseries of values and quality flags of the observation/aggregate data.
  • quality_flags (tuple of solarforecastarbiter.datamodel.QualityFlagFilter) – Flags to process and apply as filters during resampling.
  • outages (tuple of solarforecastarbiter.datamodel.TimePeriod) – Time periods to drop from data prior to filtering or alignment.
Returns:

  • forecast_values (pandas.Series or pandas.DataFrame) – Same as input data except may be coerced to a safer dtype.
  • observation_values (pandas.Series) – Observation values filtered and resampled.
  • validation_results (list) – Elements are solarforecastarbiter.datamodel.ValidationResult.

Notes

The keep/exclude result of each element of the quality_flags tuple is combined with the OR operation.

For quality_flags tuple elements where QualityFlagFilter.discard_before_resample is False, the QualityFlagFilter.quality_flags are considered during the resampling operation. The flags of the raw observations are combined with OR, the total number of flagged points within a resample period is computed, and intervals are discarded where QualityFlagFilter.resample_threshold_percentage is exceeded.

Therefore, the following examples can produce different results:

>>> # separate flags. OR computed after resampling.
>>> qflag_1 = QualityFlagFilter(('NIGHTTIME', ), discard_before_resample=False)
>>> qflag_2 = QualityFlagFilter(('CLEARSKY', ), discard_before_resample=False)
>>> # combined flags. OR computed during resampling.
>>> qflag_combined = QualityFlagFilter(('NIGHTTIME', 'CLEARSKY'),
        discard_before_resample=False)
Raises:
  • ValueError – If fx_obs.forecast.interval_length is less than fx_obs.observation.interval_length
  • ValueError – If fx_obs.forecast is an EventForecast and fx_obs.forecast.interval_length is not equal to fx_obs.observation.interval_length