solarforecastarbiter.metrics.preprocessing.filter_resample¶
-
solarforecastarbiter.metrics.preprocessing.
filter_resample
(fx_obs: Union[solarforecastarbiter.datamodel.ForecastObservation, solarforecastarbiter.datamodel.ForecastAggregate], fx_data: Union[pandas.core.series.Series, pandas.core.frame.DataFrame], obs_data: pandas.core.frame.DataFrame, quality_flags: Tuple[solarforecastarbiter.datamodel.QualityFlagFilter, ...], outages: Tuple[solarforecastarbiter.datamodel.TimePeriod, ...] = ()) → Tuple[Union[pandas.core.series.Series, pandas.core.frame.DataFrame], pandas.core.series.Series, List[solarforecastarbiter.datamodel.ValidationResult]][source]¶ Filter and resample the observation to the forecast interval length.
Parameters: - fx_obs (solarforecastarbiter.datamodel.ForecastObservation, solarforecastarbiter.datamodel.ForecastAggregate) – Pair of forecast and observation.
- fx_data (pandas.Series or pandas.DataFrame) – Timeseries data of the forecast.
- obs_data (pandas.DataFrame) – Timeseries of values and quality flags of the observation/aggregate data.
- quality_flags (tuple of solarforecastarbiter.datamodel.QualityFlagFilter) – Flags to process and apply as filters during resampling.
- outages (tuple of
solarforecastarbiter.datamodel.TimePeriod
) – Time periods to drop from data prior to filtering or alignment.
Returns: - forecast_values (pandas.Series or pandas.DataFrame) – Same as input data except may be coerced to a safer dtype.
- observation_values (pandas.Series) – Observation values filtered and resampled.
- validation_results (list) – Elements are
solarforecastarbiter.datamodel.ValidationResult
.
Notes
The keep/exclude result of each element of the
quality_flags
tuple is combined with the OR operation.For
quality_flags
tuple elements whereQualityFlagFilter.discard_before_resample
isFalse
, theQualityFlagFilter.quality_flags
are considered during the resampling operation. The flags of the raw observations are combined withOR
, the total number of flagged points within a resample period is computed, and intervals are discarded whereQualityFlagFilter.resample_threshold_percentage
is exceeded.Therefore, the following examples can produce different results:
>>> # separate flags. OR computed after resampling. >>> qflag_1 = QualityFlagFilter(('NIGHTTIME', ), discard_before_resample=False) >>> qflag_2 = QualityFlagFilter(('CLEARSKY', ), discard_before_resample=False)
>>> # combined flags. OR computed during resampling. >>> qflag_combined = QualityFlagFilter(('NIGHTTIME', 'CLEARSKY'), discard_before_resample=False)
Raises: ValueError
– If fx_obs.forecast.interval_length is less than fx_obs.observation.interval_lengthValueError
– If fx_obs.forecast is an EventForecast and fx_obs.forecast.interval_length is not equal to fx_obs.observation.interval_length