Reference Forecasts

Structure

The Solar Forecast Arbiter supports reference forecasts based on data from numerical weather prediction (NWP) models and from site observations.

The main module orchestrates reference forecast generation within the Solar Forecast Arbiter. It uses data types defined in solarforecastarbiter.datamodel.

main Make benchmark irradiance and power forecasts.
main.run_nwp(forecast, model, run_time, …) Calculate benchmark irradiance and power forecasts for a Forecast or ProbabilisticForecast.
main.run_persistence(session, observation, …) Run a persistence forecast for an observation.

NWP

Forecasts based on NWP model data are used for intraday and longer forecasts. The Solar Forecast Arbiter contains a set of functions to process data from NWP forecasts. These functions are found in the models module. Each function is specific to:

  1. A particular NWP model data set (e.g. NAM or subhourly HRRR), and
  2. The post processing steps required to obtain a particular type of irradiance or power forecast data (e.g. hourly mean or instantaneous).
models Default processing functions for data from NOAA weather models.
models.hrrr_subhourly_to_subhourly_instantaneous(…) Subhourly (15 min) instantantaneous HRRR forecast.
models.hrrr_subhourly_to_hourly_mean(…[, …]) Hourly mean HRRR forecast.
models.rap_ghi_to_instantaneous(latitude, …) Hourly instantantaneous RAP forecast.
models.rap_cloud_cover_to_hourly_mean(…[, …]) Take hourly RAP instantantaneous cloud cover and convert it to hourly average forecasts.
models.gfs_quarter_deg_3hour_to_hourly_mean(…) Take 3 hr GFS and convert it to hourly average data.
models.gfs_quarter_deg_hourly_to_hourly_mean(…) Take 1 hr GFS and convert it to hourly average data.
models.gfs_quarter_deg_to_hourly_mean(…[, …]) Hourly average forecasts derived from GFS 1, 3, and 12 hr frequency output.
models.nam_12km_hourly_to_hourly_instantaneous(…) Hourly instantantaneous forecast.
models.nam_12km_cloud_cover_to_hourly_mean(…) Hourly average forecast.
models.gefs_half_deg_to_hourly_mean(…[, …]) Hourly average forecasts derived from GEFS 3, 6, and 12 hr frequency output.

All of the above functions return weather forecast data, a resampler function, and a solar position calculation function. The weather forecast data may be supplied to a PV model and then resampled using the resampler function. This workflow allows for seperation of weather data processing and PV modeling while preserving the ability to use more accurate, shorter time interval inputs to the PV model. In the case of probabilistic forecasts, the resampler function also may define how an ensemble of deterministic forecasts should be translated to a probabilistic forecast. Finally, these functions return a solar position calculation function (rather than the actual solar position) to simplify the API while maintaining reasonable performance. (Solar position is only sometimes needed within the model processing functions and is only needed externally if power is to be calculated.) See models module for additional documentation.

Many of the functions in models rely on common functions related to forecast processing. These functions are found in forecast.

forecast Functions for forecasting.
forecast.cloud_cover_to_ghi_linear(…[, offset]) Convert cloud cover to GHI using a linear relationship.
forecast.cloud_cover_to_irradiance_ghi_clear(…) Estimates irradiance from cloud cover in the following steps:
forecast.cloud_cover_to_irradiance(latitude, …) Estimates irradiance from cloud cover in the following steps:
forecast.resample(arg[, freq, label]) Resamples an argument, allowing for None.
forecast.reindex_fill_slice(arg[, freq, …]) Reindex data to shorter intervals (create NaNs) from start to end, fill NaNs in gaps using fill_method, fill NaNs before first valid time using bfill, fill NaNs after last valid time using ffill, then slice output from start_slice to end_slice.
forecast.unmix_intervals(mixed[, lower, upper]) Convert mixed interval averages into pure interval averages.
forecast.sort_gefs_frame(frame) Sort a DataFrame from a GEFS forecast.

Persistence

The solarforecastarbiter supports several varieties of deterministic and probabilistic persistence forecasts.

persistence Functions for persistence forecasts.
persistence.persistence_scalar(observation, …) Make a persistence forecast using the mean value of the observation from data_start to data_end.
persistence.persistence_interval(…) Make a persistence forecast for an observation using the mean values of each interval_length bin from data_start to data_end.
persistence.persistence_scalar_index(…) Calculate a persistence forecast using the mean value of the observation clear sky index or AC power index from data_start to data_end.
persistence.persistence_probabilistic(…) Make a probabilistic persistence forecast using the observation from data_start to data_end.
persistence.persistence_probabilistic_timeofday(…) Make a probabilistic persistence forecast using the observation from data_start to data_end, matched by time of day (e.g.

Automated Generation

The Solar Forecast Arbiter runs workers that automatically create reference forecasts. This feature is currently limited to reference forecasts defined by the Arbiter itself. In the future we hope to extend this to user-defined reference forecasts.

Automated generation of reference NWP forecasts is achieved by adding a set of parameters to a Forecast’s extra_parameters (formatted as JSON). These parameters are:

  • is_reference_forecast - true or ‘true’ for automated generation
  • model - string of one of the functions found in models
  • piggyback_on - optional, the ID of another Forecast object to group together when making forecasts. For example, if ForecastA has variable ac_power and ForecastB has variable for ghi for the same site, ForecastB piggyback_on can be set to the forecast_id of ForecastA. Then these forecasts would be grouped together and the values of ForecastB would be the same GHI values that were used in the generation of ForecastA.

An example of a valid extra_parameters JSON for automated generation is:

{
    "is_reference_forecast": true,
    "model": "gfs_quarter_deg_hourly_to_hourly_mean",
    "piggyback_on": "da2bc386-8712-11e9-a1c7-0a580a8200ae"
}

The function make_latest_nwp_forecasts() is responsible for listing all forecasts available to a user and generating the appropriate reference NWP forecasts. In practice, the CLI script referencenwp() is called as a cronjob using an appropriate reference user account to continuously update pre-defined reference forecasts.

main.make_latest_nwp_forecasts(token, …[, …]) Make all reference NWP forecasts for run_time that are within issue_buffer of the next issue time for the forecast.