solarforecastarbiter.metrics.probabilistic.brier_score

solarforecastarbiter.metrics.probabilistic.brier_score(obs, fx, fx_prob)[source]

Brier Score (BS).

BS = 1/n sum_{i=1}^n (f_i - o_i)^2

where n is the number of forecasts, f_i is the forecasted probability of event i, and o_i is the observed event indicator (o_i=0: event did not occur, o_i=1: event occured). The forecasts are supplied as the right-hand-side of a CDF interval, e.g., forecast <= 10 MW at time i, and therefore o_i is defined as:

o_i = 1 if obs_i <= fx_i, else o_i = 0

where fx_i and obs_i are the forecast and observation at time i, respectively.

Parameters:
  • obs ((n,) array_like) – Observations (physical unit).
  • fx ((n,) array_like) – Forecasts (physical units) of the right-hand-side of a CDF interval, e.g., fx = 10 MW is interpreted as forecasting <= 10 MW.
  • fx_prob ((n,) array_like) – Probability [%] associated with the forecasts.
Returns:

bs (float) – The Brier Score [unitless], bounded between 0 and 1, where values closer to 0 indicate better forecast performance and values closer to 1 indicate worse performance.

Notes

The Brier Score implemented in this function is for binary outcomes only, rather than the more general (but less commonly used) categorical version.