solarforecastarbiter.metrics.probabilistic.continuous_ranked_probability_score

solarforecastarbiter.metrics.probabilistic.continuous_ranked_probability_score(obs, fx, fx_prob)

Continuous Ranked Probability Score (CRPS).

CRPS = 1/n sum_{i=1}^n int (F_i - O_i)^2 dx

where F_i is the CDF of the forecast at time i and O_i is the CDF associated with the observed value obs_i:

O_{i, j} = 1 if obs_i <= fx_{i, j}, else O_{i, j} = 0

where obs_i is the observation at time i, and fx_{i, j} is the forecast at time i for CDF interval j.

Parameters:

• obs ((n,) array_like) – Observations (physical unit).
• fx ((n, d) array_like) – Forecasts (physical units) of the right-hand-side of a CDF with d intervals (d >= 2), e.g., fx = [10 MW, 20 MW, 30 MW] is interpreted as <= 10 MW, <= 20 MW, <= 30 MW.
• fx_prob ((n, d) array_like) – Probability [%] associated with the forecasts.

Returns:

crps (float) – The Continuous Ranked Probability Score [unitless].

Raises:

ValueError – If the forecasts have incorrect dimensions; either a) the forecasts are for a single sample (n=1) with d CDF intervals but are given as a 1D array with d values or b) the forecasts are given as 2D arrays (n,d) but do not contain at least 2 CDF intervals (i.e. d < 2).

Examples

Forecast probabilities of <= 10 MW and <= 20 MW: >>> fx = np.array([[10, 20], [10, 20]]) >>> fx_prob = np.array([[30, 50], [65, 100]]) >>> obs = np.array([8, 12]) >>> continuous_ranked_probability_score(obs, fx, fx_prob) 4.5625

Forecast thresholds for constant probabilities (25%, 75%): >>> fx = np.array([[5, 15], [8, 14]]) >>> fx_prob = np.array([[25, 75], [25, 75]]) >>> obs = np.array([8, 10]) >>> continuous_ranked_probability_score(obs, fx, fx_prob) 0.5