solarforecastarbiter.metrics.probabilistic.quantile_score¶
-
solarforecastarbiter.metrics.probabilistic.
quantile_score
(obs, fx, fx_prob)[source]¶ Quantile Score (QS).
\[\text{QS} = \frac{1}{n} \sum_{i=1}^n (fx_i - obs_i) * (p - 1\{obs_i > fx_i\})\]where \(n\) is the number of forecasts, \(obs_i\) is an observation, \(fx_i\) is a forecast, \(1\{obs_i > fx_i\}\) is an indicator function (1 if \(obs_i > fx_i\), 0 otherwise) and \(p\) is the probability that \(obs_i <= fx_i\). [1] [2]
If \(obs > fx\), then we have:
\[\begin{split}(fx - obs) < 0 \\ (p - 1\{obs > fx\}) = (p - 1) <= 0 \\ (fx - obs) * (p - 1) >= 0\end{split}\]If instead \(obs < fx\), then we have:
\[\begin{split}(fx - obs) > 0 \\ (p - 1\{obs > fx\}) = (p - 0) >= 0 \\ (fx - obs) * p >= 0\end{split}\]Therefore, the quantile score is non-negative regardless of the obs and fx.
Parameters: - obs ((n,) array_like) – Observations (physical unit).
- fx ((n,) array_like) – Forecasts (physical units) of the right-hand-side of a CDF interval, e.g., fx = 10 MW is interpreted as forecasting <= 10 MW.
- fx_prob ((n,) array_like) – Probability [%] associated with the forecasts.
Returns: qs (float) – The Quantile Score, with the same units as the observations.
Notes
Quantile score is meant to be computed for a single probability of \(n\) samples.
Examples
>>> obs = 100 # observation [MW] >>> fx = 80 # forecast [MW] >>> fx_prob = 60 # probability [%] >>> quantile_score(obs, fx, fx_prob) # score [MW] 8.0
References
[1] Koenker and Bassett, Jr. (1978) “Regression Quantiles”, Econometrica 46 (1), pp. 33-50. doi: 10.2307/1913643 [2] Wilks (2020) “Forecast Verification”. In “Statistical Methods in the Atmospheric Sciences” (3rd edition). Academic Press. ISBN: 9780123850225