# solarforecastarbiter.metrics.probabilistic.quantile_score¶

solarforecastarbiter.metrics.probabilistic.quantile_score(obs, fx, fx_prob)[source]

Quantile Score (QS).

$\text{QS} = \frac{1}{n} \sum_{i=1}^n (fx_i - obs_i) * (p - 1\{obs_i > fx_i\})$

where $$n$$ is the number of forecasts, $$obs_i$$ is an observation, $$fx_i$$ is a forecast, $$1\{obs_i > fx_i\}$$ is an indicator function (1 if $$obs_i > fx_i$$, 0 otherwise) and $$p$$ is the probability that $$obs_i <= fx_i$$. [1] [2]

If $$obs > fx$$, then we have:

$\begin{split}(fx - obs) < 0 \\ (p - 1\{obs > fx\}) = (p - 1) <= 0 \\ (fx - obs) * (p - 1) >= 0\end{split}$

If instead $$obs < fx$$, then we have:

$\begin{split}(fx - obs) > 0 \\ (p - 1\{obs > fx\}) = (p - 0) >= 0 \\ (fx - obs) * p >= 0\end{split}$

Therefore, the quantile score is non-negative regardless of the obs and fx.

Parameters: obs ((n,) array_like) – Observations (physical unit). fx ((n,) array_like) – Forecasts (physical units) of the right-hand-side of a CDF interval, e.g., fx = 10 MW is interpreted as forecasting <= 10 MW. fx_prob ((n,) array_like) – Probability [%] associated with the forecasts. qs (float) – The Quantile Score, with the same units as the observations.

Notes

Quantile score is meant to be computed for a single probability of $$n$$ samples.

Examples

>>> obs = 100     # observation [MW]
>>> fx = 80       # forecast [MW]
>>> fx_prob = 60  # probability [%]
>>> quantile_score(obs, fx, fx_prob)   # score [MW]
8.0


References

 [1] Koenker and Bassett, Jr. (1978) “Regression Quantiles”, Econometrica 46 (1), pp. 33-50. doi: 10.2307/1913643
 [2] Wilks (2020) “Forecast Verification”. In “Statistical Methods in the Atmospheric Sciences” (3rd edition). Academic Press. ISBN: 9780123850225