Compute calibration of a quantile forecaster
compute_calibration( predictions_cards, geo_type, backfill_buffer = 10, grp_vars = c("forecaster", "forecast_date", "ahead"), avg_vars = c("geo_value") )
| predictions_cards | tibble containing at least columns actual, quantile, value and any grouping or averaging variables named in the next arguments |
|---|---|
| geo_type | String indicating geographical type, such as "county", or "state". See the COVIDcast Geographic Coding documentation for available options. |
| backfill_buffer | How many days until response is deemed trustworthy enough to be taken as correct? |
| grp_vars | character vector of named columns in the score_card at which average performance will be returned |
| avg_vars | character vector of named columns in the score_card over which averaging performance will be computed |
A tibble containing grp_vars, nominal_prob (the claimed interval coverage), prop_below (the proportion of actual values falling below the forecast quantile) and prop_above (the proportion of actual values falling above the forecast quantile). All proportions are calculated for each available probability at each combination of grouping variables by averaging over any variables listed in avg_vars.
Note that no checks are performed to ensure that averaging variables all contain the same predicted quantiles. avg_vars and grp_vars must have an empty intersection.