Compute observed coverage from a quantile forecaster
compute_coverage( predictions_cards, geo_type = c("county", "hrr", "msa", "dma", "state", "hhs", "nation"), grp_vars = c("forecaster", "forecast_date", "ahead"), avg_vars = c("geo_value"), backfill_buffer = 0 )
| predictions_cards | tibble of predictions
that are all for the same prediction task, meaning they are for the same
response, incidence period,and geo type. Forecasts may be for a
different forecast date or forecaster.
A predictions card may be created by the function
|
|---|---|
| geo_type | String indicating geographical type, such as "county", or "state". See the COVIDcast Geographic Coding documentation for available options. |
| grp_vars | character vector of named columns in the score_card at which average performance will be returned |
| avg_vars | character vector of named columns in the score_card over which averaging performance will be computed |
| backfill_buffer | How many days until response is deemed trustworthy enough to be taken as correct? |
A tibble containing grp_vars, nominal_prob (the claimed interval coverage), prop_below (the proportion of actual values falling below the lower end of the confidence interval), prop_above (the proportion of actual values falling above the upper end of the confidence interval), and prop_covered (the proportion falling inside the interval). All proportions are calculated for each available symmetric interval at each combination of grouping variables by averaging over any variables listed in avg_vars.
Checks are performed to ensure that averaging variables all contain the same confidence intervals though these may vary over grouping variables. avg_vars and grp_vars must have an empty intersection.