Publications Using CTIS

Since 2020, academic and nonprofit researchers have used data from the US COVID-19 Trends and Impact Survey in peer-reviewed research publications.

Our primary description of the survey and its results over the first year of operation was published as part of the special feature “Beyond Cases and Deaths: The Benefits of Auxiliary Data Streams in Tracking the COVID-19 Pandemic” in PNAS:

• J. Salomon, A. Reinhart, A. Bilinski, E. J. Chua, W. La Motte-Kerr, M. M. Rönn, M. B. Reitsma, K. A. Morris, S. LaRocca, T. H. Farag, F. Kreuter, R. Rosenfeld, and R. J. Tibshirani (2021). The US COVID-19 Trends and Impact Survey: Continuous real-time measurement of COVID-19 symptoms, risks, protective behaviors, testing, and vaccination. Proceedings of the National Academy of Sciences 118 (51) e2111454118.

Peer Reviewed Papers

Research publications using the survey data include:

• K.M. Geffel, H.P. Dyer, A.D. Casas, S.N. Christian-Afflu, D.D. Méndez, and T.L. Gary-Webb (2024). COVID-19 vaccine uptake and intention of Black adults: A county-wide analysis of an online survey. Journal of the National Medical Association.
• C.K. Ettman, E. Badillo-Goicoechea, E.A. Stuart, and L.T. Dean (2024). Area-level credit scores and symptoms of depression and anxiety in adults. American Journal of Epidemiology kwae275.
• C.K. Ettman, E. Badillo-Goicoechea, E.A. Stuart (2024). Financial strain, schooling modality and mental health of US adults living with children during the COVID-19 pandemic. Journal of Epidemiology & Community Health.
• K. Sasse, R. Mahabir, O. Gkountouna, A. Crooks, A. Croitoru (2024). Understanding the determinants of vaccine hesitancy in the United States: A comparison of social surveys and social media. PLoS ONE 19(6), e0301488.
• Y. Yang, W. Dempsey, P. Han, Y. Deshmukh, S. Richardson, B. Tom, and B. Mukherjee (2024). Exploring the Big Data Paradox for various estimands using vaccination data from the global COVID-19 Trends and Impact Survey (CTIS). Science Advances 10 (22).
• Perofsky, A.C., Hansen, C.L., Burstein, R. et al (2024). Impacts of human mobility on the citywide transmission dynamics of 18 respiratory viruses in pre- and post-COVID-19 pandemic years. Nature Communications 15, 4164.
• Z. Yang, R. Krishnan, and B. Li (2024). The interplay between individual mobility, health risk, and economic choice: A holistic model for COVID-19 policy intervention. INFORMS Journal on Data Science 3 (1), 6-27.
• A. Srivastava, J. M. Ramirez, S. Díaz-Aranda, J. Aguilar, A. F. Anta, A. Ortega, and R. E. Lillo (2024). Nowcasting temporal trends using indirect surveys. In Proceedings of the 38th AAAI Conference on Artificial Intelligence 38, 22359–22367.
• P. Porebski, S. Venkatramanan, A. Adiga, B. Klahn, B. Hurt, M. L. Wilson, J. Chen, A. Vullikanti, M. Marathe & B. Lewis (2024). Data-driven mechanistic framework with stratified immunity and effective transmissibility for COVID-19 scenario projections. Epidemics 47, 100761.
• V. Nelson, B. Bashyal, P.-N. Tan & Y. A. Argyris (2024). Vaccine rhetoric on social media and COVID-19 vaccine uptake rates: A triangulation using self-reported vaccine acceptance. Social Science & Medicine 348, 116775.
• R.R. Andridge (2024). Using proxy pattern-mixture models to explain bias in estimates of COVID-19 vaccine uptake from two large surveys. Journal of the Royal Statistical Society Series A: Statistics in Society.
• H. Bui, S. Ekşioğlu, R. Proano, and S. N. Pinkley (2024). An Analysis of COVID-19 Vaccine Hesitancy in the U.S.. IISE Transactions.
• de Vries, M., Kim, J.Y. & Han, H. (2023). The unequal landscape of civic opportunity in America. Nature Human Behavior 8, 256-263.
• E. Tuzhilina, T. J. Hastie, D. J. McDonald, J. K. Tay & R. Tibshirani (2023). Smooth multi-period forecasting with application to prediction of COVID-19 cases. Journal of Computational and Graphical Statistics 33 (3), 955-967.
• W. Dempsey (2023). Addressing selection bias and measurement error in COVID-19 case count data using auxiliary information. Annals of Applied Statistics 17 (4), 2903-2923.
• Ma, M.Z., Chen, S.X. (2023). Beyond the surface: accounting for confounders in understanding the link between collectivism and COVID-19 pandemic in the United States. BMC Public Health 23, 1513.
• C.K. Ettman, E. Badillo Goicoechea, and E.A. Stuart (2023). Evolution of depression and anxiety over the COVID-19 pandemic and across demographic groups in a large sample of U.S. adults. AJPM Focus 2 (4), 100140.
• M. Rubinstein, Z. Branson, and E.H. Kennedy (2023). Heterogeneous interventional effects with multiple mediators: Semiparametric and nonparametric approaches. Journal of Causal Inference 11 (1), 20220070.
• Uyheng, J., Robertson, D.C. & Carley, K.M. (2023). Bridging online and offline dynamics of the face mask infodemic. BMC Digital Health 1, 27.
• Kobayashi H, Saenz-Escarcega R, Fulk A, Agusto FB (2023). Understanding mental health trends during COVID-19 pandemic in the United States using network analysis. PLoS ONE 18(6): e0286857.
• GK Charles, SP Braunstein, JL Barker, et al (2023). How do psychobehavioural variables shed light on heterogeneity in COVID-19 vaccine acceptance? Evidence from United States general population surveys on a probability panel and social media. BMJ Open 13:e066897.
• S. Soorapanth, R. Cheung, X. Zhang, A. H. Mokdad, G. A. Mensah (2023). Rural–Urban Differences in Vaccination and Hesitancy Rates and Trust: US COVID-19 Trends and Impact Survey on a Social Media Platform, May 2021–April 2022. American Journal of Public Health 113 (6), 680-688.
  • See also the associated editorial: T. Callaghan (2023). Vaccine Uptake and Hesitancy in Rural America in the Wake of the COVID-19 Pandemic. American Journal of Public Health 113 (6), 615-617.
• M. Rubinstein, A. Haviland, and J. Breslau (2023). The effect of COVID-19 vaccinations on self-reported depression and anxiety during February 2021. Statistics and Public Policy 10 (1), 2190008.
• Rönn MM, Menzies NA, Salomon JA (2023). Vaccination and voting patterns in the United States: analysis of COVID-19 and flu surveys from 2010 to 2022. American Journal of Preventive Medicine 65 (3), 458-466.
• Taube JC, Susswein Z, Bansal S (2023). Spatiotemporal Trends in Self-Reported Mask-Wearing Behavior in the United States: Analysis of a Large Cross-sectional Survey. JMIR Public Health and Surveillance 9:e42128.
• Rebecca L. Weintraub et al (2023). Identifying COVID-19 Vaccine Deserts and Ways to Reduce Them: A Digital Tool to Support Public Health Decision-Making. American Journal of Public Health 113 (4), 363-367.
• Anzalone AJ, Sun J, Vinson AJ, Beasley WH, Hillegass WB, Murray K, et al. (2023). Community risks for SARS-CoV-2 infection among fully vaccinated US adults by rurality: A retrospective cohort study from the National COVID Cohort Collaborative. PLoS ONE 18(1): e0279968.
• Rufino, J., Baquero, C., Frey, D. et al (2023). Using survey data to estimate the impact of the omicron variant on vaccine efficacy against COVID-19 infection. Scientific Reports 13, 900.
• J. G. Lu (2023). Two large-scale global studies on COVID-19 vaccine hesitancy over time: Culture, uncertainty avoidance, and vaccine side-effect concerns. Journal of Personality and Social Psychology 124 (4), 683-706.
• C. Lupton-Smith, E. Badillo Goicoechea, M. Collins, J. Lessler, M. K. Grabowski & E. A. Stuart (2023). Consistency between Household and County Measures of Onsite Schooling during the COVID-19 Pandemic. Journal of Research on Educational Effectiveness 16 (3), 419-441.
• Rader, B., Astley, C.M., Sewalk, K. et al (2022). Spatial modeling of vaccine deserts as barriers to controlling SARS-CoV-2. Communications Medicine 2, 141.
• Nguyen, Q.C., Yardi, I., Gutierrez, F.X.M. et al. (2022). Leveraging 13 million responses to the U.S. COVID-19 Trends and Impact Survey to examine vaccine hesitancy, vaccination, and mask wearing, January 2021-February 2022. BMC Public Health 22, 1911.
• J. M. Cox-Ganser, P. K. Henneberger, D. N. Weissman, G. Guthrie, and C. P. Groth (2022). COVID-19 test positivity by occupation using the Delphi US COVID-19 Trends and Impact Survey, September–November 2020. American Journal of Industrial Medicine 65 (9), 721-730.
• M. Jahja, A. Chin, and R.J. Tibshirani (2022). Real-Time Estimation of COVID-19 Infections: Deconvolution and Sensor Fusion. Statistical Science 37 (2), 207-228.
• Henneberger, PK, Cox-Ganser, JM, Guthrie, GM, Groth, CP (2022). Estimates of COVID-19 vaccine uptake in major occupational groups and detailed occupational categories in the United States, April–May 2021. American Journal of Industrial Medicine 65 (7), 525-536.
• K. E. Wiens, C. P. Smith, E. Badillo-Goicoechea, K. H. Grantz, M. K. Grabowski, A. S. Azman, E. A. Stuart, and J. Lessler (2022). In-person schooling and associated COVID-19 risk in the United States over spring semester 2021. Science Advances 8, eabm9128.
• F. Petersen, A. Errore, and P. Karaca-Mandic (2022). Lifting statewide mask mandates and COVID-19 cases: A synthetic control study. Medical Care 60 (7), 538-544.
• F. Pierri, B. L. Perry, M. R. DeVerna, et al. (2022). Online misinformation is linked to early COVID-19 vaccination hesitancy and refusal. Scientific Reports 12, 5966.
• A. Pilehvari, J. Ton, M. R. Mohan, A. Marathe, and A. Vullikanti (2022). Drivers and Predictors of COVID-19 Vaccine Hesitancy in Virginia. In: Yang, Z., von Briesen, E. (eds), Proceedings of the 2021 Conference of The Computational Social Science Society of the Americas. CSSSA 2021.
• L. S. Flor, J. Friedman, C. N. Spencer, et al. (2022). Quantifying the effects of the COVID-19 pandemic on gender equality on health, social, and economic indicators: a comprehensive review of data from March, 2020, to September, 2021. The Lancet 399 (10344), 2381-2397.
• D. P. Do and R. Frank (2022). Prior COVID-19 infection: an underappreciated factor in vaccine hesitancy in the USA. Journal of Public Health 44 (2), 471-474.
• W. C. King, M. Rubinstein, A. Reinhart, and R. J. Mejia (2021). Time trends, factors associated with, and reasons for COVID-19 vaccine hesitancy: A massive online survey of US adults from January-May 2021. PLoS ONE 16 (12), e0260731.
• C. Lupton-Smith, E. Badillo-Goicochea, T.-H. Chang, H. Maniates, K. E. Riehm, I. Schmid, and E. A. Stuart (2021). Factors associated with county-level mental health during the COVID-19 pandemic. Journal of Community Psychology 50 (5), 2431-2442.
• D. P. Do and R. Frank (2021). U.S. frontline workers and COVID-19 inequities. Preventive Medicine 153, 106833.
• W. C. King, M. Rubinstein, A. Reinhart, and R. J. Mejia (2021). COVID-19 vaccine hesitancy January-May 2021 among 18–64 year old US adults by employment and occupation. Preventive Medicine Reports 24, 101569.
• C. H. Sudre, A. Keshet, M. S. Graham, A. D. Joshi, S. Shilo, H. Rossman, B. Murray, E. Molteni, K. Klaser, L. D. Canas, M. Antonelli, L. H. Nguyen, D. A. Drew, M. Modat, J. Capdevila Pujol, S. Ganesh, J. Wolf, T. Meir, A. T. Chan, C. J. Steves, T. D. Spector, J. S. Brownstein, E. Segal, S. Ourselin, and C. M. Astley (2021). Anosmia, ageusia, and other COVID-19-like symptoms in association with a positive SARS-CoV-2 test, across six national digital surveillance platforms: an observational study. The Lancet Digital Health 3 (9), e577-e586.
• R. Sukumaran, P. Patwa, S. T V, S. Shankar, R. Kanaparti, J. Bae, Y. Mathur, A. Singh, A. Chopra, M. Kang, P. Ramaswamy, and R. Raskar (2021). COVID-19 outbreak prediction and analysis using self reported symptoms. Journal of Behavioral Data Science 1 (1), 154–169.
• D. Adjodah, K. Dinakar, M. Chinazzi, S. P. Fraiberger, A. Pentland, S. Bates, K. Staller, A. Vespignani, and D. L. Bhatt (2021). Association between COVID-19 outcomes and mask mandates, adherence, and attitudes. PLoS ONE 16 (6), e0252315.
• C. M. Zipfel, V. Colizza, and S. Bansal (2021). The missing season: The impacts of the COVID-19 pandemic on influenza. Vaccine 39 (28), 3645-3648.
• J. Lessler, M. K. Grabowski, K. H. Grantz, E. Badillo-Goicoechea, C. J. E. Metcalf, C. Lupton-Smith, A. S. Azman, and E. A. Stuart (2021). Household COVID-19 risk and in-person schooling. Science 372 (6545), 1092-1097.
• Doerr, A.J. (2021). Locked (Down) and Loaded (Language): Effect of Policy and Speech on COVID-19 Outcomes. Journal of Leadership & Organizational Studies 28 (3), 340-348.
• Fischer CB, Adrien N, Silguero JJ, Hopper JJ, Chowdhury AI, Werler MM (2021). Mask adherence and rate of COVID-19 across the United States. PLoS ONE 16 (4), e0249891.
• Bilinski, A., Emanuel, E., Salomon, J.A. and Venkataramani, A. (2021). Better Late Than Never: Trends in COVID-19 Infection Rates, Risk Perceptions, and Behavioral Responses in the USA. Journal of General Internal Medicine 36, 1825-1828.
• Molteni, E., Astley, C.M., Ma, W. et al. (2021). Symptoms and syndromes associated with SARS-CoV-2 infection and severity in pregnant women from two community cohorts. Scientific Reports 11, 6928.
• Rebeiro, P., Aronoff, D, and Smith, M.K. (2021). The Impact of State Mask-Wearing Requirements on the Growth of COVID-19 Cases, Hospitalizations, and Deaths in the United States. Clinical Infectious Diseases 73 (9), 1703–1706.
• Rader, White, Burns, Chen, Brilliant, Cohen, Shaman, Brilliant, Kraemer, Moritz, Hawkins, Scarpino, Astley, and Brownstein (2021). Mask-wearing and control of SARS-CoV-2 transmission in the USA: a cross-sectional study. Lancet Digital Health 3 (3), e148-e157.
• Flaxman AD, Henning DJ and Duber HC (2020). The relative incidence of COVID-19 in healthcare workers versus non-healthcare workers: evidence from a web-based survey of Facebook users in the United States. Gates Open Research, 4:174.
• Kreuter, F., Barkay, N., Bilinski, A., Bradford, A., Chiu, S., Eliat, R., Fan, J., Galili, T., Haimovich, D., Kim, B., LaRocca, S., Li, Y., Morris, K., Presser, S., Sarig, T., Salomon, J. A., Stewart, K., Stuart, E. A., & Tibshirani, R. J. (2020). Partnering with a global platform to inform research and public policy making. Survey Research Methods, 14 (2), 159-163.

Blog Posts and Reports

• Surgo Ventures, Resolve to Save Lives (2021). COVID-19 Vaccine Precision Response Toolkit: An End-to-End Vaccination Improvement Framework to Improve COVID-19 Vaccine Uptake.
• Alejandra Arrieta, Emmanuela Gakidou, Heidi Larson, Erin Mullany, and Christopher Troeger, April 1, 2021. Through Understanding and Empathy, We Can Convince Women to Get the COVID-19 Vaccine. Think Global Health.
• Joseph Friedman, Silvia Montoya and Emmanuela Gakidou, March 8, 2021. Gender Equality in the Global Return to School. Think Global Health.
• Leanna Morinishi, Kris Barkume, Esther Kim, and Alex Reinhart, February 2, 2021. Home for the Holidays? The Impact of US Holidays on Social Behaviors and Preventative Measures. Delphi blog.
• Alex Reinhart, Esther Kim, Andy Garcia, and Sarah LaRocca, January 28, 2021. Using the COVID-19 Symptom Survey to Track Vaccination Uptake and Sentiment in the United States. Delphi blog.
• Alex Reinhart, October 12, 2020. New and Improved COVID Symptom Survey Tracks Testing and Mask-Wearing. Delphi blog.
• Ryan Tibshirani, September 21, 2020. Can Symptom Surveys Improve COVID-19 Forecasts? Delphi blog.
• Alex Reinhart and Ryan Tibshirani, August 26, 2020. COVID-19 Symptom Surveys through Facebook. Delphi blog.

Send Your Publications

If you have used the survey data, or the aggregate data available in the COVIDcast API, to publish research results, please contact us at delphi-survey-info@lists.andrew.cmu.edu so we can include your work here.