Bayesian Analysis Reporting Guidelines

John K. Kruschke

doi:10.1038/s41562-021-01177-7

journal article Open Access Aug 16, 2021

Bayesian Analysis Reporting Guidelines

John K. Kruschke

Nature Human Behaviour Vol. 5 No. 10 pp. 1282-1291 · Springer Science and Business Media LLC

View at Publisher Save 10.1038/s41562-021-01177-7

Abstract

AbstractPrevious surveys of the literature have shown that reports of statistical analyses often lack important information, causing lack of transparency and failure of reproducibility. Editors and authors agree that guidelines for reporting should be encouraged. This Review presents a set of Bayesian analysis reporting guidelines (BARG). The BARG encompass the features of previous guidelines, while including many additional details for contemporary Bayesian analyses, with explanations. An extensive example of applying the BARG is presented. The BARG should be useful to researchers, authors, reviewers, editors, educators and students. Utilization, endorsement and promotion of the BARG may improve the quality, transparency and reproducibility of Bayesian analyses.

Topics

No keywords indexed for this article. Browse by subject →

References

103

[1]

Vandekerckhove, J., Rouder, J. N. & Kruschke, J. K. Editorial: Bayesian methods for advancing psychological science. Psychonomic Bull. Rev. 25, 1–4 (2018). 10.3758/s13423-018-1443-8

[2]

van de Schoot, R., Winter, S. D., Ryan, O., Zondervan-Zwijnenburg, M. & Depaoli, S. A systematic review of Bayesian articles in psychology: the last 25 years. Psychol. Methods 22, 217–239 (2017). 10.1037/met0000100

[3]

Pibouleau, L. & Chevret, S. Bayesian statistical method was underused despite its advantages in the assessment of implantable medical devices. J. Clin. Epidemiol. 64, 270–279 (2011). 10.1016/j.jclinepi.2010.03.018

[4]

Sung, L. et al. Seven items were identified for inclusion when reporting a Bayesian analysis of a clinical study. J. Clin. Epidemiol. 58, 261–268 (2005). 10.1016/j.jclinepi.2004.08.010

[5]

Sobieraj, D. M. et al. Methods used to conduct and report Bayesian mixed treatment comparisons published in the medical literature: a systematic review. BMJ Open 3, e003111 (2013). 10.1136/bmjopen-2013-003111

[6]

Zhai, J. et al. Reporting of core items in hierarchical Bayesian analysis for aggregating N-of-1 trials to estimate population treatment effects is suboptimal. J. Clin. Epidemiol. 76, 99–107 (2016). 10.1016/j.jclinepi.2016.02.023

[7]

Improving transparency and replication in Bayesian statistics: The WAMBS-Checklist.

Sarah Depaoli, Rens van de Schoot

Psychological Methods 2017 10.1037/met0000065

[8]

Rietbergen, C., Debray, T. P. A., Klugkist, I., Janssen, K. J. M. & Moons, K. G. M. Reporting of Bayesian analysis in epidemiologic research should become more transparent. J. Clin. Epidemiol. 86, 51–58 (2017). 10.1016/j.jclinepi.2017.04.008

[9]

Simera, I. The EQUATOR network: supporting editors in publishing well-reported health research. Sci. Editor 37, 15–16 (2014). 18.

[10]

Fuller, T., Pearson, M., Peters, J. & Anderson, R. What affects authors’ and editors’ use of reporting guidelines? Findings from an online survey and qualitative interviews. PLoS ONE 10, e0121585 (2015). 10.1371/journal.pone.0121585

[11]

Improving the Reporting Quality of Nonrandomized Evaluations of Behavioral and Public Health Interventions: The TREND Statement

Don C. Des Jarlais, Cynthia Lyles, Nicole Crepaz

American Journal of Public Health 2004 10.2105/ajph.94.3.361

[12]

Spiegelhalter, D. J., Myles, J. P., Jones, D. R. & Abrams, K. R. Bayesian methods in health technology assessment: a review. Health Technol. Assess. 4, 1–130 (2000). 10.3310/hta4380

[13]

Gatsonis, C. & Goodman, S. Bayesian Standards in Science—Standards for Reporting of Bayesian Analyses in the Scientific Literature (The BaSiS Group, 2001); http://lib.stat.cmu.edu/bayesworkshop/2001/BaSis.html

[14]

Kruschke, J. K. Doing Bayesian Data Analysis: A Tutorial with R and BUGS (Academic, 2011).

[15]

Kruschke, J. K. Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan, 2nd edn (Academic, 2015). 10.1016/b978-0-12-405888-0.00008-8

[16]

Basic statistical reporting for articles published in Biomedical Journals: The “Statistical Analyses and Methods in the Published Literature” or the SAMPL Guidelines

Thomas A. Lang, Douglas G. Altman

International Journal of Nursing Studies 2015 10.1016/j.ijnurstu.2014.09.006

[17]

Baldwin, S. A. & Larson, M. J. An introduction to using Bayesian linear regression with clinical data. Behav. Res. Ther. 98, 58–75 (2017). 10.1016/j.brat.2016.12.016

[18]

van de Schoot, R., Veen, D., Smeets, L., Winter, S. D. & Depaoli, S. in Small Sample Size Solutions (eds van de Schoot, R. & Miocevic, M.) 30–49 (Routledge, 2020). 10.4324/9780429273872-4

[19]

van de Schoot, R. et al. Bayesian statistics and modelling. Nat. Rev. Methods Prim. 1, 3 (2021). 10.1038/s43586-020-00003-0

[20]

Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report.

Mark Appelbaum, Harris Cooper, Rex B. Kline et al.

American Psychologist 2018 10.1037/amp0000191

[21]

van Doorn, J. et al. The JASP guidelines for conducting and reporting a Bayesian analysis. Psychonomic Bull. Rev. 28, 813–826 (2021). 10.3758/s13423-020-01798-5

[22]

Aczel, B. et al. Discussion points for Bayesian inference. Nat. Hum. Behav. 4, 561–563 (2020). 10.1038/s41562-019-0807-z

[23]

Sharp, M. K., Glonti, K. & Hren, D. Online survey about the STROBE statement highlighted divergent views about its content, purpose, and value. J. Clin. Epidemiol. 123, 100–106 (2020). 10.1016/j.jclinepi.2020.03.025

[24]

Pullenayegum, E. M., Guo, Q. & Hopkins, R. B. Developing critical thinking about reporting of Bayesian analyses. J. Stat. Educ. 20, 1 (2012). 10.1080/10691898.2012.11889630

[25]

The file drawer problem and tolerance for null results.

Robert Rosenthal

Psychological Bulletin 1979 10.1037/0033-2909.86.3.638

[26]

Kerr, N. L. HARKing: hypothesizing after the results are known. Pers. Soc. Psychol. Rev. 2, 196–217 (1998). 10.1207/s15327957pspr0203_4

[27]

Simmons, J. P., Nelson, L. D. & Simonsohn, U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22, 1359–1366 (2011). 10.1177/0956797611417632

[28]

John, L. K., Loewenstein, G. & Prelec, D. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol. Sci. 23, 524–532 (2012). 10.1177/0956797611430953

[29]

Gelman, A. & Loken, E. The statistical crisis in science: data-dependent analysis—a “garden of forking paths”—explains why many statistically significant comparisons don’t hold up. Am. Sci. 102, 460–466 (2014). 10.1511/2014.111.460

[30]

Berry, S. M., Carlin, B. P., Lee, J. J. & Müller, P. Bayesian Adaptive Methods for Clinical Trials (CRC Press, 2011). 10.1201/ebk1439825488

[31]

Kruschke, J. K. & Liddell, T. M. The Bayesian new statistics: hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychon. Bull. Rev. 25, 178–206 (2018). 10.3758/s13423-016-1221-4

[32]

Schönbrodt, F. D. & Wagenmakers, E.-J. Bayes factor design analysis: planning for compelling evidence. Psychon. Bull. Rev. 25, 128–142 (2018). 10.3758/s13423-017-1230-y

[33]

Morey, R. D. & Rouder, J. N. Bayes factor approaches for testing interval null hypotheses. Psychol. Methods 16, 406–419 (2011). 10.1037/a0024377

[34]

Kelter, R. Analysis of Bayesian posterior significance and effect size indices for the two-sample t-test to support reproducible medical research. BMC Med. Res. Method. 20, 88 (2020). 10.1186/s12874-020-00968-2

[35]

Makowski, D., Ben-Shachar, M. S., Chen, S. H. A. & Lüdecke, D. Indices of effect existence and significance in the Bayesian framework. Front. Psychol. 10, 2767 (2019). 10.3389/fpsyg.2019.02767

[36]

Stanton, J. M. Evaluating equivalence and confirming the null in the organizational sciences. Organ. Res. Methods 24, 491–512 (2020). 10.1177/1094428120921934

[37]

Pek, J. & Van Zandt, T. Frequentist and Bayesian approaches to data analysis: evaluation and estimation. Psychol. Learn. Teach. 19, 21–35 (2020). 10.1177/1475725719874542

[38]

Wagenmakers, E.-J., Lee, M., Lodewyckx, T. & Iverson, G. J. in Bayesian Evaluation of Informative Hypotheses (eds Hoijtink, H., Klugkist, I. & Boelen, P. A.) 181–207 (Springer, 2008). https://doi.org/10.1007/978-0-387-09612-4_9 10.1007/978-0-387-09612-4_9

[39]

Hoeting, J. A., Madigan, D., Raftery, A. E. & Volinsky, C. T. Bayesian model averaging: a tutorial. Stat. Sci. 14, 382–401 (1999). 10.1214/ss/1009212519

[40]

Myung, I. J. & Pitt, M. A. Applying Occam’s razor in modeling cognition: a Bayesian approach. Psychon. Bull. Rev. 4, 79–95 (1997). 10.3758/bf03210778

[41]

Mindless statistics

Gerd Gigerenzer

The Journal of Socio-Economics 2004 10.1016/j.socec.2004.09.033

[42]

Gigerenzer, G., Krauss, S. & Vitouch, O. in The Sage Handbook of Quantitative Methodology for the Social Sciences (ed. Kaplan, D.) 391–408 (Sage, 2004).

[43]

The ASA Statement on p -Values: Context, Process, and Purpose

Ronald L. Wasserstein, Nicole A. Lazar

The American Statistician 2016 10.1080/00031305.2016.1154108

[44]

Schad, D. J., Betancourt, M. & Vasishth, S. Toward a principled Bayesian workflow in cognitive science. Psychol. Methods 26, 103–126 (2021). 10.1037/met0000275

[45]

Berger, J. O. & Mortera, J. Default bayes factors for nonnested hypothesis testing. J. Am. Stat. Assoc. 94, 542–554 (1999). 10.1080/01621459.1999.10474149

[46]

Jeffreys, H. Theory of Probability (Oxford Univ. Press, 1961).

[47]

Ly, A., Verhagen, J. & Wagenmakers, E.-J. Harold Jeffreys’s default Bayes factor hypothesis tests: explanation, extension, and application in psychology. J. Math. Psychol. 72, 19–32 (2016). 10.1016/j.jmp.2015.06.004

[48]

Rouder, J. N., Morey, R. D., Speckman, P. L. & Province, J. M. Default Bayes factors for ANOVA designs. J. Math. Psychol. 56, 356–374 (2012). 10.1016/j.jmp.2012.08.001

[49]

Lee, M. D. & Vanpaemel, W. Determining informative priors for cognitive models. Psychon. Bull. Rev. 25, 114–127 (2018). 10.3758/s13423-017-1238-3

[50]

Berger, J. O. & Pericchi, L. R. in IMS Lecture Notes—Monograph Series, Vol. 38 (ed. Lahiri, P.) 135–207 (2001). 10.1214/lnms/1215540968

Showing 50 of 103 references

Cited By

358

Trust, social movements, and the state

Malcolm Fairbrother, Matthias Penker · 2024

Journal of Trust Research

Rapid Alternate Monocular Deprivation Does Not Affect Binocular Balance and Correlation in Human Adults

Wenman Lin (林温曼), Junhan Wei (魏君涵) · 2022

eneuro

Statistical tools for water quality assessment and monitoring in river ecosystems – a scoping review and recommendations for data analysis

Stefan G. Schreiber, Sanja Schreiber · 2022

Water Quality Research Journal

Metrics

358

Citations

103

References

Details

Published: Aug 16, 2021
Vol/Issue: 5(10)
Pages: 1282-1291
License: View

Authors

J

John K. Kruschke

Cite This Article

John K. Kruschke (2021). Bayesian Analysis Reporting Guidelines. Nature Human Behaviour, 5(10), 1282-1291. https://doi.org/10.1038/s41562-021-01177-7

Bayesian Analysis Reporting Guidelines

You May Also Like