BishtRef
Articles Journals Publishers Authors Subjects Most Cited New Papers Year Stats Open Access
journal article Nov 02, 2018

Peer-Review Guidelines Promoting Replicability and Transparency in Psychological Science

William E. Davis Roger Giner-Sorolla D. Stephen Lindsay ORCID Jessica P. Lougheed ORCID Matthew C. Makel ORCID Matt E. Meier Jessie Sun ORCID Leigh Ann Vaughn John M. Zelenski
Advances in Methods and Practices in Psychological Science Vol. 1 No. 4 pp. 556-573 · SAGE Publications
View at Publisher Save 10.1177/2515245918806489
Abstract
More and more psychological researchers have come to appreciate the perils of common but poorly justified research practices and are rethinking commonly held standards for evaluating research. As this methodological reform expresses itself in psychological research, peer reviewers of such work must also adapt their practices to remain relevant. Reviewers of journal submissions wield considerable power to promote methodological reform, and thereby contribute to the advancement of a more robust psychological literature. We describe concrete practices that reviewers can use to encourage transparency, intellectual humility, and more valid assessments of the methods and statistics reported in articles.
Topics

No keywords indexed for this article. Browse by subject →

References
99
[1]
American Psychological Association (2010)
[2]
American Psychological Association. (2017). Ethical principles of psychologists and code of conduct. Retrieved from https://www.apa.org/ethics/code/ethics-code-2017.pdf
[4]
Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report.

Mark Appelbaum, Harris Cooper, Rex B. Kline et al.

American Psychologist 10.1037/amp0000191
[6]
Blohowiak B. B., Cohoon J., de-Wit L., Eich E., Farach F. J., Hasselman F. DeHaven A. C. (2018). Badges to acknowledge open practices. Retrieved from https://osf.io/tvyxz/
[10]
Yes, but what’s the mechanism? (don’t expect an easy answer).

John G. Bullock, Donald P. Green, SHANG E. HA

Journal of Personality and Social Psychology 10.1037/a0018933
[11]
Center for Open Science. (n.d.a). Registered Reports: Peer review before results are known to align scientific values and practices. Retrieved from https://cos.io/rr/
[12]
Center for Open Science. (n.d.b). [Transparency and Openness Promotion (TOP) guidelines]. Retrieved from https://cos.io/our-services/top-guidelines/
[13]
Champely S. (2018). pwr: Basic functions for power analysis (R package Version 1.2-2). [Computer software]. Retrieved from https://CRAN.R-project.org/package=pwr
[16]
A power primer.

Jacob Cohen

Psychological Bulletin 10.1037/0033-2909.112.1.155
[18]
Cumming G. (2011)
[20]
Cumming G. (2017)
[22]
Epskamp S., Nuijten M. B. (2016). statcheck: Extract statistics from articles and recompute p values (R package Version 1.2.2) [Computer software]. Retrieved from http://CRAN.R-project.org/package=statcheck
[24]
G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences

Franz Faul, Edgar Erdfelder, Albert-Georg Lang et al.

Behavior Research Methods 10.3758/bf03193146
[27]
Fried E. I., Flake J. K. (2018). Measurement matters. Observer. Retrieved from https://www.psychologicalscience.org/observer/measurement-matters
[28]
Gelman A. (2016, January 26). The time-reversal heuristic—a new way to think about a published finding that is followed up by a large, preregistered replication (in context of Amy Cuddy’s claims about power pose) [Web log post]. Retrieved from http://andrewgelman.com/2016/01/26/more-power-posing/
[29]
Gelman A., Loken P. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Retrieved from http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf
[31]
Mini Meta‐Analysis of Your Own Studies: Some Arguments on Why and a Primer on How

Jin X. Goh, Judith A. Hall, Robert Rosenthal

Social and Personality Psychology Compass 10.1111/spc3.12267
[34]
The weirdest people in the world?

Joseph Henrich, Steven J. Heine, Ara Norenzayan

Behavioral and Brain Sciences 10.1017/s0140525x0999152x
[36]
Howell D. C. (2010)
[37]
Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification.

Li-tze Hu, Peter M. Bentler

Psychological Methods 10.1037/1082-989x.3.4.424
[38]
Inzlicht M. (2015, November). Guest post: A tale of two papers [Web log post]. Retrieved from http://sometimesimwrong.typepad.com/wrong/2015/11/guest-post-a-tale-of-two-papers.html 10.1063/pt.5.010324
[39]
Why Most Published Research Findings Are False

John P. A. Ioannidis

PLoS Medicine 10.1371/journal.pmed.0020124
[40]
Why Most Discovered True Associations Are Inflated

John P. A. Ioannidis

Epidemiology 10.1097/ede.0b013e31818131e7
[43]
Kidd R. F. Representative Research in Social Psychology (1976)
[44]
Krueger J. I.
[47]
Lakens D. (2014b, May 29). The probability of p -values as a function of the statistical power of a test [Web log post]. Retrieved from http://daniellakens.blogspot.ca/2014/05/the-probability-of-p-values-as-function.html 10.59350/xqdjs-z6007
[48]
Equivalence Tests

Daniël Lakens

Social Psychological and Personality Science 10.1177/1948550617697177
[50]
Too True to be Bad

Daniël Lakens, Alexander J. Etz

Social Psychological and Personality Science 10.1177/1948550617693058

Showing 50 of 99 references

Cited By
15
Reporting Ethical Procedures in Suicide Prevention Research: Current Status and Recommendations

Ryan M. Hill, Zain Hussain · 2022

Archives of Suicide Research
Metrics
15
Citations
99
References
Details
Published
Nov 02, 2018
Vol/Issue
1(4)
Pages
556-573
License
View
Authors
W
William E. Davis

Department of Psychology, Wittenberg University

R
Roger Giner-Sorolla

School of Psychology, University of Kent

D
D. Stephen Lindsay

Department of Psychology, University of Victoria

J
Jessica P. Lougheed

Department of Human Development and Family Studies, Purdue University

M
Matthew C. Makel

Talent Identification Program, Duke University

M
Matt E. Meier

Psychology Department, Western Carolina University

J
Jessie Sun

Department of Psychology, University of California, Davis

L
Leigh Ann Vaughn

Department of Psychology, Ithaca College

J
John M. Zelenski

Department of Psychology, Carleton University

Cite This Article
William E. Davis, Roger Giner-Sorolla, D. Stephen Lindsay, et al. (2018). Peer-Review Guidelines Promoting Replicability and Transparency in Psychological Science. Advances in Methods and Practices in Psychological Science, 1(4), 556-573. https://doi.org/10.1177/2515245918806489
Related

You May Also Like

Evaluating Effect Size in Psychological Research: Sense and Nonsense

David C. Funder, Daniel J. Ozer · 2019

2,455 citations

Equivalence Testing for Psychological Research: A Tutorial

Daniël Lakens, Anne M. Scheel · 2018

1,152 citations

Thinking Clearly About Correlations and Causation: Graphical Causal Models for Observational Data

Julia M. Rohrer · 2018

698 citations

An Introduction to Linear Mixed-Effects Modeling in R

Violet A. Brown · 2021

400 citations

Simulation-Based Power Analysis for Factorial Analysis of Variance Designs

Daniël Lakens, Aaron R. Caldwell · 2021

383 citations