BishtRef
Articles Journals Publishers Authors Subjects Most Cited New Papers Year Stats Open Access
journal article Jul 19, 2010

Visualizing and assessing discrimination in the logistic regression model

Patrick Royston Douglas G. Altman ORCID
Statistics in Medicine Vol. 29 No. 24 pp. 2508-2520 · Wiley
View at Publisher Save 10.1002/sim.3994
Abstract
AbstractLogistic regression models are widely used in medicine for predicting patient outcome (prognosis) and constructing diagnostic tests (diagnosis). Multivariable logistic models yield an (approximately) continuous risk score, a transformation of which gives the estimated event probability for an individual. A key aspect of model performance is discrimination, that is, the model's ability to distinguish between patients who have (or will have) an event of interest and those who do not (or will not). Graphical aids are important in understanding a logistic model. The receiver‐operating characteristic (ROC) curve is familiar, but not necessarily easy to interpret. We advocate a simple graphic that provides further insight into discrimination, namely a histogram or dot plot of the risk score in the outcome groups. The most popular performance measure for the logistic model is the c‐index, numerically equivalent to the area under the ROC curve. We discuss the comparative merits of the c‐index and the (standardized) mean difference in risk score between the outcome groups. The latter statistic, sometimes known generically as the effect size, has been computed in slightly different ways by several different authors, including Glass, Cohen and Hedges. An alternative measure is the overlap between the distributions in the outcome groups, defined as the area under the minimum of the two density functions. The larger the overlap, the weaker the discrimination. Under certain assumptions about the distribution of the risk score, the c‐index, effect size and overlap are functionally related. We illustrate the ideas with simulated and real data sets. Copyright © 2010 John Wiley & Sons, Ltd.
Topics

No keywords indexed for this article. Browse by subject →

References
38
[12]
Saunders IW "A measure of the performance of biomarkers for disease" Cancer Biomarkers (2006) 10.3233/cbm-2006-23-406
[13]
The meaning and use of the area under a receiver operating characteristic (ROC) curve.

J A Hanley, B J McNeil

Radiology 10.1148/radiology.143.1.7063747
[15]
Evaluating the Yield of Medical Tests

Frank E. Harrell

JAMA 10.1001/jama.1982.03320430047030
[17]
Hand DJ (1997)
[19]
Cohen J (1988)
[20]
Bradley EL (1985)
[35]
Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction

Nancy R. Cook

Circulation 10.1161/circulationaha.106.672402
[36]
Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond

Michael J. Pencina, Ralph B. D' Agostino, Ramachandran S. Vasan

Statistics in Medicine 10.1002/sim.2929
[38]
Royston P "Approximating statistical functions by using fractional polynomial regression" The Statistician (1997) 10.1111/1467-9884.00093
Cited By
83
Inference on the overlap coefficient: The binormal approach and alternatives

Alba María Franco-Pereira, Christos T Nakas · 2021

Statistical Methods in Medical Rese...
Prediction and network modelling of self-harm through daily self-report and history of self-injury

Michael J. Kyron, Geoff R. Hooke · 2020

Psychological Medicine
Quantitative assessment of two- and three-dimensional transthoracic and two-dimensional transesophageal echocardiography, computed tomography, and magnetic resonance imaging in normal canine hearts

R.C. Fries, S.G. Gordon · 2019

Journal of Veterinary Cardiology
Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration

Karel G.M. Moons, Douglas G. Altman · 2015

Annals of Internal Medicine
Normalized Lactate Load Is Associated with Development of Acute Kidney Injury in Patients Who Underwent Cardiopulmonary Bypass Surgery

Zhongheng Zhang, Hongying Ni · 2015

PLoS ONE
Metrics
83
Citations
38
References
Details
Published
Jul 19, 2010
Vol/Issue
29(24)
Pages
2508-2520
License
View
Authors
P
Patrick Royston
D
Douglas G. Altman
Cite This Article
Patrick Royston, Douglas G. Altman (2010). Visualizing and assessing discrimination in the logistic regression model. Statistics in Medicine, 29(24), 2508-2520. https://doi.org/10.1002/sim.3994
Related

You May Also Like

Quantifying heterogeneity in a meta‐analysis

Julian P. T. Higgins, Simon G. Thompson · 2002

31,148 citations

Multiple imputation using chained equations: Issues and guidance for practice

Ian R. White, Patrick Royston · 2010

7,694 citations

MULTIVARIABLE PROGNOSTIC MODELS: ISSUES IN DEVELOPING MODELS, EVALUATING ASSUMPTIONS AND ADEQUACY, AND MEASURING AND REDUCING ERRORS

Frank E. Harrell, Kerry L. Lee · 1996

7,135 citations

Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity‐score matched samples

Peter C. Austin · 2009

5,504 citations

Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond

Michael J. Pencina, Ralph B. D' Agostino · 2007

5,289 citations