Ranking Scientific Journals Via Latent Class Models for Polytomous Item Response Data

Francesco Bartolucci; Valentino Dardanoni; Franco Peracchi

doi:10.1111/rssa.12106

journal article Feb 11, 2015

Ranking Scientific Journals Via Latent Class Models for Polytomous Item Response Data

Francesco Bartolucci

Valentino Dardanoni Franco Peracchi

Journal of the Royal Statistical Society Series A: Statistics in Society Vol. 178 No. 4 pp. 1025-1049 · Oxford University Press (OUP)

View at Publisher Save 10.1111/rssa.12106

Abstract

SummaryWe propose a model-based strategy for ranking scientific journals starting from a set of observed bibliometric indicators that represent imperfect measures of the unobserved ‘value’ of a journal. After discretizing the available indicators, we estimate an extended latent class model for polytomous item response data and use the estimated model to cluster journals. We illustrate our approach by using the data from the Italian research evaluation exercise that was carried out for the period 2004–2010, focusing on the set of journals that are considered relevant for the subarea statistics and financial mathematics. Using four bibliometric indicators (IF, IF5, AIS and the h-index), some of which are not available for all journals, and the information contained in a set of covariates, we derive a complete ordering of these journals. We show that the methodology proposed is relatively simple to implement, even when the aim is to cluster journals into a small number of ordered groups of a fixed size. We also analyse the robustness of the obtained ranking with respect to different discretization rules.

Topics

No keywords indexed for this article. Browse by subject →

References

31

[1]

Agresti (2002) 10.1002/0471249688

[2]

Akaike (1973)

[3]

Bacci "A class of multidimensional latent class irt models for ordinal polytomous item responses" Communs Statist. Theor. Meth. (2014) 10.1080/03610926.2013.827718

[4]

Bartolucci "A class of multidimensional IRT models for testing unidimensionality and clustering items" Psychometrika (2007) 10.1007/s11336-005-1376-9

[5]

Bertocchi "Bibliometric evaluation vs. informed peer review: evidence from Italy" Res. Poly (2014)

[6]

Biernacki "An improvement of the nec criterion for assessing the number of clusters in a mixture model" Pattn Recogn. Lett. (1999) 10.1016/s0167-8655(98)00144-5

[7]

Bollen "A principal component analysis of 39 scientific impact measures" PLOS ONE (2009) 10.1371/journal.pone.0006022

[8]

Celeux "An entropy criterion for assessing the number of clusters in a mixture model" J. Classificn (1996) 10.1007/bf01246098

[9]

Chang (2012)

[10]

Chang (2010)

[11]

Colombi "Marginal regression models for the analysis of positive association of ordinal response variables" Biometrika (2001) 10.1093/biomet/88.4.1007

[12]

Maximum Likelihood from Incomplete Data Via the EM Algorithm

A. P. Dempster, N. M. Laird, D. B. Rubin

Journal of the Royal Statistical Society Series B:... 1977 10.1111/j.2517-6161.1977.tb01600.x

[13]

Dias (2006) 10.1007/3-540-34416-0_11

[14]

Douglas (1991)

[15]

Fraley "Model-based clustering, discriminant analysis, and density estimation" J. Am. Statist. Ass. (2002) 10.1198/016214502760047131

[16]

The History and Meaning of the Journal Impact Factor

Eugene Garfield

JAMA 2006 10.1001/jama.295.1.90

[17]

Goodman "Exploratory latent structure analysis using both identifiable and unidentifiable models" Biometrika (1974) 10.1093/biomet/61.2.215

[18]

Hambleton (1985) 10.1007/978-94-017-1988-9

[19]

Heckman "A method for minimizing the impact of distributional assumptions in econometric models for duration data" Econometrica (1985) 10.2307/1911491

[20]

An index to quantify an individual's scientific research output

J. E. Hirsch

Proceedings of the National Academy of Sciences 2005 10.1073/pnas.0507655102

[21]

Lazarsfeld (1968)

[22]

Lindsay "Semiparametric estimation in the rasch model and related exponential response models, including a simple latent class model for item analysis" J. Am. Statist. Ass. (1991) 10.1080/01621459.1991.10475008

[23]

Little (2002) 10.1002/9781119013563

[24]

Regression Models for Ordinal Data

Peter McCullagh

Journal of the Royal Statistical Society Series B:... 1980 10.1111/j.2517-6161.1980.tb01109.x

[25]

McLachlan (2000) 10.1002/0471721182

[26]

Palacios-Huerta "The measurement of intellectual influence" Econometrica (2004) 10.1111/j.1468-0262.2004.00519.x

[27]

Plackett "A class of bivariate distributions" J. Am. Statist. Ass. (1965) 10.1080/01621459.1965.10480807

[28]

Samejima "Estimation of ability using a response pattern of graded scores" Psychmetr. Monogr. (1969)

[29]

Samejima "Evaluation of mathematical models for ordered polychotomous responses" Behaviormetrika (1996) 10.2333/bhmk.23.17

[30]

Estimating the Dimension of a Model

Gideon Schwarz

The Annals of Statistics 1978 10.1214/aos/1176344136

[31]

Zimmermann (2012)

Metrics

8

Citations

31

References

Details

Published: Feb 11, 2015
Vol/Issue: 178(4)
Pages: 1025-1049
License: View

Authors

F

Francesco Bartolucci

University of Perugia , Italy

V

Valentino Dardanoni

University of Palermo , Italy

F

Franco Peracchi

University of Rome Tor Vergata and Einaudi Institute for Economics and Finance , Rome , Italy

Cite This Article

Francesco Bartolucci, Valentino Dardanoni, Franco Peracchi (2015). Ranking Scientific Journals Via Latent Class Models for Polytomous Item Response Data. Journal of the Royal Statistical Society Series A: Statistics in Society, 178(4), 1025-1049. https://doi.org/10.1111/rssa.12106

Ranking Scientific Journals Via Latent Class Models for Polytomous Item Response Data

You May Also Like