Topic Modeling and Text Analysis for Qualitative Policy Research

Karoliina Isoaho; Daria Gritsenko; Eetu Mäkelä

doi:10.1111/psj.12343

journal article Jun 19, 2019

Topic Modeling and Text Analysis for Qualitative Policy Research

Karoliina Isoaho

Daria Gritsenko Eetu Mäkelä

Policy Studies Journal Vol. 49 No. 1 pp. 300-324 · Wiley

View at Publisher Save 10.1111/psj.12343

Abstract

This paper contributes to a critical methodological discussion that has direct ramifications for policy studies: how computational methods can be concretely incorporated into existing processes of textual analysis and interpretation without compromising scientific integrity. We focus on the computational method of topic modeling and investigate how it interacts with two larger families of qualitative methods:
content and classification
methods characterized by interest in words as communication units and
discourse and representation
methods characterized by interest in the meaning of communicative acts. Based on analysis of recent academic publications that have used topic modeling for textual analysis, our findings show that different mixed‐method research designs are appropriate when combining topic modeling with the two groups of methods. Our main concluding argument is that topic modeling enables scholars to apply policy theories and concepts to much larger sets of data. That said, the use of computational methods requires genuine understanding of these techniques to obtain substantially meaningful results. We encourage policy scholars to reflect carefully on methodological issues, and offer a simple heuristic to help identify and address critical points when designing a study using topic modeling.

Topics

No keywords indexed for this article. Browse by subject →

References

81

[1]

10.1002/asi.23786

[2]

Probabilistic topic models

David M. Blei

Communications of the ACM 10.1145/2133806.2133826

[3]

Blei David M. "Correlated Topic Models" Advances in Neural Information Processing Systems (2006)

[4]

10.1162/jmlr.2003.3.4-5.993

Applied Physics Letters 10.1162/jmlr.2003.3.4-5.993

[5]

10.1016/j.gloenvcha.2015.12.001

[6]

Using thematic analysis in psychology

Virginia Braun, Victoria Clarke

Qualitative Research in Psychology 10.1191/1478088706qp063oa

[7]

10.1177/1468794106058877

[8]

10.2307/271007

[9]

10.1016/0304-422x(94)90011-6

[10]

10.1016/j.biocon.2018.01.029

[11]

Chang Jonathan "Reading Tea Leaves: How Humans Interpret Topic Models" Advances in Neural Information Processing Systems (2009)

[12]

Creswell John W. (2003)

[13]

Creswell John W. (2007)

[14]

Green, circular, bio economy: A comparative analysis of sustainability avenues

D. D'Amato, N. Droste, B. Allen et al.

Journal of Cleaner Production 10.1016/j.jclepro.2017.09.053

[15]

Indexing by latent semantic analysis

Scott Deerwester, Susan T. Dumais, George W. Furnas et al.

Journal of the American Society for Information Sc... 10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9

[16]

10.1080/19331681.2013.826613

[17]

10.1017/pan.2017.44

[18]

Denzin Norman K. (2007)

[19]

10.1177/2053951715602908

[20]

Exploiting affinities between topic modeling and the sociological perspective on culture: Application to newspaper coverage of U.S. government arts funding

Paul DiMaggio, Manish Nag, David Blei

Poetics 10.1016/j.poetic.2013.08.004

[21]

10.1016/j.csda.2008.01.011

[22]

Framing: Toward Clarification of a Fractured Paradigm

ROBERT M. ENTMAN

Journal of Communication 10.1111/j.1460-2466.1993.tb01304.x

[23]

10.1017/s0143814x18000132

[24]

Media Discourse and Public Opinion on Nuclear Power: A Constructionist Approach

WILLIAM A. GAMSON, Andre Modigliani

American Journal of Sociology 10.1086/229213

[25]

Gilardi Fabrizio Charles R.Shipan andBrunoWueest.2017. “Policy Diffusion: The Issue‐Definition Stage.”http://fabriziogilardi.org/resources/papers/diffusion-policy-frames.pdf. Accessed January 5 2019.

[26]

Glaser Barney G. (1967)

[27]

10.1353/nlh.2014.0025

[28]

10.1017/pan.2016.7

[29]

10.1093/pan/mpp034

[30]

Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts

Justin Grimmer, Brandon M. Stewart

Political Analysis 10.1093/pan/mps028

[31]

10.1016/j.erss.2016.03.012

[32]

10.1016/j.erss.2017.05.030

[33]

Hajer Maarten A. (1995)

[34]

Hofmann Thomas (1999)

[35]

Three Approaches to Qualitative Content Analysis

Hsiu-Fang Hsieh, Sarah E. Shannon

Qualitative Health Research 10.1177/1049732305276687

[36]

10.17645/pag.v7i1.1731

[37]

10.1080/21670811.2015.1093271

[38]

Jagarlamudi Jagadeesh (2012)

[39]

Jaworska Sylvia "Doing Well by Talking Good: A Topic Modelling‐Assisted Discourse Study of Corporate Social Responsibility" Applied Linguistics (2016)

[40]

10.4135/9781473957640.n4

[41]

10.1177/000312240807300205

[42]

10.1177/0165551515608733

[43]

10.1093/llc/fqv052

[44]

Krippendorff Klaus (2004)

[45]

10.4135/9781446288719

[46]

10.1037/14763-000

[47]

10.17813/1086-671x-20-4-43

[48]

10.1177/1750481318766936

[49]

10.5465/19416520.2012.660763

[50]

10.1093/pan/mpu019

Showing 50 of 81 references

Cited By

174

Digital innovation strategies in the public sector

Ali Asker Guenduez, Mehmet Akif Demircioglu · 2025

Research Policy

Making governance agile: Exploring the role of artificial intelligence in China’s local governance

Yiran Li, Yingying Fan · 2023

Public Policy and Administration

Metrics

174

Citations

81

References

Details

Published: Jun 19, 2019
Vol/Issue: 49(1)
Pages: 300-324
License: View

Authors

Funding

Academy of Finland Award: 284972

Cite This Article

Karoliina Isoaho, Daria Gritsenko, Eetu Mäkelä (2019). Topic Modeling and Text Analysis for Qualitative Policy Research. Policy Studies Journal, 49(1), 300-324. https://doi.org/10.1111/psj.12343

Topic Modeling and Text Analysis for Qualitative Policy Research

You May Also Like