Agreeing to Disagree: Choosing Among Eight Topic-Modeling Methods

Qiang Fu, Yufan Zhuang, Jiaxin Gu, Yushu Zhu, Xin Guo

Research output: Journal article publicationJournal articleAcademic researchpeer-review

Abstract

Topic modeling is a key research area in natural language processing and has inspired innovative studies in a wide array of social-science disciplines. Yet, the use of topic modeling in computational social science has been hampered by two critical issues. First, social scientists tend to focus on a few standard ways of topic modeling. Our understanding of semantic patterns has not been informed by rapid methodological advances in topic modeling. Moreover, a systematic comparison of the performance of different methods in this field is warranted. Second, the choice of the optimal number of topics remains a challenging task. A comparison of topic-modeling techniques has rarely been situated in a social-science context and the choice appears to be arbitrary for most social scientists. Based on about 120,000 Canadian newspaper articles since 1977, we review and compare eight traditional, generative, and neural methods for topic modeling (Latent Semantic Analysis, Principal Component Analysis, Factor Analysis, Non-negative Matrix Factorization, Latent Dirichlet Allocation, Neural Autoregressive Topic Model, Neural Variational Document Model, and Hierarchical Dirichlet Process). Three measures (coherence statistics, held-out likelihood, and graph-based dimensionality selection) are then used to assess the performance of these methods. Findings are presented and discussed to guide the choice of topic-modeling methods, especially in social science research.

Original languageEnglish
Article number100173
Pages (from-to)1-9
Number of pages9
JournalBig Data Research
Volume23
DOIs
Publication statusPublished - 15 Feb 2021

Keywords

  • Computational social science
  • Natural language processing
  • Optimal number of topics
  • Topic modeling

ASJC Scopus subject areas

  • Management Information Systems
  • Information Systems
  • Computer Science Applications
  • Information Systems and Management

Cite this