From Traditional Topic Models to LLM Topic Models: Can Large Language Models Replace Traditional Topic Models?

Published

August 1, 2025

Author(s)

Zongxia Li, Lorena Calvo Bartolome, Alexander Hoyle, Daniel Stephens, Paiheng Xu, Alden Dima, Jordan Boyd-Graber, Juan Fung

Abstract

A common use of NLP is to facilitate the understanding of large document collections, with models based on Large Language Models (LLMs) replacing probabilistic topic models. Yet the effectiveness of LLM-based approaches in real-world applications remains under explored. This study measures the knowledge users acquire with topic models—including traditional, unsupervised and supervised LLM- based approaches—on two datasets. While LLM-based methods generate more human- readable topics and show higher average win probabilities than traditional models for data exploration, they produce overly generic topics for domain-specific datasets that do not easily allow users to learn much about the documents. Adding human supervision to LLM-based topic models improves data exploration by addressing hallucination and genericity but requires more human efforts. In contrast, traditional models like Latent Dirichlet Allocation (LDA) remain effective for exploration but are less user-friendly. This paper provides best practices—there is no one right model, the choice of models is situation-specific—and suggests potential improvements for scalable LLM- based topic models.

Proceedings Title

Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

Volume

7583

Conference Dates

July 27-August 1, 2025

Conference Location

Vienna, AT

Conference Title

The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

Pub Type

Conferences

Download Paper

https://doi.org/10.18653/v1/2025.acl-long.375

Local Download

Keywords

natural language processing, topic models, large language models, artificial intelligence, content analysis

Citation

Li, Z. , Calvo Bartolome, L. , Hoyle, A. , Stephens, D. , Xu, P. , Dima, A. , Boyd-Graber, J. and Fung, J. (2025), From Traditional Topic Models to LLM Topic Models: Can Large Language Models Replace Traditional Topic Models?, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025) , Vienna, AT, [online], https://doi.org/10.18653/v1/2025.acl-long.375, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=959292 (Accessed December 12, 2025)