This article introduces a scalable topic-modeling approach aimed at enabling political scientists to analyze extremely large text corpora. The authors propose tensor latent Dirichlet allocation (tensor LDA) and report theoretical identifiability and sample-complexity guarantees, an open-source GPU-based implementation, and computational performance gains that make it feasible to study social-media phenomena at scale. The article illustrates the method with two large, applied analyses: a multi-year study of the #MeToo movement on Twitter and a study of social-media conversations about alleged election fraud in the 2020 U.S. presidential election.
Data and methods
The paper presents tensor latent Dirichlet allocation as a topic-modeling method that the authors say has identifiable and recoverable parameter guarantees and provable sample complexity for large datasets. The abstract emphasizes computational and memory efficiency, a GPU-based open-source implementation, and claimed empirical speed improvements relative to earlier parallelized LDA approaches. The applied examples draw on very large social-media corpora, including over two years of Twitter conversation and datasets on 2020 election-fraud discussion, as described in the abstract.
Main findings
According to the abstract, tensor LDA scales linearly to text collections with over a billion documents and achieves multi-fold speedups (reported as over 3×–4×) compared with prior parallelized LDA methods. The implementation is presented as enabling previously prohibitive analyses; the authors then apply it to describe evolution in the #MeToo conversation and to examine social-media conversations about election fraud in 2020.
Identifiable tensor LDA guarantees GPU-based open-source implementation Scales linearly to billions of documents
- The method is claimed to be more computationally and memory efficient than prior parallel LDA variants.
- Two large applied studies (Twitter #MeToo over two years; social-media election-fraud conversation in 2020) demonstrate use cases the authors emphasize in the abstract.
