Dynamic Topic Modeling with a Higher-Order Hypergraphical Representation

Thomas M. Menino Convention & Exhibition Center 

Most traditional topic models represent text using multinomial likelihood. These models treat documents as collections of independent word counts, combining word occurrence and repetition into a single probabilistic mechanism. However, this formulation obscures higher-order word interaction patterns at the document level and limits model expressiveness. To address these limitations, we propose a higher-order, hypergraphical text representation. In this model, each document induces a hyperedge connecting all co-occurring words, and repetition intensity is encoded as a node weight. This construction yields a novel hypergraph-induced multinomial distribution that decouples word occurrence from repetition intensity via support-dependent normalization. Building on this framework, we develop a dynamic topic modeling approach based on low-rank factorization that can accommodate evolving topic semantics over time. Under suitable initialization and regularity conditions, we establish local convergence guarantees and derive non-asymptotic error bounds. Numerical experiments on the synthetic datasets and real-world corpora demonstrate the advantages of our approach over existing methods.

Topic model

Dynamic modeling

Hypergraph

Text representation

Nonconvex optimization

Low-rank factorization 

Section on Text Analysis