In general, My research focuses on building efficient learning or inference methods for probabilistic graphical models and combining graphical models with deep learning methods to do structured prediction. Topics that I am interested in include graphical models, variational inference, structured prediction and deep generative models.
Efficient Learning method for Markov Random Fields
In this project, we develop an efficent learning method for Markov random fields. We develop block belief propagation learning, which only require doing inference on a small block of the network, and use approximate gradient to update the parameters of interest. The method is more efficient than the existing methods in that the complexity of inference does not increase with the size of network. Our method does not require changing the objective function, is as easy to implement as the traditional convex belief propagation and has convergence guarantee.
- You Lu, Zhiyuan Liu, and Bert Huang. Block Belief Propagation for Parameter Learning in Markov Random Fields. AAAI 2019. Acceptance rate: 16.2%.
Analyzing Problems in Online Topic Models
In this project, we analyze the problems of online topic models with Adagrad. The problem is that to fit a topic model, the training algorithm must break the symmetry between parameters of words that are highly related to the topic and words that are not related to the topic. Before the algorithm converges, the magnitude of gradients of the parameters are very large. Since ADAGRAD uses the accumulation of previous gradients as learning rates’ denominators, the learning rates shrink very quickly. Thus, the algorithm cannot break the symmetry quickly. We also provide solutions to this problem.
- You Lu, Jeffrey Lund, and Jordan Boyd-Graber. Why ADAGRAD Fails for Online Topic Modeling. EMNLP 2017. Acceptance rate: 18%.
Topic Modeling Large Scale Text Sets
In this project, we develop topic models for large scale corpora. We develop new online topic models for better analyzing large corpora with millions of documents, e.g., Wikipedia, Pubmed.
- Ximing Li, Jihong Ouyang, and You Lu. Topic modeling for large-scale text data. Frontiers of Information Technology & Electronic Engineering, 2015.
- Jihong Ouyang, You Lu, and Ximing Li. Momentum online LDA for large-scale datasets. ECAI 2014. Acceptance rate: 41%.
Projected Gradient method for Inference in Markov Random Fields
In this project, I use projected gradient to replace belief propagation to infer the beliefs of Markov random fields. The results show that projected gradient method works for inference, but it is slower than belief propagation.
Relational Topic Model for Congressional Bills Corpus
In this project, we implement relational topic model, and use it to analyze a document network, obtained from interactive user study. By analyzing the network using relational topic model, we validated the results presented in the user study.
Improving the Performance of sLDA with SVI
In this project, we develop online supervised latent Dirichlet allocation. We use stochastic variational inference to learn the model's parameters. For the label's parameters, i.e., the parameters of the softmax distribution, we use stochastic gradient descent to optimize them. Our results show that the online sLDA is much faster than the traditional sLDA.