Research
My current research focuses on modeling complex data with deep generative models. Topics that I am interested in include structured prediction, deep generative models, variational inference and graphical models.
Current Projects
Woodbury Transformations for Normalizing Flows
In this project, we develop Woodbury transformations, which achieve efficient invertibility via the Woodbury matrix identity and efficient determinant calculation via Sylvester's determinant identity. In contrast with other operations used in state-of-the-art normalizing flows, Woodbury transformations enable (1) high-dimensional interactions, (2) efficient sampling, and (3) efficient likelihood evaluation.
Corresponding papers:
- You Lu, and Bert Huang. Woodbury Transformations for Deep Generative Flows. NeurIPS 2020.
Structured Prediction with Exact Likelihood
Traditional methods for structured output learning use energy-based models to define the conditional Likelihood. These models are usually hard to train, and require approximations and restrictive assumptions. In this project, we develop conditional generative flows. Our model has the unique advantage that can directly compute the exact conditional likelihood. The training of our model is easy and straightforward, and does not require any inference process.
Corresponding papers:
- You Lu, and Bert Huang. Structured Output Learning with Conditional Generative Flows. AAAI 2020.
Efficient Learning method for Markov Random Fields
In this project, we develop an efficent learning method for Markov random fields. We develop block belief propagation learning, which only require doing inference on a small block of the network, and use approximate gradient to update the parameters of interest. The method is more efficient than the existing methods in that the complexity of inference does not increase with the size of network. Our method does not require changing the objective function, is as easy to implement as the traditional convex belief propagation and has convergence guarantee.
Corresponding papers:
- You Lu, Zhiyuan Liu, and Bert Huang. Block Belief Propagation for Parameter Learning in Markov Random Fields. AAAI 2019.
Previous Projects
Analyzing Problems in Online Topic Models
In this project, we analyze the problems of online topic models with Adagrad. The problem is that to fit a topic model, the training algorithm must break the symmetry between parameters of words that are highly related to the topic and words that are not related to the topic. Before the algorithm converges, the magnitude of gradients of the parameters are very large. Since ADAGRAD uses the accumulation of previous gradients as learning rates’ denominators, the learning rates shrink very quickly. Thus, the algorithm cannot break the symmetry quickly. We also provide solutions to this problem.
Corresponding papers:
- You Lu, Jeffrey Lund, and Jordan Boyd-Graber. Why ADAGRAD Fails for Online Topic Modeling. EMNLP 2017.
Topic Modeling Large Scale Text Sets
In this project, we develop topic models for large scale corpora. We develop new online topic models for better analyzing large corpora with millions of documents, e.g., Wikipedia, Pubmed.
Corresponding papers:
- Ximing Li, Jihong Ouyang, and You Lu. Topic modeling for large-scale text data. Frontiers of Information Technology & Electronic Engineering, 2015.
- Jihong Ouyang, You Lu, and Ximing Li. Momentum online LDA for large-scale datasets. ECAI 2014.
Course Projects
Projected Gradient method for Inference in Markov Random Fields
In this project, I use projected gradient to replace belief propagation to infer the beliefs of Markov random fields. The results show that projected gradient method works for inference, but it is slower than belief propagation.
Project report:
Relational Topic Model for Congressional Bills Corpus
In this project, we implement relational topic model, and use it to analyze a document network, obtained from interactive user study. By analyzing the network using relational topic model, we validated the results presented in the user study.
Project report:
Improving the Performance of sLDA with SVI
In this project, we develop online supervised latent Dirichlet allocation. We use stochastic variational inference to learn the model's parameters. For the label's parameters, i.e., the parameters of the softmax distribution, we use stochastic gradient descent to optimize them. Our results show that the online sLDA is much faster than the traditional sLDA.
Project report: