An Efficient Variational Inference Method for MRF Learning and Structured Prediction Tasks




Journal Title

Journal ISSN

Volume Title



The combination of deep neural networks and probabilistic graphical models (PGMs), especially conditional random fields (CRFs), has been studied extensively in recent years, due to a large variety of real world applications that could benefit from combining these two different modeling approaches, especially in the computer vision field, e.g. stereo matching, semantic segmentation and image colorization, etc. However, the traditional methods are either too slow to be applied on large scale PGMs (say grid models over high definition images) or are too simple to yield significant performance improvements. In this dissertation, we propose a highly parallelizeable inference method that is especially suitable for combined CRF + neural network frameworks. We first apply this inference method to general MRF/CRF learning problems, using neural networks to model the potential functions. We show that the resulting model not only yields better classification performance on real-world tasks, but that it also yields a better generative model of the data. We then explain how to combine CRFs with pure deep neural networks, using our inference method as the backbone of the learning process, to solve structured prediction problems in computer vision tasks, e.g. stereo matching, image colorization, semantic segmentation, etc. We show that this strategy is not only efficient on modern GPUs, but it can also achieve superior performance to pure neural network solutions in each problem domain, sometimes dramatically so.



Computer Science, Artificial Intelligence, Statistics