# Fast Parallel SAME Gibbs Sampling on General Discrete Bayesian Networks

@article{Seita2015FastPS, title={Fast Parallel SAME Gibbs Sampling on General Discrete Bayesian Networks}, author={Daniel Seita and Haoyu Chen and John F. Canny}, journal={ArXiv}, year={2015}, volume={abs/1511.06416} }

A fundamental task in machine learning and related fields is to perform inference on Bayesian networks. Since exact inference takes exponential time in general, a variety of approximate methods are used. Gibbs sampling is one of the most accurate approaches and provides unbiased samples from the posterior but it has historically been too expensive for large models. In this paper, we present an optimized, parallel Gibbs sampler augmented with state replication (SAME or State Augmented Marginalâ€¦Â Expand

#### Figures, Tables, and Topics from this paper

#### Paper Mentions

#### 3 Citations

GPU-accelerated Gibbs sampling: a case study of the Horseshoe Probit model

- Computer Science, Mathematics
- Stat. Comput.
- 2019

This paper shows how to do Gibbs sampling in a fully data-parallel manner on a graphics processing unit, for a large class of exchangeable models that admit latent variable representations and finds that the implementation scales effectively to thousands of predictors and millions of data points simultaneously. Expand

GPU-accelerated Gibbs Sampling

- Computer Science, Mathematics
- ArXiv
- 2016

This paper shows how to do Gibbs sampling in a fully data-parallel manner on a graphics processing unit (GPU) for a large class of exchangeable models that admit latent variable representations and finds that the implementation scales effectively to thousands of predictors and millions of data points simultaneously. Expand

Using Deep Neural Network Approximate Bayesian Network

- Computer Science, Mathematics
- ArXiv
- 2018

A new method to approximate posterior probabilities of Bayesian Network using Deep Neural Network is presented, which is much faster and gains higher accuracy in medium sized Bayesian network. Expand

#### References

SHOWING 1-10 OF 24 REFERENCES

SAME but Different: Fast and High Quality Gibbs Parameter Estimation

- Computer Science, Mathematics
- KDD
- 2015

It is shown that combining SAME with factored sample representation (or approximation) gives throughput competitive with the fastest symbolic methods, but with potentially better quality, than other LDA implementations. Expand

Analyzing Hogwild Parallel Gaussian Gibbs Sampling

- Computer Science, Mathematics
- NIPS
- 2013

It is shown that if the Gaussian precision matrix is generalized diagonally dominant, then any Hogwild Gibbs sampler, with any update schedule or allocation of variables to processors, yields a stable sampling process with the correct sample mean. Expand

Parallel Gibbs Sampling: From Colored Fields to Thin Junction Trees

- Mathematics, Computer Science
- AISTATS
- 2011

This work proposes two methods to construct parallel Gibbs samplers guaranteed to draw from the targeted distribution, called the Chromatic sampler and the Splash sampler, a complementary strategy which can be used when the variables are tightly coupled. Expand

Reducing the sampling complexity of topic models

- Computer Science
- KDD
- 2014

An algorithm which scales linearly with the number of actually instantiated topics kd in the document, for large document collections and in structured hierarchical models kd ll k, yields an order of magnitude speedup. Expand

Towards high-throughput gibbs sampling at scale: a study across storage managers

- Computer Science
- SIGMOD '13
- 2013

This work studies how (and whether) traditional data processing choices about materialization, page layout, and buffer-replacement policy need to be changed to achieve high-throughput Gibbs sampling for factor graphs that are larger than main memory. Expand

Marginal maximum a posteriori estimation using Markov chain Monte Carlo

- Mathematics, Computer Science
- Stat. Comput.
- 2002

A simple and novel MCMC strategy, called State-Augmentation for Marginal Estimation (SAME), which leads to MMAP estimates for Bayesian models and illustrates the simplicity and utility of the approach for missing data interpolation in autoregressive time series and blind deconvolution of impulsive processes. Expand

Graphical Models, Exponential Families, and Variational Inference

- Mathematics, Computer Science
- Found. Trends Mach. Learn.
- 2008

The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models. Expand

A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms

- Mathematics
- 1990

Abstract The first part of this article presents the Monte Carlo implementation of the E step of the EM algorithm. Given the current guess to the maximizer of the posterior distribution, latent dataâ€¦ Expand

Accelerated Quantification of Bayesian Networks with Incomplete Data

- Computer Science
- KDD
- 1995

This paper considers statistical batch learning of the probability tables on the basis of incomplete data and expert knowledge and proposes a new class of models that allows a great variety of local functional restrictions to be imposed on the statistical model. Expand

Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning

- Computer Science
- 2009

Because uncertainty is an inescapable aspect of most real-world applications, the book focuses on probabilistic models, which make the uncertainty explicit and provide models that are more faithful to reality. Expand