Title

Properties and Bayesian fitting of restricted Boltzmann machines

Campus Units

Industrial and Manufacturing Systems Engineering, Statistics

Document Type

Article

Publication Version

Accepted Manuscript

Publication Date

2-2019

Journal or Book Title

Statistical Analysis and Data Mining

Volume

12

Issue

1

First Page

23

Last Page

38

DOI

10.1002/sam.11396

Abstract

A restricted Boltzmann machine (RBM) is an undirected graphical model constructed for discrete or continuous random variables, with two layers, one hidden and one visible, and no conditional dependency within a layer. In recent years, RBMs have risen to prominence due to their connection to deep learning. By treating a hidden layer of one RBM as the visible layer in a second RBM, a deep architecture can be created. RBMs thereby are thought to have the ability to encode very complex and rich structures in data, making them attractive for supervised learning. However, the generative behavior of RBMs largely is unexplored and typical fitting methodology does not easily allow for uncertainty quantification in addition to point estimates. In this paper, we discuss the relationship between RBM parameter specification in the binary case and model properties such as degeneracy, instability and uninterpretability. We also describe the associated difficulties that can arise with likelihood‐based inference and further discuss the potential Bayes fitting of such (highly flexible) models, especially as Gibbs sampling (quasi‐Bayes) methods often are advocated for the RBM model structure.

Comments

This is the peer-reviewed version of the following article: Kaplan, Andee, Daniel Nordman, and Stephen Vardeman. "Properties and Bayesian fitting of restricted Boltzmann machines." Statistical Analysis and Data Mining: The ASA Data Science Journal 12, no. 1 (2019): 23-38, which has been published in final form at DOI: 10.1002/sam.11396. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.

Copyright Owner

Wiley Periodicals, Inc.

Language

en

File Format

application/pdf

Published Version

Share

COinS