Generalized Boltzmann Machine with Deep Neural Structure

Restricted Boltzmann Machine (RBM) is an essential component in many machine learn-ing applications. As a probabilistic graphical model, RBM posits a shallow structure, which makes it less capable of modeling real-world applications. In this paper, to bridge the gap between RBM and articial neural net-work, we propose an energy-based probabilis-tic model that is more exible on modeling continuous data. By introducing the pair-wise inverse autoregressive ow into RBM, we propose two generalized continuous RBMs which contain deep neural network structure to more exibly track the practical data dis-tribution while still keeping the inference tractable. In addition, we extend the gen-eralized RBM structures into sequential set-ting to better model the stochastic process of time series. Performance improvements on probabilistic modeling and representation learning are demonstrated by the experiments non diverse datasets.