Fullscreen
Loading...
 
Tao
Print

Seminar28042017

April 28th

14:30 (Shannon amphitheatre, building 660) (see location ):

Jascha Sohl-dickstein (Google Brain)


Title: Deep Unsupervised Learning using Nonequilibrium Thermodynamics


Abstract:

I will present a method for building probabilistic models of complex
data-sets which is highly flexible, yet for which learning, sampling,
inference, and evaluation are still analytically or computationally
tractable. The essential idea, inspired by non-equilibrium statistical
physics, is to systematically and slowly destroy the structure in a data
distribution through an iterative forward diffusion process. We then train
a reverse diffusion process to restore that structure, yielding a
generative model of the data. By using a deep network to define the
diffusion process, we are able to rapidly learn, sample from, and evaluate
probabilities in deep generative models with thousands of layers or time
steps. I will demonstrate the effectiveness of this model on several
natural image datasets. I will also briefly discuss several other projects
applying ideas from nonequilibrium statistical mechanics to training,
sampling from, and predicting properties of machine learning models.

Publications:

- Sohl-Dickstein J, Weiss EA, Maheswaranathan N, Ganguli S. Deep
unsupervised learning using nonequilibrium thermodynamics. International
Conference on Machine Learning. 2015. http://arxiv.org/abs/1503.03585
- Sohl-Dickstein J, Battaglino PB, DeWeese? MR. New method for parameter
estimation in probabilistic models: minimum probability flow. Physical
review letters. 2011. https://journals.aps.org/prl/
abstract/10.1103/PhysRevLett.107.220601
- Sohl-Dickstein J, Mudigonda M, DeWeese? MR. Hamiltonian Monte Carlo
without detailed balance. International Conference on Machine Learning.
2014. https://arxiv.org/abs/1409.5191
- Schoenholz SS, Gilmer J, Ganguli S, Sohl-Dickstein J. Deep Information
Propagation. International Conference on Learning Representations. 2017.
https://openreview.net/forum?id=H1W1UN9gg&noteId=H1W1UN9gg




Contact: guillaume.charpiat at inria.fr

Contributors to this page: guillaume .
Page last modified on Thursday 27 of April, 2017 00:28:40 CEST by guillaume.