Sampling using Piecewise Deterministic Markov Processes

On this website I aim to collect references to papers on the use and analysis of Piecewise Deterministic Markov Processes (PDMPs) for use in MCMC. I will try to list all papers that I find relevant, will put years of first appearance on arXiv or elsewhere, and list in reverse chronological order. Inevitably this page will reflect some of my own judgements. If you want to bring some paper to my attention, or feel I should correct something, please contact me.

Main idea

Markov Chain Monte Carlo (MCMC) is a probabilistic computational method which is of vital importance in data science (statistics, machine learning) as well as in the physical sciences (physics, chemistry, biology). For example, MCMC is applied in Bayesian statistics, for the training of (deep) neural networks, as well as for the simulation of many-particle systems.

In recent years it has been established that PDMPs may play a very useful role in designing new efficient MCMC methods which have good convergence properties and allow various clever tricks in dealing with large data sets (subsampling of the data, for example).

First, a few images

The following images are generated using the R package RZigZag, which can be installed via the command install.packages("RZigZag").

What we see are two-dimensional projections of higher dimensional trajectories of two PDMP samplers: the Zig-Zag Sampler and the Bouncy Particle Sampler (BPS). The trajectories can be seen to be continuous (in contrast to most MCMC algorithms). Also shown are discrete samples drawn along the trajectories, which can be interpreted in the classical MCMC sense. Both plots in this example correspond to a 5-dimensional standard normal distribution. The BPS trajectory is generated using a refreshment rate of 0.01.

Where to start?

A Jupyter tutorial [github link] with a very accessible introduction to MCMC and Zig-Zag together with Python code has been developed by Bernardita Ried (University of Chile). If you are looking for a gentle introduction into this class of algorithms, which does not assume too much mathematical background (no measure theory...) and covers both Bouncy Particle Sampler and Zig-Zag, I can recommend Fearnhead et al. (2016). (Note: although it is an extremely interesting topic, Section 4 on Continuous-Time Sequential Importance Sampling can be skipped if you wish to understand the BPS and Zig-Zag algorithms.) If you don't mind the measure theoretic formulation of probability, consult e.g. Vanetti et al. (2017).

Mathematics of PDMPs

Here papers are listed which concern the mathematical analysis of the stochastic processes underlying PDMP based algorithms.

Methodology using PDMPs and non-reversible MCMC

Statistics literature

Physics literature

As usual many ideas concerning sampling methods originated from physics; these are a few key papers in this respect.

Machine learning literature

Last update: February 2019