Approximate Bayesian Inference by Adaptive Quantization of the Hypothesis Space.

Mathias Johansson

MaxEnt 2005, 25th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering San Jose, August 2005.

Abstract:

We introduce a method for making approximate Bayesian inference based on quantizing the hypothesis space and repartitioning it as observations become available. The method relies on approximating an optimal inference by using a probability distribution for quantized intervals of the unknown quantity, and by adjusting the intervals so as to obtain higher resolution in regions of higher probability, and vice versa.

We repartition the hypothesis space adaptively with the aim of maximizing the mutual information between the approximate distribution and the exact distribution. It is shown that this approach is equivalent to maximizing the entropy of the approximate distribution, and we provide low-complexity algorithms for approximating multi-dimensional posterior distributions with tunable complexity/performance.

The resulting quantized distribution for a one-dimensional case can be visualized as a histogram where each bar has equal area, but in general unequal width. The method can be used to provide adaptive quantization of arbitrary data sequences, or to approximate the posterior expectation of for instance some loss function by summing over a pre-specified number of terms.

Related publications:

PhD Thesis by Mathias Johansson

Source:

Pdf, (79K)