Approximate Bayesian Inference by
Adaptive Quantization of the Hypothesis Space.
MaxEnt 2005, 25th International Workshop on
Bayesian Inference and
Maximum Entropy Methods in Science and Engineering
San Jose, August 2005.
We introduce a method for making
approximate Bayesian inference based on quantizing the hypothesis
space and repartitioning it as observations become available. The
method relies on approximating an optimal inference by using a
probability distribution for quantized intervals of the unknown
quantity, and by adjusting the intervals so as to obtain higher
resolution in regions of higher probability, and vice versa.
We repartition the hypothesis space adaptively with the aim of
maximizing the mutual information between the approximate
distribution and the exact distribution. It is shown that this
approach is equivalent to maximizing the entropy of the
approximate distribution, and we provide low-complexity algorithms
for approximating multi-dimensional posterior distributions with
The resulting quantized distribution for a one-dimensional case
can be visualized as a histogram where each bar has equal area,
but in general unequal width. The method can be used to provide
adaptive quantization of arbitrary data sequences, or to
approximate the posterior expectation of for instance some loss
function by summing over a pre-specified number of terms.
by Mathias Johansson
This material is presented to ensure timely dissemination
of scholarly and technical work. Copyright and all rights
therein are retained by authors.
All persons copying this information are expected to
adhere to the terms and constraints invoked by each authors
copyright. This work may not be reposted
without the explicit permission of the copyright holders.