CIF: Small: Universal Signal Estimation from Noisy Measurements

Signal processing and communication systems often use Bayesian methods, which require a statistical characterization of the input to be available.When statistics are unavailable, one can use universal algorithms that adapt to the data at hand and perform as well as any other system asymptotically.

Unfortunately, the impact of universal algorithms has been limited to lossless compression.

To take universal algorithms beyond compression, this project will develop theory and algorithms for universal signal estimation, where an input signal is estimated from knowledge of its noisy measurements and the structure of the measurement system but without knowledge of the input statistics.

Our formulation is broad, and the research could lead to

(i) higher quality medical imaging;

(ii) improved seismic methods for predicting the likelihood of earthquakes and knowing where to perform oil and gas drilling;

(iii) communicating better-compressed data over more challenging unknown channels; and (iv) allocating capital more efficiently in financial markets.

We perform universal signal estimation in (potentially nonlinear) signal processing systems from noisy measurements. The theoretical component of our this project is inspired by Kolmogorov complexity and the minimum description length principle. The algorithmic component combines scalar quantization, universal lossless compression, and Markov chain Monte Carlo. We evaluate the estimated input over a quantized grid and optimize for the trade-off between information complexity of the estimated input and how well it explains the measurements.