Sampling, Metric Entropy, and Dimensionality Reduction

Let $Q$ be a relatively compact subset in a Hilbert space $V$. For a given $\epsilon>0$ let $N(\epsilon,Q)$ be the minimal number of linear measurements, sufficient to reconstruct any $x \in Q$ with the accuracy $\epsilon$. We call $N(\epsilon,Q)$ the sampling $\epsilon$-entropy of $Q$. Using dimensionality reduction, as provided by the Johnson--Lindenstrauss lemma, we show that, in an appropriate probabilistic setting, $N(\epsilon,Q)$ is bounded from above by Kolmogorov's $\epsilon$-entropy $H(\epsilon,Q)$, defined as $H(\epsilon,Q) = \log M(\epsilon,Q)$, with $M(\epsilon,Q)$ being the minimal number of $\epsilon$-balls covering $Q$. As the main application, we show that piecewise smooth (piecewise analytic) functions in one and several variables can be sampled with essentially the same accuracy rate as their regular counterparts. For univariate piecewise $C^k$-smooth functions this result, which settles the so-called Eckhoff conjecture, was recently established in D. Batenkov, Complete Algebraic Reconstruction of Piecewise-smooth Functions from Fourier Data, arXiv:1211.0680, 2012 via a deterministic “algebraic reconstruction” algorithm.