190 Learning Theory — Sub Gaussian
190.1 Overview
This module is the sub-Gaussian abstraction layer for the Johnson–Lindenstrauss lemma. Its main result, subgaussian_centered_sq_bernstein (Vershynin, Proposition 2.7.1(e), the MGF form of the sub-Gaussian/sub-exponential link of Lemma 2.7.6; the scalar diagonal case of the Hanson–Wright circle of results, Theorem 6.2.1), shows that if \(Z\) is strictly sub-Gaussian with variance proxy \(\sigma ^2\) and \(\mathbb {E}[Z^2] = \sigma ^2\), then the centered square \(Z^2 - \sigma ^2\) satisfies the Bernstein MGF condition with parameters \((2\sigma ^4, 1/(4\sigma ^2))\). The supporting lemmas provide a Gaussian decoupling identity (the trick used in the proof of Vershynin’s Lemma 6.2.2), an elementary quadratic bound on \(e^{-x}\), an exponential-moment bound for \(Z^2\), a fourth-moment bound, and the two one-sided centered MGF bounds.
190.2 Declarations
For every real \(t \ge 0\) and every real \(z\),
i.e. \(e^{tz^2}\) equals the moment generating function of a standard Gaussian evaluated at \(s = \sqrt{2t}\, z\).
For every real \(x \ge 0\),
Let \(Z\) be a measurable real random variable on a probability space \((\Omega , \mu )\) which is strictly sub-Gaussian with variance proxy \(\sigma ^2\): for every real \(s\), \(e^{sZ}\) is \(\mu \)-integrable and \(\mathbb {E}_\mu [e^{sZ}] \le e^{\sigma ^2 s^2/2}\). Then for every \(t \ge 0\) with \(2\sigma ^2 t \lt 1\), the function \(e^{tZ^2}\) is \(\mu \)-integrable and
the same bound that holds for \(Z \sim N(0, \sigma ^2)\).
Let \(Z\) be a measurable real random variable on a probability space \((\Omega , \mu )\) which is strictly sub-Gaussian with variance proxy \(\sigma ^2 \gt 0\) (for every real \(s\), \(e^{sZ}\) is integrable and \(\mathbb {E}_\mu [e^{sZ}] \le e^{\sigma ^2 s^2/2}\)), and suppose \(\mathbb {E}_\mu [Z^2] = \sigma ^2\). Then \(Z^4\) is \(\mu \)-integrable and
(For a Gaussian the exact value is \(3\sigma ^4\).)
Let \(Z\) be a measurable real random variable on a probability space \((\Omega , \mu )\) which is strictly sub-Gaussian with variance proxy \(\sigma ^2 \gt 0\) (for every real \(s\), \(e^{sZ}\) is integrable and \(\mathbb {E}_\mu [e^{sZ}] \le e^{\sigma ^2 s^2/2}\)), with \(\mathbb {E}_\mu [Z^2] = \sigma ^2\). Then for every \(t\) with \(-1/(4\sigma ^2) \le t \le 0\), the function \(e^{t(Z^2 - \sigma ^2)}\) is \(\mu \)-integrable and
Let \(Z\) be a measurable real random variable on a probability space \((\Omega , \mu )\) which is strictly sub-Gaussian with variance proxy \(\sigma ^2 \gt 0\) (for every real \(s\), \(e^{sZ}\) is integrable and \(\mathbb {E}_\mu [e^{sZ}] \le e^{\sigma ^2 s^2/2}\)). Then for every \(t\) with \(0 \le t \le 1/(4\sigma ^2)\), the function \(e^{t(Z^2 - \sigma ^2)}\) is \(\mu \)-integrable and
Let \(Z\) be a measurable real random variable on a probability space \((\Omega , \mu )\) which is strictly sub-Gaussian with variance proxy \(\sigma ^2 \gt 0\), i.e. for every real \(t\) the function \(e^{tZ}\) is \(\mu \)-integrable and \(\mathbb {E}_\mu [e^{tZ}] \le e^{\sigma ^2 t^2/2}\), and whose second moment is exactly \(\mathbb {E}_\mu [Z^2] = \sigma ^2\). Then the centered square \(Z^2 - \sigma ^2\) satisfies the Bernstein MGF condition ProbabilityTheory.HasBernsteinMGF with parameters \((2\sigma ^4,\, 1/(4\sigma ^2))\): for every \(t\) with \(\left\lvert t\right\rvert \le 1/(4\sigma ^2)\), \(e^{t(Z^2 - \sigma ^2)}\) is \(\mu \)-integrable and \(\mathbb {E}_\mu [e^{t(Z^2 - \sigma ^2)}] \le e^{2\sigma ^4 t^2}\). This is the sub-exponential MGF bound of Vershynin, Proposition 2.7.1(e), for the square of a sub-Gaussian variable (the sub-Gaussian/sub-exponential link of Lemma 2.7.6), i.e. the scalar case of the Hanson–Wright circle of results (Theorem 6.2.1), with the sharp Gaussian-case constants; the Gaussian decoupling identity it rests on is the one from Vershynin’s Lemma 6.2.2.