247 Boolean Analysis — Product Hessian
247.1 Overview
The mean score and Hessian of a product Gaussian density have explicit linear and quadratic coordinate formulas. Local integrable envelopes justify a second differentiation of bounded Gaussian averages and yield global Lipschitz derivative bounds.
247.2 Declarations
Let \(I\) be finite and \(v\ge 0\). Write \(p_v(c,z)\) for the \(N(c,v)\) density when \(v\gt 0\) and set \(p_0(c,z)=0\); define \(G_v(c,z)=\prod _{i\in I}p_v(c_i,z_i)\) and interpret division by zero as zero. For \(c,z,h\in \mathbb R^I\), put \(S_v(c,z)[h]=\sum _i((z_i-c_i)/v)h_i\). The mean score is the linear functional \(S_v(c,z)\).
Let \(I\) be finite and \(v\ge 0\). Write \(p_v(c,z)\) for the \(N(c,v)\) density when \(v\gt 0\) and set \(p_0(c,z)=0\); define \(G_v(c,z)=\prod _{i\in I}p_v(c_i,z_i)\) and interpret division by zero as zero. For \(c,z,h\in \mathbb R^I\), put \(S_v(c,z)[h]=\sum _i((z_i-c_i)/v)h_i\). For every \(c,z\), the derivative in \(c\) of \(S_v(c,z)\) is the bilinear map \((h,k)\mapsto -\sum _i h_i k_i/v\).
Let \(I\) be finite and \(v\ge 0\). Put \(G_v(c,z)=\prod _i p_v(c_i,z_i)\), where \(p_v\) is the Gaussian density of variance \(v\) for \(v\gt 0\) and \(p_0=0\), and put \(S_v(c,z)[h]=\sum _i((z_i-c_i)/v)h_i\), with division by zero equal to zero. Define \(A_v(c,z)[h]=G_v(c,z)S_v(c,z)[h]\) and \(T_v(c,z)[h,k]=G_v(c,z)(S_v(c,z)[h]S_v(c,z)[k]-\sum _i h_i k_i/v)\). The product-density Hessian is the bilinear form \(T_v(c,z)\).
Let \(I\) be finite and \(v\ge 0\). Put \(G_v(c,z)=\prod _i p_v(c_i,z_i)\), where \(p_v\) is the Gaussian density of variance \(v\) for \(v\gt 0\) and \(p_0=0\), and put \(S_v(c,z)[h]=\sum _i((z_i-c_i)/v)h_i\), with division by zero equal to zero. Define \(A_v(c,z)[h]=G_v(c,z)S_v(c,z)[h]\) and \(T_v(c,z)[h,k]=G_v(c,z)(S_v(c,z)[h]S_v(c,z)[k]-\sum _i h_i k_i/v)\). For all \(c,z\in \mathbb R^I\), the derivative at \(c\) of \(m\mapsto A_v(m,z)\) is \(T_v(c,z)\).
Let \(I\) be finite and \(v\ge 0\). Put \(G_v(c,z)=\prod _i p_v(c_i,z_i)\), where \(p_v\) is the Gaussian density of variance \(v\) for \(v\gt 0\) and \(p_0=0\), and put \(S_v(c,z)[h]=\sum _i((z_i-c_i)/v)h_i\), with division by zero equal to zero. Define \(A_v(c,z)[h]=G_v(c,z)S_v(c,z)[h]\) and \(T_v(c,z)[h,k]=G_v(c,z)(S_v(c,z)[h]S_v(c,z)[k]-\sum _i h_i k_i/v)\). For all \(c,z,h,k\in \mathbb R^I\), \(T_v(c,z)[h,k]=G_v(c,z)\sum _{i,j}((z_i-c_i)(z_j-c_j)/v^2-\delta _{ij}/v)h_jk_i\), where \(\delta _{ij}\) is the Kronecker delta.
Let \(I\) be finite and \(v\ge 0\). Write \(p_v(c,z)\) for the \(N(c,v)\) density when \(v\gt 0\) and set \(p_0(c,z)=0\); define \(G_v(c,z)=\prod _{i\in I}p_v(c_i,z_i)\) and interpret division by zero as zero. Assume \(c,z,a\in \mathbb R^I\), \(a_i\ge 0\), \(p_v(c_i,z_i)\le a_i\), and \(|((z_i-c_i)^2/v^2-1/v)p_v(c_i,z_i)|\le a_i\) for every \(i\in I\). For every \(i\in I\), \(((z_i-c_i)^2/v^2)G_v(c,z)\le (1+1/v)\prod _j a_j\).
Let \(I\) be finite and \(v\ge 0\). Write \(p_v(c,z)\) for the \(N(c,v)\) density when \(v\gt 0\) and set \(p_0(c,z)=0\); define \(G_v(c,z)=\prod _{i\in I}p_v(c_i,z_i)\) and interpret division by zero as zero. Assume \(c,z,a\in \mathbb R^I\), \(a_i\ge 0\), \(p_v(c_i,z_i)\le a_i\), and \(|((z_i-c_i)^2/v^2-1/v)p_v(c_i,z_i)|\le a_i\) for every \(i\in I\). If \(v\gt 0\), then for every \(i,j\in I\), \(|((z_i-c_i)(z_j-c_j)/v^2-\delta _{ij}/v)G_v(c,z)|\le (1+2/v)\prod _k a_k\), where \(\delta _{ij}\) is the Kronecker delta.
Let \(I\) be finite and \(v\ge 0\). Put \(G_v(c,z)=\prod _i p_v(c_i,z_i)\), where \(p_v\) is the Gaussian density of variance \(v\) for \(v\gt 0\) and \(p_0=0\), and put \(S_v(c,z)[h]=\sum _i((z_i-c_i)/v)h_i\), with division by zero equal to zero. Define \(A_v(c,z)[h]=G_v(c,z)S_v(c,z)[h]\) and \(T_v(c,z)[h,k]=G_v(c,z)(S_v(c,z)[h]S_v(c,z)[k]-\sum _i h_i k_i/v)\). For fixed \(v\ge 0\), the map \((c,z)\mapsto T_v(c,z)\) is continuous in bilinear operator norm.
Let \(I\) be finite and \(a_{ij}\in \mathbb R\). For the bilinear form \(T(h,k)=\sum _{i,j}a_{ij}h_jk_i\) on maximum-norm vectors in \(\mathbb R^I\), \(\| T\| \le \sum _{i,j}|a_{ij}|\).
Let \(I\) be finite and \(v\ge 0\). Put \(G_v(c,z)=\prod _i p_v(c_i,z_i)\), where \(p_v\) is the Gaussian density of variance \(v\) for \(v\gt 0\) and \(p_0=0\), and put \(S_v(c,z)[h]=\sum _i((z_i-c_i)/v)h_i\), with division by zero equal to zero. Define \(A_v(c,z)[h]=G_v(c,z)S_v(c,z)[h]\) and \(T_v(c,z)[h,k]=G_v(c,z)(S_v(c,z)[h]S_v(c,z)[k]-\sum _i h_i k_i/v)\). If \(v\gt 0\) and \(M\ge 0\), there is a nonnegative Lebesgue-integrable \(b:\mathbb R^I\to \mathbb R\) such that \(\| T_v(m,z)\| \le b(z)\) for every \(z\) and every \(m\) satisfying \(|m_i|\le M\) for all \(i\); the bilinear operator norm uses maximum norms.
Let \(I\) be finite and \(v\ge 0\). Put \(G_v(c,z)=\prod _i p_v(c_i,z_i)\), where \(p_v\) is the Gaussian density of variance \(v\) for \(v\gt 0\) and \(p_0=0\), and put \(S_v(c,z)[h]=\sum _i((z_i-c_i)/v)h_i\), with division by zero equal to zero. Define \(A_v(c,z)[h]=G_v(c,z)S_v(c,z)[h]\) and \(T_v(c,z)[h,k]=G_v(c,z)(S_v(c,z)[h]S_v(c,z)[k]-\sum _i h_i k_i/v)\). Let \(f:\mathbb R^I\to \mathbb R\) be measurable with \(|f|\le 1\). If \(v\gt 0\) and \(c\in \mathbb R^I\), then \(z\mapsto f(z)T_v(c,z)\) is Lebesgue-integrable as a bilinear-form-valued function.
Let \(I\) be finite and \(v\ge 0\). Put \(G_v(c,z)=\prod _i p_v(c_i,z_i)\), where \(p_v\) is the Gaussian density of variance \(v\) for \(v\gt 0\) and \(p_0=0\), and put \(S_v(c,z)[h]=\sum _i((z_i-c_i)/v)h_i\), with division by zero equal to zero. Define \(A_v(c,z)[h]=G_v(c,z)S_v(c,z)[h]\) and \(T_v(c,z)[h,k]=G_v(c,z)(S_v(c,z)[h]S_v(c,z)[k]-\sum _i h_i k_i/v)\). Let \(f:\mathbb R^I\to \mathbb R\) be measurable with \(|f|\le 1\). If \(v\gt 0\), then \(m\mapsto \int f(z)A_v(m,z)\, dz\) is differentiable at every \(c\), with derivative \(\int f(z)T_v(c,z)\, dz\).
Let \(I\) be finite and \(v\ge 0\). Put \(G_v(c,z)=\prod _i p_v(c_i,z_i)\), where \(p_v\) is the Gaussian density of variance \(v\) for \(v\gt 0\) and \(p_0=0\), and put \(S_v(c,z)[h]=\sum _i((z_i-c_i)/v)h_i\), with division by zero equal to zero. Define \(A_v(c,z)[h]=G_v(c,z)S_v(c,z)[h]\) and \(T_v(c,z)[h,k]=G_v(c,z)(S_v(c,z)[h]S_v(c,z)[k]-\sum _i h_i k_i/v)\). Let \(f:\mathbb R^I\to \mathbb R\) be measurable with \(|f|\le 1\). If \(v\gt 0\), \(c,h,k\in \mathbb R^I\), and \(Z\sim \bigotimes _iN(c_i,v)\), then \((\int f(z)T_v(c,z)\, dz)[h,k]=\sum _{i,j}\mathbb E[((Z_i-c_i)(Z_j-c_j)/v^2-\delta _{ij}/v)f(Z)]h_jk_i\), where \(\delta _{ij}\) is the Kronecker delta.
Let \(I\) be finite and \(v\ge 0\). Put \(G_v(c,z)=\prod _i p_v(c_i,z_i)\), where \(p_v\) is the Gaussian density of variance \(v\) for \(v\gt 0\) and \(p_0=0\), and put \(S_v(c,z)[h]=\sum _i((z_i-c_i)/v)h_i\), with division by zero equal to zero. Define \(A_v(c,z)[h]=G_v(c,z)S_v(c,z)[h]\) and \(T_v(c,z)[h,k]=G_v(c,z)(S_v(c,z)[h]S_v(c,z)[k]-\sum _i h_i k_i/v)\). Let \(f:\mathbb R^I\to \mathbb R\) be measurable with \(|f|\le 1\). If \(v\gt 0\), then \(\| \int f(z)T_v(c,z)\, dz\| \le 2|I|^2/v\) for every \(c\in \mathbb R^I\), using maximum norms for the bilinear operator norm.
Let \(I\) be finite and \(v\ge 0\). Write \(p_v(c,z)\) for the \(N(c,v)\) density when \(v\gt 0\) and set \(p_0(c,z)=0\); define \(G_v(c,z)=\prod _{i\in I}p_v(c_i,z_i)\) and interpret division by zero as zero. Let \(A_v(c,z)\) be the linear functional \(h\mapsto \sum _i\bigl(\prod _{j\ne i}p_v(c_j,z_j)\bigr)((z_i-c_i)/v)p_v(c_i,z_i)h_i\). Let \(f:\mathbb R^I\to \mathbb R\) be measurable with \(|f|\le 1\). If \(v\gt 0\), the map \(c\mapsto \int f(z)A_v(c,z)\, dz\) is globally \(2|I|^2/v\)-Lipschitz, with maximum norm on mean vectors and operator norm on linear functionals.