TCSLib

110 Error-Correcting Codes — MRRW Bound

110.1 Overview

This chapter formalizes the binary first McEliece–Rodemich–Rumsey–Welch (MRRW) bound on the asymptotic rate of binary codes: for every \(\delta \in [0,1/2]\),

\[ R(\delta ) \; \le \; H\! \left(\tfrac 12 - \sqrt{\delta (1-\delta )}\right), \]

where \(R\) is the asymptotic binary code rate and \(H\) is the binary entropy function. The development supplies the maximum code size \(A(n,d)\), the binary Krawtchouk polynomials (both as an explicit alternating sum and as genuine polynomials over \(\mathbb {R}\) defined by the three-term recurrence), Delsarte’s linear-programming bound, the truncated Christoffel–Darboux kernel \(\Phi _{n,t}\) together with its feasibility properties, and the asymptotic ingredients needed to assemble the main theorem.

110.2 Declarations

110.2.1 Statement and normalization

Definition 110.1 Maximum size of a binary code with given distance
✓
#

For naturals \(n\) and \(d\), \(A(n,d)\) is the supremum of the cardinalities \(k\) for which there is a finite set \(C\) of binary codewords \(\mathrm{Fin}\, n \to \mathrm{Bool}\) with \(|C| = k\) and such that any two distinct \(x, y \in C\) differ in at least \(d\) coordinates.

Definition 110.2 Binary entropy in base \(2\)
✓
#

The binary entropy function

\[ H(x) \; =\; -x\log _2 x - (1-x)\log _2(1-x), \]

using the base-\(2\) logarithm, with the convention \(0\log _2 0 = 0\) inherited from \(\log _2 0 = 0\).

Definition 110.3 Asymptotic binary code rate
✓
#

For \(\delta \in [0,1]\), the asymptotic rate is

\[ R(\delta ) \; =\; \limsup _{n\to \infty } \frac{1}{n}\log _2 A(n, \lfloor \delta n \rfloor ). \]

110.2.2 Krawtchouk polynomials

Definition 110.4 Binary Krawtchouk polynomial at integer arguments
✓
#

For naturals \(n\), \(j\), \(x\),

\[ K_j^{(n)}(x) \; =\; \sum _{i=0}^{j} (-1)^i \binom {x}{i}\binom {n-x}{j-i}, \]

where the subtraction \(n-x\) is natural-number subtraction; the definition gives the intended values for \(x \le n\).

Definition 110.5 Krawtchouk polynomials over \(\mathbb {R}\)
✓
#

The family of real polynomials defined by the three-term recurrence \(K_0 = 1\), \(K_1 = n - 2X\), and

\[ (j+2)\, K_{j+2} \; =\; (n - 2X)\, K_{j+1} - (n-j)\, K_j, \]

extending the integer-argument definition to real arguments.

Theorem 110.6 Value of the zeroth Krawtchouk polynomial
✓
#

For all natural numbers \(n\) and \(x\), the degree-zero binary Krawtchouk polynomial is identically \(1\); that is,

\[ K_0^{(n)}(x) = 1. \]
Theorem 110.7 Krawtchouk polynomial at zero
✓
#

Let \(n\) and \(j\) be natural numbers with \(j \le n\). Then the binary Krawtchouk polynomial \(K_j^{(n)}\) evaluated at \(0\) equals the binomial coefficient \(\binom {n}{j}\), that is,

\[ K_j^{(n)}(0) = \binom {n}{j}. \]
Theorem 110.8 Value of the first Krawtchouk polynomial
✓
#

Let \(n\) and \(x\) be natural numbers with \(x \le n\). Then the first binary Krawtchouk polynomial at \(x\) satisfies

\[ K_1^{(n)}(x) = n - 2x. \]
Theorem 110.9 Generating function of the Krawtchouk polynomials
✓
#

Let \(n\) and \(x\) be natural numbers with \(x \le n\), and let \(K_j^{(n)}(x)\) denote the binary Krawtchouk polynomial. Then for every real \(z\),

\[ \sum _{j=0}^{n} K_j^{(n)}(x)\, z^{j} \; =\; (1-z)^{x}\, (1+z)^{\, n-x}. \]
Theorem 110.10 Orthogonality of the Krawtchouk polynomials
✓
#

Fix a natural number \(n\), and let \(r, s\) be natural numbers with \(r \le n\) and \(s \le n\). Then the binary Krawtchouk polynomials satisfy

\[ \sum _{x=0}^{n} \binom {n}{x}\, K_r^{(n)}(x)\, K_s^{(n)}(x) \; =\; \begin{cases} 2^{n}\dbinom {n}{r}, & r = s,\\[2pt] 0, & r \ne s; \end{cases} \]

that is, they are orthogonal with respect to the weights \(\binom {n}{x}\) on \(\{ 0, 1, \dots , n\} \).

Theorem 110.11 Three-term recurrence for Krawtchouk polynomials
✓
#

Let \(n\), \(j\), and \(x\) be natural numbers with \(1 \le j\), \(j + 1 \le n\), and \(x \le n\). Then the binary Krawtchouk polynomials \(K_j^{(n)}\) satisfy the three-term recurrence

\[ (j+1)\, K_{j+1}^{(n)}(x) \; =\; (n - 2x)\, K_j^{(n)}(x) \; -\; (n - j + 1)\, K_{j-1}^{(n)}(x). \]
Theorem 110.12 Krawtchouk polynomial evaluated at an integer
✓

Fix a natural number \(n\), and let \(j, x\) be natural numbers with \(j \le n\) and \(x \le n\). Then the Krawtchouk polynomial \(K_j^{(n)}\), evaluated at the real number \(x\), equals the alternating sum

\[ K_j^{(n)}(x) \; =\; \sum _{i=0}^{j} (-1)^i \binom {x}{i}\binom {n-x}{j-i}. \]

110.2.3 Delsarte’s linear programming bound

Theorem 110.13 Delsarte linear-programming bound
#

Fix naturals \(n\) and \(d\), and let \(F_0, F_1, \dots , F_n\) be real coefficients defining the function \(F(x) = \sum _{j=0}^{n} F_j\, K_j^{(n)}(x)\). Suppose that \(F_0 \gt 0\), that \(F_j \ge 0\) for every \(j\) with \(1 \le j \le n\), and that \(F(x) \le 0\) for every integer \(x\) with \(d \le x \le n\). Then the maximum size of a binary code of length \(n\) and minimum distance \(d\) satisfies

\[ A(n,d) \; \le \; \frac{F(0)}{F_0}. \]

110.2.4 Christoffel–Darboux kernel and feasibility

Definition 110.14 Truncated Christoffel–Darboux kernel
✓
#

For \(t \le n\) and real \(a, x\), the truncated reproducing kernel is

\[ \Phi _{n,t}(a,x) \; =\; \sum _{j=0}^{t} \frac{K_j^{(n)}(a)\, K_j^{(n)}(x)}{\binom {n}{j}}. \]
Theorem 110.15 Christoffel–Darboux identity for Krawtchouk polynomials
✓
#

Let \(n, t\) be natural numbers with \(t + 1 \le n\), and let \(a, x\) be real numbers with \(a \ne x\). Then there is a positive constant \(c \gt 0\) for which the truncated Christoffel–Darboux kernel satisfies

\[ \Phi _{n,t}(a,x) \; =\; c\, \frac{K_t^{(n)}(a)\, K_{t+1}^{(n)}(x) - K_{t+1}^{(n)}(a)\, K_t^{(n)}(x)}{a - x}, \]

where \(K_j^{(n)}\) denotes the \(j\)-th Krawtchouk polynomial.

Theorem 110.16 Positivity of Krawtchouk values below the smallest zero

Let \(n\) and \(t\) be natural numbers with \(t + 1 \le n\), and let \(K_j^{(n)}\) denote the degree-\(j\) Krawtchouk polynomial associated with \(n\). Suppose \(a\) is a real number lying strictly below every real zero of \(K_t^{(n)}\): that is, \(a \lt \xi \) whenever \(K_t^{(n)}(\xi ) = 0\). Then \(K_j^{(n)}(a) \gt 0\) for every \(j\) with \(j \le t\).

Theorem 110.17 Nonpositivity of the kernel beyond a threshold

Fix natural numbers \(t\) and \(n\) with \(t + 1 \le n\), and let \(a\) be a real number lying strictly to the left of every real zero of the Krawtchouk polynomial \(K_t^{(n)}\); that is, \(a \lt \xi \) whenever \(K_t^{(n)}(\xi ) = 0\). Then there is a real threshold \(\eta \) such that the truncated Christoffel–Darboux kernel satisfies \(\Phi _{n,t}(a, x) \le 0\) for every integer \(x\) with \(\eta \le x \le n\).

Theorem 110.18 Finite-length MRRW linear-programming bound
✓

Fix natural numbers \(n\), \(t\), and \(d\) with \(t + 1 \le n\), and let \(a\) be a real number lying strictly below every real zero of the Krawtchouk polynomial \(K_t^{(n)}\). Suppose that the truncated Christoffel–Darboux kernel satisfies \(\Phi _{n,t}(a, x) \le 0\) for every integer \(x\) with \(d \le x \le n\). Then the maximum size of a binary code of length \(n\) and minimum distance \(d\) is bounded by

\[ A(n, d) \; \le \; \Phi _{n,t}(a, 0). \]

110.2.5 Asymptotic choice of parameters

Theorem 110.19 Asymptotics of the smallest Krawtchouk zero

Fix \(\tau \) with \(0 \lt \tau \lt 1/2\), and let \((t_n)\) be a sequence of natural numbers with \(t_n/n \to \tau \). Then there is a real sequence \((a_n)\) with the following three properties. First, for all sufficiently large \(n\), the number \(a_n\) lies strictly below every real zero of the Krawtchouk polynomial \(K_{t_n}^{(n)}\); that is, \(K_{t_n}^{(n)}(\xi ) = 0\) implies \(a_n \lt \xi \). Second, \(a_n/n \to \tfrac 12 - \sqrt{\tau (1-\tau )}\). Third, for all sufficiently large \(n\), the truncated Christoffel–Darboux kernel satisfies \(\Phi _{n,t_n}(a_n, x) \le 0\) for every integer \(x\) with \(\big\lfloor (\tfrac 12 - \sqrt{\tau (1-\tau )})\, n \big\rfloor \le x \le n\).

Theorem 110.20 Entropy growth rate of the objective

Fix \(0 \lt \tau \lt 1/2\), and let \((t_n)\) be a sequence of natural numbers and \((a_n)\) a sequence of reals such that \(t_n/n \to \tau \) and, for all sufficiently large \(n\), the point \(a_n\) lies strictly below every real zero of the Krawtchouk polynomial \(K_{t_n}^{(n)}\). Then the truncated Christoffel–Darboux kernel evaluated at \((a_n, 0)\) grows at rate at most \(H(\tau )\), in the sense that

\[ \limsup _{n\to \infty } \frac{1}{n}\log _2 \Phi _{n,t_n}(a_n, 0) \; \le \; H(\tau ), \]

where \(H\) is the binary entropy function in base \(2\).

Theorem 110.21 Entropy asymptotics for binomial tail sums
#

Fix a real number \(\tau \) with \(0 \le \tau \le \tfrac 12\). Then the normalized base-\(2\) logarithm of the lower binomial tail sum converges to the binary entropy of \(\tau \):

\[ \frac{1}{n}\log _2\! \left(\sum _{j=0}^{\lfloor \tau n\rfloor } \binom {n}{j}\right) \; \longrightarrow \; H(\tau ) \qquad (n \to \infty ). \]

110.2.6 Conclusion of the MRRW bound

Theorem 110.22 An involution of the interval \([0,\tfrac 12]\)
✓
#

Let \(\delta \) be a real number with \(0 \le \delta \le \tfrac 12\), and define \(\tau = \tfrac 12 - \sqrt{\delta (1-\delta )}\). Then \(\tau \) again lies in \([0,\tfrac 12]\) and applying the same rule to it recovers \(\delta \), that is, \(\tfrac 12 - \sqrt{\tau (1-\tau )} = \delta \). Hence the map \(x \mapsto \tfrac 12 - \sqrt{x(1-x)}\) is an involution of the interval \([0,\tfrac 12]\).

Theorem 110.23 Binary entropy at zero
✓
#

Let \(H(x) = -x\log _2 x - (1-x)\log _2(1-x)\) be the binary entropy function in base \(2\), under the convention \(0\log _2 0 = 0\). Then \(H(0) = 0\).

Theorem 110.24 Binary entropy at one-half
✓
#

The binary entropy function \(H\) attains the value \(1\) at the point \(1/2\); that is, \(H(1/2) = 1\).

Theorem 110.25 First linear-programming (MRRW) bound on binary codes
#

For every real \(\delta \) with \(0 \le \delta \le 1/2\), the asymptotic binary code rate satisfies

\[ R(\delta ) \; \le \; H\! \left(\tfrac 12 - \sqrt{\delta (1-\delta )}\right), \]

where \(H\) is the base-\(2\) binary entropy function.