User:Raman Sonkhla/Parvaresh-Vardy codes and list decoder

Introduction

We know that a Reed Solomon code (RS code), $\mathbb {F} _{q}^{k}\rightarrow \mathbb {F} _{q}^{n}$, has a distance $1-{\dfrac {k}{n}}$ and a list decoding radius $1-{\sqrt {\dfrac {k}{n}}}$. Hence the list decoding rate, R, of the code is $(1-p)^{2}$, where $p$ is the number of fraction errors. Now the next question is can we arithmetically do better than this in polynomial time? For seven years after the Guruswami-Sudan, there was no progress till the break through work of Parvaresh and Vardy. The Parvaresh-Vardy (PV) codes are based on RS codes.

The list decoding algorithm is based on two key ideas. First is the transition from bi-variate polynomial interpolation to multivariate interpolation decoding. The second key idea is to take different approach than that is taken with RS codes as a number of prior attempts to overcome the $1-{\sqrt {R}}$ rate barrier has already proved unsuccessful. Hence rather than devising a better list-decoder for RS codes, new codes were constructed.

Standard RS encoder view a message as a polynomial $f(X)$ over a field $\mathbb {F} _{q}$, and produce the corresponding codeword by evaluating $f(X)$ at $n$ distinct elements of $\mathbb {F} _{q}$. In case of PV codes, given $f(X)$, first related polynomials $g_{1}(X),g_{2}(X),...,g_{M-1}(X)$ are computed and then the corresponding codeword is produced by evaluating all these polynomials. Correlation between $g(X)$ and $f(X)$ is of form $g(X)=(f(X))^{a}$ Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. TeX parse error: Extra close brace or missing open brace"): {\displaystyle \mod } $e\left(X\right)$. Here $e(X)$ is an arbitrary irreducible (over $\mathbb {F} _{q}$) polynomial of degree k, and $a$ is an arbitrary (but sufficient large) integer. Correlation between $f(X)$ and $g(X)$ when $M\geq 2$ provides information that is exploited to break the $1-{\sqrt {R}}$ fraction of error barrier for adversarial errors.

Decoder

Input for the PV list decoding algorithm will be $(\alpha _{i},y_{i},z_{i})|_{i=1}^{N}\in \mathbb {F} ^{3}$ and $t\geq 0$. Here $t$ is called the agreement parameter. The output of algorithm will be all degree $\leq K$ polynomials $f(X)$ such that the PV codeword corresponding to $f(X)$ agrees with the received word in at least $t$ places. The algorithm consists of the following steps,

1. Find $Q(x,y,z)\not \equiv 0$ such that
   1. $Q(\alpha _{i},\beta _{i},\gamma _{i})$ for all $i=1,...,n$.
   2. $deg_{x}$ $Q\leq k^{2/3}n^{1/3},deg_{y}$ $Q,deg_{z}$ $Q\leq ({\dfrac {n}{k}})^{1/3}$.
2. While $g(x)|Q(x,y,z)$, put $Q\leftarrow Q/g$.
3. Put $Q_{x}(y,z)=Q(x,y,z)$ $(mod$ $g(x))$, put $p_{x}(y)=Q_{x}(y,y^{D}),$ and output all roots of $p_{x}\in E=\mathbb {F} _{q}[x]/g(x)$.

Here note that $Q_{x}\in E[y,z]$ and if $Q(x,y,z)\not \equiv 0$ and $h\nmid Q$ then $Q_{x}(y,z)\not \equiv 0$.

Performance analysis of PV codes

In case of PV codes the message corresponds to an element of $\mathbb {\mathbb {F} } _{q}^{k}=(\mathbb {F} _{q}^{2})^{k/2}$, i.e. $k/2$ symbols from the alphabet $\mathbb {F} _{q^{2}}$. Hence rate is $R={\dfrac {k}{2n}}$. As we can list decode from $3k^{2/3}n^{1/3}$ agreement, hence we could recover from ${\dfrac {n-3k^{2/3}n^{1/3}}{n}}=1-3(2R)^{2/3}=1-O(R^{2/3})$ fraction of errors. On the other hand RS codes achieved only a rate of $1-{\sqrt {R}}$.

Improvements

Some improvements that can be made to PV codes include,

1. We can insist that $Q$ have multiple roots at $(\alpha _{i},\beta _{i},\gamma _{i})$. This would eliminate the leading constant factor of 3 in $n-3k^{2/3}n^{1/3}$, and would improve rate to $1-(2R)^{2/3}$.
2. Additionally we can use correlated polynomials to extract additional performance from PV codes. Let $p_{1}$ be the message that we want to transmit. For $j=2,...,m$ we put $p_{j}=p_{(}j-1)^{D}(modg(x))$. This results in following encoding, $p_{1}\mapsto {(p_{1}(\alpha _{i}),...,p_{m}(\alpha _{i}))}_{i=1}^{n}$.

Now although we pay extra running time in $m$ while decoding, but its still remains a polynomial time algorithm for any fixed m and yield recovery from $1-(mR)^{\dfrac {m}{m+1}}$ fraction of errors. Asymptotically, for large $m$, this approaches (letting $R\rightarrow 0$ now) $1-O(R-log{\dfrac {1}{R}})$ but doesn’t really do much better for any fixed R. Also since alphabet becomes $m$-tuples of $\mathbb {F} _{q}$, rate can not possibly increase ${\dfrac {1}{m}}$.

Application in Guruswami-Umans-Vadhan Expander Construction

Expanders are highly connected yet sparse graphs. They have a wide variety of applications in theoretical computer science, in designing algorithms, to construct hash functions in cryptography, error correcting codes, extractors, pseudorandom generators, sorting networks and robust computer networks. The construction of expanders of Guruswami-Umans-Vadhan is based on the list decodable codes of Parvaresh and Vardy.

Let us review the basics of list decodable codes. We take C as the code which is a mapping $C:\left[N\right]\mapsto \left[M\right]^{D}$ encoding messages of bit length $n=\log _{2}\left[N\right]$ to$D$ symbols over the alphabet $\left[M\right].$ Rate of such a code will be $\rho =n/\left(D\log _{2}M\right)$. We call $C$ as $\left(\varepsilon ,K\right)$ list decodable if for every$r\in [M]^{D}$, the set LIST$(r,\varepsilon )=^{def}{x:Pr_{y}[C(x)_{y}=r_{y}]\geq \varepsilon }$ is of atmost K size. With list decodable codes, we wish to optimize the tradeoff between the agreement $\varepsilon$ and the rate $\rho$ which do not depend on message length M. Sudan showed that such a property can be achieved by Reed Solomon Codes in polynomial time. This tradeoff was then improved by Guruswami and Sudan and recently by Parvaresh and Vardy who improved the tradeoff by using a variant of Reed Solomon codes.

GUV Constructor

The construction of Guruswami-Umans-Vadhan Expander is based on Parvaresh Vardy codes.We know that a typical Parvaresh Vardy codeword has several related degree $m-1$ polynomials $f_{0},f_{1},f_{2}...f_{m-1}$ evaluated at all points in the field and $f\in \mathbb {F} _{q}\left[Y\right]$ where $q$ is a prime power over which the field $\mathbb {F}$ is defined. All such evaluations are packaged into larger alphabet $\mathbb {F} _{q^{m}}$ symbol. This extra redundancy enables a better list decoding algorithm than Reed Solomon ones. Elements of $\mathbb {F}$ are chosen such that $f_{i}={f_{0}}^{h^{i}}$ for $i\geq 1$ and $h\geq 1$ integer parameter.

We need to show that for a given set $T$ of size $L$, the set LIST$\left(T\right)=\{f_{0}:\Gamma \left(f_{0}\right)\subseteq T\}$ is small.

Expander Graphs

Lets start with some definitions : For a bipartite graph $\Gamma :\left[N\right]\times \left[D\right]\mapsto \left[M\right]$ and a set $T\subseteq \left[M\right]$, define $LIST\left(T\right)=\{x\in \left[N\right]:\Gamma \left(x\right)\subseteq T\}$. Also, a digraph $G$ is a $\left(K,A\right)$ vertex expander if for all sets $S$ of at most $K$ vertices, the neighborhood $N\left(S\right)$ is of size atleast $A.|S|$ where neighborhood $N\left(S\right)=^{def}\{u|\exists v\in S$ s.t. $\left(u,v\right)\in E\}$. Details can be found out in the paper Expander graphs and vertex expansion. This proves the following lemma:

Lemma- A graph $\Gamma$ is a$\left(K,A\right)$ expander if and only if for every set $T$ of size at most $AK-1,LIST\left(T\right)$ is of size at most $K-1$.

Construction

Fix the field $\mathbb {F} _{q}$ and let $E\left(Y\right)$be an irreducible polynomial of degree $n$ over the field $\mathbb {F} _{q}$. Elements of ${\mathbb {F} _{q}}^{n}$ are univariate polynomials over $\mathbb {F} _{q}$ with degree at most $n-1$. $h$, integer parameter is fixed. The expander is bipartite graph $\Gamma _{PV}:{\mathbb {F} _{q}}^{n}\times \mathbb {F} _{q}\mapsto {\mathbb {F} _{q}}^{m+1}$ defined as: $\Gamma \left(f,y\right)=^{def}\left[y,f\left(y\right),\left(f^{h}\mod E\right)\left(y\right),\left(f^{h^{2}}\mod E\right)\left(y\right),...,\left(f^{h^{m-1}}\mod E\right)\left(y\right)\right]$                $...(eq1)$

The bipartite graph has message polynomials on the left and the $j^{'th}$ neighbor of $f\left(Y\right)$ is the $j^{'th}$ symbol of Parvaresh-Vardy encoding of $f\left(Y\right)$. This follows a theorem which can formally be stated as:

Theorem 1: The graph $\Gamma _{PV}:{\mathbb {F} _{q}}^{n}\times \mathbb {F} _{q}\mapsto {\mathbb {F} _{q}}^{m+1}$ is a $\left(\leq K_{max},A\right)$ expander for $K_{max}=h^{m}$ and $A=q-\left(n-1\right)\left(h-1\right)m$.

Proof: Let us take any integer $K$, where $K\leq K_{max}=h$ and let $A=q-\left(n-1\right)\left(h-1\right)m$. By the lemma defined above, if we take a $T$ such that $T\subseteq {\mathbb {F} _{q}}^{m+1}$ is of at most $AK-1$ size, then we need to show that $|LIST\left(T\right)|\leq K-1$.

Parvaresh-Vardy codes view degree $n-1$ polynomials as elements of field $\mathbb {F} =\mathbb {F} _{q}\left[Y\right]/E\left(Y\right)$ where $E$ is an irreducible polynomial of degree $n$. We need $Q$ that will have non zero coefficients on monomials of the form $X^{i}M_{j}\left(X_{1},....,X_{m}\right)$ for $0\leq i\leq A-1$ and $0\leq j\leq K-1\leq h^{m}-1$, where $M_{j}(X_{1},...,X_{m})={X_{1}}^{j_{0}}....{X_{m}}^{j_{m-1}}$ and $j=j_{0}+j_{1}h+...+j_{m-1}h^{m-1}$ is the base-$h$ representation of $j$. If we impose a homogeneous linear constraint on $AK$ coefficients of $Q$, then we require that $Q\left(z\right)=0$ for every $z\in T$. Since number of constraints is less than the number of unknowns, the linear system thus made has a solution that is not 0. If $Q$ has the smallest possible degree in variable $X$, then

$Q\left(X,X_{1},....,X_{m}\right)={\Sigma _{j=0}}^{K-1}p_{j}\left(X\right).M_{j}\left(X_{1},...,X_{m}\right)$                $...(eq2)$

for univariate polynomials $p_{0}\left(X\right),...,p_{K-1}\left(X\right)$, at least one of $p_{j}$ will not be divisible by $E\left(X\right)$. If every $p_{j}$ is divisible by $E\left(X\right)$ then $Q\left(X,X_{1},....,X_{m}\right)/E\left(X\right)$ will have smaller degree in $X$ and would still vanish on $T$ (since $E$ is irreducible and therefore has no roots in $\mathbb {F} _{q}$).

Let us take $f\left(X\right)\in LIST\left(T\right)$ to be any polynomial. Then by our $Q$, $Q\left(y,f_{0}\left(y\right),f_{1}\left(y\right),.....,f_{m-1}\left(y\right)\right)=0$ $\forall y\in \mathbb {F} _{q}$. This means, the univariate polynomial $R_{f}\left(X\right)=^{def}Q\left(X,f_{0}\left(X\right),f_{1}\left(X\right),.....,f_{m-1}\left(X\right)\right)$ has $q$ zeroes. Since $R_{f}\left(X\right)$ has at most degree $\left(A-1\right)+\left(n-1\right)\left(h-1\right)m<q$, then it is $0$. Refer Polynomials and properties for proof. So,

$Q\left(X,f_{0}\left(X\right),f_{1}\left(X\right),.....,f_{m-1}\left(X\right)\right)=0$

Recall that, we have, $f_{i}(X)\equiv f(X)^{h^{i}}(\mod E(X))$. Thus, $Q(X,f(X),f(X)^{h},....,f(X)^{h^{m-1}})\equiv Q(X,f_{0}(X),...,f_{m-1}(X)\equiv 0$ since $[0(\mod E(X))]$.

Then $f\left(X\right)$ which is an element of the extended field $\mathbb {F} =\mathbb {F} _{q}\left[Y\right]/E\left(Y\right)$ $($where $E$ is an irreducible polynomial of degree $n$$)$ is the root of univariate polynomial $Q^{*}$ over $\mathbb {F}$ defined by

$Q^{*}\left(Z\right)=^{def}Q\left(X,Z,Z^{h},Z^{h^{2}},...,Z^{h^{m-1}}\right)\mod E\left(X\right)$ From equation $\left(2\right)$, the above equation is same as:

$=\Sigma _{j=0}^{K-1}\left(p_{j}\left(X\right)\mod E\left(X\right)\right).M_{j}\left(Z,Z^{h},...,Z^{h^{m-1}}\right)$

$=\Sigma _{j=0}^{K-1}\left(p_{j}\left(X\right)\mod E\left(X\right)\right).Z^{j}$

Since this is true for all $f\left(X\right)\in LIST\left(T\right)$, $Q^{*}$ has at least $|LIST\left(T\right)|$ roots in field $\mathbb {F}$. Some $p_{j}\left(X\right)$'s is not divisible by $E\left(X\right)$, $Q^{*}$ is a non zero polynomial. Thus, $|LIST\left(T\right)|$ is bounded by the degree of $Q^{*}$, which is at most $K-1$.

By proper instantiation of parameters in Theorem 1, we lead to following results:

Theorem 2: For all positive integers $N$, $K_{max}\leq N$, all $\varepsilon >0$, and all $\alpha \in \left(0,\log x/\log \log x\right)$ for $x=\left(\log N\right)\left(\log K_{max}\right)/\varepsilon$, there is an explicit $\left(\leq K_{max},\left(1-\varepsilon \right)D\right)$ expander $\Sigma :\left[N\right]\times \left[D\right]\mapsto \left[M\right]$ with degree $D=O(((\log N){(\log K_{max})/\varepsilon )}^{1+{1/\alpha }})$ and $M\leq D^{2}.{K_{max}}^{1+\alpha }$. Moreover, $D$ and $M$ are powers of $2$.

Theorem 3: For all positive integers $N$, $K_{max}\leq N$, and all $\varepsilon >0$, there is an explicit $\left(\leq K_{max},\left(1-\varepsilon \right)D\right)$ expander $\Sigma :\left[N\right]\times \left[D\right]\mapsto \left[M\right]$ with degree $D\leq 2(\log N){(\log K_{max})/\varepsilon }$ and $M\leq {(4K_{max})}^{\log D}$. Again, $D$ and $M$ are powers of $2$..

The proofs of the above two theorems can be found from GUV paper.

References

1. Farzad Parvaresh and Alexander Vardy. Correcting errors beyond the Guruswami-Sudan radius in polynomial time In Proceedings of the 43nd Annual Symposium on Foundations of Computer Science (FOCS), pages 285-294, 2005.
2. Atri Rudra. Error Correcting Codes: Combinatorics, Algorithms and Applications Lecture 41
3. Madhu Sudan. Essential coding theory Lecture 15 and Lecture 16
4. Unbalanced Expanders and Randomness Extractors from Parvaresh–Vardy Codes - GUV paper.
5. Expander graphs.
6. Expander graphs and vertex expansion.
7. Bipartite graph.
8. Digraph.
9. Polynomials and their properties.

External links

• This article incorporates text from a comparable page at the website of SUNY Buffalo