Path: blob/main/course-contents/2024-02-26--block+linear.md
908 views
------\newcommand{\F}{\mathbb{F}} \newcommand{\Z}{\mathbb{Z}} \newcommand{\NN}{\mathbb{N}} \newcommand{\e}{\mathbf{e}} \newcommand{\h}{\mathbf{h}} \newcommand{\bb}{\mathbf{b}} \newcommand{\vv}{\mathbf{v}} \newcommand{\ww}{\mathbf{w}}
\newcommand{\Null}{\operatorname{Null}}
\newcommand{\PP}{\mathbb{P}}
\newcommand{\dist}{\operatorname{dist}} \newcommand{\floor}[1]{\lfloor #1 \rfloor}
Bounds for block codes, continued
Let be a block code, where , and suppose that is the minimal distance of .
Recall that we showed that "corrects up to errors", where ParseError: KaTeX parse error: Undefined control sequence: \floor at position 5: t = \̲f̲l̲o̲o̲r̲{(d-1)/2}.
And recall that is the maximal size of a code with minimal distance .
Let The (closed) ball of radius in satisfies .
Recall that the Gilbert-Varshamov result gave in some sense a lower bound result for ; it showed that
We now give an upper bound for known as the sphere-packing bound.
Theorem (Sphere-packing bound) : Let ParseError: KaTeX parse error: Undefined control sequence: \floor at position 5: t = \̲f̲l̲o̲o̲r̲{(d-1)/2}. Then
Proof : Let be a code of minimal distance with .
Remark : A code is said to be perfect if it meets the sphere packing bound; i.e. if .
Lemma (Plotkin Lemma) : Let , , and suppose the maximal distance of is . Then
Proof : Fix and for each write for the number of times appears as the th coordinate of a codeword in .
Asymptotics of codes
(sketch/motivation)
If we wish to send a large amount of data with short length codes, we have to cut up a string of "bits" of data into strings of some fixed length .
If the probability of decoding the string of length is , then the probability of decoding the string of length is . For fixed , note that $$p^{n/n_0} \to 0 \quad \text{as $n \to \inftyParseError: KaTeX parse error: Expected 'EOF', got '}' at position 1: }̲$
On the other hand, Shanon's Theorem promises us that we should be able to send the string of bits through a channel with some given capacity which encodes almost bits of information and then decode correctly with probability approaching .
Now, a proof of Shannon's theorem e.g. for the binary symmetric channel uses the fact that the average number of errors which occur in the transmission of bits is .
Thus, to satisfy Shannon's Theorem, our code should be able to correct a number of errors that is linear in -- i.e. we want to construct codes of length for which the minimal distance grows linearly with .
Defn : A sequence of asymptotically good codes is a sequence where is a code of length , dimension and minimal distance for which and are bounded away from zero (as ).
In some sense, the goal of constructing asymptotically good codes hopefully makes clear the utility of the preceding result on bounds for codes.
Decoding
Let be a (linear) -code.
The following diagram outlines components of usage of such a code for data transmission:
{width=750px}
So, we begin with data ParseError: KaTeX parse error: Undefined control sequence: \F at position 16: \mathbf{x} \in \̲F̲_q^k. We encode it using a generator matrix for the code :
Now, this vector in ParseError: KaTeX parse error: Undefined control sequence: \F at position 1: \̲F̲_q^n is somehow transmitted through the channel; the received data is a vector ParseError: KaTeX parse error: Undefined control sequence: \vv at position 1: \̲v̲v̲ ̲\in \F_q^n, possible suffering transmission errors.
This leaves the decoding step: how do we hope to recover from ParseError: KaTeX parse error: Undefined control sequence: \vv at position 1: \̲v̲v̲ the data vector ParseError: KaTeX parse error: Undefined control sequence: \F at position 16: \mathbf{x} \in \̲F̲_q^k?
Standard array decoding
Here is a fairly simple-to-describe procedure for decoding.
For each coset in ParseError: KaTeX parse error: Undefined control sequence: \F at position 1: \̲F̲_q^n, find an element with minimal weight.
Now, to decode the vector ParseError: KaTeX parse error: Undefined control sequence: \vv at position 1: \̲v̲v̲, find the coset containing ParseError: KaTeX parse error: Undefined control sequence: \vv at position 1: \̲v̲v̲, and write ParseError: KaTeX parse error: Undefined control sequence: \ww at position 1: \̲w̲w̲ for the element (chosen previously) of minimal length.
Notice that ParseError: KaTeX parse error: Undefined control sequence: \vv at position 1: \̲v̲v̲ ̲- \ww \in C (since both vectors are in ); we decode to this vector.
Example : Let's consider a code with ParseError: KaTeX parse error: Undefined control sequence: \F at position 5: k = \̲F̲_2.