In telecommunication, a Hamming code is an error-correcting code named after its inventor, Richard Hamming. Hamming codes can detect single and double-bit errors, and correct single-bit errors as well. In contrast, the simple parity code cannot detect errors where two bits are transposed, nor can it help correct the errors it can find.

History

Hamming worked at Bell Labs in the 1940s on the Bell Model V computer, an electromechanical relay-based monster with cycle times in seconds. Input was fed in on punch cards, which would invariably have read errors. During weekdays, special code would find errors and flash lights so the operators could correct the problem. During after-hours periods and on weekends, when there were no operators, the machine simply moved on to the next job.

Hamming worked on weekends, and grew increasingly frustrated with having to restart his programs from scratch due to the unreliability of the card reader. Over the next few years he worked on the problem of error-correction, developing an increasingly powerful array of algorithms. In 1950 he published what is now known as Hamming Code, which remains in use in some applications today.

Codes predating Hamming

A number of simple error-detecting codes were used before Hamming codes, but none were nearly as effective as Hamming codes in the same overhead of space. We describe some here.

Parity

Parity adds a single bit that indicates whether the number of 1 bits in the preceding data was even or odd. If a single bit is changed in transmission, the message will change parity and the error can be detected at this point. (Note that the bit that changed may have been the parity bit itself!) Parity checking is not very robust, since if the number of bits changed is even, the check bit will be valid and the error will not be detected.

Moreover, parity does not indicate which bit contained the error, even when it can detect it. The data must be discarded entirely, and re-transmitted from scratch. On a noisy transmission medium a successful transmission could take a long time, or even never occur. Parity does have the advantage, however, that it's about the best possible code that uses only a single bit of space.

Two-of-five

In the 1940s Bell used a slightly more sophisticated code known as two-of-five. This code ensured that every block of five bits (known as a 5-block) had exactly two 1s. The computer could tell if there was an error if in its input there were not exactly two 1s in each block. Two-of-five was still only able to detect single bits; if one bit flipped to a 1 and another to a 0 in the same block, the two-of-five rule remained true and the error would go undiscovered.

Error correction

Another code in use at the time repeated every data bit several times in order to ensure that it got through. For instance, if the data bit to be sent was a 1, an n=3 repetition code would send "111". If the three bits received were not identical, an error occurred. If the channel is clean enough, most of the time only one bit will change in each triple. Therefore, 001, 010, and 100 each correspond to a 0 bit, while 110, 101, and 011 correspond to a 1 bit, as though the bits counted as "votes" towards what the original bit was. A code with this ability to reconstruct the original message in the presence of errors is known as an error-correcting code.

Such codes cannot correctly repair all errors, however. In our example, if the channel flipped two bits and the receiver got "001", the system would detect the error, but conclude that the original bit was 0, which is incorrect. If we increase the number of times we duplicate each bit to four, we can detect all two-bit errors but can't correct them (the votes "tie"); at five, we can correct all two-bit errors, but not detect all three-bit errors.

Moreover, the repetition code is extremely inefficient, reducing throughput by three times in our original case, and the efficiency drops drastically as we increase the number of times each bit is duplicated in order to detect and correct more errors.

Hamming codes

If more error-correcting bits are included with a message, and if those bits can be arranged such that different errored bits produce different error results, then bad bits could be identified. In a 7-bit message, there are seven possible single bit errors, so three error control bits could potentially specify not only that an error occurred but also which bit caused the error.

Hamming studied the existing coding schemes, including two-of-five, and generalized their concepts. To start with he developed a nomenclature to describe the system, including the number of data bits and error-correction bits in a block. For instance, parity includes a single bit for any data word, so assuming ASCII words with 7-bits, Hamming described this as an (8,7) code, with eight bits in total, of which 7 are data. The repetition example would be (3,1), following the same logic. The information rate is the second number divided by the first, for our repetition example, 1/3.

Hamming also noticed the problems with flipping two or more bits, and described this as the "distance" (it is now called the Hamming distance, after him.) Parity has a distance of 1, as any two bit flips will be invisible. The same is true of the (3,1) repetition, as two bits being flipped in the same triple lead to an error. A (4,1) repetition (each bit is repeated four times) has a distance of 2, so flipping two bits in the same group will no longer go undiscovered.

Hamming was interested in two problems at once; increasing the distance as much as possible, while at the same time increasing the information rate as much as possible. During the 1940s he developed several encoding schemes that were dramatic improvements on existing codes. Key to all of his systems was to have the parity bits overlap, such that they managed to check each other as well as the data.

Hamming (7,4)

Today, Hamming Code really refers to a specific (7,4) code Hamming introduced in 1950. Hamming Code adds three additional check bits to every four data bits of the message. Hamming's (7,4) algorithm can correct any single-bit error, and detect all two-bit errors. Since the medium would have to be uselessly noisy for 2 out of 7 bits to be lost, Hamming's (7,4) is generally lossless.

The algorithm is simple:

1. All bit positions that are powers of two are used as parity bits. (positions 1, 2, 4, 8, 16, 32, 64, etc.)
2. All other bit positions are for the data to be encoded. (positions 3, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 17, etc.)
3. Each parity bit calculates the parity for some of the bits in the code word. The position of the parity bit determines the sequence of bits that it alternately checks and skips.
   • Position 1: check 1 bit, skip 1 bit, check 1 bit, skip 1 bit, etc.
   • Position 2: skip 1 bit, check 2 bits, skip 2 bits, check 2 bits, skip 2 bits, etc.
   • Position 4: skip 3 bits, check 4 bits, skip 4 bits, check 4 bits, skip 4 bits, etc.
   • Position 8: skip 7 bits, check 8 bits, skip 8 bits, check 8 bits, skip 8 bits, etc.
   • Position 16: skip 15 bits, check 16 bits, skip 16 bits, check 16 bits, skip 16 bits, etc.
   • Position 32: skip 31 bits, check 32 bits, skip 32 bits, check 32 bits, skip 32 bits, etc.
   • And so on.

Example

Consider a data word "1001101". Hamming code inserts bits at the 1, 2, 4 and 8 positions, so the word is expanded to "__1_001_101". Following the algorithm, the first parity bit adds up 1, 3, 5 etc., which in this message returns a 0. Continuing the algorithm, we get 1, 1 and 0 for the remaining parity bits. The resulting encoded message is "01110010101".

The clever part of Hamming Code is that examining which parity bits are "wrong" allows us to determine which bit was modified, even if that bit is itself a check bit! For instance, let's say you received "01110010111", which differs from the original "01110010101" in only the 10th bit. Then, both bits 2 and 8 appear to have the wrong parity value — both of these bits are checking position 10 — but the other parity bits are fine. By adding 2 + 8 = 10, we see which bit must be flipped to repair the sequence.

Similarly, if the parity bit 4 were flipped, resulting in "01100010101", then certainly that parity bit would appear to be wrong, but all the other parity bits remain correct, since the parity bits are never based on one another. This indicates that the error is in position 4.

Finally, suppose two bits change, at positions x and y. If x and y have the same bit at the 2^k position in their binary representations, then the parity bit corresponding to that position checks them both, and so will remain the same. However, some parity bit must be altered, because x ≠ y, and so some two corresponding bits differ in x and y. Thus, the Hamming code detects all two bit errors — however, it cannot distinguish them from 1-bit errors.

See also

• Hamming distance
• Golay code
• CGI script for calculating hamming distances (from R. Tervo, UNB, Canada)