Huffman coding is a lossless data compression algorithm. The idea is to assign variable-length code to input characters, lengths of the assigned code are based on the frequencies of corresponding characters. The most frequent character gets the smallest code and the least frequent character gets the largest code.

The variable-length code assigned to input characters are Prefix Codes, meaning the code (bit sequences) is assigned in such a way that the code assigned to one character is not prefix of code assigned to any other character. This is how Huffman Coding makes sure that there is no ambiguity when decoding the generated bit stream.

Let us understand prefix codes with a counter example. Imagine there are four characters, a, b, c and d, and their corresponding variable length codes are 00, 01, 0 and 1. This coding leads to ambiguity because code assigned to c is the prefix of the code assigned to a and b. If the compressed bit stream is 0001, the decompressed output may be “cccd” or “ccb” or “acd” or “ab”.

See this for applications of Huffman Coding.

There are mainly two major parts in Huffman Coding

1) Build a Huffman Tree from input characters.

2) Traverse the Huffman Tree and assign code to the characters.

Steps to build a Huffman Tree

Input is an array of unique characters along with their frequency of occurrences and output is the Huffman Tree.

1. Create a leaf node for each unique character and build a min heap of all leaf nodes (Min Heap is used as a priority queue. The value of frequency field is used to compare two nodes in min heap. Initially, the least frequent character is at root)
2. Extract two nodes with the minimum frequency from the min heap.
3. Create a new internal node with frequency equal to the sum of the two nodes frequencies. Make the first extracted node as its left child and the other extracted node as its right child. Add this node to the min heap.
4. Repeat steps #2 and #3 until the heap contains only one node. The remaining node is the root node and the tree is complete.

Let’s clarify this algorithm with an example:

Step 1. Build a min heap that contains 6 nodes where each node represents the root of a tree with single node.

Step 2 Extract two minimum frequency nodes from min heap. Add a new internal node with the frequency 5 + 9 = 14.

Now min heap contains 5 nodes where 4 nodes are roots of trees with a single element each, and one heap node is root of tree with 3 elements

Step 3: Extract two minimum frequency nodes from the heap. Add a new internal node with frequency 12 + 13 = 25

Now min heap contains 4 nodes where 2 nodes are the roots of trees with a single element each, and the two heap nodes are the root of the tree with more than one nodes.

Step 4: Extract two minimum frequency nodes. Add a new internal node with frequency 14 + 16 = 30

Now min heap contains 3 nodes.

Step 5: Extract two minimum frequency nodes. Add a new internal node with frequency 25 + 30 = 55

Now min heap contains 2 nodes.

Step 6: Extract two minimum frequency nodes. Add a new internal node with frequency 45 + 55 = 100

Now min heap contains only one node.

Since the heap contains only one node, the algorithm stops here.

Steps to print codes from Huffman Tree:

Traverse the tree formed starting from the root. Maintain an auxiliary array. While moving to the left child, write 0 to the array. While moving to the right child, write 1 to the array. Print the array when a leaf node is encountered.

The codes are as follows:

Now finally, let’s put the algorithm into code:

// The main function that builds Huffman tree
struct MinHeapNode* buildHuffmanTree(char data[], int freq[], int size)
{
   struct MinHeapNode *left, *right, *top;

   // Step 1: Create a min heap of capacity equal to size.  Initially, there are
   // modes equal to size.
   struct MinHeap* minHeap = createAndBuildMinHeap(data, freq, size);

   // Iterate while size of heap doesn't become 1
   while (!isSizeOne(minHeap))
   {
       // Step 2: Extract the two minimum freq items from min heap
       left = extractMin(minHeap);
       right = extractMin(minHeap);

       // Step 3:  Create a new internal node with frequency equal to the
       // sum of the two nodes frequencies. Make the two extracted node as
       // left and right children of this new node. Add this node to the min heap
       // '$' is a special value for internal nodes, not used
       top = newNode('$', left->freq + right->freq);
       top->left = left;
       top->right = right;
       insertMinHeap(minHeap, top);
   }

   // Step 4: The remaining node is the root node and the tree is complete.
   return extractMin(minHeap);
}

// Prints huffman codes from the root of Huffman Tree.  It uses arr[] to
// store codes
void printCodes(struct MinHeapNode* root, int arr[], int top)
{
   // Assign 0 to left edge and recur
   if (root->left)
   {
       arr[top] = 0;
       printCodes(root->left, arr, top + 1);
   }

   // Assign 1 to right edge and recur
   if (root->right)
   {
       arr[top] = 1;
       printCodes(root->right, arr, top + 1);
   }

   // If this is a leaf node, then it contains one of the input
   // characters, print the character and its code from arr[]
   if (isLeaf(root))
   {
       printf("%c: ", root->data);
       printArr(arr, top);
   }
}

// The main function that builds a Huffman Tree and print codes by traversing
// the built Huffman Tree
void HuffmanCodes(char data[], int freq[], int size)
{
  //  Construct Huffman Tree
  struct MinHeapNode* root = buildHuffmanTree(data, freq, size);

  // Print Huffman codes using the Huffman tree built above
  int arr[MAX_TREE_HT], top = 0;
  printCodes(root, arr, top);
}

Time complexity

O(nlogn) where n is the number of unique characters. If there are n nodes, extractMin() is called 2*(n – 1) times. extractMin() takes O(logn) time as it calles minHeapify(). So, overall the complexity is O(nlogn).

If the input array is sorted, there exists a linear time algorithm.