How does the smoothness of the filter bank functions affect visibility of artifacts in an image coder?
Smooth basis functions lead to reconstructed images with less visible artifacts, since sharp discontinuities are more visible than smooth features.
Properties of the HVS
- The HVS is more sensitive to overall intensity changes (luminance), than to color changes (chrominance). Usually most of the scene information is contained in its luminance rather than its color.
- The bandwidth of the HVS for luminance components is typically around 5 times as wider than chrominance.
- The contrast sensitivity for luminance is also around 3 times better than red-green sensitivity, and 6 times better than blue-yellow.
- The luminance sensitivity also drops off at low spatial frequencies if there is no flicker.
- Activity masking occurs, such that in the presence of high image activity (i.e. texture), it is more difficult to notice coding distortions than in smoother areas of low activity.
RGB to YUV conversion
- Y = 0.3R + 0.6G + 0.1B
- U = 0.5 * (B - Y)
- V = 0.625 * (R - Y)
Explain why YUV is a good system for image compression.
- Luminance sensitivity drops off at low spatial frequencies.
- The maximum luminance bandwidth is much greater than the chrominance bandwidth, which drops off a spatial freq. of 1 cycle per degree. This means that U and V can be sampled at a lower rate than Y.
- The maximum chrominance sensitivity is much lower than the max luminance contrast sensitivity. This means that U and V channels can be quantized more coarsely.
Encoder / Decoder Process
- Energy Compression
- Quantization
- Entropy Coding
- Entropy Decoding
- Inverse Quantization
- Reconstruction
Encoder / Decoder Process
- Energy Compression
Energy compression compresses most of the energy in the input image x into a smaller proportion of coefficients in output image y.
Encoder / Decoder Process
- Quantization
The quantizer represents y by using integers q, using some predefined quantization strategy. Ideally, a high proportion of elements of q should be zero.
Encoder / Decoder Process
- Entropy Coding
The entropy coder converts quantized bits q into binary bits d, so that all information in q is retained in d, while minimizing the number of bits in d.
Encoder / Decoder Process
- Entropy Decoding
The entropy decoder recovers q exactly from the binary data-stream d.
Encoder / Decoder Process
- Inverse Quantization
The inverse quantizer generates an estimate for y (y_hat) from q, such that the mean quantization error energy of (y- y_hat) is minimized.
Encoder / Decoder Process
- Reconstruction
The reconstruction block converts y_hat back into x_hat, such that if y_hat = y, then x_hat = x (perfect reconstruction).
How can the result of the Haar transform be reordered into four subimages? Discuss the distribution of energies in these subimages and why this is helpful for image compression.
- To apply the Haar transform to a complete image, we group the pixels into 2x2 blocks and apply the transform to each block – we then group all top left components into a subimage, all top right components into a subimage, and so on to give 4 subimages.
- Most of the energy (generally over 90%) is contained in the lo-lo (top left components) subimage and similarly, lo-lo typically contains over 50% of the entropy.
- Since this top left subimage is the result of lowpass filtering in both directions, it has similar characteristics to the original image (except that it is smaller). Image compression techniques can therefore usefully be applied to this lo-lo subimage.
Image characteristics of Haar transform subimages
- Top-left: lowpass filter of each block represented by half sum of pixels in block. Subimage will be slightly blurred version of the original image, scaled in amplitude by 2, but smaller in size.
- Top-right: vertical lowpass and horizontal highpass filter, that picks up vertical edges in the image.
- Bottom-left: horizontal lowpass and vertical highpass filter, that picks up horizontal edges in the image.
- Bottom-right: high pass filter measures diagonal curvature of block. Sub-bands will pick up corners and textures in the image.
Describe the nature of blocking artifacts produced by the Haar transform.
- When sub-bands are quantized severely, the coefficients from finer levels are often set to zero, so the reconstructed image is largely made up of basis functions from the low-pass sub-bands.
- At level 1, the Haar low-pass basis functions are just 2x2 blocks of pixels, and it increases at coarser levels to 4x4, 8x8, etc.
- Hence, the final image is made up of differently sized blocky patterns, known as ‘blocking’ artefacts in the image.
How can the Haar transform be extended to become a wavelet transform that can make the coding artifacts less visible?
- To avoid the ‘blocky’ artifacts, we must modify these Haar filters so that they have smoother responses. Wavelets allow us to do this.
- Using the concept of a two-band filter bank with perfect reconstruction inverse filters, it is possible to design filters which have much better smoothness than Haar filters by forcing them to have multiple zeros at z = -1 in the z-plane.
- This makes the filters more complex but gives quantization artifacts which are less visible because the lowpass basis functions have smooth boundaries, rather than the sharp edges of the Haar functions.
Classes of Image Transform
- Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT) used in compression/reconstruction blocks of typical coding scheme.
- DCT is a block transformation that applies an N x N block transform to non-overlapping blocks of input image X. It is simple and fast to compute, but suffer from blocking artefacts due to significant discontinuities in the boundaries of the blocks.
- DWT is a filtering based transformation that uses convolution to allow basis functions to overlap in a smoothly decaying way, largely avoiding blocking artefacts. Also localizes high frequency artefacts better as the high-freq basis functions are smaller than the low-freq.
- Much more computationally complex than the DCT though.
