JPEG

10/17/2001

Types of Images
We consider here still images. Examples:
Photographs (color or grayscale) Fax (bilevel) Documents (text, handwriting, graphics and photographs)

Application examples
Slide-show graphics on the Internet (catalogs, home shopping, real-estate viewing,). Images must be presented at a variety of resolution, including low resolution for browsing and high resolution for detailed inspection. Medical applications where sophisticated high resolution images (X-rays, CAT scans, etc.) need to be stored, indexed, accessed and transmitted from site to site on demand

10/17/2001 10/17/2001

JPEG Image Coding

JPEG is the standard algorithm for compression of still images It is of reasonably low computational complexity, is capable of producing compressed images of high quality, and can provide both lossless and lossy compression of arbitrarily sized grayscale and color images

JPEG Lossy Codec Scheme

ENCODER
8x8 blocks of pixels DCT Scalar quantization Entropy coder Compressed data

Table specs

Table specs

Compressed data

Entropy decoder

Dequantization

Inverse DCT

8x8 blocks of pixels

Table specs

Table specs

10/17/2001 10/17/2001 10/17/2001

DECODER

JPEG Lossy Coding

The RGB color vectors in the image are first converted into YCrCb values (channels are encoded independently) The image is converted into a series of 8-by-8 pixel blocks, which are then processed in a raster scan sequence from left to right and from top to bottom Each 8x8 block of pixels is spectrally analyzed using DCT (transform coding) and coefficients are quantized After DCT coding and quantization, coefficients are entropy coded

What Do the DCT Coefficients Represent?

The DCT coefs represent the spatial frequency content within a 8x8 image block The (0,0) coefficient contains is called DC coefficient, and is proportional to the average of the 64 image values in the block As we move to the right of the block, the coefficients represent the energy in higher horizontal frequencies As we move down the block, the coefficients represent the energy in higher vertical frequencies

Examples of 8x8 DCTs

Horizontal frequencies
1 1 1

2 2

3 3

4 4

5 5

6 6

6
7

7 7

7
8

8 1 2 3 4 5 6 7 8

8 8 1 2 3 4 5 6 7 8 1

8 1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8

Vertical frequencies

image

DCT

image

DCT

image

DCT

1
1 1

2
2 2

3
3 3

4
4 4

5
5 5

6
6
7

7
7
8

8
1 2 3 4 5 6 7 8

8 1

image

DCT

image
2 3 4 5 6 7

DCT
3 4 5 6

1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 7 7 7 8 8 1 2 3 4 5 6 7 8 8 1 2 3 4 5 6 7 8

10/17/2001 10/17/2001 10/17/2001

8 1 2 3 4 5 6 7 8

image

DCT

image

DCT

DCT Coefficient Quantization

Each DCT coefficient is quantized independently using an uniform quantizer The number of quantization intervals per each DCT channel is chosen independently for each coefficient After quantization, the DC coefficient of a block is encoded as the difference from the DC term of the previous block in the processing order. This is because there is usually strong correlation between the DC coefficients of adjacent blocks

DCT Coefficient Quantization

Rules for choosing quantization intervals:
Low-frequency coefs usually have more power than high-frequency coefs, therefore require more quantization intervals The Human Visual System (HSV) is more sensitive to error in low frequencies. Hence, low-frequency coefs require more quantization intervals The HVS is more sensitive to luminance channels than to chrominance channels. Hence, luminance channels require more quantization intervals than chrominance channels

Note: the quantization table is not standardized!

Example
139 144 150 159 159 161 162 162 144 151 155 161 160 161 162 162 149 153 160 162 161 161 161 161 153 156 163 160 162 161 163 161 155 159 158 160 162 160 162 163 155 156 156 159 155 157 157 158 155 156 156 159 155 157 157 158 155 156 156 159 155 157 157 158
235.6 -22.6 -10.9 -7.1 -0.6 1.8 -1.3 -2.6 -1.0 -12.1 -17.5 -6.2 -9.3 -1.6 -1.9 0.2 -0.8 1.5 -0.2 1.6 -0.4 -0.3 1.6 -3.8 -5.2 -3.2 1.5 1.5 1.6 -0.3 -1.5 -1.8 2.1 -2.9 0.2 0.9 -0.1 -0.8 -0.5 1.9 -1.7 -0.1 -0.9 -0.1 -0.7 1.5 1.7 1.2 -2.7 0.4 -0.6 0.0 0.6 1.0 1.1 -0.6 1.3 -1.2 -0.1 0.3 1.3 -1.0 -0.8 -0.4

Source image sample

16 12 14 14 18 24 49 72 11 12 13 17 22 35 64 92 10 14 16 22 37 55 78 95 16 19 24 29 56 64 87 98 24 26 40 51 68 81 103 112 40 58 57 87 109 104 121 100 51 60 69 80 103 113 120 103 61 55 56 62 77 92 101 99

DCT coefficients
15 -2 -1 0 0 0 0 0 0 -1 -1 0 0 0 0 0 -1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

10/17/2001 10/17/2001 10/17/2001

Quantization table

Quantized coefficients

Entropy Coding of DCT Coefs

The quantized DCT coefficients are first processed in zig-zag order
DC coef

0 2 3 9 10 20 21 35

1 4 8 11 19 22 34 36

5 7 12 18 23 33 37 48

6 13 17 24 32 38 47 49

14 16 25 31 39 46 50 57

15 26 30 40 45 51 56 58

27 29 41 44 52 55 59 62

28 42 43 53 54 60 61 53

AC coefs

Intermediate Representation
Zig-zag ordering of coefficients places lowfrequency (which are more likely to be non-zero) coefficients before high-frequency coefficients (which are more likely to be null after quantization) In this way, the sequence of quantized coefficients is such that there often are long sequences of null values AC coefficients are represented in a intermediate run-length representation which encodes the runs of zero preceding any non-null coefficient

Intermediate Representation (contd)

Each non-zero AC coef is represented in combination with the run-length of the null coefs before it, using a pair of symbols:
Symbol-1 (RUNLENGTH,SIZE) Symbol-2 (AMPLITUDE)

RUNLENGTH is the number of consecutive null coefs before the non-zero symbol, SIZE is the size of the symbol used to encode the non-null symbol AMPLITUDE is the actual (encoded) value

10/17/2001 10/17/2001 10/17/2001

Intermediate Representation More Details

RUNLENGTH must be 15. To represent a run of >15 zeros, can use up to 3 consecutive (15,0) symbol-1 extensions If the last run of zeros contains the last DCT coef, then the special EOB (End-Of-Block) symbol-1 value (0,0) terminates the representation DC coefficients are represented by only one pair:
Symbol-1 (SIZE) Symbol-2 (AMPLITUDE)

Entropy Coding
For both DC and AC coefficients, each symbol-1 is Huffman encoded (variable length prefix code)
Huffman tables are not specified in the standard and must be input to the encoder

Each symbol-2 is encoded with a Variable Length Integer (VLI) code

It is different from a Huffman code because the symbol length is known in advance The coding table is hard-wired in the proposal

Lossless JPEG Mode

It is wholly independent of DCT processing Each sample value is predicted from a combination of nearby sample values, and the prediction residual is entropy coded (Huffman)
Prediction effectively reduces the entropy of the residual, so that small average symbol length can be achieved
C A B X

X can be predicted by a combination of A,B,C

10/17/2001 10/17/2001 10/17/2001

Progressive Encoding Modes

Layering or progressive encoding: an image can be transmitted at a low rate (and a low quality) and then progressively improved by subsequent transmission Convenient for browsing applications, where a low-quality or low-resolution image is adequate browsing Three forms of JPEG progressive encoding:
Spectral selection Successive approximation (SNR scalability) Hierarchical (pyramidal) mode

Progressive Encoding (contd)

Spectral selection
Initial transmission sends low-frequency DCT coefs, followed progressively by the higher frequency coefficients, according to the zig-zag scan Simple to implement but reconstructed images from the early scans are blurred

Successive approximation (SNR scalability)

Only the most significant bits of each coefficient are sent in the first scan, followed by the next most significant bits and so on until all bits have been transmitted

Example of Spectral Selection

1 6 4 5 7 2 3 8
20 40

60

80

100

120

10/17/2001 10/17/2001
50 100 150 200

140 20 40 60 80 100 120 140 160

Progressive Encoding (contd)

Hierarchical (pyramidal) mode
Image can be sent in one of several resolution modes to accommodate different kinds of displays Different resolution modes achieved by filtering and downsampling the image in multiples of two in each direction. The resulting image is interpolated and subtracted from the next level to create a residual The different resolution modes are transmitted together with the residuals

Pyramid Decomposition
50 100

Residuals

150

200

250
20

300

40 60 80 100 120

350

400 50 100 150 200 250 300 350 400 450

140 160 180 200 50 100 150 200

Interpolate by 2x2 Interpolate by 2x2

20 40 60
10 20 30 40 50 60 70 80 90 100 20 40 60 80 100 120

250

Filter + subsample by 2x2

80 100 120 140 160 180

Filter + subsample by 2x2

50 100 150 200

10/17/2001

300

200

350

400 50 100 150 200 250 300 350 400 450

Pyramid Decomposition (contd)

Whats inside the bit-stream:

50

100
20 40 60
10 20

150

80
30

200

40

100 120 140 160 180

50 60 70 80

250

90 100 20 40 60 80 100 120

300

200 50 100 150 200

350

400 50 100 150 200 250 300 350 400 450

Higher resolution residuals

Lowest resolution image

10/17/2001 10/17/2001 10/17/2001

JPEG Performances
Assume the pixels of an arbitrary color image are digitized to 8 bits for luminance and chrominance channels with 4:2:2 color sampling scheme.
Then effectively the source image has 16 bits/pixel

Bits/pixel 2 1.5 0.75 0.50 0.25

Quality Indistinguishable Excellent Very good Good Fair

Compression Ratio 8:1 10:1 21:1 32:1 64:1

Example of Artifacts Due to JPEG Coding

Original (280 KB)

JPEG (40 KB)

References
G.K. Wallace, The JPEG Still Picture Compression Standard, IEEE Transactions on Consumer Electronics, 38(1), Feb. 1992

10/17/2001