Multimedia Storage Techniques

Anandi Giridharan Electrical Communication Engineering, Indian Institute of Science, Bangalore 560012, India
E0262 MIS Multimedia Storage Techniques

Media and Storage Requirements

Audio, video and image require vast amount of data for their representation. There are 3 main reasons for compression: Large storage Doesn't allow playing back uncompressed multimedia data. Network bandwidth Storage requirements for multimedia application such as encyclopedia is 500,000 pages of text (2 KB per page)= total 1 GB 3000 color picture= total 3 GB.

E0262 MIS Multimedia Storage Techniques

How much storage is required for different data types Text : 8x8 pixel / char Vector Graphics : A typical still image is composed of 500 lines. Each line is defined by horizontal and vertical position and an 8 bit attribute field. Horizontal axis is represented by 10 bits [log2(640)] and vertical axis by 9 bits [log2(480)]. Uncompressed audio : If a sampling rate of 8kHz is used, and data is quantized at 8 bits/sample then 64 kbits/sec is required. CD quality stereo audio : Sampling rate used is 44.1 kHz, hence at 16 bits/sample 44.1 x 16 = 705.6 kbits/sec is required. Interaction between human users via multimedia information is involved for example, requires that the end to end delay should not exceed 150 ms. Hence compression method should be selected
E0262 MIS Multimedia Storage Techniques

DIGITAL VIDEO
Digital video is just a digital representation of the analogue video signal. Unlike analogue video that degrades in quality from one generation to the next, digital video does not. Each generation of digital video is identical to the parent. With digital video, four factors have to be kept in mind. # Frame rate # Spatial Resolution # Colour Resolution # Image Quality

E0262 MIS Multimedia Storage Techniques

Frame Rate
The standard for displaying any type of non-film video is 30 frames per second (film is 24 frames per second). Additionally these frames are split in half (odd lines and even lines), to form what are called fields. When a television set displays its analogue video signal, it displays the odd lines (the odd field) first. Then is displays the even lines (the even field). Each pair forms a frame and there are 60 of these fields displayed every second (or 30 frames per second). This is referred to as interlaced video.
E0262 MIS Multimedia Storage Techniques

Fragment of the "matrix" sequence (2 frames)

After processing the fragment on the left by the FRC filter the frame rate increased 4 times

E0262 MIS Multimedia Storage Techniques

A computer monitor, however, uses a process called "progressive scan" to update the screen. With this method, the screen is not broken into fields. Instead, the computer displays each line in sequence, from top to bottom. This entire frame is displayed 30 times every second. This is often called non-interlaced video.

E0262 MIS Multimedia Storage Techniques

Colour Resolution This second factor is a bit more complex. Colour resolution refers to the number of colours displayed on the screen at one time. Computers deal with colour in an RGB (red-green-blue) format, while video uses a variety of formats. One of the most common video formats is called YUV.

E0262 MIS Multimedia Storage Techniques

This test table was used to estimate the color resolution. First we determine the border when one of the colors on the resolution chart disappears, and color sharpness is found on the scale on the right.

E0262 MIS Multimedia Storage Techniques

Spatial Resolution
The third factor is spatial resolution - or in other words, "How big is the picture?". Since PC and Macintosh computers generally have resolutions in excess of 640 by 480, The National Television Standards Committee ( NTSC) standard used in North America and Japanese Television uses a 768 by 484 display. The Phase Alternative system (PAL) standard for European television is slightly larger at 768 by 576.

E0262 MIS Multimedia Storage Techniques

Spatial resolution is a parameter that shows how many pixels are used to represent a real object in digital form. Fig. 2 shows the same color image represented by different spatial resolution. Left flower have a much better resolution that right one

E0262 MIS Multimedia Storage Techniques

Image quality

The final objective is video that looks acceptable for your application. For some this may be 1/4 screen, 15 frames per second (fps), at 8 bits per pixel. Other require a full screen (768 by 484), full frame rate video, at 24 bits per pixel (16.7 million colours).

E0262 MIS Multimedia Storage Techniques

E0262 MIS Multimedia Storage Techniques

Need For Compression

How the four factors mentioned above (frame rate, colour resolution, spatial resolution and image quality) affect your selection. With more colours, higher resolution, faster frame rates and better quality, you will need more computer power and will require more storage space for your video. 24-bit colour video, with 640 by 480 resolution, at 30 fps, requires an astonishing 26 megabytes of data per second! Not only does this surpass the capabilities of the many home computer systems, but also overburdens existing storage systems.
E0262 MIS Multimedia Storage Techniques

640 horizontal resolution X 480 vertical resolution = 307, 200 total pixels per frame X 3 bytes per pixel = 921, 600 total bytes per frame X 30 frames per second = 27, 648, 000 total bytes per second / 1, 048 576 to convert to megabytes = 26.36 megabytes per second! Calculation to show space required for video is excessive
E0262 MIS Multimedia Storage Techniques

Factors Associated with Compression

The goal of video compression is to massively reduce the amount of data required to store the digital video file, while retaining the quality of the original video # Real-Time versus Non-Real-Time # Symmetrical versus Asymmetrical # Compression Ratios # Lossless versus Lossy # Interframe versus Intraframe # Bit Rate Control

E0262 MIS Multimedia Storage Techniques

Real-Time versus Non-Real-Time

Some compression systems capture, compress to disk, decompress and play back video (30 frames per second) all in real time; there are no delays. Other systems are only capable of capturing some of the 30 frames per second and/or are only capable of playing back some of the frames. Insufficient frame rate is one of the most noticeable video deficiencies. Without a minimum of 24 frames per second, the video will be noticeably jerky. In addition, the missing frames will contain extremely important lip synchronisation data. If the movement of a person's lips is missing due to dropped frames during capture or playback, it is impossible to match the audio correctly with the video.
E0262 MIS Multimedia Storage Techniques

Real time

Non Real time

E0262 MIS Multimedia Storage Techniques

Symmetrical Versus Asymmetrical

This refers to how video images are compressed and decompressed. Symmetrical compression means that if you can play back a sequence of 640 by 480 video at 30 frames per second, then you can also capture, compress and store it at that rate. Asymmetrical compression means just the opposite. The degree of asymmetry is usually expressed as a ratio. A ratio of 150:1 means it takes approximately 150 minutes to compress one minute of video. Asymmetrical compression can sometimes be more elaborate and more efficient for quality and speed at playback because it uses so much more time to compress the video. The two big drawbacks to asymmetrical compression are that it takes a lot longer, and often you must send the source material out to a dedicated compression company for encoding
E0262 MIS Multimedia Storage Techniques

Compression Ratio
The compression ratio relates the numerical representation of the original video in comparison to the compressed video. For example, 200:1 compression ratio means that the original video is represented by the number 200. In comparison, the compressed video is represented by the smaller number, in this case, that is 1. With MPEG, compression ratios of 100:1 are common, with good image quality. Motion JPEG provides ratios ranging from 15:1 to 80:1, although 20:1 is about the maximum for maintaining a good quality image.

E0262 MIS Multimedia Storage Techniques

Lossless Versus Lossy

The loss factor determines whether there is a loss of quality between the original image and the image after it has been compressed and played back (decompressed). The more compression, the more likely that quality will be affected. Virtually all compression methods lose some quality when you compress the data the only lossless algorithms are for still image compression. Lossless compression can usually only compress a photo-realistic image by a factor of 2:1.
E0262 MIS Multimedia Storage Techniques

Interframe Versus Intraframe

One of the most powerful techniques for compressing video is interframe compression. Interframe compression uses one or more earlier or later frames in a sequence to compress the current frame, while intraframe compression uses only the current frame, which is effectively image compression.

Since interframe compression copies data from one frame to another, if the original frame is simply cut out (or lost in transmission), the following frames cannot be reconstructed properly.

Making 'cuts' in intraframe-compressed video is almost as easy as editing uncompressed video one finds the beginning and ending of each frame, and simply copies bit-for-bit each frame that one wants to keep, and discards the frames one doesn't want. Another difference between intraframe and interframe compression is that with intraframe systems, each frame uses a similar amount of data.
E0262 MIS Multimedia Storage Techniques

Bit Rate Control

A good compression system should allow the user to instruct the compression hardware and software which parameters are most important. In some applications, frame rate may be of paramount importance, while frame size is not. In other applications, you may not care if the frame rate drops below 15 frames per second, but the quality of those frames must be of very good.

E0262 MIS Multimedia Storage Techniques

MPEG (Moving Picture Expert Group)

MPEG was set standard for Audio and Video compression and transmission

MPEG-1 is a standard for lossy compression of video and audio. It is designed to compress VHS-quality raw digital video and CD audio down to 1.5 Mbit/s (26:1 and 6:1 compression ratios respectively) without excessive quality loss, making Video CDs, digital cable/satellite TV and digital audio broadcasting (DAB) possible. MPEG-1 has become the most widely compatible lossy audio/video format in the world, and is used in a large number of products and technologies. The best-known part of the MPEG-1 standard is the MP3 audio format . The standard consists of the following five Parts: 1. Systems (storage and synchronization of video, audio, and other data together) 2. Video (compressed video content) 3. Audio (compressed audio content) 4. Conformance testing & 5. reference software
E0262 MIS Multimedia Storage Techniques

MPEG-2
was designed for coding interlaced images at transmission rates above 4 million bits per second. MPEG 2 can be used on HD-DVD and blue ray disc. handles 5 audio channels, Covers wider range of frame sizes (HDTV). Provides resolution 720*480 and 1280*720 at 60 fps with full CD quality audio used by DVD-ROM. MPEG-2 can compress 2 hours video into a few GHz. MPEG-2 is used for digital TV broadcast and DVD. An MPEG-2 is designed to offer higher quality than MPEG-1, at a higher bandwidth (between 4 and 10 Mbit/s). The scheme is very similar to MPEG-1, and scalable.
E0262 MIS Multimedia Storage Techniques

MPEG-3
Designed to handle HDTV signal in range 20 to 40 Mbits/sec. HDTV-resolution is 1920* 1080*30 Hz But MPEG-2 was fully capable of handling HDTV so MPEG -3 is no longer mentioned.

E0262 MIS Multimedia Storage Techniques

MPEG-4
MPEG-4 is a collection of methods defining compression of audio and visual (AV) digital data. MPEG-4 absorbs many of the features of MPEG-1 and MPEG-2 and other related standards, Wavelength band MPEG-4 files are smaller than JPEG. so they transmit video and images over narrower bandwidth and can mix video with text graphics and 2D and 3D animation layers. MPEG-4 provides a series of technolgies for developers for various service providers and end users. SP use for data transparency Helps end users with wide range of interaction with animated objects. MPEG-4 multiplexes and synchronizes data . Interaction with audio visual scene.
E0262 MIS Multimedia Storage Techniques

MPEG-7
MPEG-7 is a content representation standard for information search. It is also titled Multimedia Content Description Interface. It will define the manner in which audiovisual materials can be coded and classified so the materials can be easily located using search engines just as search engines are used to locate text-based information . Music, art, line drawings, photos, and videos are examples of the kinds of materials that will become searchable based on descriptive language defined by MPEG-7. * Provide a fast and efficient searching, filtering and content identification method. * Describe main issues about the content (low-level characteristics, structure, models, collections, etc.). * Index a big range of applications. * Audiovisual information that MPEG-7 deals is : Audio, voice, video, images, graphs and 3D models * Inform about how objects are combined in a scene. * Independence between description and the information itself.
E0262 MIS Multimedia Storage Techniques

MPEG-7 applications
* Digital library: Image/video catalogue, musical dictionary. * Multimedia directory services: e.g. yellow pages. * Broadcast media selection: Radio channel, TV channel. * Multimedia editing: Personalized electronic news service, media authoring. * Security services: Traffic control, production chains... * E-business: Searching process of products. * Cultural services: Art-galleries, museums... * Educational applications. * Biomedical applications.

E0262 MIS Multimedia Storage Techniques

E0262 MIS Multimedia Storage Techniques

TIFF(Tagged Image File Format)

Tagged Image File Format (abbreviated TIFF) is a file format for storing images, including photographs and line art TIFF is a flexible, adaptable file format for handling images and data within a single file, by including the header tag. TIFF file using lossless compression (or none) may be edited and resaved without losing image quality; The TIFF format is the standard in document imaging and document management systems The TIFF format can save multi-page documents to a single TIFF file rather than a series of files for each scanned page. Multi-page support and 2D compression of bitonal images.

E0262 MIS Multimedia Storage Techniques

TIFF format

E0262 MIS Multimedia Storage Techniques

JPEG
commonly used method of compression for photographic images. JPEG typically achieves 10:1 compression with little perceptible loss in image quality. JPEG compression is used in a number of image file formats. JPEG/Exif is the most common image format used by digital cameras and other photographic image capture devices; it is the most common format for storing and transmitting photographic images on the World Wide Web. These format variations are often not distinguished, and are simply called JPEG.
E0262 MIS Multimedia Storage Techniques

* Format independent of frame size * different data rates * synchronization of different streams (audio, video) * SW coding or HW coding * open systems should be allowed. During the retrieval applications for human -> database interaction such as # Fast forward and backward data retrieval # Random access # Decomposition of images, video or audio independently should be possible.

E0262 MIS Multimedia Storage Techniques

GIF Graphics Interchange Format

The Graphics Interchange Format (GIF) is a bitmap image format The format supports up to 8 bits per pixel, allowing a single image to reference a palette of up to 256 distinct colors chosen from the 24-bit RGB color space. It also supports animations GIF images are compressed using the Lempel-ZivWelch (LZW) lossless data compression technique to reduce the file size without degrading the visual quality.

E0262 MIS Multimedia Storage Techniques

# GIFs can also be used to store low-color sprite data for games. # GIFs can be used for small animations and low-resolution film clips. # In view of the general limitation on the GIF image palette to 256 colors, it is not usually used as a format for digital photography # The PNG format is a popular alternative to GIF images since it uses better compression techniques and does not have a limit of 256 colors, but PNGs do not support animations.

E0262 MIS Multimedia Storage Techniques

Despite its other limitations, fluid 3D animation is possible when using the GIF format, as this animation of Newton's Cradle demonstrates.

E0262 MIS Multimedia Storage Techniques

Other Formats
PNG (Portable Network Graphic): A higher-quality replacement for the GIF format. PNG's compression is among the best that can be had without losing image information. PNG supports three main image types: truecolor, grayscale and palettebased ("8-bit").

PNG doesnot support animation at all. It has greater compression than GIF.

E0262 MIS Multimedia Storage Techniques

E0262 MIS Multimedia Storage Techniques

PDF

PDF (Portable Document Format): provides a convenient way to view and print images at a high resolution PDF lets us to capture and view robust information from application on any computer system.

Multiplatform: PDF are viewable and printable on any platform. Extensible-1800 vendors world wide offer PDF based solutions. Trusted realiable. preserve source file information, text, drawing etc regardless of applications.

Digital sign or password pdfdoc created with acrobat SW, Searchable- Text search features.
E0262 MIS Multimedia Storage Techniques

Structuring Metadata
Metadata schemes (also called schema) are sets of metadata elements designed for a specific purpose, such as describing a particular type of information resource. The definition or meaning of the elements themselves is known as the semantics of the scheme. ASCII Text SGML (Standard Generalized Markup Language) HTML (HyperText Markup Language) XML XHTML (Extensible HyperText Markup Language) MARC (The MAchine Readable Cataloginge)

E0262 MIS Multimedia Storage Techniques