Multimedia Databases

MS
I
Arif Ghafoor
Professor
Electrical and Computer Engineering
Purdue University
 Multimedia Database Laboratory
Facilities at Purdue University
MS
I 
Distributed Multimedia Systems Lab
(ECE)
• Ghafoor, Kashyap, Khan
 Multimedia Systems Lab (CS)
• Aref, Elmagarmid, Prabhakar
 Raid Lab (CS)
• Bhargava
 Video and Image Processing Lab (ECE)
• Delp 2
 On-Going Research Projects in
Multimedia Databases
MS
I 
Multimedia database management
(NSF, Fuji Electric, AT&T)
• Video modeling and management
• Multimedia document management
 Distributed multimedia systems
(NSF, AFRL, IBM, Intel, Siemens)
 High-performance multimedia database
architecture for storage management
(NSF, AT&T) 3
 Example of Event-Based Video
Data (Touch-down Scenario)
MS
I

4
 Research Challenges for Video
Databases
MS
I  Development of video processing techniques
for automatic object identification and motion
tracking
 Development of data models with powerful
semantic expressiveness
 Designing efficient indexing, classification
and event-based searching techniques with
high degree of precision
 Providing suitable video querying
environment
 Addressing scalability
5
 Semantic Video Data Modeling
Complexity
MS
I High
Knowledge-based
Higher-level
Level of Abstraction

Semantics

Iconic-based
Motion Detection
Grouping and
Analysis
Browsing

Video Parsing and

Object Recognition
Segmentation
Low

Coarse
Granularity Fine
6
 An Architecture for Video
Database System
MS Spatio-Temporal Semantics:
I Object Definitions Formal Specification of Inter-Object Movement
(Events/Concepts) Event/Activity/Episode for (Analysis)
Content-Based Retrieval
Intra/Inter-Frame
Analysis
(Motion Analysis)
Spatial-Semantics Semantic Association
of Objects (President, Capitol,...)
(human,building,…) Object Identification
and Tracking
Physical
Image Features Object
Database Sequence of Frames
(indexed)

Object Description Raw Temporal

Image Abstraction
Database

Raw
Frame Video
Database 7
Spatial Abstraction
 Comparison of Different Video
Database Management Projects
MS
I
Model Querying Semantic Modeling Video Processing Fuzziness
Mechanism & Approch Approach Support
Attributes

Zhang (HP) Visual Browsing - video parsing No

Liu (Princeton) Visual Browsing Graphical video parsing No

Tanaka (Tokyo) Visual SQL, text Object-oriented Manual No

only

Weiss (MIT) Algebraic , text Object-oriented Manual No

only

Golshani (ASU) Algebraic , - Automatic No

motion only tracking

Chang (Columbia) Sketch & Image - Auitomatic Moderate

Features tracking

Ghafoor (Purdue) Petri-net ,Text Spatio-temporal Automatic High

and motion logic and Petri- tracking
nets

8
 Sample Multimedia Document
MS
I Exit

The Ruminant Digestive System

Four compartment stomach that:

- utilizes complex carbohydrates
such as cellulose
- utilizes non-protein nitrogen
sources
- utilizes feeds of no nutritive value
for humans

Omasum
Abomasum
Reticulum
Esophagus Rumen
Protozoa
Rumen

Stomach
The rumen, the first compartment of the
Digestive System ruminant stomach, is essentially non-
functional at birth but develops rapidly when
Small Intestine
calves are given solid feeds such as hay and
Salivary Glands grain.
Large Intestine

Multimedia Browsing Graph 9

 Document Representation and
Specification Complexity
MS
I Media Types and Attributes Complexity

HyTime

SGML/XML
Style Complexity

MPEG Word
Interleaf/
PDF Framemaker
y VRML HTML
exit
pl
m
Co
e
t ur
c
t ru 10
S
 End-to-End QoP / QoS Management
MS
I
- Reliability
- Resolution

Specification Meta Data / User Interface - Rate of Presentation

- Display Area
- Temporal Synchronization ( Intra/Inter )

Translation Network OS Database Security

- End-to-End - CPU Throughput - Storage Throughput/ - Intrusion Detection
Delays - Memory Overflow Bandwidth - Access Control
- Jitter Delay and Reliability - Storage Delays
- Bandwidth - Distributed
- Packet Loss Database
Rate Coordination (QoS)

Negotiation Dependency Model Analysis and QoS Adjustment

End-to-End
Run Time End-to-End Resource Allocation and Scheduling
11
Scheduling
 Multimedia Document Management
MS
I 
Quality of presentation requirements
• Resolution, reliability, rate
 Synchronization requirements
• Temporal, spatial, and logical structure
specification
 Media processing requirements
• Coloring, enhancements, dubbing, etc.
 Security attributes
12
 Petri-Net Based Model for Synchronization and
Quality Specification of Multimedia Documents
MS
I Parameter
Duration
Symbol
t
Size s
Deadline p
Allowable Skew a
Display Area A
Rate r
Reliability w
Contents

Text Text Video

Video Image

Audio
Audio

p1 p2 p3 p4 13
 Multimedia Browsing
Environment
MS
I Abomasum
Omasum

Reticulum
Esophagus
Protozoa
Rumen

Stomach
Text

Digestive System Audio

Text Image

Small Intestine Image

Text
Salivary Glands
Text

Video

Large Intestine
Audio

Image

14
 Research Issues for Quality and Performance of
Multimedia Document Management Systems
MS
I  Document modeling
(Dimensionality/Flexibility/Portability)
• Languages (SGML/XML)
• Graphical (OCPN)
 Transformation among models and
heterogeneity management of models
 Development of efficient search and indexing
techniques for multimedia documents using
these models
 Clustering of documents
• Based on contents for fast search
• Controlled access 15
 Comparison of Different
Multimedia Document Models
MS
I

Model Spatial/Temporal Logical Content & Consistency Meta Storage Network

Composition Structuring QoP Checking for Schema Placement Interface
Facility Specification Scheduling Translation Support Support
Facilty Complexity

HyTime/ Both Yes Content Only No Difficult No No

SGML

Object- Both Yes Both No Easy No No

Oriented
(VODAK)

OCPN Both Partial Both Yes Easy Yes Yes

16
 Software Architecture of a Multimedia
Database Management System
MS User To Presentation Device
I
Multimedia
Document Generator
Structuring Module Tool Library

HyTime/SGML/XML Translator
Multimedia Meta-Data
HyTime/SGMLXML Parser -->
Multimedia Query Language

OODBMS and MQL Temporal Synchronization Manager

Distributed Query Processor

Text Video Image Audio

Database Database Database Database 17
 Distributed Multimedia Database
Systems
MS
I

DBMS Audio DBMS Image

Presentation Device

DBMS Audio

Network A Network B

DBMS Audio

DBMS Video
DBMS Text
18
 Research Issues in Distributed
Multimedia Database Management
MS
I 
Distributed Object Management
 Efficient distributed query processing
techniques
 Network caching techniques for
composition of distributed objects
 Adaptive data filtering techniques to
satisfy diverse user QoS requirements

19
Architecture of a Distributed
Multimedia Database Management
MS
I Multimedia Database Client
Visual Tool for
Multimedia Multimedia
Multimedia Document
Database Interface Presentation Subsystem
Generation

API for SBS Network

Integrated Multimedia Information Server

Multimedia Meta Data

API for SBS Network Distributed Query

Management

Management
Directory
Processor
Multimedia Database Server

Database
Meta Media Server
Management API for SBS Network
Data Subsystem
System

Multimedia
...
Database Connectivity

Multimedia
Database Server Database Server
Text Image Video Audio 20
 Planned Experiments
MS
I 
Development of video database
management system ( using database
compute servers and storage system)
 Development of multimedia document
management system
 Development of distributed multimedia
object management system using SBS
architecture
21