Digital Watermarking

2002/10/24 Room 418, NTU Library Building Lecturer: Chun-Hsiang Huang Communication and Multimedia Lab. Department of CSIE, NTU

Outline (I)
l Introduction

to digital watermarking

History and interesting facts Definitions, models, and importance Applications of digital watermarking Properties of digital watermarking

Outline (II)
l

Important image watermarking techniques

LSB watermarking Spread-spectrum watermarking DCT-based image watermarking Watermarking techniques for different purposes
l l

Visible watermarking Fragile watermarking

Discussions and future work

Introduction to Digital Watermarking

What is a watermark?
l

Watermarking is an important mechanism applied to physical objects like bills, papers, garment labels, product packing Physical objects can be watermarked using special dyes and inks or during paper manufacturing.

Characteristics of watermarks
l

The watermark is hidden from view during normal use, only become visible by adopting a special viewing process.
E.g. hold the bill up to light

The watermark carries information about the object in which it is hidden.

E.g. the authenticity of the bill E.g. the trademark of the paper manufacturer

History of watermarking (I)

l

The term watermark was probably originated from the German term wassermarke. Since watermark is of no importance in the creation of the mark, the name is probably given because the marks resemble the effects of water on paper. Papers are invented in China over a thousand years ago. However, the first paper watermark did not appear until 1282, in Italy.

History of watermarking (II)

l By

the 18th century, watermarks on paper in Europe and America had been used as trademarks, to record the manufactured date, or to indicate the size of original sheets. l Watermarks are commonly used on bills nowadays to avoid counterfeiting

What is digital watermarking?

l

Watermarking can also be applied to digital signals!

Images

Video

Audio

IPR related information technologies

Data hiding

Steganography Imperceptible data embedding Non-robust data embedding Visible data embedding

Watermarking Imperceptible watermarking Visible watermarking

Robust data embedding

Fragile watermarking

Robust watermarking

Information hiding
l Data hiding Containing a large range of problem

beyond that of embedding message in content

l

Making the information imperceptible

E.g. watermarking

Keeping the existence of information secret

E.g. anonymous usage of network E.g. hiding portions of database for non-privileged users

Steganography
l

A term derived from the Greek words steganos and graphia (The two words mean covered and writing, respectively)
The art of concealed communication. The very existence of a message is kept secret. E.g. a story from Herodotus
l

Military Messages tatooed on the scalp of a slave

Watermarking v.s. Steganography

l Watermark

messages contain information related the cover work l In steganographic systems, the very existence of the message is kept secret.
If the message tatooed on the slave is the

slave belongs to somebody, then we can regard it as an example of watermarking

Classification of information hiding systems

Cover Work Dependent Message Existence Hidden Existence Known Steganographic Watermarking Non-Steganographic Watermarking Cover Work Independent Message Covert Communication Overt Embedded Communication

Importance of digital watermarking

l

The sudden increase in watermarking interest is most likely due to the increase in concern over copyright protection of content copyright-protected digital contents are easily recorded and distributed due to:

prevalence of high-capacity digital recording devices

the explosive growth in using Internet

Watermarking v.s. cryptography

l

Cryptography is the most common method of protecting digital content and is one of the best developed science. However, encryption cannot help the seller monitor how a legitimate customer handles the content after decryption. Digital watermarking can protect content even after it is decrypted.
Encryption

Decryption

Under Protection

Definitions about digital watermarking

l Digital

watermarking:

The practice of imperceptually alternating a

Work to embed a message about the Work. Related terms

l

Work: a specific copy of some electronic signal, such as a song, a video sequence, or a picture Cover Work: the original un-watermarked work, since it covers (hides) the watermark Watermark: the messages being embedded, indicating some information about the work

A digital watermarking system

Cover Work

Watermark Embedder

Watermarked Work

Watermark Detector

Detected Watermark Message

Watermark Message

Recording, transmissions, or processing

History of digital watermarking

l

The first watermarking example similar to the digital methods nowadays appeared in 1954. The Muzak Corporation filed a patent for watermarking musical Work. An identification Work was inserted in music by intermittently applying a narrow notch filter centered at 1KHz. About 1995, interest in digital watermarking began to mushroom.

The watermarking fever

Annual number of papers published on watermarking

Difference between watermarking and other IPR techniques

l Watermarks

are imperceptible l Watermarks are inseparable from the works in which they are embedded l Watermarks undergo the same transformations as the work

Applications of digital watermarking

identification l Proof of ownership l Broadcast monitoring l Transaction tracking l Content authentication l Copy control l Device control
l Owner

Owner identification (I)

l

Under the U.S. law, although the copyright notice is not required in every distributed copy to protect the rights of copyright holders, the award to the copyright holders whose work is misused will be significantly limited without a copyright notice found on the distributed materials. Traditional textual copyright notices Copyright date owner date owner Copr. date owner

Owner identification (II)

l

Disadvantages for textual copyright notices

Easily removed from a document when it is copied l E.g. the Lena Sjooblom picture (see the next slide) Copyright notices printed on the physical medium are not

copied along with the digital content

l

E.g. the Music CD

Occupying a portion of the image and aesthetically reducing

the value of artworks

l

Since watermarks are imperceptible and inseparable from the work, they are obviously superior to textual copyright notices.

The Lena Phenomenon

Lena is the most common test image in image processing research! l However, the copyright notice of this picture was cropped and ignored.
l

Proof of ownership
copyright notices cannot be used to solve the copyright dispute since they can be easily forged l Registering every work to a central repository is too costly!
http://www.loc.gov/copyright $30 per document

l Textual

Watermarking can be of use!

Broadcast monitoring (I)

l

TV or radio advertisements should be monitored to prevent airtime overbooking! In 1997, a scandal broke out in Japan. Advertisers are paying for thousands of commercials that were never aired!

Broadcast monitoring
By human watchers Passive monitoring Active monitoring

Broadcast monitoring (II)

l Passive

monitoring

Use computers to monitor received signal

and compares with a database of known contents Disadvantages

l l l

Comparing is not trivial Signal degraded due to broadcasting Management and maintenance of the database is quite expensive

Broadcast monitoring (III)

l

Active monitoring
Simpler to implement Identification information can be directly decoded

reliably E.g.
l

close captions on VBI or file headers

Watermarking is an obvious alternative method of

hiding identification information

l l

Existing within the content Completely compatible with the equipments

Transaction tracking
l l

Watermarks recording the recipient in each legal sale or distribution of the work. If the work is misused (leaked to the press or illegally distributed), the owner could find out who is the traitor. Visible watermarking is often adopted in this application, but Invisible watermark is even better

The defunct DiVX DVD Player

l The

DIVX Corporation sold a enhanced DVD player that implements a pay-perview model. l Each player will place a unique watermark in the video disk it played. l Once the video disk is recorded and sold, the adversary can be tracked!

Copy control (I)

l

Encryption is the first and strongest line of defense against illegal copy
Overcome an encryption mechanism
l

Decrypt a copy without a valid key

Theoretically infeasible for a well designed system

Obtain a valid key

Reverse-engineering hardware or software E.g. the DeCSS program against the CSS protecting DVD

Legally obtain a key and pirate the decrypted content

The central weakness of cryptographic protection! The content must be decrypted before it is used, but all protection is lost once decrypted!

Copy control (II)

l Watermarking

in copy control

Combining every content recorder with a

watermark detector When a copy-prohibit watermark is detected, the recording device will refuse to copy The system has been envisioned by CPTWG and SDMI to protect DVD and audio

Copy control (III)

l Problems

of adopting watermarking module in recording devices

Increasing cost Reducing the value of devices

l Solution
Include the requirement for a watermark

detector in the patent license of CSS instead of enforcing by law

Keep honest people honest

Playback control by encryption Copy control by watermarking

Legal, encrypted copy

Compliant player

Compliant recorder

Non-Compliant player Illegal, decrypted copy

Compliant recorder

Device control
l Copy

control belongs to a broader category - device control l Other applications of device control
Automatically turning on/off functions

related to special contents

l

E.g Including watermark to skip advertisements

Action toys interactive with the TV program Digimarcs MediaBridge

Properties of digital watermarking

l

Correct detection result

Embedding effectiveness False-alarm rate

l l

Fidelity (perceptual similarity) Resisting distortions

Robustness Security

l l l

Data payload (capacity) Blind/informed watermarking Cost

Effectiveness
l Effectiveness

of a watermarking system

The probability of detection after embedding A 100% effectiveness is desirable, but it is

often not the case due to other conflict requirements, such as perceptual similarity
l

E.g. watermarking system for a stock photo house

False-alarm rate
l

Detection of watermark in a work that do not actually contain one

The number of false positives occur in a given

number of runs of watermark detector

l

The false alarm rate of the watermarking system used in DVD recorder should be lower than 1/1012
E.g. a false alarm occurred in a world-series

baseball game

Fidelity (perceptual similarity)

l The fidelity of the watermarking system The perceptual similarity between the

original and the watermarked version of the cover work It is the similarity at the point at which the watermarked content is provided to the customer that counts
l

E.g. NTSC video or AM radio has different perceptual similarity requirements from the HDTV or DVD video and audio

Problems to determine the fidelity

l Commonly used image 1 N MSE: ( c[i ] - c' [i ]) 2 N i =1 N
SNR:

similarity index

( c [ i ] - c ' [ i ])
N 2

i=1

c[i] l Finding a quality index completely reflecting the characteristics of the human perceptual model is difficult
i=1

Robustness (I)
l The

ability to detect the watermark after common signal processing operations

Common images distortions l spatial filtering, lossy compression, printing/scanning, geometric distortions Common video distortions l Changes in frame rate, recording to tape Common audio distortions l temporal filtering, recording on audio tape

Robustness (II)
l Not

all watermarking applications require robustness to all possible signal processing operations. l There is a special class of watermarking techniques where robustness is undesirable
The fragile watermarking

Security
l

The ability to resist hostile attacks

Unauthorized removal
l l l

Eliminating attacks Masking attacks Collusion attacks Embed forgery watermarks into works that should not contain watermarks E.g. fragile watermarks for Authentication

Unauthorized embedding
l

Unauthorized detection

Data capacity
l The

number of bits a watermarking scheme encodes within a unit of time or within a work. l Different applications require different data capacities, e.g.
4-8 bits for a 5-minutes video of copy

control Longer messages for broadcast monitoring

Blind/informed detection
l Informed watermarking schemes The detector requires access to the un-

watermarked original
l

E.g. transaction tracking,

l Blind watermarking schemes Detectors do not require any information

related to the original

l l

E.g.DVD copy control module E.g. An automatic image IPR checking robot

Multiple watermarks
l In

certain cases, more than one watermarks are needed.

E.g. American copyright grants the right of

TV viewers to make a single copy of broadcast programs for time-shift watch. But further copies is not allowed .
l

Adding two watermarks instead of alternating the original watermark to avoid the risk caused by easily changing watermarks

Cost
l The

costs in deploying watermark embedders and detectors depends on the scenario and the business model.
Real-time constraint l Broadcast monitoring v.s. proof of copyright Embedder/detector constraint l Copy protection v.s. transaction tracking (DIV-X)

Watermarking techniques in current standards

l

The CPTWG (Copy Protection Technical Working Group) tested watermarking systems for protection of video on DVD disks. The SDMI (Secure Digital Music Initiative) made watermarking a core component in their system for music protection. Two projects sponsored by the European Union, VIVA and Talisman, tested watermarking for broadcast monitoring. The ISO (International Organization for Standardization) took an interest in the context of designing advanced MPEG standards. (MPEG-21)

Companies with watermarking products bundled its watermarking system with Adobes Photoshop l Technology from the Verance Corporation was adopted into the first phase of SDMI and used by some Internet music distributors
l Digimarc

Watermarking for different media types watermarking l Audio watermarking l Image watermarking
LSB based scheme Spread spectrum scheme DCT-based scheme

l Text

l Video

watermarking l 3D-mesh watermarking

Watermarking for text

l l l

Line-Shift Coding Word-Shift Coding Feature coding

Watermarking for images & videos

l

Watermarking in Watermarking in Watermarking with

spatial domain transform domain raw data compressed data random number visually recognizable pattern

An generic image watermarking system

Distortions Embedding
Image processing Compression ...

Watermark

Comparison

Requirements
Invisibility Robustness No ambiguity

Watermark

Extraction

LSB Flipping Method

0 1 2 Bit 1 Bit 0 Watermark 1 Watermark 2 Bit 0 Bit 2 Bit 1 Bit 2 Bit 3 Bit 3 3 4 5 0 1 2 3 4 5 0
55 120 121 130 140 158 159 195

1
73 123 123 123 133 142 178

2
71 71 70 67 120 123 122 150

3
123 72 68 68 72 123 112

4
123 147 73 70 70 69 67

5
205 199 123 117 71 70 70

0 1 2 3 4 5

l l

Generate the random walk sequence for each watermark (e.g.. 00112) Force the LSB to match the watermark bit
m

This works will not survive any modification

Spread Spectrum Method

the watermark W = w1,...,wn each wi is chosen according to zero-mean Gaussian Distribution the image X is transform by full-frame DCT n highest magnitude coefficients (except DC) are chosen: y1,...,yn Embedding: yi = yi + a wi Extracting: wi = (yi* - yi) / a similarity = correlation (W,W *)

Spread Spectrum Method (cont.)

Original Image FFT/DCT Original Image FFT/DCT Received Image

Watermark

Determine Perceptually Significant Regions

Original Watermark

Extracted Watermark

Inverse FFT/DCT Watermarked Image Similar

Spread Spectrum Method (cont.)

l

Watermark detector

Watermark detector response to 1000 randomly generated watermarks

Watermarking for Audio

Phase Coding Inserting the watermark by modifying the phase of each frequency component l Spread Spectrum Method The watermark code is spread over the available frequency band, and then attenuated and added as additive random noise l Perceptual Method The watermark is generated by filtering a PN-sequence with a filter that approximates the frequency masking characteristics of HAS Weighting the watermark in the time domain to account for temporal masking
l

Watermarking for Audio (cont.)

l

Watermark generator
Audio Signal Hanning Window FFT Frequency Masking Information Scale Factor PN sequence M(w) w Quantization w+ original signal

Extract Envelop

Watermarking for Audio (cont.)

l

Watermark detection

Watermarking for Polygonal Models

l 3D models watermarking vertex coordinates vertex topology (connectivity)

Embedded pattern

Simplified polygonal

A DCT-based Image Watermarking System

l

Reference
C.T.Hsu and J. L. Wu, Hidden digital watermarks in

images, IEEE Trans. On Image Processing, vol 8., No.1, January 1999 C. H. Huang and J. L. Wu, A Blind Watermarking Algorithm with Semantic Meaningful Watermarks, 34th Asilomar Conference on Signals. Systems, and Computers, Pacific Grove, October, 2000. D. Y. Chen, M. Ouhyoung, and J. L. Wu, "A Shift-Resisting Public Watermark System for Protecting Image Processing Software", IEEE Trans. on Comsumer Electronics, Vol. 46, No. 3, pp. 404-414, Aug. 2000.

Advantages of this scheme

l Advantages:
Semantically meaningful watermark pattern Good perceptual invisibility Acceptable robustness Various user-selected options Reasonable complexity/execution time

System overview
l Watermarks

are randomly permuted to spread their spatial relationship, and then embedded in the DCT domain of the host image, with consideration of invisibility/robustness

Block Diagrams of the Original Algorithm

Original Image Watermark Image Original Image Suspected Image FDCT Pseudo-random Permutation FDCT FDCT

Block-based Mapping Embedding (Polarity Reversing)

Extract the Permutated Data (XOR)

Reverse Blockbased Permutation IDCT

Watermarked Image

Reverse Pseudorandom Permutation

Extracted Watermark

Block DCT/IDCT
l Advantages
Fast Suitable for robustness against JPEG

compression
l Disadvantages
Block effect Effect of picture cropping

Semantic Meaningful Watermarks

l

Watermarks can be verified with naked eyes by understanding the semantics of the extracted watermark patterns

The seal of CML (in Chinese characters)

Pseudo-Random Permutation
lA

n-bit Linear Feedback Shift Register (LFSR) is used to generate the maximal length (2n-1) sequence
+

Original Watermark Permuted Watermark

The 14-bit Shift Register that permutes 1-16384

Block-based Mapping
l

Watermark blocks with more signal pixels are embedded into image blocks with higher variances
to achieve better perceptual invisibility.

Polarity Reversing
l

Polarity: the inequality relationship between DC & corresponding AC values within each DCT block.
watermark bits: 0 1 0 1 0 0 1 polarity: reversed polarity: 1100101 1001100 XOR Extraction

Embedding

Embedding(1/2)
l

Choices of embedding positions within each block:

Embedding 4 watermark pixels

0 2 3 9 10 20 21 35 1 4 8 11 19 22 34 36 5 7 12 18 23 33 37 48 6 13 17 24 32 38 47 49 14 16 25 31 39 46 50 57 15 26 30 40 45 51 56 58 27 29 41 44 52 55 59 62 28 42 43 53 54 60 61 63

Low-frequency
l

Bad invisibility Bad robustness

High-frequency
l

=> Middle-frequency
l

Fix positions in each block

Embedding(2/2)
l

Polarity:the inequality relations between the scaled DC value and the selected AC coefficients
AC (i, j ) DC * Q(i, j ) > * Q(0,0) 1, if P(i, j ) = Q(i, j ) ScaleFactor * Q(0,0) 0, otherwise

Effects of the JPEG quantization table are also considered

Extraction
l Exclusive-or

(XOR) operations are performed on the two polarity patterns to obtain the permuted binary data.

Indispensability of Original Images for Watermark Extraction

l Reasons
The relative ordering of block variance

values is changed after embedding. The watermark extraction needs both polarity information from the original and the embedded images.

VideoVR System
l

A panoramic image construction tool developed by CML

Watermarking for VideoVR

l Each

output panorama picture has been embedded with an invisible watermark.

Extraction

Difficulty in Applying the Original Algorithms

Original Image Suspected Image FDCT FDCT

l Only

panorama pictures embedded with watermarks are available.

Extract the Permutated Data (XOR)

Reverse Blockbased Permutation

Reverse Pseudorandom Permutation

Extracted Watermark

A Blind Version of the Watermarking Algorithm

Original Image Watermark Image Suspected Image FDCT Pseudo-random Permutation FDCT

Embedding Parameter Adjusting Embedding (Watermark Labeling)

Readout the Permutated Data (XOR)

IDCT

Reverse Pseudorandom Permutation

Extracted Watermark

Watermarked Image

Watermark Labeling
l Corresponding

polarities are set according to the watermark sequence.

watermark bits: 0 1 0 1 0 0 1 polarity: 1100101 0101001 Read Out Extraction

reversed polarity:

Embedding

Removing Block-Based Permutation

l Removing

the block-based permutation to eliminate the need of original images

The image quality loss is compensated by

carefully adjusting watermarking parameters The experimental results show that no perceptible quality loss was found

Experimental Results(1/2)

A panorama picture before embedding watermark

A panorama picture after embedding watermark

Experimental Results (2/2)

The watermark extracted from the watermarked picture

The watermark extracted from the picture without watermarking

System Demonstration

Introduction to visible watermarking schemes

l

G. Braudaway, K.A. Magerlein, and F. Mintzer, "Protecting Publicly Available Images with a Visible Image Watermark," IS&T/SPIE Symposium on Elect. Imaging Sci. and Tech., Proceedings of Symposium on Optical Security and Counterfeit Deterrence Techniques, Feb. 1996. C. H. Huang and J. L. Wu, Attacking Visible Watermarking Schemes, accepted by IEEE Transactions on Multimedia

Visible Watermarking
+

l l

IPR protection mechanisms for images and videos that have to be released. Unobtrusive copyright patterns are recognizable after embedding.

Requirements for Visible Watermarking Schemes

l The

perceptibility of copyright patterns (watermarks) l The perceptibility of host image details l Robustness
Difficult to remove unless exhaustive and

expensive human interventions are involved.

A General Formulation for Visible Watermarking Embedding

I ' = K 1 * I + K 2 *W D ( E I ( I ' ), E I ( I )) < Threshold I

D( EW ( I ' ), EW (W )) < ThresholdW

For a good visible watermarking scheme, the embedding parameters are assumed to be unknown.

An visible watermarking system

(mn,m -mt ) Yw Yn,m 2/3 ( ) DL* Yn,m =Yn,m + . mA -mt 38667 Yw
~

Important observations (1/4)

visible watermarking scheme means successfully recover the watermarked area. l Implication:
Similar image processing techniques can

l Attacking

be adopted
l l

Image recovery Object removal

Important observations(2/4)
clearly recognize the copyright patterns, the contours of embedded patterns must be preserved. l Implication:
An attacking scheme is effective if 1. The pattern is completely removed 2. The shape is seriously distorted without seriously degrading visual quality.

l To

Important observations(3/4)
l The

perceptibility of the host image details within watermarked area depends on the preservation of edge information. l Implication:
Available information while attacking l Surrounding pixels around watermarked area. l Edge information within watermarked area is available while attacking.

Important observations(4/4)
l The

robustness lies in the inevitability of exhaustive and expensive labors. l Implication:

Only minimum user intervention should be

adopted during attacking

l

User selection of watermarked areas

Averaging Attacks

Refill the watermarked areas by averaging surrounding pixels.

Good approximations for small areas. Blurring effects across object boundaries

Image Inpainting
n N (i , j , n ) n I (i, j ) I ( i , j ) = dL ( i , j ) N (i , j , n )
n t

M. Bertalmio, V. Caselles, and C. Ballester, Image inpainting, SIGGRAPH 2000, Aug. 2000 l Image inpainting
l is an iterative image recovery technique. prolongs the approaching isophotes into damaged areas. successfully reconstruct the edges of damaged area.

Inpainting Attacks

Attacks against visible watermarking are regarded as common image recovery problems. l Good results can be obtained for areas composed of thin copyright patterns, but areas composed of thick patterns cannot be successfully recovered.
l

Generalized Attacks (I)

Mathematical morphology

After the user selects the watermarked areas, decompose watermarked areas into parts:
Areas composed of thin patterns
l

Recovered by basic inpainting

Areas composed of thick areas

Generalized Attacks (II)

l

Classifying flat areas within watermarked area by analyzing remaining edge information of host images

Generalized Attacks (III)

Pixel values of the flat areas connecting the unwatermarked areas can be appropriately approximated by propagating that of surrounding unwatermarked areas.

Generalized Attacks (IV)

l l

The remaining areas can only be recovered by approximated prediction. The edge areas are recovered automatically by preserving the differences of intensity between edges and their surrounding flat areas, which may originally be unmarked or recovered in the previous step In order to recover the fully contained flat areas, the same algorithms applied to watermarked edge areas are adopted again.

Flowchart of the visible watermark attacking scheme

Experimental Results (I)

Experimental Results (II)

Experimental Results (III)

Fragile Watermarks
l

Definition
A watermark likely to be undetectable after a work

is modified in anyway. If a watermark is found, we can infer that this work is probably not altered.
l

Applications
Content authentication against malicious

alternation

Example
LSB embedding

Resource
l

Books
Digital Watermarking, Ingemar Cox, Jeffrey Bloom, Matthew Miller,

Morgan Kauffmann Publishers,2002 Information hiding techniques for steganography and digital watermarking, Stefan Katzenbeisser, Fabien A. P. Petitcolas (Editors), Artech House Books, 1999
l

Website
Digital Watermarking World Digital Watermarking Links by Alessandro Piva Watermarking and Data Hiding by Frank Hartung

Papers