WAVELET TRANSFORMS INTRODUCTION to THEORY POT Yat ood] iV HY | | | RAGHUVEER M. aed AJIT 5. BOPAR DIKARDa ee ee complicated theory of wavelets with 2 practical description of its various applications in image processing. Arecommended reference far the wavelet practitioner.” Se CS Cee Se ee ed stodents wity a practical understanding of wavelet transtoems and their properties. The authors intreduce the ueWerdng Dt ed image processing, and communications. Ceo ee eee eu Tr eed Co enn eee ee ee a ee ee OM eed Cee ee RC eu) Deen eee ee ee Et ned Sot Mir eee et eet ire a Soe em ee he a ead CR ec CR eM ee ee tenia No Re Ue So gg CNet e ete ee kl at GE ne ee md De eR eo De Se Mu ee ne eed Seen ene eee tee ee re ee a) De a re ea eee ee et ea Mee ae ee ea Ca be ecb ee weg eee ad De A Re Pe A nee ue ge De 0 eared Poe Tad renner ry emi Car ecWavelet Transforms Introduction to Theory and Applications Raghuveer M. Rao and Ajit S. Bopardikar A vv ADDISON-WESLEY An Imprint of Addison Wesley Longman, Inc. Reading, Massachusetts e Harlow, England e Menlo Park, California Berkeley, California # Don Mills, Ontario # Sydney Bonn e Amsterdam @ Tokyo e Mexico CityMATLAB and Simulink are registered trademarks of The MathWorks, Inc. For information, please contact: The MathWorks, Inc. 24 Prime Park Way Natick, MA 01760-1500 Phone: (508) 647-7000 Fax: (508) 647-7001 E-mail: info@mathworks.com www.mathworks.com Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and Addison Wesley Longman, Inc. was aware of a trademark claim, the designations have been printed in initial capital letters or all capital letters. The authors and publisher have taken care in preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein. The publisher offers discounts on this book when ordered in quantity for special sales. For more information please contact: Computer and Engineering Publishing Group Addison Wesley Longman, Inc. One Jacob Way Reading, Massachusetts 01867 Library of Congress Cataloging-in-Publication Data Rao, Raghuveer M. Wavelet transforms : introduction to theory and applications / Raghuveer M. Rao, Ajit S. Bopardikar. . cm, Includes bibliographical references and index. ISBN 0-201-63463-5 1, Wavelets (Mathematics) 2. Signal processing—Mathematics. I. Bopardikar, Ajit S. Il. Title. QA403.3.R36 1998 515’.2433—dce21 98-22332 CIP Copyright © 1998 by Addison Wesley Longman, Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. Printed in the United States of America. Published simultaneously in Canada. Cover image by Raghuveer M. Rao. Pseudocolor plot of the phase of the continuous wavelet transform of a complex-valued random signal. Wavelet used: Mexican hat. Text printed on recycled paper. ISBN 0-201-63463-5 1234567 8 9—MA—O0201009998 First printing, July 1998oie To Chitra and Chirag whose love and support made this possible, to Nikhil who has brought new joy, and to my parents —Raghuveer M. Rao To my parents and to the people with whom I have shared my dreams —Ajit S. BopardikarCONTENTS Preface xi Acknowledgments _ xiii 1 CONTINUOUS WAVELET TRANSFORM 1 1.1 Introduction 1 1.2 Continuous-Time Wavelets 1 1.3 Definition of the CWT 3 1.4 The CWT asa Correlation 8 1.5 Constant Q-Factor Filtering Interpretation and Time-Frequency Resolution 11 1.6 The CWT as an Operator 19 1.7 Inverse CWT 20 18 Problems 22 2 INTRODUCTION TO THE DISCRETE WAVELET TRANSFORM AND ORTHOGONAL WAVELET DECOMPOSITION 25 2.1. Introduction 25 22 Approximations of Vectors in Nested Linear Vector Subspaces 29 2.2.1 Example of Approximating Vectors in Nested Subspaces of a Finite-Dimensional Linear Vector Space 29 2.2.2. Example of Approximating Vectors in Nested Subspaces of an Infinite-Dimensional Linear Vector Space 31 2.3 Example of an MRA 32 2.3.1 Bases for the Approximation Subspaces and Haar Scaling Function 35 2.3.2 Bases for the Detail Subspaces and Haar Wavelet 36 2.3.3. Digital Filter Implementation of the Haar Wavelet Decomposition 41 24 Problems 50 viiviii Contents 3 MRA, ORTHONORMAL WAVELETS, AND THEIR RELATIONSHIP TO FILTER BANKS 51 3.1 Introduction 51 3.2. Formal Definition of an MRA 52 3.3 Construction of a General Orthonormal MRA 53 3.3.1 Scaling Function and Subspaces 53 3.3.2 Implications of the Dilation Equation and Orthogonality 55 3.4 A Wavelet Basis for the MRA 58 3.4.1 Two-scale Relation for y(t) 59 3.4.2 Basis for the Detail Subspaces 61 3.4.3 Direct Sum Decomposition 63 3.5 Digital Filtering Interpretation 64 3.5.1 Decomposition Filters 64 3.5.2 Reconstructing the Signal 68 3.6 Examples of Orthogonal Basis-Generating Wavelets 70 3.6.1 Daubechies D4 Scaling Function and Wavelet 71 3.6.2. Bandlimited Wavelets 71 3.7 Interpreting Orthonormal MRAs for Discrete-Time Signals 78 3.7.1. Continuous-Time MRA Interpretation for the DTWT 78 3.7.2. Discrete-Time MRA 79 3.7.3. Basis Functions for the DTWT 82 3.8 Miscellaneous Issues Related to PRQMF Filter Banks 84 3.9 Generating Scaling Functions and Wavelets from Filter Coefficients 89 3.10 Problems 92 4 ALTERNATIVE WAVELET REPRESENTATIONS 95 4.1 Introduction 95 4.2 Biorthogonal Wavelet Bases 95 43 Filtering Relationship for Biorthogonal Filters 102 4.4 Examples of Biorthogonal Scaling Functions and Wavelets 104 4.5 Two-Dimensional Wavelets 109 4.6 Nonseparable Multidimensional Wavelets 115 4.7 Wavelet Packets 121 4.8 Problems 126 5 WAVELET TRANSFORM AND DATA COMPRESSION 133 5.1. Introduction 133 5.2 Transform Coding 14254 Contents DTWT for Image Compression 147 5.3.1 Image Compression Using DIWT and Run-Length Encoding 149 5.3.2 Embedded Tree Image Coding 151 5.3.3. Comparison with JPEG 166 Audio Compression 170 5.4.1 Audio Masking 175 5.4.2 Standards Specifying Subband Implementation: ISO/MPEG Coding for Audio 177 5.4.3 Wavelet-Based Audio Coding 178 Video Coding Using Multiresolution Techniques: A Brief Introduction 181 OTHER APPLICATIONS OF WAVELET TRANSFORMS 183 61 6.2 63 64 65 6.6 67 Introduction 183 Wavelet Denoising 183 Speckle Removal 189 Edge Detection and Object Isolation 192 Image Fusion 200 Object Detection by Wavelet Transforms of Projections 204 Communication Applications 208 6.7.1 Scaling Functions as Signaling Pulses 208 6.7.2 Discrete Wavelet Multitone Modulation 214 ADVANCED TOPICS 219 71 7.2 7.3 74 75 7.6 77 Introduction 219 CWTs Revisited 219 7.2.1 Parseval’s Identity for the CWT 219 7.2.2. Inverse CWT Is a Many-to-One Operation 220 7.2.3. Wavelet Inner Product as a Projection Operation 222 Bridging the Gap Between CWTs and DWTs 223 7.3.1 CWT with an Orthonormal Basis-Generating Wavelet 223 73.2. ATrous Algorithm 224 Regularity and Convergence 229 Daubechies Construction of Orthonormal Scaling Functions 232 Bandlimited Biorthogonal Decomposition 236 7.6.1 Scaling Function Pair Construction 241 7.6.2 Wavelet Pair Construction 243 Design and Selection of Wavelets 244 7.7.1 The Lifting Scheme 244x Contents 78 79 7.10 7.7.2 Best Basis Selection 248 7.7.3. Wavelet Matching 250 Perfect Reconstruction Circular Convolution Filter Banks 252 7.8.1 Downsampling 252 7.8.2 Upsampling 255 7.8.3. Procedure for Implementation 255 7.8.4 Conditions for Perfect Reconstruction 257 7.8.5 Procedure for Constructing PRCC Filter Banks 258 Interpolators Matched to the Input Process 259 7.9.1 Interpolation Sampling 260 7.9.2 Frequency-Sampled Implementation of Bandlimited DWTs 262 The Scaling Operation and Self-Similar Signals 264 7.10.1 LTI Systems and Eigenfunctions 264 7.10.2 Continuous-Time Linear Scale-Invariant System 265 7.10.3 Scaling in Discrete Time 268 7.10.4 Discrete-Time LSI Systems 271 A FUNDAMENTALS OF MULTIRATE SYSTEMS 279 Al A2 A3 The Downsampler 279 The Upsampler 280 Noble Identities 282 B LINEAR ALGEBRA AND VECTOR SPACES 285 BA B.2 B23 Brief Review of Vector Spaces 285 B.1.1 Vector Subspace 286 B.1.2 Linear Independence and Bases 286 Inner Product Spaces 287 Hilbert Space and Riesz Bases 288 Bibliography 291 Index 301PREFACE The wavelet transform has been perhaps the most exciting development in the last decade to bring together researchers in several different fields such as signal pro- cessing, image processing, communications, computer science, and mathematics—to name a few. This book provides an introduction to wavelet transform theory and applications for engineers. The subject has been taught previously as part of several of our graduate courses in the Electrical Engineering Department at the Rochester Institute of Technology and is now being taught as a complete course. Most of the students who take the course are working engineers from local industries such as Eastman Kodak Company, Xerox Corporation, Harris, RF Communications, and so on. A challenge in teaching this audience has been to cast the material in a lan- guage familiar to engineers with a basic undergraduate degree while maintaining accuracy and rigor. Over the years we have been able to develop an approach using geometric analogies and filtering concepts to meet this challenge successfully. Our students and colleagues have been urging us to write a book that adopts such an approach because they find the material in the mathematically oriented books to be daunting and inaccessible. We have endeavored to keep the presentation in line with their exhortations. Application of the wavelet transform has almost come to be regarded as being synonymous with data compression. So it should come as no surprise that we have an extensive chapter on this application. However, there are properties of the wavelet transform that make it naturally suited for application in many other areas. It has been our desire for some time to bring out this fact. The reader will find a detailed chapter devoted to such applications. The book starts off with a discussion of the continuous wavelet transform. This is in keeping with the teaching approach in our classes. Rather than starting with one or two chapters devoted to notation and basic material such as Fourier analy- sis, the reader is led directly into the subject using basic linear filtering principles. The wavelet transform is introduced in the context of reconstructing a signal from the outputs of filters with impulse responses that are generated by dilation of a single function. Chapter 2 introduces the basics of discrete wavelet transforms and multiresolution analysis. -The latter concept is first developed in the famil- iar setting of linear vector spaces and Fourier series. The role of wavelets is then xixii Preface introduced through the Haar wavelet and piecewise constant approximations of signals. Chapter 3 considers multiresolution analysis based on orthogonal wavelet basis functions in greater detail. Subband filter bank implementations for signals and images as well as efficient implementation are discussed. Chapter 4 looks at al- ternative wavelet representations such as biorthogonal bases, wavelet packets, and multiresolution analysis of images. Chapter 5 provides a detailed treatment of the use of wavelet transform techniques in signal and image compression. In Chapter 6, wavelet transform application to areas such as denoising, object isolation, and de- tection are explained. Lastly, Chapter 7 treats several advanced topics. A question that arises often is which wavelet to choose in an application. Chapter 7 addresses this issue. It also deals with some of the latest research on generating wavelets and decompositions that are tailored specifically to meet certain objectives. Through- out the text, the presentation is patterned after what is usually found in electrical engineering texts and generally stays away from the “theorem-proof” model. The readers do not have to be familiar with real analysis or functional analysis. The con- cept of integration as dealt with traditionally in undergraduate engineering courses should suffice for understanding the material, and because it serves perfectly well in most applications, the attendant sacrifice in mathematical rigor should be of minor importance from a practical viewpoint. Problem sets are provided at the ends of Chapters 1 through 4. We also provide a disk with MATLAB routines that illustrate the concepts developed in the text.It is our pleasant task to acknowledge the help of many in making this book possible. The book has been influenced considerably by the suggestions and work of B.S. Adiga, Joseph Chapa, Sohail Dianat, Ajay Divakaran, and Wei Zhao. Sohail Dianat and Ajay Divakaran were also part of the fine review panel assembled by Addison Wesley Longman, the other members of which were Prashant Gandhi, Perry Li, and Mike Sablatash. Our many thanks to them. Our thanks to the Electrical Engineering Department and the Center for Imaging Science at the Rochester Institute of Technology for their support. We are indebted to the many graduate and undergraduate students for their comments. Many others have played a role in the creation of the book by providing us with opportunities to present our research work, critiquing portions of the manuscript, being research collaborators, making suggestions, and so forth. The following is our attempt to list their names: Akram Aldroubi, Mark Alford, Peter Anderson, Richard Buck- ley, V. Chander, George Cleetus, Roy Czernikowski, David Ferris, Wesley Foor, Roger Gaborski, Joe Garcia, Mark Getbehead, Sebastian Gracias, Ed Hoefer, Lyman Hurd, Shubha Kadambe, Anand Kannan, Thomas Kenney, Yogish Lavanis, James LeBlanc, Ed Leiby, Jim Leone, Jingquan Li, Richard Lyons, George Mathews, Steven McLaughlin, James Michaels, Terry Montonye, Dinesh Nadarajah, S.V. Narasimhan, Colonel Nielsen, Kannan Panchapakesan, Paul Patti, Athina Petropulu, Majid Rab- bani, P. Ravindra, V.U. Reddy, Harvey Rhody, Vincent Samar, Andreas Savakis, Carl Schauffele, John Schott, N. Sitaram, M.K. Sridharan, Venkatesh Subramaniam, Ken Swartz, Harold Szu, Brian Telfer, Raman Unnikrishnan, Madhu Vairy, Xinran Wang, Ted Wilcox, Paul Willson, Valerie Zelenty, and Michael Zoltowski. Finally, we thank Addison Wesley Longman for all their efforts. In particular, Simon Yates came up with the idea, Tara Herries bore the brunt of the review chores, and Lynn Merriam and Maureen Hurley saw the book through the final stages. Our heartfelt thanks to them. —Raghuveer Rao and Ajit Bopardikar It is my pleasure to acknowledge gratefully the mentoring of Chrysostomos Nikias early in my career. It has been a privilege and an honor to have received his guidance. —Raghuveer Rao xiiiCHAPTER ONE Continuous Wavelet Transform 1.1 Introduction The beginnings of the wavelet transform as a specialized field can be traced to the work of Grossman and Morlet [1984]. Their motivation in studying wavelet trans- forms was provided by the fact that certain seismic signals can be modeled suitably by combining translations and dilations of a simple, oscillatory function of finite du- ration called a wavelet. The early results were related to what is now known as the continuous wavelet transform (CWT) [Grossman and Morlet 1984]. However, “wavelet transformlike” expressions can be found in earlier work done in several fields such as function representation, quantum mechanics, and signal processing.! The inter- est triggered by the work of Grossman, Morlet, and colleagues was sustained and nurtured when connections were discovered between wavelet representations and developments in other fields [Meyer 1993b]. For a historical account of the wavelet transform the reader is referred to Meyer [1993b] and Daubechies [1996]. In this chapter we provide an introduction to CWTs and their properties. 1.2. Continuous-Time Wavelets Consider a real or complex-value continuous-time function #(t) with the following properties [Daubechies 1988; Grossman and Morlet 1984]: 1. The function integrates to zero: et y(t)dt =0. (1.1) 1 The term wavelet can also be found in the seismic signal processing literature in a context other than the wavelet transform (see Stone [1984]).2 Continuous Wavelet Transform 0.4 Figure 1.1 The cubic B-spline wavelet. 2. It is square integrable or, equivalently, has finite energy: 00 / Iwa)Pdt <0. (1.2) 0 The function y(t) is a mother wavelet or wavelet if it satisfies these two properties as well as the admissibility condition defined later in this chapter. While the ad- missibility condition is useful in formulating a simple inverse wavelet transform, properties 1 and 2 suffice to define the CWT, and they capture essentially the rea- sons for calling the function a wavelet. Property 2 implies that most of the energy in w(t) is confined to a finite duration. Property 1 is suggestive of a function that is oscillatory or that has a wavy appearance.” Thus, in contrast to a sinusoidal func- tion, it is a “small wave” or a wavelet. The two properties are easily satisfied and there is an infinity of functions that qualify as mother wavelets. Figure 1.1 shows the plot of a wavelet called the cubic B-spline wavelet. This particular wavelet is sup- ported compactly. In other words, the entire wavelet has a finite duration: 0 <1 < 4 sec. However, it is possible to have wavelets that are not supported compactly. For 2 A function does not necessarily have to oscillate to satisfy this property.Definition of the CWT 0.8 5 0.4 F 4 wo -0.4 -0.6 5 -08 Figure 1.2 The Morlet wavelet. example, a Morlet wavelet, which is constructed by modulating a sinusoidal function by a Gaussian function [Daubechies 1990; Vetterli and Kovacevic 1995], is a wavelet of infinite duration. But most of the energy in this wavelet is confined to a finite interval. For instance, consider the real-value Morlet wavelet: 2 2 w(t) =e cos (- ) : (1.3) Its plot is shown in Figure 1.2. More than 99% of the total energy of the function is contained in the interval |t| < 2.5 sec. 1.3. Definition of the CWT Let f(t) be any square integrable function. The CWT or continuous-time wavelet transform of f(t) with respect to a wavelet y(t) is defined as (= stands for is defined as) [Grossman and Morlet 1984; Daubechies 1992] wea. f rorew (—) ar (1.4) 3Continuous Wavelet Transform where a and b are real and + denotes complex conjugation. Thus, the wavelet transform is a function of two variables. Observe that both f(r) and w(r) belong to L?(R), the set of square integrable functions, also called the set of energy signals [Couch 1997]. Equation 1.4 can be written in a more compact form by defining Wa,p(t) as 1 t—b Vablt) = Tai” (=) : (1.5) Then, combining Equations 1.4 and 1.5, 00 Wa, b) = / SOW, dt. (1.6) 00 Notice that yno(t) = Ww). £17) The normalizing factor of 1/,/Ja| ensures that the energy stays the same for all a and 5; that is, 00 00 | Weo(Olat = [ Wer de (8) 00 00 for all a and b. For any given value of a, the function Wo,4(t) is a shift of Wa,o(t) by an amount b along the time axis. Thus, the variable b represents time shift or translation. From 1 t Wa.o(t) = Vial” (:) (1.9) it follows that Wa,o(t) is a time-scaled and amplitude-scaled version of y(r). Since a determines the amount of time scaling or dilation, it is referred to as the scale or dilation variable. Figure 1.3 shows two dilations of the Morlet wavelet. If a > 1, there is a stretching of y(t) along the time axis, whereas if 0 < a < 1, there is a contraction of y(t). Negative values of a result in a time reversal in combination with dilation. Since the CWT is generated using dilates and translates of the single function y(r), the wavelet for the transform is referred to as the mother wavelet. Example 1.1 Let the mother wavelet be 1 O 0, Wa b<1 h= 4) 1-(6+a/2) 0 1 Ih = 41-(b+a) 0 b<-a/2 -a/21-a/2 b<-a -a1l-a. Let us now obtain the CWT by computing W(a, 5) as a function of b for different values of a. Observe that associated with the expression for each of h, 12, and J; in Equations 1.18, 1.19, and 1.20 respectively, there is a pair of values of b at which the expression changes. These are 0 and 1 for h, —a/2 and 1—a/2 for h, and —a and 1—a for /3. In substituting these back into Equation 1.16, these six values have to be taken into account because the expression for W(a, b) (for a given a) will change as b (which goes from —oo to oo) crosses these values. For 0 . For 1 < a < 2, we have ~a < —a/2 < 1-a < 0 <1~—a/2 <1. Thus, over this range, b<-a -a.8 Continuous Wavelet Transform Finally, for a > 2, we have -a < 1 —a < ~a/2 <1~—a/2 < 0 <1, which yields 0 b<-a a8 ~a. For a < 0 we find that Ves) =— [we 2) - 26 (1-5-5) +ue-o-a)]. (1.24) 1 V=a Going through the integration as in the case of a > 0 we find that (see Problem 9 at the end of this chapter) Wa,b) =—W(-a, 1-5). (1.25) Figure 1.4 shows a contour plot of |W(a,5)/?.. The symmetry given by Equation 1.25 is captured by the plot. The quantity |W/(a, 6)|? is called the scalogram of f(r). The of events in the original signal. For a two-dimensional function such as an image, this property corresponds to the ability of the CWT to provide information regarding the location and size of an object in the image. The CWT for this example approaches zero as a 0 and also as a -+ +00 (see Problem 3). As this example shows, it is difficult to obtain a closed-form expression for the CWT even for simple functions and wavelets. Therefore, we typically use numerical algorithms to find wavelet transforms. 1.4 The CWT as a Correlation We now provide an interpretation of the CWT that has a bearing on its application in areas such as pattern detection and classification. The set of square integrable functions forms a linear vector space under addition and scalar multiplication.3 Fur- thermore, this vector space comes with a well-defined inner product. Given two finite energy signals x(t) and y(r), their inner product, denoted by (x(1), y(t), is given by (x(t), ¥()) =/ x(t)y*(t) dt. (1.26) oo 3 See Appendix B and [Hoffman and Kunze 1971] for a discussion on vector spaces and related topics.The CWT as a Correlation e nv -2 3 -4 -5 ad 0 5 Figure 1.4 Contour plot of W(a, b) for Example 1.1. The total energy in x(t) is given by f lx(@e)P de (1.27) 20 which is (x(t), x*(t)). The square root of the energy is called the norm of x(t) and is denoted by |[x(t)|| [Hoffman and Kunze 1971].* From Equation 1.26 we see that the CWT is essentially a collection of inner products of a signal f(t) and the translated and dilated wavelet Wa,4(¢) for all a and b: Wa, b) = (f(t), Wa.o(t)) (1.28) 4 This is known as the L? norm. With the inner product and norm as defined the space of square integrable functions, L(R) forms what is known as a Hilbert space. 910, Continuous Wavelet Transform The ‘CWT has a cross-correlational interpretation as well. The cross-correlation R,y(t) of the two functions x(1) and y(r) is defined as Rey(t) = [xorre =D =O. =D) 0.29) where t is the lag or shift parameter. Using Equations 1.5 and 1.9, Equation 1.28 can be written as W(a, b) = (F(t), Wa.olt — b)) = Rey. (b). (1.30) In other words, the CWT is the cross-correlation at lag b between f(t) and the wavelet dilated to scale factor a. Using the Cauchy-Schwarz inequality (see Ap- pendix B), Wea, by? < IFO? Wao OIF. (1.31) This in turn implies that W(a, b) always exists because the function and the wavelet have finite norms. Equality holds in this relation if and only if Wap (t) = af (t) (1.32) for some scalar a. Therefore, if there actually is a pair of values (a,b) for which Equation 1.32 is satisfied, then the scalogram |W(a, b)|? achieves its maximum at this pair of values. Even if Equation 1.32 is not satisfied, the position of the global maximum in the real part of W2(a, b) (which is the same as the scalogram when the wavelet is real-valued) provides a measure of the fit between f(t) and the corresponding Wa,o(t), as explained next. A measure of the separation between f(t) and ya.s(t) is provided by || f(t) -— Wa,b(t)||?. We have (see Problem 12), IFO — Var OIP = NFOIP + lve o@I? —2 Rel W(a, b)] (1.33) where Re[ ] stands for “real part of.” Note that w(t) is closest to f(t) for that value of the pair (a, b) for which the left-hand side is minimized. Of the quantities on the right-hand side, only Re[ (a. b)] varies with a and b (the term ||¥a,o(1)|I? is a constant by Equation 1.8). Furthermore, it has a negative sign associated with it. Thus the maximum value of Re[ W(a, b)], because it corresponds to the minimum value of the left-hand side, yields the values of a and b that provide the best positive correlation between f(r) and Wo.4(t). Similarly, working with || f(t) + Ve,o(¢)II?, which provides a measure of closeness between f(t) and —y(t), we see that the minimum value of Re[ W(a, b)] yields a andConstant Q-Factor Filtering Interpretatten and Time freyasricy RESOMTON b for which there is the best negative correlation between f(t) and w,,,(t). Thus, the maximum value of Re[ W(a, b)] points to the best possible match between a dilated and translated (and possibly negated) version of the mother wavelet and the input signal f(t). Local maxima usually disclose local features in the signal that correlate well with the wavelet for values of dilation and shift corresponding to the location of such maxima. 1.5 Constant @-Factor Filtering Interpretation and Time-Frequency Resolution The CWT offers time and frequency selectivity; that is, it is able to localize events both in time and in frequency. Its ability to localize events in time is easy to establish. The segment of f(t) that influences the value of W(a, b) for any (a, 5) is that stretch of f(t) that coincides with the interval over which Wa.5(t) has the bulk of its energy. This windowing effect results in the time selectivity of the CWT. The frequency selectivity of the CWT is explained using its interpretation as a collection of linear, time-invariant filters with impulse responses that are dilations of the mother wavelet reflected about the time axis. This follows from the fact that the cross-correlation operation on two signals can be expressed as a convolution, and the CWT was seen in Equation 1.30 to be a cross-correlation in b parameterized by the scale factor a. Denoting the convolution of two signals h(t) and x(t) by howe [ aeoxe 0 dt (1.34) we have Wa, b) = f(b) * ¥39(—4). (1.35) Thus, for any given a, the wavelet transform W(a, b) is the output of a filter with impulse response 3 9(—b) and input f(b). We have a continuum of filters param- eterized by the scale factor a. For the purpose of illustrating the frequency selectivity of the CWT let us choose the Mexican hat wavelet vO = (1 = 2%)" (1.36) which is obtained by taking the second derivative of the negative Gaussian function —e-" 72. A plot of the wavelet is shown in Figure 1.5. Let ¥(w) denote the Fourier transform of w(t): Ww) = [ w(ne te, (1.37) VW12 Continuous Wavelet Transform 0.5 a -0.5 J -10 -5 0 5 10 Figure 1.5 The Mexican hat wavelet. From the plot of |W@)|? shown in Figure 1.6, it is seen that the wavelet is essen- tially a bandpass function (see Problem 13 for a definition of bandpass functions) with the Fourier transform magnitude centered® at approximately 2 rad/sec. Since time reversal does not change the Fourier transform magnitude of a real-valued wavelet, Figure 1.6 applies to the frequency domain characterization of y(—1) as well. The 3-dB bandwidth, defined as the difference between the two frequencies on either side of the peak at which the squared magnitude of the Fourier transform is exactly half its peak value, is approximately 2 rad/sec, covering the band from 1 rad/sec to 3 rad/sec. This gives a Q-factor (ratio of the center frequency to the 3-dB bandwidth)® of roughly 1 for the bandpass function. The Q-factor is invariant 5 Centering here is not to be taken literally. The center frequency here refers to the frequency at which the Fourier transform magnitude attains its peak value. ©The term Q-factor is commonly used in assessing the quality of band rejection or the sharpness of response in tuning filtering applications such as broadcast reception. The higher the value of Q, the sharper the tuning.Constant Q-Factor Filtering Interpretation and Time-Frequency Resolution 08 Iw)? 0.6 04 0.2 0 1 2 3 4 5 6 ha w (rad/sec) Figure 1.6 Squared magnitude of the Fourier transform of the Mexican hat wavelet. with respect to wavelet dilation because Fly(t/a)] = |al¥ (aw) (1.38) where F[ ] denotes Fourier transformation. The center frequency of F[y(t/a)] for any a is at 1/|a| times the center frequency of the mother wavelet, and its 3-dB bandwidth is also 1/ja| times the 3-dB bandwidth of the mother wavelet, thus yielding the same value for the Q-factor as before. Thus the continuum of filters to which we alluded previously is actually a set of constant-Q bandpass filters. It is this bandpass nature that gives rise to the frequency selectivity of the CWT. By virtue of the relationship in Equation 1.38 it follows that for a large value of a, the associated filter has a frequency response centered at a low frequency value and, therefore, the corresponding W(a, b), seen as a function of b for this fixed a, captures the frequency content of f(t) around this low frequency. On the other hand, the smaller the value of a, the more the bandpass shifts to a higher frequency, implying that the CWT at small scales contains information about f(r) at the higher end of its frequency spectrum. The squared frequency response magnitudes for three different values of a for the Mexican hat wavelet are shown in Figure 1.7. Observe that as a increases, the frequency response shifts to the lower end of the spectrum with a corresponding shrinking of the bandwidth to maintain the constant Q-factor. 1314 Continuous Wavelet Transform 25 Wo? 1.5 0.5 0 1 2 3 4 5 6 7 @ (rad/sec) Figure 1.7 Squared magnitude frequency responses of a Mexican hat wavelet as a function of a. Here Yq(w) = F(¥a.o(—1)). Example 1.2 The combined effect of time and frequency localization is brought out in this example. Let FMO=AO+ AO (1.39) where sin(w1/100) O<1<10 A@= { 0 otherwise and fl) = sin(w1/200) 5<1<15 ow 10 otherwise. Thus f(r) is the sum of two truncated sine waves or tones of two different frequencies with a partial overlap in time. A plot of f(r) is shown in Figure 1.8. The scalogram of this signal obtained through the Mexican hat wavelet is shown in Figure 1.9 as an intensity map in which the darkness is proportional to the value of |W(a, )|?. Peaks occur in the scalogram for lower values of scale in the range 0 < b < 5 than for the range 10 < b < 15. This is a reflection of the fact that f(t) = fi(r) for 0 <1 <5 andConstant @-Factor Filtering Interpretation and Time-Frequency Resolution fo) 0 ° a Figure 1.8 Plot of f(s) for Example 1.2. F (0) = falt) for 10 <1 < 15. The frequency of f:(t) (1/400 Hz) is half the frequency of Fi(t) (1/200. Hz), and there is an inverse relationship between the scale and frequency as shown in this section. Also, for the time interval 5 1/2. (1.46) 1.6 The CWT as an Operator The CWT takes a member of the set of square integrable functions of one real variable in L?(R) and transforms it to a member of the set of functions of two real variables. Thus it can be seen as a mapping operator from L?(R) to the latter set. Define Wylf@] = Wa, 5). (1.47) Then W,[ ] is to be read “CWT with respect to w(t) of.” The notation for the operator uses w as a subscript to remind us of the fact that the transform depends not only on the function f(t) but also on the mother wavelet. We now enumerate various properties of the CWT using the operator notation: Linearity— Wylaf(t) + g(t] = eWy Lf] + By [gO] (1.48) for scalars w and f and functions f(t), g(t) € L?(R). 7 The Gaussian function is not a wavelet. Why? Although we have defined the time-bandwidth product in the context of wavelets, the definition applies to other types of functions as well. 1920 Continuous Wavelet Transform Translation— WyIf(t — 1)] = Wa, b - 1). (1.49) Scaling— 7 1 t a ab wo [zer(2)]=" (2-2) alas fora > 0. Wavelet shifting—Let W(t) = w(t —t). Then Wf] = WG@,b +ar). (1.51) Observe that the CWT obtained by shifting the wavelet is different from the one obtained by shifting the signal. Wavelet scaling—Let y(t) = [¥:(t/a)]/./Ta]. Then Wilf] = W(aa, b). (1.52) This result, which states that the new CWT is a scaled version of the original in the a variable, is not surprising because all dilations of the wavelet are actually used in obtaining the original CWT. Linear combination of wavelets—Given two wavelets y(r1) and wo(t), their linear combination ay (1) + BWe(t) for scalars a and f is also a wavelet. We have Wan +pvalf(t)] = aWy [f(91 + BWyLF OI. (1.53) 1.7. Inverse CWT Suppose W(a,b) = Wy[f(t)] for some signal f(t) and wavelet y(t). A natural question to pose is the following: Is it possible to go in the other direction? That is, given W(a, b), is it possible to obtain f(r)? In other words, does the CWT have an inverse? It is of particular interest to explore the possibility of an expression for the inverse transform that bears resemblance to the CWT of f(r). An analogy is provided by the forward and inverse Fourier transforms. Their expressions are similar in that the exponential kernel appears in both transformations (the kernel in the inverse is the complex conjugate of the one in the forward transform). Is it similarly possible to devise an inverse CWT such that it involves translates and dilates of the mother wavelet? In fact, if this could be done we would then be able toInverse CWT interpret the wavelet transform as providing a weighting function for synthesizing a given function f(t) from translates and dilates of the mother wavelet much as the Fourier transform provides a weighting function for synthesizing a function from sine and cosine functions. Fortunately, it is possible to construct such an expression. If the mother wavelet satisfies the (sufficient) condition that ar cs i wor (1.54) lol and is such that 0 < C < oo, then 1% = 1 fo=s ff wea. best da ab. (1.35) C Jaz-co Jo=-00 1a The condition in Equation 1.54 is known as the admissibility condition on w(t) [Dau- bechies 1990, 1992]. We now proceed with the derivation of the previous result [Chapa 1995; Kaiser 1994]. Equation 1.35 expresses W(a, b) as the convolution of f(b) with va,o(—b). In the frequency domain this becomes | Wa, be”? db = Jal F(w) ¥*(aw) (1.56) ‘b=—00 where F(w) and (w) are the Fourier transforms of f(t) and y(t) respectively. Multiplying both sides of Equation 1.56 by U(aw)/|a|>* and integrating with respect to a yields © |Wwaw)? 0 [ [ —1_W(a, by Y(awyeda db = F(w) da. (1.57) =-co Jb=~oo lal??? fa=-co (al It is easy to show that Co 2 00, 2 [ aol ta -f WOW do=Cc (1.58) a=-co al w=-co 1] where C is the constant from the admissibility condition. Thus, on eda db 1.59 Fo) = t ey tee Jape b)V(aw)e a db. (1.59) Taking the inverse Fourier transform on both sides of Equation 1.59, we get the expression for the inverse CWT in Equation 1.55, which shows that W(a, b)/|a|? provides the weighting function for constructing f(t) from wa.4(t)—the translates 2122 Continuous Wavelet Transform and dilates of the mother wavelet. While we have let a take both positive and negative values in the CWT, it is possible to use only positive a if the wavelet is real-valued (see Problem 14). The admissibility condition stated here is a sufficient condition that leads to the expression in Equation 1.55. It is not a necessary condition to obtain a mapping from the set of CWTs back to L?(R). Another important fact to note is that the range of the mapping Wy is not the entire set of square integrable functions of two variables, but only a subset of it. This means that it is possible to have a function, say X(a, b), which when substituted for W(a. b) in Equation 1.55 yields a function, say x(t), such that W,[x(t)] # X(a, b). Some of these issues are discussed again in Chapter 7. 1.8 Problems o . Show that the Mexican hat wavelet satisfies the admissibility condition.® n Using the Mexican hat wavelet, derive a closed-form expression for the CWT of the function _fi isi fO= { 0 otherwise. o . Show that the CWT in Example 1.1 tends to 0 as a tends to 0 and as a tends to too. Investigate whether this is generally true of CWTs. is . Suppose we obtain the function W(a, b) by putting f(t) = (1), the unit impulse or Dirac delta function in Equation 1.4. Is this a valid CWT? What happens if Equation 1.55 is evaluated with this W(a, b)? on Repeat Problem 4 with f(r) being a sinusoidal signal of arbitrary amplitude, frequency, and phase. a Show that if W(a,b) = W,[f()] for a wavelet that satisfies the admissibility condition, then 2 user = 2 f ia eee da db. acest pale This is a generalization of Parseval’s theorem to CWT. 7. The CWT is often compared to another time-frequency representation—the short-time Fourier transform (or STFT)—which can be defined in a form similar to the CWT. STFTf()] = Fone [ SOS, (t) at (1.60) where ¢..1(t) = w(t — t)e/”. The function w(r) is a windowing function, the simplest of which is a rectangular window that has a unit value over a finite interval and is zero § The authors thank Dr. Sohail Dianat for Problems 1 and 2.10. 11. 12. 13. 14. 15. Problems 23 elsewhere. How do the time and frequency resolution of this transform compare with that of the CWT? . Show that the Morlet wavelet does not satisfy the admissibility condition. Note: Even though it does not satisfy the admissibility condition, it is still possible to recover a signal from the CWT using what is called the Morlet algorithm instead of the inverse CWT expression in Equation 1.55 [Meyer 1993a]. . Obtain the wavelet transform of Example 1.1 for negative a and verify Equation 1.25. Derive an expression for how the first moment of a wavelet W(t) varies with the pa- rameters a and b. How does the first moment of the Fourier transform of was(t) vary with a and b? Discuss the implications of this on the wavelet and its ability to cover the time and frequency ranges. Compute the time-bandwidth product for the Gaussian function given in Equation 1.45. Show that the squared separation between f(t) and Was(t) is given by Equation 1.33. A function f(t) is a bandpass function if its Fourier transform F(w) is confined to a frequency interval w < |w| < @2, where w > 0 and w» is finite. Is every bandpass function a wavelet? Show that if w(t) is a real-value wavelet then a function f(r) can be recovered from its CWT Wa, d) as fO= ef. [a W (a, b)Wra.o(t) da db a=0 Jb=—c0 where C = f° “@F do, Why can we not use this expression if the wavelet is complex? Hint: The answer “has to do with the symmetry of the Fourier transform W(w) of the wavelet about w = 0. The problem in Example 1.1 has an alternative solution that leads to the same answer. Define 0 t<0 F(th= | t O0 0 1=0. MATLAB? routine cwtex distributed with this book uses this approach. ° MATLAB is a trademark of The MathWorks, Inc.24 Continuous Wavelet Transform Table 1.1 DTMF Signaling Code Frequency (Hz) Signal 697 770 852 1 xX 2 X 3 xX 4 Xx 5 x 6 x 7 X 8 x 9 xX * 0 # 941 x x XK 1209 1336 po x x x x x x x x 1477 16. Touch-tone dialing on telephones uses what is known as dual-tone multifrequency (DTMF) signaling. Each key has a unique pair of frequencies associated with it. (See Table 1.1. The pair of frequencies for a given tone are indicated by an “X” in the cor- responding row.) Compute the scalogram of the DTMF signal. The DTMF sequence is provided on the disk included with this book as the file dtmf .wav. This file can be read using the MATLAB command wavread. Experiment with different wavelets. Observe how the scalogram changes as the wavelet changes. Decode the tones in the signal from the scalogram. (Note: This was actually assigned as a classroom project at the Rochester Institute of Technology. The students found it useful to filter out the line noise before computing the scalogram.)re CHAPTER TWO Introduction to the Discrete Wavelet Transform and Orthogonal Wavelet Decomposition 2.1 Introduction We saw in the previous chapter that the CWT maps a one-dimensional function f(t) to a function W(a, b) of two continuous real variables a and b, which are the wavelet dilation and translation respectively. The region of support of W(a, b) is defined as the set of ordered pairs (a,b) for which W(a,b) # 0. In principle, the region of support of a CWT is unbounded; that is, it can be the entire plane defined by R?, the set of all ordered real pairs. This chapter introduces a type of wavelet representation that has assumed considerable practical significance because of its link to digital filtering [Daubechies 1988, 1992; Mallat 1989a, 1989b; Vaidyanathan 1992; Vetterli and Kovacevic 1995]. We take the approach of developing the under- lying ideas entirely through an example involving the Haar wavelet. A more general development is presented in Chapter 3. The CWT provides a redundant representation of the signal in the sense that the entire support of W(a, b) need not be used to recover f(t). Consider the function f(t) with the Fourier transform given by 1 2s|o} <3 0 otherwise. (2-1) Fo) = | Suppose we obtain the CWT of f(t) using the wavelet y(t) with the Fourier trans- form given by 1 1slels4 2.2. 0 otherwise. (2.2) W(o) = | 2526 Discrete Wavelet Transform and Orthogonal Wavelet Decomposition Let W(a, b) be the resulting CWT and W,() its Fourier transform in the b variable so that oo Wa(w) = [ Wa, bei db. (2.3) Because F[va.o(t)] = lal (aw), we have W,(w) = Val F(w) ¥ (aw). (2.4) We now use Equation 2.4 to derive the region of support of W(a,b). Figure 2.1A shows a plot of F(w) and Figure 2.1B shows a plot of ‘/(w). From the fact that the support of ¥(w) is 1 < || < 4, it follows that the support of (aw) for a given a is I/la| < |o| < 4/la|. Comparing this support with that of F(w), we find that the two overlap only over the range 1/3 < |a| < 2, which is illustrated in Figure 2.2. Thus the support of this CWT is restricted to that portion of R? for which 1/3 < |a| < 2. However, we also see from Figure 2.2 that for 1/2 < |a| < 4/3, the support of F(w) is contained entirely within that of (aw), and therefore W,(w) = JlalF(w) 1/2 < lal < 47. (25) From Equation 2.5 it is seen that F(w) can be obtained from W,(w) or, equivalently, F(t) can be obtained from W(a, b) by a simple amplitude scaling of the latter for any fixed value of a in the range 1/2 < |a| < 4/3 as F(w) = Wa(w)/./a. This indicates that it is not necessary to use the entire support of W(a, b) to reconstruct f(t). We have used a specific function and a specific wavelet to illustrate how redundancy arises in a CWT. However, it is possible to generalize the result to any CWT [Daubechies 1992]. In this chapter we introduce a type of nonredundant wavelet representation. In particular we look into a representation of the form fO= VY dk, or'"yar-o,. (2.6) knoe (=—00 Observe that in contrast to Equation 1.55, which involves a continuum of dila- tions and translations, Equation 2.6 uses discrete values for these parameters. The dilation takes values of the form a = 2 where k is an integer. At any dilation 2*, the translation parameter takes values of the form 2*€ where € is again an integer. The values d(k, £) are related to values of the wavelet transform W(a,b) = WIf(0)] at a = 2* and b = 2'¢. This corresponds to sampling the coordinates (a,b) on a grid such as the one shown in Figure 2.3. This process is called dyadic samplingIntroduction 27 18 _-—}YY SS 1 F(w) 0.5 0 (a) =10 “5-3-2 02 3 5 10 ® 15 1 Ww) 0.5 0 (B) =10 4-101 4 10 w Figure 2.1 Fourier transform of a signal (A) and a wavelet (8). because consecutive values of the discrete scales as well as the corresponding sam- pling intervals differ by a factor of two. The two-dimensional sequence d(k, £) is commonly referred to as the discrete wavelet transform (DWT) of f(t) [Daubechies 1992; Vetterli and Kovacevic 1995]. Observe that the DWT is still the transform of a continuous-time signal. The discretization is only in the a and b variables. In this sense it is analogous to the Fourier series, which involves a discrete frequency do- main representation of a (periodic) continuous-time signal. For this reason the DWT28 Discrete Wavelet Transform and Orthogonal Wavelet Decomposition 0.33 Ir t BB Reeion of panial overlap of ¥/(au) and F(w) EEE Region of complete overlap of ¥(aw) and F(w) — 4 Support of F(w) ak 20 Figure 2.2 Overlap of Fourier transforms of the signal and dilated wavelets. a 14 {2 1 I w Figure 2.3 The time-frequency cells that correspond to dyadic sampling.Approximations of Vectors in Nested Linear Vector Subspaces has also been referred to as a continuous-time wavelet series [Rioul and Duhamel 1992; Vetterli and Kovacevic 1995]. The description and derivation of the DWT is developed here entirely through an example involving the Haar wavelet as defined in Chapter 1. This is a precursor to descriptions of the DWT developed in a more general context in Chapter 3. 2.2 Approximations of Vectors in Nested Linear Vector Subspaces Representations such as the one in Equation 2.6 arise naturally in the context of what is called multiresolution analysis (MRA), which essentially amounts to constructing a hierarchy of approximations to functions in various subspaces of a linear vector space [Daubechies 1988; Mallat 1989a, 1989b]. We explain the concept of approx- imating vectors in nested subspaces by going through examples involving simple, familiar linear vector spaces before establishing the link between MRAs and DWTs. 2.2.1 Example of Approximating Vectors in Nested Subspaces of a Finite-Dimensional Linear Vector Space Consider the set of all N-dimensional, real-valued vectors of the form x = [x1, x2,..., xy]. This set forms an N-dimensional linear vector space and there exist N lin- early independent (not necessarily orthogonal) basis vectors aj, a2,..., an Such that any vector in this space can be expressed as a linear combination of these ba- sis vectors. In other words, there exist unique scalars ,02,...,a@y such that X = a; + aya; +... + aNan. Let us call this N-dimensional vector space Vy. We next look at approximating vectors in this N-dimensional space by vectors in a sub- space of a lower dimension. Suppose we generate all linear combinations of just N —1 of the N basis vectors, say ay, a2,...,aN—1. The set of all such linear combi- nations is also a vector space. It is of dimension N — 1 and every vector in it also lies in Vy. It is thus a vector subspace of Vy. Let us call this Vy_1. By continuing to drop the last basis vector at every step, we can similarly construct subspaces Vy-2,-..,Vi of dimensions N — 2,..., 1 respectively. The subspace Vj has as its basis the single vector a}. These vector spaces form a nested sequence of subspaces, VY.cVac...C Vy. Suppose we want to approximate a vector x in Vy by a vector in Vy_1. The best possible approximation in the minimum mean-squared error or least squares sense is made by choosing that vector (say xn-1) in Vy-1 for which the length of the error vector en-1 = X — Xn-1 is minimized. Such a vector is obtained by solving for en-3 in (en-1, ax) = 0 (2.7) 2930 Discrete Wavelet Transform and Orthogonal Wavelet Decomposition * ° a Figure 2.4 Finding the best approximating vector—a geometric analogy. where k = 1,2,..., N—1. To give a geometric analogy, xn-1 is the vector obtained by dropping a perpendicular from x to Vy-y as illustrated in Figure 2.4. The vector xN-1 is called the orthogonal projection of x on Vy-;. The error vector en-1 is orthogonal; that is, it is at “right angles” to every vector in Vy_1. Suppose we now continue this process of projecting throughout the entire se- quence of subspaces by projecting xn-1 on Vy-z2 to yield xn-2, and so on. At the end of the process we have a sequence of orthogonal projections of x in the sub- spaces Vy-1, Vw-2,..., Vi. These are xn-1, XN-2,---, X1 respectively. Note that an approximating vector x, is the orthogonal projection of not only x41 but also of the vectors Xx42,..-, XN-1 and x on Vj. With xn = x let ek = Xk+1 — Xk (2.8) for k = 1,...,N —1, denote the error between the projections on successive sub- spaces. Looking at the projections as approximations to the original vector x, we would expect the quality of the approximation either to stay the same or to get worse as we go from xn-1 to x1. Another way of stating this is that the subspace Vy-1 contains the finest approximations to vectors in Vy whereas V; contains the coarsest approximations. The act of breaking down a vector into such a hierarchy of approximations lies at the heart of MRA. The sequence of error vectors en-1,..., €1 also has an interpretation. The vector €n-1 can be seen as the amount of “detail” that is lost in x in going to its approx- imation xn-1 because x — ey-1 = Xn-1. The detail vector comes from that vector subspace of Vy with vectors that are all orthogonal to Vy_;. Let us call this vector space Wy_;. The dimension of Wy_; is one. We say that Vy_; and Wy- are orthog- onal to each other. Furthermore, since any vector in Vy can be expressed as the sum of a vector in Vy_; and a vector in Wy_1, we call these subspaces the orthogonal complements of each other in Vy. Similarly, ey_2 is the detail lost in xn_1 in going to the latter’s approximation xn-2 in Vy_2, and it comes from the one-dimensional subspace Wy-2, which consists of all vectors in Vy_; that are orthogonal to Vy-2.Approximations of Vectors in Nested Linear Vector Subspaces In general, the error vector at any level can be seen as the detail that is lost in going from a finer to a coarser level or, alternatively, as the detail to be added to an approximation at one level to move up to a finer level. The vectors en_1, €n-2,---, €1, X1 form an orthogonal set; that is, the inner prod- uct of any pair of these is zero. They belong to the one-dimensional subspaces Wy-1, Wy-2...., Wi, and V; respectively. Moreover, X=en-1 +en-2+...+e1+X1- (2.9) Thus the original vector is reconstructed from the coarsest approximation x; and the details at the various levels. Each of these detail vectors is an orthogonal projection of x on the corresponding one-dimensional subspace W; where i = 1,...,N—1. Our development shows that every vector in Vy can be written in the form of Equation 2.9; that is, as the sum of one vector apiece from the N subspaces Wy-1, Wy-2,-.-, Wi, and V;. Also, these vector subspaces are mutually orthogonal. We say that the vector space Vy is the direct sum of these subspaces. This is written Vy = Wy-1 ® Wy-2®.... Wi Ov. (2.10) As we will see, MRAs using wavelets fall into this framework in a natural fashion. 2.2.2 Example of Approximating Vectors in Nested Subspaces of an Infinite-Dimensional Linear Vector Space The concepts of the previous example involving a finite-dimensional linear vector space can also be extended to infinite-dimensional spaces. Consider the familiar Fourier series expansion of periodic signals. A real periodic signal of fundamental period T seconds with finite energy! over a period can be expressed as a linear combination of sines and cosines of frequencies that are integer multiples of the fundamental frequency 1/T Hz. The set of all periodic signals of period T with finite energy over a period forms a linear vector space. Let Vo denote the subspace generated by the constant or “DC” term. Let V, denote the subspace generated by the DC term, the fundamental, and all harmonics up to the nth harmonic. The basis for V, consists of 2n +1 “vectors”; namely, cos(2rkt/T) for k = 0,1,...,n and sin(2xk/T) for k =1,...,n. Thus V. is the linear vector space containing the periodic functions of interest. The basis vectors are mutually orthogonal under the ' The finite energy requirement leads to a less strict form of convergence than the Dirichlet conditions quoted in most texts on signal analysis. 3132 Discrete Wavelet Transform and Orthogonal Wavelet Decomposition definition of the inner product T/2 (x(t), ¥@) =/ rn dt (2.11) for x(t), y(t) € Veo. Here, in the previous example, we have a nested sequence of linear vector sub- spaces Vy CV, C.... Also, the concept of detail being gained in going from the approximation in V; to the approximation in V4; applies as well. In fact, this sim- ply amounts to adding a harmonic; that is, the sine and cosine functions at the harmonic frequency. Thus every periodic signal in Vo. is the sum of the coarsest approximation; namely, the DC component and the detail functions that are the various harmonics. 2.3 Example of an MRA Our third example of approximations in nested linear vector subspaces involves a wavelet and is an MRA [Chui 1992a; Daubechies 1988]. As we go through the example, the reader is urged to note the conceptual similarity of the constructs to those of the previous two examples. Let f(r) be a continuous, real-valued, finite energy signal. We will build approximations to this function at different levels of resolution using piecewise constant functions. Somewhere during this process we will obtain a representation for f(r) on the lines of the expression in Equation 2.6. Let us begin with an approximation that is constant over unit intervals. In other words, the approximating function has a constant value over the interval 0 —oo yields fO= Yo aw. (2.20) k 2.3.1 Bases for the Approximation Subspaces and Haar Scaling Function The subspace structure in this example differs from those in the previous two ex- amples in several of its properties. It is the possession of these differing properties, in addition to those in common with the previous examples, that makes the struc- ture of this example an MRA. We have already seen one difference: The coarsest approximation does not depend on the function f(t). It is always the zero vector. Consequently, the only common vector in all the approximation subspaces is the zero vector. This is expressed in terms of the intersection of the subspaces as co (1) % =(¢ (2.21) k=—00 where the g stands for the zero vector. Another property is that if a function a(t) € V;, then a(2t) € Vi-1 and vice versa? Yet another very interesting property, which we soon verify, is that the bases for all the subspaces in this structure can be generated using translations and dilations of a single function called the scaling function. ? This means that given a function that is piecewise constant over intervals of a certain length, then dilating the function by a factor of one half will make it piecewise constant over integrals of half this length, thus yielding a function in the next finer subspace. Doing the opposite (that is, dilating the function by a factor of two) will yield a function in the next coarser subspace.36 Discrete Wavelet Transform and Orthogonal Wavelet Decomposition Next we show that the basis for %; for any integer k is obtained by translations of an appropriately dilated version of the single function given by 1 O oy) (A) > w(t) 12 1 (B) Figure 2.6 Haar functions: (A) Scaling function. (B) Wavelet.Example of an MRA we have go(t)= D> dO, 2¥e- (2.38) t==00 which shows that go(t) can be expressed as a linear combination of integer translates of the Haar wavelet. Following the same steps as for g(t) we have the general result for any integer k, gl) = > dk, Qua -&) (2.39) =—00 where dtk,£) = Fletk ~ 1,20 ~c(k-1,2¢+ DI. (2.40) From Equations 2.23, 2.34, 2.36, and 2.40 we have y peer d(k, 2) = x ff Fw Ot = & dt. (2.41) ae Also, (WO — 0, wat — m)) =0 (2.42) for € # m. Equations 2.39 and 2.42 imply that for a specified integer k, the set {w(2-*t — £:€ integers}, obtained by dilating the wavelet y(t) by a factor of 2* and translating by integer multiples of 2*, provides a basis for the space containing the detail function g(t). Let us call this subspace Wy. Because W; contains vectors (again, piecewise constant functions) obtained as differences between vectors in V-1 and Vy, we express this symbolically as We = Ve-1 © Vie (2.43) Alternatively, Ve-1 = Ve ® Wee (2.44) This is read “V;_; is the direct sum of VY, and W,” and it simply means that every vector in V;_1 can be obtained as the sum of a vector from V; and a vector from 3940 Discrete Wavelet Transform and Orthogonal Wavelet Decomposition W,. Note that WC Vj_. Recall that the vectors in V, are orthogonal projections of vectors in V;_]. Consequently, the subspace W of details of the form g,(r) is orthogonal to V;; that is, every vector in W, is orthogonal to every vector in Vy. Each of these two subspaces is said to be the orthogonal complement of the other in V1. Carrying out the direct sum decomposition in Equation 2.44 further on Vj, Vi41, and so on, we have Veo = @B W; (2.45) =k which says that V;-; is the direct sum of the linear vector subspaces Wy, Wi+1, and so on. Repeated application of the argument used to explain the orthogonality of Vj, and W;, leads to the conclusion that the detail subspaces W,, k = 0,+1,+2, and so on, are all mutually orthogonal. The same conclusion would also follow directly from the fact that (wt — 2), ¥(2-"1 —n)) =0 (2.46) for k # m regardless of whether £ = n. The reader should notice the difference between Equations 2.42 and 2.46. The latter indicates that Haar wavelets with two different dyadic dilations are orthogonal regardless of their positions on the dyadic grid. This result is quite easy to verify. For example, if we consider k = 0 and k = 1, we see that the product of y(t — £) and (27's — n) always integrates to zero by virtue of the fact that the product is itself zero when the two functions do not overlap or, when they do for a pair of values (£,), the product is either w(t — £) or its negative. The two equations together imply that the set of functions (we*t-O: ke integers}, consisting of dyadic dilations and translations of the Haar wavelet, is an orthogonal set. Finally, when we put together Equations 2.20 and 2.39 we have f= YY dk ovets-o. (2.47) k=—20 L=—00 Equation 2.47 has a form identical to the one in Equation 2.6, indicating that we have obtained what we set out to do; namely, express the function f(t) in terms of a discrete wavelet representation. The orthogonality of the set of dyadically dilated and translated wavelets further makes this, by definition, an orthogonal wavelet decomposition or orthogonal MRA.Example of an MRA 2.3.3 Digital Filter Implementation of the Haar Wavelet Decomposition As we have seen, the coefficients of the expansion of the approximation at any level in terms of its basis functions are simply average values of the function f(t) over corresponding intervals. These are called approximation coefficients, and are obtained formally through the integration in Equation 2.23. Similarly, the detail coefficients d(k, £) can be obtained from Equation 2.41. However, as we will show, it is not necessary to perform the integration of the continuous-time function at every level. Given the coefficients at one level of approximation, the coefficients at the next coarser level of approximation, and by extension the coefficients at all coarser approximations, can be obtained by a digital filtering scheme [Daubechies 1988]. Generalizing Equation 2.31, the average value over a given interval at one level of approximation is merely the average of the two piecewise constant values of the next finer approximation over this same interval, (k,n) = sletk - 1.20) + e¢& -1,2n+1)] (2.48) where n indicates the sample number or discrete time. Now suppose we were to pass the sequence c(k — 1,) through a digital filter with impulse response ~ 1 A(n) = 368(n) + 8m + DI. (2.49) Denoting the filter output by y(k — 1,7), we then have v= 1.) = 5lelk — 1.) + elk = 10+ 1) (2.50) Note that c(k,n) = y(k - 1. 2n). (2.51) This is a result that says that c(k,n) is obtained by passing c(k — 1.) through the digital filter of impulse response h(n) and then retaining only alternate samples of the output. This process of retaining alternate samples is referred to as downsampling by a factor of two. The combination of filtering and downsampling is called decimation. Similarly, from Equation 2.40, with a(n) = 518¢n) —8(n+1)] (2.52) we can show that d(k, n) is obtained from c(k — 1. n) by passing the latter through a digital filter with impulse response g(n) and then downsampling by a factor of two. 4142 Discrete Wavelet Transform and Orthogonal Wavelet Decomposition Let us refer to the two previous digital filters as the H and G filters respectively. It is instructive to look at their frequency responses. The frequency response of His? Hw) = e/? cos(w/2) (2.53) whereas the frequency response of G is G(w) = jel? sin(w/2). (2.54) From the frequency response magnitudes plotted in Figure 2.7 we see that the H and G filters are (albeit poor) low-pass filters (LPFs) and high-pass filters (HPFs) respectively. From Equations 2.53 and 2.54 we have \A@)P+|H+zx)P =1 1G@)P +|G@+n)P =1 (2.55) |A(o)? + |G)? = 1. Being a real digital filter,’ A satisfies |A(@ +o)| =|A(r -o)]. (2.56) Similarly, G(x +)| = |G —@)}. (2.57) Substituting w = 7/2 — w into the first two equations of Equation 2.55 and using Equations 2.56 and 2.57 gives |A(x/2 + w)/? + |A (2/2 -@)? = 1 - - (2.58) |G(x/2 +o)? + |G(x/2 - w)/? = 1. Because the previous equations indicate a mirroring of the squared frequency re- sponse magnitudes about w = 7/2 (known as the quadrature frequency in various engineering applications), filters satisfying Equation 2.58 are called quadrature mirror filters or QMFs [Daubechies 1988; Mallat 1989a, 1989b]. 3 The notation here is that # without the argument w refers to the filter, whereas H with the argument denotes the frequency response function. The same is true of G. 4 By a “real filter” we mean that its impulse response is real-valued.Example of an MRA 43 1 0.9 0.8 0.7 0.6 \A(w) 0.5 0.4 03 0.2 0.1 0 (A) 0 0.5 1 1.5 2 2.5 3 1 0.9 0.8 0.7 0.6 IG)! 0.5 0.4 0.3 0.2 0. 0 (B) 0 0.5 1 1.5 2 25 3 o Figure 2.7 Frequency response magnitudes of the H (A) and G (8) filters. The range of the horizontal axis only covers 0 < w < x because the filters are digital with real impulse responses. The process of starting with a sequence of the approximation coefficients at some level of resolution and then generating the approximation and detail coefficients at coarser levels through decimation is referred to as a decomposition or analysis of the sequence. The reverse process of combining the coarser approximation and detail coefficients to yield the approximation coefficients at a finer resolution can be performed by digital filtering as well, and is referred to as reconstruction or synthesis. Let us develop the filtering construct for reconstruction by first looking at how c(—1, n) is obtained from the coarser coefficients c(0,7) and d(0, n).Discrete Wavelet Transform and Orthogonal Wavelet Decomposition From Equations 2.31 and 2.37, it follows that c(—1, 2n) = c(0,n) + d(0,n) (2.59) c(—1, 22 +1) = c(0,n) — d(0,n). Consider two filters called H and G with impulse responses given by h(n) = 8(n) +8(n —1) (2.60) g(n) = 6(n) — 6(n - 1) respectively. Now suppose we have sequence p(r) defined as _ fc@,n/2) neven PIM= 19 n odd. (2.61) That is, the sequence p(n) is obtained by inserting zeros between two successive values of c(0, 1). This process is called upsampling by a factor of two and is depicted in Figure 2.8. Let q(n) be a similarly upsampled version of d(0, 1): d(0,n/2) neven 0 n odd. (2.62) qin) = | Let r(n) be the output of filter H with p(n) as its input. Then from Equation 2.60, r(n) = p(n) + p(n - 1). (2.63) pin) Figure 2.8 Upsampling by c factor of two.Example of an MRA From Equation 2.61 (see also Figure 2.8) we see that p(n) is zero for odd values of n. Therefore, Equation 2.63 simplifies to _ | p(n) n even ra) = fe —1) nodd. (2.64) Putting Equation 2.61 in Equation 2.64 we get _ [ c(0, 2/2) n even r= | c(0,(n—1)/2) nodd. (2.65) The relationship expressed between r(n) and c(0, n) in Equation 2.65 simply amounts to creating r(n) by repeating each of the values of c(m) twice in succession. For example, r(0) = c(0,0), r(1) = c(0,0), r(2) = c(0, 1), r(3) = c(0,1), and so on. Suppose s(n) is the output of filter G with q(n) as its input. Then, following steps similar to those that led to Equation 2.65, we arrive at (0, n/2) n even —d(0,(n -—1)/2) nn odd. (2.66) s(n) = This means that s(n) is a sequence obtained from d(0,n) by repeating each of the values of d(0,n)s twice, but with alternating signs. For example, s(0) = d(0,0), s(1) = —d(0, 0), s(2) = d(0, 1), s(3) = —d(0, 1), and so on. We now add the outputs of the H and G filters: (0, n/2) + d(0, n/2) n even c(0, (n — 1)/2) —d(0,(n-1)/2)__ nn odd. (2.67) r(n) +s(n) = Let us try out a few values of n to understand the implication of Equation 2.67. Putting n = 0, 1,2,3 we get (0) +s(0) = c(0, 0) + d(0, 0) r(1) +s(1) = c(0, 0) — d(0, 0) r(2) +5(2) = c(0, 1) +. d(0, 1) r(3) +5(3) = ¢(0, 1) — d(0, 1). (2.68) Observe from Equation 2.59 that these values are c(—1,0), c(—1, 1), c(—1, 2), and c(—1, 3) respectively. Thus, c(-1,n) =r(n) +s(n). (2.69) 4546 Discrete Wavelet Transform and Orthogonal Wavelet Decomposition c(O.n) (0,n) Figure 2.9 A two-level decomposition and reconstruction system. In summary, the result states that c(—1,7) is obtained by adding the outputs of the filters H and G with the upsampled version of c(0,), and d(0,n) respectively as their inputs. The process of upsampling followed by filtering is called interpolation. The [H, G] pair is called the decomposition or analysis filter pair and the [H, G] pair is called the reconstruction or synthesis pair. The two filter sets are related by h(n) = 2h(—n) (2.70) g(n) = 2g(—n) from which it follows that these filters are also of the QMF type. The block diagram of a two-level decomposition and reconstruction arrangement is shown in Figure 2.9. Starting with c(0,n), we work down to c(2,n) and then back up to c(0,n), with intermediate detail coefficients d(1,n) and d(2,n) being generated in the process. Such filtering schemes and their relation to wavelet transforms are discussed in much more detail in Chapter 3. We conclude Chapter 2 by providing an example MRA using the Haar wavelet. Example 2.1 Consider the function F(t) = exp(—1/4)u(1) (2.71) where u(t) is the unit step function 1 120 Ee | 0 otherwise. (2.72)Examolets? an MRA f(t) 0.5 0 L 1 L T tea) (A) 0 2 4 6 8 10 12 14 16 1 T T T T T T T fnx(t) 0.5 0 (B) 0 2 4 6 8 10 12 14 16 © Folt) 0.5 a 0 ©) gh a ) F2(t) 0.5 gL ee 0 nv s a co S nD = a Figure 2.10 (A-F) MRA using the Haar wavelet. Plot A shows the plot of exp(—r/4). Plots B through F show piecewise constant approximations over intervals of length 1/4, 1/2, 1, 2, and 4 respectively. The average value of this function over the interval a Oandb>a is given by lexp(-a/4) — exp(—b/4)]. > rf exp(—t/4) dt = 4748 Discrete Wavelet Transform and Orthogonal Wavelet Decomposition 0.05 + 7 1 1 + + + 0 9 MIMI iann inn a 1 1 1 1 \ 1 : (A) 095 6 2 4 6 8 wo 12 14 16 8 ol + + 7 r + + 1 a) 0 Lf Loma ae - 4 ! \ \ \ © 2 4 8 10 612)si«dSSCSG 02 + r 1 t + sl) 0 | a -02 i L \ 1 L \ m7 () 0 2 4 6 8 1 ~612)~SCOdSSSC«*G Figure 2.11 (A-D) Detail functions g_}(r) through go(r). Then, by our definition in Equation 2.23, we have kg 2 c(k, €) = sex (- =) f — exp (-3)] 7 (2.74) The approximation f,(r) at resolution level k is given by AM=ckO. Be <1< 2% e+. (2.75)Example of an MRA 0.05 T T T T 4.0740e — 5) 0 IANNIS =0.05 1 1 1 L (a) 0 2 10 12 14 16 0.1 + + + 1 1,6325e — 4 0 Eat ; PEELE (B) 0 2 10 12 14 16 0.2 T 7 T T 6.5042e — 4 9 ANAK ~02 1 1 : : (c) 0 2 10 12 14 16 02 + + _—— 0.0026 : { 02 \ 1 ‘ i (D) o 2 0 2 14 16 05 + 1 1 1 Pee ee 0.0095 0 -05 “1 : . 1 ©) 0 2 10 12 14 16 > Figure 2.12 (A-E) Error between the original function and the approximations shown in Figure 2.10. Figure 2.10 shows these approximations over the range 0 <1 < 16 for values k = ~2,-1,0, 1,2. The detail functions are shown in Figure 2.11, whereas the approxima- tion error between the original function f(t) and the approximations at various levels are shown in Figure 2.12. By simply looking at the amplitude swings of the error plots it is possible to see that the approximation error increases with increasing k; that is, with increasing coarseness. We have also included the mean squared error com- puted over the interval 0 f(t) as k — 00 for any t=. . Suppose in Problem 2 that the function f(r) is discontinuous and the number of discon- tinuities is finite. Then show that f_.o(to) # f (to) if f(1) is discontinuous at to. Further, show that the integral of the squared error between f(r) and f_.0(t) is still zero. This form of convergence is known as convergence in the mean-squared sense. . Let f(r) be piecewise linear approximations to a continuous, finite-energy signal f(z) over intervals of length 2". Let 1. = 2*€ for integer values of k and @. Then f(t) = (Cf eat) — fF te.0))/(hen — te.) /f (e.e)- Can you find a function $(1) such that its integer translates form a basis for fo(1)? What would be the basis for f,(t)? Hint: Unlike the Haar scaling function, which has support over 0 <1 < 1 and whose integer translates were used to generate the basis functions for fo(r) in Equation 2.22, the function ¢(r) for this problem has support over 0 <1 <2. . Suppose the input to the LPF decimator at level zero in Figure 2.9 consists of a sequence of length eight, say x(0), x(1), .... x(7), and the output after downsampling at level one are the four values denoted y(0), y(1). y(2), y(3). If we write y = Hx for some matrix H, what is H? Here, x = [x(0), x(1),..-.x(7)I” and y = [y(0), y(1), y(2), (3)]7. Now suppose that the y values are interpolated back to level zero using the LPF interpolator to yield the eight values we call z(0), z(1), .... 2(7). Let z = Hy where z = [z(0), 2(1), ..., 2(7)]”. What is H? What is the relationship between the two matrices?CHAPTER THREE MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks 3.1 Introduction Chapter 2 introduced the concept of the DWT. What is common to the two forms of the wavelet transform seen so far (the CWT in Equation 1.55 and the DWT in Equation 2.6) is the fact that the given signal is expressed in terms of translations and dilations of a single mother wavelet. Indeed, the theme of this book is that for anything to be discussed in the wavelet transform setting, these two elements must be present. In other words, no translation and no dilation means no wavelets. The introduction to the concepts of DWT and MRA in Chapter 2 was through a particular wavelet—the Haar wavelet. DWTs would not be of much use if only this one wavelet existed. It is natural, therefore, to pose questions such as: Are there mother wavelets other than the Haar that can be used to express a function using Equation 2.6? How would one go about constructing such wavelets? Is there always an interpretation in terms of a multiresolution decomposition for such a DWT? The purpose of this chapter is to address these questions by deriving results that characterize all orthogonal wavelet decompositions. An understanding of this chapter will enable the reader to comprehend the material on applications in subsequent chapters. Chapter 2 showed a link between the DWT and subband filtering. A fuller development of that link is provided in this chapter. The derivations have been kept as simple as possible and key results are presented in italics. This was done to let the reader who chooses to focus on the results, and not worry about the detailed derivations, do so without much inconvenience. Also, some of the steps in certain derivations are not presented within the main body of the text but are posed as exercises at the end of the chapter. 5152 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks 3.2 Formal Definition of an MRA First we provide a precise definition of an MRA by its defining properties or axioms. As we have seen, an MRA involves approximation of functions in a sequence of nested linear vector spaces. Not every sequence of nested vector spaces yields an MRA. We saw in Chapter 2 that in addition to the nesting property, the vector spaces need to satisfy certain other conditions. We formalize these conditions here [Daubechies 1988, 1992; Mallat 1989a, 1989b]. The definition of an MRA is the following: An MRA consists of the nested linear vector spaces ...C 4 CW) C V1 C... such that 1, The union of these subspaces is dense in the space of square integrable functions DR) 2. The intersection of these subspaces is a singleton set containing the all-zero function or zero vector 3. If f(t) € YW then f(2r) € V1, and vice versa 4. There exists a function (called a scaling function) @(t) such that {o(t — k) : k integer} is a basis for Vo There are alternative logically equivalent axioms that can be used to define an MRA [Vaidyanathan 1992]. We next examine the meaning of these statements. The first axiom mentions one set being dense in another. For a rigorous treatment of the concept of denseness the reader is referred to texts on mathematical analy- sis or topology (see, for example, Royden [1978] and Rudin [1976]). We provide a simple explanation of the concept that, although not very rigorous, should suffice for an understanding. Suppose A and B are two sets of real numbers such that AC B. The set A is said to be dense in B if for every element b € B we can find an element in A that is as close to b as we please. The most straightforward example of this notion is provided by the set of rational numbers Q and the set of real numbers R. The set Q is dense in R since for every real number x, it is possible to find a rational number that is as close to x as desired. Stated another way, it is possible to find a sequence of rational numbers qi, g2, and so on, such that as n > oo we have qn — x. It is important to observe that although q, € Q for any n, the real number x does not have to be rational. A classic example is provided in the definition of e, the base of the natural logarithm, as e= lim ( + *) (3.1) n=00 n where the limit is taken as n steps through positive integer values. The right-hand side of Equation 3.1 is a rational number for any finite n, but the limit e is not. The fact that Q is dense in R lets us make approximations to irrational numbers with rational numbers in practical situations, such as numeric computation. For example,Construction of a General Orthonormal MRA we do not use the true values of x or /2 when we perform computations on digital machines. In this sense, density lets us treat Q as being almost as good as R, although mathematically the two sets are very different in their properties [Royden 1978]. Thus the first axiom of the MRA definition implies that the union of all nested linear vector spaces yields a space (let us call this set V_,.) that is not the same as the space of signals with finite energy, L?(R), but is dense init. In other words, for every signal f(t) € L?(R) there is a signal in V_.. that is as close to f(r) as we please. Here the closeness of two signals is measured by the energy in their difference. Thus, we can regard V_.. as being as good as L?(R) for all practical purposes. he second property states that the only signal common to all the vector spaces is the all-zero signal or the zero vector. The third property brings in dilation and states that a factor-of-two dilation of a vector belonging to a subspace at a certain level yields a vector in the next coarser subspace. Conversely, dilating by a factor of one half yields a function in the next finer subspace. It should be noted that the notion of coarseness here has been generalized from the Haar example in Chapter 2. Vector space V; is coarser than V, ifi > j. The final property requires that there be a scaling function ¢(r) such that the set {¢(¢—n): n integer} is linearly independent, and any function f(t) € Vo is expressible as co Jolt) =~ a(0,n)otr —n) (3.2) n==00 for a sequence of scalars a(0,n) where n = 0, +1, +2, and so on. It should be noted that the statement does not call for the set of ¢(r) and its integer translates to be orthogonal. However, an important class of MRAs results when they are orthogonal (Daubechies 1988]. Also, there is no mention of a wavelet in the definition of an MRA. Yet, as shown in this chapter, an MRA provides a natural setting for the DWT. Chapter 2 first presented an MRA and then derived the bases for the approximation subspaces. In Section 3.3 we do the opposite. We start with the scaling function and then construct an MRA that satisfies the properties listed at the beginning of this section. 3.3. Construction of a General Orthonormal MRA 3.3.1 Scaling Function and Subspaces Let ¢(t) be a function that satisfies the following: 1. It integrates to one!: | o(t) dt =1. (3.3) | Although we explicitly state this requirement, it actually follows from the next two requirements and Equation 3.10. 5354 — MRA. Orthonormal Wavelets, and Their Relationship to Filter Banks 2. It has unit energy: ioc = f Ie@P dt =1. (3.4) 00 3. The set consisting of ¢(t) and its integer translates is orthogonal*: (@@), O( —n)) = 5). (3.5) Because a set of orthogonal vectors is linearly independent, the set {(r — k) : k integer} consists of linearly independent vectors. Therefore, it serves as a basis for a linear vector space that we will call Vo. Substituting 2~*r for 1 in Equation 3.5 we get (@Q*1), 621 — n)) = 25(n) (3.6) which indicates that the set (g(2“1 —@) : € integer} (3.7) is also an orthogonal set for a given integer k. Let us call the linear vector space generated by this set V;. Similar to the subspaces of the Haar wavelet decomposition in Chapter 2, VY, is a “coarser” vector space than Vi-1. The subspaces ..., Vi, Vo, V-1, and so on, satisfy? F(t) Ee @ F2t) € Va (3.8) Suppose we now add the condition YC Via. (3.9) The condition in Equation 3.9 requires every vector in Vp to belong to V_; as well. In particular ¢(t), which is in Vo, must also be in V_;. Thus it should be possible to express $(1) as a linear combination of the basis for V_;, namely, {¢(2t — n) : n integer}. With this additional property, $(r) qualifies as a scaling function. This property is stated as the following dilation equation [Daubechies 1988; Mallat 1989a, 2 Im this particular instance we have a specialized form of orthogonality called orthonormality, which refers to the fact that, in addition to orthogonality, we have set ||(r — k)|| = 1 in Equation 3.4. 3 Read the symbol « in Equation 3.8 as “implies and is implied by.”Constnittion:of a General Orthonormal MRA 1989b; Strang 1989]: There is a sequence of scalars c(a) n =0,+1 £2, and so on, such that ~ (0) = J? e(n)p(2t —n). (3.10) n=—0o In Equation 3.10 we have used the fact that the basis for V_; is given by translates of ¢(24) by integer multiples of half-unit intervals. Since ¢(t) is expressed in terms of its own dyadic dilation and translation, Equation 3.10 is referred to as a dilation equation or a two-scale difference equation [Daubechies 1992; Daubechies and Lagarias 1991, 1992]. Replacing 1 in Equation 3.10 by 2~*t, we have oo ot) = S> e(nyg(2-@-2 = ny (3.11) n=—20 which implies that the basis for 4. is contained in 1. In other words, the linear vector spaces ..., Vi, Vo. V-1,... form a nested sequence: Vie C Vet. (3.12) Thus the condition in Equation 3.9 is equivalent to a nesting of all the subspaces involved. The union of the subspaces is dense in L?(R). The steps involved in the proof of this statement are presented as an exercise (see Problem 2 at the end of this chapter). At this stage we have an MRA because all the axioms for one are satisfied. Observe that the initial set of properties of $(t), along with the dilation equation, was sufficient to arrive at an MRA. 3.3.2 Implications of the Dilation Equation and Orthogonality A constraint on the coefficients: By integrating both sides of the dilation equation, Equation 3.10, with respect to t and canceling the common terms we get 3 DS ce =2. (3.13) n==00 From Equations 3.6 and 3.10 we have (O(t), b(t — £)) = d(0) (3.14) (6), 6-8) = ; Do compe(m — 26). (3.15) m=—0o 5556 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks Define R(t) = (6(1), @ — 7) (3.16) the autocorrelation function of ¢(r). Equation 3.14 states that R(t) is zero for all integer arguments except t = 0. Let O() = FId(t)] (3.17) be the Fourier transform of ¢(r). Then, the orthonormality condition in Equa- tion 3.14 has the frequency domain equivalent 2 Dd 1o@ +270)? =1 (3.18) BoC which is referred to as the Poisson summation formula [Chui 1992a]. Equation 3.15 says something about the autocorrelation sequence of the coefficient sequence c(n). Defining the autocorrelation sequence 2 rel€) = D> c(me(m = &) (3.19) we have, according to Equation 3.15, r,(2€) = 25(€) (3.20) which says that the autocorrelation sequence of c(n) is zero at even lags except lag zero [Daubechies 1988; Strang 1989]. Let us now look at a corresponding frequency domain interpretation. Let oo. Cw) = Yo eine" (3.21) n==00 be the Fourier transform of the sequence c(n). Then the frequency domain equiva- lent of Equation 3.20 is* IC(w)}? + |\C( +x)? =4. (3.22) 4 See Problem 8 at the end of the chapter. The following result comes in handy: If X(w) is the Fourier transform of a discrete time signal x(n), then the Fourier transform of x(2n) is [X(w/2) + X(w/2 +7)]/2.Construction of a General Orthonormal MRA This is often called the power complementarity condition [Vaidyanathan 1990, 1992; Vetterli and Herley 1992; Vetterli and Kovacevic 1995]. Suppose we have a function f(t) from which we obtain f,(t), k = 0, £1, £2, and so on, the orthogonal projections of f(t) on the subspaces VY, k = 0, +1. £2, and so on. Let us represent f;(t) in terms of the basis functions in Equation 3.7 as Silt) = YO alk, nyo" =n). (3.23) nso The sequences a(k,n) are the coordinates or coefficients of the expansion of the functions f;,(t) in terms of their respective basis functions. We refer to f,(t) as the approximation at level k. Let us now establish a relationship between the coefficients at one level of approximation and the next. We start with the particular case of f_;(t) and f(t). From Equation 3.23 fall) = D> a(-1,meer =n) (24) folt) = S* a, n)o(r -n). 3.25) Because fo(t) is the orthogonal projection of f_(t) on Vp, the error or detail function Bolt) = f(t) — folr) (3.26) is orthogonal to the subspace Vp and, in particular, to its basis. Therefore, (f-r@ — folt), 6@-n)) =0 (3.27) for integer n. Equivalently, (folt), @@ —n)) = (fr), 6 —n)). (3.28) From Equation 3.25 and the orthonormality of ¢(t), and its integer translates in Equation 3.5, it follows that the left-hand side of Equation 3.28 is given by (fo(t), o(t —n)) = a(0, n). (3.29) Next we evaluate the right-hand side of Equation 3.28. From Equation 3.10, co (P(t), b(2t — m)) = >» e(n)(b (2t — m), 6(2t — n)) (3.30) n=—00 5758 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks Using Equation 3.6 in the right-hand side of Equation 3.30 we have (@(), 621 — m)) = S (3.31) From Equation 3.24, (f-1(t), @(¢— n)) = xy a(—1, m)(¢(t — n), d(2t — m)) (3.32) m=—00 which, in combination with Equation 3.31, yields (m,ee-m)= 3S a(t, mE. (3.33) m=—00 Substituting Equations 3.29 and 3.33 in Equation 3.28 we get a(0,n) = Sect, my (3.34) Our derivation started with the particular case of f_;(r) and fo(t), and led to the relationship between a(0,n) and a(—1,n) in Equation 3.34. Starting with the more general case of f,_1(t) and f;(r), and following the previous steps, leads to the relationship cS a(k,n) = DL a(k —1,m) m=—00 c(m — 2n) (3.35) 2 where a(k,n) are coefficients of the expansion of fi (t) in terms of the basis for Vy; as in Equation 3.23. Equation 3.35 reveals that with knowledge of the coefficients c(n) in Equation 3.10 it is possible to obtain the coefficients a(k, n) at one level using the coefficients at the next finer level. The interpretation of the operation in Equation 3.35 is discussed in greater detail in Section 3.5. 3.4 A Wavelet Basis for the MRA We now turn our attention to the spaces containing the detail functions. Define 8x) = feat) — A) (3.36) as the detail function at level k. The functions g,(t) and f,(t) are orthogonal to each other. Recall that in the case of the Haar wavelet decomposition, the detail functionA Wavelet Basis for the MRA at a particular level was expressed in terms of translates of a dilated version of the Haar wavelet. We now try to construct a similar expression for the general case. Suppose y(t) is a function in V_; satisfying the following: | w(t) dt =0 E WP de =1 (¥@), v(t —n)) = d(n) (1), O(t =n) = 0. (3.37) The function y(t) is a wavelet that provides the DWT corresponding to this MRA. The rest of this section is devoted to establishing this fact. 3.4.1 Two-scale Relation for (ft) The second and third requirements in Equation 3.37 state that y(t) and its integer translates form an orthonormal set. Also, the last condition in Equation 3.37 requires w(t) to be orthogonal to all vectors in Vp. Because w(t) is in V_;, it can be expressed in terms of the basis of the subspace [Daubechies 1988]. Let d(x), » = 0, £1, +2, and so on, be the corresponding coefficients so that oo. Yo ager — 2) (3.38) =—00 voO= which is similar in appearance to Equation 3.10 and is a two-scale relation for y(t). There are constraints on d(n) that follow from these conditions. Integrating both sides of Equation 3.38 and using the fact that the integral of y(t) is required to be zero leads to co YS din) =0. (3.39) namo Similar to Equation 3.15 we have 1 @ : x d(m)d(m — 28) = 6(€) (3.40) which means that the autocorrelation of d(n) is zero at all even lags except at lag zero [Daubechies 1988; Strang 1989]. Using Equations 3.10 and 3.38 along with the 5960 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks mutual orthogonality of y(t) and translates of (1) we have, by proceeding on lines similar to the derivation of Equation 3.15, (W(t), o@ — £)) = 0 5 > d(m)c(m — 2€) m=—00 (3.41) which implies that the cross-correlation between d(n) and c(n) is zero at even lags [Daubechies 1988; Strang 1989]. Equations 3.15, 3.40, and 3.41 imply that the set of sequences formed by the even shifts of c(n) and d(n) is an orthogonal set. We now move to the frequency domain. Defining Dw) = > dine" (3.42) we have the frequency domain equivalent of Equation 3.40 as |D@)? +|D@+mP=4 3.43) and that of Equation 3.41 as C(w) D*(w) + C@+7)D*(w+7) =0. (3.44) Equations 3.22, 3.43, and 3.44 can be written compactly in matrix form as Cw) Cwt+x)]f C@) D*(w) _f4 0 (3.45) D@) Dwt+x)| | Cw+xz) Diw+m | [0 4] ; By interchanging® the matrices on the left-hand side of Equation 3.45 we have Cw) D*(w) C@) Cww+z)|_[4 0 (3.46) Cit) Diw+x2)},|D@) Dwt+mj Lo 4 : which is known as the paraunitary condition [Burrus, Gopinath, and Guo 1997; Vaidyanathan 1990, 1992; Vetterli and Kovacevic 1995]. Carrying out the matrix 5 Although matrix multiplication is not commutative in general, it works when the product is a multiple of the identity matrix, as in this case.A Wavelet Basis for the MRA multiplication on the left-hand side of Equation 3.46 and equating it to the right- hand side, we get the following®: IC(w)? +|D(@)P = 4 (3.47) C*(w)C(w +7) + D*(w)D(w+ 7) = 0. (3.48) On taking the inverse Fourier transform of Equation 3.48 we get the corresponding time domain equivalents: d= ceein $e) + D> dn)d(n + 2) = 45(0) (3.49) Dd preimen +.2) + D2 (-"d dn + &) = 0. (3.50) In deriving Equation 3.50 from Equation 3.48 we have used the fact that translating the Fourier transform by z radians in the frequency domain is equivalent to multi- plying the corresponding time domain sequence by (—1)". Adding Equations 3.49 and 3.50 yields D> c@ndeQn +2) + Y> dQnjd(2n + &) = 45(0. (3.51) n=—co n=—00 3.4.2 Basis for the Detail Subspaces Let 62, » 7 pute y+ » se yttn n). (3.52) n=—00 n=—20 v(t) Using Equations 3.10 and 3.38 in the right-hand side of Equation 3.52 we get ie v(t) = ie SO compar +20 =m) + ee (3.53) > > oC demyo(2t +20 — m) m=—00 n=—00 ® Even though the matrix multiplication yields four equations, only two of these results are independent. 6162 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks which expresses v(t) as a linear combination of ¢(2r) and its translates. Substituting m = 2n + € in Equation 3.53 we obtain v(t) = ; YS | YE (endeQ@n + &) +d (2nd Qn +20))} 21 - O. (3.54) n=—00 | £=—00 Substituting Equation 3.51 in Equation 3.54 we see that v(t) = ¢(2r). (3.55) Thus ¢(2r) is the same as the function v(t) defined in Equation 3.52, indicating that ¢(2t) (and hence any of its translates by half-unit intervals) is a linear combination of integer translates of $(r1) and y(t). Since {¢(2t—£) : £ integer} is the basis for V_1, the implication then is that every vector in V_; is expressible as a linear combination of integer translates of ¢(r) and y(t). That is, in general, f_1(t) can be expressed as fal) = S° aO,n)@-n)+ D> 60,m)ye =n) (3.56) n=—00 n=—00 for some pair of sequences a(0, 7) and b(0,n). The first sum in Equation 3.56 is fo(t) as given in Equation 3.25. By Equation 3.26, the second sum is go(r); that is, 0. golt) = D> bO.n)y(t —n). (3.57) n=—00 Just as fo(t) is in the space Vo spanned by ¢(t) and its unit translates, go(t) is in the subspace spanned by y(r) and its integer translates. This subspace can be regarded as the difference between the subspaces V_; and Vo. Although we have established a relationship between a(0,n) and a(—1,n) in Equation 3.34, we have not done so for b(0,). We next derive a relationship between a(—1,n) and b(0, n). Taking the inner product of both sides of Equation 3.56 with respect to y(t — n) we have (f(t), Wr = n)) = (go(t), We — 0). (3.58) The term involving fo(t) drops out because of the orthogonality of ¢(r) and its translates to (r) and its translates. From Equation 3.57 and the orthonormality of w(t) and its integer translates, it follows that the right-hand side of Equation 3.58 is given by (go(t), W(t —2)) = b(0,n). (3.59)A Wavelet Basis for the MRA We next evaluate the left-hand side of Equation 3.58. From Equation 3.38, ~~ (2), @(2t —m)) = ee d(n)($(2t — m), 6 (2t —n)). (3.60) Using Equation 3.6 in the right-hand side of Equation 3.60 we have d( (W(.62e =m) = 2. (3.61) From Equation 3.24, (Fa, ve =m) = D> a(-1, mh = 0), bt —m)) (3.62) which, in combination with Equation 3.61, yields co. (Fx, We=m) = Yo atm, (3.63) m=—co Substituting Equations 3.59 and 3.63 in Equation 3.58 we get bO,n)= >> a(-1, SE) m=—c0 (3.64) Observe the similarity between the results in Equations 3.34 and 3.64. By general- izing the result we have bn) = So ak - 1m). (3.65) 3.4.3. Direct Sum Decomposition Let Wo be the linear vector space that has the set {w(t —n) : n integer} for its basis. According to Equation 3.57, Wo is the vector subspace of detail functions at level zero. Equation 3.56 implies that every vector in V_; can be expressed as the sum of a vector in Vo and a vector in Wo. Also, the two subspaces Vp and Wo are orthogonal complements of each other in the space V_; [Daubechies 1988, 1992; Strang and Nguyen 1996]. Using the direct sum notation of Chapter 2 we can write V-1=VW@ Wo. (3.66) 6364 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks The result is generalized easily to other levels as VY = Ve BW (3.67) where W;, is the subspace generated by the set {w@-*t—n)in integer) for a given integer k. Applying the direct sum recursively we have [Daubechies 1988, 1992; Strang and Nguyen 1996] © Yin = Q Wi (3.68) =k If we let k — —oo in Equation 3.68, Vio= QB W;- (3.69) In other words, given a square integrable function f(r), we have feolt)= > > bk, Over -O. (3.70) =—00 f=—00 It is clear from the discussion in Section 3.2 that Equation 3.70 gives the best possible approximation to the given input function f(r). In fact, for most signals of interest, F(@) = f-eo(t). Thus, y(t) is a wavelet for the MRA associated with ¢(r). The dyadic dilates and translates of y(t) form an orthogonal basis for a linear vector space that is dense in L?(R). Such a basis is called a Riesz basis’ of L?(R). 3.5 Digital Filtering Interpretation 3.5.1 Decomposition Filters Let h(n) = cer) + (3.71) 2 g(n) = “2 » (3.72) h(n) = h(-n) _ 3.73) &(n) = g(-n) (3.74) 7 A formal definition of a Riesz basis and a related discussion are provided in Appendix B.Digital Filtering Interpretation where c(n) and d(n) are coefficients of the two-scale relations in Equations 3.10 and 3.38 respectively. We can now rewrite Equations 3.35 and 3.65 as [Daubechies 1988, 1992] oo a(k,n) = > alk -1.m)hQn — m) (3.75) mance oo b(k,n) = > alk -1,m)gQn-m). (3.76) Equation 3.75 shows that a(k,n) is obtained from a(k — 1,n) by convolving the latter with the sequence h(n) and retaining only the even indexed samples. In other words, by passing a(k —1,n) through a digital filter with impulse response f(n) and retaining every other sample, we obtain a(k,n). As mentioned in Chapter 2, this process is called decimation by a factor of two. Similarly, b(k,n) is obtained by convolving a(k —1,n) with the sequence g(n) and downsampling by a factor of two to retain only the even indexed samples. We continue to use the notation of Chapter 2 of using the same uppercase letter as the impulse response to denote the filter. For example, H stands for the filter with an impulse response of h(n), H stands for the filter with an impulse response of A(n), and so on. Let us now examine properties associated with H and H. By Equation 3.13 and the definition of h(n), dian) =1. (3.77) We deliberately left the limits unspecified in the summation in Equation 3.77. If H is an infinite impulse response (IIR) filter, then the limits could conceivably extend from —oo to oo, in which case the filters would be noncausal. If the filter is of the finite impulse response type, then the summation extends over a finite range of values of n. Let H(w) = Yo Ate" (3.78) be the frequency response of the filter H. Then, by Equation 3.77, H(0)=1 (3.79) which says that the frequency response at DC is unity. Also, from Equation 3.22, |H(w)? +|H( +2)? =1. (3.80) 6566 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks IH(w + m)P Figure 3.1 Squared magnitude frequency responses of H(w) and H(w +7). Equation 3.80 implies that H is a QMF, that is, |H(w + x)| is a reflection of |H(w)| about the quadrature frequency (namely, w = 1/2). However, not every QMF satisfies the power complementarity property. A QMF that satisfies that property is known as a perfect reconstruction quadrature mirror filter (PRQMF) for reasons soon to be explained. The PRQMF nature of H is a direct consequence of the dilation equation and the orthogonality of the set {¢(t —n): n integer). An example of the squared magnitude frequency response of such a filter is shown in Figure 3.1 along with |H(@ + 7)|?. Since h(n) is a reflection of h(n) about n = 0, the magnitude frequency response of H is the same as that of H, and we have |A(o)P + |H(o + 2)P =1. (3.81) Thus is also a PRQME. Using A) = H(0) = 1 in Equation 3.81 yields A(z) = H(x) =0. (3.82) Based on the fact that their DC gain is unity and the frequency tesponse at w = is zero, we might conclude that H and H have the characteristics of LPFs.6 The condition in Equation 3.81 is also known as the halfband condition. A filter with a frequency response equal to |H(«)|? is said to be a halfband filter. 8 As we saw in the Haar case, they need not be good LPFs. See Figure 2.7 for plots of corresponding filter frequency responses for the Haar case.Digital Filtering Interpretation Equation 3.76 shows that the sequence b(k, 7), like a(k,n), is also obtained by filtering a(k — 1,n) with the filter G and downsampling the filtered sequence bya factor of two. The G and G filters are also QMFs on the basis of Equation 3.43 and the definition of g(n) and g(n); that is, IG)? +|Gw+n)P =1 (3.83) IG@)P + |G +x)? =1. (3.84) From Equation 3.39 and the definition of G and G, G(0) = G0) =0 (3.85) which, in conjunction with Equations 3.83 and 3.84, yields 1GGr)| = |G@x)| = 1. (3.86) Equations 3.85 and 3.86 show that the G and G filters have the characteristics of HPFs. Expressing Equation 3.44 in terms of the filter frequency responses we have H(w)G*(@) + H(@+7)G*(wo+7) =0 ae (3.87) H(@)G*(@) + H(w+7)G(o+ 7) =0. Also, from Equations 3.47 and 3.48, |H(w)? + |G(o)P = 1 H*(w)H(w+7) + G*(w)G(wtn) =0 ssemraeetoaunant (3.88) IH) + |G@)/F = 1 At (w)H(w +2) + G*(o)G(w +7) = 0. It is important to observe that the sequences b(k, n) in Equation 3.71 are in fact the coefficients of the DWT in Equation 3.70. The time domain equivalent of Equation 3.87 is obtained by substituting Equa- tions 3.71 and 3.72 in Equations 3.15, 3.40, and 3.41: So h(mph(n $ 2k) = Pt) (3.89) diggin + 2) = 55) (3.90) YD Aldgin $ 2k) = 0. (3.91) 6768 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks The equations imply that the LPF impulse response h(n) is orthogonal to even translates of itself, the HPF impulse response g(m) is orthogonal to even translates of itself, and h(n) is orthogonal to g(7) and its even translates. A similar set of conditions holds for h(n) and 3(n) because they are proportional to reflections of h(n) and g(n) about the origin respectively. 3.5.2 Reconstructing the Signal From Equation 3.56 and the two-scale relations for the scaling function $(r) and wavelet (1), we have oe fat) = >» » a(0, £)c(m)p(21 — 26 — m) + mao b=— oe Sd YE 6, Odem)o er — 26 - m). misao (= —00 Substituting » = 2¢ +m in Equation 3.92 we get fat) = YOY a, Helm - 2€)G2r = 0) + n=—06 f=-90 dS YS 00, Ode - 20)42 - 0). n=—% f=—90 From Equations 3.23 and 3.93 we have Ea oo a(-1,n) = > a, cin = 20 + D> bO, Hain — 26) (2-00 =—00 which can be rewritten in terms of h(n) and g(n) as oe oe a(-1,) =2 D> a0, Ohm — 2 +2 D> dO, Hgin — 26). f==00 (==00 Let 2 (0,2) = Y° a(0, £c(n — 28) a b@.n) = > v0. e)d(n — 28) f= (3.92) G.93) (3.94) (3.95) (3.96)Digital Filtering Interpretation Now consider the expression for 4(0,n) in Equation 3.96. Expanding the right- hand side of this expression we get a(0,n) =... + .a(0, 0)c(n) +.a(0, Ic(n — 2) +.a(0, 2)c(n —4) +... (3.97) This can be rewritten as a(0,n) = ...+ (0, 0)c(n) + Oc(n — 1) + a(O, I)c(n — 2) + Oc(n — 3) + a(0, 2)c(n—4) +... (3.98) Equation 3.98 can be interpreted as inserting zeros between adjacent samples of the sequence a(0,n) and filtering the resulting sequence with c(n). As described in Chapter 2 this process of zero insertion is called upsampling and is depicted in Figure 2.8. An alternative interpretation is that of interpolating a(0,n) using the filter H, followed by scalar multiplication by a factor of two? We refer to this process as interpolating a(0,n) with 2H. In a similar fashion, the sequence b(n) can be obtained by a factor ot two interpolation of b(0,n) using G, followed by scalar multiplication by two. Let us refer to this process as interpolating b(0,n) with 2G. Therefore by Equation 3.94, a(—1, 7), the sequence of coefficients used for expanding f_1(t) in terms of the basis for V_;, can be obtained from a(0, n) and b(0, n) by summing their scaled interpolations using the H and G filters respectively. Equations 3.95 and 3.96 can be generalized to any integer k: a(k,n) = 2a(k +1, n) + 2b(k +1, ny (3.99) where co ak+1n) = D> alk +1,He(n - 28) ate (3.100) : oo bk+1n) = Yo bK+1,0d(n - 20). t=-00 Figure 3.2 shows the general structure of decomposition and reconstruction. The H and G filters that are associated with decomposition form what is known as the analysis filter bank. Similarly, the H and G filters that effect reconstruction form the synthesis filter bank. Because the sequence going into the analysis filter bank is 9 Some authors define h(n) = c(n)/V(@ and g(n) = d(n)/V@._With h(n) = h(n) and §(n) = g(-n), this definition leads to decomposition being effected using the H and G filters, and reconstruction being carried out with the H and G filters without a factor of two multiplication. 6970 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks Ld 2s. tte a(k,n) ak ak 1) Tien) ~{) —{} | anny | 28 i ‘atk = 1.n) >| ¢(—n) -{) —} >} 2e(n) b(k,n) 2 (km) 2 7 Analysis bank Synthesis bank Figure 3.2 Analysis and synthesis filter banks. recovered perfectly by the synthesis bank, these filters form a perfect reconstruction pair (hence the name perfect reconstruction quadrature mirror filter banks). 3.6 Examples of Orthogonal Basis-Generating Wavelets So far we have seen one concrete example of a wavelet that can provide an orthog- onal decomposition—the Haar wavelet. But there is an infinite number of wavelets that can provide an orthogonal basis. Associated with these wavelets are scaling functions. As in the general development of the previous sections, the scaling func- tion defines the MRA and the associated filters H and H. More importantly, the wavelet is derived in a simple manner from the scaling function, as we show next. Suppose we have a scaling function ¢(r) that satisfies all the properties associated with the scaling function of Section 3.3. Then, from the definition of the frequency response of H and Equation 3.82, Eph) = 0. (3.101) 7 Suppose we set g(n) = (—1)'"A(1 — 2). (3.102) Then, Equation 3.102 and the fact that H and H satisfy power complementarity imply that G and G also satisfy this property and hence are QMFs [Mintzer 1985;Examples of Orthogonal Basis-Generating Wavelets 71 Table 3.1 LPF Coefficients for D4 n h(n) 0 (a+ v3)/8 1 (34+ V3)/8 2 GB- V3)/8 3 a ¥3)/8 Smith and Barnwell 1986; Vaidyanathan 1992]. Furthermore, Equations 3.87 and 3.88 are also satisfied. Finally, the wavelet w(t) is obtained using Equation 3.38 as HH) =2 D> gin)g(2r - 2) (3.103) n=—co 3.6.1 Daubechies D4 Scaling Function and Wavelet The plot of the D4 wavelet [Daubechies 1988, 1992] shown in Figure 3.3 is perhaps the picture that is most associated with the term wavelet.!? The LPF associated with this wavelet is of the finite impulse response (FIR) type of length four (hence the term D4). The nonzero coefficients of the impulse response are listed in Table 3.1. The corresponding HPF g(n) is obtained from Equation 3.102. Figure 3.3 also shows the D4 scaling function. Neither the scaling function nor the wavelet has a closed-form expression. They are both of finite extent; that is, they are identically zero outside of a finite interval on the time axis. Such functions are said to be supported compactly. Figure 3.1 shows the squared magnitude frequency response of the H and G filters corresponding to D4 (|G(w)| = |H() + 2}). 3.6.2 Bandlimited Wavelets The orthonormal basis-generating wavelets seen so far, Haar and D4, are both supported compactly in time. It is possible to have orthonormal basis-generating wavelets that are bandlimited; that is, their Fourier transforms are supported com- pactly in frequency [Meyer 1993a]. Let us begin with a simple example. Let sinrt = 7 mt g(t) 7 (3.104) 10 Some authors use the term D2 instead of D4. The reason for the use of this terminology will become clear in Chapter 7. A formal method for designing the LPFs associated with the Daubechies scaling functions is presented in Chapter 7.72 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks 1S 0.5 (A) (B) 1S 3 > Figure 3.3 Scaling function (A) and wavelet (B) generated using the D4 filter. Its Fourier transform, given by 1 jol Vw) = | The scaling function and wavelet are shown in Figure 3.4. Because of the association of the sinc function with sampling theory, the scaling function shown in Figure 3.4 is known as the Shannon scaling function, and the wavelet is known as the Shannon wavelet in honor of Claude Shannon, who played a pioneering role in the devel- opment of the theory associated with reconstructing continuous-time signals from their samples [Daubechies 1992; Shannon 1949; Zayed 1993]. The first orthonormal wavelets were constructed by Meyer [1993a] and they were bandlimited. Although the construction is often presented using trigonometric poly- nomials, we take a simpler approach here [Chapa 1995; Vetterli and Kovacevic 1995]. Let ¢(r) be a real-valued function with a Fourier transform ®(w) that satisfies the following: 1. It is real and positive in the range |w| < 47/3 and is zero at frequencies outside this range. Because ©(w) has been assumed to be real, $(r) is symmetric about the origin. It also follows that (w) is itself symmetric about w = 0. 2. It is smooth in the sense that (w) is differentiable. 3. 1 0 < || < 27/3 [d(o) 7? =} 1-j@r-o)? 27/3 < lol < 40/3 (3.115) 0 otherwise. This type of function satisfies the Poisson summation formula in Equation 3.18. Thus, $(r) and its integer translates form an orthonormal set. It can be shown that any bandlimited scaling function necessarily satisfies the Meyer bandlimit [Chapa 1995] (see Problem 7). The function H(w) obtained by substituting such a ®(w) in Equation 3.109 is the frequency response of an LPF with cutoff frequency 27/3 rad. Furthermore, it is real-valued and satisfies the PRQMF property. Thus, its impulse response serves as the coefficient sequence of the dilation equation for $(t), indicating that it is a scaling function for an orthonormal MRA.Examples of Orthogonal Basis-Generating Wavelets 75 0.8 0.6 ~04 (A) (B) Figure 3.4 The Shannon scaling function (A) and wavelet (8).76 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks The Fourier transform “(w) of the corresponding wavelet is generated using Equations 3.111 and 3.113. Combining these two equations, V(o) = e #2 Ht (5 + 7) O(w/2). (3.116) Because H has a real-valued impulse response as well as a real-valued frequency response, H(a+o)=H(x-«@). (3.117) Substituting Equation 3.117 in Equation 3.116 we have Wo) =e PH (7-2) Ow). (3.118) 2 Substituting Equation 3.109 in Equation 3.118 we get =jon POR =O) py, (3.119) MOS Ber =o Using the properties presented earlier for ®(w), Equation 3.119 yields forw>0 e 2 @(2n —|w|) 22/3 < lol < 40/3 W(w) = 4 e/#? &(\w|/2) 4n/3 < |o| < 8x3 3.120) 0 otherwise. The inverse Fourier transform of ¥(w) is the Meyer wavelet y(t) associated with the function ®(w). Example 3.1 7 Let 1 0s jo) < 27/3 ©(w) = } cos WE On /3 < [wl] < 40/3 (3.121) 0 lol > 403. This function satisfies all the properties of the Fourier transform of a Meyer scaling function. Its inverse Fourier transform is the scaling function ¢(1), shown in Figure 3.5. Also shown in Figure 3.5 is the wavelet obtained by the inverse Fourier transform of Ww), which is obtained from Equation 3.120. We see that the Meyer wavelet is bandlimited to the frequency range 27/3 < |w| < 87/3. Therefore, the corresponding wavelet y(r) is of infinite duration in time, although its amplitude decays rapidly, as seen in Figure 3.5. As a consequence, theExamples of Orthogonal Basis-Generating Wavelets on) OS (A) -10 0 10 20 t Ls iE OS- wr) OF -0.5b -1 J ©) -20 -10 0 10 20 Figure 3.5 The Meyer scaling function (A) and wavelet (8). H and G filters are of the noncausal IIR type. This has been seen as a disadvantage in using Meyer scaling functions and wavelets in practical applications. However, there are several situations related to sampling theory when Meyer-type scaling functions arise naturally [Bopardikar, Raghuveer, Adiga 1997a, 1997b]. One such application, related to the choice of signaling waveforms for bandlimited digital communication channels [Jones 1994; Jones and Dill 1998a, 1998b], is discussed in Chapter 6. 7778 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks 3.7 Interpreting Orthonormal. MRAs for Discrete-Time Signals In Section 3.4 we showed that a DWT can be associated with an MRA. The MRA itself consists of nested vector spaces of continuous-time functions. In many appli- cations the signal to be wavelet transformed is a discrete-time signal. The practice is to perform decimation as described in Section 3.5 on this discrete-time signal and offer the resulting sequences (the ones corresponding to b(k,) of Equation 3.76) as the “wavelet transform.” Let us call such a transform a discrete-time wavelet transform (DTWT) [Vaidyanathan 1992]. We now pose the following questions: 1. Can the DIWT be reconciled with our development of the DWT as a repre- sentation of a continuous-time signal using discrete (that is, dyadic) dilations and translations of a continuous wavelet? 2. Is there an equivalent MRA interpretation for the resulting sequences? 3.7.1 Continuous-Time MRA Interpretation for the DTWT To answer the first question, let us begin with a discrete-time signal, say f(n). Define 0 fO= LY foe -a) (3.122) n=—00 where (1) is a scaling function that gives rise to an orthogonal MRA of the type discussed so far. Let f,(t) denote the projection of f(t) on the subspace Vj. Then, with p(k, n) = 2-*( F(t), 62 — n)) (3.123) we have i x Alt) = YO pk, mo Q-*t 1). (3.124) In particular, f(n) = p(0,n) (3.125) which implies that f(t) € Vo. In turn, by the nesting of the subspaces, it follows that f(t) € Ve (3.126)Interpreting Orthonormal MRAs for Discrete-Time Signals for k < 0. Let %&(t) be the detail function at level k, and q(k,n) be the coefficients of its expansion"! in terms of the appropriately dilated and translated wavelets. As we have seen, the sequence p(k,m) is the output of the # filter at the kth level. Similarly, g(k, n) is the output of the G filter at level k. By Equation 3.126, S(t) =0 (3.127) for k < 0, which implies that q(k,n) =0 (3.128) for k < 0. Therefore, A x © fo= > alk, Ey 2*r — e) fet (3.129) = ak Over —o. k=1 t=-00 Thus the DIWT of f(n), which essentially consists of the sequences q(k,n) for k > 0, can be interpreted as the DWT of the continuous-time signal f(t) defined in Equation 3.122. 3.7.2 Discrete-Time MRA We now provide an MRA interpretation for the DIWT of f(n). Let p(1,) and q(1,n) respectively be the outputs at level one of the reconstruction filters 2H and 2G (Figure 3.6). Substituting k = 0 along with an appropriate change of variables in Equations 3.99 and 3.100, oo B(1,n) =2 D> pC, h(n - 28) (3.130) =-0o G(1,n) =2 Y° g(1,£)g(n — 20) (3.131) &=-00 and (see also Equation 3.125) f(x) = p(0,n) = pn) + G(1,n). (3.132) " Here, p(k,n) and q(k.n) play the roles of a(k, n) and b(k, n), respectively, discussed in Section 3.5. 7980 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks Hie) fn) (B) Figure 3.6 Filter bank structure for a three-level DIWT. (A) Analysis bank. (8) Synthesis bank. The sequences p(1,n) and 9(1,”) are orthogonal to each other since h(n — 2€) and g(n — 2) are orthogonal sequences according to Equation 3.91. Suppose we carry out the DIWT decomposition down to K levels. Figure 3.6A shows decom- position for K = 3. The resulting DIWT coefficient sequences (that is, the outputs of the decomposition filter bank) are the sequences q(1,),q(2,n),...,q(K,n), and p(K,n). Let p(K,n) be the final output of the synthesis filter bank with all the q(k,n), k =1,..., K forced to zero. See the illustration for K = 3 in Figure 3.7A. The sequence /(K, 7) can be regarded as the coarse approximation at level K to the input f(n). Similarly, let g(m,) denote the final output of the synthesis filter bank with p(K, n) and all the other g(k, 7) (that is, for 1 9k, n) + B(K, n) (3.133) k=l Letting K > co oo fa) = 9kn) = YP Gk) (3.134) k=l S00Interpreting Orthonormal MrAs for Discrete-Time Signals .-0,0,0, 2g(n) PG.m) 2h(n) G. PG.) —O> 2h(n) (A) +-0,0.0,.. +O 22 FQn) 2h(n) Figure 3.7 Examples of generation of approximation and detail functions for a DIWT for (3, n) (A) and 4(2, n) (8). where q(k,n) =0 (3.135) for k < 0. Also, because p(k,n) is obtained as the sum of an interpolation of the sequence p(k +1, ) with the filter 2H and an interpolation of the q(k +1) sequence with the filter 2G, Blk, n) = G(k + 1,0) + pk +1,n). (3.136) Again, the sequences p(k,n) and (k,n); k = 1,...,K are mutually orthogonal because of the orthogonality of h(n — 2é) and g(n — 20). Let vp be the linear vector space of all discrete-time signals with finite energy (that is, the space containing the input f(n)), u, be the space containing the approximation sequences at level k (such as the sequence p(k, n)), and w; be the linear vector space containing the detail sequences at level k (such as g(k,n)). As we have seen, the sequences 4(k,m) are mutually orthogonal for different values of k. It follows that the vector spaces w; are also mutually orthogonal [Soman and Vaidyanathan 1993]. 8182 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks Then, by Equation 3.134 oo v9 = Pur (3.137) =1 which is the MRA corresponding to the DIWT. Notice the similarity of structure between this MRA and the MRA in Equation 3.68. 3.7.3. Basis Functions for the DTWT From Equations 3.90 and 3.131 we see that g(n) and its even translates form an orthogonal basis for w;. However, by substituting k = 0 in Equation 3.90, we see that the norm of g(n) is 1/2. Therefore, it follows that the set of sequences (V2g(n — 2k): k integer} is an orthonormal basis for w;. The basis for wz is found by looking at how the sequence 4(2,n) is expanded in terms of a basis. From Figure 3.7 it is seen that g(n) is obtained by two stages of interpolation of the sequence q(2,n)—first with the filter 2G and then with the filter 2H. Applying the interpolation expressions in Equation 3.96 twice, G2,n) =4) g2,b) Dg (E - 2k)h(n — 26). (3.138) k é Define gin) =) g@h(n— 20). (3.139) 7 Substituting Equation 3.139 in Equation 3.138, g(2,n) =4) 902, bai(n - 4h). (3.140) K We see that the sequences g(n) and its translates by integer multiples of four (let us call these the fourth translates) form an orthogonal basis for w2—the vector space that contains 9(2.1). Because |[¢(n)|| = ||h(n)|] = 1/2, we require 2g,(m) and its fourth translates to ensure orthonormality. The definition of g;(7) in Equation 3.139 shows that 2g;(n) is obtained by upsampling /2g(n) by a factor of two and then convolving it with V2h(n). Thus, the orthonormal basis for w; is provided by /2g(n) and its even trans- lates. The basis for w is provided by fourth translates of the sequence obtained by upsampling V2g(n) by a factor of two and convolving with J/2h(n). Generalizing the result, an orthonormal basis for w;,() consists of 2" translates of the sequence generated by upsampling V2g(n), and convolving it with /2h(n), then upsamplingInterpreting Orthonormal MRAs for Discrete-Time Signals the result and again convolving it with V2h(n), and so on, repeating the process k—1 times. In other words, there is a total of kK—1 upsampling operations and k—1 convolutions involving one stage of V28(n) and k —1 stages of V2h(n). Figure 3.8 shows the generation of sequences with 2" translates that yield orthonormal basis functions for w;. In Figure 3.8, V2g(n) is itself initially generated as the impulse response of a filter. What about a basis for u%.? The process of generating basis functions for v; follows on lines very similar to those for generating basis functions for wz. Recall that vg is the subspace containing p(k,n), which is generated (see Figure 3.7) by passing p(k,n) through a chain of interpolators. But for one level of interpolation, both p(k, n) and q(k,n) use the same LPF interpolator chain for a given k. The first stage of interpolation for q(n) is with the filter 2G, whereas for p(k,n) it is with 2H. Thus, orthonormal basis functions for vz are obtained as in Figure 3.9 by replacing the g(n) of the basis generation process for w, with h(n). An orthonormal basis for the space of all square summable functions is provided by the collection of the basis functions of wz, k =1,..., K and vx for any K > 1. Examples of such basis functions for the D4 filters are given in Figure 3.10. Even translates of output are basis functions of w, 4(n) a Fourth translates of output are basis functions of w, Eighth translates. of output are basis functions of w, 3(n) (C) Figure 3.8 Generation of sequences with even transiares that yield orthonormal bases for w; (A), w2 (8), and w3 (C). 8384 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks Even translates of output are basis functions of v, 3(n) [ae | (A) Fourth translates of output are basis functions of v, 5(n) (B) Eighth translates. of output are basis functions of v, d(n) * [a } +O g Co} Figure 3.9 Generation of sequences with even translates that yield orthonormal bases for vy (A), v2 (B), and v3 (C). 3.8 Miscellaneous Issues Related to PRAMF Filter Banks We have shown that an orthonormal dyadic wavelet decomposition leads naturally to a PRQMF bank approach to finding the approximation and detail coefficients at subsequent levels after we have the approximation coefficients at a given level of the MRA. However, perfect reconstruction filter bank theory was developed indepen- dently of the wavelet transform. By simply starting with the structure in Figure 3.2 and requiring that the output at the reconstruction end be equal to the input at the decomposition end, one arrives at the paraunitary condition [Vaidyanathan 1990, 1992] (see Problem 14). H*() Go) Hw) Hw+x)]_f1 0 (141 Hw+nx) Gtwt+n) || Gw) Got }]~lo 1]° 141) This also includes the power complementarity condition, as can be seen by substi- tuting the relationship from Equation 3.111 in Equation 3.141.Miscellaneous Issues Related to PRQMF Filter Banks 85 0.5 -0.5 - (A) 0.8 T T 04 0.2 4 (B) 1 Figure 3.10 Functions with translates that form the basis for w; (A) and w2 (8).86 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks 0.6 — 0.2 © n> 0.4 03 0.2 0.1 0 =, | -0.3 -0.4 0 20 25 30 35 40 45 50 Ss a Figure 3.10 (continued) Functions with translates that form the basis for w3 (C) and ug (D).Miscellaneous Issues Related to PRQMF Filter Banks 0.35 0.3 0.2 0.15 4 i oI 2 _— «© 0.1 n> Figure 3.10 (continued) Functions with translates that form the basis for ug (E). The orthogonality condition in Equation 3.20 rules out the possibility of FIR filters of odd length for H. We show this by a simple example using a length-three filter. Suppose h(n) has values a, b, and c forn =0, 1,2 respectively, and zero for all other values of n. Then, by the requirement that the autocorrelation sequence of h(n) be zero at even lags (see Equation 3.19), we have ac = 0, which leads us to set either a=0orc=0, rendering the filter length even. The example is generalized easily to all other odd values larger than three. Also, the orthogonality condition leads to trivial filters if we insist on linear-phase FIR filters. The linear-phase property is equivalent to requiring symmetry. Consider, for example, a symmetric filter of length four, say with impulse response values {a.b,b,a}. Again, the requirement of the autocorrelation of the impulse response being zero at even lags leads to the condition ab = 0 at lag two, which means b = 0. Thus, the filter has only two nonzero coefficients—a result that. can be generalized to symmetric filters of other lengths. As explained in Section 3.5, one level of the DIWT of a given sequence is com- puted by passing the input sequence through the # and the G filters, and downsam- pling the results by two. Now, let the input x(n) be an N-sample sequence. Also, suppose that H and G are FIR filters of length L. Then the convolution of x(m) with the H and G filters will give low-pass and high-pass sequences of N + L—1 samples 8788 MRA, Orthonormal Wavelets, and Their Relationship to Filter Banks each. Downsampling these sequences gives two sequences of length [(N+L—1)/2] where [.] rounds the argument to the next higher integer. Clearly, the total number of samples at the output is greater than the total number of samples of the input sequence. Thus, the DTWT based on the linear convolution does not conserve the number of samples. In applications involving finite-length signals or images, it is necessary to conserve the total number of samples [Smith and Eddins 1990]. One way of doing this is by replicating the signal at the boundary. In other words, we create a sequence (mn) such that E(+pN)=x(n) 0 } h (B) Figure 3.11 Original signal (A) and circular extension of the signal (B).Generating Scaling Functions and Wavelets from Filter Coefficients two N/2-periodic sequences to give N-periodic sequences, passing these N-periodic sequences through the H and G filters and adding the results. This gives ¥(n) from which x(n) can be extracted. Note that for the circular extension process to work, it is necessary that the input sequence x(n) have an even length. If this is not the case then the downsampled sequences will not be N/2-periodic and the number of samples will not be conserved [Smith and Eddins 1990]. Extending our analysis from the one-level decomposition case to M-levels we find that at the kth level the sequences are N/2*-periodic. At each level it is necessary that the sequences be divisible by two, so that the next level of decomposition can be effected. Thus, it is necessary that the length of the input sequence to an M-level filter analysis bank be divisible by 2”. 3.9 Generating Scaling Functions and Wavelets from Filter Coefficients Repeated application of Equation 3.108 leads to O(w) = H(w/2) H(w/4) ... H(0) (3.143) where we have set (0) = 1. Suppose we are given the coefficients of the impulse response of a discrete-time filter h(n) satisfying the paraunitary conditions. The sequence 2h(n) can serve as the set of coefficients for the dilation equation to gen- erate a potential scaling function $(t) for an orthonormal decomposition. In fact, if substitution of the frequency response H(w) in the right-hand side of Equation 3.143 leads to a function (w), then its inverse Fourier transform is such a scaling function. There is a simple time-domain iteration method based on this result. The steps of the algorithm are [Daubechies 1988; Strang 1989]: 1. Set c(n) = 2h(n). 2. Let the initial scaling function be the Haar scaling function: 1 O» a(k, n)o(2“'t — n) where ¢(1) is the scaling function associated with an orthonormal MRA. Show that f¢(t) has finite energy if and only if the sequence a(k, n) has finite energy; that is, it is square summable: Yak, mp < 00. 5. Show that if ¢(r) and its integer translates form an orthonormal set—that is, (¢(t), o(t — £)) = 6(€)—then it satisfies the Poisson summation formula o Y @+27HP =1 (e220 where ©(w) is the Fourier transform of ¢(r). Hint: The autocorrelation of ¢(t) given by R(t) = (d(t).¢( — t)) when converted to a discrete-time sequence by sampling at unit lags is, by orthonormality, the discrete unit impulse function. Use the result from sampling theory that relates the Fourier transform of the sampled signal to the Fourier transform of the original continuous-time signal. 6. Show that the converse of Problem 5 is also true: If a function satisfies the Poisson summation formula then the set of its integer translates is orthonormal. 7. Show that any bandlimited scaling function that satisfies the dilation equation (Equa- tion 3.106) is bandlimited to the frequency range |w| < 47/3 (the Meyer bandlimit) [Chapa 1995]. 8. If X(@) is the Fourier transform of a discrete-time sequence x(n), show that the Fourier transform of its downsampled version x(2m) is [(X (@/2) + X(w/2 + 7)]/2.Problems 9. Derive Equation 3.22 using the result of Problem 8. 10. In this problem we generalize the dyadic case to the M-adic case. Suppose we had a scaling function that satisfies the dilation equation 6() = MAM -h). a This scaling function would have M — 1 corresponding wavelets W(¢),..., Yw—i(t) that would satisfy similar M-adic dilation equations: Wilt) = MY? 8 (OO(Ms — K). 7 The scaling function ¢(t) and the wavelets y(t), i = 1,...,M—1 are orthogonal to their integer translates. Derive the corresponding paraunitary conditions satisfied by the filters A(n) and gi(n),i=1,...,M@—1. 11. Suppose ¢(t) is a scaling function that gives rise to an orthonormal MRA. Would you then say that ¢(—r) also gives rise to an orthonorma! MRA? How about ¢(1 — m) where m is an integer? 12. If y(t) is a wavelet associated with an orthonormal MRA, can we then replace y(t) by —w(s) and still get the same MRA? Repeat the problem for y(—1) and y(t — m) where m is an integer. 13. Show that [OP = SVP. k=] Hint: First show that [O(w)/? = |@(2o)/? + |W 2w)/?. 14. Consider the analysis and synthesis filter bank system shown in Figure 3.2. As noted in Section 3.8 we can arrive at the perfect reconstruction conditions for this arrangement without invoking its connection to wavelet transforms and MRAs. Let x(n) be the input and x,(n) be the output. Express the Fourier transform X,(w) of the output x-(m) of the system in terms of the filter frequency responses H (#) and G(w) and input signal Fourier transform X(w). Show by equating X,(w) to X(w) that the necessary and sufficient conditions for perfect reconstruction are the paraunitary conditions of Equations 3.87 and 3.88. You will find that the transfer function contains an aliasing term, due to the presence of X(w + 7), that is canceled when the paraunitary conditions are satisfied. 15. Run anal given in the disk provided with this book with the input sequence f(n) = er (3.147) 0 » a(0,n)o(t —n) (4.6) where a(0,n) = (f@), b(t =n). (4.7) Thus, the coefficients of the expansion are obtained by projecting f(r) not on Vo but rather on Vo, which is simply a generalization of our introduction to biorthogonality provided earlier in this chapter with a pair of finite-dimensional vectors. In general, 0 felt) = D> alk, nya" —n) (4.8) where a(k,n) = 2 FO), G2“ — n)). (4.9) Generating approximations in this fashion has the nice property (also seen with the orthogonal MRA of Chapter 3) of yielding the same approximation to the function f(t) at a certain level whether we use the original function f(r) or one of its ap- proximations at a finer level. We illustrate this using the particular case of fo(t) and f_1(t). From Equation 4.8, fat) = YO a(-1,n)6 2 =n) (4.10) where a(-1.n) =2(f(), G(2t =n). (4.11) Suppose we were to start with f_;(r) instead of f(r). Then, similar to Equation 4.6, we would form its approximation in Vp as Ea DS (Fae). $= mp0 = 0) (4.12) n=—Biorthogonal Wavelet Bases whereas if we used f(r) directly, we would form its approximation as f(t) in Equa- tion 4.6 with the coefficients given by a(0, n) in Equation 4.7. From Equation 4.4, b(t =n) =2 D> h(m)b(2t — 2n — m). (4.13) m=—co Using Equation 4.13 in Equation 4.7 we have a(0,n) =2) h(m)(f), d(2t - 2n = m)). (4.14) From Equations 4.9 and 4.14 we have a(0,n) =)“ A(m)a(-1, 2n +m) (4.15) which can be rewritten as a(0,n) = )\a(-1, myh(m — 2n). (4.16) It can be shown that the quantity (f_1(t), 6(¢ — n)) in Equation 4.12 is also equal to the right-hand side of Equation 4.16, thus showing that by using Equation 4.8, whether we approximate f(t) in Vo or f_1(t) in Vo, we get the same function. Observe from Equation 4.16 that (as in the orthogonal case) coefficients at one level can be obtained by filtering and downsampling the coefficient sequence at the next finer level. The filter in question has impulse response h(—n). How does one associate wavelets with a biorthogonal MRA? Not surprisingly, given that there are two scaling functions, we also have two wavelets that are duals of each other {[Cohen, Daubechies, and Feauveau 1992; Daubechies 1992]. Let u(t) and v(t) be two functions that integrate to zero, with the additional properties that (WO), VO =n) = 8m) (W(t), dt —n)) = 0 (W), o@ —n)) = 0 (4.17) VO = 2° g)g2t -n) Vit) = 2° Gn) G2r —m). Also, let y(t) and its integer translates, and v(t) and its integer translates, span subspaces denoted by Wo and Wo respectively. With these properties, y(t) and v(t) 99100 Alternative Wavelet Representations serve as a wavelet and its dual, respectively, for the biorthogonal MRA. Properties 2 and 3 in Equation 4.17 require the wavelet to be orthogonal to the dual scaling function and its integer translates, and require the dual wavelet to be orthogonal to the scaling function and its translates. This means that the subspace Wo is or- thogonal to subspace Vo, and the subspace Wo is orthogonal to Vp. Analogous to the orthogonal MRA, the subspaces Wo and Wo are the detail subspaces and they are biorthogonal to each other by property 1 in Equation 4.17. Properties 4 and 5 require the wavelets to satisfy two-scale relations with coefficient sequences g(n) and g(n). Following steps similar to those in Chapter 3 showing that the detail function at any level is expressible as a linear combination of translations of an appropriate dilation of the wavelet, it can be shown that the detail functions Bit) = fri) — felt) (4.18) for the biorthogonal decomposition can be expressed as a linear combination of translations of a dilation of y(r): &(t) = > b(k, n)w(2*t — 1). (4.19) Furthermore, the coefficients b(k,n) can be obtained by decimating a(k —1,n). We show this by first considering 80o(t) = fl) — fol) z (4.20) _ > b(O,n) Wt — 7). From Equation 4.20 and the relationship between y(t) and #(t—n) in Equation 4.17, it follows that b(0,n) = (go(t). Wr — n)) = (f) — fot), He —n)) (4.21) = (furl), W(t = m)) = (fol), We =n). The last term in Equation 4.21 (namely, (fo(t), vt —n))) is zero because, as we have seen in Equation 4.6, f(t) is a linear combination of $(r — n), n integer, and (h(t), 6 —n)) is zero by Equation 4.17. Thus bO,n) = (ft), We -n). (4.22)Biorthogonal Wavelet Bases Using Equation 4.10 and the two-scale relation for y(t) in Equation 4.22 along with the biorthogonality of ¢(2t) and $(2t), we get 5(0,n) = )* a(—1,m)Z(m — 2n) (4.23) m=—00 which indicates that b(0,) can be computed from a(—1,n) by decimating it by a factor of two using a filter with an impulse response of g(—n). The result can be generalized to oo blk,n) = S> alk -1.m)g(m — 2n). (4.24) m=—00 For any continuous square integrable function f(r) we have co fO= > al) (4.25) k=-00 which, because of Equation 4.19, leads to oo fO= DY bk nya -n) (4.26) =—00 N=—00 which is the reconstruction expression for the DWT of f(t) with respect to the wavelet w(t). We conclude this section by deriving the synthesis filtering arrangement for the biorthogonal DWT. We have fA) = fo® + golt) (4.27) =} aO,mde—n) + D> bO,n)ve-n). (4.28) On substituting the respective two-scale relations in Equation 4.28 we have Yo a1, mor —n) = 2 YS Ya. Qacmp2r — 2 — m) + n=—0O f=-co m (4.29) 2 S72 S000. Lg¢mg(2r - 2¢ - m). =—co m 10°102 Alternative Wavelet Representations > a) ~) BOO.n) ~h) >| 2¢in) a(—1,n) a(—1,n) — Ll ign a 2hn) a(0,n) Figure 4.2 Decomposition and reconstruction filter banks for the biorthogonal case. Substituting n = 2+ m in the right-hand side of Equation 4.29 leads to co ce SO a(-1,n)$@1—n) = 2 D> Ya, a(n — 26621 —n) + n=—00 i 7 (4.30) 2 > $200, Agim - 2€)6 21 — 2). From Equation 4.30, a(-1,n) =2)-a(0, £)h(n — 26) +2 b(0, £)g(n — 26) (4.31) é if which shows that a(—1, n) is the sum of two sequences: one, an interpolation by a factor of two of a(0, n) using the filter with impulse response 2h(n); and the other, an interpolation by a factor of two of b(0, n) using the filter with impulse response 2g(n). The combined decomposition and reconstruction is depicted in Figure 4.2. 4.3 Filtering Relationship for Biorthogonal Filters We next derive expressions relating the various filters to each other in both the time and frequency domains. Equations 4.4 and 4.5 yield (00), 60 —n)) = 8(n) = 4° n(pyh@y (Pr — p)d(2t — 2n — g)) (4.32) Pog = 22 DA yh@)s(p ~ 2n - @) P fiFiltering Relationship for Biorthogonal Filters from which it follows that Slain + 2nyh(m) = o (4.33) In the frequency domain the corresponding relationship is H*(w)H(w) + H*(w+)A(o +m) =1. (4.34) Similarly, from Equation 4.17 we have 3 Digin + 2am) = a Z (4.35) G*(w)G(w) + G'(w+2)G(wt+72) =1 and Yo AGn + 2n)g(m) = 0 H*(w)G(w) + H(w+2)Glo +x) =0 (4.36) hon +2n)g(m) = 0 H(w)G"(w) + H(w+2)G(o+z) = 0. The filter equations presented in this section have been derived as necessary consequences of biorthogonal decomposition. They can be represented compactly in matrix form as [2 ee ae G*(w) |= Gi) Gw+zn) H*(w+7) G*(w+7) (437) where I is the two-by-two identity matrix. Interchanging the order of multiplication of the two matrices in Equation 4.37 gives H*(w) Go) Hw) H(wt+n)] _ eons eee fer |= (4.38) which gives rise to the set of equations H*(w) H(w) + G"(w)G(w) = 1 W(otn)Awt+n)+Gwtn)Gwt+m) =1 H*(w)H(o+n)+G(o)Gw+n) =0 H*(o+2)H(w) + Gi(w+7)G(w) = 0. (4.39) Note that Equations 4.34, 4.35, 4.36, and 4.39 are all biorthogonal generalizations of the paraunitary conditions mentioned in Chapter 3. As in the orthogonal case, 103104 Alternative Wavelet Representations the satisfaction of the paraunitary conditions by the analysis and synthesis filter banks in the biorthogonal case are necessary and sufficient conditions for perfect reconstruction. Lastly, consider the following relationships between the H and G filters Gi) = ce’ H*(w +7) oe (4.40) G(w) = e!°H*(w+n). It is easy to see [Cohen, Daubechies, and Feauveau 1992] that these relationships between the H and G filters satisfy automatically the conditions given in Equations 4.34, 4.35, and 4.39. The equations for the orthogonal filters follow as a special case if we replace h(n) by A(n), A(o) by H(w), and so on (that is, if we simply remove the tildes). 4.4 Examples of Biorthogonal Scaling Functions and Wavelets The B-splines are examples of scaling functions with underlying, symmetric FIR filters. An N tap B-spline LPF has the impulse response 1 = no) = ses ( *) forn =0,1,...,N—1. Substituting this filter in the dilation equation yields a scaling function. The higher the value of N, the smoother the scaling function. Thus, for N = 2 we get the Haar filter and scaling function. Table 4.1 gives h(n) for N = 3, 4, and 5. A dual filter h(n) can be obtained using the biorthogonality relationship given in Equation 4.33. The filter h(n) is also symmetric. This fact along with Equation 4.33 gives a set of linear equations that yield the coefficients of h(n) (see Problem 1 at the end of the chapter). The scaling functions corresponding to these B-spline filters are shown in Figure 4.3. These were generated using the MATLAB routine presented in Chapter 3 for solution of dilation equations. (4.41) Table 4.1 B-Spline Filter Impulse Responses No. of Taps B-Spline Type Impulse Response /(n) 3 Linear {3. i, 5 4 Quadratic {i 2, 3, 3 5 Cubic {5.5-24.4}Examples of Biorthogonal Scaling Functions and Wavelets 105 0.8 0.6 04 (A) 0.8 0.7 0.6 04 0.3 0.2 0.1 (B) 0.7 0.6 0.5 0.4 0.3 (C) Figure 4.3 Scaling functions corresponding to linear (A), quadratic (B), and cubic (C) B-spline filters.106 Alternative Wavelet Representations Table 4.2 Analysis Filter Banks Used by the FBI Filter Type Right-side Filter Coefficients Filter Length Low pass a x {0.853, 0.3774, —0.11062, 9 —0.023849, 0.03783} High pass a x (0.7885, —0.4181, —0.04069, 7 0.06454} Another example of biorthogonal analysis and synthesis filters is the one em- ployed by the Federal Bureau of Investigation (FBI) for fingerprint image compres- sion. The coefficients of the right half of the analysis bank are given in Table 4.2. The left side of the filter is generated by making use of the symmetry of the filters. The synthesis bank is generated using the following relations that can be obtained as an inverse discrete-time Fourier transform of the relationships in Equation 4.40: Bn) = (-1)"h - 2) h(n) = (-1)"g(1 — 2). (4.42) The scaling functions and wavelets corresponding to these filters are shown in Fig- ure 4.4. A method of designing practical biorthogonal filters [Herley 1993; Vetterli and Herley 1992] involves first designing a halfband filter to desired specifications. Let P(w) be the frequency response of the halfband filter. Standard filter design meth- ods, such as the Parks-McClellan algorithm [Oppenheim and Schafer 1989], can be used to obtain P(w) subject to the halfband constraint P(w)+ P(w+m)=1 (4.43) and other passband and stopband constraints. The function P(w) can then be fac- tored as P(w) = H*(w) A) (4.44) to give a pair of biorthogonal filter frequency responses H(w) and H(w). For the specific case when P(w) is of the form |H(w)|* = H*(w)H(w), we can factor it to yield H(w) and H(w) = H*(w), resulting in a PRQMF bank for an orthogonal decomposition. Other methods for the design of biorthogonal scaling functions and wavelets such as the lifting scheme and biorthogonal generalization of the Meyer construction are presented in Chapter 7.