INVERSION

1. Inversion and Vector Spaces

data: a set of discrete observations

e = [e1 , e2 , e 3 , e 4 ,..., e N ]
in reality, the set is finite and the data are inaccurate example 1: observations of the strength of gravity as a function of position on Earths surface example 2: observations of concentration of a chemical in a 2-reservoir system

model: a mathematical model of reality, represented by a set of model parameters m = [ m1 , m2 , m 3 , m 4 ,..., m M ] M can be infinite example 1: 3D density structure within Earth example 2: diffusion constant

forward modeling: given a model, compute data simulate an experiment for a known model incorporates the laws of physics, plus the experiment design e = F{m} F is any mathematical functional (it is often difficult to separate the data from the general equation example 1: Newtons Law of gravity, location of gravity observations example 2: diffusion equation, time of composition observations

G{m, e} = 0

GEOS 5004: Analytical Geosciences Hole: 1.1 Inversion: Inversion and Vector Spaces

forward modeling:

inverse modeling (inversion):

given data, directly compute a model that is consistent with it m = F 1 {e} F-1 is the mathematical inverse of F in general, the inverse function does not exist in a unique form => inverse theory

GEOS 5004: Analytical Geosciences

Hole: 1.2

Inversion: Inversion and Vector Spaces

the discrete, linear inverse problem the inverse problem is discrete if the number of unknown model parameters the inverse problem is linear if

mj

is finite:

F{m} = F{m}

and

F{m1 + m 2 } = F{m1 } + F{m 2 }

then each datum is a linear combination of the model parameters

ei = gi1m1 + gi 2 m2 + L + giM m M M ei = gij m j

j =1

example: fitting a straight line assumed physics: y = + x model parameters: intercept and slope

m = [ m1 , m2 ] = [, ]
data: values of

yi

as measured by sampling at different locations

data equations are:

e = [e1 , e2 ,..., e N ] = [ y1 , y2 ,..., y N ] yi = + x i ei = 1m1 + x i m2 gi = [1, x i ]

Hole: 1.3

xi

GEOS 5004: Analytical Geosciences

Inversion: Inversion and Vector Spaces

example: fitting a 4th order polynomial 2 assumed physics: y = m1 + m2 x + m 3 x

+ m4 x 3 m = [ m1 , m2 , m 3 , m 4 ] e = [e1 , e2 ,..., e N ] = [ y1 , y2 ,..., y N ]

gi = 1, x i , x i 2 , x i 3

data equations are:

ei = 1m1 + x i m2 + x i 2 m 3 + x i 3 m 4

example: gravity anomaly measure the strength of Earths gravity field at different positions on a flat surface subtract the background average gravity to obtain the gravity anomaly g this gravity anomaly is due to anomalous mass, or , the density minus the average density Newtons Law of Gravitation says: (where x,y,z are measured from the ith gravity station)

gi = G
M

earth

(x

+y +z

( x , y, z)dzdydz

if we assume density is constant in a set of M blocks in the region of interest

then

z gi = G dzdydz j 3 2 2 2 2 j =1 block ( x + y + z ) m j = j ei = gi and gij = the integral above

j

GEOS 5004: Analytical Geosciences

Hole: 1.4

Inversion: Inversion and Vector Spaces

example: straight-ray x-ray tomography (CT scan) or seismic tomography tomography = slice picture (Greek)

ei =

( raypath ) i

m( x, y, z)dl

for medical tomography: m(x,y,z) is the x-ray attenuation coefficient at each point in space ei are the natural logarithm of the observed x-ray amplitudes ln(A) for seismic tomography: m(x,y) is one divided by the seismic wave speed ei are the observed travel times

1/v

if the model is discretized into a set of grid cells, with constant physical properties in each cell mj is the attenuation or slowness in each grid cell and the integral becomes a summation
M

ei = Lij m j
j =1

where

Lij

is the length of the i'th ray in the jth grid cell

note that both data and model in the tomography example are derived properties the equations have been rearranged to make them linear
GEOS 5004: Analytical Geosciences Hole: 1.5 Inversion: Inversion and Vector Spaces

vector and matrix notation

scalar:

v1 vector: v = M or v = [v1 K v n ] v n addition: C = A + B Cij = Aij + Bij multiplication:

K

A11 A12 L A1 M O A M matrix: A = M ij AN 1 L L ANM C, A, B all have dimensions NxM

A is NxK, v has dimension K, u has N

multiplication: C = AB
T transpose: Cij = C ji

u = Av

ui = Aik v k
k =1 K

Cij = Aik Bkj A is NxK, B is KxM, C is NxM

k =1

identity matrix:

=1 i = j Iij = ij =0 i j

inverse of a matrix:

2x2 matrix:

AA 1 = I a b A = c d

determinant of a 2x2 matrix: determinant of a 3x3 matrix:

A is NxN (must be a square matrix) 1 d b 1 A = c a ad bc A = ad bc

a b c d e g h e f =a h i f d b i g f d e +c i g h

GEOS 5004: Analytical Geosciences

Hole: 1.6

Inversion: Inversion and Vector Spaces

vector and matrix practice let

1 2 A = 3 4 A=

write

AT =

compute

1 compute A =

compute

AA 1 = I
?

does it equal

GEOS 5004: Analytical Geosciences

Hole: 1.7

Inversion: Inversion and Vector Spaces

linear inverse problem the data equations are

M

ei = gij m j
j =1

i = 1,2,..., N

this is a system of

equations in

M unknowns e1 = g11m1 + g12 m2 + L + g1M m M

in matrix notation

e2 = g21m1 + g22 m2 + L + g2 M m M M e N = g N1m1 + g N 2 m2 + L + g NM m M

e = Gm
e1 e e = 2 M e N

m1 g1 m g2 m = 2 G = M M m g M N

g i = [ gi1

gi 2 L giM ]

to find the unknown model parameters: however, the solution only exists if

m = G 1e G is square ( N equations, N unknowns) and well behaved

Hole: 1.8 Inversion: Inversion and Vector Spaces

GEOS 5004: Analytical Geosciences

vector spaces our (Euclidean) universe is a three-dimensional space ,y , and z the Cartesian coordinate system (x,y,z) is defined by three unit vectors x every vector in this 3D space can be written in terms of its components in each direction

+ vy y + vz z = (v x , v y , v z ) v = vx x
the norm that defines length in our universe is Pythagoruss Theorem

angle

2 2 2 2 v = v x + v y + vz = vi i= x ,y ,z
1 2

between two vectors in our universe is defined by the vector dot product

u v = u v cos = u x v x + u y v y + u z v z =

u v
i= x , y ,z

i i

two vectors are perpendicular (orthogonal) if their dot product equals zero
the unit vectors are orthonormal: length of 1 and perpendicular to each other

= y = z =1 x
= v cos x vx = v x

y =x z =y z =0 x
= v cos y vy = v y = v cos z vz = v z

these vectors define a basis for the vector space (a coordinate system)

the coordinates of the vector are the projections (dot products) of the vector on the unit vectors

the model vector m lies in an M-dimensional vector space the data vector e lies in an N-dimensional vector space
GEOS 5004: Analytical Geosciences Hole: 1.9 Inversion: Inversion and Vector Spaces

norm need to define a norm to use as a ruler to measure length

N

l1 norm:

v 1 = vi
i =1

l norm:

2 v 2 = vi i =1
N N

1 2

(also called the Euclidean norm)

1 p

l norm: l norm:

p v p = vi i =1
v
2

= lim v p = max vi
p i=1

(also called the minimax norm)

1 2

weighted l norm:

v 2w

e.g.,

2 = wi vi i =1
N

wi

are a pre-determined set of weights

v = [ 1, 3, 4, 4] u = [ 5,0,1, 4] v 1 = 12 v 2 = 42 u 1 = 10 u 2 = 42
v1> u1 v2 = u2
Hole: 1.10

v =4 u =5 v< u
Inversion: Inversion and Vector Spaces

length, and even relative length, depends on the norm

GEOS 5004: Analytical Geosciences

dot product need to define angles between vectors need to define when vectors are perpendicular the dot product is defined as
N

u v = u v cos = ui vi
note that this definition of angle is associated with the l 2 norm (should not be used with others) two vectors are perpendicular (orthogonal) if their dot product equals zero compute the dot product of v = [ 1, 3, 4, 4] and what is the angle between them?
i=1

u = [ 5,0,1, 4]
gi
vectors

the data equations are the dot product of the model with the
M

ei = gij m j
j =1

ei = g i m

the gi vectors (the rows of G ) are called the data kernels each vector gi points in a direction in the models M-dimensional vector space what we know the data are the projections of the model upon the data kernels however, the data kernels do not form a very good coordinate system for the models vector space they are not orthogonal to each other; in fact, they are usually close to parallel
GEOS 5004: Analytical Geosciences Hole: 1.11 Inversion: Inversion and Vector Spaces

eigenvalues and eigenvectors for a square

NxN matrix, the eigenvalues i (i = 1,2,N) A i I = 0

are defined by:

where the vertical bars indicate the determinant of the matrix inside e.g., for a 2x2 matrix:

b = (a )(d ) bc = 2 + (a d ) + (ad bc) = 0 d solve the quadratic equation in for the two eigenvalues B B 2 4 AC 2 for the quadratic equation Ax + Bx + C = 0 solutions are x = 2A an NxN matrix will involve the solution of an Nth order polynomial to find N eigenvalues

a c

the corresponding eigenvectors

i (i = 1,2,N) are defined r by i = 0 i = 1 r (A i I)r finding each eigenvector involves solving a system of N equations in N unknowns (the
components of the vector), subject to the additional unit-length constraint (note that some algorithms will not force the eigenvectors to have unit length)

GEOS 5004: Analytical Geosciences

Hole: 1.12 Inversion: Inversion and Vector Spaces

eigenvalues and eigenvectors compute the eigenvalues and eigenvectors of

1 2 A = 3 4

GEOS 5004: Analytical Geosciences

Hole: 1.13

Inversion: Inversion and Vector Spaces

spectral decomposition if there exist N different, real, non-zero eigenvalues for the matrix A , the eigenvectors form a new basis (coordinate system) for the N-dimensional vector space create a diagonal matrix whose values are the eigenvalues create a matrix

whose columns are the eigenvectors (sorted in the same order)

1 0 0 =0 O 0 0 0 N
then the original matrix can be diagonalized as

1 L r N R = r

A = RR1
assuming

N = M

find the eigenvalues and eigenvectors of the data equations of

defining new, rotated vectors

1

(and the inverse un-rotation)

RR1m = e R1m = R1e

m =R m e = R1e m = Rm e = Re m ,e are the coordinates of m , e in the coordinate system defined by the eigenvectors this gives a system of equations that is trivial to invert i m =e m i = e i the only problem with the method above is that the R matrix still needs to be inverted
GEOS 5004: Analytical Geosciences Hole: 1.14 Inversion: Inversion and Vector Spaces

spectral decomposition in the special case of a symmetric matrix if the eigenvalues are different (and non-zero), the eigenvectors will be orthonormal

i r j = ij r the eigenvectors then form an orthonormal basis for the N-dimensional vector space because the eigenvalues are orthonormal, the inverse of R is the transpose of R RRT = RTR = I then the original matrix can be diagonalized as A = RRT
assuming

N=M

and

is symmetric (a very special case, but one we will come back to)

defining new, rotated vectors

(and the inverse un-rotation)

RRT m = e RT m = RT e

m = RT m e = RT e m = Rm e = Re m ,e are the coordinates of m , e in the coordinate system defined by the eigenvectors that is trivial to invert gives a system of equations i m =e m i = e i

GEOS 5004: Analytical Geosciences

Hole: 1.15

Inversion: Inversion and Vector Spaces

linear dependence the data equations are the dot product of the model with the

ei = g i m each data kernel gi points in a direction within the models M-dimensional vector space each vector defines a direction in which you know the length of the model (from ei ) the N data kernels together define the N directions in which you know the length of the model
if one (or more) of the data kernels
N

gi

data kernel vectors

gi

can be expressed as a linear combination of the other rows, is satisfied for some

c g
i i=1

=0

ci 0

then the system of equations (or the matrix if not, the the system is linearly independent

G ) is linearly dependent

if a particular data kernel is linearly depended upon the previous ones, then it measures the length of the model in a direction that is already known => adds no new information the total number of directions in which you know the model is less than N g 2 = [0,1,0] g 3 = [1,2,0] e.g., in 3D space: g1 = [1,0,0] direction these three vectors are linearly dependent; none of the three vectors samples the z
GEOS 5004: Analytical Geosciences Hole: 1.16 Inversion: Inversion and Vector Spaces

over-determined and under-determined problems if the number of data exceed the number of unknowns: N > M the problem is over-determined some of the data kernels must be linear combinations of each other because they only span the M-dimensional model space; G is linearly dependent since the data include measurement errors, the system of equations is usually inconsistent => no solution exactly matches all of the data often the model parameterization is not an exact representation of reality e.g., the physics is only approximated by the equation you are trying to fit e.g., the earth is only approximated by a set of grid cells => no solution can exactly match all of the data solution: find the best fitting model but need to define best if the number of data is less than the number of unknowns: N < M the problem is under-determined some directions in the M-dimensional model space are not sampled by any of the data kernels the length of the model in directions perpendicular to the set of data kernels is unknown models that fit the data can have any length in these directions => the model is non-uniquely determined => an infinite number of models match the data solution: find the best model that fits the data but need to define best
GEOS 5004: Analytical Geosciences Hole: 1.17 Inversion: Inversion and Vector Spaces

mixed-determined problems in practice, we often have the mixed case: some of the data kernels are linear combinations of each other (linearly dependent) => cannot match all of the data but the total number of independent data is less than M => the model is non-uniquely determined this problem is mixed determined solution: find the best model that best fits the data but need to define both bests!

inverse theory is about making choices on best, and choosing tools to solve the problem

GEOS 5004: Analytical Geosciences

Hole: 1.18

Inversion: Inversion and Vector Spaces