AMath 585 Homework 4

Name: Jennifer Hanson UWNetID: 0631824

Part 1: Exercise 3.1a - code for Poisson problem from http://www.amath.washington.edu/~rjl/fdmbook. Notice this system is being solved with Gaussian Elimination using Matlabs backslash command, but sparse storage is being used. Other useful Matlab commands can be learned from this exercise, including contour plotting, and reshaping vectors and matrices for convenience easily. Exercise: The matlab script poisson.m solves the Poisson problem on a square m m grid with x = y = h, using the 5-point Laplacian. It is set up to solve a test problem for which the exact solution is u(x, y) = exp(x + y/2), using Dirichlet boundary conditions and the right hand side f (x, y) = 1.25 exp(x + y/2). Test this script by performing a grid renement study to verify that it is second order accurate. Solution: The errors associated with grids containing mxm internal points with spacing h can been seen in the table below. Fitting the log of the spacing h to the log of the errors with the matlab polyt command gives the equation: ln(|Error|) = 1.9934 ln(h) 4.2659 2 ln(h) 4.2659 This indicates that the method is second-order accurate with respect to h. m 5 10 15 20 25 30 40 50 60 70 80 90 100 150 h 1.667e-001 9.091e-002 6.250e-002 4.762e-002 3.846e-002 3.226e-002 2.439e-002 1.961e-002 1.639e-002 1.408e-002 1.235e-002 1.099e-002 9.901e-003 6.623e-003 |Error| 3.837e-004 1.184e-004 5.621e-005 3.273e-005 2.138e-005 1.504e-005 8.601e-006 5.561e-006 3.889e-006 2.870e-006 2.205e-006 1.748e-006 1.419e-006 6.347e-007 1

200 250 300

4.975e-003 3.984e-003 3.322e-003

3.582e-007 2.297e-007 1.598e-007

Code: The code for this problem contains two les. The primary program is in problem1Main.m, which cycles through computations with dierent grid spacings and plots the results. The auxiliary le is gridRene.m which computes the error associated with a specic grid spacing. problem1Main.m %vector of the number of internal grid points to use for different iterations ms = [5;10;15;20;25;30;40;50;60;70;80;90;100;150;200; 250; 300]; a = 0; b = 1; %compute the errors associated with using different grid sizes for n=1:max(size(ms)) m = ms(n); err(n) = gridRefine(a,b,m); h(n) = (b-a)/(m+1); fprintf(%i %10.3e %10.3e \n, m ,h(n), err(n)); end

x = log(h); y = log(err); polyfit(x,y,1) plot(x, y,o-) hold on, xlabel(log(h)) ylabel(log(Error)) title(log(h) vs log(Error)) gridRene.m function err = gridRefine(a,b,m) % % % % % % % % Solves the Poisson problem u_{xx} + u_{yy} = f(x,y) on [a,b] x [a,b] using m interior grid points in each direction for at total of m^2 internal grid points. The 5-point Laplacian is used at interior grid points. This system of equations is then solved using backslash. Returns the error relative to true solution of PDE.

h = (b-a)/(m+1); x = linspace(a,b,m+2); y = linspace(a,b,m+2);

% grid points x including boundaries % grid points y including boundaries

[X,Y] = meshgrid(x,y); X = X; Y = Y; Iint Jint Xint Yint = = = = 2:m+1; 2:m+1; X(Iint,Jint); Y(Iint,Jint);

% 2d arrays of x,y values % transpose so that X(i,j),Y(i,j) are % coordinates of (i,j) point % indices of interior points in x % indices of interior points in y % interior points

f = @(x,y) 1.25*exp(x+y/2); rhs = f(Xint,Yint);

% f(x,y) function

% evaluate f at interior points for right hand side % rhs is modified below for boundary conditions. % true solution for test problem

utrue = exp(X+Y/2);

% set boundary conditions around edges of usoln array: 3

usoln = utrue;

% % % % %

use true solution for this test problem This sets full array, but only boundary values are used below. For a problem where utrue is not known, would have to set each edge of usoln to the desired Dirichlet boundary values.

% adjust rhs(:,1) rhs(:,m) rhs(1,:) rhs(m,:)

the rhs to = rhs(:,1) = rhs(:,m) = rhs(1,:) = rhs(m,:)

include boundary terms: - usoln(Iint,1)/h^2; - usoln(Iint,m+2)/h^2; - usoln(1,Jint)/h^2; - usoln(m+2,Jint)/h^2;

% convert the 2d grid function rhs into a column vector for rhs of system: F = reshape(rhs,m*m,1); % I e T S A form matrix A: = speye(m); = ones(m,1); = spdiags([e -4*e e],[-1 0 1],m,m); = spdiags([e e],[-1 1],m,m); = (kron(I,T) + kron(S,I)) / h^2;

% Solve the linear system: uvec = A\F; % reshape vector solution uvec as a grid function and % insert this interior solution into usoln for plotting purposes: % (recall boundary conditions in usoln are already set) usoln(Iint,Jint) = reshape(uvec,m,m); % assuming true solution is known and stored in utrue: err = max(max(abs(usoln-utrue))); Part 2 - Gauss-Siedel and SOR: Note: All code for this problem is at the end of the problem. Part 2.a: Solve the problem of Exercise 3.1a (same mesh spacing h in x and y directions) using the GaussSeidel, and SOR methods for h = .1, h = .05, and h = .025. For SOR use the optimal for each h value. Since the routine poisson.m is set up for the matrix versions of the methods, it will be easier to go that route rather than attempt to use two dimensional storage. Let udiscrete 4

be the solution obtained by Gaussian Eliminations backslash command for each problem. Let ek = uk udiscrete be the dierence between the solution to the discrete system and the iterate at the k-th iteration, and stop the iteration when ||uk udiscrete||/||u0 udiscrete|| < 104 . Plot the value of ||ek ||/||e0 || versus k until convergence to compare the various methods. Solution:

From the graph above of Gauss-Siedel, the error term quickly drops for courser grids. This suggests that having lots of interior points points, and adding ner details and complexity to the solution obviously comes at a computational cost.

From the graph above of the SOR errors for dierent grid sizes, the trend of computational cost per grid renement is less obvious than with the Gauss-Siedel graph. However, the error smoothly drops o for the coarsest grid with spacing h = 0.1. Having ner grids introduced variations in magnitude of the error per increasing iteration of the method. Clearly we dont get as smooth a rate of convergence with ner grids and with renement you can see the issues with the initial iterations. Eventually (approximately 11 iterations in) the trend is visible and the errors for the ner grids are larger than the h = 0.1 spaced grid. In the following three graphs, the Gauss-Siedel error is clearly the dominant error as compared to SOR for any of the grid spacings tested. After a few initial iterations, its always larger. From lecture (and countless other sources), we know that the work for SOR is O(m3 logm) and for Gauss-Siedel is O(m4 logm). The results below at least qualitatively/visually concur with these big-O estimations in that SOR is faster.

Part 2.b: Since you know the exact solution to the PDE for this problem, dene E k = uk upde where upde is the true solution to the PDE at the grid points. Also plot the value of ||E k ||/||E 0 || versus k for the SOR method with the optimal for the problem h = .1. Compare this to the curve obtained earlier for the discrete error. (Note: For h, where h2 >> 104 , you shouldnt be surprised to see ||E k ||/||E 0 || stagnating before you have solved the DISCRETE system exactly, since truncation error will be reached earlier than the tolerance that was set for the solution of the discrete system. Do you observe this?) Solution: When plotted directly against each other (see graph below), there appears to be no dierence (both errors were computed with tolerance of 104 ).

When taking dierence between the two errors at each respective iteration, we can see dierences on the order of 105 . The error computed with the discrete solution, had larger errors at every iteration. Using the discrete solution also took one iteration longer to converge within tolerance (23 iterations vs 22). So, yes, in a sense the ||E k ||/||E 0 || stagnates before we solve the DISCRETE system exactly. However, it wasnt directly apparent from the graphs and it doesnt stagnate that much.

Part 2.c: For h = .025, plot the number of iterations required by SOR as a function of . You should see that the predicted by the theory gives the lowest number of iterations. Notice that the number of iterations might change rapidly for < opt , but close to opt . Solution: The optimal occurs at =
2 1+sin(h)

2 1+sin(0.025)

1.8545.

Visually from the graphs below we can see that below this optimal value, the number of iterations does rapidly change. In general, the number of iterations rapidly decreases to opt from the left. From the right, using larger > opt gives a slight increase number of iterations. This can be seen in better detail in the second graph.

10

Code: There are 4 matlab les for this problem. The main program is in problem2Main.m. 11

This le runs through all parts of the problem, computing the respective normed errors and plotting them. The three remaining les are helper functions that compute solutions to the problem. discreteSoln.m computes the discrete solution to the problem and is heavily modied from books poisson.m le. GS.m computes the solution using Gauss-Siedel method. SOR.m computes the solution using the SOR method. problem2Main.m a = 0; b = 1; tol = 10^-4; maxItr = 10000; m = [ 9; 19; 39]; %number of internal grid points h = [ 0.1; 0.05; 0.025]; %grid spacing, same in x,y dirs omegaOpt = 2*((1 + sin(pi*h)).^-1); %optimal omega based on h

n = 1; %compute discrete solution via backslash command (Gaussian elimination) [Udiscrete1, F1, Upde1] = discreteSoln(a,b,m(n)); %want to keep the boundary conditions but guess at the internals U1 = Udiscrete1; U1(2:m+1,2:m+1) = ones(m,m); %find errors from Gauss-Seidel [errInfGS_1, err0GS_1] = GS(tol, m(n), h(n), maxItr, F1, U1, Udiscrete1); %find errors from SOR [errInfSOR_1, err0SOR_1] = SOR(tol, m(n), h(n), maxItr, F1, U1, Udiscrete1, omegaOpt(n));

%now repeat for the other two desired %grid spacings to get info for plots n = 2; [Udiscrete2, F2, Upde2] = discreteSoln(a,b,m(n)); U2 = Udiscrete2; U2(2:m+1,2:m+1) = ones(m,m); [errInfGS_2, err0GS_2] = GS(tol, m(n), h(n), maxItr, F2, U2, Udiscrete2); [errInfSOR_2, err0SOR_2] = SOR(tol, m(n), h(n), maxItr, F2, U2, Udiscrete2, omegaOpt(n)); n = 3; [Udiscrete3, F3, Upde3] = discreteSoln(a,b,m(n)); U3 = Udiscrete3; U3(2:m+1,2:m+1) = ones(m,m); 12

[errInfGS_3, err0GS_3] = GS(tol, m(n), h(n), maxItr, F3, U3, Udiscrete3); [errInfSOR_3, err0SOR_3] = SOR(tol, m(n), h(n), maxItr, F3, U3, Udiscrete3, omegaOpt(n));

%plot the errors figure(1) plot(errInfGS_1, k); hold on, plot(errInfGS_2, g); plot(errInfGS_3, b); title(||e^k||/||e^0|| Gauss-Seidel for varying grid spacings) legend(h = 0.1,h=0.05,h=0.025) ylabel(||e^k||/||e^0||) xlabel(iteration #, k) figure(2) plot(errInfSOR_1, k); hold on, plot(errInfSOR_2, g); plot(errInfSOR_3, b); title(||e^k||/||e^0|| Gauss-Seidel for varying grid spacings) legend(h = 0.1,h=0.05,h=0.025) ylabel(||e^k||/||e^0||) xlabel(iteration #, k) figure(3) plot(errInfGS_1, k); hold on, plot(errInfSOR_1, g); title(||e^k||/||e^0|| Gauss-Seidel vs SOR for h = 0.1) legend(GS, SOR) ylabel(||e^k||/||e^0||) xlabel(iteration #, k) figure(4) plot(errInfGS_1, k); hold on, plot(errInfSOR_1, g); title(||e^k||/||e^0|| Gauss-Seidel vs SOR for h = 0.05) legend(GS, SOR) ylabel(||e^k||/||e^0||) xlabel(iteration #, k) figure(5) plot(errInfGS_1, k); 13

hold on, plot(errInfSOR_1, g); title(||e^k||/||e^0|| Gauss-Seidel vs SOR for h = 0.025) legend(GS, SOR) ylabel(||e^k||/||e^0||) xlabel(iteration #, k)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %part b - discrete vs exact/pde error in SOR for h = 0.1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% n = 1; %find errors from SOR using pde solution as true solution [errInfSOR_1pde, err0SOR_1pde] = SOR(tol, m(n), h(n), maxItr, F1, U1, Upde1, omegaOpt(n)); %they look the same figure(6) plot(errInfSOR_1, -ok); hold on, plot(errInfSOR_1pde, -xg); title(||e^k||/||e^0|| vs ||E^k||/||E^0|| SOR errors for h = 0.1) legend(discrete, pde) ylabel(||e^k||/||e^0||) xlabel(iteration #, k) %the two vector lengths are off by 1 %so just cut off the extra value and then subtract errDiff = errInfSOR_1(1:22) - errInfSOR_1pde; %slight diff figure(7) plot(errDiff, -ok) title(difference in SOR errors (discrete - pde error) ylabel(||e^k||/||e^0|| - ||E^k||/||E^0||)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %part c - SOR iterations vs omega using h = 0.025, or m = 39 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% h = 0.025; m = 39; tol = 10^-10 maxItr = 10000; 14

omega = (0.1:0.1:1.9); for n = 1:size(omega) [errInfSOR, err0SOR_] = SOR(tol, m, h, maxItr, F3, U3, Udiscrete3, omega(n)); numItr(n) = max(size(errInfSOR)); end figure(8) plot(omega, numItr, -ok) title(# iterations required by SOR as func of omega, tol = 10^-10, maxItr = 10000) xlabel(omega) ylabel(# iterations)

%now look close to omega_opt tol = 10^-10 omega = (1.65:0.01:1.95); for n = 1:size(omega) [errInfSOR, err0SOR_] = SOR(tol, m, h, maxItr, F3, U3, Udiscrete3, omega(n)); numItr2(n) = max(size(errInfSOR)); end figure(9) plot(omega, numItr2, -ok) title(Number of iterations required by SOR as a function of omega, tol = 10^-10, maxItr = xlabel(omega) ylabel(# iterations) discreteSoln.m function [Udiscrete, F, Upde] = discreteSoln(a,b,m) % %computes discetized solution for problem 3.1 %Inputs: %a = left/lower end of interval for grid %b = right/upper end of interval for grid %m = number of internal grid points perdirection (m*m total) % %Outputs %Udiscrete = discrete solution %F = right side of AU = F %Upde = true solution for the grid using the known pde h = (b-a)/(m+1); x = linspace(a,b,m+2); y = linspace(a,b,m+2); [X,Y] = meshgrid(x,y);

% grid points x including boundaries % grid points y including boundaries % 2d arrays of x,y values 15

X = X; Y = Y; Iint = 2:m+1; Jint = 2:m+1; Xint = X(Iint,Jint); Yint = Y(Iint,Jint);

% transpose so that X(i,j),Y(i,j) are % coordinates of (i,j) point % indices of interior points in x % indices of interior points in y % interior points

f = @(x,y) 1.25*exp(x+y/2); rhs = f(Xint,Yint);

% f(x,y) function

% evaluate f at interior points for right hand side % rhs is modified below for boundary conditions. % true solution for test problem

Upde = exp(X+Y/2);

% set boundary conditions around edges of usoln array: Udiscrete = Upde; % % % % % adjust rhs(:,1) rhs(:,m) rhs(1,:) rhs(m,:) the rhs to = rhs(:,1) = rhs(:,m) = rhs(1,:) = rhs(m,:) % use true solution for this test problem This sets full array, but only boundary values are used below. For a problem where utrue is not known, would have to set each edge of usoln to the desired Dirichlet boundary values.

include boundary terms: - Udiscrete(Iint,1)/h^2; - Udiscrete(Iint,m+2)/h^2; - Udiscrete(1,Jint)/h^2; - Udiscrete(m+2,Jint)/h^2;

% convert the 2d grid function rhs into a column vector for rhs of system: F = reshape(rhs,m*m,1); % I e T S A form matrix A: = speye(m); = ones(m,1); = spdiags([e -4*e e],[-1 0 1],m,m); = spdiags([e e],[-1 1],m,m); = (kron(I,T) + kron(S,I)) / h^2;

% Solve the linear system: Uvec = A\F; Udiscrete(Iint,Jint) = reshape(Uvec,m,m); F = Udiscrete; 16

F(2:m+1,2:m+1) = f(Xint,Yint); GS.m function [errInf, err0] = GS(tol, m, h, maxItr, F, U, Usoln) % %computes the solution to the problem given in 3.1 using the SOR method %Inputs: % tol = tolerance % m = number of internal grid points per axis (so m^2 total internal points) % h = grid spacing %maxIter = max number of iterations for convergence to error within tol before quitting %F = right hand side of AU = F %U = initial guess at solution %Usoln =discritized solution used in computing error % %Outputs: %errInf = vector of ||E^k||/||E^0|| values for each iteration performed using inf norm %err0 = initial ||E^0||

err = U - Usoln; err0 = norm(reshape(err, (m+2)*(m+2),1), inf); for k = 1:maxItr for j = 2:(m+1) for i = 2:(m+1) U(i,j) = 0.25*(U(i-1,j) + U(i+1,j) + U(i,j-1) + U(i,j+1) - (h^2)*F(i,j)); end end err = U - Usoln; % get difference for every internal point errInf(k) = norm(reshape(err, (m+2)*(m+2),1), inf); errInf(k) = errInf(k)/err0; if errInf(k) < tol break end end SOR.m function [errInf, err0] = SOR(tol, m, h, maxItr, F, U, Usoln, omega) % %computes the solution to the problem given in 3.1 using the SOR method 17

%Inputs: % tol = tolerance % m = number of internal grid points per axis (so m^2 total internal points) % h = grid spacing %maxIter = max number of iterations for convergence to error within tol before quitting %F = right hand side of AU = F %U = initial guess at solution %Usoln =discritized solution used in computing error %omega = SOR parameter between 0 and 2 % %Outputs: %errInf = vector of ||E^k||/||E^0|| values for each iteration performed using inf norm %err0 = initial ||E^0|| Unew = U; err0 = norm(reshape(U - Usoln, (m+2)*(m+2),1), inf);

for k = 1:maxItr for j = 2:m+1 for i = 2:m+1 Unew(i,j) = omega*(0.25*(Unew(i-1,j) + U(i+1,j) + Unew(i,j-1) + U(i,j+1) - (h^2 end end err = U - Usoln; % get difference for every internal point errInf(k) = norm(reshape(err, (m+2)*(m+2),1), inf)/err0; if errInf(k) < tol break end U = Unew; end Part 3 - Conjugate Gradient: Part 3.a: In developing the conjugate gradient algorithm, we chose k to make pk+1 A-conjugate to pk . Prove that this choice of k also ensures that pk+1 is A-conjugate to all previous direction vectors pj , j k. Here A is SPD. (Hint: You will have to make an inductive assumption.) Solution: Via inductive hypothesis, assume that pj is A-conjugate to pk for all j < k. We know from the lecture notes that rk+1 is orthogonal to p0 , ..., pk . Then the next step in the sequence begins with denition.

18

pK+1 = rk+1 + k pk pk+1 (Apj ) = (Apk Apj ) = k (pk Apj ) From here we also know that pj is in the p0 , ..., pk = r0 , Ar0 , ..., Aj r0 so that Apj will be in the space A-conjugate to... Part 3.b: Suppose A is an m m matrix that is SPD. Suppose we use an initial guess with conjugate gradient that gives e0 = c2 v2 + c8 v8 where Av2 = 2 v2 , Av8 = 8 v8 . After two steps of conjugate gradient e2 = 2 (A)e0 where 2 (A) is a quadratic polynominal in A. What is 2 (A) and what is e2 if the three s are distinct? Solution: From the problem statement, we know that 2 and 8 are eigenvalues of the matrix A, with corresponding/respective eigenvectors v2 and v8 . Hence both 2 and 8 satisfy the equation det(A I) = 0. From this we can derive a second-order characteristic polynomial for which 2 and 8 are roots. Requiring this polynomial to be equal to I at 0, gives the error e2 as 0. So our desired values are: 2 (A) = (A 2 I)(A 8 I) e2 = 0 Part 3.c: Please do help on pcg in Matlab to see how to use Matlabs conjugate gradient algorithm, which is pcg with no preconditioning. Solve the problem of Exercise 3.1a using the conjugate gradient method and Matlabs solver. Use all zeros as the initial guess, so the right hand side becomes the initial residual! Then give Matlabs solver a tolerance of 104 which will enforce ||rk ||2 /||r0 ||2 104 . Note that this Matlab program allows you to tell it how to multiply a matrix A times a vector through afun and how to nd z where M z = r through mfun (used for preconditioning). You can ignore this if you choose to specify the matrix A (and matrix M if preconditioning is used). Solution:

19

The computed error using Matlabs conjugate gradient method relative to true solution of the PDE was 5.326 10004 . The relative residual norm norm(F A usoln)/norm(F ) for the system was 8.567 10005 and the method converged within error tolerance on iteration 40. Code: problem3.m

%solves the Poisson problem u_{xx} + u_{yy} = f(x,y) % on [a,b] x [a,b] using a conjugate gradient method. % %Mostly the same as poisson.m, new code at bottom! tol = 10^-4; a = 0; b = 1; m = 20; h = (b-a)/(m+1); x = linspace(a,b,m+2); y = linspace(a,b,m+2); [X,Y] = meshgrid(x,y); X = X; Y = Y; Iint = 2:m+1; Jint = 2:m+1;

% grid points x including boundaries % grid points y including boundaries % 2d arrays of x,y values % transpose so that X(i,j),Y(i,j) are % coordinates of (i,j) point % indices of interior points in x % indices of interior points in y 20

Xint = X(Iint,Jint); % interior points Yint = Y(Iint,Jint); f = @(x,y) 1.25*exp(x+y/2); % f(x,y) function rhs = f(Xint,Yint); utrue = exp(X+Y/2); usoln = utrue; % adjust the rhs to include boundary terms: rhs(:,1) = rhs(:,1) - usoln(Iint,1)/h^2; rhs(:,m) = rhs(:,m) - usoln(Iint,m+2)/h^2; rhs(1,:) = rhs(1,:) - usoln(1,Jint)/h^2; rhs(m,:) = rhs(m,:) - usoln(m+2,Jint)/h^2; % convert the 2d grid function rhs into a column vector for rhs of system: F = reshape(rhs,m*m,1); % form matrix A: I = speye(m); e = ones(m,1); T = spdiags([e -4*e e],[-1 0 1],m,m); S = spdiags([e e],[-1 1],m,m); A = (kron(I,T) + kron(S,I)) / h^2; %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %NEW/DIFFERENT CODE from poisson.m %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% tol = 10^-4; maxItr = 10000; %A is negative definite, so we want to solve the related system -Au=-F %so that well have a SPD matrix to through into matlab A = -1.*A; F = -1*F; [x,flag,relres,iter,resvec] = pcg(A,F,tol, maxItr, [], [], []); usoln(Iint,Jint) = reshape(x,m,m);

%same old plot code err = max(max(abs(usoln-utrue))); fprintf(Error relative to true solution of PDE = %10.3e \n,err) contour(X,Y,usoln,30,k) axis([a b a b]) daspect([1 1 1]) title(Contour plot of computed solution) hold off

21