Numerical Methods

Preface xiii
Chapter 1: MATHEMATICAL MODELING 1
1.1 Modeling in Computer Animation 2
1.1.1 A Model Robe 2
1.2 Modeling in Physics: Radiation Transport 4
1.3 Modeling in Sports 6
1.4 Ecological Models 8
1.5 Modeling a Web Surfer and Google 11
1.5.1 The Vector Space Model 11
1.5.2 Google's PageRank 13
1.6 Chapter 1 Exercises 14
Chapter 2: BASIC OPERATIONS WITH MATLAB 19
2.1 Launching MATLAB 19
2.2 Vectors 20
2.3 Getting Help 22
2.4 Matrices 23
2.5 Creating and Running .m Files 24
2.6 Comments 25
2.7 Plotting 25
2.8 Creating Your Own Functions 27
2.9 Printing 28
2.10 More Loops and Conditionals 29
2.11 Clearing Variables 31
2.12 Logging Your Session 31
2.13 More Advanced Commands 31
2.14 Chapter 2 Exercises 32
Chapter 3: MONTE CARLO METHODS 41
3.1 A Mathematical Game of Cards 41
3.1.1 The Odds in Texas Holdem 42
3.2 Basic Statistics 46
3.2.1 Discrete Random Variables 48
3.2.2 Continuous Random Variables 51
3.2.3 The Central Limit Theorem 53
3.3 Monte Carlo Integration 56
3.3.1 Buffon's Needle 56
3.3.2 Estimating p 58
3.3.3 Another Example of Monte Carlo Integration 60
3.4 Monte Carlo Simulation of Web Surfing 64
3.5 Chapter 3 Exercises 67
Chapter 4: SOLUTION OF A SINGLE NONLINEAR EQUATION IN ONE UNKNOWN 71
4.1 Bisection 75
4.2 Taylor's Theorem 80
4.3 Newton's Method 83
4.4 Quasi-Newton Methods 89
4.4.1 Avoiding Derivatives 89
4.4.2 Constant Slope Method 89
4.4.3 Secant Method 90
4.5 Analysis of Fixed Point Methods 93
4.6 Fractals, Julia Sets, and Mandelbrot Sets 98
4.7 Chapter 4 Exercises 102
Chapter 5: FLOATING-POINT ARITHMETIC 107
5.1 Costly Disasters Caused by Rounding Errors 108
5.2 Binary Representation and Base 2 Arithmetic 110
5.3 Floating-Point Representation 112
5.4 IEEE Floating-Point Arithmetic 114
5.5 Rounding 116
5.6 Correctly Rounded Floating-Point Operations 118
5.7 Exceptions 119
5.8 Chapter 5 Exercises 120
Chapter 6: CONDITIONING OF PROBLEMS; STABILITY OF ALGORITHMS 124
6.1 Conditioning of Problems 125
6.2 Stability of Algorithms 126
6.3 Chapter 6 Exercises 129
Chapter 7: DIRECT METHODS FOR SOLVING LINEAR SYSTEMS AND LEAST SQUARES PROBLEMS 131
7.1 Review of Matrix Multiplication 132
7.2 Gaussian Elimination 133
7.2.1 Operation Counts 137
7.2.2 LU Factorization 139
7.2.3 Pivoting 141
7.2.4 Banded Matrices and Matrices for Which Pivoting Is Not Required 144
7.2.5 Implementation Considerations for High Performance 148
7.3 Other Methods for Solving Ax = b 151
7.4 Conditioning of Linear Systems 154
7.4.1 Norms 154
7.4.2 Sensitivity of Solutions of Linear Systems 158
7.5 Stability of Gaussian Elimination with Partial Pivoting 164
7.6 Least Squares Problems 166
7.6.1 The Normal Equations 167
7.6.2 QR Decomposition 168
7.6.3 Fitting Polynomials to Data 171
7.7 Chapter 7 Exercises 175
Chapter 8: POLYNOMIAL AND PIECEWISE POLYNOMIAL INTERPOLATION 181
8.1 The Vandermonde System 181
8.2 The Lagrange Form of the Interpolation Polynomial 181
8.3 The Newton Form of the Interpolation Polynomial 185
8.3.1 Divided Differences 187
8.4 The Error in Polynomial Interpolation 190
8.5 Interpolation at Chebyshev Points and chebfun 192
8.6 Piecewise Polynomial Interpolation 197
8.6.1 Piecewise Cubic Hermite Interpolation 200
8.6.2 Cubic Spline Interpolation 201
8.7 Some Applications 204
8.8 Chapter 8 Exercises 206
Chapter 9: NUMERICAL DIFFERENTIATION AND RICHARDSON EXTRAPOLATION 212
9.1 Numerical Differentiation 213
9.2 Richardson Extrapolation 221
9.3 Chapter 9 Exercises 225
Chapter 10: NUMERICAL INTEGRATION 227
10.1 Newton-Cotes Formulas 227
10.2 Formulas Based on Piecewise Polynomial Interpolation 232
10.3 Gauss Quadrature 234
10.3.1 Orthogonal Polynomials 236
10.4 Clenshaw-Curtis Quadrature 240
10.5 Romberg Integration 242
10.6 Periodic Functions and the Euler-Maclaurin Formula 243
10.7 Singularities 247
10.8 Chapter 10 Exercises 248
Chapter 11: NUMERICAL SOLUTION OF THE INITIAL VALUE PROBLEM FOR ORDINARY DIFFERENTIAL EQUATIONS 251
11.1 Existence and Uniqueness of Solutions 253
11.2 One-Step Methods 257
11.2.1 Euler's Method 257
11.2.2 Higher-Order Methods Based on Taylor Series 262
11.2.3 Midpoint Method 262
11.2.4 Methods Based on Quadrature Formulas 264
11.2.5 Classical Fourth-Order Runge-Kutta and Runge-Kutta-Fehlberg Methods 265
11.2.6 An Example Using MATLAB's ODE Solver 267
11.2.7 Analysis of One-Step Methods 270
11.2.8 Practical Implementation Considerations 272
11.2.9 Systems of Equations 274
11.3 Multistep Methods 275
11.3.1 Adams-Bashforth and Adams-Moulton Methods 275
11.3.2 General Linear m-Step Methods 277
11.3.3 Linear Difference Equations 280
11.3.4 The Dahlquist Equivalence Theorem 283
11.4 Stiff Equations 284
11.4.1 Absolute Stability 285
11.4.2 Backward Differentiation Formulas (BDF Methods) 289
11.4.3 Implicit Runge-Kutta (IRK) Methods 290
11.5 Solving Systems of Nonlinear Equations in Implicit Methods 291
11.5.1 Fixed Point Iteration 292
11.5.2 Newton's Method 293
11.6 Chapter 11 Exercises 295
Chapter 12: MORE NUMERICAL LINEAR ALGEBRA: EIGENVALUES AND ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS 300
12.1 Eigenvalue Problems 300
12.1.1 The Power Method for Computing the Largest Eigenpair 310
12.1.2 Inverse Iteration 313
12.1.3 Rayleigh Quotient Iteration 315
12.1.4 The QR Algorithm 316
12.1.5 Google's PageRank 320
12.2 Iterative Methods for Solving Linear Systems 327
12.2.1 Basic Iterative Methods for Solving Linear Systems 327
12.2.2 Simple Iteration 328
12.2.3 Analysis of Convergence 332
12.2.4 The Conjugate Gradient Algorithm 336
12.2.5 Methods for Nonsymmetric Linear Systems 334
12.3 Chapter 12 Exercises 345
Chapter 13: NUMERICAL SOLUTION OF TWO-POINT BOUNDARY VALUE PROBLEMS 350
13.1 An Application: Steady-State Temperature Distribution 350
13.2 Finite Difference Methods 352
13.2.1 Accuracy 354
13.2.2 More General Equations and Boundary Conditions 360
13.3 Finite Element Methods 365
13.3.1 Accuracy 372
13.4 Spectral Methods 374
13.5 Chapter 13 Exercises 376
Chapter 14: NUMERICAL SOLUTION OF PARTIAL DIFFERENTIAL EQUATIONS 379
14.1 Elliptic Equations 381
14.1.1 Finite Difference Methods 381
14.1.2 Finite Element Methods 386
14.2 Parabolic Equations 388
14.2.1 Semidiscretization and the Method of Lines 389
14.2.2 Discretization in Time 389
14.3 Separation of Variables 396
14.3.1 Separation of Variables for Difference Equations 400
14.4 Hyperbolic Equations 402
14.4.1 Characteristics 402
14.4.2 Systems of Hyperbolic Equations 403
14.4.3 Boundary Conditions 404
14.4.4 Finite Difference Methods 404
14.5 Fast Methods for Poisson's Equation 409
14.5.1 The Fast Fourier Transform 411
14.6 Multigrid Methods 414
14.7 Chapter 14 Exercises 418
APPENDIX A REVIEW OF LINEAR ALGEBRA 421
A.1 Vectors and Vector Spaces 421
A.2 Linear Independence and Dependence 422
A.3 Span of a Set of Vectors; Bases and Coordinates; Dimension of a Vector Space 423
A.4 The Dot Product; Orthogonal and Orthonormal Sets; the Gram-Schmidt Algorithm 423
A.5 Matrices and Linear Equations 425
A.6 Existence and Uniqueness of Solutions; the Inverse; Conditions for Invertibility 427
A.7 Linear Transformations; the Matrix of a Linear Transformation 431
A.8 Similarity Transformations; Eigenvalues and Eigenvectors 432
APPENDIX B TAYLOR'S THEOREM IN MULTIDIMENSIONS 436
References 439
Index 445