Preface

Once in a while you get shown the light

In the strangest of places if you look at it right.

Robert Hunter

Preface to the First Edition

We intend this book to serve as a first graduate-level text for applied mathematicians, scientists, and engineers. We hope that these students have had some exposure to numerics, but the book is self-contained enough to accommodate students with no numerical background. Students should know a computer programming language, though.

In writing the text, we have tried to adhere to three principles:

1. The book should cover a significant range of numerical methods now used in applications, especially in scientific computation involving differential equations.
2. The book should be appropriate for mathematics students interested in the theory behind the methods.
3. The book should also appeal to students who care less for rigorous theory than for the heuristics and practical aspects of the methods.

The first principle is a matter of taste. Our omissions may appall some readers; they include polynomial root finders, linear and nonlinear programming, digital filtering, and most topics in statistics. On the other hand, we have included topics that receive short shrift in many other texts at this level. Examples include:

• Multidimensional interpolation, including interpolation on triangles.
• Quasi-Newton methods in several variables.
• A brief introduction to multigrid methods.
• Conjugate-gradient methods, including error estimates.
• Rigorous treatment of the QR method for eigenvalues.
• An introduction to adaptive methods for numerical integration and ordinary differential equations.
• A thorough treatment of multistep schemes for ordinary differential equations (ODEs).
• Consistency, stability, and convergence of finite-difference schemes for partial differential equations (PDEs).
• An introduction to finite-element methods, including basic convergence arguments and methods for time-dependent problems.

All of these topics are prominent in scientific applications.

The second and third principles conflict. Our strategy for addressing this conflict is threefold. First, most sections of the book have a "pyramid" structure. We begin with the motivation and construction of the methods, then discuss practical considerations associated with their implementation, then present rigorous mathematical details. Thus, students in a "methods" course can concentrate on motivation, construction, and practical considerations, perhaps grazing from the mathematical details according to the instructor's tastes. Students in an "analysis" course should delve into the mathematical details as well as the practical considerations.

Second, we have included Chapter 1, "Some Useful Tools," which reviews essential notions from undergraduate analysis and linear algebra. Mathematics students should regard this chapter as a review; engineering and applied science students may profit by reading it thoroughly.

Third, at the end of each chapter are both theoretical and computational exercises. Engineers and applied scientists will probably concentrate on the computational exercises. Mathematicians should work a variety of both theoretical and computational problems. Numerical analysis without computation is a sterile enterprise.

The book's format allows instructors to use it in either of two modes. For a "methods" course, one can cover a significant set of topics in a single semester by covering the motivation, construction, and practical considerations. At the University of Wyoming, we teach such a course for graduate engineers and geophysicists. For an "analysis" course, one can construct a two- or three-semester sequence that involves proofs, computer exercises, and projects requiring written papers. At Wyoming, we offer a two-semester course along these lines for students in applied mathematics.

Most instructors will want to skip topics. The following remarks may help avoid infelicitous gaps:

• We typically start our courses with Chapter 2. Sections 2.2 and 2.3 (on polynomial interpolation) and 2.7 (on least squares) seem essential.
• Even if one has an aversion to direct methods for linear systems, it is worthwhile to discuss Sections 3.1 and 3.3. Also, the introduction to matrix norms and condition numbers in Sections 1.4 and 3.6 is central to much of numerical analysis.
• While Sections 4.1-4.4 contain the traditional core material on nonlinear equations, our experience suggests that engineering students profit from some coverage of the multidimensional methods discussed in Sections 4.6 and 4.7.
• Even in a proof-oriented course, one might reasonably leave some of the theory in Sections 5.3 and 5.4 for independent reading. Section 5.6, The Conjugate-Gradient Method, is independent of earlier sections in that chapter.
• Taste permitting, one can skip Chapter 6, Eigenvalue Problems, completely.
• One should cover Section 7.1 and at least some of Section 7.2, Newton-Cotes Formulas, in preparation for Chapter 8. Engineers use Gauss quadrature so often, and the basic theory is so elegant, that we seldom skip Section 7.4.
• We rarely cover Chapter 8 (on ODEs) completely. Still, in preparation for Chapter 9, one should cover at least the most basic material - through Euler methods - from Sections 8.1 and 8.2.
• While many first courses in numerics omit the treatment of PDEs, at least some coverage of Chapter 9 seems crucial for virtually all of the students who take our courses.
• Chapter 10, on finite-element methods, emphasizes analysis at the expense of coding, since the latter seems to lie at the heart of most semester-length engineering courses on the subject. It is hard to get this far in a one-semester "methods" course.

We owe tremendous gratitude to many people, including former teachers and many remarkable colleagues too numerous to list. We thank the students and colleagues who graciously endured our drafts and uncovered an embarrassing number of errors. Especially helpful were the efforts of Marian Anghel, Damian Betebenner, Bryan Bornholdt, Derek Mitchum, Patrick O'Leary, Eun-Jae Park, Gamini Wickramage, and the amazingly keen-eyed Li Wu. (Errors undoubtedly remain; they are our fault.) The first author wishes to thank the College of Engineering and Mathematics at the University of Vermont, at which he wrote early drafts during a sabbatical year. Finally, we thank our wives, Adele Aldrich and Lynne Ipiña, to whom we dedicate the book. Their patience greatly exceeds that required to watch a book being written.

Midnight on a carousel ride,

Reaching for the gold ring down inside.

Never could reach

It just slips away when I try.

Robert Hunter

Preface to the Second Edition

In producing this second edition of Numerical Analysis for Applied Science, I pursued two goals. First, I incorporated many suggestions and corrections made by people who have used the book since it first appeared in print. I owe my sincerest thanks to colleagues who have shared these improvements over the years. Professor Scott Fulton of Clarkson University and Professor Aleksey Telyakovskiy of the University of Nevada at Reno deserve special thanks for their extraordinary generosity in this respect.

Second, I have incorporated new topics or expanded treatments of existing topics, to reflect some of the evolving applications of numerical analysis during the past two decades. Among the new contents are the following:

• A description of the symmetric successive overrelaxation method in Chapter 5, to facilitate an expanded discussion of preconditioners later in the chapter.
• A separate section in Chapter 5 on multigrid methods for solving linear systems, including more detail than the first edition's brief discussion.
• A revised section in Chapter 5 on the conjugate-gradient method, including a more detailed discussion of preconditioners.
• A short discussion in Chapter 5 of the method of steepest descent.
• More details on the power and QR methods for computing eigenvalues in Chapter 6.
• An introduction in Chapter 6 to the singular value decomposition and its application to principal components analysis.
• Revised and expanded discussion in Chapter 9 of the approximation of elliptic PDEs by finite-difference methods, including the treatment of irregular boundaries in two dimensions.
• Revised approximation error estimates for the finite-element method in Chapter 10.
• An additional section in Chapter 10 on the condition number of the stiffness matrix, a major motivation for many advances in numerical linear algebra over the past three decades.
• Seven new pseudocodes, bringing the total to 32.
• More than twice as many problems as appeared in the first edition.

Also, I moved a section on eigenvalues and matrix norms to Chapter 1 and a section on the condition number to Section 3.2, to make it easier to skip most of Chapter 3 in favor of...