1
Introduction to Surrogate Models

1.1 What Is Surrogate Modeling?

In engineering design, a quantity of interest (QoI) is a function of many input parameters or design variables. For example, the maximum stress in a cantilevered beam is a function of the beam length, cross-sectional dimensions, and applied loads. Such a functional relationship is often governed by complex physics, and it may not exist in a simple analytical form. Many engineering mechanics problems are governed by integro-differential equations, which often require computationally intensive numerical methods to find the relationship. Finite element methods, computational fluid dynamics, and rigid-body dynamics comprise a short list of examples of numerical methods used to establish the relationship between QoI and input parameters or design variables. Even if modern technology has enhanced computational power significantly, engineers still anticipate a significant amount of computational time to calculate the QoI.

Figure 1.1(a) shows an example of dispersed multiphase flow, where numerous particles interact with shock waves generated from explosives [1]. Physics of shock interaction with a cloud of particles has many interesting applications, such as explosive volcanic eruptions, dust explosions in coal mines, and supernovae [2]. Simulation of shock interaction with distributed particles, as shown in Figure 1.1(b), creates many computational challenges. Problems involving distributed particles present a wide range of length scales, from hundreds of nanometers (shock thickness) to meters (size of the domain) and an associated wide range of time scales from microseconds to minutes. Given this wide range of length and time scales, fully resolved particle simulations of dispersed multiphase flows involving a large number of particles are practically impossible, where the complete details of both the flow around the particles and the particle motion are captured. Instead, Euler-Lagrange formations are often adopted with point-particle force and heat transfer models to couple the particle momentum and energy with those of the surrounding fluid. While the governing Navier-Stokes equations are solved in the region outside the particles with a sufficiently high resolution in the Eulerian frame of reference, the motion of each particle is tracked by solving its equations of motion in the Lagrangian frame of reference. Even such an approximate model pushes the limit of current computational resources. The simulation in Figure 1.1(b), for example, took 15,000 seconds with 2,400 clusters in a high-performance computing facility at US National Laboratories.

In many engineering processes, multiple simulations are frequently required by changing input parameters or design variables. To find the optimal airfoil shape, for example, an engineer simulates the airflow around a wing for different design variables, such as length, curvature, aspect ratio, chord length, and even material. During design optimization, design variables are updated at each iteration to improve the objective function while satisfying constraints. Commonly, the design variables are updated more than 100 times before reaching an optimum design. Since the objective function and constraints need to be recalculated with each updated design, numerous simulations are required during the design optimization process. Other examples are design space exploration, sensitivity analysis, what-if analysis, reliability analysis, and uncertainty quantification. All of these examples require numerous simulations by changing input parameters or design variables. As shown in the previous example, if a single simulation is already computationally expensive, it becomes easily impractical to run numerous simulations.

Figure 1.1 Experiment and simulation of dispersed multiphase flow: shock wave interaction with a particle bed. (a) Experiment at Eglin Air Force Base and (b) simulation at the University of Florida.

Even if the process of calculating QoI is governed by complex physics and expensive numerical simulations, the relationship between the QoI and input parameters can be simplified. In the case of a cantilevered beam shown in Figure 1.2, for example, the maximum stress can be written as

where F is the applied force at the tip, L is the length of the beam, b is the width, and h is the height of the cross-section. Even if in this simple example the QoI (maximum stress) is expressed in an analytical form, we may consider that it is from complex physics and expensive numerical simulation. However, the relationship between maximum stress and applied force is linear, smax = aF, and it is inversely proportional to the height of the beam, smax = ßh-2. Therefore, without considering complex physics, it is possible to express the relationship using simple functions, such as smax = f(F, L, b, h). However, the issues of determining the functional form and calculating the coefficients remain.

Figure 1.2 Bending of a cantilevered beam.

Surrogate modeling is a powerful engineering tool when a QoI cannot be easily calculated (or measured), so an approximation of the QoI is used instead. It mimics (i.e., approximates) the behavior of the simulation model as closely as possible while being computationally cheap in evaluating the QoI. It is a so-called data-driven approach where it does not concern physics or underlying theory. Instead, it only concerns input-output behavior, and the relationship is represented using mathematical functions. This approach is also known as a response surface model [3], meta-model [4], reduced model [5], model emulator [6], proxy model [7], lower-fidelity model [8], behavioral modeling, or black-box modeling in different communities. When only a single input parameter is involved, the modeling process is known as curve-fitting.

1.2 Surrogate Models

The idea of surrogate modeling is to represent the relationship between the QoI and input parameters using known functional forms with unknown function coefficients. Different surrogate models can be defined based on how to select the functional forms and how to determine unknown coefficients. However, all of them share some common characteristics. As will be discussed in more detail later, the most important characteristic relates to samples. All surrogate models utilize samples to determine the unknown coefficients. Samples in a surrogate model mean data between input parameters and output QoIs. The dimension of a surrogate model refers to the number of input variables. Figure 1.3(a) shows an example of samples in a two-dimensional surrogate model. Each sample consists of the paired values of input parameters and output QoI. Let input parameters be x1 and x2 and output QoI be y. Then, a sample can be represented by S = (x1, x2, y). In general, the k-th sample of N-dimensional surrogate can be written as S(k) = (x(k), y(k)), where x = {x1, x2, ., xN}T is the vector of input sample locations. In this text, a sample refers to S(k), while a sample location refers to x(k). The example in Figure 1.3(a) has 21 samples. For a given input sample location, x(k), it is necessary to evaluate the output QoI sample y(k), which requires expensive numerical simulation or a physical experiment. The major cost of the surrogate model occurs at this stage; that is, each sample requires simulation/experimentation. Once m numbers of samples, S = {S(1), S(2), ., S(m)}, are obtained, Figure 1.3(b) shows a surrogate model that fits using these samples. The surrogate model can be simple polynomials or complicated statistical functions. However, it is assumed that the cost of constructing the surrogate model and evaluating the output QoI using the surrogate model is assumed to be negligible compared to that of evaluating the output QoI samples y(k). Under this assumption, the cost of a surrogate model can be determined based on the number of samples to construct it. In general, the accuracy of a surrogate model is proportional to the number of samples. Therefore, there is a trade-off between the accuracy of a surrogate model and the cost of constructing it.

Figure 1.3 An example of samples in a 2D domain and the corresponding surrogate model. (a) Sample locations and (b) samples and fitted surrogate model.

Surrogate modeling began in field experiments where the trend of a response change was measured by varying input parameters [3]. Normally, the input parameters were varied by two or three levels, and the change in the response was modeled using linear or quadratic polynomials. Since measured data from an experiment normally contained noise, it was important that the surrogate model filters out the measurement noise. The polynomial response surface (PRS) was designed for this particular purpose, where the functional form was assumed to be a linear combination of monomial bases with unknown coefficients. These coefficients were determined to minimize the error between the surrogate prediction and measured data. The errors...