Chapter 1
Principles of Black-Box Optimization
Unlocking optimal solutions when you cannot peek inside the box: This chapter dives into the theoretical and practical bedrock of black-box optimization, arming you to tackle problems where function structure, gradients, or even the search space itself resist illumination. We dissect the fundamental challenges, contrasting algorithmic strategies and formalizing how uncertainty can be harnessed rather than feared, to make informed, efficient, and robust decisions-setting the stage for world-class optimization in real and simulated domains.
1.1 Nature of Black-Box Functions
Black-box functions are defined by the absence of a known analytic form and the inaccessibility of explicit gradient information. Unlike classical functions that are represented by closed-form expressions or those amenable to symbolic differentiation, black-box functions resist direct analysis due to intrinsic complexity or the way they are defined. The only method to obtain information about such functions is by querying an underlying system-often a simulation, experiment, or deployed model-without knowledge of the internal workings. As a result, function evaluations become atomic operations whose internal mechanics remain entirely concealed.
Consider a function f :
