Introduction To Global Optimization

2y ago
52 Views
3 Downloads
586.94 KB
43 Pages
Last View : 2m ago
Last Download : 2m ago
Upload by : Camryn Boren
Transcription

Introduction to Global OptimizationLeo LibertiLIX, École Polytechnique, Palaiseau F-91128, France(liberti@lix.polytechnique.fr)February 15, 2008AbstractAccurate modelling of real-world problems often requires nonconvex terms to be introduced in themodel, either in the objective function or in the constraints. Nonconvex programming is one of thehardest fields of optimization, presenting many challenges in both practical and theoretical aspects.The presence of multiple local minima calls for the application of global optimization techniques.This paper is a mini-course about global optimization techniques in nonconvex programming; it dealswith some theoretical aspects of nonlinear programming as well as with some of the current stateof-the-art algorithms in global optimization. The syllabus is as follows. Some examples of NonlinearProgramming Problems (NLPs). General description of two-phase algorithms. Local optimizationof NLPs: derivation of KKT conditions. Short notes about stochastic global multistart algorithmswith a concrete example (SobolOpt). In-depth study of a deterministic spatial Branch-and-Boundalgorithm, and convex relaxation of an NLP. Latest advances in bilinear programming: the theory ofreduction constraints.Contents1 Introduction21.1Scope of this paper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31.2Examples which require Global Optimization . . . . . . . . . . . . . . . . . . . . . . . . .41.3A brief history of global optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .62 The structure of Global Optimization Algorithms82.1Stochastic global phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .92.2Deterministic global phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .112.2.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13Example of solution by Branch-and-Select . . . . . . . . . . . . . . . . . . . . . . . . . . .132.3Fathoming3 Local Optimization of NLPs163.1Fundamental notions of convex analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . .163.2Necessary and sufficient conditions for local optimality . . . . . . . . . . . . . . . . . . . .183.3A local optimization algorithm: SQP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24

1 INTRODUCTION24 The SobolOpt algorithm255 The spatial Branch-and-Bound algorithm275.1Bounds tightening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .285.1.1Optimization-based bounds tightening (step 1) . . . . . . . . . . . . . . . . . . . .285.1.2Feasibility-based bounds tightening (step 2) . . . . . . . . . . . . . . . . . . . . . .285.2Choice of region (step 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .295.3Convex relaxation (step 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .295.3.1Reformulation to standard form. . . . . . . . . . . . . . . . . . . . . . . . . . . .295.3.2Convexification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32Local solution of the original problem (step 4) . . . . . . . . . . . . . . . . . . . . . . . . .325.4.1Branching on added variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .335.4.2Branching on original variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33Branching (step 7) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .335.45.56 Tight convex relaxations of bilinear problems347 Conclusion361IntroductionOptimization is a field of applied mathematics that deals with finding the extremal value of a functionin a domain of definition, subject to various constraints on the variable values. Historically, optimizationtechniques first came about in conjunction with problems linked with the logistics of personnel andtransportation management. Typically, the problems were modelled in terms of finding the minimum costconfiguration subject to all constraints be satisfied, where both the cost and the constraints were linearfunctions of the decision variables. These kinds of problems are all in the class of Linear Programming(LP). The most celebrated algorithm for solving such problems is the Simplex Algorithm, proposedby Dantzig in the late 40s/early 50s [Dan63]; its worst-case complexity is exponential in the number ofproblem variables, but it performs extremely efficiently in most practical instances.One other technique forsolving LP problems, the Ellipsoid Algorithm, exhibits polynomial complexity. Many free and commercialimplementations of the Simplex Algorithm exist nowadays, which are capable of solving extremely largescale instances with up to millions of variables and constraints.In many cases the nature of the problem explicitly requires integer or binary decision variables.Imposing such constraints on the variables makes the problem much more difficult to solve. MILP (MixedInteger Linear Programming) problems are problems where some (or all) variables are constrained totake integer values, and where the objective function and constraints are linear functions of the variables.Notwithstanding the linearity of the functions in the problem, the problem itself is nonlinear, since theintegrality constraints on the variables can be expressed as nonlinear functions. The techniques for solvingMILPs are very different from those for solving LPs. Typically, the solution of an entire LP problem(obtained by relaxing the integrality constraints on the integer variables, and called the LP relaxation) isrequired at each step of an algorithm that solves a MILP. The two most common algorithms employed in

1 INTRODUCTION3solving MILPs are the Cutting Plane algorithm and the Branch-and-Bound algorithm. Their worst-casecomplexity is exponential in the size of the instance.The true discriminant between “easy” and “hard” optimization problems in mathematical programming is the convex/nonconvex issue. A problem is convex if it is a minimization of a convex function(or a maximization of a concave function) where the admissible points are in a convex set. The fundamental result in convex analysis says that a locally optimal solution of a convex problem is also globallyoptimal (see Theorem 3.6 below). This is practically very important: since algorithmic termination testsoccur at each iteration in the algorithm, they must be computationally very efficient; thus, virtually alltermination conditions of optimization algorithms are local conditions, i.e. they test whether the currentsolution is locally optimal with respect to a pre-defined neighbourhood. If the problem is convex thisis enough to guarantee that the solution is globally optimal. Nonconvex problems, on the other hand,may have many different local optima, and choosing the best one is an extremely hard task. The field ofapplied mathematics that studies extremal locations of nonconvex functions subject to (possibly) nonconvex constraints is called Global Optimization. If some, or all the problem variables are constrained totake discrete values only, the problem is nonconvex even if the mathematical expressions in the problemare linear (this happens because the solution set is a discrete set of points, which is a nonconvex set).Such problems are often termed Combinatorial Optimization problem due to the combinatorial nature ofa discrete decision variable.In general, the global optimization problem is formulated in terms of finding the point x in a solutionspace set X (called the feasible region) where a certain function f : X T (called the objective function),attains a minimum or a maximum. T is any ordered set (usually a subset of R). The set X is usuallya subset of Rn defined by constraints g(x) T 0, where g is a set of m possibly nonlinear functions ofx and T are relations in { , , }. The extremal point x can then be written as (x1 , . . . , xn ), and thexi ’s are sometimes called decision variables. It is often practically useful to express the variable boundsexplicitly as xL x xU (xL , xU Rn ). Some of the variables may be constrained to only take integervalues (xi Z for all i in an index set Z {1, . . . n}). It is customary to write the global optimizationMixed-Integer Nonlinear Programming problem (MINLP) as follows: minxf (x) g(x)Tb(1) xL x xU xi Z i Z.Notice that expressing the integrality constraints on the variables can be reduced to imposing an appropriate constraint in form of a function. E.g., x1 only taking values 0,1 can be expressed by imposing theconstraint x1 (x1 1) 0. The variable bounds are also called variable ranges. It is sometimes convenientto introduce the problem constraints in the forma g(x) bwhere a, b Rm are lower and upper limits on the constraint values. Equality constraints are expressedby setting a b and one-way inequality constraints are expressed by setting a or b . In therest of the paper, both forms of expressing the constraints will be employed.1.1Scope of this paperThis paper has a didactical purpose, and therefore includes many details which would normally beassumed as known. It is meant to give a substantial overview of the field of global optimization ofnonconvex MINLPs. We focus our treatment on deterministic global optimization algorithms with an indepth treatment of the spatial Branch-and-Bound algorithm (see Sections 2.2 and 5); however, we also givesome insight to stochastic methods (see Sections 2.1, 4). Since most modern global optimization methodsuse local optimization as a tool, an overview of local optimization of NLPs is also given (see Section

1 INTRODUCTION43), which also contains much basic material about convexity and elementary multi-variate geometry andanalysis. Although this is mainly a theoretical paper, some examples have been included (see Section1.2). Section 6 covers more advanced material about an automatic way to provide very tight convexrelaxations of bilinear problems.1.2Examples which require Global OptimizationIn this section we shall see some examples of multi-extremal nonconvex programming problems. Solvingthese problems requires global optimization methods.1.1 Example (Haverly’s Pooling Problem)Haverly’s Pooling Problem, first introduced in [Hav78], is described visually in Fig. 1. We have threeinput feeds with different levels of percentage of sulphur in the crude. Accordingly, the unit costs of theinput feeds vary. Two of the feeds go into a mixing pool, which makes the level of sulphur percentagethe same in the two streams coming out of the pool. The various streams are then blended to make twoend products with different requirements on the sulphur percentage and different unit revenues. We havesome market demands on the end products and we want to determine the quantities of input crude ofeach type to feed into the networks so that the operations costs are minimized. This problem can bex113% Sulphur 6y11Blend 1 2.5% Sulphur 100 9Blend 2 1.5% Sulphur 200 15Poolx211% Sulphur 16x122% Sulphur 10y21y12y22Figure 1: Haverly’s Pooling Problem.formulated in various ways. We present here the what is known as the p-formulation of the HPP.minx,y,ps.t.6x11 16x21 10x12 9(y11 y21 ) 15(y12 y22 ) costx11 x21 y11 y12 0x12 y21 y22 0y11 y21 100y12 y22 2003x11 x21 p(y11 y12 ) 0py11 2y21 2.5(y11 y21 )py12 2y22 1.5(y12 y22 )mass balancemass balancedemandsdemandssulphur balancequality req.quality req. where x, y are the input and intermediate stream quantities (as in Fig. 1) and p is the percentage ofsulphur of the streams out of the pool. Notice that this problem has three constraints involving bilinearterms, so global optimization techniques are mandatory for its solution.1.2 Example (Pooling/Blending Problems)The HPP (Example 1.1) is a special instance of a class of problems termed Pooling/Blending problems.Various types of crude oils of different qualities, coming from different feeds, are mixed together invarious pools to produce several end-products subject to quality requirements and quantity demands; theobjective is to minimize the costs. These problems are usually modelled as continuous NLPs involvingbilinear terms. In general they may have many local optima, and so the use of global optimization

1 INTRODUCTION5methods is required. The literature on Pooling/Blending problems is vast [ATS99, FHJ92, VF96, TS02,ABH 02, LP04]. We present the general blending problem formulation found in [ATS99], p. 1958. Theformulation refers to a setting with q pools each with nj input streams (j q) and r end products. Eachstream has l qualities.PrPqPq Pnj cij xij k 1 dk j 1 yjkminx,y,p j 1 i 1 PrPnj j q ij i 1 xPk 1 yjk q yjk Sk k rj 1PnjPr(2)λijw xij pjw Pk 1 xjk j q w l Pi 1 qq k r w l j 1 pjw xjk zkwj 1 yjk LULULUx x x ,p p p ,y y y ,where xij is the flow of input stream i into pool j, yjk is the total flow from pool j to product k and pjwis the w-th quality of pool j; cij , dk , Sk , Zkw , λijw are, respectively: the unit cost of the i-th stream intopool j, the unit price of product k, the demand for product k, the w-th quality requirement for productk and the w-th quality specification of the i-th stream into pool j.Euclidean location problems can also be expressed as NLPs:1.3 Example (Euclidean Location Problems)There are n plants with geographical positions expressed in Euclidean coordinates (ai , bi ) (1 i n)w.r.t. to an arbitrary point (0, 0). Daily, the i-th plant needs ri tons of raw material, which can bedelivered from m storage points each with storage capacity cj (1 j m). For security reasons, eachstorage point must be located at least at D 1Km distance from the plant. The cost of raw materialtransportation between each storage point and each plant is directly proportional to their distance aswell as the quantity of raw material. Find geographical positions where to place each storage point sothat the transportation costs are minimized. The mathematical formulation is as follows: Pn Pmmin i 1j 1 wij dij x,y,d,w P n s.t. Pi 1 wij cj j mstorage capacitymwij ri i nplant requirement j 1 p dij (xj ai )2 (yj bi )2 i m, j n Euclidean distance dij D i n, j mminimum distancewhere (xj , yj ) is the geographical position of the j-th storage point, dij is the distance between the i-thplant and the j-th storage point, wij is the quantity of raw material transported from the j-th storagepoint to the i-th plant (i n, j m).Lastly, we present an example which shows that global optimization techniques can be a research aidto questions in pure mathematics.1.4 Example (Kissing Number Problem)When rigid balls touch each other, in technical terms, they “kiss” (this is the etimology of the term “kissingnumber”). In mathematical terms, the kissing number in D dimensions is the number of D-spheres ofradius R that can be arranged around a central D-sphere of radius R so that each of the surroundingspheres touches the central one without overlapping. Determining the maximum kissing number in variousdimensions has become a well-known problem in Combinatorial Geometry. Notationally, we indicate theKissing Number Problem in D dimensions by KNP(D). In R2 the result is trivial: the maximum kissingnumber is 6 (see Fig. 2). The situation is far from trivial in R3 . The problem earned its fame because,according to Newton, the maximum kissing number in 3D is 12, whereas according to his contemporaryfellow mathematician David Gregory, the maximum kissing number in 3D is 13 (this conjecture wasstated without proof). This question was settled, at long last, more than 250 years after having beenstated, when J. Leech finally proved that the solution in 3D is 12 [Lee56]. Given parameters D (numberof dimensions) and N (number of spheres), the variables xi (xi1 , . . . , xiD ), 1 i N determine the

1 INTRODUCTION6Figure 2: The Kissing Number Problem in 2D.position of the center of the i-th sphere around the central one. We maximize a decision variable α 0which represents the minimum pairwise sphere separation distance in the N -sphere configuration beingtested, subject to the necessary geometric constraints. Since the constraints are nonconvex, there maybe multiple local minima. If the global optimization of the model determines that the global maximumis at α 1, then there is enough space for N spheres; if the globally optimal α is strictly less than 1, itmeans that the N configuration has overlapping spheres, hence the kissing number is N 1. By solvingthis decision problem repeatedly for different values of N , we are able to quickly pinpoint the maximumN for which α 1. The following formulation correctly models the problem:max i N i j N i Nα xi 2 2R xi xj 2 2Rα(3)(4)(5)α 0xi RD(6)(7)Constraints (4) ensure that the centers of the N spheres all have distance 2R from the center of the centralsphere (i.e., the N spheres kiss the central sphere). Constraints (5) make the N spheres non-overlapping.This problem has been solved correctly up to D 4 (and N 24, which is KNP(4)) with the SobolOptalgorithm (see Sect. 4) [LMK04].1.3A brief history of global optimizationGeneric optimization problems have been important throughout history in engineering applications. Thefirst significant work in optimization was carried out by Lagrange in 1797 [Lag97]. Nevertheless, beforethe introduction and extensive usage of electronic computers, the requirement for global optimization wasnot even an issue given the tremendous amount of computational effort it requires. Global optimalityof solutions was guaranteed only when locally optimizing convex functions, a rather limited class ofproblems.The methods that were first used in global optimization were deterministic techniques, mostly basedon the divide-and-conquer principle. This was introduced in the late 1950s with the advent of the firstelectronic computers into the research environment. In contrast with the computational principles employed before the introduction of computers, whereby people would try to minimize the amount of actualcomputations to perform with respect to the theoretical framework, the divide-and-conquer principle

1 INTRODUCTION7applies to the intrinsic iterative nature of methods used when working with electronic computers: keepthe complexity of the theoretical structures involved to a minimum, and rely on intensive computationto explore the solution space. Thus, instead of trying to locate a minimum by solving a set of equationsby symbolic/algebraic methods, one would try to construct a sequence of approximate solutions whichconverges to the solution by dividing the problem into smaller subproblems. One typical algorithm whichembodies the divide-and-conquer principle is the Branch-and-Bound algorithm (BB) [PS98]. Because ofthe nature of the algorithm, where the subproblems are produced by branching a problem entity (e.g.variable) into its possible instances, the BB algorithm applies very well to cases where problem entitiesare discrete in nature. Thus, the first applications of BB to global optimization problems were devotedto discrete problems such as the Travelling Salesman Problem (TSP).The first paper concerning continuous global optimization with a BB (deterministic) technique datesfrom 1969 [FS69]. In the 1970s and 1980s, work on continuous or mixed-integer deterministic globaloptimization was scarce. Most of the papers published in this period dealt either with applicationsof global optimization to very specific cases, or with theoretical results concerning convergence proofs.One notable exception was the work of McCormick [McC76] who considered symbolic transformations ofproblems. His methods are such that they can, in theory, be carried out automatically by a computer.A major reason for the slow pace of pr

The presence of multiple local minima calls for the application of global optimization techniques. This paper is a mini-course about global optimization techniques in nonconvex programming; it deals with some theoretical aspects of nonlinear programming as well as with some of the current state-of-the-art algorithms in global optimization.

Related Documents:

Since the eld { also referred to as black-box optimization, gradient-free optimization, optimization without derivatives, simulation-based optimization and zeroth-order optimization { is now far too expansive for a single survey, we focus on methods for local optimization of continuous-valued, single-objective problems.

Global Optimization, ESI 6492 Page 2 Panos Pardalos, Fall 2020 1. Fundamental Results on Convexity and Optimization 2. Quadratic Programming 3. General Concave Minimization 4. D.C. Programming 5. Lipschitz Optimization 6. Global Optimization on Networks Attendance Policy, Class Expectations, and Make-Up Policy

work/products (Beading, Candles, Carving, Food Products, Soap, Weaving, etc.) ⃝I understand that if my work contains Indigenous visual representation that it is a reflection of the Indigenous culture of my native region. ⃝To the best of my knowledge, my work/products fall within Craft Council standards and expectations with respect to

global optimization (Pint er ,1991), black-box optimization (Jones et al.,1998) or derivative-free optimization (Rios & Sahinidis,2013). There is a large number of algorithms based on various heuristics which have been introduced in order to solve this problem such as genetic algorithms, model-based methods or Bayesian optimization. We focus

Structure topology optimization design is a complex multi-standard, multi-disciplinary optimization theory, which can be divided into three category Sizing optimization, Shape optimization and material selection, Topology optimization according to the structura

An approach for the combined topology, shape and sizing optimization of profile cross-sections is the method of Graph and Heuristic Based Topology Optimization (GHT) [4], which separates the optimization problem into an outer optimization loop for the topology modification and an inner optimization loo

alculus In Motion “Related Rates” * Related Rates MORE” 4.7 Applied Optimization Pg. 262-269 #2-8E, 12, 19 WS –Optimization(LL) NC #45(SM) MMM 19 Optimization MMM 20 Economic Optimization Problems WS – Optimization(KM) Calculus In Motion “Optimization-Applications” TEST: CH

Additif très dangereux E249 : Nitrite de potassium . Conservateur chimique. Risques : essoufflements, vertiges, maux de tête, chez les nourrissons les nitrites peuvent provoquer la mort par asphyxie car ils empêchent les globules rouges de transporter l'oxygène, cancérigène. Très répandu dans les charcuteries, les salaisons, le foie gras et le bacon traité, MÊME DANS LES .