Recent Trends in Optimization Theory and Applications

The following sections are included:

• Contributors

• Preface

• CONTENTS

This paper discusses some recent developments in the static optimization of queueing systems. Special attention is given to three problem classes: (i) the optimal allocation of servers, or service capacity, to queues in a network; (ii) the optimal allocation of the visits of a single server to several queues (a polling system); (iii) the optimal allocation of a single arrival stream to several single server queues.

Variational inequalities, nonlinear programming, complementarity problems and other problems can be reduced to nonsmooth equations, for which some generalizations of Newton's method are known. The Newton path, as a natural generalization of the Newton direction, was suggested by D. Ralph for enlarging the convergence region (globalization) of Newton-Robinson's method in the nonsmooth case. We investigate some properties of both the Newton direction and the Newton path, which seem to be basic for various globalization strategies. In particular, a simple formula for the derivative of an arbitrary norm of residuals along the Newton direction, derived earlier by the author for the smooth equations, is generalized here for the derivative along the Newton path.

Optimization problems under invex hypotheses preserve many properties of convex problems. However a vector function may be invex at a point without being convexifiable by a transformation. It is shown that a generalization of Weir and Mond's preinvex property, not requiring derivatives, characterizes invex at a point, and a further generalization characterizes the convexifiable property. The results are applied to Lagrangian duality.

Kuhn–Tucker conditions for mathematical programming problems with fuzzy objective functions or fuzzy constraints require the characterization of dual cones of spaces of convex homogeneous functions. This is trivial when the fuzzy sets are all defined on the real line. Here the characterization is established for fuzzy sets defined on a two–dimensional base space.

A surjective space transformation technique is used to convert an original dual linear programming problem with equality and inequality constraints into a problem involving only equality constraints. Continuous and discrete versions of the stable gradient projection method are applied to the reduced problem. The numerical methods involve performing inverse transformations. The convergence rate analysis for dual linear programming methods is presented. By choosing a particular exponential space-transformation function we obtain the dual affine scaling algorithm. Variants of methods which have linear local convergence are given.

In this paper we analyze a common procedure used for the identification of parameters in distributed systems in order to obtain an efficient solver of the parameter identification problem on a parallel-vector computer, such as the Cray Y-MP. The results of some numerical experiments are presented.

A constrained extremum problem is embedded in a generalized system and is associated with the space where the images of its functions run. In such a space, called image space, the image of the extremum problem is denned. This approach is shown to be effective for other kinds of problems, such as Variational Inequalites. In the last two decades there has been an increasing interest in analyzing the properties of the image of an extremum problem. Here we report on some results, concerned with optimality conditions, duality, penalty methods, which have been obtained through this approach. This is extended to other fields, including Variational Inequalities. Further research in the field is discussed.

The convex semi-infinite quadratic program dealt with in this paper originates from a least-squares parameter estimation problem for systems in linear regression form subject to distributed constraints. A batch algorithm and a recursive algorithm for the considered problem are presented, and their convergence proved. The former is a proper extension of Kelley's convex cutting plane algorithm, the latter a suitably adjusted projection algorithm.

The actual accelerations of constrained mechanical systems are characterized implicitly by Appell's equations. However, the generalized inverse form provides an explicit characterization. It is shown that Appell's equations and the generalized inverse form minimize the same function. Both methods are then applied to a non-holonomic system, and provide exactly the same actual accelerations.

World-wide competitive trends require large-scale industrial processes to be fully optimised across all levels of the process hierarchy. This paper reports a methodological framework for the problems to be solved. Industrial examples are used to motivate the use of generic performance monitoring indices for different types of industrial process control problems. Proposals to extend the statistical process control paradigm to incorporate new tools are also discussed. The potential industrial benefits for adopting a holistic plant optimization strategy close the paper.

In this note, the tool of the calculus of variations is used to determine optimal schedules in preparation for a major earthquake. The simple model presented here is one of the many applications of the dynamic optimization techniques to planning and management problems.

The Wiener model of a nonlinear process is an optimal input/output representation of the system when the input is a Gaussian process. If the control input is not Gaussian, the Wiener model parameters are difficult to calculate. To circumvent this problem a set of gate functions, whose orthognality does not depend on the statistical properties of the input, is used. This paper is concerned with the application of the gate function approach to the identification and control of nonlinear systems. The model parameters are estimated by minimising the mean-square error between the measurement and the model output. An interpolation scheme is employed to update the model output and a predictive type controller is used to design a nonlinear compensator. The novel control design approach is applicable to general nonlinear systems and no a priori information is required. Simulation results suggest that the gate function approach is superior to the Volterra series based methods, neural network and fuzzy logic techniques for some applications. The proposed method is applied to pH control problems.

A technique is presented which enables the optimal control designer to compensate for the approximations usually made to obtain an analytical optimal control. Instead of discarding the approximation terms, they are treated as bounded controls (called error compensation controls), and the bounds are replaced by penalty terms in the performance index. The resulting optimal control, expected to be analytical, contains the penalty weights as parameters. The error compensation control is then used in the exact optimal control problem, and optimal weights are computed by nonlinear programming. What results is an improved approximate analytical optimal control. An optimal guidance example shows that error compensation improves the performance of the approximate optimal control.

Two System Identification paradigms using Fuzzy Logic modeling are explored. The first entails the representation of an unknown input/output mapping by a fuzzy inference engine, using the input/output data. The second addresses the class of problems where the prior information about the system is imbedded in the structure of a fuzzy logic model and where one is concerned with the identification of the underlying fuzzy rules from input/output data. Examples, using polynomial models and a logical XOR device, respectively, illustrate the two fuzzy logic modeling/identification paradigms.

The following sections are included:

• Introduction

• Problem Formulation

• Stabilizing Control

• Simulation Results

• Conclusion

• APPENDIX

• References

There have been a lot of efforts made in deriving various second-order sufficient conditions for local minima, strict local minima and isolated local minima in nonlinear programming. In the paper we aim at trying to synthesize ideas scattering in the literature concerning second-order sufficient conditions in smooth nonlinear programs, and in a sense to exhaust several approaches from a systematic investigation. Sharp second-order sufficient conditions for local minima, strict local minima, and isolated local minima are derived. The results obtained here recover many known second-order sufficient conditions in the literature.

In this paper, we study the iterate convergence of a class of primal-dual interior point algorithms for solving convex quadratic programs. We establish the linear convergence of the iterate sequence without any nondegeneracy assumptions (such as the strict complementarity or the Jacobian being nonsingular). Our proof is based on an error bound for polynomial systems. Our work extends the iterate convergence results for linear programming by Tapia, Zhang and Ye.

The technique of direct relaxation developed in^1,2 is applied to problems of optimal layout for plates. Optimal microstructures are explicitly indicated for the case when the original and conjugate strain tensors are coaxial. The paper develops and extends results obtained in².

In this paper the problem of maximizing a quadratic function defined in {−1, 1}ⁿ is considered. We propose a technique to obtain an upper bound and a lower bound to the maximum of a quadratic function on the set {−1, 1}ⁿ and a feasible point where the lower bound is attained. The problem of the approximability of the quadratic maximization problem with integer constraints by the method proposed here is studied and solved negatively. Moreover a special class of matrices such that the feasible point obtained with our method is the solution of the maximization problem considered is given. Numerical implementation of the method proposed and related numerical experience are shown.

Spectral matrices that have unit circle transmission zeros arise in the consideration of H_∞, control, the bounded-real lemma and discrete spectral factorization problems. Spectral matrices with unit circle invariant (but not transmission) zeros arise when considering Kalman filtering for systems with unit circle modes which are not corrupted by process noise. It is well known that if a spectral matrix is generically nonsingular, minimum phase spectral factors can be constructed from a strong solution of an algebraic Riccati equation associated with a state-space realization of the spectral matrix. Closely associated with the algebraic equation is a Riccati difference equation whose iterates are shown to converge to the strong solution under fairly mild conditions on the realization of the spectral matrix. The key observations made in this paper concern the fine structure of the Riccati difference equation iterates from which a convergence rate of is deduced.

This paper presents a simple algorithm for solving the optimal control problems of discrete-time systems with constraints on control, but without constraints on the trajectory or the terminal state. In this algorithm, monotonous reduction of cost values at each iteration is guaranteed. It is proved that the converged control satisfies the Pontryagin type necessary conditions for optimality under some conditions.

The objective of this paper is to derive sufficient conditions for optimally of decentralized control using the classical method of Lagrange. By formulating the decentralized information structure constraints on the input variables as differential equations we can add the constraints to the equations of motion and minimize the cost functional. To obtain the globally optimal solution we introduce suitable Lagrange–Lyapunov functions of time and state as multipliers and solve the constrained optimization problem. By assuming a Gaussian nature of the state evolution we provide a feedback control structure determined by Riccati-type equations.

We describe an approach to estimate parameters in explicit model functions, dynamic systems of equations, Laplace transformations, systems of ordinary differential equations, differential algebraic equations and systems of partial differential equations. Proceeding from given experimental data, i.e. observation times and measurements, the minimum least squares distance of measured data from a fitting criterion is computed, that depends on the solution of the dynamic system. The practical impact of parameter identification is illustrated by a couple of real life examples. Also some details about implementation and numerical algorithms used, are presented.

Traditionally, given a system, people try to find an optimal point (solution). This solution only can be used to handle a certain decision situation. In this paper, we introduce the framework of designing optimal systems that can cope with various decision situations under uncertainty to the Optimization Community. We also discuss a number of effective methods for finding the optimal systems as well as their contingency plans. We sincerely invite all interested readers to jointly work on this growing significant research field.

In many hierarchal systems, the lead decision maker does not control all system inputs and thus cannot directly implement a team optimal solution. In this situation the leader must develop incentives to influence the lower level decision makers (followers) into choosing strategies that lead to the leader's desired solution. This paper shows that, for a certain class of systems, affine sliding manifolds may be used as incentives to influence decision makers, level by level, into adopting the team strategy.

The following sections are included:

• Introduction

• The Optimization Systems

• The Statistical Mechanics of Optimization Systems

• The Constraint Selection Function

• Binary Problems

• The Elastic Net Algorithm

• References

We give a simplified analysis of an infeasible predictor-corrector path-following method for solving monotone linear complementarity problems. This method, like those studied by Mizuno et al. and by Potra and Sheng, (i) requires two factorizations and two backsolves per iteration, (ii) can find a solution in or O(nL) iterations, depending on the quality of the starting point, and (iii) has local quadratic convergence, provided a strictly complementary solution exists. The method decreases the centering parameter and the in-feasibility at both predictor step and corrector step, and it is flexible in that either a primal-scaling or dual-scaling or primal-dual scaling can be used for the corrector step without affecting the global and local convergence properties of the method.

Computationally simple multivariable adaptive controllers with improved control quality during the adaptation are synthesised using bicriterial optimization and Lyapunov functions. The suggested design method exploits the dual effect of adaptive control and an uncertainty measure to improve the performance and decrease the adaptation time. Two performance indices are introduced for control optimization which correspond to two goals: to control the plant and to provide an optimal excitation for speeding up the parameter estimation process. Lyapunov functions are used for designing a stable closed-loop control scheme. The interconnections between the suggested approach and linear quadratic control are discussed. A simulated example of a multivariable system demonstrates the potential and superiority of the designed controller over a usually applied controller based on the certainty equivalence assumption.

Numerical methods for a class of optimization problem with functional inequality constraints are proposed by approximating these with conventional constrained or unconstrained optimization problems. Each of the approximate problems has two interdependent parameters. The sub-optimal solution can be made arbitrarily close to the optimal solution by choosing the parameters of the approximate problem appropriately.

For a narrower class of problems where the cost is quadratic and the functional constraints are convex, a closed form expression for the approximation error can be derived. Globally convergent algorithms are developed for the case where the constraints functions are affine.

Control system design problems are generally multiobjective, in that they require several, generally conflicting, requirements to be simultaneously met. One procedure for multiobjective computer-aided control system design is the method of inequalities (MOI). Here, the MOI is combined with an H^∞-optimization loop-shaping design procedure in a mixed-optimization approach which designs for both robustness and explicit closed-loop performance. This mixed-optimization approach has been incorporated into a MATLAB-based interactive multiobjective computer-aided control system design environment.