AF: Small: Approximate Augmented Lagrangians: First-Order and Parallel Optimization Methods, with Applications to Stochastic Programming

Continuous optimization is a mathematical discipline with extensive applications in engineering design and business/logistical planning. Its currently most common solution techniques are difficult to adapt to newly evolving computer architectures comprising dozens to thousands of processing elements working in parallel. Combining several existing techniques with some recent results of the principal investigator, this project explores a means of solving continuous optimization problems that should adapt more readily to parallel computer architectures than present standard solvers, allowing the architectures' full power to be brought to bear on large, time-consuming problems. Without such new solution approaches, solution of critical design and planning problems may not benefit from most of the advances in computing power anticipated for the next decade. The project will also involve cooperative work with the Brazilian research community. The technical approach is to capitalize on recent advances in augmented Lagrangian and conjugate gradient algorithms to produce a new kind of modular parallel continuous constrained optimization solver. The solver consists of a classical augmented Lagrangian outer loop, with subproblems solved by the a state-of-the art box-constrained conjugate gradient method terminated by a recently developed relative error criterion. The research consists of three stages: the goal of stage one is to create an object-oriented, modular serial implementation, test it extensively, and address some theoretical issues. Stage two aims to evolve the stage-one substrate into a parallel solver for which the user explicitly specifies how to map the problem structure to multiple processing elements. Stage three's goal is to automate the structure detection and mapping process. Stages two and three will use stochastic programming problems as test cases.