Optimization problems of large scale appear as important mathematical models, starting with Big Data analytics and Neural Network training to models of complex natural and scientific phenomena. The principal challenges are large dimensions, huge data sets and presence of noise which implies intrinsic inexactness of objective function and constraints. The need to extract knowledge from huge data sets or complex models generates a number of mathematical challenges. In this project we will consider three main frameworks which are mutually interlaced - distributed optimization, stochastic optimization and multiobjective optimization.

Challenges

For all three types of problems we will built on the state-of-the-art algorithms and nontrivial adaptations of concepts from classical optimization and hence expand the theoretical knowledge and improve numerical performance in the sense of computational costs. The main research tasks will be the following: globalization strategies with adaptive step sizes, provable convergence and ability to retain fast local convergence for distributed and stochastic optimization problems, including the heavy-tailed noise problems, as well as theoretical and numerical properties of methods based on probabilistic models for all three classes of problems. The key challenges are efficient computation of the search direction in the presence of noise or in the distributed environment and computation of step size that ensures global convergence but retains fast local convergence. We will investigate adaptive ways of balance between accuracy and costs with the goal of defining methods that are both theoretically and numerically improving the state-of-the-art algorithms.

Results and impact

Obtained results will be further specialized for problems with special structure like convex clustering and other machine learning models. LASCADO will have academic impact but given the applicability of the expected results it will also have impact on industry and society in general.