arXiv subjects

Anastasiya Ivanova

Publications and source records attributed to Anastasiya Ivanova.

8 recordsLinked to original sources

arXiv2023

Near-optimal tensor methods for minimizing the gradient norm of convex functions and accelerated primal-dual tensor methods

Motivated, in particular, by the entropy-regularized optimal transport problem, we consider convex optimization problems with linear equality constraints, where the dual objective has Lipschitz $p$-th order derivatives, and develop two approaches for solving such problems. The first approach is based on the minimization of the norm of the gradient in the dual problem and then the reconstruction of an approximate primal solution. Recently, Grapiglia and Nesterov in their work showed lower complexity bounds for the problem of minimizing the gradient norm of the function with Lipschitz $p$-th order derivatives. Still, the question of optimal or near-optimal methods remained open as the algorithms presented in the paper achieve suboptimal bounds only. We close this gap by proposing two near-optimal (up to logarithmic factors) methods with complexity bounds $\tilde{O}(\varepsilon^{-2(p+1)/(3p+1)})$ and $\tilde{O}(\varepsilon^{-2/(3p+1)})$ with respect to the initial objective residual and the distance between the starting point and solution respectively. We then apply these results (having independent interest) to our primal-dual setting. As the second approach, we propose a direct accelerated primal-dual tensor method for convex problems with linear equality constraints, where the dual objective has Lipschitz $p$-th order derivatives. For this algorithm, we prove $\tilde O (\varepsilon^{-1 / (p + 1)})$ complexity in terms of the duality gap and the residual in the constraints. We illustrate the practical performance of the proposed algorithms in experiments on logistic regression, entropy-regularized optimal transport problem, and the minimal mutual information problem.

math.OC↗

arXiv2022

Oracle Complexity Separation in Convex Optimization

Many convex optimization problems have structured objective function written as a sum of functions with different types of oracles (full gradient, coordinate derivative, stochastic gradient) and different evaluation complexity of these oracles. In the strongly convex case these functions also have different condition numbers, which eventually define the iteration complexity of first-order methods and the number of oracle calls required to achieve given accuracy. Motivated by the desire to call more expensive oracle less number of times, in this paper we consider minimization of a sum of two functions and propose a generic algorithmic framework to separate oracle complexities for each component in the sum. As a specific example, for the $μ$-strongly convex problem $\min_{x\in \mathbb{R}^n} h(x) + g(x)$ with $L_h$-smooth function $h$ and $L_g$-smooth function $g$, a special case of our algorithm requires, up to a logarithmic factor, $O(\sqrt{L_h/μ})$ first-order oracle calls for $h$ and $O(\sqrt{L_g/μ})$ first-order oracle calls for $g$. Our general framework covers also the setting of strongly convex objectives, the setting when $g$ is given by coordinate derivative oracle, and the setting when $g$ has a finite-sum structure and is available through stochastic gradient oracle. In the latter two cases we obtain respectively accelerated random coordinate descent and accelerated variance reduction methods with oracle complexity separation.

math.OC↗

arXiv2021

The recovery model for the calculation of correspondence matrix for Moscow

In this paper, we consider the problem of restoring the correspondence matrix based on the observations of real correspondences in Moscow. Following the conventional approach, the transport network is considered as a directed graph whose edges correspond to road sections and the graph vertices correspond to areas that the traffic participants leave or enter. The number of city residents is considered constant. The problem of restoring the correspondence matrix is to calculate all the correspondence from the $i$ area to the $j$ area. To restore the matrix, we propose to use one of the most popular methods of calculating the correspondence matrix in urban studies -- the entropy model. In our work, we describe the evolutionary justification of the entropy model and the main idea of the transition to solving the problem of entropy-linear programming (ELP) in calculating the correspondence matrix. To solve the ELP problem, it is proposed to pass to the dual problem. In this paper, we describe several numerical optimization methods for solving this problem: the Sinkhorn method and the Accelerated Sinkhorn method. We provide numerical experiments for the following variants of cost functions: a linear cost function and a superposition of the power and logarithmic cost functions. In these functions, the cost is a combination of average time and distance between areas, which depends on the parameters. The correspondence matrix is calculated for multiple sets of parameters and then we calculate the quality of the restored matrix relative to the known correspondence matrix. We assume that the noise in the restored correspondence matrix is Gaussian, as a result, we use the standard deviation as a quality metric.

math.OC↗

arXiv2021

Adaptive Catalyst for Smooth Convex Optimization

In this paper, we present a generic framework that allows accelerating almost arbitrary non-accelerated deterministic and randomized algorithms for smooth convex optimization problems. The main approach of our envelope is the same as in Catalyst (Lin et al., 2015): an accelerated proximal outer gradient method, which is used as an envelope for a non-accelerated inner method for the $\ell_2$ regularized auxiliary problem. Our algorithm has two key differences: 1) easily verifiable stopping criteria for inner algorithm; 2) the regularization parameter can be tunned along the way. As a result, the main contribution of our work is a new framework that applies to adaptive inner algorithms: Steepest Descent, Adaptive Coordinate Descent, Alternating Minimization. Moreover, in the non-adaptive case, our approach allows obtaining Catalyst without a logarithmic factor, which appears in the standard Catalyst (Lin et al., 2015, 2018).

math.OC↗

arXiv2021

Numerical methods for the resource allocation problem in networks

In this paper, we consider the resource allocation problem in a network with a large number of connections which are used by a huge number of users. The resource allocation problem under discussion is a maximization problem with linear inequality constraints. To solve this problem we construct the dual problem and propose to use the following numerical optimization methods for the dual: a fast gradient method, a stochastic projected subgradient method, an ellipsoid method, and a random gradient extrapolation method. A special focus is made on the primal-dual analysis of these methods. For each method we estimate the convergence rate. We also provide some modifications of these methods in the setup of distributed computations, taking into account their application to networks.

math.OC↗

arXiv2019

Adaptive Mirror Descent for the Network Utility Maximization Problem

Network utility maximization is the most important problem in network traffic management. Given the growth of modern communication networks, we consider the utility maximization problem in a network with a large number of connections (links) that are used by a huge number of users. To solve this problem an adaptive mirror descent algorithm for many constraints is proposed. The key feature of the algorithm is that it has a dimension-free convergence rate. The convergence of the proposed scheme is proved theoretically. The theoretical analysis is verified with numerical simulations. We compare the algorithm with another approach, using the ellipsoid method (EM) for the dual problem. Numerical experiments showed that the performance of the proposed algorithm against EM is significantly better in large networks and when very high solution accuracy is not required. Our approach can be used in many network design paradigms, in particular, in software-defined networks.

math.OC↗

arXiv2019

Composite optimization for the resource allocation problem

In this paper we consider resource allocation problem stated as a convex minimization problem with linear constraints. To solve this problem, we use gradient and accelerated gradient descent applied to the dual problem and prove the convergence rate both for the primal iterates and the dual iterates. We obtain faster convergence rates than the ones known in the literature. We also provide economic interpretation for these two methods. This means that iterations of the algorithms naturally correspond to the process of price and production adjustment in order to obtain the desired production volume in the economy. Overall, we show how these actions of the economic agents lead the whole system to the equilibrium.

math.OC↗

arXiv2019

Walrasian Equilibrium and Centralized Distributed Optimization from the point of view of Modern Convex Optimization Methods on the Example of Resource Allocation Problem

We consider the resource allocation problem and its numerical solution. The following constructions are demonstrated: 1) Walrasian price-adjustment mechanism for determining the equilibrium; 2) Decentralized role of the prices; 3) Slater's method for price restrictions (dual Lagrange multipliers); 4) A new mechanism for determining equilibrium prices, in which prices are fully controlled not by Center (Government), but by economic agents -- nodes (factories). In economic literature the convergence of the considered methods is only proved. In contrast, this paper provides an accurate analysis of the convergence rate of the described procedures for determining the equilibrium. The analysis is based on the primal-dual nature of the suggested algorithms. More precisely, in this article we propose the economic interpretation of the following numerical primal-dual methods of convex optimization: dichotomy and subgradient projection method. Numerical experiments conclude the paper.

math.OC↗