Searcharxiv⌕ Search

arXiv subjects

Arkadi Nemirovski

Publications and source records attributed to Arkadi Nemirovski.

At least 19 recordsLinked to original sources
arXiv2026

Recovering linear images of sparse signals from indirect observations

In this paper, we develop and analyze techniques for recovering a linear image $Bx$ of an unknown signal $x$ from indirect noisy observation $\omega=Ax+\xi$. It is {\em a priori} known that $x\in \cX$, a given convex compact set, and that $x$ is $s$-sparse---has at most $s$ nonvanishing entries. The proposed estimates belong to a large family of recovery routines by $\ell_1$-minimization. However, unlike the classical result describing performance of such estimates, we do not make any special (and hard to check) assumptions about the sensing matrix $A$ such as nullspace or Restricted Isometry condition and the like. As a consequence, parameters of the estimates and the upper bounds on their risks are not available in a closed analytic form, but are delivered instead by efficient computation as solutions to explicit convex optimization problems.

stat.ML↗
arXiv2025

On robust recovery of signals from indirect observations

We consider an uncertain linear inverse problem as follows. Given observation $\omega=Ax_*+\zeta$ where $A\in {\bf R}^{m\times p}$ and $\zeta\in {\bf R}^{m}$ is observation noise, we want to recover unknown signal $x_*$, known to belong to a convex set ${\cal X}\subset{\bf R}^{n}$. As opposed to the "standard" setting of such problem, we suppose that the model noise $\zeta$ is "corrupted" -- contains an uncertain (deterministic dense or singular) component. Specifically, we assume that $\zeta$ decomposes into $\zeta=N\nu_*+\xi$ where $\xi$ is the random noise and $N\nu_*$ is the "adversarial contamination" with known $\cal N\subset {\bf R}^n$ such that $\nu_*\in \cal N$ and $N\in {\bf R}^{m\times n}$. We consider two "uncertainty setups" in which $\cal N$ is either a convex bounded set or is the set of sparse vectors (with at most $s$ nonvanishing entries). We analyse the performance of "uncertainty-immunized" polyhedral estimates -- a particular class of nonlinear estimates as introduced in [15, 16] -- and show how "presumably good" estimates of the sort may be constructed in the situation where the signal set is an ellitope (essentially, a symmetric convex set delimited by quadratic surfaces) by means of efficient convex optimization routines.

math.ST↗
arXiv2023

First order algorithms for computing linear and polyhedral estimates

It was recently shown [7, 9] that "properly built" linear and polyhedral estimates nearly attain minimax accuracy bounds in the problem of recovery of unknown signal from noisy observations of linear images of the signal when the signal set is an ellitope. However, design of nearly optimal estimates relies upon solving semidefinite optimization problems with matrix variables, what puts the synthesis of such estimates beyond the rich of the standard Interior Point algorithms of semidefinite optimization even for moderate size recovery problems. Our goal is to develop First Order Optimization algorithms for the computationally efficient design of linear and polyhedral estimates. In this paper we (a) explain how to eliminate matrix variables, thus reducing dramatically the design dimension when passing from Interior Point to First Order optimization algorithms and (2) develop and analyse a dedicated algorithm of the latter type -- Composite Truncated Level method.

math.OC↗
arXiv2023

On Robust Recovery of Signals from Indirect Observations

Our focus is on robust recovery algorithms in statistical linear inverse problem. We consider two recovery routines - the much studied linear estimate originating from Kuks and Olman [42] and polyhedral estimate introduced in [37]. It was shown in [38] that risk of these estimates can be tightly upper-bounded for a wide range of a priori information about the model through solving a convex optimization problem, leading to a computationally efficient implementation of nearly optimal estimates of these types. The subject of the present paper is design and analysis of linear and polyhedral estimates which are robust with respect to the uncertainty in the observation matrix. We evaluate performance of robust estimates under stochastic and deterministic matrix uncertainty and show how the estimation risk can be bounded by the optimal value of efficiently solvable convex optimization problem; "presumably good" estimates of both types are then obtained through optimization of the risk bounds with respect to estimate parameters.

math.ST↗
arXiv2023

Generalized generalized linear models: Convex estimation and online bounds

We introduce a new computational framework for estimating parameters in generalized generalized linear models (GGLM), a class of models that extends the popular generalized linear models (GLM) to account for dependencies among observations in spatio-temporal data. The proposed approach uses a monotone operator-based variational inequality method to overcome non-convexity in parameter estimation and provide guarantees for parameter recovery. The results can be applied to GLM and GGLM, focusing on spatio-temporal models. We also present online instance-based bounds using martingale concentrations inequalities. Finally, we demonstrate the performance of the algorithm using numerical simulations and a real data example for wildfire incidents.

stat.ME↗
arXiv2023

Radiation design in computed tomography via convex optimization

Proper X-ray radiation design (via dynamic fluence field modulation, FFM) allows to reduce effective radiation dose in computed tomography without compromising image quality. It takes into account patient anatomy, radiation sensitivity of different organs and tissues, and location of regions of interest. We account all these factors within a general convex optimization framework.

physics.med-ph↗
arXiv2022

On Design of Polyhedral Estimates in Linear Inverse Problems

Polyhedral estimate is a generic efficiently computable nonlinear in observations routine for recovering unknown signal belonging to a given convex compact set from noisy observation of signal's linear image. Risk analysis and optimal design of polyhedral estimates may be addressed through efficient bounding of optimal values of optimization problems. Such problems are typically hard; yet, it was shown in Juditsky, Nemirovski 2019 that nearly minimax optimal ("up to logarithmic factors") estimates can be efficiently constructed when the signal set is an ellitope - a member of a wide family of convex and compact sets of special geometry (see, e.g., Juditsky, Nemirovski 2018). The subject of this paper is a new risk analysis for polyhedral estimate in the situation where the signal set is an intersection of an ellitope and an arbitrary polytope allowing for improved polyhedral estimate design in this situation.

math.ST↗
arXiv2022

Aggregating regular norms

The subject of this paper is regularity-preserving aggregation of regular norms on finite-dimensional linear spaces. Regular norms were introduced in [5] and are closely related to ``type 2'' spaces [9, Chapter 9] playing important role in 1) high-dimensional convex geometry and probability in Banach spaces [0.9.12.13.15], and in 2) design of proximal first-order algorithms for large-scale convex optimization with dimension-independent, or nearly so, complexity. Regularity, with moderate parameters, of a norm makes applicable, in a dimension-independent fashion, numerous geometric, probabilistic, and optimization-related results, which motivates our interest in aggregating regular norms with controlled (and moderate) inflation of regularity parameters.

math.OC↗
arXiv2021

Tight Computationally Efficient Approximation of Matrix Norms with Applications

We address the problems of computing operator norms of matrices induced by given norms on the argument and the image space. It is known that aside of a fistful of "solvable cases," most notably, the case when both given norms are Euclidean, computing operator norm of a matrix is NP-hard. We specify rather general families of norms on the argument and the images space ("ellitopic" and "co-ellitopic," respectively) allowing for reasonably tight computationally efficient upper-bounding of the associated operator norms. We extend these results to bounding "robust operator norm of uncertain matrix with box uncertainty," that is, the maximum of operator norms of matrices representable as a linear combination, with coefficients of magnitude $\leq1$, of a collection of given matrices. Finally, we consider some applications of norm bounding, in particular, (1) computationally efficient synthesis of affine non-anticipative finite-horizon control of discrete time linear dynamical systems under bounds on the peak-to-peak gains, (2) signal recovery with uncertainties in sensing matrix, and (3) identification of parameters of time invariant discrete time linear dynamical systems via noisy observations of states and inputs on a given time horizon, in the case of "uncertain-but-bounded" noise varying in a box.

math.OC↗
arXiv2021

Aggregating estimates by convex optimization

We discuss the approach to estimate aggregation and adaptive estimation based upon (nearly optimal) testing of convex hypotheses. We show that in the situation where the observations stem from {\em simple observation schemes} and where set of unknown signals is a finite union of convex and compact sets, the proposed approach leads to aggregation and adaptation routines with nearly optimal performance. As an illustration, we consider application of the proposed estimates to the problem of recovery of unknown signal known to belong to a union of ellitopes in Gaussian observation scheme. The proposed approach can be implemented efficiently when the number of sets in the union is "not very large." We conclude the paper with a small simulation study illustrating practical performance of the proposed procedures in the problem of signal estimation in the single-index model.

math.ST↗
arXiv2021

On Well-Structured Convex-Concave Saddle Point Problems and Variational Inequalities with Monotone Operators

For those acquainted with CVX (aka disciplined convex programming) of M. Grant and S. Boyd, the motivation of this work is the desire to extend the scope of CVX beyond convex minimization -- to convex-concave saddle point problems and variational inequalities with monotone operators. To attain this goal, given a family K of cones (e.g., Lorentz, semidefinite{, geometric, etc.), we introduce the notions of K-conic representation of a convex-concave saddle point problem and of variational inequality with monotone operator. We demonstrate that given such a representation of the problem of interest, the latter can be reduced straightforwardly to a conic problem on a cone from K and thus can be solved by (any) solver capable to handle conic problems on cones from K (e.g., Mosek or SDPT3 in the case of semidefinite cones). We also show that K-representations of convex-concave functions and monotone vector fields admit a fully algorithmic calculus which helps to recognize the cases when a saddle point problem or variational inequality can be converted into a conic problem on a cone from K and to carry out such conversion.

math.OC↗
arXiv2020

Convex optimization for finite horizon robust covariance control of linear stochastic systems

This work addresses the finite-horizon robust covariance control problem for discrete-time, partially observable, linear system affected by random zero mean noise and deterministic but unknown disturbances restricted to lie in what is called ellitopic uncertainty set (e.g., finite intersection of centered at the origin ellipsoids/elliptic cylinders). Performance specifications are imposed on the random state-control trajectory via averaged convex quadratic inequalities, linear inequalities on the mean, as well as pre-specified upper bounds on the covariance matrix. For this problem we develop a computationally tractable procedure for designing affine control policies, in the sense that the parameters of the policy that guarantees the aforementioned performance specifications are obtained as solutions to an explicit convex program. Our theoretical findings are illustrated by a numerical example.

math.OC↗
arXiv2020

Constant Depth Decision Rules for multistage optimization under uncertainty

In this paper, we introduce a new class of decision rules, referred to as Constant Depth Decision Rules (CDDRs), for multistage optimization under linear constraints with uncertainty-affected right-hand sides. We consider two uncertainty classes: discrete uncertainties which can take at each stage at most a fixed number d of different values, and polytopic uncertainties which, at each stage, are elements of a convex hull of at most d points. Given the depth mu of the decision rule, the decision at stage t is expressed as the sum of t functions of mu consecutive values of the underlying uncertain parameters. These functions are arbitrary in the case of discrete uncertainties and are poly-affine in the case of polytopic uncertainties. For these uncertainty classes, we show that when the uncertain right-hand sides of the constraints of the multistage problem are of the same additive structure as the decision rules, these constraints can be reformulated as a system of linear inequality constraints where the numbers of variables and constraints is O(1)(n+m)d^mu N^2 with n the maximal dimension of control variables, m the maximal number of inequality constraints at each stage, and N the number of stages. As an illustration, we discuss an application of the proposed approach to a Multistage Stochastic Program arising in the problem of hydro-thermal production planning with interstage dependent inflows. For problems with a small number of stages, we present the results of a numerical study in which optimal CDDRs show similar performance, in terms of optimization objective, to that of Stochastic Dual Dynamic Programming (SDDP) policies, often at much smaller computational cost.

math.OC↗
arXiv2020

Convex Parameter Recovery for Interacting Marked Processes

We introduce a new general modeling approach for multivariate discrete event data with categorical interacting marks, which we refer to as marked Bernoulli processes. In the proposed model, the probability of an event of a specific category to occur in a location may be influenced by past events at this and other locations. We do not restrict interactions to be positive or decaying over time as it is commonly adopted, allowing us to capture an arbitrary shape of influence from historical events, locations, and events of different categories. In our modeling, prior knowledge is incorporated by allowing general convex constraints on model parameters. We develop two parameter estimation procedures utilizing the constrained Least Squares (LS) and Maximum Likelihood (ML) estimation, which are solved using variational inequalities with monotone operators. We discuss different applications of our approach and illustrate the performance of proposed recovery routines on synthetic examples and a real-world police dataset.

math.ST↗
arXiv2019

Signal recovery by Stochastic Optimization

We discuss an approach to signal recovery in Generalized Linear Models (GLM) in which the signal estimation problem is reduced to the problem of solving a stochastic monotone variational inequality (VI). The solution to the stochastic VI can be found in a computationally efficient way, and in the case when the VI is strongly monotone we derive finite-time upper bounds on the expected $\|\cdot\|_2^2$ error converging to 0 at the rate $O(1/K)$ as the number $K$ of observations grows. Our structural assumptions are essentially weaker than those necessary to ensure convexity of the optimization problem resulting from Maximum Likelihood estimation. In hindsight, the approach we promote can be traced back directly to the ideas behind the Rosenblatt's perceptron algorithm.

math.ST↗
arXiv2018

Adaptive Denoising of Signals with Local Shift-Invariant Structure

We discuss the problem of adaptive discrete-time signal denoising in the situation where the signal to be recovered admits a "linear oracle" -- an unknown linear estimate that takes the form of convolution of observations with a time-invariant filter. It was shown by Juditsky and Nemirovski (2009) that when the $\ell_2$-norm of the oracle filter is small enough, such oracle can be "mimicked" by an efficiently computable adaptive estimate of the same structure with an observation-driven filter. The filter in question was obtained as a solution to the optimization problem in which the $\ell_\infty$-norm of the Discrete Fourier Transform (DFT) of the estimation residual is minimized under constraint on the $\ell_1$-norm of the filter DFT. In this paper, we discuss a new family of adaptive estimates which rely upon minimizing the $\ell_2$-norm of the estimation residual. We show that such estimators possess better statistical properties than those based on $\ell_\infty$-fit; in particular, we prove oracle inequalities for their $\ell_2$-loss and improved bounds for $\ell_2$- and pointwise losses. The oracle inequalities rely on the "approximate shift-invariance" assumption stating that the signal to be recovered is close to an (unknown) shift-invariant subspace. We also study the relationship of the approximate shift-invariance assumption with the "signal simplicity" assumption introduced in Juditsky and Nemirovski (2009) and discuss the application of the proposed approach to harmonic oscillations denoising.

math.ST↗
arXiv2018

Near-Optimal Recovery of Linear and N-Convex Functions on Unions of Convex Sets

In this paper we build provably near-optimal, in the minimax sense, estimates of linear forms and, more generally, "$N$-convex functionals" (the simplest example being the maximum of several fractional-linear functions) of unknown "signal" known to belong to the union of finitely many convex compact sets from indirect noisy observations of the signal. Our main assumption is that the observation scheme in question is good in the sense of A. Goldenshluger, A. Juditsky, A. Nemirovski, Electr. J. Stat. 9(2) (2015), arXiv:1311.6765, the simplest example being the Gaussian scheme where the observation is the sum of linear image of the signal and the standard Gaussian noise. The proposed estimates, same as upper bounds on their worst-case risks, stem from solutions to explicit convex optimization problems, making the estimates "computation-friendly."

math.ST↗