Searcharxiv⌕ Search

arXiv subjects

Irad Yavneh

Publications and source records attributed to Irad Yavneh.

15 recordsLinked to original sources
arXiv2026

On the recursive structure of multigrid cycles

A new fixed (non-adaptive) recursive scheme for multigrid algorithms is introduced. Governed by a positive parameter $κ$ called the cycle counter, this scheme generates a family of multigrid cycles dubbed $κ$-cycles. The well-known $V$-cycle, $F$-cycle, and $W$-cycle are shown to be particular members of this rich $κ$-cycle family, which satisfies the property that the total number of recursive calls in a single cycle is a polynomial of degree $κ$ in the number of levels of the cycle. This broadening of the scope of fixed multigrid cycles is shown to be potentially significant for the solution of some large problems on platforms, such as graphics processing units, where the overhead induced by numerous sequential calls to the coarser levels may be relatively significant. In cases of problems for which the convergence of standard $V$-cycles or $F$-cycles (corresponding to $κ=1$ and $κ=2$, respectively) is particularly slow, and yet the cost of $W$-cycles is very high due to the large number of coarse-level calls (which is exponential in the number of levels), intermediate values of $κ$ may prove to yield significantly faster run-times. This is demonstrated in examples where $κ$-cycles are used for the solution of rotated anisotropic diffusion problems, both as a stand-alone solver and as a preconditioner. Moreover, a simple model is presented for predicting the approximate run-time of the $κ$-cycle, which is useful in pre-selecting an appropriate cycle counter for a given problem on a given platform. Implementing the $κ$-cycle requires making just a small change in the classical multigrid cycle.

math.NA↗
arXiv2026

A Mini-Batch Quasi-Newton Proximal Method for Constrained Total-Variation Nonlinear Image Reconstruction

Over the years, computational imaging with accurate nonlinear physical models has garnered considerable interest due to its ability to achieve high-quality reconstructions. However, using such nonlinear models for reconstruction is computationally demanding. A popular choice for solving the corresponding inverse problems is the accelerated stochastic proximal method (ASPM), with the caveat that each iteration is still expensive. To overcome this issue, we propose a mini-batch quasi-Newton proximal method (BQNPM) tailored to image reconstruction problems with constrained total variation regularization. Compared to ASPM, BQNPM requires fewer iterations to converge. Moreover, we propose an efficient approach to compute a weighted proximal mapping at a cost similar to that of the proximal mapping in ASPM. We also analyze the convergence of BQNPM in the nonconvex setting. We assess the performance of BQNPM on three-dimensional inverse-scattering problems with linear and nonlinear physical models. Our results on simulated and real data demonstrate the effectiveness and efficiency of BQNPM, while also validating our theoretical analysis.

math.OC↗
arXiv2023

pISTA: preconditioned Iterative Soft Thresholding Algorithm for Graphical Lasso

We propose a novel quasi-Newton method for solving the sparse inverse covariance estimation problem also known as the graphical least absolute shrinkage and selection operator (GLASSO). This problem is often solved using a second-order quadratic approximation. However, in such algorithms the Hessian term is complex and computationally expensive to handle. Therefore, our method uses the inverse of the Hessian as a preconditioner to simplify and approximate the quadratic element at the cost of a more complex \(\ell_1\) element. The variables of the resulting preconditioned problem are coupled only by the \(\ell_1\) sub-derivative of each other, which can be guessed with minimal cost using the gradient itself, allowing the algorithm to be parallelized and implemented efficiently on GPU hardware accelerators. Numerical results on synthetic and real data demonstrate that our method is competitive with other state-of-the-art approaches.

math.NA↗
arXiv2021

Merging Multigrid Optimization with SESOP

A merger of two optimization frameworks is introduced: SEquential Subspace OPtimization (SESOP) with MultiGrid (MG) optimization. At each iteration of the algorithm, the search direction implied by the coarse-grid correction process of MG is added to the low dimensional search-space of SESOP, which includes the preconditioned gradient and search directions involving the previous iterates, called {\em history}. Numerical experiments demonstrate the effectiveness of this approach. We then study the asymptotic convergence factor of the two-level version of SESOP-MG (dubbed SESOP-TG) for optimization of quadratic functions, and derive approximately optimal fixed parameters, which may reduce the computational overhead for such problems significantly.

math.OC↗
arXiv2021

On Adapting Nesterov's Scheme to Accelerate Iterative Methods for Linear Problems

Nesterov's well-known scheme for accelerating gradient descent in convex optimization problems is adapted to accelerating stationary iterative solvers for linear systems. Compared with classical Krylov subspace acceleration methods, the proposed scheme requires more iterations, but it is trivial to implement and retains essentially the same computational cost as the unaccelerated method. An explicit formula for a fixed optimal parameter is derived in the case where the stationary iteration matrix has only real eigenvalues, based only on the smallest and largest eigenvalues. The fixed parameter, and corresponding convergence factor, are shown to maintain their optimality when the iteration matrix also has complex eigenvalues that are contained within an explicitly defined disk in the complex plane. A comparison to Chebyshev acceleration based on the same information of the smallest and largest real eigenvalues (dubbed Restricted Information Chebyshev acceleration) demonstrates that Nesterov's scheme is more robust in the sense that it remains optimal over a larger domain when the iteration matrix does have some complex eigenvalues. Numerical tests validate the efficiency of the proposed scheme. This work generalizes and extends the results of [1, Lemmas 3.1 and 3.2 and Theorem 3.3].

math.OC↗
arXiv2020

Learning Algebraic Multigrid Using Graph Neural Networks

Efficient numerical solvers for sparse linear systems are crucial in science and engineering. One of the fastest methods for solving large-scale sparse linear systems is algebraic multigrid (AMG). The main challenge in the construction of AMG algorithms is the selection of the prolongation operator -- a problem-dependent sparse matrix which governs the multiscale hierarchy of the solver and is critical to its efficiency. Over many years, numerous methods have been developed for this task, and yet there is no known single right answer except in very special cases. Here we propose a framework for learning AMG prolongation operators for linear systems with sparse symmetric positive (semi-) definite matrices. We train a single graph neural network to learn a mapping from an entire class of such matrices to prolongation operators, using an efficient unsupervised loss function. Experiments on a broad class of problems demonstrate improved convergence rates compared to classical AMG, demonstrating the potential utility of neural networks for developing sparse system solvers.

cs.LG↗
arXiv2020

Adjustable Coins

In this paper we consider a scenario where there are several algorithms for solving a given problem. Each algorithm is associated with a probability of success and a cost, and there is also a penalty for failing to solve the problem. The user may run one algorithm at a time for the specified cost, or give up and pay the penalty. The probability of success may be implied by randomization in the algorithm, or by assuming a probability distribution on the input space, which lead to different variants of the problem. The goal is to minimize the expected cost of the process under the assumption that the algorithms are independent. We study several variants of this problem, and present possible solution strategies and a hardness result.

cs.DS↗
arXiv2020

Solving RED with Weighted Proximal Methods

REgularization by Denoising (RED) is an attractive framework for solving inverse problems by incorporating state-of-the-art denoising algorithms as the priors. A drawback of this approach is the high computational complexity of denoisers, which dominate the computation time. In this paper, we apply a general framework called weighted proximal methods (WPMs) to solve RED efficiently. We first show that two recently introduced RED solvers (using the fixed point and accelerated proximal gradient methods) are particular cases of WPMs. Then we show by numerical experiments that slightly more sophisticated variants of WPM can lead to reduced run times for RED by requiring a significantly smaller number of calls to the denoiser.

eess.IV↗
arXiv2019

Learning to Optimize Multigrid PDE Solvers

Constructing fast numerical solvers for partial differential equations (PDEs) is crucial for many scientific disciplines. A leading technique for solving large-scale PDEs is using multigrid methods. At the core of a multigrid solver is the prolongation matrix, which relates between different scales of the problem. This matrix is strongly problem-dependent, and its optimal construction is critical to the efficiency of the solver. In practice, however, devising multigrid algorithms for new problems often poses formidable challenges. In this paper we propose a framework for learning multigrid solvers. Our method learns a (single) mapping from a family of parameterized PDEs to prolongation operators. We train a neural network once for the entire class of PDEs, using an efficient and unsupervised loss function. Experiments on a broad class of 2D diffusion problems demonstrate improved convergence rates compared to the widely used Black-Box multigrid scheme, suggesting that our method successfully learned rules for constructing prolongation matrices.

math.NA↗
arXiv2016

A multilevel framework for sparse optimization with application to inverse covariance estimation and logistic regression

Solving l1 regularized optimization problems is common in the fields of computational biology, signal processing and machine learning. Such l1 regularization is utilized to find sparse minimizers of convex functions. A well-known example is the LASSO problem, where the l1 norm regularizes a quadratic function. A multilevel framework is presented for solving such l1 regularized sparse optimization problems efficiently. We take advantage of the expected sparseness of the solution, and create a hierarchy of problems of similar type, which is traversed in order to accelerate the optimization process. This framework is applied for solving two problems: (1) the sparse inverse covariance estimation problem, and (2) l1-regularized logistic regression. In the first problem, the inverse of an unknown covariance matrix of a multivariate normal distribution is estimated, under the assumption that it is sparse. To this end, an l1 regularized log-determinant optimization problem needs to be solved. This task is challenging especially for large-scale datasets, due to time and memory limitations. In the second problem, the l1-regularization is added to the logistic regression classification objective to reduce overfitting to the data and obtain a sparse model. Numerical experiments demonstrate the efficiency of the multilevel framework in accelerating existing iterative solvers for both of these problems.

math.NA↗
arXiv2012

Phase retrieval combined with digital holography

We present a new method for real- and complex-valued image reconstruction from two intensity measurements made in the Fourier plane: the Fourier magnitude of the unknown image, and the intensity of the interference pattern arising from superimposition of the original signal with a reference beam. This approach can provide significant advantages in digital holography since it poses less stringent requirements on the reference beam. In particular, it does not require spatial separation between the sought signal and the reference beam. Moreover, the reference beam need not be known precisely, and in fact, may contain severe errors, without leading to a deterioration in the reconstruction quality. Numerical simulations are presented to demonstrate the speed and quality of reconstruction.

physics.optics↗
arXiv2012

Designing and using prior knowledge for phase retrieval

In this work we develop an algorithm for signal reconstruction from the magnitude of its Fourier transform in a situation where some (non-zero) parts of the sought signal are known. Although our method does not assume that the known part comprises the boundary of the sought signal, this is often the case in microscopy: a specimen is placed inside a known mask, which can be thought of as a known light source that surrounds the unknown signal. Therefore, in the past, several algorithms were suggested that solve the phase retrieval problem assuming known boundary values. Unlike our method, these methods do rely on the fact that the known part is on the boundary. Besides the reconstruction method we give an explanation of the phenomena observed in previous work: the reconstruction is much faster when there is more energy concentrated in the known part. Quite surprisingly, this can be explained using our previous results on phase retrieval with approximately known Fourier phase.

physics.optics↗
arXiv2010

On MMSE and MAP Denoising Under Sparse Representation Modeling Over a Unitary Dictionary

Among the many ways to model signals, a recent approach that draws considerable attention is sparse representation modeling. In this model, the signal is assumed to be generated as a random linear combination of a few atoms from a pre-specified dictionary. In this work we analyze two Bayesian denoising algorithms -- the Maximum-Aposteriori Probability (MAP) and the Minimum-Mean-Squared-Error (MMSE) estimators, under the assumption that the dictionary is unitary. It is well known that both these estimators lead to a scalar shrinkage on the transformed coefficients, albeit with a different response curve. In this work we start by deriving closed-form expressions for these shrinkage curves and then analyze their performance. Upper bounds on the MAP and the MMSE estimation errors are derived. We tie these to the error obtained by a so-called oracle estimator, where the support is given, establishing a worst-case gain-factor between the MAP/MMSE estimation errors and the oracle's performance. These denoising algorithms are demonstrated on synthetic signals and on true data (images).

cs.CV↗
arXiv2004

Propagation of interacting force chains in the continuum limit

We study the effect of mergers in the force chain model describing the stress profile in static granular materials. Combining numerical and analytical calculations we show that granular materials do not generally behave in an elastic-like manner, however they may under specific conditions, which are elaborated. Non-elastic behavior resulting from the non-linearity of the full force chain model is discussed.

cond-mat.soft↗