arXiv subjects

Deren Han

Publications and source records attributed to Deren Han.

At least 19 recordsLinked to original sources

arXiv2026

Randomized iterative methods with Polyak step-size for solving generalized absolute value equations

In this paper, we systematically incorporate the Polyak step-size into the randomized iterative method to improve its efficiency for solving generalized absolute value equations. In particular, we adopt the Polyak step-size within a stochastic iterative setting where the objective function updates dynamically at every step, unlike the classical Polyak step-size designed for deterministic optimization with fixed objective functions. Consequently, this novel implementation differs from the conventional Polyak scheme and demands a dedicated convergence analysis. We rigorously analyze the convergence properties of the proposed method and establish its linear convergence in expectation. Numerical experiments demonstrate that the incorporation of the Polyak step-size substantially improves the computational performance of randomized iterative methods with constant step-sizes.

math.NA↗

arXiv2026

On subspace-constrained preconditioning for randomized iterative methods

In this paper, we further investigate and refine the subspace-constrained preconditioning technique to enhance the theoretical and numerical convergence properties of randomized iterative methods for solving linear systems. In particular, we design a QR-like factorization that transforms the original linear system into an equivalent block-orthogonal form, thus avoiding the full-rank assumptions adopted in existing work. Moreover, this reformulation reduces the problem to solving a smaller linear system with a favorable singular value distribution, provided an appropriate initial point is employed. The proposed framework can be implemented implicitly within the iteration and does not require explicitly constructing either a preconditioner matrix or a preconditioned linear system, which eliminates the prohibitive cost of forming a fully preconditioned system. Furthermore, we construct orthogonalized search directions from stochastic gradients and develop accelerated variants of the framework. We prove that the proposed algorithmic framework converges linearly in expectation. Numerical experiments demonstrate the benefits of the proposed preconditioning strategy.

math.NA↗

arXiv2026

Randomized conjugate gradient least squares

We develop a novel randomized conjugate gradient least squares (RCGLS) method for solving least-squares problems, in which iterative sketching is employed at each step to reduce the dimension and hence the computational cost. In particular, we propose a new perspective on the classical CGLS method, where the next descent direction is determined via a constraint correction problem associated with the gradient. Based on this insight, we replace the gradient with a randomized coordinate gradient that naturally satisfies the variance reduction property, leading directly to the proposed RCGLS method. We prove that RCGLS converges linearly in expectation, with a better convergence bound compared to the randomized coordinate descent method. Furthermore, we investigate an implementation of the method that avoids full-dimensional vector operations, which are the major bottleneck of vanilla RCGLS for sparse matrices and render it impractical. We also show how to apply the RCGLS method to solve the ridge regression problem, yielding a lightweight, parallelizable, and accelerated method for such problems. Numerical experiments are provided to confirm our results.

math.NA↗

arXiv2026

Linear convergence of Gearhart-Koshy accelerated Kaczmarz methods for tensor linear systems

The generalized Gearhart-Koshy acceleration is a recent exact affine search technique designed for the method of cyclic projections onto hyperplanes, i.e., the Kaczmarz method. However, its convergence properties, particularly the linear convergence rate, have not been thoroughly established. In this paper, we systematically establish the linear convergence of the generalized Gearhart-Koshy accelerated Kaczmarz method for tensor linear systems, proving that it converges linearly to the unique least-norm solution. Our analysis is general and applies to several popular Kaczmarz variants, including incremental, shuffle-once, and random-reshuffling schemes, and demonstrates that this acceleration approach yields a better convergence upper bound compared to the plain Kaczmarz method. We also propose an efficient Gram-Schmidt-based implementation that computes the next iterate in linear time. Building on this implementation, we establish a connection between this acceleration framework and Arnoldi-type Krylov subspace methods, further highlighting its efficiency and potential. Our theoretical results are supported by numerical experiments.

math.NA↗

arXiv2026

NS-RGS: Newton-Schulz based Riemannian gradient method for orthogonal group synchronization

Group synchronization is a fundamental task involving the recovery of group elements from pairwise measurements. For orthogonal group synchronization, the most common approach reformulates the problem as a constrained nonconvex optimization and solves it using projection-based methods, such as the generalized power method. However, these methods rely on exact SVD or QR decompositions in each iteration, which are computationally expensive and become a bottleneck for large-scale problems. In this paper, we propose a Newton-Schulz-based Riemannian Gradient Scheme (NS-RGS) for orthogonal group synchronization that significantly reduces computational cost by replacing the SVD or QR step with the Newton-Schulz iteration. This approach leverages efficient matrix multiplications and aligns perfectly with modern GPU/TPU architectures. By employing a refined leave-one-out analysis, we overcome the challenge arising from statistical dependencies, and establish that NS-RGS with spectral initialization achieves linear convergence to the target solution up to near-optimal statistical noise levels. Experiments on synthetic data and real-world global alignment tasks demonstrate that NS-RGS attains accuracy comparable to state-of-the-art methods such as the generalized power method, while achieving nearly a 2$\times$ speedup.

stat.ML↗

arXiv2026

Parallelizable Riemannian Alternating Direction Method of Multipliers for Non-convex Pose Graph Optimization

Pose graph optimization (PGO) is fundamental to robot perception and navigation systems, serving as the mathematical backbone for solving simultaneous localization and mapping (SLAM). Existing solvers suffer from polynomial growth in computational complexity with graph size, hindering real-time deployment in large-scale scenarios. In this paper, by duplicating variables and introducing equality constraints, we reformulate the problem and propose a Parallelizable Riemannian Alternating Direction Method of Multipliers (PRADMM) to solve it efficiently. Compared with the state-of-the-art methods that usually exhibit polynomial time complexity growth with graph size, PRADMM enables efficient parallel computation across vertices regardless of graph size. Crucially, all subproblems admit closed-form solutions, ensuring PRADMM maintains exceptionally stable performance. Furthermore, by carefully exploiting the structures of the coefficient matrices in the constraints, we establish the global convergence of PRADMM under mild conditions, enabling larger relaxation step sizes within the interval $(0,2)$. Extensive empirical validation on two synthetic datasets and multiple real-world 3D SLAM benchmarks confirms the superior computational performance of PRADMM.

math.OC↗

arXiv2025

Research on the descent direction of prediction correction algorithms for pseudo-convex/convex optimization problems

Prediction-correction algorithms are a highly effective class of methods for solving pseudo-convex optimization problems. The descent direction of these algorithms can be viewed as an adjustment to the gradient direction based on the prediction step. This paper investigates the adjustment coefficients of these descent directions and offers explanations from the perspective of differential equations. Unlike existing algorithms where the adjustment coefficient is always set to 1, we establish that the range of the adjustment coefficient lies within (1/2,1] for pseudo-convex optimization problems, and [0,1] for convex optimization problems. We also provide rigorous convergence proofs for these proposed algorithms. Numerical experiment results show that the algorithms perform best when the value of the adjustment coefficient makes the algorithm approach or equal to those in differential equations with higher-order global discrete error.

math.OC↗

arXiv2025

A Two-step Krasnosel'skii-Mann Algorithm with Adaptive Momentum and Its Applications to Image Denoising and Matrix Completion

In this paper, we propose a Two-step Krasnosel'skii-Mann (KM) Algorithm (TKMA) with adaptive momentum for solving convex optimization problems arising in image processing. Such optimization problems can often be reformulated as fixed-point problems for certain operators, which are then solved using iterative methods based on the same operator, including the KM iteration, to ultimately obtain the solution to the original optimization problem. Prior to developing TKMA, we first introduce a KM iteration enhanced with adaptive momentum, derived from geometric properties of an $\alpha$-averaged nonexpansive operator T with $\alpha\in(0,1)$, KM acceleration technique, and information from the composite operator $T^2$. The proposed TKMA is constructed as a convex combination of this adaptive-momentum KM iteration and the Picard iteration of $T^2$. We prove that the sequence generated by TKMA converges weakly to a fixed point of T in a real Hilbert space. Moreover, under $\alpha\in(0,1/2]$ and specific assumptions on the adaptive momentum parameters, we prove that the algorithm achieves an $o\left(1/\sqrt{k}\right)$ convergence rate in terms of the distance between successive iterates. Numerical experiments demonstrate that TKMA outperforms the FPPA, PGA, Fast KM algorithm, and Halpern algorithm on tasks such as image denoising and low-rank matrix completion.

math.OC↗

arXiv2025

Approximating Quantum States with Positive Partial Transposes in Multipartite System via Linearized Proximal Alternative Direction Method of Multipliers

Numerical approximation of quantum states via convex combinations of states with positive partial transposes (bi-PPT state) in multipartite systems constitutes a fundamental challenge in quantum information science. We reformulate this problem as a linearly constrained optimization problem. An approximate model is constructed through an auxiliary variable and a suitable penalty parameter, balancing constraint violation and approximation error. To slove the approximate model, we design a Linearized Proximal Alternating Direction Method of Multipliers (LPADMM), proving its convergence under a prescribed inequality condition on regularization parameters. The algorithm achieves an iteration complexity of $O(1/\epsilon^2)$ for attaining $\epsilon$-stationary solutions. Numerical validation on diverse quantum systems, including three-qubit W/GHZ states and five-partite GHZ and multiGHZ states with noises, confirms high-quality bi-PPT approximations and decomposability certification, demonstrating the utility of our method for quantum information applications.

math-ph↗

arXiv2025

A fixed-time stable dynamical model for solving EVLCPs

A fixed-time stable dynamical system for solving the extended vertical linear complementarity problem (EVLCP) is developed. The system is based on the reformulation of EVLCP as a special case of a new kind of generalized absolute value equations. Some properties of the new kind of generalized absolute value equations are explored which are useful for developing a fixed-time stable dynamical system for solving it. Without using any smoothing technique, we develop a dynamical system for solving the new kind of generalized absolute value equations and prove its fixed-time stability. The model is applicable for solving EVLCP. As two by-products, a new condition which guarantees the unique solvability of EVLCP and a new error bound of EVLCP are given. Numerical results are given to demonstrate our claims.

math.NA↗

arXiv2025

An inverse-free fixed-time stable dynamical system and its forward-Euler discretization for solving generalized absolute value equations

An inverse-free dynamical system is proposed to solve the generalized absolute value equation (GAVE) with a fixed time convergence, where the time of convergence is finite and is uniformly bounded for all initial points. Moreover, an iterative method obtained by using the forward-Euler discretization of the proposed dynamic model is developed and sufficient conditions which guarantee that the discrete iteration globally converge to an arbitrarily small neighborhood of the unique solution of GAVE within a finite number of iterative steps are given. Numerical results illustrate the effectiveness of the proposed methods.

math.NA↗

arXiv2025

The Condition Number in Phase Retrieval from Intensity Measurements

This paper investigates the stability of phase retrieval by analyzing the condition number of the nonlinear map $\Psi_{\boldsymbol{A}}(\boldsymbol{x}) = \bigl(\lvert \langle {\boldsymbol{a}}_j, \boldsymbol{x} \rangle \rvert^2 \bigr)_{1 \le j \le m}$, where $\boldsymbol{a}_j \in \mathbb{H}^n$ are known sensing vectors with $\mathbb{H} \in \{\mathbb{R}, \mathbb{C}\}$. For each $p \ge 1$, we define the condition number $\beta_{\Psi_{\boldsymbol{A}}}^{\ell_p}$ as the ratio of optimal upper and lower Lipschitz constants of $\Psi_{\boldsymbol{A}}$ measured in the $\ell_p$ norm, with respect to the metric $\mathrm {dist}_\mathbb{H}\left(\boldsymbol{x}, \boldsymbol{y}\right) = \|\boldsymbol{x} \boldsymbol{x}^\ast - \boldsymbol{y} \boldsymbol{y}^\ast\|_*$. We establish universal lower bounds on $\beta_{\Psi_{\boldsymbol{A}}}^{\ell_p}$ for any sensing matrix $\boldsymbol{A} \in \mathbb{H}^{m \times d}$, proving that $\beta_{\Psi_{\boldsymbol{A}}}^{\ell_1} \ge \pi/2$ and $\beta_{\Psi_{\boldsymbol{A}}}^{\ell_2} \ge \sqrt{3}$ in the real case $(\mathbb{H} = \mathbb{R})$, and $\beta_{\Psi_{\boldsymbol{A}}}^{\ell_p} \ge 2$ for $p=1,2$ in the complex case $(\mathbb{H} = \mathbb{C})$. These bounds are shown to be asymptotically tight: both a deterministic harmonic frame $\boldsymbol{E}_m \in \mathbb{R}^{m \times 2}$ and Gaussian random matrices $\boldsymbol{A} \in \mathbb{H}^{m \times d}$ asymptotically attain them. Notably, the harmonic frame $\boldsymbol{E}_m \in \mathbb{R}^{m \times 2}$ achieves the optimal lower bound $\sqrt{3}$ for all $m \ge 3$ when $p=2$, thus serving as an optimal sensing matrix within $\boldsymbol{A} \in \mathbb{R}^{m \times 2}$. Our results provide the first explicit uniform lower bounds on $\beta_{\Psi_{\boldsymbol{A}}}^{\ell_p}$ and offer insights into the fundamental stability limits of phase retrieval.

cs.IT↗

arXiv2025

Enhanced randomized Douglas-Rachford method: Improved probabilities and adaptive momentum

Randomized iterative methods have gained recent interest in machine learning and signal processing for solving large-scale linear systems. One such example is the randomized Douglas-Rachford (RDR) method, which updates the iterate by reflecting it through two randomly selected hyperplanes and taking a convex combination with the current point. In this work, we enhance RDR by introducing improved sampling strategies and an adaptive heavy-ball momentum scheme. Specifically, we incorporate without-replacement and volume sampling into RDR, and establish stronger convergence guarantees compared to conventional i.i.d. sampling. Furthermore, we develop an adaptive momentum mechanism that dynamically adjusts step sizes and momentum parameters based on previous iterates, and prove that the resulting method achieves linear convergence in expectation with improved convergence bounds. Numerical experiments demonstrate that the enhanced RDR method consistently outperforms the original version, providing substantial practical benefits across a range of problem settings.

math.NA↗

arXiv2025

Connecting randomized iterative methods with Krylov subspaces

Randomized iterative methods, such as the randomized Kaczmarz method, have gained significant attention for solving large-scale linear systems due to their simplicity and efficiency. Meanwhile, Krylov subspace methods have emerged as a powerful class of algorithms, known for their robust theoretical foundations and rapid convergence properties. Despite the individual successes of these two paradigms, their underlying connection has remained largely unexplored. In this paper, we develop a unified framework that bridges randomized iterative methods and Krylov subspace techniques, supported by both rigorous theoretical analysis and practical implementation. The core idea is to formulate each iteration as an adaptively weighted linear combination of the sketched normal vector and previous iterates, with the weights optimally determined via a projection-based mechanism. This formulation not only reveals how subspace techniques can enhance the efficiency of randomized iterative methods, but also enables the design of a new class of iterative-sketching-based Krylov subspace algorithms. We prove that our method converges linearly in expectation and validate our findings with numerical experiments.

math.NA↗

arXiv2025

Inexact Regularized Quasi-Newton Algorithm for Solving Monotone Variational Inequality Problems

Newton's method has been an important approach for solving variational inequalities, quasi-Newton method is a good alternative choice to save computational cost. In this paper, we propose a new method for solving monotone variational inequalities where we introduce a merit function based on the merit function. With the help of the merit function, we can locally accepts unit step size. And a globalization technique based on the hyperplane is applied to the method. The proposed method applied to monotone variational inequality problems is globally convergent in the sense that subproblems always have unique solutions, and the whole sequence of iterates converges to a solution of the problem without any regularity assumptions. We also provide extensive numerical results to demonstrate the efficiency of the proposed algorithm.

math.OC↗

arXiv2025

A Single-loop Proximal Subgradient Algorithm for A Class Structured Fractional Programs

In this paper, we investigate a class of nonconvex and nonsmooth fractional programming problems, where the numerator composed of two parts: a convex, nonsmooth function and a differentiable, nonconvex function, and the denominator consists of a convex, nonsmooth function composed of a linear operator. These structured fractional programming problems have broad applications, including CT reconstruction, sparse signal recovery, the single-period optimal portfolio selection problem and standard Sharpe ratio minimization problem. We develop a single-loop proximal subgradient algorithm that alleviates computational complexity by decoupling the evaluation of the linear operator from the nonsmooth component. We prove the global convergence of the proposed single-loop algorithm to an exact lifted stationary point under the Kurdyka-\L ojasiewicz assumption. Additionally, we present a practical variant incorporating a nonmonotone line search to improve computational efficiency. Finally, through extensive numerical simulations, we showcase the superiority of the proposed approach over the existing state-of-the-art methods for three applications: $L_{1}/S_{\kappa}$ sparse signal recovery, limited-angle CT reconstruction, and optimal portfolio selection.

math.OC↗

arXiv2025

An Accelerated Alternating Partial Bregman Algorithm for ReLU-based Matrix Decomposition

Despite the remarkable success of low-rank estimation in data mining, its effectiveness diminishes when applied to data that inherently lacks low-rank structure. To address this limitation, in this paper, we focus on non-negative sparse matrices and aim to investigate the intrinsic low-rank characteristics of the rectified linear unit (ReLU) activation function. We first propose a novel nonlinear matrix decomposition framework incorporating a comprehensive regularization term designed to simultaneously promote useful structures in clustering and compression tasks, such as low-rankness, sparsity, and non-negativity in the resulting factors. This formulation presents significant computational challenges due to its multi-block structure, non-convexity, non-smoothness, and the absence of global gradient Lipschitz continuity. To address these challenges, we develop an accelerated alternating partial Bregman proximal gradient method (AAPB), whose distinctive feature lies in its capability to enable simultaneous updates of multiple variables. Under mild and theoretically justified assumptions, we establish both sublinear and global convergence properties of the proposed algorithm. Through careful selection of kernel generating distances tailored to various regularization terms, we derive corresponding closed-form solutions while maintaining the $L$-smooth adaptable property always holds for any $L\ge 1$. Numerical experiments, on graph regularized clustering and sparse NMF basis compression confirm the effectiveness of our model and algorithm.

cs.LG↗

arXiv2024

SOR-like iteration and FPI are consistent when they are equipped with certain optimal iterative parameters

Two common methods for solving absolute value equations (AVE) are SOR-like iteration method and fixed point iteration (FPI) method. In this paper, novel convergence analysis, which result wider convergence range, of the SOR-like iteration and the FPI are given. Based on the new analysis, a new optimal iterative parameter with a analytical form is obtained for the SOR-like iteration. In addition, an optimal iterative parameter with a analytical form is also obtained for FPI. Surprisingly, the SOR-like iteration and the FPI are the same whenever they are equipped with our optimal iterative parameters. As a by product, we give two new constructive proof for a well known sufficient condition such that AVE has a unique solution for any right hand side. Numerical results demonstrate our claims.

math.NA↗