SearcharxivSearch

arXiv subjects

Lei-Hong Zhang

Publications and source records attributed to Lei-Hong Zhang.

At least 19 recordsLinked to original sources

NEP_MiniMax: An approach for NEPs based on minimax approximation of the split coefficient vector

We propose \textsf{NEP\_MiniMax}, a novel computational method for solving nonlinear eigenvalue problems (NEPs) $T(λ)\mathbf{u} = \mathbf{0}$ on a Jordan domain $Ω\subset \mathbb{C}$. For an NEP in split form $T(x)=\sum_{i=1}^s t_i(x)E_i$, the method applies the \textsf{m-d-Lawson} algorithm to the split coefficient vector-valued function $\mathbf{t}(x)=[t_1(x),\ldots,t_s(x)]^{\textrm{T}}$ on $Ω$, rather than directly minimizing an error for the matrix-valued function $T$. The resulting rational minimax approximant ${\boldsymbolξ}^*(x)=[r_1^*(x),\ldots,r_s^*(x)]^{\textrm{T}}$ induces the rational matrix surrogate $R^*(x)=\sum_{i=1}^s r_i^*(x)E_i=P^*(x)/q^*(x)\approx T(x)$. The method combines this coefficient-vector approximation with a structure-exploiting linearization technique. A matrix error bound transfers the uniform coefficient-vector approximation accuracy to $R^*$ and yields computable eigenpair residual estimates. Eigenpairs are then computed by solving a polynomial eigenvalue problem $P^*(λ) \mathbf{u} = \mathbf{0}$ via a strong linearization that exactly preserves eigenvalue multiplicities of $P^*$. Numerical experiments on some problems from the NLEVP collection demonstrate competitiveness with state-of-the-art methods (e.g., Beyn, NLEIGS, SV-AAA) in efficiency and accuracy, with theoretical error bounds directly relating eigenpair {residuals} to the {split coefficient-vector approximation} quality.

math.NA

A Sharp Unitarily Invariant Norm Bound for the Off-Diagonal Block Perturbation of a Hermitian Matrix

Let $$ A=\begin{bmatrix} H_1 & E^* \\ E & H_2 \end{bmatrix} \quad\text{and}\quad \widetilde A=\begin{bmatrix} H_1 & 0 \\ 0 & H_2 \end{bmatrix} $$ be two partitioned Hermitian matrices, where $\widetilde A$ is obtained from $A$ by simply dropping the off-diagonal blocks, and let $η$ be the gap between the spectra ${\rm eig}(H_1)$ of $H_1$ and ${\rm eig}(H_2)$ of $H_2$. Define, for $δ\ge 0$ and $ε\ge 0$, $$ ϕ(δ,ε)= \begin{cases} 2ε/(δ+\sqrt{δ^2+4ε^2}), &\quad\mbox{if $(δ,ε)\ne (0,0)$}, 1, &\quad\mbox{if $(δ,ε) = (0,0)$}, \end{cases} $$ and let $V=A-\widetilde A$ and $ε_2=\|E\|_2=\|V\|_2$, the matrix spectral norm. Li and Li [{\em Linear Algebra Appl.}, 395:183--190, 2005] established a sharp spectral-norm bound on the changes in the eigenvalues of $A$: $$ \big\|{\rm diag}\big(\pmbλ(A)-\pmbλ(\widetilde A)\big)\big\|_2 \le ϕ(η,ε_2)\,\|E\|_2, $$ where $\pmbλ(A)$ is the vector whose components are the eigenvalues of $A$ in descending order and similarly for $\pmbλ(\widetilde A)$. The goal of this paper is to resolve the question: how far an extension of this result in the form $$ \big\|{\rm diag}\big(\pmbλ(A)-\pmbλ(\widetilde A)\big)\big\|_{\rm UI} \le ϕ(η,ε_2)\,\|A-\widetilde A\|_{\rm UI} $$ remains valid for some or all unitarily invariant norms $\|\cdot\|_{\rm UI}$? Two results are obtained: (a) the extension holds for any $Q$-norm, a subclass of unitarily invariant norms that encompasses the Schatten $p$-norm for $2\le p\le\infty$ (particularly, the Frobenius norm and the spectral norm included), and (b) the extension holds for any unitarily invariant norm if ${\rm rank}(E)\le 1$. It is demonstrated that the equality is attained on the $2\times 2$ matrix $A$.

math.NA

Quality of Approximate Balanced Truncation

Model reduction is a powerful tool in dealing with numerical simulation of large scale dynamic systems for studying complex physical systems. Two major types of model reduction methods for linear time-invariant dynamic systems are Krylov subspace-based methods and balanced truncation-based methods. The methods of the second type are much more theoretically sound than the first type in that there is a fairly tight global error bound on the approximation error between the original system and the reduced one. It is noted that the error bound is established based upon the availability of the exact controllability and observability Gramians. However, numerically, the Gramians are not available and have to be numerically calculated, and for a large scale system, a viable option is to compute low-rank approximations of the Gramians from which an approximate balanced truncation is then performed. Hence, rigorously speaking, the existing global error bound is not applicable to any reduced system obtained via approximate Gramians. The goal of this paper is to address this issue by establishing global error bounds for reduced systems via approximate balanced truncation.

math.NA

Rational Minimax Approximations for Matrix-Valued Functions: Existence, Optimality and Algorithms

In this paper, we study rational minimax approximation for continuous complex matrix-valued functions in the Frobenius norm, where all approximant entries share a common denominator. This generalizes classical scalar rational approximation, with applications in system modeling, microwave design, and nonlinear eigenvalue problems. We first prove the existence of such matrix-valued approximants on point sets dense in themselves, extending Walsh's foundational scalar result. Next, we establish characterizations of the local and global minimax approximants by deriving primal/dual matrix-valued Kolmogorov criteria and a Ruttan-type sufficient condition for global optimality. For analytic functions on a continuum, we link continuum minimax approximation to approximation on its boundary, and finite boundary samples via the maximum norm principle. We show that Ruttan's sufficient optimality condition provides a certificate under which a minimax approximant obtained from the boundary or from a discrete set of boundary nodes also solves the original continuum problem. Finally, for discrete approximation, we connect these conditions to a dual problem and the related dual-based numerical method m-d-Lawson: when the original minimax problem admits a solution, strong duality is equivalent to Ruttan's sufficient optimality condition, and, the optimality equations underlying the m-d-Lawson iteration coincide with Kolmogorov's dual criteria. These results provide a theoretical basis for certifying and computing matrix-valued rational minimax approximants.

math.OC

Interpolation constrained rational minimax approximation with barycentric representation

In this paper, we propose a novel dual-based Lawson's method, termed {b-d-Lawson}, designed for addressing the rational minimax approximation under specific interpolation conditions. The {b-d-Lawson} approach incorporates two pivotal components that have been recently gained prominence in the realm of the rational approximations: the barycentric representation of the rational function and the dual framework for tackling minimax approximation challenges. The employment of barycentric formulae enables a streamlined parameterization of the rational function, ensuring natural satisfaction of interpolation conditions while mitigating numerical instability typically associated with Vandermonde basis matrices when monomial bases are utilized. This enhances both the accuracy and computational stability of the method. To address the bi-level min-max structure, the dual framework effectively transforms the challenge into a max-min dual problem, thereby facilitating the efficient application of Lawson's iteration. The integration of this dual perspective is crucial for optimizing the approximation process. We will discuss several applications of interpolation-constrained rational minimax approximation and illustrate numerical results to evaluate the performance of the {b-d-Lawson} method.

math.NA

Optimality Conditions for Rational Minimax Approximations: Bridging Ruttan's Criteria to Dual-Based Methods

This paper presents a theoretical discussion on Ruttan's optimality conditions for rational minimax approximations in discrete and continuum settings, integrating analytical foundations with computational practice. We develop extended second-order optimality criteria for the discrete case, demonstrating that Ruttan's sufficient condition for global solutions [Ruttan, {Constr. Approx.}, 1 (1985), 287-296] becomes necessary when the number of extreme points is minimal. Our analysis further uncovers fundamental relationships between these conditions and the dual-based {d-Lawson} method [L.-H. Zhang et al., {Math. Comp.}, 94 (2025), 2457-2494], proving that strong duality in {d-Lawson} ensures simultaneous satisfaction of both Ruttan's and Kolmogorov's criteria. Additionally, we show that minimax approximants on a continuum satisfying Ruttan's sufficient global optimality can be captured through discrete minimax approximations at properly chosen boundary points, thereby enabling efficient computation of minimax approximants on a continuum using discrete methods.

math.NA

An NPDo Approach for Principal Joint Block Diagonalization

Matrix joint block-diagonalization (JBD) frequently arises from diverse applications such as independent component analysis, blind source separation, and common principal component analysis (CPCA), among others. Particularly, CPCA aims at joint diagonalization, i.e., each block size being $1$-by-$1$. This paper is concerned with {\em principal joint block-diagonalization\/} (\pjbd), which aim to achieve two goals: 1)~partial joint block-diagonalization, and 2)~identification of dominant common block-diagonal parts for all involved matrices. This is in contrast to most existing methods, especially the popular ones based on Givens rotation, which focus on full joint diagonalization and quickly become impractical for matrices of even moderate size ($300$-by-$300$ or larger). An NPDo approach is proposed and it is built on a {\em nonlinear polar decomposition with orthogonal polar factor dependency} that characterizes the solutions of the optimization problem designed to achieve \pjbd, and it is shown the associated SCF iteration is globally convergent to a stationary point while the objective function increases monotonically during the iterative process. Numerical experiments are presented to illustrate the effectiveness of the NPDo approach and its superiority to Givens rotation-based methods.

math.NA

Multivariate confluent Vandermonde with G-Arnoldi and applications

In the least-squares fitting framework, the Vandermonde with Arnoldi (V+A) method presented in [Brubeck, Nakatsukasa, and Trefethen, {SIAM Review}, 63 (2021), pp. 405-415] is an effective approach to compute a polynomial that approximates an underlying univariate function $f$. Extensions of V+A include its multivariate version and the univariate confluent V+A; the latter enables us to use the information of the derivative of $f$ in obtaining the approximation polynomial. In this paper, we shall extend V+A further to the multivariate confluent V+A. Besides the technical generalization of the univariate confluent V+A, we also introduce a general and application-dependent $G$-orthogonalization in the Arnoldi process. We shall demonstrate with several applications that, by specifying an application-related $G$-inner product, the desired approximate multivariate polynomial, as well as certain of its partial derivatives can be computed accurately from a well-conditioned least-squares problem whose coefficient matrix is orthonormal. The desired multivariate polynomial is represented in a discrete $G$-orthogonal polynomials basis which admits an explicit recurrence, and therefore, facilitates evaluating function values and certain partial derivatives at new nodes efficiently. We demonstrate its flexibility by applying it to solve the multivariate Hermite least-squares problem and PDEs with various boundary conditions in irregular domains.

math.NA

Rational minimax approximation of matrix-valued functions

In this paper, we present a rigorous framework for rational minimax approximation of matrix-valued functions that generalizes classical scalar approximation theory. Given sampled data $\{(x_\ell, {F}(x_\ell))\}_{\ell=1}^m$ where ${F}:\mathbb{C} \to \mathbb{C}^{s \times t}$ is a matrix-valued function, we study the problem of finding a matrix-valued rational approximant ${R}(x) = {P}(x)/q(x)$ (with ${P}:\mathbb{C} \to \mathbb{C}^{s \times t}$ a matrix-valued polynomial and $q(x)$ a nonzero scalar polynomial of prescribed degrees) that minimizes the worst-case Frobenius norm error over the given nodes: $$ \inf_{{R}(x) = {P}(x)/q(x)} \max_{1 \leq \ell \leq m} \|{F}(x_\ell) - {R}(x_\ell)\|_{\rm F}. $$ By reformulating this min-max optimization problem through Lagrangian duality, we derive a maximization dual problem over the probability simplex. We analyze weak and strong duality properties and establish a sufficient condition ensuring that the solution of the dual problem yields the minimax approximant $R(x)$. For numerical implementation, we propose an efficient method (\textsf{m-d-Lawson}) to solve the dual problem, generalizing Lawson's iteration to matrix-valued functions. Convergence analysis of \textsf{m-d-Lawson} is established. Numerical experiments are conducted and compared to state-of-the-art approaches, demonstrating its efficiency as a novel computational framework for matrix-valued rational approximation.

math.NA

A Minimal Perturbation Approach For The Rectangular Multiparameter Eigenvalue Problem

The rectangular multiparameter eigenvalue problem (RMEP) involves rectangular coefficient matrices (usually with more rows than columns) and may potentially have no solution in its original form. A minimal perturbation framework is proposed to defines approximate solutions. Computationally, two particular scenarios are considered: computing one approximate eigen-tuple or a complete set of approximate eigen-tuples. For computing one approximate eigen-tuple, an alternating iterative scheme with proven convergence is devised, while for a complete set of approximate eigen-tuples, the framework leads to a standard MEP (RMEP with square coefficient matrices) for numerical solutions. The proposed approach is validated on RMEPs from discretizing the multiparameter Sturm-Liouville equation and the Helmholtz equations by the least-squares spectral method.

math.NA

Kahan's Automatic Step-Size Control for Unconstrained Optimization

The Barzilai and Borwein (BB) gradient method is one of the most widely-used line-search gradient methods. It computes the step-size for the current iterate by using the information carried in the previous iteration. Recently, William Kahan [Kahan, Automatic Step-Size Control for Minimization Iterations, Technical report, University of California, Berkeley CA, USA, 2019] proposed new Gradient Descent (KGD) step-size strategies which iterate the step-size itself by effectively utilizing the information in the previous iteration. In the quadratic model, such a new step-size is shown to be mathematically equivalent to the long BB step, but no rigorous mathematical proof of its efficiency and effectiveness for the general unconstrained minimization is available. In this paper, by this equivalence with the long BB step, we first derive a short version of KGD step-size and show that, for the strongly convex quadratic model with a Hessian matrix $H$, both the long and short KGD step-size (and hence BB step-sizes) gradient methods converge at least R-linearly with a rate $1-\frac{1}{{\rm cond}(H)}$. For the general unconstrained minimization, we further propose an adaptive framework to effectively use the KGD step-sizes; global convergence and local R-linear convergence rate are proved. Numerical experiments are conducted on the CUTEst collection as well as the practical logistic regression problems, and we compare the performance of the proposed methods with various BB step-size approaches and other recently proposed adaptive gradient methods to demonstrate the efficiency and robustness.

math.OC

A convergence analysis of Lawson's iteration for computing polynomial and rational minimax approximations

Lawson's iteration is a classical and effective method for solving the linear (polynomial) minimax approximation problem in the complex plane. Extension of Lawson's iteration for the rational minimax approximation problem with both computationally high efficiency and theoretical guarantee is challenging. A recent work [L.-H. Zhang, L. Yang, W. H. Yang and Y.-N. Zhang, A convex dual problem for the rational minimax approximation and Lawson's iteration, Math. Comp., 94(2025), 2457-2494.] reveals that Lawson's iteration can be viewed as a method for solving the dual problem of the original rational minimax approximation problem, and a new type of Lawson's iteration, namely, d-Lawson, was proposed, which reduces to the classical Lawson's iteration for the linear minimax approximation problem. For the rational case, such a dual problem is guaranteed to obtain the original minimax solution under Ruttan's sufficient condition, and numerically, d-Lawson was observed to converge monotonically with respect to the dual objective function. In this paper, we present a theoretical convergence analysis of d-Lawson for both the linear and rational minimax approximation problems. In particular, we show that (i) for the linear minimax approximation problem, $β=1$ is a near-optimal Lawson exponent in Lawson's iteration, and (ii) for the rational minimax approximation problem, under certain conditions, d-Lawson converges monotonically with respect to the dual objective function for any sufficiently small $β>0$, and the limiting approximant satisfies the complementary slackness condition: any node associated with positive weight either is an interpolation point or has a constant error.

math.NA

An NEPv Approach for Feature Selection via Orthogonal OCCA with the (2,1)-norm Regularization

A novel feature selection model via orthogonal canonical correlation analysis with the $(2,1)$-norm regularization is proposed, and the model is solved by a practical NEPv approach (nonlinear eigenvalue problem with eigenvector dependency), yielding a feature selection method named OCCA-FS. It is proved that OCCA-FS always produces a sequence of approximations with monotonic objective values and is globally convergent. Extensive numerical experiments are performed to compare OCCA-FS against existing feature selection methods. The numerical results demonstrate that OCCA-FS produces superior classification performance and often comes out on the top among all feature selection methods in comparison.

math.NA

A convex dual problem for the rational minimax approximation and Lawson's iteration

Computing the discrete rational minimax approximation in the complex plane is challenging. Apart from Ruttan's sufficient condition, there are few other sufficient conditions for global optimality. The state-of-the-art rational approximation algorithms, such as the adaptive Antoulas-Anderson (AAA), AAA-Lawson, and the rational Krylov fitting (RKFIT) method, perform highly efficiently, but the computed rational approximations may not be minimax solutions. In this paper, we propose a convex programming approach, the solution of which is guaranteed to be the rational minimax approximation under Ruttan's sufficient condition. Furthermore, we present a new version of Lawson's iteration for solving this convex programming problem. The computed solution can be easily verified as the rational minimax approximation. Our numerical experiments demonstrate that this updated version of Lawson's iteration generally converges monotonically with respect to the objective function of the convex optimization. It is an effective competitive approach for computing the rational minimax approximation, compared to the highly efficient AAA, AAA-Lawson, and the stabilized Sanathanan-Koerner iteration.

math.NA

On Stewart's Perturbation Theorem for SVD

This paper establishes a variant of Stewart's theorem (Theorem~6.4 of Stewart, {\em SIAM Rev.}, 15:727--764, 1973) for the singular subspaces associated with the SVD of a matrix subject to perturbations. Stewart's original version uses both the Frobenius and spectral norms, whereas the new variant uses the spectral norm and any unitarily invariant norm that offer choices per convenience of particular applications and lead to sharper bounds than that straightforwardly derived from Stewart's original theorem with the help of the well-known equivalence inequalities between matrix norms. Of interest in their own right, bounds on the solution to two couple Sylvester equations are established for a few different circumstances.

math.NA

The $L_q$-weighted dual programming of the linear Chebyshev approximation and an interior-point method

Given samples of a real or complex-valued function on a set of distinct nodes, the traditional linear Chebyshev approximation is to compute the best minimax approximation on a prescribed linear functional space. Lawson's iteration is a classical and well-known method for that task. However, Lawson's iteration converges linearly and in many cases, the convergence is very slow. In this paper, by the duality theory of linear programming, we first provide an elementary and self-contained proof for the well-known Alternation Theorem in the real case. Also, relying upon the Lagrange duality, we further establish an $L_q$-weighted dual programming for the linear Chebyshev approximation. In this framework, we revisit the convergence of Lawson's iteration, and moreover, propose a Newton type iteration, the interior-point method, to solve the $L_2$-weighted dual programming. Numerical experiments are reported to demonstrate its fast convergence and its capability in finding the reference points that characterize the unique minimax approximation.

math.NA

On Generalizing Trace Minimization

Ky Fan's trace minimization principle is extended along the line of the Brockett cost function $\mathrm{trace}(DX^H AX)$ in $X$ on the Stiefel manifold, where $D$ of an apt size is positive definite. Specifically, we investigate $\inf_X \mathrm{trace}(DX^H AX)$ subject to $X^H BX=I_k$ or $J_k=\mathrm{diag}(\pm 1)$. We establish conditions under which the infimum is finite and when it is finite, analytic solutions are obtained in terms of the eigenvalues and eigenvectors of the matrix pencil $A-λB$, where $B$ is possibly indefinite and singular, and $D$ is also possibly indefinite.

math.NA

Trace Ratio Optimization with an Application to Multi-view Learning

A trace ratio optimization problem over the Stiefel manifold is investigated from the perspectives of both theory and numerical computations. At least three special cases of the problem have arisen from Fisher linear discriminant analysis, canonical correlation analysis, and unbalanced Procrustes problem, respectively. Necessary conditions in the form of nonlinear eigenvalue problem with eigenvector dependency are established and a numerical method based on the self-consistent field (SCF) iteration is designed and proved to be always convergent. As an application to multi-view subspace learning, a new framework and its instantiated concrete models are proposed and demonstrated on real world data sets. Numerical results show that the efficiency of the proposed numerical methods and effectiveness of the new multi-view subspace learning models.

math.OC