arXiv subjects

Gaohang Yu

Publications and source records attributed to Gaohang Yu.

18 recordsLinked to original sources

arXiv2026

A Two-Sided Sketching Algorithm for Low-rank Tensor Train Approximation

Tensor train (TT) decomposition is a powerful method to acquire low-rank tensors. However, the computational process is frequently obstructed by the large-scale matrix singular value decomposition (SVD). The sketching algorithm serves as an efficient data compression technique that can quickly derive low-rank matrix approximations. In this paper, we propose a randomized algorithm to obtain the TT approximation of tensors using a one-pass sketching algorithm and subspace iteration, and offer thorough error-bound and robustness analysis. Numerical experiments on synthetic and real-world datasets demonstrate the effectiveness and efficiency of the proposed algorithm.

math.NA↗

arXiv2026

Explicit Convergence Regions of PID-Damped Accelerated Gradient Methods in Nonconvex Optimization

Momentum-based accelerated gradient methods are widely adopted to expedite convergence in nonconvex optimization, but are prone to overshooting and oscillatory behavior. A class of PID-damped accelerated gradient methods mitigates this issue by augmenting classical momentum methods with a discrete-time derivative damping term. However, the coupling among the step size, momentum, and derivative gain renders the explicit characterization of their convergence regions analytically intractable, leaving explicit theoretical convergence boundaries unexplored. In this paper, we model this class of algorithms as a third-order nonlinear feedback dynamical system and establish explicit three-dimensional convergence regions for the step size, momentum, and derivative gain via a robust control-theoretic analysis based on the Kalman-Yakubovich-Popov (KYP) lemma, formally guaranteeing linear convergence under the Regularity Condition. Furthermore, we reveal a strict geometric upper bound on the derivative gain dictated by the nonconvex curvature, beyond which over-damping severely contracts the feasible step-size region, providing a rigorous theoretical explanation for the overdamped stagnation phenomenon. Numerical experiments corroborate the theoretical boundaries and illustrate practical parameter selection guidelines for the derivative gain.

math.OC↗

arXiv2026

Optimization Models and Computational Bounds for Limited Augmented Zarankiewicz Numbers in the Incidence-Graph Family of Complete Graphs

Let $G_1$ denote the incidence graph of the complete graph $K_{q+1}$. We study limited augmented Zarankiewicz numbers in this family through structured 0--1 optimization models for admissible 2-edge augmentation. For the original framework, we combine exact ILP formulations for the smallest instances with constructive search followed by exact admissibility verification for larger instances. This yields \[ {z_L(6,4)=14,\qquad z_L(10,5)=26,\qquad z_L(15,6)\ge 43,\qquad z_L(21,7)\ge 64,\qquad z_L(28,8)\ge 88.} \] The first two values are exact, whereas the latter three are rigorous lower bounds obtained from explicitly verified admissible families. We then formulate the global weak problem determined by $(S),(W2),(W2'),(W3)$. On the larger incidence-family cases, the corresponding global exact optimization problem is currently computationally out of reach, and our computations provide the certified lower bounds \[ {z_{WL}(6,4)=14,\quad z_{WL}(10,5)\ge 29,\quad z_{WL}(15,6)\ge 46,\quad z_{WL}(21,7)\ge 65,\quad z_{WL}(28,8)\ge 90.} \] On the fourteen literal-grid benchmarks from $(4,3)$ through $(7,7)$, by contrast, the global weak problem is solved exactly. Across this exact benchmark block, the weak framework is never worse than the strong framework and is strictly better in several cases. These computations improve the corresponding classical Zarankiewicz numbers and therefore strengthen available lower bounds for $\operatorname{BSR}(m,n)$ within this family.

math.CO↗

arXiv2026

GLT-PEFT: Gated Lie-Tucker Parameter-Efficient Fine-Tuning for Alzheimer's Disease Diagnosis with Hippocampal Segmentation Pretraining

Parameter-efficient fine-tuning (PEFT) has emerged as a promising paradigm for adapting pretrained models under limited data conditions. However, most existing PEFT methods are designed for matrix-structured parameters and are not well suited for high-dimensional convolutional kernels in medical imaging models. Moreover, they typically rely on additive updates and lack mechanisms to preserve the geometric structure of pretrained parameters, while multiplicative (geometry-aware) updates are difficult to integrate within a unified framework. To address this issue, this paper proposes GLT-PEFT, a gated Lie-Tucker parameter-efficient fine-tuning framework for Alzheimer's disease (AD) diagnosis. The proposed approach transfers a hippocampal segmentation pretrained model to a downstream classification task. Tucker decomposition enables tensor-aware low-rank adaptation of 3D convolutional kernels, while Lie group-based transformations provide structure-preserving multiplicative updates. A gating mechanism further reconciles additive and multiplicative update forms, resulting in a unified and more stable fine-tuning strategy. Extensive experiments demonstrate that GLT-PEFT achieves effective cross-task transfer while significantly reducing trainable parameters, highlighting its effectiveness for efficient and robust adaptation in medical imaging models.

cs.CV↗

arXiv2025

tCURLoRA: Tensor CUR Decomposition Based Low-Rank Parameter Adaptation and Its Application in Medical Image Segmentation

Transfer learning, by leveraging knowledge from pre-trained models, has significantly enhanced the performance of target tasks. However, as deep neural networks scale up, full fine-tuning introduces substantial computational and storage challenges in resource-constrained environments, limiting its widespread adoption. To address this, parameter-efficient fine-tuning (PEFT) methods have been developed to reduce computational complexity and storage requirements by minimizing the number of updated parameters. While matrix decomposition-based PEFT methods, such as LoRA, show promise, they struggle to fully capture the high-dimensional structural characteristics of model weights. In contrast, high-dimensional tensors offer a more natural representation of neural network weights, allowing for a more comprehensive capture of higher-order features and multi-dimensional interactions. In this paper, we propose tCURLoRA, a novel fine-tuning method based on tensor CUR decomposition. By concatenating pre-trained weight matrices into a three-dimensional tensor and applying tensor CUR decomposition, we update only the lower-order tensor components during fine-tuning, effectively reducing computational and storage overhead. Experimental results demonstrate that tCURLoRA outperforms existing PEFT methods in medical image segmentation tasks.

eess.IV↗

arXiv2024

LoRA-PT: Low-Rank Adapting UNETR for Hippocampus Segmentation Using Principal Tensor Singular Values and Vectors

The hippocampus is an important brain structure involved in various psychiatric disorders, and its automatic and accurate segmentation is vital for studying these diseases. Recently, deep learning-based methods have made significant progress in hippocampus segmentation. However, training deep neural network models requires substantial computational resources, time, and a large amount of labeled training data, which is frequently scarce in medical image segmentation. To address these issues, we propose LoRA-PT, a novel parameter-efficient fine-tuning (PEFT) method that transfers the pre-trained UNETR model from the BraTS2021 dataset to the hippocampus segmentation task. Specifically, LoRA-PT divides the parameter matrix of the transformer structure into three distinct sizes, yielding three third-order tensors. These tensors are decomposed using tensor singular value decomposition to generate low-rank tensors consisting of the principal singular values and vectors, with the remaining singular values and vectors forming the residual tensor. During fine-tuning, only the low-rank tensors (i.e., the principal tensor singular values and vectors) are updated, while the residual tensors remain unchanged. We validated the proposed method on three public hippocampus datasets, and the experimental results show that LoRA-PT outperformed state-of-the-art PEFT methods in segmentation accuracy while significantly reducing the number of parameter updates. Our source code is available at https://github.com/WangangCheng/LoRA-PT/tree/LoRA-PT.

cs.CV↗

arXiv2024

An Efficient Two-Sided Sketching Method for Large-Scale Tensor Decomposition Based on Transformed Domains

Large tensors are frequently encountered in various fields such as computer vision, scientific simulations, sensor networks, and data mining. However, these tensors are often too large for convenient processing, transfer, or storage. Fortunately, they typically exhibit a low-rank structure that can be leveraged through tensor decomposition. However, performing large-scale tensor decomposition can be time-consuming. Sketching is a useful technique to reduce the dimensionality of the data. In this paper, we propose a novel two-sided sketching method based on the $\star_{L}$-product decomposition and transformed domains like the discrete cosine transformation. A rigorous theoretical analysis is also conducted to assess the approximation error of the proposed method. Specifically, we improve our method with power iteration to achieve more precise approximate solutions. Extensive numerical experiments and comparisons on low-rank approximation of synthetic large tensors and real-world data like color images and grayscale videos illustrate the efficiency and effectiveness of the proposed approach in terms of both CPU time and approximation accuracy.

math.OC↗

arXiv2023

Low-rank Tensor Train Decomposition Using TensorSketch

Tensor train decomposition is one of the most powerful approaches for processing high-dimensional data. For low-rank tensor train decomposition of large tensors, the alternating least squares (ALS) algorithm is widely used by updating each core tensor alternatively. However, it may suffer from the curse of dimensionality due to the large scale of subproblems. In this paper, a novel randomized proximal ALS algorithm is proposed for low-rank tensor train decomposition by using TensorSketch, which allows for efficient implementation via fast Fourier transform. The theoretical lower bounds of sketch size are estimated for approximating the optimal value of subproblems. Numerical experiments on synthetic and real-world data also demonstrate the effectiveness and efficiency of the proposed algorithm.

math.NA↗

arXiv2023

A Randomized Block Krylov Method for Tensor Train Approximation

Tensor train decomposition is a powerful tool for dealing with high-dimensional, large-scale tensor data, which is not suffering from the curse of dimensionality. To accelerate the calculation of the auxiliary unfolding matrix, some randomized algorithms have been proposed; however, they are not suitable for noisy data. The randomized block Krylov method is capable of dealing with heavy-tailed noisy data in the low-rank approximation of matrices. In this paper, we present a randomized algorithm for low-rank tensor train approximation of large-scale tensors based on randomized block Krylov subspace iteration and provide theoretical guarantees. Numerical experiments on synthetic and real-world tensor data demonstrate the effectiveness of the proposed algorithm.

math.NA↗

arXiv2023

Variable T-Product and Zero-Padding Tensor Completion with Applications

The T-product method based upon Discrete Fourier Transformation (DFT) has found wide applications in engineering, in particular, in image processing. In this paper, we propose variable T-product, and apply the Zero-Padding Discrete Fourier Transformation (ZDFT), to convert third order tensor problems to the variable Fourier domain. An additional positive integer parameter is introduced, which is greater than the tubal dimension of the third order tensor. When the additional parameter is equal to the tubal dimension, the ZDFT reduces to DFT. Then, we propose a new tensor completion method based on ZDFT and TV regularization, called VTCTF-TV. Extensive numerical experiment results on visual data demonstrate that the superior performance of the proposed method. In addition, the zero-padding is near the size of the original tubal dimension, the resulting tensor completion can be performed much better.

math.OC↗

arXiv2023

Practical Sketching Algorithms for Low-Rank Tucker Approximation of Large Tensors

Low-rank approximation of tensors has been widely used in high-dimensional data analysis. It usually involves singular value decomposition (SVD) of large-scale matrices with high computational complexity. Sketching is an effective data compression and dimensionality reduction technique applied to the low-rank approximation of large matrices. This paper presents two practical randomized algorithms for low-rank Tucker approximation of large tensors based on sketching and power scheme, with a rigorous error-bound analysis. Numerical experiments on synthetic and real-world tensor data demonstrate the competitive performance of the proposed algorithms.

math.NA↗

arXiv2022

Multi-mode Tensor Train Factorization with Spatial-spectral Regularization for Remote Sensing Images Recovery

Tensor train (TT) factorization and corresponding TT rank, which can well express the low-rankness and mode correlations of higher-order tensors, have attracted much attention in recent years. However, TT factorization based methods are generally not sufficient to characterize low-rankness along each mode of third-order tensor. Inspired by this, we generalize the tensor train factorization to the mode-k tensor train factorization and introduce a corresponding multi-mode tensor train (MTT) rank. Then, we proposed a novel low-MTT-rank tensor completion model via multi-mode TT factorization and spatial-spectral smoothness regularization. To tackle the proposed model, we develop an efficient proximal alternating minimization (PAM) algorithm. Extensive numerical experiment results on visual data demonstrate that the proposed MTTD3R method outperforms compared methods in terms of visual and quantitative measures.

eess.IV↗

arXiv2021

T-Singular Values and T-Sketching for Third Order Tensors

Based upon the T-SVD (tensor SVD) of third order tensors, introduced by Kilmer and her collaborators, we define T-singular values of third order tensors. T-singular values of third order tensors are nonnegative scalars. The number of nonzero T-singular values is the tensor tubal rank of the tensor. We then use T-singular values to define the tail energy of a third order tensor, and apply it to the error estimation of a tensor sketching algorithm for low rank tensor approximation. Numerical experiments on real world data show that our algorithm is efficient.

math.NA↗

arXiv2020

A Parallelizable Method for Missing Internet Traffic Tensor Data

Recovery of internet network traffic data from incomplete observed data is an important issue in internet network engineering and management. In this paper, by fully combining the temporal stability and periodicity features in internet traffic data, a new separable optimization model for internet data recovery is proposed, which is based upon the t-product and the rapid discrete Fourier transform of tensors. Moreover, by using generalized inverse matrices, an easy-to-operate and effective algorithm is proposed. In theory, we prove that under suitable conditions, every accumulation point of the sequence generated by the proposed algorithm is a stationary point of the established model. Numerical simulation results carried on the widely used real-world internet network datasets, show good performance of the proposed method. In the case of moderate sampling rates, the proposed method works very well, its effect is better than that of some existing internet traffic data recovery methods in the literature. The separable structural features presented in the optimization model provide the possibility to design more efficient parallel algorithms.

math.NA↗

arXiv2020

Accelerating Power Methods for Higher-order Markov Chains

Higher-order Markov chains play a very important role in many fields, ranging from multilinear PageRank to financial modeling. In this paper, we propose three accelerated higher-order power methods for computing the limiting probability distribution of higher-order Markov chains, namely higher-order power method with momentum and higher-order quadratic extrapolation method. The convergence results are established, and numerical experiments are reported to show the efficiency of the proposed algorithms. In particular, the non-parametric quadratic extrapolation method is very competitive, and outperforms state-of-the-art competitions.

math.OC↗

arXiv2016

Spectral projected gradient methods for generalized tensor eigenvalue complementarity problem

This paper looks at the tensor eigenvalue complementarity problem (TEiCP) which arises from the stability analysis of finite dimensional mechanical systems and is closely related to the optimality conditions for polynomial optimization. We investigate two monotone ascent spectral projected gradient (SPG) methods for TEiCP. We also present a shifted scaling-and-projection algorithm (SPA), which is a great improvement of the original SPA method proposed by Ling, He and Qi [Comput. Optim. Appl., DOI 10.1007/s10589-015-9767-z]. Numerical comparisons with some existed gradient methods in the literature are reported to illustrate the efficiency of the proposed methods.

math.OC↗

arXiv2016

An adaptive gradient method for computing generalized tensor eigenpairs

High order tensor arises more and more often in signal processing,data analysis, higher-order statistics, as well as imaging sciences. In this paper, an adaptive gradient (AG) method is presented for generalized tensor eigenpairs. Global convergence and linear convergence rate are established under some suitable conditions. Numerical results are reported to illustrate the efficiency of the proposed method. Comparing with the GEAP method, an adaptive shifted power method proposed by Tamara G. Kolda and Jackson R. Mayo [SIAM J. Matrix Anal. Appl., 35 (2014), pp. 1563-1581], the AG method is much faster and could reach the largest eigenpair with a higher probability.

math.OC↗

arXiv2015

Properties of Solution set of Tensor Complementarity Problem

The tensor complementarity problem is a specially structured nonlinear complementarity problem, then it has its particular and nice properties other than ones of the classical nonlinear complementarity problem. In this paper, it is proved that a tensor is an S-tensor if and only if the tensor complementarity problem is feasible, and each Q-tensor is an S-tensor. Furthermore, the boundedness of solution set of the tensor complementarity problem is equivalent to the uniqueness of solution for such a problem with zero vector. For the tensor complementarity problem with a strictly semi-positive tensor, we proved the global upper bounds for solution of such a problem. In particular, the upper bounds keep in close contact with the smallest Pareto $H-$($Z-$)eigenvalue.

math.OC↗