arXiv subjects

Yiting Yang

Publications and source records attributed to Yiting Yang.

8 recordsLinked to original sources

arXiv2026

Group-Shared Low-Rank Approximation for Mobile-Efficient Pointwise Convolutions in Large-Kernel CNNs

Large-kernel Convolutional Neural Networks (CNNs) deliver remarkable performance in vision tasks by significantly expanding receptive fields, yet their quadratic parameter growth critically impedes storage-efficient edge deployment. While existing efficient architectures adopt parameter-efficient depthwise separable convolution backbones that leverage techniques like low-rank approximation and weight sharing to compress depthwise convolutions, we identify a critical oversight: pointwise convolutions dominate parameter volume (>87% in models like RepLKNet-31B) and constitute the primary deployment bottleneck on resource-constrained edge devices. This results in prohibitive storage costs and severe memory-loading constraints on resource-limited devices (e.g., smartphones with 4-12 GB Random Access Memory (RAM)). To overcome this, we propose Channel Group-Shared (CGS) low-rank approximation, a novel Singular Value Decomposition (SVD)-based parameter-sharing strategy. CGS constructs a structured low-rank paradigm isomorphic to SVD decomposition, comprising shared (high-parameter-cost) down/up-projection matrices across channel groups within a layer and channel-group-specific (low-parameter-cost) scalable diagonal matrices. This group-sharing design achieves significant parameter reduction. Extensive experiments demonstrate that large-kernel CNNs (RepLKNet, ConvNeXt, SLaK) enhanced with CGS strike an empirically favorable balance between competitive performance and substantially reduced storage costs. Crucially, by alleviating storage constraints, reducing memory bandwidth pressure during loading, and minimizing model loading latency, CGS enables the feasible deployment of pre-trained large-kernel CNN models on edge devices, thereby bridging the gap between high-performance vision models and practical edge deployment.

cs.LG↗

arXiv2025

On the number of linear uniform hypergraphs with linear girth constraint

For an integer $r\geqslant 3$, a hypergraph on vertex set $[n]$ is $r$-uniform if each edge is a set of $r$ vertices, and is said to be linear if every two distinct edges share at most one vertex. Given a family $\mathcal{H}$ of linear $r$-uniform hypergraphs,let $Forb_r^L(n,\mathcal{H})$ be the set of linear $r$-uniform hypergraphs on vertex set $[n]$, which does not contain any member from $\mathcal{H}$ as a subgraph. An $r$-uniform linear cycle of length $\ell$, denoted by $C_\ell^r$, is a linear $r$-uniform hypergraph on $(r-1)\ell$ vertices whose edges can be ordered as $\boldsymbol{e}_1,\ldots,\boldsymbol{e}_\ell$ such that $|\boldsymbol{e}_i\cap \boldsymbol{e}_j|=1$ if $j=i\pm 1$ (indices taken modulo $\ell$) and $|\boldsymbol{e}_i\cap \boldsymbol{e}_j|=0$ otherwise. The linear girth of a linear $r$-uniform hypergraph is the smallest integer $\ell$ such that it contains a $C_\ell^r$. Let $Forb_L(n,r,\ell)=Forb_r^L(n,\mathcal{H})$ when $\mathcal{H}=\{C_i^r:\, 3\leqslant i\leqslant \ell\}$, that is, $Forb_L(n,r,\ell)$ is the set of all linear $r$-uniform hypergraphs on $[n]$ with linear girth greater than $\ell$. For integers $r\geqslant 3$ and $\ell\geqslant 4$, Balogh and Li [On the number of linear hypergraphs of large girth, J. Graph Theory, 93(1) (2020), 113-141] showed that $|Forb_L(n,r,\ell)|= 2^{O(n^{1+1/\lfloor \ell/2\rfloor})}$ based on the graph container method. It is natural to obtain $|Forb_L(n,r,\ell)|\geqslant 2^{c\cdot n^{1+1/\ell}}$ for some constant $c$ by probabilistic deletion method. Combined with the known results that $|Forb_L(n,r,3)|= 2^{o (n^{2})}$ and $|Forb_L(n,3,4)|= 2^{\Theta (n^{3/2})}$, by analyzing the random greedy high linear girth linear $r$-uniform hypergraph process, we show $|Forb_L(n,r,\ell)|\geqslant 2^{n^{1+1/(\ell-1)-O(\log\log n/\log n)}}$ for every pair of fixed integers $r,\ell\geqslant 4$, or $r= 3$ and $\ell\geqslant 5$.

math.CO↗

arXiv2025

Efficient Adaptation of Pre-trained Vision Transformer underpinned by Approximately Orthogonal Fine-Tuning Strategy

A prevalent approach in Parameter-Efficient Fine-Tuning (PEFT) of pre-trained Vision Transformers (ViT) involves freezing the majority of the backbone parameters and solely learning low-rank adaptation weight matrices to accommodate downstream tasks. These low-rank matrices are commonly derived through the multiplication structure of down-projection and up-projection matrices, exemplified by methods such as LoRA and Adapter. In this work, we observe an approximate orthogonality among any two row or column vectors within any weight matrix of the backbone parameters; however, this property is absent in the vectors of the down/up-projection matrices. Approximate orthogonality implies a reduction in the upper bound of the model's generalization error, signifying that the model possesses enhanced generalization capability. If the fine-tuned down/up-projection matrices were to exhibit this same property as the pre-trained backbone matrices, could the generalization capability of fine-tuned ViTs be further augmented? To address this question, we propose an Approximately Orthogonal Fine-Tuning (AOFT) strategy for representing the low-rank weight matrices. This strategy employs a single learnable vector to generate a set of approximately orthogonal vectors, which form the down/up-projection matrices, thereby aligning the properties of these matrices with those of the backbone. Extensive experimental results demonstrate that our method achieves competitive performance across a range of downstream image classification tasks, confirming the efficacy of the enhanced generalization capability embedded in the down/up-projection matrices.

cs.CV↗

arXiv2024

Efficient Adaptation of Pre-trained Vision Transformer via Householder Transformation

A common strategy for Parameter-Efficient Fine-Tuning (PEFT) of pre-trained Vision Transformers (ViTs) involves adapting the model to downstream tasks by learning a low-rank adaptation matrix. This matrix is decomposed into a product of down-projection and up-projection matrices, with the bottleneck dimensionality being crucial for reducing the number of learnable parameters, as exemplified by prevalent methods like LoRA and Adapter. However, these low-rank strategies typically employ a fixed bottleneck dimensionality, which limits their flexibility in handling layer-wise variations. To address this limitation, we propose a novel PEFT approach inspired by Singular Value Decomposition (SVD) for representing the adaptation matrix. SVD decomposes a matrix into the product of a left unitary matrix, a diagonal matrix of scaling values, and a right unitary matrix. We utilize Householder transformations to construct orthogonal matrices that efficiently mimic the unitary matrices, requiring only a vector. The diagonal values are learned in a layer-wise manner, allowing them to flexibly capture the unique properties of each layer. This approach enables the generation of adaptation matrices with varying ranks across different layers, providing greater flexibility in adapting pre-trained models. Experiments on standard downstream vision tasks demonstrate that our method achieves promising fine-tuning performance.

cs.CV↗

arXiv2022

A note on the random triadic process

For a fixed integer $r\geqslant 3$, let $\mathbb{H}_r(n,p)$ be a random $r$-uniform hypergraph on the vertex set $[n]$, where each $r$-set is an edge randomly and independently with probability $p$. The random $r$-generalized triadic process starts with a complete bipartite graph $K_{r-2,n-r+2}$ on the same vertex set, chooses two distinct vertices $x$ and $y$ uniformly at random and iteratively adds $\{x,y\}$ as an edge if there is a subset $Z$ with size $r-2$, denoted as $Z=\{z_1,\cdots,z_{r-2}\}$, such that $\{x,z_i\}$ and $\{y,z_i\}$ for $1\leqslant i\leqslant r-2$ are already edges in the graph and $\{x,y, z_1,\cdots,z_{r-2}\}$ is an edge in $\mathbb{H}_r(n,p)$. The random triadic process is an abbreviation for the random $3$-generalized triadic process. Kor\'{a}ndi et al. proved a sharp threshold probability for the propagation of the random triadic process, that is, if $p= cn^{ - \frac 12}$ for some positive constant $c$, with high probability, the triadic process reaches the complete graph when $c> \frac 12$ and stops at $O(n^{\frac 32})$ edges when $c< \frac 12$. In this note, we consider the final size of the random $r$-generalized triadic process when $p=o( n^{- \frac 12}\log^{ \alpha(3-r)} n)$ with a constant $\alpha> \frac 12$. We show that the generated graph of the process essentially behaves like $\mathbb{G}(n,p)$. The final number of added edges in the process, with high probability, equals $ \frac {1}{2}n^{2}p(1\pm o(1))$ provided that $p=\omega(n^{-2})$. The results partially complement the ones on the case of $r=3$.

math.CO↗

arXiv2017

Proof of a conjecture of Kløve on permutation codes under the Chebychev distance

Let $d$ be a positive integer and $x$ a real number. Let $A_{d, x}$ be a $d\times 2d$ matrix with its entries $$ a_{i,j}=\left\{ \begin{array}{ll} x\ \ & \mbox{for} \ 1\leqslant j\leqslant d+1-i, 1\ \ & \mbox{for} \ d+2-i\leqslant j\leqslant d+i, 0\ \ & \mbox{for} \ d+1+i\leqslant j\leqslant 2d. \end{array} \right. $$ Further, let $R_d$ be a set of sequences of integers as follows: $$R_d=\{(ρ_1, ρ_2,\ldots, ρ_d)|1\leqslant ρ_i\leqslant d+i, 1\leqslant i \leqslant d,\ \mbox{and}\ ρ_r\neq ρ_s\ \mbox{for}\ r\neq s\}.$$ and define $$Ω_d(x)=\sum_{ρ\in R_d}a_{1,ρ_1}a_{2, ρ_2}\ldots a_{d,ρ_d}.$$ In order to give a better bound on the size of spheres of permutation codes under the Chebychev distance, Kløve introduced the above function and conjectured that $$Ω_d(x)=\sum_{m=0}^d{d\choose m}(m+1)^d(x-1)^{d-m}.$$ In this paper, we settle down this conjecture positively.

cs.IT↗

arXiv2016

New bounds of permutation codes under Hamming metric and Kendall's $τ$-metric

Permutation codes are widely studied objects due to their numerous applications in various areas, such as power line communications, block ciphers, and the rank modulation scheme for flash memories. Several kinds of metrics are considered for permutation codes according to their specific applications. This paper concerns some improvements on the bounds of permutation codes under Hamming metric and Kendall's $τ$-metric respectively, using mainly a graph coloring approach. Specifically, under Hamming metric, we improve the Gilbert-Varshamov bound asymptotically by a factor $n$, when the minimum Hamming distance $d$ is fixed and the code length $n$ goes to infinity. Under Kendall's $τ$-metric, we narrow the gap between the known lower bounds and upper bounds. Besides, we also obtain some sporadic results under Kendall's $τ$-metric for small parameters.

cs.IT↗

arXiv2011

The Randic index and the diameter of graphs

The {\it Randić index} $R(G)$ of a graph $G$ is defined as the sum of 1/\sqrt{d_ud_v} over all edges $uv$ of $G$, where $d_u$ and $d_v$ are the degrees of vertices $u$ and $v,$ respectively. Let $D(G)$ be the diameter of $G$ when $G$ is connected. Aouchiche-Hansen-Zheng conjectured that among all connected graphs $G$ on $n$ vertices the path $P_n$ achieves the minimum values for both $R(G)/D(G)$ and $R(G)- D(G)$. We prove this conjecture completely. In fact, we prove a stronger theorem: If $G$ is a connected graph, then $R(G)-(1/2)D(G)\geq \sqrt{2}-1$, with equality if and only if $G$ is a path with at least three vertices.

math.CO↗