arXiv subjects

Yaoyu Zhang

Publications and source records attributed to Yaoyu Zhang.

At least 19 recordsLinked to original sources

arXiv2026

Uplink Positioning for PASS in Multipath Environments

Pinching-antenna systems (PASS) enhance wireless propagation by activating or placing pinching antennas (PAs) near users. Therefore, accurate uplink positioning is essential for efficient communication. In this paper, an uplink multi-carrier positioning framework is established for PASS in multipath environments. Matrix pencil (MP)-based and low-complexity Rank-1 ranging algorithms are proposed to estimate the distances between the PAs and the user. For the MP-based ranging algorithm, the line-of-sight (LoS) component is separated from non-line-of-sight components by exploiting the shift-invariance property of the Hankel matrix, thereby enabling accurate distance estimation. For the Rank-1 ranging algorithm, the dominant LoS delay is directly isolated through truncated singular value decomposition, thereby avoiding matrix inversions. Subsequently, a two-stage weighted nonlinear least-squares (WNLS) positioning algorithm is designed to estimate the three-dimensional user position. To gain further insights, a comprehensive theoretical performance analysis of the proposed ranging and positioning algorithms is conducted. The closed-form ranging variances and position error bound (PEB) are derived to reveal the error propagation mechanism. Numerical results demonstrate that: i) The MP-based algorithm achieves higher accuracy and robustness than the Rank-1-based algorithm, while the Rank-1-based algorithm has lower computational complexity. ii) The positioning error of the MP-based algorithm follows the same trend as the derived PEB, whereas the Rank-1 algorithm exhibits an error floor due to multipath bias. iii) The positioning accuracy of the MP algorithm improves as the number of subcarriers increases.

eess.SP↗

arXiv2026

Gradient Flow Dynamics and Implicit Bias of Diagonal Linear Networks under Infinitesimal Initialization

We study the gradient flow dynamics of diagonal linear networks for regression tasks under infinitesimal initialization. Extending Theorem 1 from Pesme & Flammarion (2023), we generalize the analysis to both deep diagonal linear networks and a broader class of two-layer diagonal linear networks (as defined in Definition 4.1). Specifically, we demonstrate that the training trajectories of these models can be equivalently characterized by the proposed Algorithm 1. We further prove that this algorithm converges to the solution of a modified $ \mathcal{l}_1 $ norm minimization problem. As a result, we establish that the implicit bias of both network architectures corresponds to a modified $ \mathcal{l}_1 $ norm in the regime of infinitesimal initialization. Additionally, we provide insights into the underlying mechanisms governing these dynamics by identifying the Structural Invariant Manifold (SIM) (Zhao et al., 2026) as the key geometric structure that shapes the learning process.

cs.LG↗

arXiv2026

Theory of the Frequency Principle for General Deep Neural Networks

Along with fruitful applications of Deep Neural Networks (DNNs) to realistic problems, recently, some empirical studies of DNNs reported a universal phenomenon of Frequency Principle (F-Principle): a DNN tends to learn a target function from low to high frequencies during the training. The F-Principle has been very useful in providing both qualitative and quantitative understandings of DNNs. In this paper, we rigorously investigate the F-Principle for the training dynamics of a general DNN at three stages: initial stage, intermediate stage, and final stage. For each stage, a theorem is provided in terms of proper quantities characterizing the F-Principle. Our results are general in the sense that they work for multilayer networks with general activation functions, population densities of data, and a large class of loss functions. Our work lays a theoretical foundation of the F-Principle for a better understanding of the training process of DNNs.

cs.LG↗

arXiv2026

Integrated Positioning and Communications for PASS: A Robust Approach

The pinching-antenna systems (PASS), which dynamically activate and relocate the pinching-antennas (PAs) along the dielectric waveguide, offer unprecedented potential for integrated positioning and communication. The multi-waveguide-based uplink positioning approaches for indoor environments are first proposed in this paper, and the downlink communication performance is analyzed. Two possible scenarios, multi-waveguide single-PA (MWSP) and multi-waveguide multi-PA (MWMP), are considered under the assumptions of line-of-sight channels and a single, stationary user. For the MWSP scenario, the received signal strength indication (RSSI)-based ranging method and the MWSP-based least square (LS) positioning algorithm are developed. To gain deeper insights, a comprehensive error analysis of the LS positioning algorithm is conducted. Subsequently, for the MWMP scenario, the closed-form expression of the superposed signal is derived. According to the signal power, the MWMP-based grid search algorithm is proposed and the estimation error of proposed algorithm is analyzed. Then, based on the user's positioning result, the PAs are relocated to provide downlink communication service, and the achievable data rate of MWSP and MWMP scenarios are analyzed. Numerical results validate the correctness of our analysis, which show that: i) For the MWSP scenario, a smaller geometric dilution of precision (GDoP) leads to a lower average positioning error. Furthermore, even when the GDoP is large, the regions where the distances to PAs are nearly equal achieve the best accuracy. ii) For the MWMP scenario, non-parallel waveguide deployment improves positioning accuracy, although errors increase with the number of PAs. iii) The noise has a serious double-impact on data rate. There is a trade-off between positioning accuracy and communication performance.

eess.SP↗

arXiv2026

Multi-/Uni-Cast Non-Orthogonal Multiple Access-Based INAC

With the rapid development of satellite communication and navigation, there is an urgent need to integrate both technologies to achieve reliable communication and precise navigation services within the same satellite system. By combining multi-/uni-cast (MUC) and non-orthogonal multiple access (NOMA) technologies, we propose a novel MUC-NOMA-based integrated navigation and communication (INAC) signal structure, in which the navigation and communication signals share a common pseudo noise (PN) sequence, thereby integrating satellite communication and navigation at the signal level. According to different power allocation strategies, two scenarios are defined: multi-cast-oriented (MO-) INAC and uni-cast-oriented (UO-) INAC, where a greater portion of power is assigned to either the multi-cast or the uni-cast signal, respectively. To mitigate co-channel interference, we employ successive interference cancellation (SIC) at the receiver and design a signal processing algorithm for the proposed INAC signal. Then, closed-form expressions are subsequently derived for the bit error rates (BER) of both the navigation and communication signals, along with the positioning accuracy of the navigation signal. To gain further insights, the impacts of power allocation factors and communication rates are evaluated. Our analysis results show that: i) In the MO-INAC scenario, the positioning and BER performance of navigation signal are excellent when more power is assigned to the multi-cast signal; ii) In the UO-INAC scenario, interference in the shared resources is reduced when more power is assigned to the uni-cast signal; iii) The ranging accuracy decreases as the communication data rate increases. Numerical results confirm the superior BER and positioning accuracy of the MO-INAC scenario for MEO satellites.

eess.SP↗

arXiv2026

Adaptive Preconditioners Trigger Loss Spikes in Adam

Loss spikes commonly emerge during neural network training with the Adam optimizer across diverse architectures and scales, yet their underlying mechanism remains elusive. While previous explanations attribute these phenomena to sharper loss landscapes at lower loss, we show that landscape geometry alone is insufficient to explain the phenomenon. In this work, we pinpoint the root cause in the internal dynamics of Adam's second moment estimator. We identify a critical ``decoupling'' mechanism where the adaptive preconditioner $v_t$ fails to track the instantaneous squared gradients $g_t^2$, causing the adaptive mechanism to effectively fail. This decoupling allows the preconditioner to decay autonomously despite rising gradients, which pushes the maximum eigenvalue of the preconditioned Hessian beyond the stability threshold $2/η$ for sustained periods, manifesting as dramatic loss spikes. Through a quadratic approximation analysis, we theoretically and experimentally characterize five distinct stages of spike evolution and propose a predictor for anticipating spikes based on gradient-directional curvature. We empirically find that the proposed loss spike mechanism, although derived from simplified models, generalizes well to practical scenarios ranging from small neural networks to large-scale Transformers.

cs.LG↗

arXiv2026

Towards Understanding Adam Convergence on Highly Degenerate Polynomials

Adam is a widely used optimization algorithm in deep learning, yet the specific class of objective functions where it exhibits inherent advantages remains underexplored. Unlike prior studies requiring external schedulers and $β_2$ near 1 for convergence, this work investigates the ``natural'' auto-convergence properties of Adam. We identify a class of highly degenerate polynomials where Adam converges automatically without additional schedulers. Specifically, we derive theoretical conditions for local asymptotic stability on degenerate polynomials and demonstrate strong alignment between theoretical bounds and experimental results. We prove that Adam achieves local linear convergence on these degenerate functions, significantly outperforming the sub-linear convergence of Gradient Descent and Momentum. This acceleration stems from a decoupling mechanism between the second moment $v_t$ and squared gradient $g_t^2$, which exponentially amplifies the effective learning rate. Finally, we characterize Adam's hyperparameter phase diagram, identifying three distinct behavioral regimes: stable convergence, spikes, and SignGD-like oscillation.

cs.LG↗

arXiv2026

Architecture Induces Structural Invariant Manifolds of Neural Network Training Dynamics

While architecture is recognized as key to the performance of deep neural networks, its precise effect on training dynamics has been unclear due to the confounding influence of data and loss functions. This paper proposed an analytic framework based on the geometric control theory to characterize the dynamical properties intrinsic to a model's parameterization. We prove that the Structural Invariant Manifolds (SIMs) of an analytic model $F(\mathbfθ)(\mathbf{x})$--submanifolds that confine gradient flow trajectories independent of data and loss--are unions of orbits of the vector field family $\{\nabla_{\mathbfθ} F(\cdot)(\mathbf{x})\mid\mathbf{x}\in\mathbb{R}^d\}$. We then prove that a model's symmetry, e.g., permutation symmetry for neural networks, induces SIMs. Applying this, we characterize the hierarchy of symmetry-induced SIMs in fully-connected networks, where dynamics exhibit neuron condensation and equivalence to reduced-width networks. For two-layer networks, we prove all SIMs are symmetry-induced, closing the gap between known symmetries and all possible invariants. Overall, by establishing the framework for analyzing SIMs induced by architecture, our work paves the way for a deeper analysis of neural network training dynamics and generalization in the near future.

math.DS↗

arXiv2026

An overview of condensation phenomenon in deep learning

In this paper, we provide an overview of a common phenomenon, condensation, observed during the nonlinear training of neural networks: During the nonlinear training of neural networks, neurons in the same layer tend to condense into groups with similar outputs. Empirical observations suggest that the number of condensed clusters of neurons in the same layer typically increases monotonically as training progresses. Neural networks with small weight initializations or Dropout optimization can facilitate this condensation process. We also examine the underlying mechanisms of condensation from the perspectives of training dynamics and the structure of the loss landscape. The condensation phenomenon offers valuable insights into the generalization abilities of neural networks and correlates to stronger reasoning abilities in transformer-based language models.

cs.LG↗

arXiv2026

Determinism in the Undetermined: Deterministic Output in Charge-Conserving Continuous-Time Neuromorphic Systems with Temporal Stochasticity

Achieving deterministic computation results in asynchronous neuromorphic systems remains a fundamental challenge due to the inherent temporal stochasticity of continuous-time hardware. To address this, we develop a unified continuous-time framework for spiking neural networks (SNNs) that couples the Law of Charge Conservation with minimal neuron-level constraints. This integration ensures that the terminal state depends solely on the aggregate input charge, providing a unique cumulated output invariant to temporal stochasticity. We prove that this mapping is strictly invariant to spike timing in acyclic networks, whereas recurrent connectivity can introduce temporal sensitivity. Furthermore, we establish an exact representational correspondence between these charge-conserving SNNs and quantized artificial neural networks, bridging the gap between static deep learning and event-driven dynamics without approximation errors. These results establish a rigorous theoretical basis for designing continuous-time neuromorphic systems that harness the efficiency of asynchronous processing while maintaining algorithmic determinism.

cs.LG↗

arXiv2025

Embedding principle of homogeneous neural network for classification problem

In this paper, we study the Karush-Kuhn-Tucker (KKT) points of the associated maximum-margin problem in homogeneous neural networks, including fully-connected and convolutional neural networks. In particular, We investigates the relationship between such KKT points across networks of different widths generated. We introduce and formalize the \textbf{KKT point embedding principle}, establishing that KKT points of a homogeneous network's max-margin problem ($P_Φ$) can be embedded into the KKT points of a larger network's problem ($P_{\tildeΦ}$) via specific linear isometric transformations. We rigorously prove this principle holds for neuron splitting in fully-connected networks and channel splitting in convolutional neural networks. Furthermore, we connect this static embedding to the dynamics of gradient flow training with smooth losses. We demonstrate that trajectories initiated from appropriately mapped points remain mapped throughout training and that the resulting $ω$-limit sets of directions are correspondingly mapped, thereby preserving the alignment with KKT directions dynamically when directional convergence occurs. We conduct several experiments to justify that trajectories are preserved. Our findings offer insights into the effects of network width, parameter redundancy, and the structural connections between solutions found via optimization in homogeneous networks of varying sizes.

cs.LG↗

arXiv2025

Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism

Large language models have consistently struggled with complex reasoning tasks, such as mathematical problem-solving. Investigating the internal reasoning mechanisms of these models can help us design better model architectures and training strategies, ultimately enhancing their reasoning capability. In this study, we constructed a symbolic multi-step reasoning task to investigate the information propagation mechanisms in Transformer models when solving the task through direct answering and Chain-of-Thought (CoT) reasoning. We introduced the concept of buffer mechanism: the model stores various information in distinct buffers and selectively extracts it through the query-key matrix. We proposed a random matrix-based algorithm to enhance the model's reasoning ability. This algorithm introduces only 132 trainable parameters, yet leads to significant performance improvements on 7 multi-step reasoning datasets, including PrOntoQA, LogicAsker, and LogicInference. These findings provide new insights into understanding the large language models.

cs.AI↗

arXiv2025

Solving multiscale dynamical systems by deep learning

Multiscale dynamical systems, modeled by high-dimensional stiff ordinary differential equations (ODEs) with wide-ranging characteristic timescales, arise across diverse fields of science and engineering, but their numerical solvers often encounter severe efficiency bottlenecks. This paper introduces a novel DeePODE method, which consists of an Evolutionary Monte Carlo Sampling method (EMCS) and an efficient end-to-end deep neural network (DNN) to predict multiscale dynamical systems. We validate this finding across dynamical systems from ecological systems to reactive flows, including a predator-prey model, a power system oscillation, a battery electrolyte thermal runaway, and turbulent reaction-diffusion systems with complex chemical kinetics. The method demonstrates robust generalization capabilities, allowing pre-trained DNN models to accurately predict the behavior in previously unseen scenarios, largely due to the delicately constructed dataset. While theoretical guarantees remain to be established, empirical evidence shows that DeePODE achieves the accuracy of implicit numerical schemes while maintaining the computational efficiency of explicit schemes. This work underscores the crucial relationship between training data distribution and neural network generalization performance. This work demonstrates the potential of deep learning approaches in modeling complex dynamical systems across scientific and engineering domains.

math.NA↗

arXiv2025

Pinching-Antenna Systems (PASS)-based Indoor Positioning

Pinching antenna (PA), a flexible waveguide integrated with dielectric particles, intelligently reconstructs line-of-sight channels. Utilizing its geometric deterministic model and meter-level reconstruction, PA systems (PASS) are applied to uplink indoor positioning. In this paper, the uplink positioning system model for PASS is firstly proposed. A PASS-based received signal strength indication (RSSI) method is proposed to measure the distance from the users to each PA, which is efficient and suitable for PASS. PASS-based weighted least squares (WLS) algorithm is designed to calculate the two-dimensional coordinates of the users. Several critical observations can be drawn from our results: i) More PAs on the waveguide improves the positioning accuracy and robustness. ii) When the number of PAs exceeds a certain threshold, the performance gain becomes marginal. iii) User locations between and near PAs yield superior positioning accuracy.

eess.SY↗

arXiv2025

A Contrastive Diffusion-based Network (CDNet) for Time Series Classification

Deep learning models are widely used for time series classification (TSC) due to their scalability and efficiency. However, their performance degrades under challenging data conditions such as class similarity, multimodal distributions, and noise. To address these limitations, we propose CDNet, a Contrastive Diffusion-based Network that enhances existing classifiers by generating informative positive and negative samples via a learned diffusion process. Unlike traditional diffusion models that denoise individual samples, CDNet learns transitions between samples--both within and across classes--through convolutional approximations of reverse diffusion steps. We introduce a theoretically grounded CNN-based mechanism to enable both denoising and mode coverage, and incorporate an uncertainty-weighted composite loss for robust training. Extensive experiments on the UCR Archive and simulated datasets demonstrate that CDNet significantly improves state-of-the-art (SOTA) deep learning classifiers, particularly under noisy, similar, and multimodal conditions.

cs.LG↗

arXiv2025

Scalable Complexity Control Facilitates Reasoning Ability of LLMs

The reasoning ability of large language models (LLMs) has been rapidly advancing in recent years, attracting interest in more fundamental approaches that can reliably enhance their generalizability. This work demonstrates that model complexity control, conveniently implementable by adjusting the initialization rate and weight decay coefficient, improves the scaling law of LLMs consistently over varying model sizes and data sizes. This gain is further illustrated by comparing the benchmark performance of 2.4B models pretrained on 1T tokens with different complexity hyperparameters. Instead of fixing the initialization std, we found that a constant initialization rate (the exponent of std) enables the scaling law to descend faster in both model and data sizes. These results indicate that complexity control is a promising direction for the continual advancement of LLMs.

cs.LG↗

arXiv2025

Uncovering Critical Sets of Deep Neural Networks via Sample-Independent Critical Lifting

This paper investigates the sample dependence of critical points for neural networks. We introduce a sample-independent critical lifting operator that associates a parameter of one network with a set of parameters of another, thus defining sample-dependent and sample-independent lifted critical points. We then show by example that previously studied critical embeddings do not capture all sample-independent lifted critical points. Finally, we demonstrate the existence of sample-dependent lifted critical points for sufficiently large sample sizes and prove that saddles appear among them.

cs.LG↗

arXiv2025

Connectivity Shapes Implicit Regularization in Matrix Factorization Models for Matrix Completion

Matrix factorization models have been extensively studied as a valuable test-bed for understanding the implicit biases of overparameterized models. Although both low nuclear norm and low rank regularization have been studied for these models, a unified understanding of when, how, and why they achieve different implicit regularization effects remains elusive. In this work, we systematically investigate the implicit regularization of matrix factorization for solving matrix completion problems. We empirically discover that the connectivity of observed data plays a crucial role in the implicit bias, with a transition from low nuclear norm to low rank as data shifts from disconnected to connected with increased observations. We identify a hierarchy of intrinsic invariant manifolds in the loss landscape that guide the training trajectory to evolve from low-rank to higher-rank solutions. Based on this finding, we theoretically characterize the training trajectory as following the hierarchical invariant manifold traversal process, generalizing the characterization of Li et al. (2020) to include the disconnected case. Furthermore, we establish conditions that guarantee minimum nuclear norm, closely aligning with our experimental findings, and we provide a dynamics characterization condition for ensuring minimum rank. Our work reveals the intricate interplay between data connectivity, training dynamics, and implicit regularization in matrix factorization models.

cs.LG↗