arXiv subjects

Qixuan Zhou

Publications and source records attributed to Qixuan Zhou.

10 recordsLinked to original sources

arXiv2026

UCPO: Uncertainty-Aware Policy Optimization

The key to building trustworthy large language models (LLMs) lies in endowing them with inherent uncertainty expression capabilities, thereby mitigating overconfident errors in high-stakes applications. However, existing RL paradigms such as GRPO often suffer from Advantage Bias due to binary decision spaces and static uncertainty rewards, inducing either excessive conservatism or overconfidence. To tackle this challenge, this paper unveils the root causes of reward hacking and overconfidence in current RL paradigms incorporating uncertainty-based rewards, based on which we propose the UnCertainty-Aware Policy Optimization (UCPO) framework. UCPO employs Ternary Advantage Decoupling to separate and independently normalize deterministic and uncertain rollouts, thereby eliminating advantage bias. Furthermore, a Dynamic Uncertainty Reward Adjustment mechanism adapts uncertainty weights in real-time according to model evolution and instance difficulty. Experimental results in mathematical reasoning and general tasks demonstrate that UCPO effectively resolves the reward imbalance, significantly improving the reliability of the model beyond their knowledge boundaries.

cs.AI↗

arXiv2026

Fourth-harmonic UV light generation in integrated silicon nitride microresonators

Integrated silicon nitride (Si3N4) resonators have emerged as a leading platform for nonlinear photonics, yet generating light at wavelength in the ultraviolet (UV) has remained elusive in single-resonator systems. Here we report the first observation of fourth-harmonic generation reaching the blue and ultraviolet spectral regions in an integrated Si3N4 microring resonator. We systematically investigate the input-power dependence of the wavelength ranges supporting second-, third-, and fourth-harmonic generation, and study the input-power-dependent variation of the circulating fourth-harmonic signal in the UV. These results extend the operational bandwidth of integrated Si3N4 nonlinear photonic platforms to the lower edge of the material transparency window, enabling on-chip UV frequency conversion. Near-ultraviolet generation around 400 nm will enable on-chip excitation of defect-based quantum emitters in hexagonal boron nitride, enhance Raman spectroscopy through increased scattering cross-sections at shorter wavelengths, and support compact fluorescence-based bio-imaging platforms exploiting intrinsic cellular fluorophores.

physics.optics↗

arXiv2026

SSBE-PINN: A Sobolev Boundary Scheme Boosting Stability and Accuracy in Elliptic/Parabolic PDE Learning

Physics-Informed Neural Networks (PINNs) have emerged as a powerful framework for solving partial differential equations (PDEs), yet they often fail to achieve accurate convergence in the H1 norm, especially in the presence of boundary approximation errors. In this work, we propose a novel method called Sobolev-Stable Boundary Enforcement (SSBE), which redefines the boundary loss using Sobolev norms to incorporate boundary regularity directly into the training process. We provide rigorous theoretical analysis demonstrating that SSBE ensures bounded H1 error via a stability guarantee and derive generalization bounds that characterize its robustness under finite-sample regimes. Extensive numerical experiments on linear and nonlinear PDEs, including Poisson, heat, and elliptic problems, show that SSBE consistently outperforms standard PINNs in terms of both relative L2 and H1 errors, even in high-dimensional settings. The proposed approach offers a principled and practical solution for improving gradient fidelity and overall solution accuracy in neural network based PDE solvers.

math.NA↗

arXiv2025

NGRPO: Negative-enhanced Group Relative Policy Optimization

RLVR has enhanced the reasoning capabilities of Large Language Models (LLMs) across various tasks. However, GRPO, a representative RLVR algorithm, suffers from a critical limitation: when all responses within a group are either entirely correct or entirely incorrect, the model fails to learn from these homogeneous responses. This is particularly problematic for homogeneously incorrect groups, where GRPO's advantage function yields a value of zero, leading to null gradients and the loss of valuable learning signals. To overcome this issue, we propose NGRPO (Negative-enhanced Group Relative Policy Optimization), an algorithm designed to convert homogeneous errors into robust learning signals. First, NGRPO introduces Advantage Calibration. This mechanism hypothesizes the existence of a virtual maximum-reward sample during advantage calculation, thereby altering the mean and variance of rewards within a group and ensuring that the advantages for homogeneously incorrect samples are no longer zero. Second, NGRPO employs Asymmetric Clipping, which relaxes the update magnitude for positive samples while imposing stricter constraints on that of negative samples. This serves to stabilize the exploration pressure introduced by the advantage calibration. Our experiments on Qwen2.5-Math-7B demonstrate that NGRPO significantly outperforms baselines such as PPO, GRPO, DAPO, and PSR-NSR on mathematical benchmarks including MATH500, AMC23, and AIME2025. These results validate NGRPO's ability to learn from homogeneous errors, leading to stable and substantial improvements in mathematical reasoning. Our code is available at https://github.com/nangongrui-ngr/NGRPO.

cs.LG↗

arXiv2024

Quantifying Training Difficulty and Accelerating Convergence in Neural Network-Based PDE Solvers

Neural network-based methods have emerged as powerful tools for solving partial differential equations (PDEs) in scientific and engineering applications, particularly when handling complex domains or incorporating empirical data. These methods leverage neural networks as basis functions to approximate PDE solutions. However, training such networks can be challenging, often resulting in limited accuracy. In this paper, we investigate the training dynamics of neural network-based PDE solvers with a focus on the impact of initialization techniques. We assess training difficulty by analyzing the eigenvalue distribution of the kernel and apply the concept of effective rank to quantify this difficulty, where a larger effective rank correlates with faster convergence of the training error. Building upon this, we discover through theoretical analysis and numerical experiments that two initialization techniques, partition of unity (PoU) and variance scaling (VS), enhance the effective rank, thereby accelerating the convergence of training error. Furthermore, comprehensive experiments using popular PDE-solving frameworks, such as PINN, Deep Ritz, and the operator learning framework DeepOnet, confirm that these initialization techniques consistently speed up convergence, in line with our theoretical findings.

math.NA↗

arXiv2024

Demystifying Lazy Training of Neural Networks from a Macroscopic Viewpoint

In this paper, we advance the understanding of neural network training dynamics by examining the intricate interplay of various factors introduced by weight parameters in the initialization process. Motivated by the foundational work of Luo et al. (J. Mach. Learn. Res., Vol. 22, Iss. 1, No. 71, pp 3327-3373), we explore the gradient descent dynamics of neural networks through the lens of macroscopic limits, where we analyze its behavior as width $m$ tends to infinity. Our study presents a unified approach with refined techniques designed for multi-layer fully connected neural networks, which can be readily extended to other neural network architectures. Our investigation reveals that gradient descent can rapidly drive deep neural networks to zero training loss, irrespective of the specific initialization schemes employed by weight parameters, provided that the initial scale of the output function $κ$ surpasses a certain threshold. This regime, characterized as the theta-lazy area, accentuates the predominant influence of the initial scale $κ$ over other factors on the training behavior of neural networks. Furthermore, our approach draws inspiration from the Neural Tangent Kernel (NTK) paradigm, and we expand its applicability. While NTK typically assumes that $\lim_{m\to\infty}\frac{\log κ}{\log m}=\frac{1}{2}$, and imposes each weight parameters to scale by the factor $\frac{1}{\sqrt{m}}$, in our theta-lazy regime, we discard the factor and relax the conditions to $\lim_{m\to\infty}\frac{\log κ}{\log m}>0$. Similar to NTK, the behavior of overparameterized neural networks within the theta-lazy regime trained by gradient descent can be effectively described by a specific kernel. Through rigorous analysis, our investigation illuminates the pivotal role of $κ$ in governing the training dynamics of neural networks.

cs.LG↗

arXiv2024

A priori Estimates for Deep Residual Network in Continuous-time Reinforcement Learning

Deep reinforcement learning excels in numerous large-scale practical applications. However, existing performance analyses ignores the unique characteristics of continuous-time control problems, is unable to directly estimate the generalization error of the Bellman optimal loss and require a boundedness assumption. Our work focuses on continuous-time control problems and proposes a method that is applicable to all such problems where the transition function satisfies semi-group and Lipschitz properties. Under this method, we can directly analyze the \emph{a priori} generalization error of the Bellman optimal loss. The core of this method lies in two transformations of the loss function. To complete the transformation, we propose a decomposition method for the maximum operator. Additionally, this analysis method does not require a boundedness assumption. Finally, we obtain an \emph{a priori} generalization error without the curse of dimensionality.

cs.LG↗

arXiv2023

On Residual Minimization for PDEs: Failure of PINN, Modified Equation, and Implicit Bias

As a popular and easy-to-implement machine learning method for solving differential equations, the physics-informed neural network (PINN) sometimes may fail and find poor solutions which bias against the exact ones. In this paper, we establish a framework of modified equation to explain the failure phenomenon and characterize the implicit bias of a general residual minimization (RM) method. We provide a simple way to derive the modified equation which models the numerical solution obtained by RM methods. Next, we show the modified solution deviates from the original exact solution. The proof uses a by-product of this paper, that is, a necessary and sufficient condition on characterizing the singularity of the coefficients. This equivalent condition can be extended to other types of equations in the future. Finally, we prove, as a complete characterization of the implicit bias, that RM method implicitly biases the numerical solution against the exact solution and towards a modified solution. In this work, we focus on elliptic equations with discontinuous coefficients, but our approach can be extended to other types of equations and our understanding of the implicit bias may shed light on further development of deep learning based methods for solving equations.

math.AP↗

arXiv2022

Towards Understanding the Condensation of Neural Networks at Initial Training

Empirical works show that for ReLU neural networks (NNs) with small initialization, input weights of hidden neurons (the input weight of a hidden neuron consists of the weight from its input layer to the hidden neuron and its bias term) condense on isolated orientations. The condensation dynamics implies that the training implicitly regularizes a NN towards one with a much smaller effective size. In this work, we illustrate the formation of the condensation in multi-layer fully connected NNs and show that the maximal number of condensed orientations in the initial training stage is twice the multiplicity of the activation function, where "multiplicity" indicates the multiple roots of activation function at origin. Our theoretical analysis confirms experiments for two cases, one is for the activation function of multiplicity one with arbitrary dimension input, which contains many common activation functions, and the other is for the layer with one-dimensional input and arbitrary multiplicity. This work makes a step towards understanding how small initialization leads NNs to condensation at the initial training stage.

cs.LG↗

arXiv2022

Empirical Phase Diagram for Three-layer Neural Networks with Infinite Width

Substantial work indicates that the dynamics of neural networks (NNs) is closely related to their initialization of parameters. Inspired by the phase diagram for two-layer ReLU NNs with infinite width (Luo et al., 2021), we make a step towards drawing a phase diagram for three-layer ReLU NNs with infinite width. First, we derive a normalized gradient flow for three-layer ReLU NNs and obtain two key independent quantities to distinguish different dynamical regimes for common initialization methods. With carefully designed experiments and a large computation cost, for both synthetic datasets and real datasets, we find that the dynamics of each layer also could be divided into a linear regime and a condensed regime, separated by a critical regime. The criteria is the relative change of input weights (the input weight of a hidden neuron consists of the weight from its input layer to the hidden neuron and its bias term) as the width approaches infinity during the training, which tends to $0$, $+\infty$ and $O(1)$, respectively. In addition, we also demonstrate that different layers can lie in different dynamical regimes in a training process within a deep NN. In the condensed regime, we also observe the condensation of weights in isolated orientations with low complexity. Through experiments under three-layer condition, our phase diagram suggests a complicated dynamical regimes consisting of three possible regimes, together with their mixture, for deep NNs and provides a guidance for studying deep NNs in different initialization regimes, which reveals the possibility of completely different dynamics emerging within a deep NN for its different layers.

cs.LG↗