arXiv subjects

Zirui Zhang

Publications and source records attributed to Zirui Zhang.

34 records · Page 2Linked to original sources

arXiv2025

Reactivation: Empirical NTK Dynamics Under Task Shifts

The Neural Tangent Kernel (NTK) offers a powerful tool to study the functional dynamics of neural networks. In the so-called lazy, or kernel regime, the NTK remains static during training and the network function is linear in the static neural tangents feature space. The evolution of the NTK during training is necessary for feature learning, a key driver of deep learning success. The study of the NTK dynamics has led to several critical discoveries in recent years, in generalization and scaling behaviours. However, this body of work has been limited to the single task setting, where the data distribution is assumed constant over time. In this work, we present a comprehensive empirical analysis of NTK dynamics in continual learning, where the data distribution shifts over time. Our findings highlight continual learning as a rich and underutilized testbed for probing the dynamics of neural training. At the same time, they challenge the validity of static-kernel approximations in theoretical treatments of continual learning, even at large scale.

cs.LG↗

arXiv2025

Yi-Lightning Technical Report

This technical report presents Yi-Lightning, our latest flagship large language model (LLM). It achieves exceptional performance, ranking 6th overall on Chatbot Arena, with particularly strong results (2nd to 4th place) in specialized categories including Chinese, Math, Coding, and Hard Prompts. Yi-Lightning leverages an enhanced Mixture-of-Experts (MoE) architecture, featuring advanced expert segmentation and routing mechanisms coupled with optimized KV-caching techniques. Our development process encompasses comprehensive pre-training, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF), where we devise deliberate strategies for multi-stage training, synthetic data construction, and reward modeling. Furthermore, we implement RAISE (Responsible AI Safety Engine), a four-component framework to address safety issues across pre-training, post-training, and serving phases. Empowered by our scalable super-computing infrastructure, all these innovations substantially reduce training, deployment and inference costs while maintaining high-performance standards. With further evaluations on public academic benchmarks, Yi-Lightning demonstrates competitive performance against top-tier LLMs, while we observe a notable disparity between traditional, static benchmark results and real-world, dynamic human preferences. This observation prompts a critical reassessment of conventional benchmarks' utility in guiding the development of more intelligent and powerful AI systems for practical applications. Yi-Lightning is now available through our developer platform at https://platform.lingyiwanwu.com.

cs.CL↗

arXiv2024

I Can Hear You: Selective Robust Training for Deepfake Audio Detection

Recent advances in AI-generated voices have intensified the challenge of detecting deepfake audio, posing risks for scams and the spread of disinformation. To tackle this issue, we establish the largest public voice dataset to date, named DeepFakeVox-HQ, comprising 1.3 million samples, including 270,000 high-quality deepfake samples from 14 diverse sources. Despite previously reported high accuracy, existing deepfake voice detectors struggle with our diversely collected dataset, and their detection success rates drop even further under realistic corruptions and adversarial attacks. We conduct a holistic investigation into factors that enhance model robustness and show that incorporating a diversified set of voice augmentations is beneficial. Moreover, we find that the best detection models often rely on high-frequency features, which are imperceptible to humans and can be easily manipulated by an attacker. To address this, we propose the F-SAT: Frequency-Selective Adversarial Training method focusing on high-frequency components. Empirical results demonstrate that using our training dataset boosts baseline model performance (without robust training) by 33%, and our robust training further improves accuracy by 7.7% on clean samples and by 29.3% on corrupted and attacked samples, over the state-of-the-art RawNet3 model.

cs.SD↗

arXiv2024

BA-Net: Bridge Attention in Deep Neural Networks

Attention mechanisms, particularly channel attention, have become highly influential in numerous computer vision tasks. Despite their effectiveness, many existing methods primarily focus on optimizing performance through complex attention modules applied at individual convolutional layers, often overlooking the synergistic interactions that can occur across multiple layers. In response to this gap, we introduce bridge attention, a novel approach designed to facilitate more effective integration and information flow between different convolutional layers. Our work extends the original bridge attention model (BAv1) by introducing an adaptive selection operator, which reduces information redundancy and optimizes the overall information exchange. This enhancement results in the development of BAv2, which achieves substantial performance improvements in the ImageNet classification task, obtaining Top-1 accuracies of 80.49% and 81.75% when using ResNet50 and ResNet101 as backbone networks, respectively. These results surpass the retrained baselines by 1.61% and 0.77%, respectively. Furthermore, BAv2 outperforms other existing channel attention techniques, such as the classical SENet101, exceeding its retrained performance by 0.52% Additionally, integrating BAv2 into advanced convolutional networks and vision transformers has led to significant gains in performance across a wide range of computer vision tasks, underscoring its broad applicability.

cs.CV↗

arXiv2024

DSDFormer: An Innovative Transformer-Mamba Framework for Robust High-Precision Driver Distraction Identification

Driver distraction remains a leading cause of traffic accidents, posing a critical threat to road safety globally. As intelligent transportation systems evolve, accurate and real-time identification of driver distraction has become essential. However, existing methods struggle to capture both global contextual and fine-grained local features while contending with noisy labels in training datasets. To address these challenges, we propose DSDFormer, a novel framework that integrates the strengths of Transformer and Mamba architectures through a Dual State Domain Attention (DSDA) mechanism, enabling a balance between long-range dependencies and detailed feature extraction for robust driver behavior recognition. Additionally, we introduce Temporal Reasoning Confident Learning (TRCL), an unsupervised approach that refines noisy labels by leveraging spatiotemporal correlations in video sequences. Our model achieves state-of-the-art performance on the AUC-V1, AUC-V2, and 100-Driver datasets and demonstrates real-time processing efficiency on the NVIDIA Jetson AGX Orin platform. Extensive experimental results confirm that DSDFormer and TRCL significantly improve both the accuracy and robustness of driver distraction detection, offering a scalable solution to enhance road safety.

cs.CV↗

arXiv2024

CMBFSCNN: Cosmic Microwave Background Polarization Foreground Subtraction with Convolutional Neural Network

In our previous study, we introduced a machine-learning technique, namely CMBFSCNN, for the removal of foreground contamination in cosmic microwave background (CMB) polarization data. This method was successfully employed on actual observational data from the Planck mission. In this study, we extend our investigation by considering the CMB lensing effect in simulated data and utilizing the CMBFSCNN approach to recover the CMB lensing B-mode power spectrum from multi-frequency observational maps. Our method is first applied to simulated data with the performance of CMB-S4 experiment. We achieve reliable recovery of the noisy CMB Q (or U) maps with a mean absolute difference of $0.016\pm0.008\ μ$K (or $0.021\pm0.002\ μ$K) for the CMB-S4 experiment. To address the residual instrumental noise in the foreground-cleaned map, we employ a "half-split maps" approach, where the entire dataset is divided into two segments sharing the same sky signal but having uncorrelated noise. Using cross-correlation techniques between two recovered half-split maps, we effectively reduce instrumental noise effects at the power spectrum level. As a result, we achieve precise recovery of the CMB EE and lensing B-mode power spectra. Furthermore, we also extend our pipeline to full-sky simulated data with the performance of LiteBIRD experiment. As expected, various foregrounds are cleanly removed from the foreground contamination observational maps, and recovered EE and lensing B-mode power spectra exhibit excellent agreement with the true results. Finally, we discuss the dependency of our method on the foreground models.

astro-ph.CO↗

arXiv2024

A Constrained NILC method for CMB B mode observations

The Internal Linear Combination (ILC) method is commonly employed to extract the cosmic microwave background (CMB) signal from multi-frequency observation maps. However, the performance of the ILC method tends to degrade when the signal-to-noise ratio (SNR) is relatively low, particularly when measuring the primordial $B$-modes to detect the primordial gravitational waves. To address this issue, an enhanced version of the ILC method, known as constrained ILC, is proposed. This method is designed to be more suitable for situations with low signal-to-noise ratio (SNR) by incorporating additional prior foreground information. In our study, we have modified the constraint Needlet ILC method and successfully improved its performance at low SNR. We illustrate our methods using mock data generated from the combination of WMAP, Planck and a ground-based experiment in the northern hemisphere, and the chosen noise level for the ground-based experiment are very conservative which can be easily achieved in the very near future. The results show that the level of foreground residual can be well controlled. In comparison to the standard NILC method, which introduces a bias to the tensor-to-scalar ratio ($r$) of approximately $0.05$, the constrained NILC method exhibits a significantly reduced bias of only around $5\times10^{-3}$ towards $r$ which is much smaller than the statistical error.

astro-ph.CO↗

arXiv2024

Optimal map-making with singularities

In this work, we investigate the optimal map-making technique for the linear system $d=Ax+n$ while carefully taking into account singularities that may come from either the covariance matrix $C = \langle nn^t \rangle$ or the main matrix $A$. We first describe the general optimal solution, which is quite complex, and then use the modified pseudo inverse to create a near-optimal solution, which is simple, robust, and can significantly alleviate the unwanted noise amplification during map-making. The effectiveness of the nearly optimal solution is then compared to that of the naive co-adding solution and the standard pseudo inverse solution, showing noticeable improvements. Interestingly, all one needs to get the near-optimal solution with singularity is just a tiny change to the classical solution, which is designed for the case without singularity.

astro-ph.CO↗

arXiv2024

Dark matter search with CMB: a study of foregrounds

The energy injected from dark matter annihilation and decay processes potentially raises the ionisation of the intergalactic medium and leaves visible footprints on the anisotropy maps of the cosmic microwave background (CMB). Galactic foregrounds emission in the microwave bands contaminate the CMB measurement and may affect the search for dark matter's signature. In this paper, we construct a full CMB data and foreground simulation based on the design of the next-generation ground-based CMB experiments. The foreground residual after the components separation on maps is fully considered in our data analysis, accounting for various contamination from the emission of synchrotron, thermal dust, free-free and spinning dust. We analyse the corresponding sensitivity on dark matter parameters from the temperature and polarization maps, and we find that the CMB foregrounds leave a non-zero yet controllable impact on the sensitivity. Comparing with statistics-only analysis, the CMB foreground residual leads to a factor of at most 19% weakening on energy-injection constraints, depending on the specific dark matter process and experimental configuration. Strong limits on dark matter annihilation rate and decay lifetime can be expected after foreground subtraction.

astro-ph.CO↗

arXiv2023

Study on the filters of atmospheric contamination in ground based CMB observation

The atmosphere is one of the most important contamination sources in the ground-based Cosmic Microwave Background (CMB) observations. In this paper, we study three kinds of filters, which are polynomial filter, high-pass filter, and Wiener filter, to investigate their ability for removing atmospheric noise, as well as their impact on the data analysis process through the end-to-end simulations of CMB experiment. We track their performance by analyzing the response of different components of the data, including both signals and noise. In the time domain, the calculation shows that the high-pass filter has the smallest root mean square error and can achieve high filtering efficiency, followed by the Wiener filter and polynomial filter. We then perform map-making with the filtered time ordered data (TOD) to trace the effects from filters on the map domain, and the results show that the polynomial filter gives high noise residual at low frequency, which gives rise to serious leakage to small scales in map domain during the map-making process, while the high-pass filter and Wiener filter do not have such significant leakage. Then we estimate the angular power spectra of residual noise, as well as those of the input signal for comparing the filter effects in the power spectra domain. Finally, we estimate the standard deviation of the filter corrected power spectra to compare the effects from different filters, and the results show that, at low noise level, the three filters give almost comparable standard deviations on the medium and small scales, but at high noise level, the standard deviation of the polynomial filter is significantly larger. These studies can be used for the reduction of atmospheric noise in future ground-based CMB data processing.

astro-ph.CO↗

arXiv2022

Rethinking Person Re-Identification via Semantic-Based Pretraining

Pretraining is a dominant paradigm in computer vision. Generally, supervised ImageNet pretraining is commonly used to initialize the backbones of person re-identification (Re-ID) models. However, recent works show a surprising result that CNN-based pretraining on ImageNet has limited impacts on Re-ID system due to the large domain gap between ImageNet and person Re-ID data. To seek an alternative to traditional pretraining, here we investigate semantic-based pretraining as another method to utilize additional textual data against ImageNet pretraining. Specifically, we manually construct a diversified FineGPR-C caption dataset for the first time on person Re-ID events. Based on it, a pure semantic-based pretraining approach named VTBR is proposed to adopt dense captions to learn visual representations with fewer images. We train convolutional neural networks from scratch on the captions of FineGPR-C dataset, and then transfer them to downstream Re-ID tasks. Comprehensive experiments conducted on benchmark datasets show that our VTBR can achieve competitive performance compared with ImageNet pretraining - despite using up to 1.4x fewer images, revealing its potential in Re-ID pretraining.

cs.CV↗

arXiv2022

Explicit-Solute Implicit-Solvent Molecular Simulation with Binary Level-Set, Adaptive-Mobility, and GPU

Coarse-grained modeling and efficient computer simulations are critical to the study of complex molecular processes with many degrees of freedom and multiple spatiotemporal scales. Variational implicit-solvent model (VISM) for biomolecular solvation is such a modeling framework, and its initial success has been demonstrated consistently. In VISM, an effective free-energy functional of solute-solvent interfaces is minimized, and the surface energy is a key component of the free energy. In this work, we extend VISM to include the solute mechanical interactions, and develop fast algorithms and GPU implementation for the extended variational explicit-solute implicit-solvent (VESIS) molecular simulations to determine the underlying molecular equilibrium conformations. We employ a fast binary level-set method for minimizing the solvation free energy of solute-solvent interfaces and construct an adaptive-mobility gradient descent method for solute atomic optimization. We also implement our methods in GPU. Numerical tests and applications to several molecular systems verify the accuracy, stability, and efficiency of our methods and algorithms. It is found that our new methods and GPU implementation improve the efficiency of the molecular simulation significantly over the CPU implementation. Our fast computational techniques may enable us to simulate very large systems such as protein-protein interactions and membrane dynamics for which explicit-solvent all-atom molecular dynamics simulations can be very expensive.

physics.chem-ph↗

arXiv2022

Efficient ILC analysis on polarization maps after EB leakage correction

The Internal Linear Combination (ILC) is widely used to extract the cosmic microwave background (CMB) signal from multi-frequency observation maps, especially for Satellite experiments with quasi-full sky coverage. We extend ILC method to CMB polarization map analysis with a small sky patch which is especially typical for ground-based experiments, by combing ILC with a template cleaning method which can give pure $B$ map free from $EB$ leakage caused by partial sky coverage. The feature of our methods is that we do the ILC analysis on pseudo-scalar $B$ maps, and the advantage is that it totally avoids the impact of $EB$ leakage on ILC, so that it can improve the efficiency of component separation dramatically. We demonstrate our methods with mock data of a future ground-based experiment with a deep survey on a clean patch in the northern sky, and the results show that the level of foreground residual can be well controlled, it biases the tensor to scalar ratio ($r$) at the order of $10^{-3}$ which is comparable to the statistical error by noise.

astro-ph.CO↗

arXiv2022

BA-Net: Bridge Attention for Deep Convolutional Neural Networks

In recent years, channel attention mechanism has been widely investigated due to its great potential in improving the performance of deep convolutional neural networks (CNNs) in many vision tasks. However, in most of the existing methods, only the output of the adjacent convolution layer is fed into the attention layer for calculating the channel weights. Information from other convolution layers has been ignored. With these observations, a simple strategy, named Bridge Attention Net (BA-Net), is proposed in this paper for better performance with channel attention mechanisms. The core idea of this design is to bridge the outputs of the previous convolution layers through skip connections for channel weights generation. Based on our experiment and theory analysis, we find that features from previous layers also contribute to the weights significantly. The Comprehensive evaluation demonstrates that the proposed approach achieves state-of-the-art(SOTA) performance compared with the existing methods in accuracy and speed. which shows that Bridge Attention provides a new perspective on the design of neural network architectures with great potential in improving performance. The code is available at https://github.com/zhaoy376/Bridge-Attention.

cs.CV↗

arXiv2021

Binary Level Set Method for Variational Implicit Solvation Model

In this article, we apply the binary level set method to the Variational Implicit Solvent Model (VISM), which is a theoretical and computational tool to study biomolecular systems with complex topology. Central in VISM is an effective free energy of all possible interfaces separating solutes (e.g., proteins) from solvent (e.g., water). Previously, such a functional is minimized numerically by the level set method to determine the stable equilibrium conformations and solvation free energies. We vastly improve the speed by applying the binary level set method, in which the interface is approximated by a binary level set function that only takes value $\pm 1$ on the solute/solvent region, leading to a discrete formulation of VISM energy. The surface area is approximated by convolution of an indicator function with a compactly supported kernel. The VISM energy can be minimized by iteratively "flipping" the binary level set function in a steepest descent fashion. Numerical experiments are performed to demonstrate the accuracy and performance of our method.

math.NA↗

arXiv2021

Association: Remind Your GAN not to Forget

Neural networks are susceptible to catastrophic forgetting. They fail to preserve previously acquired knowledge when adapting to new tasks. Inspired by human associative memory system, we propose a brain-like approach that imitates the associative learning process to achieve continual learning. We design a heuristics mechanism to potentiatively stimulate the model, which guides the model to recall the historical episodes based on the current circumstance and obtained association experience. Besides, a distillation measure is added to depressively alter the efficacy of synaptic transmission, which dampens the feature reconstruction learning for new task. The framework is mediated by potentiation and depression stimulation that play opposing roles in directing synaptic and behavioral plasticity. It requires no access to the original data and is more similar to human cognitive process. Experiments demonstrate the effectiveness of our method in alleviating catastrophic forgetting on image-to-image translation tasks.

cs.CV↗