arXiv subjects

Yang Luo

Publications and source records attributed to Yang Luo.

At least 55 records · Page 3Linked to original sources

arXiv2025

MERIT: Maximum-normalized Element-wise Ratio for Language Model Large-batch Training

Large-batch training has become a cornerstone in accelerating the training of deep neural networks, yet it poses challenges in optimization and generalization. Existing optimizers like AdamW present performance degradation during language models' large-batch training, due to the information bottleneck in attention layers caused by the sharp increase of max attention logit. While the LAMB optimizer partially addresses this issue, some attention layers still face this issue. The reason is that $l_2$-norm-based trust ratios in LAMB are less effective in directly influencing the max value of query/key weights. Furthermore, the weight-wise trust ratio in LAMB is error-prone as it overlooks relationships of weight values within rows or columns. Building on these observations, we propose a novel optimizer, MERIT, which leverages the max-norm to calculate the trust ratio to constrain the max attention logit more effectively. Moreover, we further construct element-wise trust ratios to provide more robust update scaling by focusing on local weight structures. Extensive experiments of large-batch training across various sizes of GPT-2 models demonstrate the superior performance of MERIT. Notably, during the training of GPT-2 Medium, MERIT enables a 6k batch size without any performance degradation compared to the standard batch size (480) with 48B training tokens. This work highlights the importance of considering the max attention logit and finer-granularity trust ratio in large-batch training. It successfully improves the training stability and paves the way for larger batch usage, enabling faster development and iteration of large language models. Code is available at https://github.com/NUS-HPC-AI-Lab/MERIT.

cs.LG↗

arXiv2025

Anomalous narrow-band correlation in a natural superconducting heterostructure

A new frontier in condensed matter physics is to stack atomically thin layered-materials with different properties and create intriguing phenomena which do not exist in any of the constituent layers. Transition metal dichalcogenide 4Hb-TaS$_2$, with an alternating stacking of a spin liquid candidate 1T-TaS$_2$ and a superconductor 1H-TaS$_2$, is a natural heterostructure for such a purpose. Recently, rare phenomena are indeed observed, including chiral superconductivity, two-component nematic superconductivity, topological surface superconductivity and enigmatic magnetic memory. A widely proposed starting point to understand such a mysterious heterostructure requires strong electronic correlation, presumably provided by 1T-TaS$_2$ layers with a narrow flat band near the Fermi level ($E_F$). Here, by using angle-resolved photoemission spectroscopy, we reveal the theoretically expected flat band near $E_F$ in the energy-momentum space for the first time. However, this flat band only exists on the 1T-TaS$_2$ terminated surface layer with broken translational symmetry, but not on the 1T-TaS$_2$ layers buried in the bulk. These results directly challenge the foundation of the current theoretical paradigm. On the 1T-TaS$_2$ terminated surface layer, we further reveal a pseudogap and an anomalous doping effect. These phenomena and the dichotomy between surface and bulk layers also shed new light on the unusual coexistence of distinct electronic orders in this mysterious heterostructure.

cond-mat.supr-con↗

arXiv2025

PAF-Net: Phase-Aligned Frequency Decoupling Network for Multi-Process Manufacturing Quality Prediction

Accurate quality prediction in multi-process manufacturing is critical for industrial efficiency but hindered by three core challenges: time-lagged process interactions, overlapping operations with mixed periodicity, and inter-process dependencies in shared frequency bands. To address these, we propose PAF-Net, a frequency decoupled time series prediction framework with three key innovations: (1) A phase-correlation alignment method guided by frequency domain energy to synchronize time-lagged quality series, resolving temporal misalignment. (2) A frequency independent patch attention mechanism paired with Discrete Cosine Transform (DCT) decomposition to capture heterogeneous operational features within individual series. (3) A frequency decoupled cross attention module that suppresses noise from irrelevant frequencies, focusing exclusively on meaningful dependencies within shared bands. Experiments on 4 real-world datasets demonstrate PAF-Net's superiority. It outperforms 10 well-acknowledged baselines by 7.06% lower MSE and 3.88% lower MAE. Our code is available at https://github.com/StevenLuan904/PAF-Net-Official.

cs.LG↗

arXiv2025

One-stage Modality Distillation for Incomplete Multimodal Learning

Learning based on multimodal data has attracted increasing interest recently. While a variety of sensory modalities can be collected for training, not all of them are always available in development scenarios, which raises the challenge to infer with incomplete modality. To address this issue, this paper presents a one-stage modality distillation framework that unifies the privileged knowledge transfer and modality information fusion into a single optimization procedure via multi-task learning. Compared with the conventional modality distillation that performs them independently, this helps to capture the valuable representation that can assist the final model inference directly. Specifically, we propose the joint adaptation network for the modality transfer task to preserve the privileged information. This addresses the representation heterogeneity caused by input discrepancy via the joint distribution adaptation. Then, we introduce the cross translation network for the modality fusion task to aggregate the restored and available modality features. It leverages the parameters-sharing strategy to capture the cross-modal cues explicitly. Extensive experiments on RGB-D classification and segmentation tasks demonstrate the proposed multimodal inheritance framework can overcome the problem of incomplete modality input in various scenes and achieve state-of-the-art performance.

cs.CV↗

arXiv2025

Improving Multimodal Learning via Imbalanced Learning

Multimodal learning often encounters the under-optimized problem and may perform worse than unimodal learning. Existing approaches attribute this issue to imbalanced learning across modalities and tend to address it through gradient balancing. However, this paper argues that balanced learning is not the optimal setting for multimodal learning. With bias-variance analysis, we prove that imbalanced dependency on each modality obeying the inverse ratio of their variances contributes to optimal performance. To this end, we propose the Asymmetric Representation Learning(ARL) strategy to assist multimodal learning via imbalanced optimization. ARL introduces auxiliary regularizers for each modality encoder to calculate their prediction variance. ARL then calculates coefficients via the unimodal variance to re-weight the optimization of each modality, forcing the modality dependence ratio to be inversely proportional to the modality variance ratio. Moreover, to minimize the generalization error, ARL further introduces the prediction bias of each modality and jointly optimizes them with multimodal loss. Notably, all auxiliary regularizers share parameters with the multimodal model and rely only on the modality representation. Thus the proposed ARL strategy introduces no extra parameters and is independent of the structures and fusion methods of the multimodal model. Finally, extensive experiments on various datasets validate the effectiveness and versatility of ARL. Code is available at \href{https://github.com/shicaiwei123/ICCV2025-ARL}{https://github.com/shicaiwei123/ICCV2025-ARL}

cs.CV↗

arXiv2025

Boosting Multimodal Learning via Disentangled Gradient Learning

Multimodal learning often encounters the under-optimized problem and may have worse performance than unimodal learning. Existing methods attribute this problem to the imbalanced learning between modalities and rebalance them through gradient modulation. However, they fail to explain why the dominant modality in multimodal models also underperforms that in unimodal learning. In this work, we reveal the optimization conflict between the modality encoder and modality fusion module in multimodal models. Specifically, we prove that the cross-modal fusion in multimodal models decreases the gradient passed back to each modality encoder compared with unimodal models. Consequently, the performance of each modality in the multimodal model is inferior to that in the unimodal model. To this end, we propose a disentangled gradient learning (DGL) framework to decouple the optimization of the modality encoder and modality fusion module in the multimodal model. DGL truncates the gradient back-propagated from the multimodal loss to the modality encoder and replaces it with the gradient from unimodal loss. Besides, DGL removes the gradient back-propagated from the unimodal loss to the modality fusion module. This helps eliminate the gradient interference between the modality encoder and modality fusion module while ensuring their respective optimization processes. Finally, extensive experiments on multiple types of modalities, tasks, and frameworks with dense cross-modal interaction demonstrate the effectiveness and versatility of the proposed DGL. Code is available at \href{https://github.com/shicaiwei123/ICCV2025-GDL}{https://github.com/shicaiwei123/ICCV2025-GDL}

cs.CV↗

arXiv2025

Direct Collapse Pre-supermassive Black Hole Objects as Ly$α$ Emitters

The Direct Collapse scenario to form the supermassive black hole (SMBH) seeds offers the most promising way to explain the origin of quasars at $z>7$. Assuming atomic primordial gas, can Ly$α$ photons escape from the central regions of the collapse and serve as a diagnostic for the detection of these pre-SMBH objects? Models of spherical collapse have found these photons to be trapped and destroyed. We use Ly$α$ radiation transfer within the inflow-outflow geometry, based on earlier zoom-in cosmological modeling involving radiation transfer and magnetic forces. Adopting geometry that includes ongoing disk and spherical accretion, and formation of a biconical outflow funnel, we obtain the formation of a dense radiatively driven expanding shell. The Ly$α$ transfer is performed using a Monte Carlo algorithm, accounting for the destruction of Ly$α$ photons and the emergence of two-photon emission. We find that a substantial fraction of Ly$α$ photons can escape through the funnel and calculate the line profiles, the line peak velocity shift, asymmetry, and cuspiness, by varying basic model parameters. The escaping Ly$α$ emission is anisotropic and sensitive to the overall inflow-outflow geometry. The escaping fraction of Ly$α$ radiation exceeds 95% from a $z=10$ pre-SMBH object -- in principle detectable by the JWST NIRSpec in the MOS mode, during $\sim 10^4$ seconds for a $10σ$ signal-to-noise ratio. Moreover, comparisons with line shapes from high-$z$ galaxies and quasars allow us to separate them from pre-SMBH objects based on the line shape: the pre-SMBH lines show a profound asymmetry and extended red tail.

astro-ph.GA↗

arXiv2025

Orbital-resolved imaging of coherent femtosecond exciton dynamics in coupled molecules

Optical excitation and control of excitonic wavepackets in organic molecules is the basis to energy conversion processes. To gain insights into such processes, it is essential to establish the relationship between the coherence timescales of excitons with the electronic inhomogeneity in the molecules, as well as the influence of intermolecular interactions on exciton dynamics. Here, we demonstrate orbital-resolved imaging of optically induced coherent exciton dynamics in single copper napthalocyanine (CuNc) molecules, and selective coherent excitation of dark and bright triplet excitons in coupled molecular dimers. Ultrafast photon-induced tunneling current enabled atomic-scale imaging and control of the excitons in resonantly excited molecules by employing excitonic wavepacket interferometry. Our results reveal an ultrafast exciton coherence time of ~ 70 fs in a single molecule, which decreases for the triplet excitons in interacting molecules.

physics.optics↗

arXiv2025

Info-Coevolution: An Efficient Framework for Data Model Coevolution

Machine learning relies heavily on data, yet the continuous growth of real-world data poses challenges for efficient dataset construction and training. A fundamental yet unsolved question is: given our current model and data, does a new data (sample/batch) need annotation/learning? Conventional approaches retain all available data, leading to non-optimal data and training efficiency. Active learning aims to reduce data redundancy by selecting a subset of samples to annotate, while it increases pipeline complexity and introduces bias. In this work, we propose Info-Coevolution, a novel framework that efficiently enables models and data to coevolve through online selective annotation with no bias. Leveraging task-specific models (and open-source models), it selectively annotates and integrates online and web data to improve datasets efficiently. For real-world datasets like ImageNet-1K, Info-Coevolution reduces annotation and training costs by 32\% without performance loss. It is able to automatically give the saving ratio without tuning the ratio. It can further reduce the annotation ratio to 50\% with semi-supervised learning. We also explore retrieval-based dataset enhancement using unlabeled open-source data. Code is available at https://github.com/NUS-HPC-AI-Lab/Info-Coevolution/.

cs.LG↗

arXiv2025

Widely Linear Augmented Extreme Learning Machine Based Impairments Compensation for Satellite Communications

Satellite communications are crucial for the evolution beyond fifth-generation networks. However, the dynamic nature of satellite channels and their inherent impairments present significant challenges. In this paper, a novel post-compensation scheme that combines the complex-valued extreme learning machine with augmented hidden layer (CELMAH) architecture and widely linear processing (WLP) is developed to address these issues by exploiting signal impropriety in satellite communications. Although CELMAH shares structural similarities with WLP, it employs a different core algorithm and does not fully exploit the signal impropriety. By incorporating WLP principles, we derive a tailored formulation suited to the network structure and propose the CELM augmented by widely linear least squares (CELM-WLLS) for post-distortion. The proposed approach offers enhanced communication robustness and is highly effective for satellite communication scenarios characterized by dynamic channel conditions and non-linear impairments. CELM-WLLS is designed to improve signal recovery performance and outperform traditional methods such as least square (LS) and minimum mean square error (MMSE). Compared to CELMAH, CELM-WLLS demonstrates approximately 0.8 dB gain in BER performance, and also achieves a two-thirds reduction in computational complexity, making it a more efficient solution.

eess.SP↗

arXiv2025

Randomized Block-Coordinate Optimistic Gradient Algorithms for Root-Finding Problems

In this paper, we develop two new randomized block-coordinate optimistic gradient algorithms to approximate a solution of nonlinear equations in large-scale settings, which are called root-finding problems. Our first algorithm is non-accelerated with constant stepsizes, and achieves $\mathcal{O}(1/k)$ best-iterate convergence rate on $\mathbb{E}[ \Vert Gx^k\Vert^2]$ when the underlying operator $G$ is Lipschitz continuous and satisfies a weak Minty solution condition, where $\mathbb{E}[\cdot]$ is the expectation and $k$ is the iteration counter. Our second method is a new accelerated randomized block-coordinate optimistic gradient algorithm. We establish both $\mathcal{O}(1/k^2)$ and $o(1/k^2)$ last-iterate convergence rates on both $\mathbb{E}[ \Vert Gx^k\Vert^2]$ and $\mathbb{E}[ \Vert x^{k+1} - x^{k}\Vert^2]$ for this algorithm under the co-coerciveness of $G$. In addition, we prove that the iterate sequence $\{x^k\}$ converges to a solution almost surely, and $k\Vert Gx^k\Vert$ attains a $o(1/k)$ almost sure convergence rate. Then, we apply our methods to a class of large-scale finite-sum inclusions, which covers prominent applications in machine learning, statistical learning, and network optimization, especially in federated learning. We obtain two new federated learning-type algorithms and their convergence rate guarantees for solving this problem class.

math.OC↗

arXiv2025

Numerical modeling the mass feeding rates onto accretion-modified stars embedded within AGN disks

Accretion disks surrounding supermassive black holes can potentially form stars within the self-gravitating region. These stars undergo high accretion rates because of the dense environment of the active galactic nuclei (AGN) accretion disk. The vorticity of the AGN disk may influence the ultimate mass feeding rate toward the star. In our study, we simulate mass feeding rates onto stars at different AGN disk thicknesses through 3D numerical models to explore the relationship between feeding rates and the thermal mass of the star ($q_{\rm th}$), defined as the ratio of the star's Bondi radius to the AGN disk thickness. Our findings indicate that disk shearing with angular momentum can notably decrease the feeding rate, and we provide an approximate formula that links the feeding rate based on the angular momentum of the surrounding gas and the thermal mass $q_{\rm th}$. Lastly, we examine the potential feedback of the rapidly accreting stars on the AGN disk and their subsequent evolution.

astro-ph.GA↗

arXiv2025

ScalerEval: Automated and Consistent Evaluation Testbed for Auto-scalers in Microservices

Auto-scaling is an automated approach that dynamically provisions resources for microservices to accommodate fluctuating workloads. Despite the introduction of many sophisticated auto-scaling algorithms, evaluating auto-scalers remains time-consuming and labor-intensive, as it requires the implementation of numerous fundamental interfaces, complex manual operations, and in-depth domain knowledge. Besides, frequent human intervention can inevitably introduce operational errors, leading to inconsistencies in the evaluation of different auto-scalers. To address these issues, we present ScalerEval, an end-to-end automated and consistent testbed for auto-scalers in microservices. ScalerEval integrates essential fundamental interfaces for implementation of auto-scalers and further orchestrates a one-click evaluation workflow for researchers. The source code is publicly available at \href{https://github.com/WHU-AISE/ScalerEval}{https://github.com/WHU-AISE/ScalerEval}.

cs.SE↗

arXiv2025

Enhance-A-Video: Better Generated Video for Free

DiT-based video generation has achieved remarkable results, but research into enhancing existing models remains relatively unexplored. In this work, we introduce a training-free approach to enhance the coherence and quality of DiT-based generated videos, named Enhance-A-Video. The core idea is enhancing the cross-frame correlations based on non-diagonal temporal attention distributions. Thanks to its simple design, our approach can be easily applied to most DiT-based video generation frameworks without any retraining or fine-tuning. Across various DiT-based video generation models, our approach demonstrates promising improvements in both temporal consistency and visual quality. We hope this research can inspire future explorations in video generation enhancement.

cs.CV↗

arXiv2025

PICBench: Benchmarking LLMs for Photonic Integrated Circuits Design

While large language models (LLMs) have shown remarkable potential in automating various tasks in digital chip design, the field of Photonic Integrated Circuits (PICs)-a promising solution to advanced chip designs-remains relatively unexplored in this context. The design of PICs is time-consuming and prone to errors due to the extensive and repetitive nature of code involved in photonic chip design. In this paper, we introduce PICBench, the first benchmarking and evaluation framework specifically designed to automate PIC design generation using LLMs, where the generated output takes the form of a netlist. Our benchmark consists of dozens of meticulously crafted PIC design problems, spanning from fundamental device designs to more complex circuit-level designs. It automatically evaluates both the syntax and functionality of generated PIC designs by comparing simulation outputs with expert-written solutions, leveraging an open-source simulator. We evaluate a range of existing LLMs, while also conducting comparative tests on various prompt engineering techniques to enhance LLM performance in automated PIC design. The results reveal the challenges and potential of LLMs in the PIC design domain, offering insights into the key areas that require further research and development to optimize automation in this field. Our benchmark and evaluation code is available at https://github.com/PICDA/PICBench.

cs.LG↗

arXiv2024

The Schur-Weyl duality and Invariants for classical Lie superalgebras

In this article, we provide a comprehensive characterization of invariants of classical Lie superalgebras from the super-analog of the Schur-Weyl duality in a unified way. We establish $\mathfrak{g}$-invariants of the tensor algebra $T(\mathfrak{g})$, the supersymmetric algebra $S(\mathfrak{g})$, and the universal enveloping algebra $\mathrm{U}(\mathfrak{g})$ of a classical Lie superalgebra $\mathfrak{g}$ corresponding to every element in centralizer algebras and their relationship under supersymmetrization. As a byproduct, we prove that the restriction on $T(\mathfrak{g})^{\mathfrak{g}}$ of the projection from $T(\mathfrak{g})$ to $\mathrm{U}(\mathfrak{g})$ is surjective, which enables us to determine the generators of the center $\mathcal{Z}(\mathfrak{g})$ except for $\mathfrak{g}=\mathfrak{osp}_{2m|2n}$. Additionally, we present an alternative algebraic proof of the triviality of $\mathcal{Z}(\mathfrak{p}_n)$. The key ingredient involves a technique lemma related to the symmetric group and Brauer diagrams.

math.RT↗

arXiv2024

A Bilayer Segmentation-Recombination Network for Accurate Segmentation of Overlapping C. elegans

Caenorhabditis elegans (C. elegans) is an excellent model organism because of its short lifespan and high degree of homology with human genes, and it has been widely used in a variety of human health and disease models. However, the segmentation of C. elegans remains challenging due to the following reasons: 1) the activity trajectory of C. elegans is uncontrollable, and multiple nematodes often overlap, resulting in blurred boundaries of C. elegans. This makes it impossible to clearly study the life trajectory of a certain nematode; and 2) in the microscope images of overlapping C. elegans, the translucent tissues at the edges obscure each other, leading to inaccurate boundary segmentation. To solve these problems, a Bilayer Segmentation-Recombination Network (BR-Net) for the segmentation of C. elegans instances is proposed. The network consists of three parts: A Coarse Mask Segmentation Module (CMSM), a Bilayer Segmentation Module (BSM), and a Semantic Consistency Recombination Module (SCRM). The CMSM is used to extract the coarse mask, and we introduce a Unified Attention Module (UAM) in CMSM to make CMSM better aware of nematode instances. The Bilayer Segmentation Module (BSM) segments the aggregated C. elegans into overlapping and non-overlapping regions. This is followed by integration by the SCRM, where semantic consistency regularization is introduced to segment nematode instances more accurately. Finally, the effectiveness of the method is verified on the C. elegans dataset. The experimental results show that BR-Net exhibits good competitiveness and outperforms other recently proposed instance segmentation methods in processing C. elegans occlusion images.

cs.CV↗

arXiv2024

AE-DENet: Enhancement for Deep Learning-based Channel Estimation in OFDM Systems

Deep learning (DL)-based methods have demonstrated remarkable achievements in addressing orthogonal frequency division multiplexing (OFDM) channel estimation challenges. However, existing DL-based methods mainly rely on separate real and imaginary inputs while ignoring the inherent correlation between the two streams, such as amplitude and phase information that are fundamental in communication signal processing. This paper proposes AE-DENet, a novel autoencoder(AE)-based data enhancement network to improve the performance of existing DL-based channel estimation methods. AE-DENet focuses on enriching the classic least square (LS) estimation input commonly used in DL-based methods by employing a learning-based data enhancement method, which extracts interaction features from the real and imaginary components and fuses them with the original real/imaginary streams to generate an enhanced input for better channel inference. Experimental findings in terms of the mean square error (MSE) results demonstrate that the proposed method enhances the performance of all state-of-the-art DL-based channel estimators with negligible added complexity. Furthermore, the proposed approach is shown to be robust to channel variations and high user mobility.

eess.SP↗