Searcharxiv⌕ Search

arXiv subjects

Zhiqiang Zou

Publications and source records attributed to Zhiqiang Zou.

At least 19 recordsLinked to original sources

HiFA4: Training-Free 4-bit FlashAttention on Ascend HIF4 NPUs for LLM Inference

We present HiFA4, a post-training operator-level design that executes both QK^T and PV in FlashAttention as 4-bit HIF4 Cube GEMMs for LLM inference on Ascend NPUs, while maintaining the online softmax state in FP16. To our knowledge, HiFA4 is the first Ascend-HIF4-targeted design of this kind evaluated on standard NLP benchmarks. HiFA4 combines two mechanisms. Smooth-QK applies a calibration-static per-channel equivalent rescaling to Q and K after RoPE, transferring quantization difficulty from K to Q without per-tile online reduction at inference. P-Reordering accumulates the softmax normalizer from the same quantized attention weights P_hat used in the PV GEMM, rather than from a higher-precision reconstruction. We show that this inconsistent formulation introduces a coherent output-scaling error, and validate the effect on a Qwen3-8B Layer-0 MMLU trace, where all 3.6M measured attention tiles exhibit net probability-mass loss with median epsilon_bar = -0.064. P-Reordering also allows the normalizer to be fused into the PV Cube GEMM. Across five LLMs, HiFA4 reduces quantization-induced decision drift. On Qwen3-8B, it recovers 37.5% of the accuracy gap introduced by direct HIF4 quantization, narrows the sample-weighted accuracy loss from 1.12 pp to 0.70 pp, reduces BF16-inconsistent MMLU predictions from 16.3% to 8.2%, and cuts MMLU accuracy regressions by 57% (1071 to 465). On Gemma2-9B, mild smoothing keeps HiFA4 within 0.7 pp of BF16 while reducing MMLU regressions by 27%. On LLaMA3.1-8B, Mistral-7B, and Phi-4B, where Smooth-QK is disabled, P-Reordering with the adopted Q-Mean auxiliary still reduces full-set MMLU regressions by 41-52%. A preliminary instruction-scheduling analysis projects a 35.4% critical-path latency reduction relative to BF16 by fusing the softmax normalizer into the PV Cube GEMM; on-hardware validation is left to future work.

cs.LG↗

Training-free sparse attention based on cumulative energy filtering

Sparse attention accelerates Diffusion Transformers (DiTs) for video generation by computing only the important tokens while skipping the rest. The token selection strategy is key to balancing sparsity and accuracy. We formulate the token filtering process as a dual-goal optimization problem: maximizing sparsity and minimizing accuracy degradation. Existing algorithms cannot fulfill both objectives simultaneously. For example, Top-p only considers the accuracy constraint, while Top-k maintains a fixed computational budget but loosens the accuracy constraint. This paper demonstrates that maintaining a fixed recall rate is sufficient for ensuring accuracy, whereas a fixed threshold is suboptimal for reducing computational cost. Therefore, we propose a dynamic thresholding scheme to improve sparsity while maintaining the same level of accuracy. Furthermore, our algorithm is deeply integrated with Flash Attention (FA), eliminating the need for any additional masking computation overhead. Experimental results on Wan 2.2 validate that, compared to the BLASST algorithm which is also integrated with FA, our dynamic thresholding strategy enhances sparsity from 61.42\% to 82\% with a VBench metric drop of less than 5\%. This results in an approximate 15\% in attention computation and a $1.61\times$ increase in computational efficiency, which is 1.18x higher than that of BLASST.

cs.CV↗

OSC: Hardware Efficient W4A4 Quantization via Outlier Separation in Channel Dimension

While 4-bit quantization is essential for high-throughput deployment of Large Language Models, activation outliers often lead to significant accuracy degradation due to the restricted dynamic range of low-bit formats. In this paper, we systematically investigate the spatial distribution of outliers and demonstrate a token-persistent structural clustering effect, where high-magnitude outliers consistently occupy fixed channels across tokens. Building on this insight, we propose OSC, a hardware-efficient framework for outlier suppression. During inference, OSC executes a dual-path computation consisting of a low-precision 4-bit General Matrix Multiplication (GEMM) path and a high-precision 16-bit branch GEMM path. Specifically, OSC uses an offline group-wise strategy to identify the channels where outliers are located and then performs structured sub-tensor extraction to coalesce these scattered activation channels into a compact dense tensor online. This mechanism implements outlier protection through regularized and high-throughput GEMM operations, achieving a seamless fit with modern 4-bit micro-scaling hardware. Furthermore, for the inputs of W2 where outlier clustering is less pronounced, we integrate a fallback strategy to FP8. Evaluation on Qwen3-8B and Qwen3-30B restricts the average accuracy drop to 2.19 and 1.12 points, respectively. Notably, OSC is highly hardware-friendly, achieving a peak speedup of 1.78x over the W8A8 GEMM baseline on a modern AI accelerator.

cs.LG↗

VFA: Relieving Vector Operations in Flash Attention with Global Maximum Pre-computation

FlashAttention-style online softmax enables exact attention computation with linear memory by streaming score tiles through on-chip memory and maintaining a running maximum and normalizer. However, as attention kernels approach peak tensor-core/cube-core throughput on modern accelerators, non-matmul components of online softmax -- especially per-tile rowmax and rowsum reductions and rescale chains -- can become vector or SIMD limited and dominate latency. This paper revisits FlashAttention and proposes Vector Relieved Flash Attention (VFA), a hardware-friendly method that reduces rowmax-driven updates of the running maximum while retaining the online-softmax structure. VFA initializes the running maximum via a cheap approximation from key-block representations, reorders key-block traversal to prioritize high-impact sink and local blocks, and freezes the maximum for remaining blocks to avoid repeated reductions and rescaling. We further integrate VFA with block-sparse skipping methods such as BLASST to form Vector Relieved Sparse Attention (VSA), which reduces both block count and per-block overhead. Notably, VFA and VSA completely avoid the conditional rescale operation in the update stage used in FA4.0. Extensive evaluations on benchmarks including MMLU and MATH500, together with attention statistics, verify our design: (i) sink and local reordering stabilizes the running maximum early; (ii) simple Q and K block summaries fail due to intra-block heterogeneity; (iii) m-initialization is required when maxima appear in middle blocks. Overall, VFA and VSA efficiently alleviate online-softmax reduction bottlenecks without performance loss. Compared to the C16V32 baseline, C8V32, C4V32 and C4V16 achieve nearly two times speedup on modern hardware while hitting the vector bottleneck. With upcoming architecture improvements, C4V16 will deliver six times speedup by enhancing exponent capacity.

cs.LG↗

StellarF: A Physics-Informed LoRA Framework for Stellar Flare Forecasting with Historical & Statistical Data

Stellar flare forecasting represents a critical frontier in astrophysics, offering profound insights into stellar activity mechanisms and exoplanetary habitability assessments. Yet the inherent unpredictability of flare activity, rooted in stellar diversity and evolutionary stages, underpins the field's core challenges: (1) sparse, incomplete, noisy lightcurve data from traditional observations; (2) ineffective multi-scale flare evolution capture via single representations; (3) poor physical interpretability in data-driven models lacking physics-informed priors. To address these challenges, we propose StellarF, a physics-informed framework synergizing general Al with astrophysical domain knowledge via three core components: a unified preprocessing pipeline for lightcurve refinement (missing-value imputation, temporal patch partitioning, adaptive sample filtering); a Low-Rank Adaptation (LoRA)-finetuned large language model (LLM) backbone enhanced by first-order difference augmentation, flare statistical information, and flare historical record modules for multimodal fusion instead of only simple representations; and a novel physics-informed loss embedding a minimum rising rate prior, appended to the cross-entropy loss, to align with flare physics. Extensive experiments on Kepler and TESS datasets show StellarF achieves state-of-the-art performance across key metrics, setting new benchmarks for flare forecasting. This work bridges general AI with astrophysics, offering a practical, physically interpretable paradigm for transient event forecasting in time-domain astronomy.

cs.LG↗

Identifying eclipsing binary stars with TESS data based on a new hybrid deep learning model

Eclipsing binary systems (EBs), as foundational objects in stellar astrophysics, have garnered significant attention in recent years. These systems exhibit periodic decreases in light intensity when one star obscures the other from the observer's perspective, producing characteristic light curves (LCs). With the advent of the Transiting Exoplanet Survey Satellite (TESS), a vast repository of stellar LCs has become available, offering unprecedented opportunities for discovering new EBs. To efficiently identify such systems, we propose a novel method that combines LC data and generalized Lomb-Scargle periodograms (GLS) data to classify EBs. At the core of this method is CNN Attention LSTM Net (CALNet), a hybrid deep learning model integrating Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and an Attention Mechanism based on the Convolutional Block Attention Module (CBAM). We collected 4,225 EB samples, utilizing their 2-minute cadence LCs for model training and validation. CALNet achieved a recall rate of 99.1%, demonstrating its robustness and effectiveness. Applying it to TESS 2-minute LCs from Sectors 1 to 74, we identified 9,351 new EBs after manual visual inspection, significantly expanding the known sample size. This work highlights the potential of advanced deep-learning techniques in large-scale astronomical surveys and provides a valuable resource for further studies on EBs.

astro-ph.SR↗

A Versatile Framework for Analyzing Galaxy Image Data by Implanting Human-in-the-loop on a Large Vision Model

The exponential growth of astronomical datasets provides an unprecedented opportunity for humans to gain insight into the Universe. However, effectively analyzing this vast amount of data poses a significant challenge. Astronomers are turning to deep learning techniques to address this, but the methods are limited by their specific training sets, leading to considerable duplicate workloads too. Hence, as an example to present how to overcome the issue, we built a framework for general analysis of galaxy images, based on a large vision model (LVM) plus downstream tasks (DST), including galaxy morphological classification, image restoration, object detection, parameter extraction, and more. Considering the low signal-to-noise ratio of galaxy images and the imbalanced distribution of galaxy categories, we have incorporated a Human-in-the-loop (HITL) module into our large vision model, which leverages human knowledge to enhance the reliability and interpretability of processing galaxy images interactively. The proposed framework exhibits notable few-shot learning capabilities and versatile adaptability to all the abovementioned tasks on galaxy images in the DESI legacy imaging surveys. Expressly, for object detection, trained by 1000 data points, our DST upon the LVM achieves an accuracy of 96.7%, while ResNet50 plus Mask R-CNN gives an accuracy of 93.1%; for morphology classification, to obtain AUC ~0.9, LVM plus DST and HITL only requests 1/50 training sets compared to ResNet18. Expectedly, multimodal data can be integrated similarly, which opens up possibilities for conducting joint analyses with datasets spanning diverse domains in the era of multi-message astronomy.

astro-ph.IM↗

A Multimodal Ecological Civilization Pattern Recommendation Method Based on Large Language Models and Knowledge Graph

The Ecological Civilization Pattern Recommendation System (ECPRS) aims to recommend suitable ecological civilization patterns for target regions, promoting sustainable development and reducing regional disparities. However, the current representative recommendation methods are not suitable for recommending ecological civilization patterns in a geographical context. There are two reasons for this. Firstly, regions have spatial heterogeneity, and the (ECPRS)needs to consider factors like climate, topography, vegetation, etc., to recommend civilization patterns adapted to specific ecological environments, ensuring the feasibility and practicality of the recommendations. Secondly, the abstract features of the ecological civilization patterns in the real world have not been fully utilized., resulting in poor richness in their embedding representations and consequently, lower performance of the recommendation system. Considering these limitations, we propose the ECPR-MML method. Initially, based on the novel method UGPIG, we construct a knowledge graph to extract regional representations incorporating spatial heterogeneity features. Following that, inspired by the significant progress made by Large Language Models (LLMs) in the field of Natural Language Processing (NLP), we employ Large LLMs to generate multimodal features for ecological civilization patterns in the form of text and images. We extract and integrate these multimodal features to obtain semantically rich representations of ecological civilization. Through extensive experiments, we validate the performance of our ECPR-MML model. Our results show that F1@5 is 2.11% higher compared to state-of-the-art models, 2.02% higher than NGCF, and 1.16% higher than UGPIG. Furthermore, multimodal data can indeed enhance recommendation performance. However, the data generated by LLM is not as effective as real data to a certain extent.

cs.IR↗

Unveiling Optimal SDG Pathways: An Innovative Approach Leveraging Graph Pruning and Intent Graph for Effective Recommendations

The recommendation of appropriate development pathways, also known as ecological civilization patterns for achieving Sustainable Development Goals (namely, sustainable development patterns), are of utmost importance for promoting ecological, economic, social, and resource sustainability in a specific region. To achieve this, the recommendation process must carefully consider the region's natural, environmental, resource, and economic characteristics. However, current recommendation algorithms in the field of computer science fall short in adequately addressing the spatial heterogeneity related to environment and sparsity of regional historical interaction data, which limits their effectiveness in recommending sustainable development patterns. To overcome these challenges, this paper proposes a method called User Graph after Pruning and Intent Graph (UGPIG). Firstly, we utilize the high-density linking capability of the pruned User Graph to address the issue of spatial heterogeneity neglect in recommendation algorithms. Secondly, we construct an Intent Graph by incorporating the intent network, which captures the preferences for attributes including environmental elements of target regions. This approach effectively alleviates the problem of sparse historical interaction data in the region. Through extensive experiments, we demonstrate that UGPIG outperforms state-of-the-art recommendation algorithms like KGCN, KGAT, and KGIN in sustainable development pattern recommendations, with a maximum improvement of 9.61% in Top-3 recommendation performance.

cs.IR↗

PhotoRedshift-MML: a multimodal machine learning method for estimating photometric redshifts of quasars

We propose a Multimodal Machine Learning method for estimating the Photometric Redshifts of quasars (PhotoRedshift-MML for short), which has long been the subject of many investigations. Our method includes two main models, i.e. the feature transformation model by multimodal representation learning, and the photometric redshift estimation model by multimodal transfer learning. The prediction accuracy of the photometric redshift was significantly improved owing to the large amount of information offered by the generated spectral features learned from photometric data via the MML. A total of 415,930 quasars from Sloan Digital Sky Survey (SDSS) Data Release 17, with redshifts between 1 and 5, were screened for our experiments. We used |Δz| = |(z_phot-z_spec)/(1+z_spec)| to evaluate the redshift prediction and demonstrated a 4.04% increase in accuracy. With the help of the generated spectral features, the proportion of data with |Δz| < 0.1 can reach 84.45% of the total test samples, whereas it reaches 80.41% for single-modal photometric data. Moreover, the Root Mean Square (RMS) of |Δz| is shown to decreases from 0.1332 to 0.1235. Our method has the potential to be generalized to other astronomical data analyses such as galaxy classification and redshift prediction. The algorithm code can be found at https://github.com/HongShuxin/PhotoRedshift-MML .

astro-ph.GA↗

Identifying outliers in astronomical images with unsupervised machine learning

Astronomical outliers, such as unusual, rare or unknown types of astronomical objects or phenomena, constantly lead to the discovery of genuinely unforeseen knowledge in astronomy. More unpredictable outliers will be uncovered in principle with the increment of the coverage and quality of upcoming survey data. However, it is a severe challenge to mine rare and unexpected targets from enormous data with human inspection due to a significant workload. Supervised learning is also unsuitable for this purpose since designing proper training sets for unanticipated signals is unworkable. Motivated by these challenges, we adopt unsupervised machine learning approaches to identify outliers in the data of galaxy images to explore the paths for detecting astronomical outliers. For comparison, we construct three methods, which are built upon the k-nearest neighbors (KNN), Convolutional Auto-Encoder (CAE)+ KNN, and CAE + KNN + Attention Mechanism (attCAE KNN) separately. Testing sets are created based on the Galaxy Zoo image data published online to evaluate the performance of the above methods. Results show that attCAE KNN achieves the best recall (78%), which is 53% higher than the classical KNN method and 22% higher than CAE+KNN. The efficiency of attCAE KNN (10 minutes) is also superior to KNN (4 hours) and equal to CAE+KNN(10 minutes) for accomplishing the same task. Thus, we believe it is feasible to detect astronomical outliers in the data of galaxy images in an unsupervised manner. Next, we will apply attCAE KNN to available survey datasets to assess its applicability and reliability.

cs.CV↗

Nontrivial topological states in BaSn5 superconductor probed by de Haas-van Alphen quantum oscillations

We report herein the nontrivial topological states in an intrinsic type-II superconductor BaSn5 (Tc ~ 4.4 K) probed via measuring the magnetizations, specific heat, de Haas-van Alphen (dHvA) effect and performing first principles calculations. The first principles calculations reveal a topological nodal ring structure centering at the H point in the kz = π plane of the Brillouin zone (BZ), which could be gapped by spin-orbit coupling (SOC), yielding rather small gaps below and above the Fermi level about 0.04 eV and 0.14 eV, respectively. The SOC also results in a pair of Dirac points along the Γ-A direction and located ~ 0.2 eV above the Fermi level. The analysis of the dHvA quantum oscillations supports the calculations by revealing nontrivial Berry phase originated from three hole and one electron pockets related to the bands forming the Dirac cones. Our study thus provides an excellent avenue for investigating the interplay between superconductivity and nontrivial topological states.

cond-mat.supr-con↗

Classifying Galaxy Morphologies with Few-Shot Learning

The taxonomy of galaxy morphology is critical in astrophysics as the morphological properties are powerful tracers of galaxy evolution. With the upcoming Large-scale Imaging Surveys, billions of galaxy images challenge astronomers to accomplish the classification task by applying traditional methods or human inspection. Consequently, machine learning, in particular supervised deep learning, has been widely employed to classify galaxy morphologies recently due to its exceptional automation, efficiency, and accuracy. However, supervised deep learning requires extensive training sets, which causes considerable workloads; also, the results are strongly dependent on the characteristics of training sets, which leads to biased outcomes potentially. In this study, we attempt Few-shot Learning to bypass the two issues. Our research adopts the dataset from Galaxy Zoo Challenge Project on Kaggle, and we divide it into five categories according to the corresponding truth table. By classifying the above dataset utilizing few-shot learning based on Siamese Networks and supervised deep learning based on AlexNet, VGG_16, and ResNet_50 trained with different volumes of training sets separately, we find that few-shot learning achieves the highest accuracy in most cases, and the most significant improvement is $21\%$ compared to AlexNet when the training sets contain 1000 images. In addition, to guarantee the accuracy is no less than 90\%, few-shot learning needs $\sim$6300 images for training, while ResNet_50 requires 13000 images. Considering the advantages stated above, foreseeably, few-shot learning is suitable for the taxonomy of galaxy morphology and even for identifying rare astrophysical objects, despite limited training sets consisting of observational data only.

astro-ph.GA↗

Pressure-induced superconductivity reentrant in transition metal dichalcogenide TiSe2

Through either elements intercalation or application of pressure, transition metal dichalcogenide 1T-TiSe2 exhibits superconductivity in proximity to a charge density wave (CDW) quantum critical point (QCP), thus providing an ideal avenue to study the correlation between the two symmetry-breaking exotic quantum electronic states. We report herein that, in addition to the well-known superconducting dome that emerges within the low pressure range of 2 - 4 GPa and peaks with the maximal Tc of about 1.8 K, the pressure induces another separate superconducting transition starting around 15 GPa with a substantially higher Tc that reaches 5.6 K at about 21.5 GPa. The high-pressure X-ray diffraction and Raman spectroscopy measurements unveil that the superconductivity reentrant is caused by a first-order structural phase transition (from P-3m1 space group to Pnma space group), which is also supported by the density functional theory calculation. A comparative theoretical calculation also reveals that the conventional phonon-mediated mechanism can account for the superconductivity of 1T-TiSe2 under low pressure, while the electron-phonon coupling of 4O-TiSe2 under high pressure is too weak to induce the superconductivity with a Tc as high as 5.6 K. This implies that the emergent superconductivity in the 4O-TiSe2 may have an unconventional origin. Our finding would open a new window toward the discovery of more exotic quantum states in transition metal dichalcogenides via high pressure.

cond-mat.supr-con↗

Pressure-induced ideal Weyl semimetal state in the layered antiferromagnet EuCd2As2

The rich nontrivial topological phases rooted in the interplay between magnetism and topology in the layered antiferromagnet EuCd2As2 have captured vast attention, especially the ideal Weyl semimetal state realized in the spin-polarized ferromagnetic (FM) structure driven by a moderate external magnetic field. In this work, combining high-pressure magnetotransport measurements, structure chracterizations and first principles calculations, we find that application of pressure can also realize the ideal Weyl state in EuCd2As2 through driving the in-plane antiferromagnetic state across an intermediate in-plane FM state then into the out-of-plane FM state. Our high-pressure angle dispersive X-ray diffraction and X-ray absorption near-edge spectroscopy measurements excluded structure transition and/or change of Eu2+ valence state as the sources for the magnetic phase transitions. Alternatively, the apparently reduced axial ratio (c/a) and compressed Eu-layer space distance should play important roles. Our result provides an alternative way to realize the ideal Weyl semimetal state in EuCd2As2 and would be instructive for the exploration of exotic topological properties in such layered magnetic topological phase with strongly competing magnetic exchanges by using high pressure.

cond-mat.mtrl-sci↗

Tunable intrinsic ferromagnetic topological phases in bulk van der Waals crystal MnSb6Te10

Intrinsic ferromagnetism is a crucial ingredient to realize quantum anomalous Hall effect in quasi two dimensional materials, thus the search of intrinsic ferromagnetic topological materials is one of the most concerned issues in the field of topological phases of matter. In this work, combining magnetotransport measurements, first principles calculations, and angle-resolved photoemission spectroscopy studies, we find that in MnSb6Te10, the n = 2 member of the MnSb2Te4/(Sb2Te3)n family, the strong magnetic competition realizes a fragile ferromagnetic ground state, which whereas easily enters into ferrimagnetic and the Z_2 antiferromagnetic topological insulator phase with warming to higher temperature. Interestingly, the system stays in an inversion-symmetry-protected axion insulator phase in the ferromagnetic ground state as well as in the external magnetic field driven spin-polarized FM phase and can be converted into a Weyl semimetal with multiple Weyl nodes in the valence bands with hole doping, which are manifested by the measured notable intrinsic anomalous Hall effect. Our work thus provides an intrinsic magnetic topological material which is highly tunable into versatile topological phases by temperature, magnetic field, as well as carrier doping.

cond-mat.mtrl-sci↗

Magnetization tunable Weyl states in EuB6

The interplay between magnetism and topological band structure offers extraordinary opportunities to realize rich exotic magnetic topological phases such as axion insulators, Weyl semimetals, and quantum anomalous Hall insulators, which therefore has attracted fast growing research interest. The rare-earth hexaborides EuB6 represents an interesting magnetic topological phase with tunable magnetizations along different crystallographic directions, while the correlation with the topological properties remains scarcely explored. In this work, combining magnetotransport measurements and first principles calculations, we demonstrate that EuB6 exhibits versatile magnetic topological phases along different crystallographic directions, which tightly correlate with the varied magnetizations. Moreover, by virtue of the weak magneto-crystalline anisotropy and the relatively strong coupling between the local magnetization and the conduction electrons, we show that the magnetic ground state of the system can be directly probed by the anisotropy in the magnetotransport properties. Our work thus introduces an excellent platform to study the rich topological phases that are tunable by magnetic orders.

cond-mat.str-el↗

Weak Antilocalization Effect up to ~ 120 K in the van der Waals Crystal Fe5-xGeTe2 with Near Room Temperature Ferromagnetism

The weak antilocalization (WAL) effect is known as a quantum correction to the classical conductivity, which never appeared in two-dimensional magnets. In this work, we reported the observation of a WAL effect in the van der Waals ferromagnet Fe5-xGeTe2 with a Curie temperature Tc ~ 270 K, which can even reach as high as ~ 120 K. The WAL effect could be well described by the Hikami-Larkin-Nagaoka and Maekawa-Fukuyama theories in the presence of strong spin-orbit coupling (SOC). Moreover, A crossover from a peak to dip behavior around 60 K in both the magnetoresistance and magnetoconductance was observed, which could be ascribed to a rare example of temperature driven Lifshitz transition as indicated by the angle-resolved photoemission spectroscopy measurements and first principles calculations. The reflective magnetic circular dichroism measurements indicate a possible spin reorientation that kills the WAL effect above 120 K. Our findings present a rare example of WAL effect in two-dimensional ferromagnet and also a magnetotransport fingerprint of the strong SOC in Fe5-xGeTe2. The results would be instructive for understanding the interaction Hamiltonian for such high Tc itinerant ferromagnetism as well as be helpful for the design of next-generation room temperature spintronic or twistronic devices.

cond-mat.mtrl-sci↗