arXiv subjects

Jiajun Zhang

Publications and source records attributed to Jiajun Zhang.

At least 109 records · Page 6Linked to original sources

arXiv2025

LR^2Bench: Evaluating Long-chain Reflective Reasoning Capabilities of Large Language Models via Constraint Satisfaction Problems

Recent progress in Large Reasoning Models (LRMs) has significantly enhanced the reasoning abilities of Large Language Models (LLMs), empowering them to tackle increasingly complex tasks through reflection capabilities, such as making assumptions, backtracking, and self-refinement. However, effectively evaluating such reflection capabilities remains challenging due to the lack of appropriate benchmarks. To bridge this gap, we introduce LR$^2$Bench, a novel benchmark designed to evaluate the Long-chain Reflective Reasoning capabilities of LLMs. LR$^2$Bench comprises 850 samples across six Constraint Satisfaction Problems (CSPs) where reflective reasoning is crucial for deriving solutions that meet all given constraints. Each type of task focuses on distinct constraint patterns, such as knowledge-based, logical, and spatial constraints, providing a comprehensive evaluation of diverse problem-solving scenarios. Our extensive evaluation on both conventional LLMs and LRMs reveals that even the most advanced LRMs, such as DeepSeek-R1 and OpenAI o1-preview, struggle with tasks in LR$^2$Bench, achieving an average Exact Match score of only 20.0% and 23.6%, respectively. These findings underscore the significant room for improvement in the reflective reasoning capabilities of current LLMs.

cs.CL↗

arXiv2025

Detectability of the 21 cm signal with BINGO through cross-correlation with photometric surveys

21 cm intensity mapping (HI IM) can efficiently map large cosmic volumes with good redshift resolution, but systematics and foreground contamination pose major challenges for extracting accurate cosmological information. Cross-correlation with galaxy surveys offers an efficient mitigation strategy, as both datasets have largely uncorrelated systematics. We evaluate the detectability of the 21 cm signal from the BINGO radio telescope by cross-correlating with the LSST photometric survey, given their strong overlap in area and redshift. Using lognormal simulations, we model the cosmological signal in the BINGO frequency range (980 - 1260 MHz), incorporating thermal noise, foregrounds, and cleanup. The LSST simulations include uncertainties in photometric redshift (photo-z) and galaxy number density in the first three redshift intervals (mean redshift approximately equal to 0.25, 0.35, 0.45), corresponding to the expected performance after 10 years of the survey. We show that photo-z errors significantly increase the noise in the cross-correlation, reducing its statistical significance to levels comparable to those of the autocorrelation. Still, the HI signal remains detectable through the cross-correlation, even with photo-z uncertainties similar to those of the LSST. Our results corroborate the feasibility of this approach under realistic conditions and motivate further refinement of the current analysis methods.

astro-ph.CO↗

arXiv2025

Cosmological Remapping for Efficient Generation of 21 cm Intensity Mapping Mocks

We present a novel application of cosmological rescaling, or "remapping," to generate 21 cm intensity mapping mocks for different cosmologies. The remapping method allows for computationally efficient generation of N-body catalogs by rescaling existing simulations. In this work, we employ the remapping method to construct dark matter halo catalogs, starting from the Horizon Run 4 simulation with WMAP5 cosmology, and apply it to different target cosmologies, including WMAP7, Planck18 and Chevallier-Polarski-Linder (CPL) models. These catalogs are then used to simulate 21 cm intensity maps. We use the halo occupation distribution (HOD) method to populate halos with neutral hydrogen (HI) and derive 21 cm brightness temperature maps. Our results demonstrate the effectiveness of the remapping approach in generating cosmological simulations for large-scale structure studies, offering an alternative for testing observational data pipelines and performing cosmological parameter forecasts without the need for computationally expensive full N-body simulations. We also analyze the precision and limitations of the remapping, in light of the rescaling parameters $s$ and $s_m$, as well as the effects of the halo mass and box size thresholds.

astro-ph.CO↗

arXiv2025

Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model

The curse of multilinguality phenomenon is a fundamental problem of multilingual Large Language Models (LLMs), where the competition between massive languages results in inferior performance. It mainly comes from limited capacity and negative transfer between dissimilar languages. To address this issue, we propose a method to dynamically group and scale up the parameters of multilingual LLM while boosting positive transfer among similar languages. Specifically, the model is first tuned on monolingual corpus to determine the parameter deviation in each layer and quantify the similarity between languages. Layers with more deviations are extended to mixture-of-experts layers to reduce competition between languages, where one expert module serves one group of similar languages. Experimental results on 18 to 128 languages show that our method reduces the negative transfer between languages and significantly boosts multilingual performance with fewer parameters. Such language group specialization on experts benefits the new language adaptation and reduces the inference on the previous multilingual knowledge learned.

cs.CL↗

arXiv2025

SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner

We introduce **SWE-Flow**, a novel data synthesis framework grounded in Test-Driven Development (TDD). Unlike existing software engineering data that rely on human-submitted issues, **SWE-Flow** automatically infers incremental development steps directly from unit tests, which inherently encapsulate high-level requirements. The core of **SWE-Flow** is the construction of a Runtime Dependency Graph (RDG), which precisely captures function interactions, enabling the generation of a structured, step-by-step *development schedule*. At each step, **SWE-Flow** produces a partial codebase, the corresponding unit tests, and the necessary code modifications, resulting in fully verifiable TDD tasks. With this approach, we generated 16,061 training instances and 2,020 test instances from real-world GitHub projects, creating the **SWE-Flow-Eval** benchmark. Our experiments show that fine-tuning open model on this dataset significantly improves performance in TDD-based coding. To facilitate further research, we release all code, datasets, models, and Docker images at [Github](https://github.com/Hambaobao/SWE-Flow).

cs.CL↗

arXiv2025

Implicit Cross-Lingual Rewarding for Efficient Multilingual Preference Alignment

Direct Preference Optimization (DPO) has become a prominent method for aligning Large Language Models (LLMs) with human preferences. While DPO has enabled significant progress in aligning English LLMs, multilingual preference alignment is hampered by data scarcity. To address this, we propose a novel approach that $\textit{captures}$ learned preferences from well-aligned English models by implicit rewards and $\textit{transfers}$ them to other languages through iterative training. Specifically, we derive an implicit reward model from the logits of an English DPO-aligned model and its corresponding reference model. This reward model is then leveraged to annotate preference relations in cross-lingual instruction-following pairs, using English instructions to evaluate multilingual responses. The annotated data is subsequently used for multilingual DPO fine-tuning, facilitating preference knowledge transfer from English to other languages. Fine-tuning Llama3 for two iterations resulted in a 12.72% average improvement in Win Rate and a 5.97% increase in Length Control Win Rate across all training languages on the X-AlpacaEval leaderboard. Our findings demonstrate that leveraging existing English-aligned models can enable efficient and effective multilingual preference alignment, significantly reducing the need for extensive multilingual preference data. The code is available at https://github.com/ZNLP/Implicit-Cross-Lingual-Rewarding

cs.CL↗

arXiv2025

TokAlign: Efficient Vocabulary Adaptation via Token Alignment

Tokenization serves as a foundational step for Large Language Models (LLMs) to process text. In new domains or languages, the inefficiency of the tokenizer will slow down the training and generation of LLM. The mismatch in vocabulary also hinders deep knowledge transfer between LLMs like token-level distillation. To mitigate this gap, we propose an efficient method named TokAlign to replace the vocabulary of LLM from the token co-occurrences view, and further transfer the token-level knowledge between models. It first aligns the source vocabulary to the target one by learning a one-to-one mapping matrix for token IDs. Model parameters, including embeddings, are rearranged and progressively fine-tuned for the new vocabulary. Our method significantly improves multilingual text compression rates and vocabulary initialization for LLMs, decreasing the perplexity from 3.4$\text{e}^2$ of strong baseline methods to 1.2$\text{e}^2$ after initialization. Experimental results on models across multiple parameter scales demonstrate the effectiveness and generalization of TokAlign, which costs as few as 5k steps to restore the performance of the vanilla model. After unifying vocabularies between LLMs, token-level distillation can remarkably boost (+4.4% than sentence-level distillation) the base model, costing only 235M tokens.

cs.CL↗

arXiv2025

The Jiutian simulations for the CSST extra-galactic surveys

We provide an overview of the Jiutian simulations, a hybrid simulation suite for the China Space Survey Telescope (CSST) extragalactic surveys. It consists of four complementary modules: the primary runs with high resolutions with the fiducial concordance cosmology, the emulator runs exploring the parameter uncertainties around the fiducial cosmology, the reconstruction runs intended for recovering the observed Universe position by position, and the extension runs employing extended cosmologies beyond the standard model. For the primary runs, two independent pipelines are adopted to construct subhaloes and merger trees. On top of them, four sets of mock galaxy light-cone catalogs are produced from semi-analytical models and subhalo abundance matching, providing a variety of observational properties including galaxy SED, emission lines, lensing distortions, and mock images. The 129 emulator runs are used to train the CSST emulator, achieving one percent accuracy in predicting the matter power spectrum over $k\leq 10h{\rm Mpc}^{-1}$ and $z\leq 2$. The reconstruction runs employ a number of subgrid baryonic models to predict the evolution and galaxy population resembling certain regions in the real Universe with constrained initial conditions, enabling controlled investigation of galaxy formation on top of structure formation. The extension runs cover models with warm dark matter, $f(R)$ gravity, interacting dark energy, and nonzero neutrino masses, revealing differences in the cosmic structure under alternative cosmological models. We introduce the specifications for each run, the data products derived from them, the corresponding pipeline developments, and present some main tests. Using the primary runs, we also show that the subhalo peak mass functions of different levels are approximately universal. These simulations form a comprehensive and open library for CSST surveys and beyond.

astro-ph.CO↗

arXiv2025

Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning

Fine-tuning large language models (LLMs) on task-specific data is essential for their effective deployment. As dataset sizes grow, efficiently selecting optimal subsets for training becomes crucial to balancing performance and computational costs. Traditional data selection methods often require fine-tuning a scoring model on the target dataset, which is time-consuming and resource-intensive, or rely on heuristics that fail to fully leverage the model's predictive capabilities. To address these challenges, we propose Data Whisperer, an efficient, training-free, attention-based method that leverages few-shot in-context learning with the model to be fine-tuned. Comprehensive evaluations were conducted on both raw and synthetic datasets across diverse tasks and models. Notably, Data Whisperer achieves superior performance compared to the full GSM8K dataset on the Llama-3-8B-Instruct model, using just 10% of the data, and outperforms existing methods with a 3.1-point improvement and a 7.4$\times$ speedup. The code is available at https://github.com/gszfwsb/Data-Whisperer.

cs.CL↗

arXiv2025

Design and Validation of the Digital Receiver System for the next-generation radio interferometer

This paper presents the design and validation of a digital receiver system developed for the next-generation radio interferometer projects. The receiver supports 8 analog inputs with 12-bit, 4GHz sampling and performs real-time signal processing using FPGA-based channelization. Field experiments were conducted to observe the Sun, a satellite beacon, and Cassiopeia A. Interference fringes were analyzed and modeled. Time delay compensation was implemented in two ways: theoretical calculation and Gaussian Process Regression (GPR) fitting. Results show sub-nanosecond consistency between the two methods. The field experiments demonstrate the receiver's suitability for future radio telescopes such as the BINGO-ABDUS project.

astro-ph.IM↗

arXiv2025

Cosmological Constraints with Void Lensing I: the Simulation-Based Inference Framework

We present a Simulation-Based Inference (SBI) framework for cosmological parameter estimation via void lensing analysis. Despite the absence of an analytical model of void lensing, SBI can effectively learn posterior distributions through forward modeling of mock data. We develop a forward modeling pipeline that accounts for both cosmology and the galaxy-halo connection. By training a neural density estimator on simulated data, we infer the posteriors of two cosmological parameters, $Ω_m$ and $S_8$. Validation tests are conducted on posteriors derived from different cosmological parameters and a fiducial sample. The results demonstrate that SBI provides unbiased estimates of mean values and accurate uncertainties. These findings highlight the potential to apply void lensing analysis to observational data even without an analytical void lensing model.

astro-ph.CO↗

arXiv2025

Forecast measurement of the 21 cm global spectrum from Lunar orbit with the Vari-Zeroth-Order Polynomial (VZOP) method

The cosmic 21 cm signal serves as a crucial probe for studying the evolutionary history of the Universe. However, detecting the 21 cm signal poses significant challenges due to its extremely faint nature. To mitigate the interference from the Earth's radio frequency interference (RFI), the ground and the ionospheric effects, the Discovering the Sky at the Longest Wavelength (DSL) project will deploy a constellation of satellites in Lunar orbit, with its high-frequency daughter satellite tasked with detecting the global 21 cm signal from cosmic dawn and reionization era (CD/EoR). We intend to employ the Vari-Zeroth-Order Polynomial (VZOP) for foreground fitting and subtracting. We have studied the effect of thermal noise, thermal radiation from the Moon, the Lunar reflection, anisotropic frequency-dependent beam, inaccurate antenna beam pattern, and RFI contamination. We discovered that the RFI contamination can significantly affect the fitting process and thus prevent us from detecting the signal. Therefore, experimenting on the far side of the moon is crucial. We also discovered that using VZOP together with DSL, after 1080 orbits around the Moon, which takes about 103 days, we can successfully detect the CD/EoR 21 cm signal.

astro-ph.IM↗

arXiv2025

Designing Human-AI System for Legal Research: A Case Study of Precedent Search in Chinese Law

Recent advancements in AI technology have seen researchers and industry professionals actively exploring the application of AI tools in legal workflows. Despite this prevailing trend, legal practitioners found that AI tools had limited effectiveness in supporting everyday tasks, which can be partly attributed to their design. Typically, AI legal tools only offer end-to-end interaction: practitioners can only manipulate the input and output but have no control over the intermediate steps, raising concerns about AI tools' performance and ethical use. To design an effective AI legal tool, as a first step, we explore users' needs with one specific use case: precedent search. Through a qualitative study with five legal practitioners, we uncovered the precedent search workflow, the challenges they face using current systems, and their concerns and expectations regarding AI tools. We conclude our exploration with an initial prototype to reflect the design implications derived from our findings.

cs.HC↗

arXiv2025

SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking

Recent advancements in large language models (LLMs) with billions of parameters have improved performance in various applications, but their inference processes demand significant energy and computational resources. In contrast, the human brain, with approximately 86 billion neurons, is much more energy-efficient than LLMs with similar parameters. Inspired by this, we redesign 7$\sim$70 billion parameter LLMs using bio-plausible spiking mechanisms, emulating the efficient behavior of the human brain. We propose the first spiking large language model, SpikeLLM. Coupled with the proposed model, two essential approaches are proposed to improve spike training efficiency: Generalized Integrate-and-Fire (GIF) neurons to compress spike length from $T$ to $\frac{T}{L} \log_2 L$ bits, and an Optimal Brain Spiking framework to divide outlier channels and allocate different $T$ for GIF neurons, which further compresses spike length to approximate $log_2T$ bits. The necessity of spike-driven LLM is proved by comparison with quantized LLMs with similar operations. In the OmniQuant pipeline, SpikeLLM reduces 11.01% WikiText2 perplexity and improves 2.55% accuracy of common scene reasoning on a LLAMA-7B W4A4 model. In the GPTQ pipeline, SpikeLLM achieves direct additive in linear layers, significantly exceeding PB-LLMs.

cs.LG↗

arXiv2025

Loop I/NPS morphology predictions in the ultralong-wavelength band

Loop I/North Polar Spur (NPS) is the giant arc structure above the Galactic plane observed at radio wavelengths ($\lesssim 10$ GHz). There has been long-standing debate about its origin. While many people believe that it black consists of nearby supernova remnants (SNRs), some others consider it as a giant bubble close to the Galactic Center (GC), associated with the Fermi Bubble and the eROSITA X-ray bubble. At ultra-long wavelengths (wavelength $\gtrsim 10$ m or frequency $\lesssim 30$ MHz), particularly below $\sim 10$ MHz, the free-free absorption of the radio signal by diffuse electrons in the interstellar medium (ISM) becomes significant, resulting in different sky morphologies from those at higher frequencies. In this paper, we develop emissivity models for the two Loop I/NPS origin scenarios, and predict the Loop I/NPS morphology at ultra-long wavelengths in both scenarios, taking into account the free-free absorption effect.We find that in the SNRs model, the full Loop I/NPS will still be a bright arc even at $\sim 1$ MHz. In the GC model, the arc is fully visible only above $\sim 3$ MHz. While below this frequency, it is visible only at Galactic latitudes $b\gtrsim 30\degree$; the $b\lesssim 30 \degree$ part becomes invisible due to the absorption by the ISM electrons between the GC and the Sun. The upcoming space missions aiming at ultra-long wavelengths, such as the DSL and the FARSIDE, can potentially distinguish these two scenarios and provide decisive information about the origin of the Loop I/NPS.

astro-ph.GA↗

arXiv2025

A Multi-Mechanism Hybrid Model of Peaked-Spectrum Radio Sources

The peaked-spectrum (PS) sources exhibit turnover characteristics in their broad radio spectra. However, the mechanism underlying this phenomenon remains elusive. The two most common hypotheses are synchrotron self-absorption (SSA) and free-free absorption (FFA). By incorporating multiple absorption scenarios, we propose a multi-mechanism hybrid (MMH) model, which aligns well with current observational data and provides a good physical explanation. Using the GLEAM survey data, we identified a sample of 4,315 sources with peak frequencies approximately between 72--3000 MHz, most of which are MHz-peaked-spectrum sources (MPS). Our analysis shows that instead of SSA, the FFA is the dominant mechanism in producing the spectral turnover for most of the sources in this sample. The index of the optically thick spectrum alpha_thick has a lower boundary due to the FFA, and the steeper alpha_thick indicates a complex multi-absorption mechanism. In particular, the external FFA produces substantial alpha_thick, which exhibits a weak correlation with the peak frequency. Future ultra-long wavelength observations would also provide data on the spectrum of these sources at even lower frequencies. Determining the absorption mechanism that shaped the spectrum of these sources would be a crucial part of understanding their nature.

astro-ph.GA↗

arXiv2025

LOCAL: Learning with Orientation Matrix to Infer Causal Structure from Time Series Data

Discovering the underlying Directed Acyclic Graph (DAG) from time series observational data is highly challenging due to the dynamic nature and complex nonlinear interactions between variables. Existing methods typically search for the optimal DAG by optimizing an objective function but face scalability challenges, as their computational demands grow exponentially with the dimensional expansion of variables. To this end, we propose LOCAL, a highly efficient, easy-to-implement, and constraint-free method for recovering dynamic causal structures. LOCAL is the first attempt to formulate a quasi-maximum likelihood-based score function for learning the dynamic DAG equivalent to the ground truth. Building on this, we introduce two adaptive modules that enhance the algebraic characterization of acyclicity: Asymptotic Causal Mask Learning (ACML) and Dynamic Graph Parameter Learning (DGPL). ACML constructs causal masks using learnable priority vectors and the Gumbel-Sigmoid function, ensuring DAG formation while optimizing computational efficiency. DGPL transforms causal learning into decomposed matrix products, capturing dynamic causal structure in high-dimensional data and improving interpretability. Extensive experiments on synthetic and real-world datasets demonstrate that LOCAL significantly outperforms existing methods and highlight LOCAL's potential as a robust and efficient method for dynamic causal discovery.

cs.LG↗

arXiv2025

Language Imbalance Driven Rewarding for Multilingual Self-improving

Large Language Models (LLMs) have achieved state-of-the-art performance across numerous tasks. However, these advancements have predominantly benefited "first-class" languages such as English and Chinese, leaving many other languages underrepresented. This imbalance, while limiting broader applications, generates a natural preference ranking between languages, offering an opportunity to bootstrap the multilingual capabilities of LLM in a self-improving manner. Thus, we propose $\textit{Language Imbalance Driven Rewarding}$, where the inherent imbalance between dominant and non-dominant languages within LLMs is leveraged as a reward signal. Iterative DPO training demonstrates that this approach not only enhances LLM performance in non-dominant languages but also improves the dominant language's capacity, thereby yielding an iterative reward signal. Fine-tuning Meta-Llama-3-8B-Instruct over two iterations of this approach results in continuous improvements in multilingual performance across instruction-following and arithmetic reasoning tasks, evidenced by an average improvement of 7.46% win rate on the X-AlpacaEval leaderboard and 13.9% accuracy on the MGSM benchmark. This work serves as an initial exploration, paving the way for multilingual self-improvement of LLMs. The code is available at https://github.com/ZNLP/Language-Imbalance-Driven-Rewarding

cs.CL↗