arXiv subjects

Jiaqi Hu

Publications and source records attributed to Jiaqi Hu.

At least 37 records · Page 2Linked to original sources

arXiv2025

Merge-of-Thought Distillation

Efficient reasoning distillation for long chain-of-thought (CoT) models is increasingly constrained by the assumption of a single oracle teacher, despite the practical availability of multiple candidate teachers and growing CoT corpora. We revisit teacher selection and observe that different students have different "best teachers," and even for the same student, the best teacher can vary across datasets. Therefore, to unify multiple teachers' reasoning abilities into a student to overcome conflicts among various teachers' supervision, we propose Merge-of-Thought Distillation (MoT), a lightweight framework that alternates between teacher-specific supervised fine-tuning branches and weight-space merging of the resulting student variants. On competition math benchmarks, using only about 200 CoT samples, applying MoT to a Qwen3-14B student surpasses strong models including Deepseek-R1, Qwen3-32B, and OpenAI-O1, demonstrating substantial gains. Besides, MoT consistently outperforms the best single-teacher distillation, improves general reasoning beyond mathematics while reducing catastrophic forgetting, and shows robustness to distribution-shifted and peer-level teachers. Finally, we have demonstrated MoT possesses consensus CoT by eliminating teacher-specific inductive biases and inter-teacher conflicts while repeatedly reinforcing the learning of consensus reasoning features. These results position MoT as a simple, effective route to efficiently distilling long CoT capabilities from diverse teachers into compact students.

cs.LG↗

arXiv2025

Neural Operator Feedback for a First-Order PIDE with Spatially-Varying State Delay

A transport PDE with a spatial integral and recirculation with constant delay has been a benchmark for neural operator approximations of PDE backstepping controllers. Introducing a spatially-varying delay into the model gives rise to a gain operator defined through integral equations which the operator's input -- the varying delay function -- enters in previously unencountered manners, including in the limits of integration and as the inverse of the `delayED time' function. This, in turn, introduces novel mathematical challenges in estimating the operator's Lipschitz constant. The backstepping kernel function having two branches endows the feedback law with a two-branch structure, where only one of the two feedback branches depends on both of the kernel branches. For this rich feedback structure, we propose a neural operator approximation of such a two-branch feedback law and prove the approximator to be semiglobally practically stabilizing. With numerical results we illustrate the training of the neural operator and its stabilizing capability.

eess.SY↗

arXiv2025

LLaDA-MoE: A Sparse MoE Diffusion Language Model

We introduce LLaDA-MoE, a large language diffusion model with the Mixture-of-Experts (MoE) architecture, trained from scratch on approximately 20T tokens. LLaDA-MoE achieves competitive performance with significantly reduced computational overhead by maintaining a 7B-parameter capacity while activating only 1.4B parameters during inference. Our empirical evaluation reveals that LLaDA-MoE achieves state-of-the-art performance among diffusion language models with larger parameters, surpassing previous diffusion language models LLaDA, LLaDA 1.5, and Dream across multiple benchmarks. The instruct-tuned model LLaDA-MoE-7B-A1B-Instruct demonstrates capabilities comparable to Qwen2.5-3B-Instruct in knowledge understanding, code generation, mathematical reasoning, agent and alignment tasks, despite using fewer active parameters. Our results show that integrating a sparse MoE architecture into the training objective of masked diffusion language models still brings out MoE's strengths under efficient inference with few active parameters, and opens ample room for further exploration of diffusion language models. LLaDA-MoE models are available at Huggingface.

cs.CL↗

arXiv2025

No Pixel Left Behind: A Detail-Preserving Architecture for Robust High-Resolution AI-Generated Image Detection

The rapid growth of high-resolution, meticulously crafted AI-generated images poses a significant challenge to existing detection methods, which are often trained and evaluated on low-resolution, automatically generated datasets that do not align with the complexities of high-resolution scenarios. A common practice is to resize or center-crop high-resolution images to fit standard network inputs. However, without full coverage of all pixels, such strategies risk either obscuring subtle, high-frequency artifacts or discarding information from uncovered regions, leading to input information loss. In this paper, we introduce the High-Resolution Detail-Aggregation Network (HiDA-Net), a novel framework that ensures no pixel is left behind. We use the Feature Aggregation Module (FAM), which fuses features from multiple full-resolution local tiles with a down-sampled global view of the image. These local features are aggregated and fused with global representations for final prediction, ensuring that native-resolution details are preserved and utilized for detection. To enhance robustness against challenges such as localized AI manipulations and compression, we introduce Token-wise Forgery Localization (TFL) module for fine-grained spatial sensitivity and JPEG Quality Factor Estimation (QFE) module to disentangle generative artifacts from compression noise explicitly. Furthermore, to facilitate future research, we introduce HiRes-50K, a new challenging benchmark consisting of 50,568 images with up to 64 megapixels. Extensive experiments show that HiDA-Net achieves state-of-the-art, increasing accuracy by over 13% on the challenging Chameleon dataset and 10% on our HiRes-50K.

cs.CV↗

arXiv2025

Reinforcement Learning with Rubric Anchors

Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a powerful paradigm for enhancing Large Language Models (LLMs), exemplified by the success of OpenAI's o-series. In RLVR, rewards are derived from verifiable signals-such as passing unit tests in code generation or matching correct answers in mathematical reasoning. While effective, this requirement largely confines RLVR to domains with automatically checkable outcomes. To overcome this, we extend the RLVR paradigm to open-ended tasks by integrating rubric-based rewards, where carefully designed rubrics serve as structured, model-interpretable criteria for automatic scoring of subjective outputs. We construct, to our knowledge, the largest rubric reward system to date, with over 10,000 rubrics from humans, LLMs, or a hybrid human-LLM collaboration. Implementing rubric-based RL is challenging; we tackle these issues with a clear framework and present an open-sourced Qwen-30B-A3B model with notable gains: 1) With only 5K+ samples, our system improves by +5.2% on open-ended benchmarks (especially humanities), outperforming a 671B DeepSeek-V3 model by +2.4%, while preserving general and reasoning abilities. 2) Our method provides fine-grained stylistic control, using rubrics as anchors to mitigate the "AI-like" tone and produce more human-like, expressive responses. We share key lessons in rubric construction, data selection, and training, and discuss limitations and future releases.

cs.AI↗

arXiv2025

Distributed Quantized Average Consensus in Open Multi-Agent Systems with Dynamic Communication Links

In this paper, we focus on the distributed quantized average consensus problem in open multi-agent systems consisting of communication links that change dynamically over time. Open multi-agent systems exhibiting the aforementioned characteristic are referred to as \textit{open dynamic multi-agent systems} in this work. We present a distributed algorithm that enables active nodes in the open dynamic multi-agent system to calculate the quantized average of their initial states. Our algorithm consists of the following advantages: (i) ensures efficient communication by enabling nodes to exchange quantized valued messages, and (ii) exhibits finite time convergence to the desired solution. We establish the correctness of our algorithm and we present necessary and sufficient topological conditions for it to successfully solve the quantized average consensus problem in an open dynamic multi-agent system. Finally, we illustrate the performance of our algorithm with numerical simulations.

eess.SY↗

arXiv2025

Nonlocal Coherent Optical Nonlinearities of a Macroscopic Quantum System

The optical responses of solids are typically understood to be local in space. Whether locality holds for the optical response of a macroscopic quantum system has, however, been largely unexplored. Here, we use multidimensional coherent spectroscopy at the optical diffraction limit to demonstrate nonlocal optical nonlinearities in a semiconductor microcavity. These nonlocal optical responses are both coherent and quantum in nature, deriving from the macroscopic length scale of confined exciton-polariton wavefunctions.

physics.optics↗

arXiv2025

StarGen: A Spatiotemporal Autoregression Framework with Video Diffusion Model for Scalable and Controllable Scene Generation

Recent advances in large reconstruction and generative models have significantly improved scene reconstruction and novel view generation. However, due to compute limitations, each inference with these large models is confined to a small area, making long-range consistent scene generation challenging. To address this, we propose StarGen, a novel framework that employs a pre-trained video diffusion model in an autoregressive manner for long-range scene generation. The generation of each video clip is conditioned on the 3D warping of spatially adjacent images and the temporally overlapping image from previously generated clips, improving spatiotemporal consistency in long-range scene generation with precise pose control. The spatiotemporal condition is compatible with various input conditions, facilitating diverse tasks, including sparse view interpolation, perpetual view generation, and layout-conditioned city generation. Quantitative and qualitative evaluations demonstrate StarGen's superior scalability, fidelity, and pose accuracy compared to state-of-the-art methods. Project page: https://zju3dv.github.io/StarGen.

cs.CV↗

arXiv2025

Absence of dehydration due to superionic transition at Earth's core-mantle boundary

The properties and stability of hydrous phases are key to unraveling the mysteries of the water cycle in Earth's interior. Under the deep lower mantle conditions, hydrous phases transition into a superionic state. However, the influence of the superionic effect on their stability and dehydration processes remains poorly understood. Using ab initio calculations and deep-learning potential molecular dynamics simulations, we discovered a doubly superionic transition in delta-AlOOH, characterized by the highly diffusive behavior of ionic hydrogen and aluminum within the oxygen sub-lattice. These highly diffusive elements contribute significant external entropy into the system, resulting in exceptional thermostability. Free energy calculations indicate that dehydration is energetically and kinetically unfavorable when water exists in a superionic state under core-mantle boundary (CMB) conditions. Consequently, water can accumulate in the deep lower mantle over Earth's history. This deep water reservoir plays a crucial role in the global deep water and hydrogen cycles.

physics.geo-ph↗

arXiv2025

Consumption-portfolio choice with preferences for liquid assets

This paper investigates an infinite horizon, discounted, consumption-portfolio problem in a market with one bond, one liquid risky asset, and one illiquid risky asset with proportional transaction costs. We consider an agent with liquidity preference, modeled by a Cobb-Douglas utility function that includes the liquid wealth. We analyze the properties of the value function and divide the solvency region into three regions: the buying region, the no-trading region, and the selling region, and prove that all three regions are non-empty. We mathematically characterize and numerically solve the optimal policy and prove its optimality. Our numerical analysis sheds light on the impact of various parameters on the optimal policy, and some intuition and economic insights behind it are also analyzed. We find that liquidity preference encourages agents to retain more liquid wealth and inhibits consumption, and may even result in a negative allocation to the illiquid asset. The liquid risky asset not only affects the location of the three regions but also has an impact on consumption. However, whether this impact on consumption is promoted or inhibited depends on the degree of risk aversion of agents.

q-fin.PM↗

arXiv2025

N-player and mean field games among fund managers considering excess logarithmic returns

This paper studies the competition among multiple fund managers with relative performance over the excess logarithmic return. Fund managers compete with each other and have expected utility or mean-variance criteria for excess logarithmic return. Each fund manager possesses a unique risky asset, and all fund managers can also invest in a public risk-free asset and a public risk asset. We construct both an $n$-player game and a mean field game (MFG) to address the competition problem under these two criteria. We explicitly define and rigorously solve the equilibrium and mean field equilibrium (MFE) for each criteria. In the four models, the excess logarithmic return as the evaluation criterion of the fund leads to the { allocation fractions} being constant. The introduction of the public risky asset yields different outcomes, with competition primarily affecting the investment in public assets, particularly evident in the MFG. We demonstrate that the MFE of the MFG represents the limit of the $n$-player game's equilibrium as the competitive scale $n$ approaches infinity. Finally, the sensitivity analyses of the equilibrium are given.

q-fin.PM↗

arXiv2025

Neural Operator based Reinforcement Learning for Control of first-order PDEs with Spatially-Varying State Delay

Control of distributed parameter systems affected by delays is a challenging task, particularly when the delays depend on spatial variables. The idea of integrating analytical control theory with learning-based control within a unified control scheme is becoming increasingly promising and advantageous. In this paper, we address the problem of controlling an unstable first-order hyperbolic PDE with spatially-varying delays by combining PDE backstepping control strategies and deep reinforcement learning (RL). To eliminate the assumption on the delay function required for the backstepping design, we propose a soft actor-critic (SAC) architecture incorporating a DeepONet to approximate the backstepping controller. The DeepONet extracts features from the backstepping controller and feeds them into the policy network. In simulations, our algorithm outperforms the baseline SAC without prior backstepping knowledge and the analytical controller.

cs.AI↗

arXiv2024

Thermodynamic signatures of a potential Fulde-Ferrell-Larkin Ovchinnikov state in the isotropic superconductor Ti4Ir2O

This study presents an investigation of the isotropic superconductor Ti4Ir2O, focusing on its magnetic field vs. temperature phase diagram near and above the Pauli limit for superconductivity. The data exhibits characteristic features that align with the formation of a Fulde-Ferrell-Larkin-Ovchinnikov (FFLO) state, including a deviation of the upper critical field line from the standard WHH model upon approaching the Pauli limiting field and a phase transition line separating the ordinary low-field superconducting phase from an unusual high field superconducting state above the Pauli limit. We discuss why the upper critical field line in this isotropic superconductor can approach the Pauli limit, necessitating a particularly high orbital limit for superconductivity. The research also identifies peculiar features in the specific heat data set taken at 0.6 K, which may be an example of the segmentation of the FFLO phase with multiple q phases. The findings provide an explanation for the violation of the Pauli limit in this superconductor, adding a fully isotropic superconductor to the list of potential FFLO superconductors and providing new insights for contemporary theories of the FFLO state.

cond-mat.supr-con↗

arXiv2024

Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decoding

Although Visual-Language Models (VLMs) have shown impressive capabilities in tasks like visual question answering and image captioning, they still struggle with hallucinations. Analysis of attention distribution in these models shows that VLMs tend to processing textual tokens rather than visual tokens. This imbalance of attention distribution causes VLMs to favor textual knowledge in the case of multimodal knowledge conflicts, resulting in differences from the image information. In this paper, we propose Re-Balancing Contrastive Decoding (RBD) method, which employs textual and visual branches to recalibrate attention distribution in VLMs. Specifically, the textual branch injects image noise to stimulate the model's dependency on text, thereby reducing textual bias. Concurrently, the visual branch focuses on the selection of significant tokens, refining the attention mechanism to highlight the primary subject. This dual-branch strategy enables the RBD method to diminish textual bias while enhancing visual information. Experimental results demonstrate that our method, RBD, outperforms the existing methods by the CHAIR and POPE metrics, mitigate hallucinations without reducing the model's general capabilities.

cs.CV↗

arXiv2024

FALIP: Visual Prompt as Foveal Attention Boosts CLIP Zero-Shot Performance

CLIP has achieved impressive zero-shot performance after pre-training on a large-scale dataset consisting of paired image-text data. Previous works have utilized CLIP by incorporating manually designed visual prompts like colored circles and blur masks into the images to guide the model's attention, showing enhanced zero-shot performance in downstream tasks. Although these methods have achieved promising results, they inevitably alter the original information of the images, which can lead to failure in specific tasks. We propose a train-free method Foveal-Attention CLIP (FALIP), which adjusts the CLIP's attention by inserting foveal attention masks into the multi-head self-attention module. We demonstrate FALIP effectively boosts CLIP zero-shot performance in tasks such as referring expressions comprehension, image classification, and 3D point cloud recognition. Experimental results further show that FALIP outperforms existing methods on most metrics and can augment current methods to enhance their performance.

cs.CV↗

arXiv2024

Unleashing the Expressive Power of Pulse-Based Quantum Neural Networks

Quantum machine learning (QML) based on Noisy Intermediate-Scale Quantum (NISQ) devices hinges on the optimal utilization of limited quantum resources. While gate-based QML models are user-friendly for software engineers, their expressivity is restricted by the permissible circuit depth within a finite coherence time. In contrast, pulse-based models enable the construction of "infinitely" deep quantum neural networks within the same time, which may unleash greater expressive power for complex learning tasks. In this paper, this potential is investigated from the perspective of quantum control theory. We first indicate that the nonlinearity of pulse-based models comes from the encoding process that can be viewed as the continuous limit of data-reuploading in gate-based models. Subsequently, we prove that the pulse-based model can approximate arbitrary nonlinear functions when the underlying physical system is ensemble controllable. Under this condition, numerical simulations demonstrate the enhanced expressivity by either increasing the pulse length or the number of qubits. As anticipated, we show through numerical examples that the pulse-based model can unleash more expressive power compared to the gate-based model. These findings lay a theoretical foundation for understanding and designing expressive QML models using NISQ devices.

quant-ph↗

arXiv2024

Electrically controlled photonic circuits of field-induced dipolaritons with huge nonlinearities

Electrically controlled photonic circuits hold promise for information technologies with greatly improved energy efficiency and quantum information processing capabilities. However, weak nonlinearity and electrical response of typical photonic materials have been two critical challenges. Therefore hybrid electronic-photonic systems, such as semiconductor exciton-polaritons, have been intensely investigated for their potential to allow higher nonlinearity and electrical control, with limited success so far. Here we demonstrate an electrically-gated waveguide architecture for dipolar-polaritons that allows enhanced and electrically-controllable polariton nonlinearities, enabling an electrically-tuned reflecting switch (mirror) and transistor of the dipolar-polaritons. The polariton transistor displays blockade and anti-blockade by compressing a dilute dipolar-polariton pulse exhibiting very strong dipolar interactions. The large nonlinearities are explained using a simple density-dependent polarization field that very effectively screens the external electric field, with an order of magnitude enhancement of the nonlinearities as compared to fixed dipoles. We project that a quantum blockade at the single polariton level is feasible in such a device.

physics.optics↗

arXiv2024

Data Contamination Calibration for Black-box LLMs

The rapid advancements of Large Language Models (LLMs) tightly associate with the expansion of the training data size. However, the unchecked ultra-large-scale training sets introduce a series of potential risks like data contamination, i.e. the benchmark data is used for training. In this work, we propose a holistic method named Polarized Augment Calibration (PAC) along with a new to-be-released dataset to detect the contaminated data and diminish the contamination effect. PAC extends the popular MIA (Membership Inference Attack) -- from machine learning community -- by forming a more global target at detecting training data to Clarify invisible training data. As a pioneering work, PAC is very much plug-and-play that can be integrated with most (if not all) current white- and black-box LLMs. By extensive experiments, PAC outperforms existing methods by at least 4.5%, towards data contamination detection on more 4 dataset formats, with more than 10 base LLMs. Besides, our application in real-world scenarios highlights the prominent presence of contamination and related issues.

cs.LG↗