arXiv subjects

Qi Jia

Publications and source records attributed to Qi Jia.

At least 55 records · Page 3Linked to original sources

arXiv2025

A Multi-To-One Interview Paradigm for Efficient MLLM Evaluation

The rapid progress of Multi-Modal Large Language Models (MLLMs) has spurred the creation of numerous benchmarks. However, conventional full-coverage Question-Answering evaluations suffer from high redundancy and low efficiency. Inspired by human interview processes, we propose a multi-to-one interview paradigm for efficient MLLM evaluation. Our framework consists of (i) a two-stage interview strategy with pre-interview and formal interview phases, (ii) dynamic adjustment of interviewer weights to ensure fairness, and (iii) an adaptive mechanism for question difficulty-level chosen. Experiments on different benchmarks show that the proposed paradigm achieves significantly higher correlation with full-coverage results than random sampling, with improvements of up to 17.6% in PLCC and 16.7% in SRCC, while reducing the number of required questions. These findings demonstrate that the proposed paradigm provides a reliable and efficient alternative for large-scale MLLM benchmarking.

cs.CL↗

arXiv2025

Droplet3D: Commonsense Priors from Videos Facilitate 3D Generation

Scaling laws have validated the success and promise of large-data-trained models in creative generation across text, image, and video domains. However, this paradigm faces data scarcity in the 3D domain, as there is far less of it available on the internet compared to the aforementioned modalities. Fortunately, there exist adequate videos that inherently contain commonsense priors, offering an alternative supervisory signal to mitigate the generalization bottleneck caused by limited native 3D data. On the one hand, videos capturing multiple views of an object or scene provide a spatial consistency prior for 3D generation. On the other hand, the rich semantic information contained within the videos enables the generated content to be more faithful to the text prompts and semantically plausible. This paper explores how to apply the video modality in 3D asset generation, spanning datasets to models. We introduce Droplet3D-4M, the first large-scale video dataset with multi-view level annotations, and train Droplet3D, a generative model supporting both image and dense text input. Extensive experiments validate the effectiveness of our approach, demonstrating its ability to produce spatially consistent and semantically plausible content. Moreover, in contrast to the prevailing 3D solutions, our approach exhibits the potential for extension to scene-level applications. This indicates that the commonsense priors from the videos significantly facilitate 3D creation. We have open-sourced all resources including the dataset, code, technical framework, and model weights: https://dropletx.github.io/.

cs.CV↗

arXiv2025

Sycophancy under Pressure: Evaluating and Mitigating Sycophantic Bias via Adversarial Dialogues in Scientific QA

Large language models (LLMs), while increasingly used in domains requiring factual rigor, often display a troubling behavior: sycophancy, the tendency to align with user beliefs regardless of correctness. This tendency is reinforced by preference-based alignment techniques that optimize for user satisfaction but can undermine truthfulness. While relatively benign in casual dialogue, sycophancy poses serious risks in high-stakes settings such as scientific question answering (QA), where model outputs may shape collaborative reasoning, decision-making, and knowledge formation. Despite its importance, this phenomenon remains underexamined in factual QA contexts. We address this gap by introducing a unified evaluation framework to quantify the impact of sycophantic context on model behavior in scientific QA, measuring how much user-imposed social pressure distorts model outputs. The framework incorporates adversarial prompting setups and targeted metrics, such as misleading resistance and sycophancy resistance, that capture a model's ability to maintain factual consistency under misleading cues. Systematic evaluations across open-source and proprietary models reveal pervasive sycophantic tendencies, driven more by alignment strategy than by model size. To mitigate this issue, we propose Pressure-Tune, a lightweight post-training method that fine-tunes models on synthetic adversarial dialogues paired with chain-of-thought rationales. These rationales reject user misinformation while reinforcing factual commitments. Experiments on challenging scientific QA benchmarks show that Pressure-Tune significantly enhances sycophancy resistance without compromising accuracy or responsiveness to valid feedback, offering a practical pathway toward more truthful and principled model behavior.

cs.CL↗

arXiv2025

User-centric Subjective Leaderboard by Customizable Reward Modeling

Existing benchmarks for large language models (LLMs) predominantely focus on assessing their capabilities through verifiable tasks. Such objective and static benchmarks offer limited utility for practical LLM selection, making it difficult for users to find suitable models for their individual needs. To bridge this gap, we present the first User-Centric Subjective Leaderboard (USL), which provides a preference-driven, dynamic ranking of LLMs across diverse real-world scenarios. Our work is built upon a thorough investigation of real human preference data, involving more than 10K subjective queries. Our investigation reveals significant diversity and contradictions in human preferences, which limit the effectiveness of state-of-the-art reward models. To address this, we introduce Customizable Reward Models (CRMs). With only 4B parameters, our CRM surpasses the performance of leading models such as GPT-4.1 and Gemini-2.5-pro, showing exceptional generalization capabilities across new topics and criteria. The USL, powered by CRMs, exhibits strong negative correlations to contradictory preferences.

cs.CL↗

arXiv2025

Can Large Models Fool the Eye? A New Turing Test for Biological Animation

Evaluating the abilities of large models and manifesting their gaps are challenging. Current benchmarks adopt either ground-truth-based score-form evaluation on static datasets or indistinct textual chatbot-style human preferences collection, which may not provide users with immediate, intuitive, and perceptible feedback on performance differences. In this paper, we introduce BioMotion Arena, a novel framework for evaluating large language models (LLMs) and multimodal large language models (MLLMs) via visual animation. Our methodology draws inspiration from the inherent visual perception of motion patterns characteristic of living organisms that utilizes point-light source imaging to amplify the performance discrepancies between models. Specifically, we employ a pairwise comparison evaluation and collect more than 45k votes for 53 mainstream LLMs and MLLMs on 90 biological motion variants. Data analyses show that the crowd-sourced human votes are in good agreement with those of expert raters, demonstrating the superiority of our BioMotion Arena in offering discriminative feedback. We also find that over 90\% of evaluated models, including the cutting-edge open-source InternVL3 and proprietary Claude-4 series, fail to produce fundamental humanoid point-light groups, much less smooth and biologically plausible motions. This enables BioMotion Arena to serve as a challenging benchmark for performance visualization and a flexible evaluation framework without restrictions on ground-truth.

cs.CV↗

arXiv2025

Demonstration of Electron-Mediated Voltage-Controlled Exchange Coupling in Perpendicular Magnetic Tunnel Junctions

Electron-mediated voltage control of exchange coupling (EM-VCEC) has been proposed as a mechanism for magnetization switching via modulation of spin-dependent electron reflection. However, its experimental verification has been challenging due to the coexistence of slower, voltage-induced ionic effects. Here, we fabricate magnetic tunnel junction (MTJ) devices that enable nanosecond timescale voltage application. Our results reveal rapid exchange coupling modulation on the nanosecond timescale, consistent with an electronic origin. The observed enhancement and saturation at low temperatures further rule out ionic migration, conclusively confirming the electronic nature of the mechanism. These results establish EM-VCEC as a viable mechanism for fast and energy-efficient voltage-driven magnetic switching.

physics.app-ph↗

arXiv2025

Info-Coevolution: An Efficient Framework for Data Model Coevolution

Machine learning relies heavily on data, yet the continuous growth of real-world data poses challenges for efficient dataset construction and training. A fundamental yet unsolved question is: given our current model and data, does a new data (sample/batch) need annotation/learning? Conventional approaches retain all available data, leading to non-optimal data and training efficiency. Active learning aims to reduce data redundancy by selecting a subset of samples to annotate, while it increases pipeline complexity and introduces bias. In this work, we propose Info-Coevolution, a novel framework that efficiently enables models and data to coevolve through online selective annotation with no bias. Leveraging task-specific models (and open-source models), it selectively annotates and integrates online and web data to improve datasets efficiently. For real-world datasets like ImageNet-1K, Info-Coevolution reduces annotation and training costs by 32\% without performance loss. It is able to automatically give the saving ratio without tuning the ratio. It can further reduce the annotation ratio to 50\% with semi-supervised learning. We also explore retrieval-based dataset enhancement using unlabeled open-source data. Code is available at https://github.com/NUS-HPC-AI-Lab/Info-Coevolution/.

cs.LG↗

arXiv2025

Period-luminosity and period-luminosity-metallicity relation for $δ$ Scuti Stars

$δ$ Scuti ($δ$ Sct) stars are potential distance tracers for studying the Milky Way structure. We conduct a comprehensive analysis of the period-luminosity (PL) and period-luminosity-metallicity (PLZ) relation for $δ$ Sct stars, integrating data from the Zwicky Transient Facility (ZTF), the Transiting Exoplanet Survey Satellite (TESS), Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST), Apache Point Observatory Galactic Evolution Experiment (APOGEE), and Gaia. To mitigate the impact of the Gaia parallax zero point offset, we applied a correction method, determining the optimal zero point value to be $zp_\varpi = 35 \pm 2 \, μ\text{as}$. Using the three best bands, by varying the parallax error threshold, we found that the total error of the PLR zero point was minimized to 0.9\% at a parallax error threshold of 6\%. With this threshold, we derived the PL and PLZ relation for nine bands (from optical to mid-infrared) and five Wesenheit bands. Through our analysis, we conclude that the influence of metallicity on the PLR of $δ$ Sct stars is not significant, and the differences across various bands are minimal.

astro-ph.SR↗

arXiv2025

Not Just Object, But State: Compositional Incremental Learning without Forgetting

Most incremental learners excessively prioritize coarse classes of objects while neglecting various kinds of states (e.g. color and material) attached to the objects. As a result, they are limited in the ability to reason fine-grained compositionality of state-object pairs. To remedy this limitation, we propose a novel task called Compositional Incremental Learning (composition-IL), enabling the model to recognize state-object compositions as a whole in an incremental learning fashion. Since the lack of suitable benchmarks, we re-organize two existing datasets and make them tailored for composition-IL. Then, we propose a prompt-based Composition Incremental Learner (CompILer), to overcome the ambiguous composition boundary problem which challenges composition-IL largely. Specifically, we exploit multi-pool prompt learning, which is regularized by inter-pool prompt discrepancy and intra-pool prompt diversity. Besides, we devise object-injected state prompting by using object prompts to guide the selection of state prompts. Furthermore, we fuse the selected prompts by a generalized-mean strategy, to eliminate irrelevant information learned in the prompts. Extensive experiments on two datasets exhibit state-of-the-art performance achieved by CompILer.

cs.CV↗

arXiv2025

Period--Luminosity Relations for Double-mode $δ$ Sct Stars

Previous studies of the Period--Luminosity relations (PLRs) of Delta Scuti ($δ$ Sct) stars have focused on those with a single pulsation mode. However, for $δ$ Sct stars with many different pulsation modes, classifying a single mode is difficult. In this study, an all-sky dataset is constructed using double-mode $δ$ Sct stars from ZTF and OGLE, and is used to determine F-mode and 1O-mode PLRs for eight single bands and six Wesenheit bands. In the $W1$ band, the PLR dispersion is about 0.171 mag and the total zero point error is 1\%. Our results show that to accurately classify the 1O modes of $δ$ Sct stars requires authentication based on multiple modes. Classification based on amplitude alone leads to impure 1O-mode $δ$ Sct stars and significant deviations in the PLRs. We compare the PLRs of the different sequences in the Petersen diagram and find that they are consistent after a strict criterion filtering, suggesting that their evolutionary state is similar. In addition, we find a weak period--metallicity relation for double-mode $δ$ Sct stars, unlike double-mode RR Lyrae stars. As distance tracers, large-amplitude F-mode $δ$ Sct and double-mode $δ$ Sct stars are the most suitable of the $δ$ Sct family.

astro-ph.SR↗

arXiv2025

Coexistence of unconventional spin Hall effect and antisymmetric planar Hall effect in IrO$_2$

Crystal symmetry plays an important role in the Hall effects. Unconventional spin Hall effect (USHE), characterized by Dresselhaus and out-of-plane spins, has been observed in materials with low crystal symmetry. Recently, antisymmetric planar Hall effect (APHE) was discovered in rutile RuO2 and IrO2 (101) thin films, which also exhibit low crystal symmetry. In this study, we report the observation of both USHE and APHE in IrO2 (111) films, using spin-torque ferromagnetic resonance (ST-FMR) and harmonic Hall measurements, respectively. Notably, the unconventional spin torque efficiency from Dresselhaus spin was more than double that of a previous report. Additionally, the temperature dependence of APHE suggests that it arises from the Lorentz force, constrained by crystal symmetry. Symmetry analysis supports the coexistence of USHE and APHE and demonstrates that both originate from the crystal symmetry of IrO2 (111), paving the way for a deeper understanding of Hall effects and related physical phenomena.

cond-mat.mes-hall↗

arXiv2025

DropletVideo: A Dataset and Approach to Explore Integral Spatio-Temporal Consistent Video Generation

Spatio-temporal consistency is a critical research topic in video generation. A qualified generated video segment must ensure plot plausibility and coherence while maintaining visual consistency of objects and scenes across varying viewpoints. Prior research, especially in open-source projects, primarily focuses on either temporal or spatial consistency, or their basic combination, such as appending a description of a camera movement after a prompt without constraining the outcomes of this movement. However, camera movement may introduce new objects to the scene or eliminate existing ones, thereby overlaying and affecting the preceding narrative. Especially in videos with numerous camera movements, the interplay between multiple plots becomes increasingly complex. This paper introduces and examines integral spatio-temporal consistency, considering the synergy between plot progression and camera techniques, and the long-term impact of prior content on subsequent generation. Our research encompasses dataset construction through to the development of the model. Initially, we constructed a DropletVideo-10M dataset, which comprises 10 million videos featuring dynamic camera motion and object actions. Each video is annotated with an average caption of 206 words, detailing various camera movements and plot developments. Following this, we developed and trained the DropletVideo model, which excels in preserving spatio-temporal coherence during video generation. The DropletVideo dataset and model are accessible at https://dropletx.github.io.

cs.CV↗

arXiv2024

Boosting LLM via Learning from Data Iteratively and Selectively

Datasets nowadays are generally constructed from multiple sources and using different synthetic techniques, making data de-noising and de-duplication crucial before being used for post-training. In this work, we propose to perform instruction tuning by iterative data selection (\ApproachName{}). We measure the quality of a sample from complexity and diversity simultaneously. Instead of calculating the complexity score once for all before fine-tuning, we highlight the importance of updating this model-specific score during fine-tuning to accurately accommodate the dynamic changes of the model. On the other hand, the diversity score is defined on top of the samples' responses under the consideration of their informativeness. IterIT integrates the strengths of both worlds by iteratively updating the complexity score for the top-ranked samples and greedily selecting the ones with the highest complexity-diversity score. Experiments on multiple instruction-tuning data demonstrate consistent improvements of IterIT over strong baselines. Moreover, our approach also generalizes well to domain-specific scenarios and different backbone models. All resources will be available at https://github.com/JiaQiSJTU/IterIT.

cs.CL↗

arXiv2024

Energy Efficient Stochastic Signal Manipulation in Superparamagnetic Tunnel Junctions via Voltage-Controlled Exchange Coupling

Superparamagnetic tunnel junctions (sMTJs) are emerging as promising components for stochastic units in neuromorphic computing, owing to their tunable random switching behavior. Conventional MTJ control methods, such as spin-transfer torque (STT) and spin-orbit torque (SOT), often require substantial power. Here, we introduce the voltage-controlled exchange coupling (VCEC) mechanism, enabling switching between antiparallel and parallel states in sMTJs with an ultralow power consumption of only 40 nW, approximately two orders of magnitude lower than conventional STT-based sMTJs. This mechanism yields a sigmoid-shaped output response, making it ideally suited for neuromorphic computing applications. Furthermore, we validate the feasibility of integrating VCEC with the SOT current control, offering an additional dimension for magnetic state manipulation. This work marks the first practical demonstration of VCEC effect in sMTJs, highlighting its potential as a low-power control solution for probabilistic bits in advanced computing systems.

physics.app-ph↗

arXiv2024

Giant spin Hall effect with multi-directional spin components in Ni4W

Spin-orbit torque (SOT) can be used to efficiently manipulate the magnetic state of magnetic materials, which is an essential element for memory and logic applications. Due to symmetry constraints, only in-plane spins can be injected into the ferromagnet from the underlying SOT layer for conventional SOT materials such as heavy metals and topological materials. Through the use of materials with low symmetries, or other symmetry breaking approaches, unconventional spin currents with out-of-plane polarization has been demonstrated and enabled field-free deterministic switching of perpendicular magnetization. Despite this progress, the SOT efficiency of these materials has typically remained low. Here, we report a giant SOT efficiency of 0.85 in sputtered Ni4W/CoFeB heterostructure at room temperature, as evaluated by second harmonic Hall measurements. In addition, due to the low crystal symmetry of Ni4W, unconventional out-of-plane and Dresselhaus-like spin components were observed. Macro-spin simulation suggests our spin Hall tensor to provide about an order of magnitude improvement in the magnetization switching efficiency, thus broadening the path towards energy efficient spintronic devices using low-symmetry materials.

cond-mat.mes-hall↗

arXiv2024

Kun: Answer Polishment for Chinese Self-Alignment with Instruction Back-Translation

In this paper, we introduce Kun, a novel approach for creating high-quality instruction-tuning datasets for large language models (LLMs) without relying on manual annotations. Adapting a self-training algorithm based on instruction back-translation and answer polishment, Kun leverages unlabelled data from diverse sources such as Wudao, Wanjuan, and SkyPile to generate a substantial dataset of over a million Chinese instructional data points. This approach significantly deviates from traditional methods by using a self-curation process to refine and select the most effective instruction-output pairs. Our experiments with the 6B-parameter Yi model across various benchmarks demonstrate Kun's robustness and scalability. Our method's core contributions lie in its algorithmic advancement, which enhances data retention and clarity, and its innovative data generation approach that substantially reduces the reliance on costly and time-consuming manual annotations. This methodology presents a scalable and efficient solution for improving the instruction-following capabilities of LLMs, with significant implications for their application across diverse fields. The code and dataset can be found at https://github.com/Zheng0428/COIG-Kun

cs.CL↗

arXiv2024

SimulBench: Evaluating Language Models with Creative Simulation Tasks

We introduce SimulBench, a benchmark designed to evaluate large language models (LLMs) across a diverse collection of creative simulation scenarios, such as acting as a Linux terminal or playing text games with users. While these simulation tasks serve as effective measures of an LLM's general intelligence, they are seldom incorporated into existing benchmarks. A major challenge is to develop an evaluation framework for testing different LLMs fairly while preserving the multi-round interactive nature of simulation tasks between users and AI. To tackle this issue, we suggest using a fixed LLM as a user agent to engage with an LLM to collect dialogues first under different tasks. Then, challenging dialogue scripts are extracted for evaluating different target LLMs. To facilitate automatic assessment on \DataName{}, GPT-4 is employed as the evaluator, tasked with reviewing the quality of the final response generated by the target LLMs given multi-turn dialogue scripts. Our comprehensive experiments indicate that these simulation tasks continue to pose a significant challenge with their unique natures and show the gap between proprietary models and the most advanced open LLMs. For example, GPT-4-turbo outperforms LLaMA-3-70b-Chat on 18.55\% more cases.

cs.CL↗

arXiv2024

Unseen No More: Unlocking the Potential of CLIP for Generative Zero-shot HOI Detection

Zero-shot human-object interaction (HOI) detector is capable of generalizing to HOI categories even not encountered during training. Inspired by the impressive zero-shot capabilities offered by CLIP, latest methods strive to leverage CLIP embeddings for improving zero-shot HOI detection. However, these embedding-based methods train the classifier on seen classes only, inevitably resulting in seen-unseen confusion for the model during inference. Besides, we find that using prompt-tuning and adapters further increases the gap between seen and unseen accuracy. To tackle this challenge, we present the first generation-based model using CLIP for zero-shot HOI detection, coined HOIGen. It allows to unlock the potential of CLIP for feature generation instead of feature extraction only. To achieve it, we develop a CLIP-injected feature generator in accordance with the generation of human, object and union features. Then, we extract realistic features of seen samples and mix them with synthetic features together, allowing the model to train seen and unseen classes jointly. To enrich the HOI scores, we construct a generative prototype bank in a pairwise HOI recognition branch, and a multi-knowledge prototype bank in an image-wise HOI recognition branch, respectively. Extensive experiments on HICO-DET benchmark demonstrate our HOIGen achieves superior performance for both seen and unseen classes under various zero-shot settings, compared with other top-performing methods. Code is available at: https://github.com/soberguo/HOIGen

cs.CV↗