arXiv subjects

Jiaming Hu

Publications and source records attributed to Jiaming Hu.

At least 19 recordsLinked to original sources

arXiv2026

SCALE: State-Calibrated Latent Embeddings for JEPA Planning in the Right Geometry

Joint-embedding predictive world models plan by scoring predicted terminal embeddings against a goal embedding using a cost defined on the representation itself. Two prominent strategies for obtaining non-collapsed representations are to inherit a pretrained feature space, as in DINO-WM, and to learn an embedding end to end with anti-collapse regularization, as in LeWorldModel (LeWM) with SIGReg. These strategies show complementary strengths across tasks. Although task-relevant state is decodable from the full embeddings of both models, DINO-WM's leading principal components usually retain substantially more state information than LeWM's. Because Euclidean planning costs are dominated by high-variance directions, this difference affects how strongly state can influence candidate selection. We propose SCALE (State-CAlibrated Latent Embeddings) to give the end-to-end LeWM representation the favorable geometric property observed in DINO-WM. SCALE induces this property by correlating sampled pairwise latent distances with distances in a standardized task-relevant state space, without replacing LeWM's learned encoder. Across five tasks, three planning solvers, and five compute budgets, SCALE improves every task--solver average over LeWM. A latent-to-state regression control matches or exceeds SCALE's full-embedding decodability yet leaves latent--state distance alignment essentially unchanged and yields less consistent planning gains. SCALE adds a single lightweight training-time regularizer and no planning-time overhead. These results show that planning depends not only on whether task-relevant information is present, but also on whether it shapes the geometry consumed by the planner.

cs.LG↗

arXiv2026

Agents' Last Exam

Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional domains. We argue that this gap is largely an evaluation problem: widely used benchmarks lack sustained performance measurement on real and economically valuable workflows. This paper introduces Agents' Last Exam (ALE), a benchmark designed to evaluate AI agents on long horizon, economically valuable, real world tasks with verifiable outcomes. Developed in collaboration with 250+ industry experts, ALE covers non-physical industries defined with reference to O*NET / SOC 2018 (the U.S. federal occupational taxonomy). It is organized around a task taxonomy with 55 sub fields grouped into 13 industry clusters covering 1K+ tasks. Current results show that the hardest tier remains far from saturated: across mainstream harness and backbone configurations, the average full pass rate is below 1%. ALE is designed as a living benchmark: its task pool grows continuously as new workflows and industries are onboarded. More broadly, ALE is intended not merely as another leaderboard, but as an instrument for closing the gap between benchmark success and GDP relevant impact.

cs.AI↗

arXiv2026

Generalized Shift Vector as the Intrinsic Dipole of Many-Body Correlated Electronic States

Shift vectors play a central role in nonlinear optics and transport phenomena, where they are usually understood as charge-center shifts associated with transitions between quantum states. Here we show that the same geometric structure can be more fundamentally understood as the intrinsic dipole moment of a single correlated state. Our derivation clarifies the local and global aspects of gauge invariance, the origin of the phase-gradient term, and its connection to the internal coherence structure of many-body correlations. The single-state shift character appears both as a displacement of the real-space joint probability density and as a linear electric-field modification in energy space. Applying this framework to optically induced correlations, electron-phonon-mediated processes, and excitonic electron-hole states, we recover previously proposed shift vectors and the standard expression for the shift current as special cases. Our results establish a common physical foundation for shift vectors as intrinsic dipolar properties of correlated electronic states.

cond-mat.mes-hall↗

arXiv2026

BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning

Image captioning is one of the most fundamental tasks in computer vision. Owing to its open-ended nature, it has received significant attention in the era of multimodal large language models (MLLMs). In pursuit of ever more detailed and accurate captions, recent work has increasingly turned to reinforcement learning (RL). However, existing captioning-RL methods and evaluation metrics often emphasize a narrow notion of caption quality, inducing trade-offs across core dimensions of captioning. For example, utility-oriented objectives can encourage noisy, hallucinated, or overlong captions that improve downstream question answering while harming fluency, whereas arena-style objectives can favor fluent but generic descriptions with limited usefulness. To address this, we propose a more balanced RL framework that jointly optimizes utility-aware correctness, reference coverage, and linguistic quality. In order to effectively optimize the resulting continuous multi-objective reward formulation, we apply GDPO-style reward-decoupled normalization to continuous-valued captioning rewards and show that it improves performance over vanilla GRPO. Additionally, we introduce length-conditional reward masking, yielding a more suitable length penalty for captioning. Across LLaVA-1.5-7B and Qwen2.5-VL 3B and 7B base models, our method consistently improves caption quality, with peak gains of +13.6 DCScore, +9.0 CaptionQA, and +29.0 CapArena across different models.

cs.CV↗

arXiv2026

Towards General Preference Alignment: Diffusion Models at Nash Equilibrium

Reinforcement learning from human feedback (RLHF) has been popular for aligning text-to-image (T2I) diffusion models with human preferences. As a mainstream branch of RLHF, Direct Preference Optimization (DPO) offers a computationally efficient alternative that avoids explicit reward modeling and has been widely adopted in diffusion alignment. However, existing preference-based methods for diffusion alignment still rely on reward-induced preference signals and typically assume that human preferences can be adequately modeled by the Bradley--Terry (BT) model, which may fail to capture the full complexity of human preferences. In this paper, we formulate diffusion alignment from a game-theoretic perspective. We propose Diffusion Nash Preference Optimization (Diff.-NPO), an intuitive general preference framework for diffusion alignment. Diff.-NPO encourages the current policy to play against itself to achieve self improvement and lead to a better alignment. Empirically, we demonstrate the effectiveness of Diff.-NPO on the text-to-image generation task via various metrics. Diff.-NPO consistently outperforms existing preference-based diffusion alignment methods.

cs.LG↗

arXiv2026

Microscopic Theory of Superionic Phase Transitions: Nonadiabatic Dynamics and Many-Body Effects

Superionic phase transitions have attracted extensive interest for decades due to their promising applications and rich underlying physics. In particular, complicated many-body effects and nonadiabatic dynamics are believed to play essential roles, limiting the explanatory power of phenomenological approaches and obscuring the microscopic mechanisms at play. In this work, we develop a unified theoretical framework for describing solid-state ionic conduction. After reviewing the conventional approximations, we construct a general lattice model that applies to both normal ionic and superionic conductors. By incorporating the nonadiabatic concerted-hopping mechanism and the many-body Coulomb interaction within a self-consistent mean-field scheme, we identify these two effects as the fundamental driving forces behind type-I and type-II superionic phase transitions, respectively. Our model directly reproduces key experimental observations. Within this unified framework, we further provide a comprehensive comparison between the two types of transitions. Overall, our work offers microscopic insight into superionic phase transitions and provides guidance for the design and optimization of advanced solid-state ionic conductors.

cond-mat.mtrl-sci↗

arXiv2026

Distributionally Robust Token Optimization in RLHF

Large Language Models (LLMs) tend to respond correctly to prompts that align well with the data they were trained and fine-tuned on. Yet, small shifts in wording, format, or language can trigger surprisingly large failures, especially on multi-step reasoning problems. To address this problem, we propose a Distributionally Robust Token Optimization (DRTO) approach, which combines token-level Reinforcement Learning from Human Feedback (RLHF) with Distributionally Robust Optimization (DRO). DRTO constructs f-divergence ambiguity sets over span-level actor losses, providing a principled way to emphasize difficult response segments during policy optimization. Empirically, DRTO enhances consistency under distribution shifts in multiple reasoning benchmarks among different tasks, achieving $+4.4$ percentage points on MATH-500 and $+2.7$ percentage points on LiveCodeBench over standard RTO.

cs.LG↗

arXiv2026

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

AI agents may soon become capable of autonomously completing valuable, long-horizon tasks in diverse domains. Current benchmarks either do not measure real-world tasks, or are not sufficiently difficult to meaningfully measure frontier models. To this end, we present Terminal-Bench 2.0: a carefully curated hard benchmark composed of 89 tasks in computer terminal environments inspired by problems from real workflows. Each task features a unique environment, human-written solution, and comprehensive tests for verification. We show that frontier models and agents score less than 65\% on the benchmark and conduct an error analysis to identify areas for model and agent improvement. We publish the dataset and evaluation harness to assist developers and researchers in future work at https://www.tbench.ai/ .

cs.SE↗

arXiv2026

Ab initio study of carrier mobility in Bi$_2$O$_2$Se

Bi$_2$O$_2$Se is an emerging high-performance layered semiconductor with excellent stability. While experimental studies have explored carrier transport across various doping levels for both $n$-type and $p$-type conduction, a comprehensive theoretical understanding remains incomplete. In this work, we present parameter-free first-principles calculations of the electron and hole mobilities in Bi$_2$O$_2$Se, based on iterative solution of the Boltzmann transport equation that includes electron-phonon scattering and ionized impurity scattering on an equal footing. Intriguingly, we find that Bi$_2$O$_2$Se exhibits high electron mobilities in both the in-plane and out-of-plane directions, whereas the hole mobilities are only significant in the in-plane direction, displaying a unique three-dimensional (3D) electron transport and two-dimensional (2D) hole transport behavior. At 300~K, the calculated intrinsic electron and hole mobilities along the in-plane direction are 447~$\mathrm{cm^2\,V^{-1}\,s^{-1}}$ and 29~$\mathrm{cm^2\,V^{-1}\,s^{-1}}$, respectively, which are primarily affected by Fr\"ohlich electron-phonon interactions. Due to its large static dielectric permittivity, Bi$_2$O$_2$Se exhibits an exceptionally high low-temperature electron mobilities above $1.0\times10^5~\mathrm{cm^2\,V^{-1}\,s^{-1}}$, and its electron mobilities above 50~K is robust against ionized impurity scattering over a wide range of impurity concentrations. By incorporating the Hall effect into our analysis, we predict an in-plane electron Hall mobility of 517~$\mathrm{cm^2\,V^{-1}\,s^{-1}}$ at 300~K, in excellent agreement with experimental data. These results provide valuable insights into the carrier transport mechanisms in Bi$_2$O$_2$Se, and offer predictive benchmarks for future theoretical and experimental investigations.

cond-mat.mtrl-sci↗

arXiv2025

Kinematify: Open-Vocabulary Synthesis of High-DoF Articulated Objects

A deep understanding of kinematic structures and movable components is essential for enabling robots to manipulate objects and model their own articulated forms. Such understanding is captured through articulated objects, which are essential for tasks such as physical simulation, motion planning, and policy learning. However, creating these models, particularly for objects with high degrees of freedom (DoF), remains a significant challenge. Existing methods typically rely on motion sequences or strong assumptions from hand-curated datasets, which hinders scalability. In this paper, we introduce Kinematify, an automated framework that synthesizes articulated objects directly from arbitrary RGB images or textual descriptions. Our method addresses two core challenges: (i) inferring kinematic topologies for high-DoF objects and (ii) estimating joint parameters from static geometry. To achieve this, we combine MCTS search for structural inference with geometry-driven optimization for joint reasoning, producing physically consistent and functionally valid descriptions. We evaluate Kinematify on diverse inputs from both synthetic and real-world environments, demonstrating improvements in registration and kinematic topology accuracy over prior work.

cs.RO↗

arXiv2025

Optical Computation-in-Communication enables low-latency, high-fidelity perception in telesurgery

Artificial intelligence (AI) holds significant promise for enhancing intraoperative perception and decision-making in telesurgery, where physical separation impairs sensory feedback and control. Despite advances in medical AI and surgical robotics, conventional electronic AI architectures remain fundamentally constrained by the compounded latency from serial processing of inference and communication. This limitation is especially critical in latency-sensitive procedures such as endovascular interventions, where delays over 200 ms can compromise real-time AI reliability and patient safety. Here, we introduce an Optical Computation-in-Communication (OCiC) framework that reduces end-to-end latency significantly by performing AI inference concurrently with optical communication. OCiC integrates Optical Remote Computing Units (ORCUs) directly into the optical communication pathway, with each ORCU experimentally achieving up to 69 tera-operations per second per channel through spectrally efficient two-dimensional photonic convolution. The system maintains ultrahigh inference fidelity within 0.1% of CPU/GPU baselines on classification and coronary angiography segmentation, while intrinsically mitigating cumulative error propagation, a longstanding barrier to deep optical network scalability. We validated the robustness of OCiC through outdoor dark fibre deployments, confirming consistent and stable performance across varying environmental conditions. When scaled globally, OCiC transforms long-haul fibre infrastructure into a distributed photonic AI fabric with exascale potential, enabling reliable, low-latency telesurgery across distances up to 10,000 km and opening a new optical frontier for distributed medical intelligence.

physics.optics↗

arXiv2025

CCFC: Core & Core-Full-Core Dual-Track Defense for LLM Jailbreak Protection

Jailbreak attacks pose a serious challenge to the safe deployment of large language models (LLMs). We introduce CCFC (Core & Core-Full-Core), a dual-track, prompt-level defense framework designed to mitigate LLMs' vulnerabilities from prompt injection and structure-aware jailbreak attacks. CCFC operates by first isolating the semantic core of a user query via few-shot prompting, and then evaluating the query using two complementary tracks: a core-only track to ignore adversarial distractions (e.g., toxic suffixes or prefix injections), and a core-full-core (CFC) track to disrupt the structural patterns exploited by gradient-based or edit-based attacks. The final response is selected based on a safety consistency check across both tracks, ensuring robustness without compromising on response quality. We demonstrate that CCFC cuts attack success rates by 50-75% versus state-of-the-art defenses against strong adversaries (e.g., DeepInception, GCG), without sacrificing fidelity on benign queries. Our method consistently outperforms state-of-the-art prompt-level defenses, offering a practical and effective solution for safer LLM deployment.

cs.CR↗

arXiv2025

Microscopic Theory of Light-Induced Coherent Phonons Mediated by Quantum Geometry

Light-induced coherent phonons provide a powerful platform for ultrafast control of material properties. However, the microscopic theory and quantum geometric nature of this phenomenon remain underexplored. Here, we develop a fully quantum-mechanical framework based on Feynman diagrams to systematically describe the generation of coherent phonons by light. We identify a dominant second-order, double-resonant process in noncentrosymmetric semiconductors that efficiently couples light to both electronic and phononic excitations. Crucially, we uncover the quantum geometric origin, encoded in the electron-phonon coupling (EPC) shift vector and the EPC quantum geometric tensor. Applying our theory to ferroelectric BaTiO$_3$ and SnSe, we demonstrate the potential for light-induced modulation of ferroelectric polarization driven by coherent phonons. This work provides fundamental insights for designing efficient optical control strategies for both coherent phonons and ferroelectric polarization.

cond-mat.mes-hall↗

arXiv2025

DRO-Augment Framework: Robustness by Synergizing Wasserstein Distributionally Robust Optimization and Data Augmentation

In many real-world applications, ensuring the robustness and stability of deep neural networks (DNNs) is crucial, particularly for image classification tasks that encounter various input perturbations. While data augmentation techniques have been widely adopted to enhance the resilience of a trained model against such perturbations, there remains significant room for improvement in robustness against corrupted data and adversarial attacks simultaneously. To address this challenge, we introduce DRO-Augment, a novel framework that integrates Wasserstein Distributionally Robust Optimization (W-DRO) with various data augmentation strategies to improve the robustness of the models significantly across a broad spectrum of corruptions. Our method outperforms existing augmentation methods under severe data perturbations and adversarial attack scenarios while maintaining the accuracy on the clean datasets on a range of benchmark datasets, including but not limited to CIFAR-10-C, CIFAR-100-C, MNIST, and Fashion-MNIST. On the theoretical side, we establish novel generalization error bounds for neural networks trained using a computationally efficient, variation-regularized loss function closely related to the W-DRO problem.

stat.ML↗

arXiv2025

Mobile Manipulation Planning for Tabletop Rearrangement

Efficient tabletop rearrangement planning seeks to find high-quality solutions while minimizing total cost. However, the task is challenging due to object dependencies and limited buffer space for temporary placements. The complexity increases for mobile robots, which must navigate around the table with restricted access. A*-based methods yield high-quality solutions, but struggle to scale as the number of objects increases. Monte Carlo Tree Search (MCTS) has been introduced as an anytime algorithm, but its convergence speed to high-quality solutions remains slow. Previous work~\cite{strap2024} accelerated convergence but required the robot to move to the closest position to the object for each pick and place operation, leading to inefficiencies. To address these limitations, we extend the planner by introducing a more efficient strategy for mobile robots. Instead of selecting the nearest available location for each action, our approach allows multiple operations (e.g., pick-and-place) from a single standing position, reducing unnecessary movement. Additionally, we incorporate state re-exploration to further improve plan quality. Experimental results show that our planner outperforms existing planners both in terms of solution quality and planning time.

cs.RO↗

arXiv2025

UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation

We introduce UniGen, a unified multimodal large language model (MLLM) capable of image understanding and generation. We study the full training pipeline of UniGen from a data-centric perspective, including multi-stage pre-training, supervised fine-tuning, and direct preference optimization. More importantly, we propose a new Chain-of-Thought Verification (CoT-V) strategy for test-time scaling, which significantly boosts UniGen's image generation quality using a simple Best-of-N test-time strategy. Specifically, CoT-V enables UniGen to act as both image generator and verifier at test time, assessing the semantic alignment between a text prompt and its generated image in a step-by-step CoT manner. Trained entirely on open-source datasets across all stages, UniGen achieves state-of-the-art performance on a range of image understanding and generation benchmarks, with a final score of 0.78 on GenEval and 85.19 on DPG-Bench. Through extensive ablation studies, our work provides actionable insights and addresses key challenges in the full life cycle of building unified MLLMs, contributing meaningful directions to the future research.

cs.CV↗

arXiv2025

cpRRTC: GPU-Parallel RRT-Connect for Constrained Motion Planning

Motion planning is a fundamental problem in robotics that involves generating feasible trajectories for a robot to follow. Recent advances in parallel computing, particularly through CPU and GPU architectures, have significantly reduced planning times to the order of milliseconds. However, constrained motion planning especially using sampling based methods on GPUs remains underexplored. Prior work such as pRRTC leverages a tracking compiler with a CUDA backend to accelerate forward kinematics and collision checking. While effective in simple settings, their approach struggles with increased complexity in robot models or environments. In this paper, we propose a novel GPU based framework utilizing NVRTC for runtime compilation, enabling efficient handling of high complexity scenarios and supporting constrained motion planning. Experimental results demonstrate that our method achieves superior performance compared to existing approaches.

cs.RO↗

arXiv2025

Quantum Geometric Origin of Strain-Induced Ferroelectric Phase Transitions

Strain-regulated ferroelectric (FE) materials have long attracted significant attention due to their diverse applications. While soft-phonon theory and the (pseudo) Jahn-Teller effect have achieved considerable success in providing phenomenological descriptions and general understanding, the detailed connection between these perspectives and their microscopic dependence on strain regulation remains unclear. Here, under the framework of density-functional perturbation theory (DFPT), we demonstrate that the Berry curvature of electron-phonon coupling (EPC) plays a pivotal role in the interatomic force matrix (IFM). A subsequent model analysis shows that external strain can reverse the polarity of the EPC Berry curvature in (quasi)-degenerate electronic subsystems through band inversion, thereby directly leading to phonon softening. The general theory is then applied to the BiOCl monolayer as a benchmark, which offers an accurate description of the density functional theory (DFT) calculations. This mechanism is further observed across a broad range of materials through ab initio calculations, providing an insightful perspective on EPC quantum geometry in lattice dynamics and FE phase transitions.

cond-mat.mtrl-sci↗