arXiv subjects

Jiayi Zhao

Publications and source records attributed to Jiayi Zhao.

At least 19 recordsLinked to original sources

arXiv2026

FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers

Activation steering has emerged as a lightweight approach for modifying language model behavior without parameter updates, yet existing methods remain brittle: unstable across layers and prone to disturbing behavior unrelated to the target concept. We trace these failures to a hidden assumption shared by widely-used methods such as CAA, ActAdd, and ITI: that the intermediate activation space is Euclidean. We show this assumption is fundamentally flawed. The metric that actually governs how a hidden-state perturbation changes the output is the Fisher information metric of the softmax layer, pulled back to the intermediate layer through the Jacobian of the intervening layers. From it we derive a closed-form steering direction, applied to a hidden state at an intermediate layer, that reaches a target concept change with the least non-target distortion. The framework is sharpest in the early and middle intermediate layers, where the metric is strongly non-Euclidean and geometric correction matters most. We evaluate it on three verb-morphology concepts: third-person-singular, progressive, and past-tense inflection, following standard counterfactual-concept evaluation. On GPT-2 Small, this non-Euclidean geometry is borne out empirically, and our method lowers off-target KL divergence by median factors of 1.4--6.5x against individual steering baselines. On Llama-3-8B and Qwen3-8B, it lowers off-target KL by median factors of 1.8--3.6x against individual baselines at the early and middle layers. These results show that geometric correction retains its advantage on larger models with more complex internal structure.

cs.LG↗

arXiv2026

On determinantal formulas for hermitian random matrices

In this paper, we give a direct proof of determinantal formulas for connected $k$-point functions for hermitian matrix models. We also give a new proof of KP integrability for them. From the viewpoint of KP hierarchy, we further give a new proof of the explicit formula for the corresponding affine coordinates. Furthermore, duality for some hermitian matrix models is proved.

math-ph↗

arXiv2026

Distilling Unitary Operations: A No-Go Theorem and Minimal Realization

Quantum gates executed on physical hardware are inevitably degraded by environmental noise. While state purification effectively distills static quantum resources, the dynamic execution of quantum algorithms requires a higher-order approach to mitigate errors on the operations themselves. In this work, we investigate universal unitary purification: the task of utilizing a quantum higher-order operation to partially restore the ideal action of an unknown unitary corrupted by a known noise model. Focusing on canonical depolarizing noise, we first reveal a fundamental operational obstruction. We prove that within the indefinite causal order framework, no nontrivial 2-slot higher-order operation can universally purify the set of single-qubit unitaries. Overcoming this strict limitation, we establish that a 3-slot parallel architecture provides the minimal realization for non-trivial purification. We analytically derive the optimal average fidelity within the parallel 3-slot class, demonstrating that it strictly surpasses trivial strategies by systematically utilizing ancillary qubits as a quantum memory to absorb errors. Furthermore, we provide a concrete quantum circuit construction attaining this parallel optimum. Our results establish the strict theoretical boundaries of distilling clean operations from noisy gates, offering immediate architectural insights for robust gate design.

quant-ph↗

arXiv2026

GUE Correlators and Large Genus Asymptotics

In this paper, we use a formula obtained by Dubrovin and Yang (2017) to study certain asymptotic behaviors of GUE (Gaussian unitary ensemble) correlators. More precisely, we obtain large genus asymptotics of enumerations of ordinary graphs and ribbon graphs with 1 face.

math-ph↗

arXiv2026

MotionPRO: Exploring the Role of Pressure in Human MoCap and Beyond

Existing human Motion Capture (MoCap) methods mostly focus on the visual similarity while neglecting the physical plausibility. As a result, downstream tasks such as driving virtual human in 3D scene or humanoid robots in real world suffer from issues such as timing drift and jitter, spatial problems like sliding and penetration, and poor global trajectory accuracy. In this paper, we revisit human MoCap from the perspective of interaction between human body and physical world by exploring the role of pressure. Firstly, we construct a large-scale human Motion capture dataset with Pressure, RGB and Optical sensors (named MotionPRO), which comprises 70 volunteers performing 400 types of motion, encompassing a total of 12.4M pose frames. Secondly, we examine both the necessity and effectiveness of the pressure signal through two challenging tasks: (1) pose and trajectory estimation based solely on pressure: We propose a network that incorporates a small kernel decoder and a long-short-term attention module, and proof that pressure could provide accurate global trajectory and plausible lower body pose. (2) pose and trajectory estimation by fusing pressure and RGB: We impose constraints on orthographic similarity along the camera axis and whole-body contact along the vertical axis to enhance the cross-attention strategy to fuse pressure and RGB feature maps. Experiments demonstrate that fusing pressure with RGB features not only significantly improves performance in terms of objective metrics, but also plausibly drives virtual humans (SMPL) in 3D scene. Furthermore, we demonstrate that incorporating physical perception enables humanoid robots to perform more precise and stable actions, which is highly beneficial for the development of embodied artificial intelligence. Project page is available at: https://nju-cite-mocaphumanoid.github.io/MotionPRO/

cs.CV↗

arXiv2026

Matrix-Valued Optimism is Matrix-Valued Augmentation: Additive Hybrid Designs for Constrained Optimization

Augmented Lagrangian and optimistic primal--dual methods stabilize equality-constrained optimization through seemingly different mechanisms: the former adds constraint-dependent primal curvature, while the latter adds dual memory. Recent work has shown that these mechanisms are equivalent for scalar parameters. We extend this equivalence to matrix-valued correction. We prove an additivity principle: for symmetric matrix parameters, the ideal primal trajectory depends only on the summed correction matrix, not on how it is split between augmented and optimistic channels. This exposes a design freedom: algebraically equivalent decompositions can have different finite-step feasibility because augmented correction affects primal curvature, whereas optimistic correction affects the scale of the dual memory correction. We formulate the resulting step-size-limited design problem and derive a closed-form hybrid rule that selects a matrix correction, splits it between the two channels, and chooses primal and dual steps using local spectral weights. Experiments on nonlinear equality-constrained problems with controlled constraint-Jacobian conditioning show that the hybrid design improves over pure augmented and pure optimistic endpoints, closely tracks a grid-search hybrid oracle, and is competitive with first-order primal--dual baselines under mild-to-moderate ill-conditioning. The experiments also identify the expected limitation: exact cancellation requires increasingly large matrix corrections as the constraint Jacobian becomes ill-conditioned.

cs.LG↗

arXiv2026

ComPASS: Towards Personalized Agentic Social Support via Tool-Augmented Companionship

Developing compassionate interactive systems requires agents to not only understand user emotions but also provide diverse, substantive support. While recent works explore empathetic dialogue generation, they remain limited in response form and content, struggling to satisfy diverse needs across users and contexts. To address this, we explore empowering agents with external tools to execute diverse actions. Grounded in the psychological concept of "social support", this paradigm delivers substantive, human-like companionship. Specifically, we first design a dozen user-centric tools simulating various multimedia applications, which can cover different types of social support behaviors in human-agent interaction scenarios. We then construct ComPASS-Bench, the first personalized social support benchmark for LLM-based agents, via multi-step automated synthesis and manual refinement. Based on ComPASS-Bench, we further synthesize tool use records to fine-tune the Qwen3-8B model, yielding a task-specific ComPASS-Qwen. Comprehensive evaluations across two settings reveal that while the evaluated LLMs can generate valid tool-calling requests with high success rates, significant gaps remain in final response quality. Moreover, tool-augmented responses achieve better overall performance than directly producing conversational empathy. Notably, our trained ComPASS-Qwen demonstrates substantial improvements over its base model, achieving comparable performance to several large-scale models. Our code and data are available at https://github.com/hzp3517/ComPASS.

cs.CL↗

arXiv2026

CloudMatch: Weak-to-Strong Consistency Learning for Semi-Supervised Cloud Detection

Due to the high cost of annotating accurate pixel-level labels, semi-supervised learning has emerged as a promising approach for cloud detection. In this paper, we propose CloudMatch, a semi-supervised framework that effectively leverages unlabeled remote sensing imagery through view-consistency learning combined with scene-mixing augmentations. An observation behind CloudMatch is that cloud patterns exhibit structural diversity and contextual variability across different scenes and within the same scene category. Our key insight is that enforcing prediction consistency across diversely augmented views, incorporating both inter-scene and intra-scene mixing, enables the model to capture the structural diversity and contextual richness of cloud patterns. Specifically, CloudMatch generates one weakly augmented view along with two complementary strongly augmented views for each unlabeled image: one integrates inter-scene patches to simulate contextual variety, while the other employs intra-scene mixing to preserve semantic coherence. This approach guides pseudolabel generation and enhances generalization. Extensive experiments show that CloudMatch achieves good performance, demonstrating its capability to utilize unlabeled data efficiently and advance semi-supervised cloud detection.

cs.CV↗

arXiv2025

DriveExplorer: Images-Only Decoupled 4D Reconstruction with Progressive Restoration for Driving View Extrapolation

This paper presents an effective solution for view extrapolation in autonomous driving scenarios. Recent approaches focus on generating shifted novel view images from given viewpoints using diffusion models. However, these methods heavily rely on priors such as LiDAR point clouds, 3D bounding boxes, and lane annotations, which demand expensive sensors or labor-intensive labeling, limiting applicability in real-world deployment. In this work, with only images and optional camera poses, we first estimate a global static point cloud and per-frame dynamic point clouds, fusing them into a unified representation. We then employ a deformable 4D Gaussian framework to reconstruct the scene. The initially trained 4D Gaussian model renders degraded and pseudo-images to train a video diffusion model. Subsequently, progressively shifted Gaussian renderings are iteratively refined by the diffusion model,and the enhanced results are incorporated back as training data for 4DGS. This process continues until extrapolation reaches the target viewpoints. Compared with baselines, our method produces higher-quality images at novel extrapolated viewpoints.

cs.CV↗

arXiv2025

Dynamic LOCC Circuits for Automated Entanglement Manipulation

Due to the limited qubit number of quantum devices, distributed quantum computing is considered a promising pathway to overcome this constraint. In this paradigm, multiple quantum processors are interconnected to form a cohesive computational network, and the most natural set of free operations is local operations and classical communication (LOCC). However, designing a practical LOCC protocol for a particular task has been a tough problem. In this work, we propose a general and flexible framework called dynamic LOCCNet (DLOCCNet) to simulate and design LOCC protocols. We demonstrate its effectiveness in two key applications: entanglement distillation and distributed state discrimination. The protocols designed by DLOCCNet, in contrast to conventional ones, can solve larger-sized problems with reduced training time, making the framework a practical and scalable tool for current quantum devices. This work advances our understanding of the capabilities and limitations of LOCC while providing a powerful methodology for protocol design.

quant-ph↗

arXiv2025

CD-Mamba: Cloud detection with long-range spatial dependency modeling

Remote sensing images are frequently obscured by cloud cover, posing significant challenges to data integrity and reliability. Effective cloud detection requires addressing both short-range spatial redundancies and long-range atmospheric similarities among cloud patches. Convolutional neural networks are effective at capturing local spatial dependencies, while Mamba has strong capabilities in modeling long-range dependencies. To fully leverage both local spatial relations and long-range dependencies, we propose CD-Mamba, a hybrid model that integrates convolution and Mamba's state-space modeling into a unified cloud detection network. CD-Mamba is designed to comprehensively capture pixelwise textural details and long term patchwise dependencies for cloud detection. This design enables CD-Mamba to manage both pixel-wise interactions and extensive patch-wise dependencies simultaneously, improving detection accuracy across diverse spatial scales. Extensive experiments validate the effectiveness of CD-Mamba and demonstrate its superior performance over existing methods.

cs.CV↗

arXiv2025

Unlocking Mixed Reality for Medical Education: A See-Through Perspective on Head Anatomy

Extended reality (XR), encompassing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), is emerging as a transformative platform for medical education. Traditional methods such as textbooks, physical models, and cadaveric dissections often lack interactivity and fail to convey complex spatial relationships effectively. The emerging MR technology addresses these limitations by providing immersive environments that blend virtual elements with real-world contexts. This study presents an MR application for head anatomy education, enabling learners to intuitively interact with see-through 3D anatomical structures via hand gestures and controllers. Our hierarchical information design supports progressive learning, guiding users from basic anatomical labels to detailed structural insights. Additionally, the system incorporates an automatic calibration module that aligns virtual anatomical models with a real human head, thereby facilitating realistic human-model interactions. Experiments show that the system can effectively match the anatomical model with real-time scenes, thus enhancing the interactivity and immersion of medical education, providing an innovative tool for teaching anatomy.

cs.HC↗

arXiv2025

PS-GS: Gaussian Splatting for Multi-View Photometric Stereo

Integrating inverse rendering with multi-view photometric stereo (MVPS) yields more accurate 3D reconstructions than the inverse rendering approaches that rely on fixed environment illumination. However, efficient inverse rendering with MVPS remains challenging. To fill this gap, we introduce the Gaussian Splatting for Multi-view Photometric Stereo (PS-GS), which efficiently and jointly estimates the geometry, materials, and lighting of the object that is illuminated by diverse directional lights (multi-light). Our method first reconstructs a standard 2D Gaussian splatting model as the initial geometry. Based on the initialization model, it then proceeds with the deferred inverse rendering by the full rendering equation containing a lighting-computing multi-layer perceptron. During the whole optimization, we regularize the rendered normal maps by the uncalibrated photometric stereo estimated normals. We also propose the 2D Gaussian ray-tracing for single directional light to refine the incident lighting. The regularizations and the use of multi-view and multi-light images mitigate the ill-posed problem of inverse rendering. After optimization, the reconstructed object can be used for novel-view synthesis, relighting, and material and shape editing. Experiments on both synthetic and real datasets demonstrate that our method outperforms prior works in terms of reconstruction accuracy and computational efficiency.

cs.GR↗

arXiv2025

Unveiling the Potential of Segment Anything Model 2 for RGB-Thermal Semantic Segmentation with Language Guidance

The perception capability of robotic systems relies on the richness of the dataset. Although Segment Anything Model 2 (SAM2), trained on large datasets, demonstrates strong perception potential in perception tasks, its inherent training paradigm prevents it from being suitable for RGB-T tasks. To address these challenges, we propose SHIFNet, a novel SAM2-driven Hybrid Interaction Paradigm that unlocks the potential of SAM2 with linguistic guidance for efficient RGB-Thermal perception. Our framework consists of two key components: (1) Semantic-Aware Cross-modal Fusion (SACF) module that dynamically balances modality contributions through text-guided affinity learning, overcoming SAM2's inherent RGB bias; (2) Heterogeneous Prompting Decoder (HPD) that enhances global semantic information through a semantic enhancement module and then combined with category embeddings to amplify cross-modal semantic consistency. With 32.27M trainable parameters, SHIFNet achieves state-of-the-art segmentation performance on public benchmarks, reaching 89.8% on PST900 and 67.8% on FMB, respectively. The framework facilitates the adaptation of pre-trained large models to RGB-T segmentation tasks, effectively mitigating the high costs associated with data collection while endowing robotic systems with comprehensive perception capabilities. The source code will be made publicly available at https://github.com/iAsakiT3T/SHIFNet.

cs.CV↗

arXiv2025

SARGes: Semantically Aligned Reliable Gesture Generation via Intent Chain

Co-speech gesture generation enhances human-computer interaction realism through speech-synchronized gesture synthesis. However, generating semantically meaningful gestures remains a challenging problem. We propose SARGes, a novel framework that leverages large language models (LLMs) to parse speech content and generate reliable semantic gesture labels, which subsequently guide the synthesis of meaningful co-speech gestures.First, we constructed a comprehensive co-speech gesture ethogram and developed an LLM-based intent chain reasoning mechanism that systematically parses and decomposes gesture semantics into structured inference steps following ethogram criteria, effectively guiding LLMs to generate context-aware gesture labels. Subsequently, we constructed an intent chain-annotated text-to-gesture label dataset and trained a lightweight gesture label generation model, which then guides the generation of credible and semantically coherent co-speech gestures. Experimental results demonstrate that SARGes achieves highly semantically-aligned gesture labeling (50.2% accuracy) with efficient single-pass inference (0.4 seconds). The proposed method provides an interpretable intent reasoning pathway for semantic gesture synthesis.

cs.CL↗

arXiv2025

Human Balancing on a Log: A Switched Multi-Layer Controller

We study the task of balancing a human on a log that is fixed in place. Balancing on a log is substantially more challenging than balancing on a flat surface due to increased instability -- nonetheless, we are able to balance by composing simple (e.g., PID, LQR) controllers in a bio-inspired switched multi-layer configuration. The controller consists of an upper-layer LQR planner (akin to the central nervous system) that coordinates ankle and hip torques, and lower-layer PID trackers (akin to local motor units) that follow this plan subject to nonlinear dynamics. The controller switches between three operational modes depending on the state of the human. The efficacy of the controller is verified in simulation, where our controller is able to stabilize the human for a variety of initial conditions and disturbances. We also introduce a controller that outputs muscle activations to perform the same balancing task.

eess.SY↗

arXiv2025

Tâtonnement in Homothetic Fisher Markets

A prevalent theme in the economics and computation literature is to identify natural price-adjustment processes by which sellers and buyers in a market can discover equilibrium prices. An example of such a process is tâtonnement, an auction-like algorithm first proposed in 1874 by French economist Walras in which sellers adjust prices based on the Marshallian demands of buyers. A dual concept in consumer theory is a buyer's Hicksian demand. In this paper, we identify the maximum of the absolute value of the elasticity of the Hicksian demand, as an economic parameter sufficient to capture and explain a range of convergent and non-convergent tâtonnement behaviors in a broad class of markets. In particular, we prove the convergence of tâtonnement at a rate of $O((1+\varepsilon^2)/T)$, in homothetic Fisher markets with bounded price elasticity of Hicksian demand, i.e., Fisher markets in which consumers have preferences represented by homogeneous utility functions and the price elasticity of their Hicksian demand is bounded, where $\varepsilon \geq 0$ is the maximum absolute value of the price elasticity of Hicksian demand across all buyers. Our result not only generalizes known convergence results for CES Fisher markets, but extends them to mixed nested CES markets and Fisher markets with continuous, possibly non-concave, homogeneous utility functions. Our convergence rate covers the full spectrum of nested CES utilities, including Leontief and linear utilities, unifying previously existing disparate convergence and non-convergence results. In particular, for $\varepsilon = 0$, i.e., Leontief markets, we recover the best-known convergence rate of $O(1/T)$, and as $\varepsilon \to \infty$, e.g., linear Fisher markets, we obtain non-convergent behavior, as expected.

cs.GT↗

arXiv2024

Motion Generation Review: Exploring Deep Learning for Lifelike Animation with Manifold

Human motion generation involves creating natural sequences of human body poses, widely used in gaming, virtual reality, and human-computer interaction. It aims to produce lifelike virtual characters with realistic movements, enhancing virtual agents and immersive experiences. While previous work has focused on motion generation based on signals like movement, music, text, or scene background, the complexity of human motion and its relationships with these signals often results in unsatisfactory outputs. Manifold learning offers a solution by reducing data dimensionality and capturing subspaces of effective motion. In this review, we present a comprehensive overview of manifold applications in human motion generation, one of the first in this domain. We explore methods for extracting manifolds from unstructured data, their application in motion generation, and discuss their advantages and future directions. This survey aims to provide a broad perspective on the field and stimulate new approaches to ongoing challenges.

cs.CV↗