arXiv subjects

Xiaowen Sun

Publications and source records attributed to Xiaowen Sun.

9 recordsLinked to original sources

arXiv2026

Should This Case Be Adapted? Prediction Fragmentation Controls Test-Time Adaptation

Episodic test-time adaptation resets a frozen segmenter to source weights $M_0$ on each case and adapts for a fixed step count. A fixed horizon conflates a cohort-level question, how far to adapt, with an irreducibly per-case one, whether this case should be adapted at all. Cohort means hide that decision: on cross-vendor cardiac MRI the mean $Δ$Dice from adaptation is statistically indistinguishable from zero while 58.7% of cases are individually made worse. We quantify this harm as harmful accepted area (HA), the harmful fraction of the edited area a controller deploys. Held-out tuning gives a stronger baseline than a fixed horizon, but the budget it selects transfers on neither of the two main medical benchmarks, and no global budget can condition on the case. We show that prediction fragmentation---the disagreement geometry between $M_0$ and the adapted mask $M_k$---predicts HA with no labels or extra backward passes at decision time, comparably on three benchmarks (Spearman $ρ$ 0.50--0.60), at a quarter of gradient-norm's latency. A case-level router built on it cuts HA from 0.228 to 0.139 on a benchmark that took no part in its design, with the design frozen and only cut-points recalibrated there. On the cardiac benchmark the design was selected on, the router cuts HA from 0.129 to 0.013 at matched Dice and 1.10 deployed updates, against the retrospective-best budget found post hoc on evaluation labels, and reduces that 58.7% to 20.0%, an upper bound we quantify. Where the retained cases are not net-helped (as on prostate), the router still cuts HA but concedes accuracy, a boundary we report. Thresholds are fit once on a labeled split disjoint from evaluation; decisions use no labels or gradients. The template ports across architecture and domain (nnU-Net$\to$SegFormer, Cityscapes$\to$ACDC) with coordinate, thresholds and per-bucket actions instantiated per domain.

cs.CV↗

arXiv2026

StateVLM: A State-Aware Vision-Language Model for Robotic Affordance Reasoning

Vision-language models have demonstrated strong performance across robotic perception and instruction-following tasks. However, they still struggle with precise spatial reasoning, particularly in predicting object locations and fine-grained object states. We propose StateVLM, a vision-language model designed to learn fine-grained object representations, including object localization and grasp-relevant region prediction. We introduce a joint training objective that integrates an auxiliary regression loss (ARL) with the standard causal language modeling (CLM) objective to improve numerical reasoning and spatial understanding. To evaluate whether models can move beyond category-level grounding toward state-aware spatial understanding, we introduce an open-source benchmark, Object State Affordance Reasoning (OSAR), comprising 1,172 scenes with 7,746 individual objects and their corresponding bounding boxes. Empirical experiments on the Referring Expression Comprehension datasets demonstrate that integrating ARL improves model performance compared with CLM only. Experiments on the OSAR benchmark further demonstrate that StateVLM with ARL achieves an average improvement of 5.2% over models trained with CLM only. These results show that ARL is particularly beneficial for the complex affordance reasoning tasks that require object state understanding. The joint training objective in StateVLM demonstrates that integrating an auxiliary regression objective into VLM training improves numerical reasoning, and the OSAR benchmark provides a new testbed for understanding object state affordances in robotics.

cs.CV↗

arXiv2026

Uncertainty, Vagueness, and Ambiguity in Human-Robot Interaction: Why Conceptualization Matters

Uncertainty, vagueness, and ambiguity are closely related and often confused concepts in human-robot interaction (HRI). In earlier studies, these concepts have been defined in contradictory ways and described using inconsistent terminology. This conceptual confusion and lack of terminological consistency undermine empirical comparability, thereby slowing the accumulation of theory. Consequently, consistent concepts that clarify these challenges, including their definitions, distinctions, and interrelationships, are needed in HRI. To address this lack of clarity, this paper proposes a consistent conceptual foundation for the challenges of uncertainty, vagueness, and ambiguity in HRI. First, we examine the meanings of these three terms in dictionaries. We then analyze the nature of their distinctions and interrelationships within the context of HRI. We further illustrate these characteristics through examples. Finally, we demonstrate how this consistent conceptual foundation facilitates the design of novel methods and the evaluation of existing methodologies for these phenomena.

cs.HC↗

arXiv2025

A Framework for Adapting Human-Robot Interaction to Diverse User Groups

To facilitate natural and intuitive interactions with diverse user groups in real-world settings, social robots must be capable of addressing the varying requirements and expectations of these groups while adapting their behavior based on user feedback. While previous research often focuses on specific demographics, we present a novel framework for adaptive Human-Robot Interaction (HRI) that tailors interactions to different user groups and enables individual users to modulate interactions through both minor and major interruptions. Our primary contributions include the development of an adaptive, ROS-based HRI framework with an open-source code base. This framework supports natural interactions through advanced speech recognition and voice activity detection, and leverages a large language model (LLM) as a dialogue bridge. We validate the efficiency of our framework through module tests and system trials, demonstrating its high accuracy in age recognition and its robustness to repeated user inputs and plan changes.

cs.RO↗

arXiv2024

Details Make a Difference: Object State-Sensitive Neurorobotic Task Planning

The state of an object reflects its current status or condition and is important for a robot's task planning and manipulation. However, detecting an object's state and generating a state-sensitive plan for robots is challenging. Recently, pre-trained Large Language Models (LLMs) and Vision-Language Models (VLMs) have shown impressive capabilities in generating plans. However, to the best of our knowledge, there is hardly any investigation on whether LLMs or VLMs can also generate object state-sensitive plans. To study this, we introduce an Object State-Sensitive Agent (OSSA), a task-planning agent empowered by pre-trained neural networks. We propose two methods for OSSA: (i) a modular model consisting of a pre-trained vision processing module (dense captioning model, DCM) and a natural language processing model (LLM), and (ii) a monolithic model consisting only of a VLM. To quantitatively evaluate the performances of the two methods, we use tabletop scenarios where the task is to clear the table. We contribute a multimodal benchmark dataset that takes object states into consideration. Our results show that both methods can be used for object state-sensitive tasks, but the monolithic approach outperforms the modular approach. The code for OSSA is available at https://github.com/Xiao-wen-Sun/OSSA

cs.AI↗

arXiv2024

Fire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning

The rapid progress in Deep Learning (DL) and Large Language Models (LLMs) has exponentially increased demands of computational power and bandwidth. This, combined with the high costs of faster computing chips and interconnects, has significantly inflated High Performance Computing (HPC) construction costs. To address these challenges, we introduce the Fire-Flyer AI-HPC architecture, a synergistic hardware-software co-design framework and its best practices. For DL training, we deployed the Fire-Flyer 2 with 10,000 PCIe A100 GPUs, achieved performance approximating the DGX-A100 while reducing costs by half and energy consumption by 40%. We specifically engineered HFReduce to accelerate allreduce communication and implemented numerous measures to keep our Computation-Storage Integrated Network congestion-free. Through our software stack, including HaiScale, 3FS, and HAI-Platform, we achieved substantial scalability by overlapping computation and communication. Our system-oriented experience from DL training provides valuable insights to drive future advancements in AI-HPC.

cs.DC↗

arXiv2024

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

We present DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. It comprises 236B total parameters, of which 21B are activated for each token, and supports a context length of 128K tokens. DeepSeek-V2 adopts innovative architectures including Multi-head Latent Attention (MLA) and DeepSeekMoE. MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation. Compared with DeepSeek 67B, DeepSeek-V2 achieves significantly stronger performance, and meanwhile saves 42.5% of training costs, reduces the KV cache by 93.3%, and boosts the maximum generation throughput to 5.76 times. We pretrain DeepSeek-V2 on a high-quality and multi-source corpus consisting of 8.1T tokens, and further perform Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to fully unlock its potential. Evaluation results show that, even with only 21B activated parameters, DeepSeek-V2 and its chat versions still achieve top-tier performance among open-source models.

cs.CL↗

arXiv2024

Teaching Text-to-Image Models to Communicate in Dialog

A picture is worth a thousand words, thus, it is crucial for conversational agents to understand, perceive, and effectively respond with pictures. However, we find that directly employing conventional image generation techniques is inadequate for conversational agents to produce image responses effectively. In this paper, we focus on the innovative dialog-to-image generation task, where the model synthesizes a high-resolution image aligned with the given dialog context as a response. To tackle this problem, we design a tailored fine-tuning approach on the top of state-of-the-art text-to-image generation models to fully exploit the structural and semantic features in dialog context during image generation. Concretely, we linearize the dialog context with specific indicators to maintain the dialog structure, and employ in-domain data to alleviate the style mismatch between dialog-to-image and conventional image generation tasks. Empirical results on PhotoChat and MMDialog Corpus show that our approach brings consistent and remarkable improvement with 3 state-of-the-art pre-trained text-to-image generation backbones.

cs.CL↗

arXiv2023

Voltage-tunable giant nonvolatile multiple-state resistance in sliding-interlayer ferroelectric h-BN van der Waals multiferroic tunnel junction

Multiferroic tunnel junctions (MFTJs) based on two-dimensional (2D) van der Waals heterostructures with sharp and clean interfaces at the atomic scale are crucial for applications in nanoscale multi-resistive logic memory devices. The recently discovered sliding ferroelectricity in 2D van der Waals materials has opened new avenues for ferroelectric-based devices. Here, we theoretically investigate the spin-dependent electronic transport properties of Fe$_3$GeTe$_2$/graphene/bilayer-$h$-BN/graphene/CrI$_3$ (FGT/Gr-BBN-Gr/CrI) all-vdW MFTJs by employing the nonequilibrium Green's function combined with density functional theory. We demonstrate that such FGT/Gr-BBN-Gr/CrI MFTJs exhibit four non-volatile resistance states associated with different staking orders of sliding ferroelectric BBN and magnetization alignment of ferromagnetic free layer CrI$_3$, with a maximum tunnel magnetoresistance (electroresistance) ratio, i.e., TMR (TER) up to $\sim$$3.36\times10^{4}$\% ($\sim$$6.68\times10^{3}$\%) at a specific bias voltage. Furthermore, the perfect spin filtering and remarkable negative differential resistance effects are evident in our MFTJs. We further discover that the TMR, TER, and spin polarization ratio under an equilibrium state can be enhanced by the application of in-plane biaxial strain. This work shows that the giant tunneling resistance ratio, multiple resistance states, and excellent spin-polarized transport properties of sliding ferroelectric BBN-based MFTJs indicate its significant potential in nonvolatile memories.

cond-mat.mes-hall↗