arXiv subjects

Xinyu Sun

Publications and source records attributed to Xinyu Sun.

At least 37 records · Page 2Linked to original sources

arXiv2025

A note on the Diversity Owen values

Béal et al. (Int J Game Theory 54, 2025) introduce the Diversity Owen value for TU-games with diversity constraints, and provide axiomatic characterizations using the axioms of fairness and balanced contributions. However, there exist logical flaws in the proofs of the uniqueness of these characterizations. In this note we provide the corrected proofs of the characterizations by introducing the null player for diverse games axiom. Also, we establish an alternative characterization of the Diversity Owen value by modifying the axioms of the above characterizations.

econ.TH↗

arXiv2025

Boundary operator expansion and extraordinary phase transition in the tricritical O(N) model

We study the boundary extraordinary transition of a three-dimensional (3D) tricritical $O(N)$ model. We first compute the mean-field Green's function with a general coupling of $|\vec ϕ|^{2n}$ (with $n=3$ corresponding to the tricritical model) at the extraordinary phase transition. Then, using layer susceptibility, we obtain the boundary operator expansion for the transverse and longitudinal modes within the $ε=3 - d$ expansion. Based on these results, we demonstrate that the tricritical point exhibits an extraordinary transition characterized by an ordered boundary for any $N$. This provides the first nontrivial example of continuous symmetry breaking in 2D in the context of boundary criticality.

cond-mat.str-el↗

arXiv2025

LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences

Research on 3D Vision-Language Models (3D-VLMs) is gaining increasing attention, which is crucial for developing embodied AI within 3D scenes, such as visual navigation and embodied question answering. Due to the high density of visual features, especially in large 3D scenes, accurately locating task-relevant visual information is challenging. Existing works attempt to segment all objects and consider their features as scene representations. However, these task-agnostic object features include much redundant information and missing details for the task-relevant area. To tackle these problems, we propose LSceneLLM, an adaptive framework that automatically identifies task-relevant areas by leveraging LLM's visual preference for different tasks, followed by a plug-and-play scene magnifier module to capture fine-grained details in focused areas. Specifically, a dense token selector examines the attention map of LLM to identify visual preferences for the instruction input. It then magnifies fine-grained details of the focusing area. An adaptive self-attention module is leveraged to fuse the coarse-grained and selected fine-grained visual information. To comprehensively evaluate the large scene understanding ability of 3D-VLMs, we further introduce a cross-room understanding benchmark, XR-Scene, which contains a series of large scene understanding tasks including XR-QA, XR-EmbodiedPlanning, and XR-SceneCaption. Experiments show that our method surpasses existing methods on both large scene understanding and existing scene understanding benchmarks. Plunging our scene magnifier module into the existing 3D-VLMs also brings significant improvement.

cs.CV↗

arXiv2025

Guided Debugging of Auto-Translated Code Using Differential Testing

Large Language Models (LLMs) hold great promise in the task of code translation. However, the lack of explainability complicates the identification of the inevitable translation errors. In this paper, we propose tHinter, a debugging tool to locate translation errors in auto-translated code. The core idea of tHinter is that correctly translated, the source and translated code should present the same functionalities, giving the same output for the same input. Hence, lines in the translated code responsible for output differences are possibly translation errors. First, tHinter employs fuzzing to generate diverse test cases that thoroughly explore the translated code. Then, tHinter relies on a heuristic algorithm to pinpoint translation errors from coverage information and differential testing execution results of those test cases. This heuristic algorithm is designed to leverage both the statistics and the expertise of developers. Comprehensive experiments with real code show its effectiveness. It reduces 71% lines developers need to review during debugging and increases the likelihood of the LLM fixing translation errors in a single query by 59%. Developers generally consider it satisfactory and helpful.

cs.SE↗

arXiv2024

Designing Phase Sensitive Probes of Monopole Superconducting Order

Distinct from familiar $s$-, $p$-, or $d$-wave pairings, the monopole superconducting order represents a novel class of pairing order arising from nontrivial monopole charge of the Cooper pair. In the weak-coupling regime, this order can emerge when pairing occurs between Fermi surfaces with different Chern numbers in, for example, doped Weyl semimetal systems. However, the phase of monopole pairing order is not well-defined over an entire Fermi surface, making it challenging to design experiments sensitive to both its symmetry and topology. To address this, we propose a scheme based on symmetry and topological principles to identify this elusive pairing order through a set of phase-sensitive Josephson experiments. By examining the discrepancy between global and local angular momentum of the pairing order, we can unveil the monopole charge of the pairing order. We demonstrate the proposed probe of monopole pairing order through analytic and numerical studies of Josephson coupling in models of monopole superconductor junctions. This work opens a promising avenue to uncover the unique topological properties of monopole pairing orders and to distinguish them from known pairing orders based on spherical harmonic symmetry.

cond-mat.supr-con↗

arXiv2024

Pro-PRIME: A general Temperature-Guided Language model to engineer enhanced Stability and Activity in Proteins

Designing protein mutants of both high stability and activity is a critical yet challenging task in protein engineering. Here, we introduce PRIME, a deep learning model, which can suggest protein mutants of improved stability and activity without any prior experimental mutagenesis data of the specified protein. Leveraging temperature-aware language modeling, PRIME demonstrated superior predictive power compared to current state-of-the-art models on the public mutagenesis dataset over 283 protein assays. Furthermore, we validated PRIME's predictions on five proteins, examining the top 30-45 single-site mutations' impact on various protein properties, including thermal stability, antigen-antibody binding affinity, and the ability to polymerize non-natural nucleic acid or resilience to extreme alkaline conditions. Remarkably, over 30% of the AI-recommended mutants exhibited superior performance compared to their pre-mutation counterparts across all proteins and desired properties. Moreover, we have developed an efficient, and successful method based on PRIME to rapidly obtain multi-site mutants with enhanced activity and stability. Hence, PRIME demonstrates the general applicability in protein engineering.

q-bio.BM↗

arXiv2024

Holographic dual of defect CFT with corner contributions

We study defect CFT within the framework of holographic duality, emphasizing the impact of corner contributions. We model distinct conformal defects using interface branes that differ in tensions and are connected by a corner. Employing the relationship between CFT scaling dimensions and Euclidean gravity actions, we outline a general procedure for calculating the anomalous dimensions of defect changing operators at nontrivial cusps. Several analytical results are obtained, including the cusp anomalous dimensions at big and small angles. While $1/ϕ$ universal divergence appears for small cusp angles due to the fusion of two defects, more interestingly, we uncover a bubble phase rendered by a near zero angle cusp, in which the divergence is absent.

hep-th↗

arXiv2024

Prioritized Semantic Learning for Zero-shot Instance Navigation

We study zero-shot instance navigation, in which the agent navigates to a specific object without using object annotations for training. Previous object navigation approaches apply the image-goal navigation (ImageNav) task (go to the location of an image) for pretraining, and transfer the agent to achieve object goals using a vision-language model. However, these approaches lead to issues of semantic neglect, where the model fails to learn meaningful semantic alignments. In this paper, we propose a Prioritized Semantic Learning (PSL) method to improve the semantic understanding ability of navigation agents. Specifically, a semantic-enhanced PSL agent is proposed and a prioritized semantic training strategy is introduced to select goal images that exhibit clear semantic supervision and relax the reward function from strict exact view matching. At inference time, a semantic expansion inference scheme is designed to preserve the same granularity level of the goal semantic as training. Furthermore, for the popular HM3D environment, we present an Instance Navigation (InstanceNav) task that requires going to a specific object instance with detailed descriptions, as opposed to the Object Navigation (ObjectNav) task where the goal is defined merely by the object category. Our PSL agent outperforms the previous state-of-the-art by 66% on zero-shot ObjectNav in terms of success rate and is also superior on the new InstanceNav task. Code will be released at https://github.com/XinyuSun/PSL-InstanceNav.

cs.CV↗

arXiv2024

CoNav: A Benchmark for Human-Centered Collaborative Navigation

Human-robot collaboration, in which the robot intelligently assists the human with the upcoming task, is an appealing objective. To achieve this goal, the agent needs to be equipped with a fundamental collaborative navigation ability, where the agent should reason human intention by observing human activities and then navigate to the human's intended destination in advance of the human. However, this vital ability has not been well studied in previous literature. To fill this gap, we propose a collaborative navigation (CoNav) benchmark. Our CoNav tackles the critical challenge of constructing a 3D navigation environment with realistic and diverse human activities. To achieve this, we design a novel LLM-based humanoid animation generation framework, which is conditioned on both text descriptions and environmental context. The generated humanoid trajectory obeys the environmental context and can be easily integrated into popular simulators. We empirically find that the existing navigation methods struggle in CoNav task since they neglect the perception of human intention. To solve this problem, we propose an intention-aware agent for reasoning both long-term and short-term human intention. The agent predicts navigation action based on the predicted intention and panoramic observation. The emergent agent behavior including observing humans, avoiding human collision, and navigation reveals the efficiency of the proposed datasets and agents.

cs.CV↗

arXiv2024

SKDF: A Simple Knowledge Distillation Framework for Distilling Open-Vocabulary Knowledge to Open-world Object Detector

In this paper, we attempt to specialize the VLM model for OWOD tasks by distilling its open-world knowledge into a language-agnostic detector. Surprisingly, we observe that the combination of a simple \textbf{knowledge distillation} approach and the automatic pseudo-labeling mechanism in OWOD can achieve better performance for unknown object detection, even with a small amount of data. Unfortunately, knowledge distillation for unknown objects severely affects the learning of detectors with conventional structures for known objects, leading to catastrophic forgetting. To alleviate these problems, we propose the \textbf{down-weight loss function} for knowledge distillation from vision-language to single vision modality. Meanwhile, we propose the \textbf{cascade decouple decoding structure} that decouples the learning of localization and recognition to reduce the impact of category interactions of known and unknown objects on the localization learning process. Ablation experiments demonstrate that both of them are effective in mitigating the impact of open-world knowledge distillation on the learning of known objects. Additionally, to alleviate the current lack of comprehensive benchmarks for evaluating the ability of the open-world detector to detect unknown objects in the open world, we propose two benchmarks, which we name "\textbf{StandardSet}$\heartsuit$" and "\textbf{IntensiveSet}$\spadesuit$" respectively, based on the complexity of their testing scenarios. Comprehensive experiments performed on OWOD, MS-COCO, and our proposed benchmarks demonstrate the effectiveness of our methods. The code and proposed dataset are available at \url{https://github.com/xiaomabufei/SKDF}.

cs.CV↗

arXiv2024

RGM: A Robust Generalizable Matching Model

Finding corresponding pixels within a pair of images is a fundamental computer vision task with various applications. Due to the specific requirements of different tasks like optical flow estimation and local feature matching, previous works are primarily categorized into dense matching and sparse feature matching focusing on specialized architectures along with task-specific datasets, which may somewhat hinder the generalization performance of specialized models. In this paper, we propose a deep model for sparse and dense matching, termed RGM (Robust Generalist Matching). In particular, we elaborately design a cascaded GRU module for refinement by exploring the geometric similarity iteratively at multiple scales following an additional uncertainty estimation module for sparsification. To narrow the gap between synthetic training samples and real-world scenarios, we build a new, large-scale dataset with sparse correspondence ground truth by generating optical flow supervision with greater intervals. As such, we are able to mix up various dense and sparse matching datasets, significantly improving the training diversity. The generalization capacity of our proposed RGM is greatly improved by learning the matching and uncertainty estimation in a two-stage manner on the large, mixed data. Superior performance is achieved for zero-shot matching and downstream geometry estimation across multiple datasets, outperforming the previous methods by a large margin.

cs.CV↗

arXiv2023

Holographic Weak Measurement

In this paper, we study a holographic description of weak measurements in conformal field theories (CFTs). Weak measurements can be viewed as a soft projection that interpolates between an identity operator and a projection operator, and can induce an effective central charge distinct from the unmeasured CFT. We model the weak measurement by an interface brane, separating different geometries dual to the post-measurement state and the unmeasured CFT, respectively. In an infinite system, the weak measurement is related to ICFT via a spacetime rotation. We find that the holographic entanglement entropy with twist operators located on the defect is consistent in both calculations for ICFT and weak measurements. We additionally calculate the boundary entropy via holographic entanglement as well as partition function. In a finite system, the weak measurement can lead to a rich phase diagram: for marginal measurements the emergent brane separates two AdS geometries, while for irrelevant measurements the post-measurement geometry features an AdS spacetime and a black hole spacetime that are separated by the brane. Although the measurement is irrelevant in the later phase, the post-measurement geometry can realize a Python's lunch.

hep-th↗

arXiv2023

Contrastive Vision-Language Alignment Makes Efficient Instruction Learner

We study the task of extending the large language model (LLM) into a vision-language instruction-following model. This task is crucial but challenging since the LLM is trained on text modality only, making it hard to effectively digest the visual modality. To address this, existing methods typically train a visual adapter to align the representation between a pre-trained vision transformer (ViT) and the LLM by a generative image captioning loss. However, we find that the generative objective can only produce weak alignment for vision and language, making the aligned vision-language model very hungry for the instruction fine-tuning data. In this paper, we propose CG-VLM that applies both Contrastive and Generative alignment objectives to effectively align the representation of ViT and LLM. Different from image level and sentence level alignment in common contrastive learning settings, CG-VLM aligns the image-patch level features and text-token level embeddings, which, however, is very hard to achieve as no explicit grounding patch-token relation provided in standard image captioning datasets. To address this issue, we propose to maximize the averaged similarity between pooled image-patch features and text-token embeddings. Extensive experiments demonstrate that the proposed CG-VLM produces strong vision-language alignment and is an efficient instruction learner. For example, using only 10% instruction tuning data, we reach 95% performance of state-of-the-art method LLaVA [29] on the zero-shot ScienceQA-Image benchmark.

cs.CV↗

arXiv2023

FGPrompt: Fine-grained Goal Prompting for Image-goal Navigation

Learning to navigate to an image-specified goal is an important but challenging task for autonomous systems. The agent is required to reason the goal location from where a picture is shot. Existing methods try to solve this problem by learning a navigation policy, which captures semantic features of the goal image and observation image independently and lastly fuses them for predicting a sequence of navigation actions. However, these methods suffer from two major limitations. 1) They may miss detailed information in the goal image, and thus fail to reason the goal location. 2) More critically, it is hard to focus on the goal-relevant regions in the observation image, because they attempt to understand observation without goal conditioning. In this paper, we aim to overcome these limitations by designing a Fine-grained Goal Prompting (FGPrompt) method for image-goal navigation. In particular, we leverage fine-grained and high-resolution feature maps in the goal image as prompts to perform conditioned embedding, which preserves detailed information in the goal image and guides the observation encoder to pay attention to goal-relevant regions. Compared with existing methods on the image-goal navigation benchmark, our method brings significant performance improvement on 3 benchmark datasets (i.e., Gibson, MP3D, and HM3D). Especially on Gibson, we surpass the state-of-the-art success rate by 8% with only 1/50 model size. Project page: https://xinyusun.github.io/fgprompt-pages

cs.CV↗

arXiv2023

$A^2$Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Models

We study the task of zero-shot vision-and-language navigation (ZS-VLN), a practical yet challenging problem in which an agent learns to navigate following a path described by language instructions without requiring any path-instruction annotation data. Normally, the instructions have complex grammatical structures and often contain various action descriptions (e.g., "proceed beyond", "depart from"). How to correctly understand and execute these action demands is a critical problem, and the absence of annotated data makes it even more challenging. Note that a well-educated human being can easily understand path instructions without the need for any special training. In this paper, we propose an action-aware zero-shot VLN method ($A^2$Nav) by exploiting the vision-and-language ability of foundation models. Specifically, the proposed method consists of an instruction parser and an action-aware navigation policy. The instruction parser utilizes the advanced reasoning ability of large language models (e.g., GPT-3) to decompose complex navigation instructions into a sequence of action-specific object navigation sub-tasks. Each sub-task requires the agent to localize the object and navigate to a specific goal position according to the associated action demand. To accomplish these sub-tasks, an action-aware navigation policy is learned from freely collected action-specific datasets that reveal distinct characteristics of each action demand. We use the learned navigation policy for executing sub-tasks sequentially to follow the navigation instruction. Extensive experiments show $A^2$Nav achieves promising ZS-VLN performance and even surpasses the supervised learning methods on R2R-Habitat and RxR-Habitat datasets.

cs.CV↗

arXiv2023

New critical states induced by measurement

Finding new critical states of matter is an important subject in modern many-body physics. Here we study the effect of measurement and postselection on the critical ground state of a Luttinger liquid theory and show that it can lead to qualitatively new critical states. Depending on the Luttinger parameter $K$, the effect of measurement is irrelevant (relevant) at $K>1$ ($K<1$). We reveal that this causes an entanglement transition between two phases, one with logarithmic entanglement entropy for a subregion ($K>1$), and the other with algebraic entanglement entropy ($K<1$). At the critical point $K=1$, the measurement is marginal, and we find new critical states whose entanglement entropy exhibits a logarithmic behavior with a continuous effective central charge as a function of measurement strength. We also performed numerical density matrix renormalization group and fermionic Gaussian state simulations to support our results. We further discuss promising and feasible routes to experimentally realize new critical states in our work.

quant-ph↗

arXiv2023

Masked Motion Encoding for Self-Supervised Video Representation Learning

How to learn discriminative video representation from unlabeled videos is challenging but crucial for video analysis. The latest attempts seek to learn a representation model by predicting the appearance contents in the masked regions. However, simply masking and recovering appearance contents may not be sufficient to model temporal clues as the appearance contents can be easily reconstructed from a single frame. To overcome this limitation, we present Masked Motion Encoding (MME), a new pre-training paradigm that reconstructs both appearance and motion information to explore temporal clues. In MME, we focus on addressing two critical challenges to improve the representation performance: 1) how to well represent the possible long-term motion across multiple frames; and 2) how to obtain fine-grained temporal clues from sparsely sampled videos. Motivated by the fact that human is able to recognize an action by tracking objects' position changes and shape changes, we propose to reconstruct a motion trajectory that represents these two kinds of change in the masked regions. Besides, given the sparse video input, we enforce the model to reconstruct dense motion trajectories in both spatial and temporal dimensions. Pre-trained with our MME paradigm, the model is able to anticipate long-term and fine-grained motion details. Code is available at https://github.com/XinyuSun/MME.

cs.CV↗

arXiv2022

Plug-Play Plasmonic Metafibers for Ultrafast Fiber Lasers

Metafibers expand the functionalities of conventional optical fibers to unprecedented nanoscale light manipulations by integrating metasurfaces on the fiber tips, becoming an emerging light-coupling platform for both nanoscience and fiber optics communities. Mostly exploring the isolated bare fibers, current metafibers remain as proof-of-concept demonstrations due to a lack of standard interfaces with the universal fiber networks. Here, we develop new methodologies to fabricate well-defined plasmonic metasurfaces directly on the end facets of commercial single mode fiber jumpers using standard planar technologies and provide a first demonstration of their practical applications in the nonlinear optics regime. Featuring plug-play connections with fiber circuitry and arbitrary metasurfaces landscapes, the metafibers with tunable plasmonic resonances are implemented into fiber laser cavities, yielding all-fiber sub-picosecond (minimum 513 fs) soliton mode locked lasers at optical wavelengths of 1.5 micrometer and 2 micrometer, demonstrating their unusual polarimetric nonlinear transfer functions and superior saturation absorption responses. Novel insights into the physical mechanisms behind the saturable absorption of plasmonic metasurfaces are provided. The nanofabrication process flow is compatible with existing cleanroom technologies, offering metafibers an avenue to be a regular member of functionalized fiber components. The work paves the way towards next generation of ultrafast fiber lasers, optical frequency combs, optical neural networks and ultracompact "all-in-fibers" optical systems for sensing, imaging, communications, and many others.

physics.optics↗