arXiv subjects

Yan Xia

Publications and source records attributed to Yan Xia.

At least 145 records · Page 8Linked to original sources

arXiv2024

Low-code LLM: Graphical User Interface over Large Language Models

Utilizing Large Language Models (LLMs) for complex tasks is challenging, often involving a time-consuming and uncontrollable prompt engineering process. This paper introduces a novel human-LLM interaction framework, Low-code LLM. It incorporates six types of simple low-code visual programming interactions to achieve more controllable and stable responses. Through visual interaction with a graphical user interface, users can incorporate their ideas into the process without writing trivial prompts. The proposed Low-code LLM framework consists of a Planning LLM that designs a structured planning workflow for complex tasks, which can be correspondingly edited and confirmed by users through low-code visual programming operations, and an Executing LLM that generates responses following the user-confirmed workflow. We highlight three advantages of the low-code LLM: user-friendly interaction, controllable generation, and wide applicability. We demonstrate its benefits using four typical applications. By introducing this framework, we aim to bridge the gap between humans and LLMs, enabling more effective and efficient utilization of LLMs for complex tasks. The code, prompts, and experimental details are available at https://github.com/moymix/TaskMatrix/tree/main/LowCodeLLM. A system demonstration video can be found at https://www.youtube.com/watch?v=jb2C1vaeO3E.

cs.CL↗

arXiv2024

LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models

This paper presents a comprehensive survey of the current status and opportunities for Large Language Models (LLMs) in strategic reasoning, a sophisticated form of reasoning that necessitates understanding and predicting adversary actions in multi-agent settings while adjusting strategies accordingly. Strategic reasoning is distinguished by its focus on the dynamic and uncertain nature of interactions among multi-agents, where comprehending the environment and anticipating the behavior of others is crucial. We explore the scopes, applications, methodologies, and evaluation metrics related to strategic reasoning with LLMs, highlighting the burgeoning development in this area and the interdisciplinary approaches enhancing their decision-making performance. It aims to systematize and clarify the scattered literature on this subject, providing a systematic review that underscores the importance of strategic reasoning as a critical cognitive capability and offers insights into future research directions and potential improvements.

cs.CL↗

arXiv2024

Text2Loc: 3D Point Cloud Localization from Natural Language

We tackle the problem of 3D point cloud localization based on a few natural linguistic descriptions and introduce a novel neural network, Text2Loc, that fully interprets the semantic relationship between points and text. Text2Loc follows a coarse-to-fine localization pipeline: text-submap global place recognition, followed by fine localization. In global place recognition, relational dynamics among each textual hint are captured in a hierarchical transformer with max-pooling (HTM), whereas a balance between positive and negative pairs is maintained using text-submap contrastive learning. Moreover, we propose a novel matching-free fine localization method to further refine the location predictions, which completely removes the need for complicated text-instance matching and is lighter, faster, and more accurate than previous methods. Extensive experiments show that Text2Loc improves the localization accuracy by up to $2\times$ over the state-of-the-art on the KITTI360Pose dataset. Our project page is publicly available at \url{https://yan-xia.github.io/projects/text2loc/}.

cs.CV↗

arXiv2024

Unsupervised Domain Adaptation for Brain Vessel Segmentation through Transwarp Contrastive Learning

Unsupervised domain adaptation (UDA) aims to align the labelled source distribution with the unlabelled target distribution to obtain domain-invariant predictive models. Since cross-modality medical data exhibit significant intra and inter-domain shifts and most are unlabelled, UDA is more important while challenging in medical image analysis. This paper proposes a simple yet potent contrastive learning framework for UDA to narrow the inter-domain gap between labelled source and unlabelled target distribution. Our method is validated on cerebral vessel datasets. Experimental results show that our approach can learn latent features from labelled 3DRA modality data and improve vessel segmentation performance in unlabelled MRA modality data.

cs.CV↗

arXiv2024

GS-EMA: Integrating Gradient Surgery Exponential Moving Average with Boundary-Aware Contrastive Learning for Enhanced Domain Generalization in Aneurysm Segmentation

The automated segmentation of cerebral aneurysms is pivotal for accurate diagnosis and treatment planning. Confronted with significant domain shifts and class imbalance in 3D Rotational Angiography (3DRA) data from various medical institutions, the task becomes challenging. These shifts include differences in image appearance, intensity distribution, resolution, and aneurysm size, all of which complicate the segmentation process. To tackle these issues, we propose a novel domain generalization strategy that employs gradient surgery exponential moving average (GS-EMA) optimization technique coupled with boundary-aware contrastive learning (BACL). Our approach is distinct in its ability to adapt to new, unseen domains by learning domain-invariant features, thereby improving the robustness and accuracy of aneurysm segmentation across diverse clinical datasets. The results demonstrate that our proposed approach can extract more domain-invariant features, minimizing over-segmentation and capturing more complete aneurysm structures.

cs.CV↗

arXiv2024

ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agents

This paper introduces Alympics (Olympics for Agents), a systematic simulation framework utilizing Large Language Model (LLM) agents for game theory research. Alympics creates a versatile platform for studying complex game theory problems, bridging the gap between theoretical game theory and empirical investigations by providing a controlled environment for simulating human-like strategic interactions with LLM agents. In our pilot case study, the "Water Allocation Challenge," we explore Alympics through a challenging strategic game focused on the multi-round auction on scarce survival resources. This study demonstrates the framework's ability to qualitatively and quantitatively analyze game determinants, strategies, and outcomes. Additionally, we conduct a comprehensive human assessment and an in-depth evaluation of LLM agents in strategic decision-making scenarios. Our findings not only expand the understanding of LLM agents' proficiency in emulating human strategic behavior but also highlight their potential in advancing game theory knowledge, thereby enriching our understanding of both game theory and empowering further research into strategic decision-making domains with LLM agents. Codes, prompts, and all related resources are available at https://github.com/microsoft/Alympics.

cs.CL↗

arXiv2024

Sudden change of the photon output field marks phase transitions in the quantum Rabi model

The experimental observation of quantum phase transitions predicted by the quantum Rabi model in quantum critical systems is usually challenging due to the lack of signature experimental observables associated with them. Here, we describe a method to identify the dynamical critical phenomenon in the quantum Rabi model consisting of a three-level atom and a cavity at the quantum phase transition. Such a critical phenomenon manifests itself as a sudden change of steady-state output photons in the system driven by two classical fields, when both the atom and the cavity are initially unexcited. The process occurs as the high-frequency pump field is converted into the low-frequency Stokes field and multiple cavity photons in the normal phase, while this conversion cannot occur in the superradiant phase. The sudden change of steady-state output photons is an experimentally accessible measure to probe quantum phase transitions, as it does not require preparing the equilibrium state.

quant-ph↗

arXiv2024

Three-state coherent control using narrowband and passband sequences

In this work, we propose a comprehensive design for narrowband and passband composite pulse sequences by involving the dynamics of all states in the three-state system. The design is quite universal as all pulse parameters can be freely employed to modify the coefficients of error terms. Two modulation techniques, the strength and phase modulations, are used to achieve arbitrary population transfer with a desired excitation profile, while the system keeps minimal leakage to the third state. Furthermore, the current sequences are capable of tolerating inaccurate waveforms, detunings errors, and work well when rotating wave approximation is not strictly justified. Therefore, this work provides versatile adaptability for shaping various excitation profiles in both narrowband and passband sequences.

quant-ph↗

arXiv2023

Multi-Modal Domain Adaptation Across Video Scenes for Temporal Video Grounding

Temporal Video Grounding (TVG) aims to localize the temporal boundary of a specific segment in an untrimmed video based on a given language query. Since datasets in this domain are often gathered from limited video scenes, models tend to overfit to scene-specific factors, which leads to suboptimal performance when encountering new scenes in real-world applications. In a new scene, the fine-grained annotations are often insufficient due to the expensive labor cost, while the coarse-grained video-query pairs are easier to obtain. Thus, to address this issue and enhance model performance on new scenes, we explore the TVG task in an unsupervised domain adaptation (UDA) setting across scenes for the first time, where the video-query pairs in the source scene (domain) are labeled with temporal boundaries, while those in the target scene are not. Under the UDA setting, we introduce a novel Adversarial Multi-modal Domain Adaptation (AMDA) method to adaptively adjust the model's scene-related knowledge by incorporating insights from the target data. Specifically, we tackle the domain gap by utilizing domain discriminators, which help identify valuable scene-related features effective across both domains. Concurrently, we mitigate the semantic gap between different modalities by aligning video-query pairs with related semantics. Furthermore, we employ a mask-reconstruction approach to enhance the understanding of temporal semantics within a scene. Extensive experiments on Charades-STA, ActivityNet Captions, and YouCook2 demonstrate the effectiveness of our proposed method.

cs.CV↗

arXiv2023

Cross-modal Prompts: Adapting Large Pre-trained Models for Audio-Visual Downstream Tasks

In recent years, the deployment of large-scale pre-trained models in audio-visual downstream tasks has yielded remarkable outcomes. However, these models, primarily trained on single-modality unconstrained datasets, still encounter challenges in feature extraction for multi-modal tasks, leading to suboptimal performance. This limitation arises due to the introduction of irrelevant modality-specific information during encoding, which adversely affects the performance of downstream tasks. To address this challenge, this paper proposes a novel Dual-Guided Spatial-Channel-Temporal (DG-SCT) attention mechanism. This mechanism leverages audio and visual modalities as soft prompts to dynamically adjust the parameters of pre-trained models based on the current multi-modal input features. Specifically, the DG-SCT module incorporates trainable cross-modal interaction layers into pre-trained audio-visual encoders, allowing adaptive extraction of crucial information from the current modality across spatial, channel, and temporal dimensions, while preserving the frozen parameters of large-scale pre-trained models. Experimental evaluations demonstrate that our proposed model achieves state-of-the-art results across multiple downstream tasks, including AVE, AVVP, AVS, and AVQA. Furthermore, our model exhibits promising performance in challenging few-shot and zero-shot scenarios. The source code and pre-trained models are available at https://github.com/haoyi-duan/DG-SCT.

cs.LG↗

arXiv2023

Extensible Prompts for Language Models on Zero-shot Language Style Customization

We propose eXtensible Prompt (X-Prompt) for prompting a large language model (LLM) beyond natural language (NL). X-Prompt instructs an LLM with not only NL but also an extensible vocabulary of imaginary words. Registering new imaginary words allows us to instruct the LLM to comprehend concepts that are difficult to describe with NL words, thereby making a prompt more descriptive. Also, these imaginary words are designed to be out-of-distribution (OOD) robust so that they can be (re)used like NL words in various prompts, distinguishing X-Prompt from soft prompt that is for fitting in-distribution data. We propose context-augmented learning (CAL) to learn imaginary words for general usability, enabling them to work properly in OOD (unseen) prompts. We experiment X-Prompt for zero-shot language style customization as a case study. The promising results of X-Prompt demonstrate its potential to facilitate advanced interaction beyond the natural language interface, bridging the communication gap between humans and LLMs.

cs.CL↗

arXiv2023

Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting

Large language models (LLMs) demonstrate impressive multilingual capability, but their performance varies substantially across different languages. In this work, we introduce a simple yet effective method, called cross-lingual-thought prompting (XLT), to systematically improve the multilingual capability of LLMs. Specifically, XLT is a generic template prompt that stimulates cross-lingual and logical reasoning skills to enhance task performance across languages. We conduct comprehensive evaluations on 7 typical benchmarks related to reasoning, understanding, and generation tasks, covering both high-resource and low-resource languages. Experimental results show that XLT not only remarkably enhances the performance of various multilingual tasks but also significantly reduces the gap between the average performance and the best performance of each task in different languages. Notably, XLT brings over 10 points of average improvement in arithmetic reasoning and open-domain question-answering tasks.

cs.CL↗

arXiv2023

EIPE-text: Evaluation-Guided Iterative Plan Extraction for Long-Form Narrative Text Generation

Plan-and-Write is a common hierarchical approach in long-form narrative text generation, which first creates a plan to guide the narrative writing. Following this approach, several studies rely on simply prompting large language models for planning, which often yields suboptimal results. In this paper, we propose a new framework called Evaluation-guided Iterative Plan Extraction for long-form narrative text generation (EIPE-text), which extracts plans from the corpus of narratives and utilizes the extracted plans to construct a better planner. EIPE-text has three stages: plan extraction, learning, and inference. In the plan extraction stage, it iteratively extracts and improves plans from the narrative corpus and constructs a plan corpus. We propose a question answer (QA) based evaluation mechanism to automatically evaluate the plans and generate detailed plan refinement instructions to guide the iterative improvement. In the learning stage, we build a better planner by fine-tuning with the plan corpus or in-context learning with examples in the plan corpus. Finally, we leverage a hierarchical approach to generate long-form narratives. We evaluate the effectiveness of EIPE-text in the domains of novels and storytelling. Both GPT-4-based evaluations and human evaluations demonstrate that our method can generate more coherent and relevant long-form narratives. Our code will be released in the future.

cs.CL↗

arXiv2023

Revealing inherent quantum interference and entanglement of a Dirac particle

Although originally predicted in relativistic quantum mechanics, Zitterbewegung can also appear in some classical systems, which leads to the important question of whether Zitterbewegung of Dirac particles is underlain by a more fundamental and universal interference behavior without classical analogs. We here reveal such an interference pattern in phase space, which underlies but goes beyond Zitterbewegung, and whose nonclassicality is manifested by the negativity of the phase space quasiprobability distribution, and the associated pseudospin-momentum entanglement. We confirm this discovery by numerical simulation and an on-chip experiment, where a superconducting qubit and a quantized microwave field respectively emulate the internal and external degrees of freedom of a Dirac particle. The measured quasiprobability negativities agree well with the numerical simulation. Besides being of fundamental importance, the demonstrated nonclassical effects are useful in quantum technology.

quant-ph↗

arXiv2023

Thermal-dephasing-tolerant generation of Schrödinger cat states with Rydberg dressed blockade

Multipartite entangled states involving non-locality are one of the most fascinating characteristics of quantum mechanics. In this work, we propose a thermal-dephasing-tolerant generation of mesoscopic entangled states with Rydberg dressed atoms. We encode logical state on dressed states rather than Rydberg states. Such treatment can increase the lifetime of multipartite entanglement coherence to around 3 times compared to the Rydberg-state-coding one at the same system size, and therefore induce solid fidelities of mesoscopic superposition states generation. The current work theoretically verifies the advantages of using Rydberg dressed states in many-body quantum entanglement, which is helpful for large-scale quantum computation and many-body Rydberg quantum simulation.

quant-ph↗

arXiv2023

Machine-learning-inspired quantum optimal control of nonadiabatic geometric quantum computation via reverse engineering

Quantum control plays an irreplaceable role in practical use of quantum computers. However, some challenges have to be overcome to find more suitable and diverse control parameters. We propose a promising and generalizable average-fidelity-based machine-learning-inspired method to optimize the control parameters, in which a neural network with periodic feature enhancement is used as an ansatz. In the implementation of a single-qubit gate by cat-state nonadiabatic geometric quantum computation via reverse engineering, compared with the control parameters in the simple form of a trigonometric function, our approach can yield significantly higher-fidelity ($>99.99\%$) phase gates, such as the $π/ 8$ gate (T gate). Single-qubit gates are robust against systematic noise, additive white Gaussian noise and decoherence. We numerically demonstrate that the neural network possesses the ability to expand the model space. With the help of our optimization, we provide a feasible way to implement cascaded multi-qubit gates with high quality in a bosonic system. Therefore, the machine-learning-inspired method may be feasible in quantum optimal control of nonadiabatic geometric quantum computation.

quant-ph↗

arXiv2023

SCP: Scene Completion Pre-training for 3D Object Detection

3D object detection using LiDAR point clouds is a fundamental task in the fields of computer vision, robotics, and autonomous driving. However, existing 3D detectors heavily rely on annotated datasets, which are both time-consuming and prone to errors during the process of labeling 3D bounding boxes. In this paper, we propose a Scene Completion Pre-training (SCP) method to enhance the performance of 3D object detectors with less labeled data. SCP offers three key advantages: (1) Improved initialization of the point cloud model. By completing the scene point clouds, SCP effectively captures the spatial and semantic relationships among objects within urban environments. (2) Elimination of the need for additional datasets. SCP serves as a valuable auxiliary network that does not impose any additional efforts or data requirements on the 3D detectors. (3) Reduction of the amount of labeled data for detection. With the help of SCP, the existing state-of-the-art 3D detectors can achieve comparable performance while only relying on 20% labeled data.

cs.CV↗

arXiv2023

Observation of a superradiant phase transition with emergent cat states

Superradiant phase transitions (SPTs) are important for understanding light-matter interactions at the quantum level, and play a central role in criticality-enhanced quantum sensing. So far, SPTs have been observed in driven-dissipative systems, but the emergent light fields did not show any nonclassical characteristic due to the presence of strong dissipation. Here we report an experimental demonstration of the SPT featuring the emergence of a highly nonclassical photonic field, realized with a resonator coupled to a superconducting qubit, implementing the quantum Rabi model. We fully characterize the light-matter state by Wigner matrix tomography. The measured matrix elements exhibit quantum interference intrinsic of a photonic mesoscopic superposition, and reveal light-matter entanglement

quant-ph↗