arXiv subjects

Ke Wang

Publications and source records attributed to Ke Wang.

At least 289 records · Page 16Linked to original sources

arXiv2024

Emergence of steady quantum transport in a superconducting processor

Non-equilibrium quantum transport is crucial to technological advances ranging from nanoelectronics to thermal management. In essence, it deals with the coherent transfer of energy and (quasi-)particles through quantum channels between thermodynamic baths. A complete understanding of quantum transport thus requires the ability to simulate and probe macroscopic and microscopic physics on equal footing. Using a superconducting quantum processor, we demonstrate the emergence of non-equilibrium steady quantum transport by emulating the baths with qubit ladders and realising steady particle currents between the baths. We experimentally show that the currents are independent of the microscopic details of bath initialisation, and their temporal fluctuations decrease rapidly with the size of the baths, emulating those predicted by thermodynamic baths. The above characteristics are experimental evidence of pure-state statistical mechanics and prethermalisation in non-equilibrium many-body quantum systems. Furthermore, by utilising precise controls and measurements with single-site resolution, we demonstrate the capability to tune steady currents by manipulating the macroscopic properties of the baths, including filling and spectral properties. Our investigation paves the way for a new generation of experimental exploration of non-equilibrium quantum transport in strongly correlated quantum matter.

quant-ph↗

arXiv2024

OPUS: Occupancy Prediction Using a Sparse Set

Occupancy prediction, aiming at predicting the occupancy status within voxelized 3D environment, is quickly gaining momentum within the autonomous driving community. Mainstream occupancy prediction works first discretize the 3D environment into voxels, then perform classification on such dense grids. However, inspection on sample data reveals that the vast majority of voxels is unoccupied. Performing classification on these empty voxels demands suboptimal computation resource allocation, and reducing such empty voxels necessitates complex algorithm designs. To this end, we present a novel perspective on the occupancy prediction task: formulating it as a streamlined set prediction paradigm without the need for explicit space modeling or complex sparsification procedures. Our proposed framework, called OPUS, utilizes a transformer encoder-decoder architecture to simultaneously predict occupied locations and classes using a set of learnable queries. Firstly, we employ the Chamfer distance loss to scale the set-to-set comparison problem to unprecedented magnitudes, making training such model end-to-end a reality. Subsequently, semantic classes are adaptively assigned using nearest neighbor search based on the learned locations. In addition, OPUS incorporates a suite of non-trivial strategies to enhance model performance, including coarse-to-fine learning, consistent point sampling, and adaptive re-weighting, etc. Finally, compared with current state-of-the-art methods, our lightest model achieves superior RayIoU on the Occ3D-nuScenes dataset at near 2x FPS, while our heaviest model surpasses previous best results by 6.1 RayIoU.

cs.CV↗

arXiv2024

FM-Fusion: Instance-aware Semantic Mapping Boosted by Vision-Language Foundation Models

Semantic mapping based on the supervised object detectors is sensitive to image distribution. In real-world environments, the object detection and segmentation performance can lead to a major drop, preventing the use of semantic mapping in a wider domain. On the other hand, the development of vision-language foundation models demonstrates a strong zero-shot transferability across data distribution. It provides an opportunity to construct generalizable instance-aware semantic maps. Hence, this work explores how to boost instance-aware semantic mapping from object detection generated from foundation models. We propose a probabilistic label fusion method to predict close-set semantic classes from open-set label measurements. An instance refinement module merges the over-segmented instances caused by inconsistent segmentation. We integrate all the modules into a unified semantic mapping system. Reading a sequence of RGB-D input, our work incrementally reconstructs an instance-aware semantic map. We evaluate the zero-shot performance of our method in ScanNet and SceneNN datasets. Our method achieves 40.3 mean average precision (mAP) on the ScanNet semantic instance segmentation task. It outperforms the traditional semantic mapping method significantly.

cs.CV↗

arXiv2024

Beyond the Phase Ordering Problem: Finding the Globally Optimal Code w.r.t. Optimization Phases

In this paper, we propose a new concept called \textit{semantically equivalence} \wrt \textit{optimization phases} \textit{(\sep)}, which defines the set of programs a compiler considers semantically equivalent to the input using a set of optimization phases. We show both theoretically and empirically that solving the phase ordering problem does not necessarily result in the most efficient code among all programs that a compiler deems semantically equivalent to the input, hereinafter referred to as the global optimal code \wrt optimization phases. To find the global optimal code \wrt optimization phases, we present a conceptual framework, leveraging the reverse of existing optimization phases. In theory, we prove that the framework is capable of finding the global optimal code for any program. We realize this framework into a technique, called \textit{iterative bi-directional optimization (\tool)}, which performs both the normal and reverse optimizations to increase and decrease the efficiency of the generated code, respectively. We evaluate \tool on C/C++ files randomly extracted from highly mature and influential programs (\eg, Linux kernel, OpenSSL, Z3). Results show that \tool frequently generates more efficient code -- measured by either code size or runtime performance -- than exhaustive search, which is the solution to the phase ordering problem. We also find by simply incorporating \tool's reverse optimization phases, the effectiveness of the optimization of state-of-the-art compilers (\eg, GCC/LLVM) can be significantly improved.

cs.PL↗

arXiv2024

ATOMS: ALMA three-millimeter observations of massive star-forming regions -- XVIII. On the origin and evolution of dense gas fragments in molecular shells of compact HII regions

Fragmentation and evolution for the molecular shells of the compact HII regions are less explored compared to their evolved counterparts. We map nine compact HII regions with a typical diameter of 0.4 pc that are surrounded by molecular shells traced by CCH. Several to a dozen dense gas fragments probed by H13CO+ are embedded in these molecular shells. These gas fragments, strongly affected by the HII region, have a higher surface density, mass, and turbulence than those outside the shells but within the same pc-scale natal clump. These features suggest that the shells swept up by the early HII regions can enhance the formation of massive dense structures that may host the birth of higher-mass stars. We examine the formation of fragments and find that fragmentation of the swept-up shell is unlikely to occur in these early HII regions, by comparing the expected time scale of shell fragmentation with the age of HII region. We propose that the appearance of gas fragments in these shells is probably the result of sweeping up pre-existing fragments into the molecular shell that has not yet fragmented. Taken together, this work provides a basis for understanding the interplay of star-forming sites with an intricate environment containing ionization feedback such as those observed in starburst regions.

astro-ph.GA↗

arXiv2024

An Electoral Approach to Diversify LLM-based Multi-Agent Collective Decision-Making

Modern large language models (LLMs) have exhibited cooperative synergy on complex task-solving, and collective decision-making (CDM) is a pivotal component in LLM-based multi-agent collaboration frameworks. Our survey on 52 recent such systems uncovers a severe lack of diversity, with a heavy reliance on dictatorial and plurality voting for CDM. Through the lens of social choice theory, we scrutinize widely-adopted CDM methods and identify their limitations. To enrich current landscape of LLM-based CDM, we present GEDI, an electoral CDM module that incorporates various ordinal preferential voting mechanisms. Our empirical case study across three benchmarks shows that the integration of certain CDM methods can markedly improve the reasoning capabilities and robustness of some leading LLMs, all without requiring intricate system designs. Additionally, we find that some CDM mechanisms generate positive synergies even with as few as three agents. The voting-based methods also demonstrate robustness against single points of failure, as well as diversity in terms of hit-rate@k and subject-wise impacts.

cs.CL↗

arXiv2024

The Milky Way atlas for linear filaments II. clump rotation versus filament orientation

Dense clumps distributed along filaments are the immediate medium for star formation. Kinematic properties of the clumps, such as velocity gradient and angular momentum, combined with filament orientation, provide important clues to the formation mechanism of filament-clump configurations and the role of filaments in star formation. By cross-matching the Milky Way atlas for linear filaments and the Structure, Excitation and Dynamics of the Inner Galactic Interstellar Medium (SEDIGISM) 13CO (2-1) data, we aim to derive the velocity gradient and its direction, the specific angular momentum (J/M), and the ratio (β) between the rotational energy and gravitational energy of clumps, as well as to investigate the alignment between clump rotation and filament orientation. We found a monotonic increase in J/M as a function of clump size (R), following a power-law relation J/M~\propto~R^{1.5\pm0.2}. The ratio βranges from 1.1~\times~10^{-5} to 0.1, with a median value 1.0~\times~10^{-3}, suggesting that clump rotation provides insignificant support against gravitational collapse. The distribution of the angle between clump rotation and natal filament orientation is random, indicating that the clumps' rotational axes have no discernible correlation with the orientation of their hosting filaments. Counting only the most massive clump in each filament also finds no alignment between clump rotation and filament orientation.

astro-ph.GA↗

arXiv2024

ORCHID: A Chinese Debate Corpus for Target-Independent Stance Detection and Argumentative Dialogue Summarization

Dialogue agents have been receiving increasing attention for years, and this trend has been further boosted by the recent progress of large language models (LLMs). Stance detection and dialogue summarization are two core tasks of dialogue agents in application scenarios that involve argumentative dialogues. However, research on these tasks is limited by the insufficiency of public datasets, especially for non-English languages. To address this language resource gap in Chinese, we present ORCHID (Oral Chinese Debate), the first Chinese dataset for benchmarking target-independent stance detection and debate summarization. Our dataset consists of 1,218 real-world debates that were conducted in Chinese on 476 unique topics, containing 2,436 stance-specific summaries and 14,133 fully annotated utterances. Besides providing a versatile testbed for future research, we also conduct an empirical study on the dataset and propose an integrated task. The results show the challenging nature of the dataset and suggest a potential of incorporating stance detection in summarization for argumentative dialogue.

cs.CL↗

arXiv2024

A Survey on Data Synthesis and Augmentation for Large Language Models

The success of Large Language Models (LLMs) is inherently linked to the availability of vast, diverse, and high-quality data for training and evaluation. However, the growth rate of high-quality data is significantly outpaced by the expansion of training datasets, leading to a looming data exhaustion crisis. This underscores the urgent need to enhance data efficiency and explore new data sources. In this context, synthetic data has emerged as a promising solution. Currently, data generation primarily consists of two major approaches: data augmentation and synthesis. This paper comprehensively reviews and summarizes data generation techniques throughout the lifecycle of LLMs, including data preparation, pre-training, fine-tuning, instruction-tuning, preference alignment, and applications. Furthermore, We discuss the current constraints faced by these methods and investigate potential pathways for future development and research. Our aspiration is to equip researchers with a clear understanding of these methodologies, enabling them to swiftly identify appropriate data generation strategies in the construction of LLMs, while providing valuable insights for future exploration.

cs.CL↗

arXiv2024

Creating and controlling global Greenberger-Horne-Zeilinger entanglement on quantum processors

Greenberger-Horne-Zeilinger (GHZ) states, also known as two-component Schrödinger cats, play vital roles in the foundation of quantum physics and, more attractively, in future quantum technologies such as fault-tolerant quantum computation. Enlargement in size and coherent control of GHZ states are both crucial for harnessing entanglement in advanced computational tasks with practical advantages, which unfortunately pose tremendous challenges as GHZ states are vulnerable to noise. Here we propose a general strategy for creating, preserving, and manipulating large-scale GHZ entanglement, and demonstrate a series of experiments underlined by high-fidelity digital quantum circuits. For initialization, we employ a scalable protocol to create genuinely entangled GHZ states with up to 60 qubits, almost doubling the previous size record. For protection, we take a new perspective on discrete time crystals (DTCs), originally for exploring exotic nonequilibrium quantum matters, and embed a GHZ state into the eigenstates of a tailor-made cat scar DTC to extend its lifetime. For manipulation, we switch the DTC eigenstates with in-situ quantum gates to modify the effectiveness of the GHZ protection. Our findings establish a viable path towards coherent operations on large-scale entanglement, and further highlight superconducting processors as a promising platform to explore nonequilibrium quantum matters and emerging applications.

quant-ph↗

arXiv2024

Measuring the Inconsistency of Large Language Models in Preferential Ranking

Despite large language models' (LLMs) recent advancements, their bias and hallucination issues persist, and their ability to offer consistent preferential rankings remains underexplored. This study investigates the capacity of LLMs to provide consistent ordinal preferences, a crucial aspect in scenarios with dense decision space or lacking absolute answers. We introduce a formalization of consistency based on order theory, outlining criteria such as transitivity, asymmetry, reversibility, and independence from irrelevant alternatives. Our diagnostic experiments on selected state-of-the-art LLMs reveal their inability to meet these criteria, indicating a strong positional bias and poor transitivity, with preferences easily swayed by irrelevant alternatives. These findings highlight a significant inconsistency in LLM-generated preferential rankings, underscoring the need for further research to address these limitations.

cs.CL↗

arXiv2024

AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories

Fine-tuning on agent-environment interaction trajectory data holds significant promise for surfacing generalized agent capabilities in open-source large language models (LLMs). In this work, we introduce AgentBank, by far the largest trajectory tuning data collection featuring more than 50k diverse high-quality interaction trajectories which comprises 16 tasks covering five distinct agent skill dimensions. Leveraging a novel annotation pipeline, we are able to scale the annotated trajectories and generate a trajectory dataset with minimized difficulty bias. Furthermore, we fine-tune LLMs on AgentBank to get a series of agent models, Samoyed. Our comparative experiments demonstrate the effectiveness of scaling the interaction trajectory data to acquire generalized agent capabilities. Additional studies also reveal some key observations regarding trajectory tuning and agent skill generalization.

cs.CL↗

arXiv2024

MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code

Code has been shown to be effective in enhancing the mathematical reasoning abilities of large language models due to its precision and accuracy. Previous works involving continued mathematical pretraining often include code that utilizes math-related packages, which are primarily designed for fields such as engineering, machine learning, signal processing, or module testing, rather than being directly focused on mathematical reasoning. In this paper, we introduce a novel method for generating mathematical code accompanied with corresponding reasoning steps for continued pretraining. Our approach begins with the construction of a high-quality mathematical continued pretraining dataset by incorporating math-related web data, code using mathematical packages, math textbooks, and synthetic data. Next, we construct reasoning steps by extracting LaTeX expressions, the conditions needed for the expressions, and the results of the expressions from the previously collected dataset. Based on this extracted information, we generate corresponding code to accurately capture the mathematical reasoning process. Appending the generated code to each reasoning step results in data consisting of paired natural language reasoning steps and their corresponding code. Combining this data with the original dataset results in a 19.2B-token high-performing mathematical pretraining corpus, which we name MathCode-Pile. Training several popular base models with this corpus significantly improves their mathematical abilities, leading to the creation of the MathCoder2 family of models. All of our data processing and training code is open-sourced, ensuring full transparency and easy reproducibility of the entire data collection and training pipeline. The code is released at https://github.com/mathllm/MathCoder2 .

cs.CL↗

arXiv2024

Helium atom micro-diffraction as a characterisation tool for 2D materials

We present helium atom micro-diffraction as an ideal technique for characterization of 2D materials due to its ultimate surface sensitivity combined with sub-micron spatial resolution. Thermal energy neutral helium scatters from the valence electron density, 2-3A above the ionic cores of a surface, making the technique ideal for studying 2D materials, where other approaches can struggle due to small interaction cross-sections with few-layer samples. Sub-micron spatial resolution is key development in neutral atom scattering to allow measurements from device-scale samples. We present measurements of monolayer-substrate interactions, thermal expansion coefficients, the electron-phonon coupling constant and vacancy-type defect density on monolayer-MoS2. We also discuss extensions to the presented methods which can be immediately implemented on existing instruments to perform spatial mapping of these material properties.

physics.app-ph↗

arXiv2024

Giant and Tunable Bosonic Quantum Interference Induced by Two-Dimensional Metals

Harnessing quantum interference among bosons provides significant opportunities as bosons often carry longer coherence time than fermions. As an example of quantum interference, Fano resonance involving phonons or photons describes the coupling between discrete and continuous states, signified by an asymmetric spectral lineshape. Utilizing photon-based Fano resonance, molecule sensing with ultra-high sensitivity and ultrafast optical switching has been realized. However, phonon-based Fano resonance, which would expand the application space to a vaster regime, has been less exploited because of the weak coupling between discrete phonons with continuous states such as electronic continuum. In this work, we report the discovery of giant phonon-based Fano resonance in a graphene/2D Ag/SiC heterostructure. The Fano asymmetry, being proportional to the coupling strength, exceeds prior reports by two orders of magnitude. This Fano asymmetry arises from simultaneous frequency and lifetime matching between discrete and continuous phonons of SiC. The introduction of 2D Ag layers restructures SiC at the interface and facilitates resonant scattering to further enhance the Fano asymmetry, which is not achievable with conventional Ag thin films. With these unique properties, we demonstrated that the phonon-based Fano resonance can be used for ultrasensitive molecule detection at the single-molecule level. Our work highlights strong Fano resonance in the phononic system, opening avenues for engineering quantum interference based on bosons. Further, our findings provide opportunities for advancing phonon-related applications, including biochemical sensing, quantum transduction, and superconductor-based quantum computing.

cond-mat.mtrl-sci↗

arXiv2024

Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinement

Large language model agents have exhibited exceptional performance across a range of complex interactive tasks. Recent approaches have utilized tuning with expert trajectories to enhance agent performance, yet they primarily concentrate on outcome rewards, which may lead to errors or suboptimal actions due to the absence of process supervision signals. In this paper, we introduce the Iterative step-level Process Refinement (IPR) framework, which provides detailed step-by-step guidance to enhance agent training. Specifically, we adopt the Monte Carlo method to estimate step-level rewards. During each iteration, the agent explores along the expert trajectory and generates new actions. These actions are then evaluated against the corresponding step of expert trajectory using step-level rewards. Such comparison helps identify discrepancies, yielding contrastive action pairs that serve as training data for the agent. Our experiments on three complex agent tasks demonstrate that our framework outperforms a variety of strong baselines. Moreover, our analytical findings highlight the effectiveness of IPR in augmenting action efficiency and its applicability to diverse models.

cs.CL↗

arXiv2024

Quantum continual learning on a programmable superconducting processor

Quantum computers may outperform classical computers on machine learning tasks. In recent years, a variety of quantum algorithms promising unparalleled potential to enhance, speed up, or innovate machine learning have been proposed. Yet, quantum learning systems, similar to their classical counterparts, may likewise suffer from the catastrophic forgetting problem, where training a model with new tasks would result in a dramatic performance drop for the previously learned ones. This problem is widely believed to be a crucial obstacle to achieving continual learning of multiple sequential tasks. Here, we report an experimental demonstration of quantum continual learning on a fully programmable superconducting processor. In particular, we sequentially train a quantum classifier with three tasks, two about identifying real-life images and the other on classifying quantum states, and demonstrate its catastrophic forgetting through experimentally observed rapid performance drops for prior tasks. To overcome this dilemma, we exploit the elastic weight consolidation strategy and show that the quantum classifier can incrementally learn and retain knowledge across the three distinct tasks, with an average prediction accuracy exceeding 92.3%. In addition, for sequential tasks involving quantum-engineered data, we demonstrate that the quantum classifier can achieve a better continual learning performance than a commonly used classical feedforward network with a comparable number of variational parameters. Our results establish a viable strategy for empowering quantum learning systems with desirable adaptability to multiple sequential tasks, marking an important primary experimental step towards the long-term goal of achieving quantum artificial general intelligence.

quant-ph↗

arXiv2024

Massive Star Formation Starts in Sub-virial Dense Clumps Unless Resisted by Strong Magnetic Fields

The initial conditions are critical for understanding high-mass star formation, but are not well observed. Built on our previous characterization of a Galaxy-wide sample of 463 candidate high-mass starless clumps (HMSCs), here we investigate the dynamical state of a representative subsample of 44 HMSCs (radii 0.13-1.12 pc) using GBT NH3 (1,1) and (2,2) data from the Radio Ammonia Mid-Plane Survey (RAMPS) pilot data release. By fitting the two NH3 lines simultaneously, we obtain velocity dispersion, gas kinetic temperature, NH3 column density and abundance, Mach number, and virial parameter. Thermodynamic analysis reveals that most HMSCs have Mach number $<$5, inconsistent to what have been considered in theoretical models. All but one (43/44) of the HMSCs are gravitationally bound with virial parameter $α_{\mathrm{vir}} < 2$. Either these massive clumps are in collapsing or magnetic field strengths of 0.10-2.65 mG (average 0.51 mG) would be needed to support them against collapsing. The estimated B-field strength correlates tightly with density, $B_{\rm est}/{\rm mG}=0.269\,(n_{\rm H_2}/10^4\,{\rm cm^{-3}})^{0.61}$, with a similar power-law index as found in observations, but a factor of 4.6 higher in strength. For the first time, the initial dynamical state of high-mass formation regions has been statistically constrained to be sub-virial, in contradictory to theoretical models in virial equilibrium, and in agreement with the lack of observed massive starless cores. The findings urge future observations to quantify the magnetic field support in the prestellar stage of massive clumps, which are rarely explored so far, towards a full understanding of the physical conditions that initiate massive star formation.

astro-ph.GA↗