arXiv subjects

Xiaolong Wang

Publications and source records attributed to Xiaolong Wang.

At least 163 records · Page 9Linked to original sources

arXiv2024

DKINet: Medication Recommendation via Domain Knowledge Informed Deep Learning

Medication recommendation is a fundamental yet crucial branch of healthcare that presents opportunities to assist physicians in making more accurate medication prescriptions for patients with complex health conditions. Previous studies have primarily focused on learning patient representation from electronic health records (EHR). While considering the clinical manifestations of the patient is important, incorporating domain-specific prior knowledge is equally significant in diagnosing the patient's health conditions. However, effectively integrating domain knowledge with the patient's clinical manifestations can be challenging, particularly when dealing with complex clinical manifestations. Therefore, in this paper, we first identify comprehensive domain-specific prior knowledge, namely the Unified Medical Language System (UMLS), which is a comprehensive repository of biomedical vocabularies and standards, for knowledge extraction. Subsequently, we propose a knowledge injection module that addresses the effective integration of domain knowledge with complex clinical manifestations, enabling an effective characterization of the health conditions of the patient. Furthermore, considering the significant impact of a patient's medication history on their current medication, we introduce a historical medication-aware patient representation module to capture the longitudinal influence of historical medication information on the representation of current patients. Extensive experiments on three publicly benchmark datasets verify the superiority of our proposed method, which outperformed other methods by a significant margin. The code is available at: https://github.com/sherry6247/DKINet.

cs.AI↗

arXiv2024

DEEM: Dynamic Experienced Expert Modeling for Stance Detection

Recent work has made a preliminary attempt to use large language models (LLMs) to solve the stance detection task, showing promising results. However, considering that stance detection usually requires detailed background knowledge, the vanilla reasoning method may neglect the domain knowledge to make a professional and accurate analysis. Thus, there is still room for improvement of LLMs reasoning, especially in leveraging the generation capability of LLMs to simulate specific experts (i.e., multi-agents) to detect the stance. In this paper, different from existing multi-agent works that require detailed descriptions and use fixed experts, we propose a Dynamic Experienced Expert Modeling (DEEM) method which can leverage the generated experienced experts and let LLMs reason in a semi-parametric way, making the experts more generalizable and reliable. Experimental results demonstrate that DEEM consistently achieves the best results on three standard benchmarks, outperforms methods with self-consistency reasoning, and reduces the bias of LLMs.

cs.CL↗

arXiv2024

The Multilayer Nature of Molecular Gas toward the Cygnus Region

We study the physical properties and 3D distribution of molecular clouds (MCs) toward the Cygnus region using the MWISP CO survey and Gaia DR3 data. Based on Gaussian decomposition and clustering for $\rm ^{13}CO$ lines, over 70% of the fluxes are recovered. With the identification result of $\rm ^{13}CO$ structures, two models are designed to measure the distances of the molecular gas in velocity crowding regions. The distances of more than 200 large $\rm ^{13}CO$ structures are obtained toward the 150 square degree region. Additionally, tens of the identified MC structures coincide well with masers and/or intense mid-IR emission. We find multiple gas layers toward the region: (1) the extensive gas structures composing the Cygnus Rift from 700 pc to 1 kpc across the whole region; (2) the $\sim$ 1.3 kpc gas layer mainly in the Cygnus X South region; and (3) the 1.5 kpc dense filament at the Cygnus X North region and many cometary clouds shaped by Cygnus OB2. We also note that the spatial distribution of YSO candidates is generally consistent with the molecular gas structures. The total molecular mass of the Cygnus region is estimated to be $\sim 2.7\times10^{6}~M_{\odot}$ assuming an X-factor ratio $X_{\rm CO} = 2 \times 10^{20} \rm cm^{-2} (K\cdot km\cdot s^{-1})^{-1}$. The foreground Cygnus Rift contributes $\sim$25% of the molecular mass in the whole region. Our work presents a new 3D view of the MCs' distribution toward the Cygnus X region, as well as the exact molecular gas mass distribution in the foreground Cygnus Rift.

astro-ph.GA↗

arXiv2024

Editable Image Elements for Controllable Synthesis

Diffusion models have made significant advances in text-guided synthesis tasks. However, editing user-provided images remains challenging, as the high dimensional noise input space of diffusion models is not naturally suited for image inversion or spatial editing. In this work, we propose an image representation that promotes spatial editing of input images using a diffusion model. Concretely, we learn to encode an input into "image elements" that can faithfully reconstruct an input image. These elements can be intuitively edited by a user, and are decoded by a diffusion model into realistic images. We show the effectiveness of our representation on various image editing tasks, such as object resizing, rearrangement, dragging, de-occlusion, removal, variation, and image composition. Project page: https://jitengmu.github.io/Editable_Image_Elements/

cs.CV↗

arXiv2024

Feature Splatting: Language-Driven Physics-Based Scene Synthesis and Editing

Scene representations using 3D Gaussian primitives have produced excellent results in modeling the appearance of static and dynamic 3D scenes. Many graphics applications, however, demand the ability to manipulate both the appearance and the physical properties of objects. We introduce Feature Splatting, an approach that unifies physics-based dynamic scene synthesis with rich semantics from vision language foundation models that are grounded by natural language. Our first contribution is a way to distill high-quality, object-centric vision-language features into 3D Gaussians, that enables semi-automatic scene decomposition using text queries. Our second contribution is a way to synthesize physics-based dynamics from an otherwise static scene using a particle-based simulator, in which material properties are assigned automatically via text queries. We ablate key techniques used in this pipeline, to illustrate the challenge and opportunities in using feature-carrying 3D Gaussians as a unified format for appearance, geometry, material properties and semantics grounded on natural language. Project website: https://feature-splatting.github.io/

cs.CV↗

arXiv2024

Investigating and Mitigating the Side Effects of Noisy Views for Self-Supervised Clustering Algorithms in Practical Multi-View Scenarios

Multi-view clustering (MVC) aims at exploring category structures among multi-view data in self-supervised manners. Multiple views provide more information than single views and thus existing MVC methods can achieve satisfactory performance. However, their performance might seriously degenerate when the views are noisy in practical multi-view scenarios. In this paper, we formally investigate the drawback of noisy views and then propose a theoretically grounded deep MVC method (namely MVCAN) to address this issue. Specifically, we propose a novel MVC objective that enables un-shared parameters and inconsistent clustering predictions across multiple views to reduce the side effects of noisy views. Furthermore, a two-level multi-view iterative optimization is designed to generate robust learning targets for refining individual views' representation learning. Theoretical analysis reveals that MVCAN works by achieving the multi-view consistency, complementarity, and noise robustness. Finally, experiments on extensive public datasets demonstrate that MVCAN outperforms state-of-the-art methods and is robust against the existence of noisy views.

cs.LG↗

arXiv2024

Pluggable Neural Machine Translation Models via Memory-augmented Adapters

Although neural machine translation (NMT) models perform well in the general domain, it remains rather challenging to control their generation behavior to satisfy the requirement of different users. Given the expensive training cost and the data scarcity challenge of learning a new model from scratch for each user requirement, we propose a memory-augmented adapter to steer pretrained NMT models in a pluggable manner. Specifically, we construct a multi-granular memory based on the user-provided text samples and propose a new adapter architecture to combine the model representations and the retrieved results. We also propose a training strategy using memory dropout to reduce spurious dependencies between the NMT model and the memory. We validate our approach on both style- and domain-specific experiments and the results indicate that our method can outperform several representative pluggable baselines.

cs.CL↗

arXiv2024

TD-MPC2: Scalable, Robust World Models for Continuous Control

TD-MPC is a model-based reinforcement learning (RL) algorithm that performs local trajectory optimization in the latent space of a learned implicit (decoder-free) world model. In this work, we present TD-MPC2: a series of improvements upon the TD-MPC algorithm. We demonstrate that TD-MPC2 improves significantly over baselines across 104 online RL tasks spanning 4 diverse task domains, achieving consistently strong results with a single set of hyperparameters. We further show that agent capabilities increase with model and data size, and successfully train a single 317M parameter agent to perform 80 tasks across multiple task domains, embodiments, and action spaces. We conclude with an account of lessons, opportunities, and risks associated with large TD-MPC2 agents. Explore videos, models, data, code, and more at https://tdmpc2.com

cs.LG↗

arXiv2024

HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data

3D hand-object interaction data is scarce due to the hardware constraints in scaling up the data collection process. In this paper, we propose HOIDiffusion for generating realistic and diverse 3D hand-object interaction data. Our model is a conditional diffusion model that takes both the 3D hand-object geometric structure and text description as inputs for image synthesis. This offers a more controllable and realistic synthesis as we can specify the structure and style inputs in a disentangled manner. HOIDiffusion is trained by leveraging a diffusion model pre-trained on large-scale natural images and a few 3D human demonstrations. Beyond controllable image synthesis, we adopt the generated 3D data for learning 6D object pose estimation and show its effectiveness in improving perception systems. Project page: https://mq-zhang1.github.io/HOIDiffusion

cs.CV↗

arXiv2024

Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning

Using tactile sensors for manipulation remains one of the most challenging problems in robotics. At the heart of these challenges is generalization: How can we train a tactile-based policy that can manipulate unseen and diverse objects? In this paper, we propose to perform Reinforcement Learning with only visual tactile sensing inputs on diverse objects in a physical simulator. By training with diverse objects in simulation, it enables the policy to generalize to unseen objects. However, leveraging simulation introduces the Sim2Real transfer problem. To mitigate this problem, we study different tactile representations and evaluate how each affects real-robot manipulation results after transfer. We conduct our experiments on diverse real-world objects and show significant improvements over baselines for the pivoting task. Our project page is available at https://tactilerl.github.io/.

cs.RO↗

arXiv2024

DNAct: Diffusion Guided Multi-Task 3D Policy Learning

This paper presents DNAct, a language-conditioned multi-task policy framework that integrates neural rendering pre-training and diffusion training to enforce multi-modality learning in action sequence spaces. To learn a generalizable multi-task policy with few demonstrations, the pre-training phase of DNAct leverages neural rendering to distill 2D semantic features from foundation models such as Stable Diffusion to a 3D space, which provides a comprehensive semantic understanding regarding the scene. Consequently, it allows various applications to challenging robotic tasks requiring rich 3D semantics and accurate geometry. Furthermore, we introduce a novel approach utilizing diffusion training to learn a vision and language feature that encapsulates the inherent multi-modality in the multi-task demonstrations. By reconstructing the action sequences from different tasks via the diffusion process, the model is capable of distinguishing different modalities and thus improving the robustness and the generalizability of the learned representation. DNAct significantly surpasses SOTA NeRF-based multi-task manipulation approaches with over 30% improvement in success rate. Project website: dnact.github.io.

cs.RO↗

arXiv2024

Obligations and permissions, algebraically

We further develop the algebraic approach to input/output logic initiated in \cite{wollic22}, where subordination algebras and a family of their generalizations were proposed as a semantic environment of various input/output logics. In particular, we consider precontact algebras as a suitable algebraic environment for negative permission, and we characterize properties of several types of permission (negative, static, dynamic), as well as their interactions with normative systems, by means of suitable modal languages encoding outputs.

math.LO↗

arXiv2024

Expressive Whole-Body Control for Humanoid Robots

Can we enable humanoid robots to generate rich, diverse, and expressive motions in the real world? We propose to learn a whole-body control policy on a human-sized robot to mimic human motions as realistic as possible. To train such a policy, we leverage the large-scale human motion capture data from the graphics community in a Reinforcement Learning framework. However, directly performing imitation learning with the motion capture dataset would not work on the real humanoid robot, given the large gap in degrees of freedom and physical capabilities. Our method Expressive Whole-Body Control (Exbody) tackles this problem by encouraging the upper humanoid body to imitate a reference motion, while relaxing the imitation constraint on its two legs and only requiring them to follow a given velocity robustly. With training in simulation and Sim2Real transfer, our policy can control a humanoid robot to walk in different styles, shake hands with humans, and even dance with a human in the real world. We conduct extensive studies and comparisons on diverse motions in both simulation and the real world to show the effectiveness of our approach.

cs.RO↗

arXiv2024

CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulation

We introduce CyberDemo, a novel approach to robotic imitation learning that leverages simulated human demonstrations for real-world tasks. By incorporating extensive data augmentation in a simulated environment, CyberDemo outperforms traditional in-domain real-world demonstrations when transferred to the real world, handling diverse physical and visual conditions. Regardless of its affordability and convenience in data collection, CyberDemo outperforms baseline methods in terms of success rates across various tasks and exhibits generalizability with previously unseen objects. For example, it can rotate novel tetra-valve and penta-valve, despite human demonstrations only involving tri-valves. Our research demonstrates the significant potential of simulated human demonstrations for real-world dexterous manipulation tasks. More details can be found at https://cyber-demo.github.io

cs.RO↗

arXiv2024

Reasoning in Conversation: Solving Subjective Tasks through Dialogue Simulation for Large Language Models

Large Language Models (LLMs) have achieved remarkable performance in objective tasks such as open-domain question answering and mathematical reasoning, which can often be solved through recalling learned factual knowledge or chain-of-thought style reasoning. However, we find that the performance of LLMs in subjective tasks is still unsatisfactory, such as metaphor recognition, dark humor detection, etc. Compared to objective tasks, subjective tasks focus more on interpretation or emotional response rather than a universally accepted reasoning pathway. Based on the characteristics of the tasks and the strong dialogue-generation capabilities of LLMs, we propose RiC (Reasoning in Conversation), a method that focuses on solving subjective tasks through dialogue simulation. The motivation of RiC is to mine useful contextual information by simulating dialogues instead of supplying chain-of-thought style rationales, thereby offering potential useful knowledge behind dialogues for giving the final answers. We evaluate both API-based and open-source LLMs including GPT-4, ChatGPT, and OpenChat across twelve tasks. Experimental results show that RiC can yield significant improvement compared with various baselines.

cs.CL↗

arXiv2024

On the spectral extremal problem of planar graphs

The spectral extremal problem of planar graphs has aroused a lot of interest over the past three decades. In 1991, Boots and Royle [Geogr. Anal. 23(3) (1991) 276--282] (and Cao and Vince [Linear Algebra Appl. 187 (1993) 251--257] independently) conjectured that $K_2 + P_{n-2}$ is the unique graph attaining the maximum spectral radius among all planar graphs on $n$ vertices, where $K_2 + P_{n-2}$ is the graph obtained from $K_2\cup P_{n-2}$ by adding all possible edges between $K_2$ and $P_{n-2}$. In 2017, Tait and Tobin [J. Combin. Theory Ser. B 126 (2017) 137--161] confirmed this conjecture for all sufficiently large $n$. In this paper, we consider the spectral extremal problem for planar graphs without specified subgraphs. For a fixed graph $F$, let $\mathrm{SPEX}_{\mathcal{P}}(n,F)$ denote the set of graphs attaining the maximum spectral radius among all $F$-free planar graphs on $n$ vertices. We describe a rough sturcture for the connected extremal graphs in $\mathrm{SPEX}_{\mathcal{P}}(n,F)$ when $F$ is a planar graph not contained in $K_{2,n-2}$. As applications, we determine the extremal graphs in $\mathrm{SPEX}_{\mathcal{P}}(n,W_k)$, $\mathrm{SPEX}_{\mathcal{P}}(n,F_k)$ and $\mathrm{SPEX}_{\mathcal{P}}(n,(k+1)K_2)$ for all sufficiently large $n$, where $W_k$, $F_k$ and $(k+1)K_2$ are the wheel graph of order $k$, the friendship graph of order $2k+1$ and the disjoint union of $k+1$ copies of $K_2$, respectively.

math.CO↗

arXiv2024

Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages

While large language models (LLMs) have been pre-trained on multilingual corpora, their performance still lags behind in most languages compared to a few resource-rich languages. One common approach to mitigate this issue is to translate training data from resource-rich languages into other languages and then continue training. However, using the data obtained solely relying on translation while ignoring the original capabilities of LLMs across languages is not always effective, which we show will limit the performance of cross-lingual knowledge transfer. In this work, we propose SDRRL, a method based on Self-Distillation from Resource-Rich Languages that effectively improve multilingual performance by leveraging the internal capabilities of LLMs on resource-rich languages. We evaluate on different LLMs (LLaMA-2 and SeaLLM) and source languages across various comprehension and generation tasks, experimental results demonstrate that SDRRL can significantly enhance multilingual capabilities while minimizing the impact on original performance in resource-rich languages.

cs.CL↗

arXiv2024

Obligations and permissions on selfextensional logics

We further develop the abstract algebraic logic approach to input/output logic initiated in \cite{wollic22}, where the family of selfextensional logics was proposed as a general background environment for input/output logics. In this paper, we introduce and discuss the generalizations of several types of permission (negative, dual negative, static, dynamic), as well as their interactions with normative systems, to various families of selfextensional logics, thereby proposing a systematic approach to the definition of normative and permission systems on nonclassical propositional bases.

math.LO↗