arXiv subjects

Xiangrui Kong

Publications and source records attributed to Xiangrui Kong.

14 recordsLinked to original sources

arXiv2026

An Inexact Variable Metric Proximal Gradient-subgradient Algorithm for a Class of Fractional Optimization Problems

In this paper, we study a class of fractional optimization problems, in which the numerator of the objective is the sum of a convex function and a differentiable function with a Lipschitz continuous gradient, while the denominator is a nonsmooth convex function. This model captures ratio-type formulations arising in scale-invariant sparse learning and related applications. To address this class of problems, we propose an inexact variable metric proximal gradient-subgradient algorithm (iVPGSA), which, to the best of our knowledge, is the first inexact proximal algorithm specifically designed for such type of fractional problems. By incorporating a variable metric proximal term and allowing for approximate subproblem solutions under a flexible error criterion, the proposed algorithm is highly adaptable to a broader range of problems while achieving favorable computational efficiency. Under suitable assumptions, we establish that any accumulation point of the generated sequence is a critical point of the target problem. Moreover, we develop a new Kurdyka-Łojasiewicz (KL)-based analysis framework, relying only on the classical KL property and its associated exponent, to prove the global convergence of the entire sequence and characterize its convergence rate, \textit{without} requiring a strict sufficient descent property. Our results clarify how the classical KL exponent and inexactness jointly influence the convergence rate. Finally, numerical experiments on the $\ell_1/\ell_2$ Lasso problem and the constrained $\ell_1/\ell_2$ sparse optimization problem demonstrate the computational advantages of the iVPGSA over existing representative algorithms.

math.OC↗

arXiv2025

Enhancing Reliability in LLM-Integrated Robotic Systems: A Unified Approach to Security and Safety

Integrating large language models (LLMs) into robotic systems has revolutionised embodied artificial intelligence, enabling advanced decision-making and adaptability. However, ensuring reliability, encompassing both security against adversarial attacks and safety in complex environments, remains a critical challenge. To address this, we propose a unified framework that mitigates prompt injection attacks while enforcing operational safety through robust validation mechanisms. Our approach combines prompt assembling, state management, and safety validation, evaluated using both performance and security metrics. Experiments show a 30.8% improvement under injection attacks and up to a 325% improvement in complex environment settings under adversarial conditions compared to baseline scenarios. This work bridges the gap between safety and security in LLM-based robotic systems, offering actionable insights for deploying reliable LLM-integrated mobile robots in real-world settings. The framework is open-sourced with simulation and physical deployment demos at https://llmeyesim.vercel.app/

cs.RO↗

arXiv2025

Cooperative Hybrid Multi-Agent Pathfinding Based on Shared Exploration Maps

Multi-Agent Pathfinding is used in areas including multi-robot formations, warehouse logistics, and intelligent vehicles. However, many environments are incomplete or frequently change, making it difficult for standard centralized planning or pure reinforcement learning to maintain both global solution quality and local flexibility. This paper introduces a hybrid framework that integrates D* Lite global search with multi-agent reinforcement learning, using a switching mechanism and a freeze-prevention strategy to handle dynamic conditions and crowded settings. We evaluate the framework in the discrete POGEMA environment and compare it with baseline methods. Experimental outcomes indicate that the proposed framework substantially improves success rate, collision rate, and path efficiency. The model is further tested on the EyeSim platform, where it maintains feasible Pathfinding under frequent changes and large-scale robot deployments.

cs.RO↗

arXiv2025

Skip Mamba Diffusion for Monocular 3D Semantic Scene Completion

3D semantic scene completion is critical for multiple downstream tasks in autonomous systems. It estimates missing geometric and semantic information in the acquired scene data. Due to the challenging real-world conditions, this task usually demands complex models that process multi-modal data to achieve acceptable performance. We propose a unique neural model, leveraging advances from the state space and diffusion generative modeling to achieve remarkable 3D semantic scene completion performance with monocular image input. Our technique processes the data in the conditioned latent space of a variational autoencoder where diffusion modeling is carried out with an innovative state space technique. A key component of our neural network is the proposed Skimba (Skip Mamba) denoiser, which is adept at efficiently processing long-sequence data. The Skimba diffusion model is integral to our 3D scene completion network, incorporating a triple Mamba structure, dimensional decomposition residuals and varying dilations along three directions. We also adopt a variant of this network for the subsequent semantic segmentation stage of our method. Extensive evaluation on the standard SemanticKITTI and SSCBench-KITTI360 datasets show that our approach not only outperforms other monocular techniques by a large margin, it also achieves competitive performance against stereo methods. The code is available at https://github.com/xrkong/skimba

cs.CV↗

arXiv2024

A Study on Prompt Injection Attack Against LLM-Integrated Mobile Robotic Systems

The integration of Large Language Models (LLMs) like GPT-4o into robotic systems represents a significant advancement in embodied artificial intelligence. These models can process multi-modal prompts, enabling them to generate more context-aware responses. However, this integration is not without challenges. One of the primary concerns is the potential security risks associated with using LLMs in robotic navigation tasks. These tasks require precise and reliable responses to ensure safe and effective operation. Multi-modal prompts, while enhancing the robot's understanding, also introduce complexities that can be exploited maliciously. For instance, adversarial inputs designed to mislead the model can lead to incorrect or dangerous navigational decisions. This study investigates the impact of prompt injections on mobile robot performance in LLM-integrated systems and explores secure prompt strategies to mitigate these risks. Our findings demonstrate a substantial overall improvement of approximately 30.8% in both attack detection and system performance with the implementation of robust defence mechanisms, highlighting their critical role in enhancing security and reliability in mission-oriented tasks.

cs.RO↗

arXiv2024

SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systems

Embodied AI systems, including AI-powered robots that autonomously interact with the physical world, stand to be significantly advanced by Large Language Models (LLMs), which enable robots to better understand complex language commands and perform advanced tasks with enhanced comprehension and adaptability, highlighting their potential to improve embodied AI capabilities. However, this advancement also introduces safety challenges, particularly in robotic navigation tasks. Improper safety management can lead to failures in complex environments and make the system vulnerable to malicious command injections, resulting in unsafe behaviours such as detours or collisions. To address these issues, we propose \textit{SafeEmbodAI}, a safety framework for integrating mobile robots into embodied AI systems. \textit{SafeEmbodAI} incorporates secure prompting, state management, and safety validation mechanisms to secure and assist LLMs in reasoning through multi-modal data and validating responses. We designed a metric to evaluate mission-oriented exploration, and evaluations in simulated environments demonstrate that our framework effectively mitigates threats from malicious commands and improves performance in various environment settings, ensuring the safety of embodied AI systems. Notably, In complex environments with mixed obstacles, our method demonstrates a significant performance increase of 267\% compared to the baseline in attack scenarios, highlighting its robustness in challenging conditions.

cs.RO↗

arXiv2024

Embodied AI in Mobile Robots: Coverage Path Planning with Large Language Models

In recent years, Large Language Models (LLMs) have demonstrated remarkable capabilities in understanding and solving mathematical problems, leading to advancements in various fields. We propose an LLM-embodied path planning framework for mobile agents, focusing on solving high-level coverage path planning issues and low-level control. Our proposed multi-layer architecture uses prompted LLMs in the path planning phase and integrates them with the mobile agents' low-level actuators. To evaluate the performance of various LLMs, we propose a coverage-weighted path planning metric to assess the performance of the embodied models. Our experiments show that the proposed framework improves LLMs' spatial inference abilities. We demonstrate that the proposed multi-layer framework significantly enhances the efficiency and accuracy of these tasks by leveraging the natural language understanding and generative capabilities of LLMs. Our experiments show that this framework can improve LLMs' 2D plane reasoning abilities and complete coverage path planning tasks. We also tested three LLM kernels: gpt-4o, gemini-1.5-flash, and claude-3.5-sonnet. The experimental results show that claude-3.5 can complete the coverage planning task in different scenarios, and its indicators are better than those of the other models.

cs.RO↗

arXiv2024

A Superalignment Framework in Autonomous Driving with Large Language Models

Over the last year, significant advancements have been made in the realms of large language models (LLMs) and multi-modal large language models (MLLMs), particularly in their application to autonomous driving. These models have showcased remarkable abilities in processing and interacting with complex information. In autonomous driving, LLMs and MLLMs are extensively used, requiring access to sensitive vehicle data such as precise locations, images, and road conditions. These data are transmitted to an LLM-based inference cloud for advanced analysis. However, concerns arise regarding data security, as the protection against data and privacy breaches primarily depends on the LLM's inherent security measures, without additional scrutiny or evaluation of the LLM's inference outputs. Despite its importance, the security aspect of LLMs in autonomous driving remains underexplored. Addressing this gap, our research introduces a novel security framework for autonomous vehicles, utilizing a multi-agent LLM approach. This framework is designed to safeguard sensitive information associated with autonomous vehicles from potential leaks, while also ensuring that LLM outputs adhere to driving regulations and align with human values. It includes mechanisms to filter out irrelevant queries and verify the safety and reliability of LLM outputs. Utilizing this framework, we evaluated the security, privacy, and cost aspects of eleven large language model-driven autonomous driving cues. Additionally, we performed QA tests on these driving prompts, which successfully demonstrated the framework's efficacy.

cs.RO↗

arXiv2024

Docs2KG: Unified Knowledge Graph Construction from Heterogeneous Documents Assisted by Large Language Models

Even for a conservative estimate, 80% of enterprise data reside in unstructured files, stored in data lakes that accommodate heterogeneous formats. Classical search engines can no longer meet information seeking needs, especially when the task is to browse and explore for insight formulation. In other words, there are no obvious search keywords to use. Knowledge graphs, due to their natural visual appeals that reduce the human cognitive load, become the winning candidate for heterogeneous data integration and knowledge representation. In this paper, we introduce Docs2KG, a novel framework designed to extract multimodal information from diverse and heterogeneous unstructured documents, including emails, web pages, PDF files, and Excel files. Dynamically generates a unified knowledge graph that represents the extracted key information, Docs2KG enables efficient querying and exploration of document data lakes. Unlike existing approaches that focus on domain-specific data sources or pre-designed schemas, Docs2KG offers a flexible and extensible solution that can adapt to various document structures and content types. The proposed framework unifies data processing supporting a multitude of downstream tasks with improved domain interpretability. Docs2KG is publicly accessible at https://docs2kg.ai4wa.com, and a demonstration video is available at https://docs2kg.ai4wa.com/Video.

cs.CL↗

arXiv2024

Assessing Electricity Service Unfairness with Transfer Counterfactual Learning

Energy justice is a growing area of interest in interdisciplinary energy research. However, identifying systematic biases in the energy sector remains challenging due to confounding variables, intricate heterogeneity in counterfactual effects, and limited data availability. First, this paper demonstrates how one can evaluate counterfactual unfairness in a power system by analyzing the average causal effect of a specific protected attribute. Subsequently, we use subgroup analysis to handle model heterogeneity and introduce a novel method for estimating counterfactual unfairness based on transfer learning, which helps to alleviate the data scarcity in each subgroup. In our numerical analysis, we apply our method to a unique large-scale customer-level power outage data set and investigate the counterfactual effect of demographic factors, such as income and age of the population, on power outage durations. Our results indicate that low-income and elderly-populated areas consistently experience longer power outages under both daily and post-disaster operations, and such discrimination is exacerbated under severe conditions. These findings suggest a widespread, systematic issue of injustice in the power service systems and emphasize the necessity for focused interventions in disadvantaged communities.

cs.LG↗

arXiv2023

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision

We have witnessed significant progress in deep learning-based 3D vision, ranging from neural radiance field (NeRF) based 3D representation learning to applications in novel view synthesis (NVS). However, existing scene-level datasets for deep learning-based 3D vision, limited to either synthetic environments or a narrow selection of real-world scenes, are quite insufficient. This insufficiency not only hinders a comprehensive benchmark of existing methods but also caps what could be explored in deep learning-based 3D analysis. To address this critical gap, we present DL3DV-10K, a large-scale scene dataset, featuring 51.2 million frames from 10,510 videos captured from 65 types of point-of-interest (POI) locations, covering both bounded and unbounded scenes, with different levels of reflection, transparency, and lighting. We conducted a comprehensive benchmark of recent NVS methods on DL3DV-10K, which revealed valuable insights for future research in NVS. In addition, we have obtained encouraging results in a pilot study to learn generalizable NeRF from DL3DV-10K, which manifests the necessity of a large-scale scene-level dataset to forge a path toward a foundation model for learning 3D representation. Our DL3DV-10K dataset, benchmark results, and models will be publicly accessible at https://dl3dv-10k.github.io/DL3DV-10K/.

cs.CV↗

arXiv2013

Surface Transformations and Water Uptake on Liquid and Solid Butanol near the Melting Temperature

Water interactions with organic surfaces are of central importance in biological systems and many Earth system processes. Here we describe experimental studies of water collisions and uptake kinetics on liquid and solid butanol from 160 to 200 K. Hyperthermal D2O molecules (0.32 eV) undergo efficient trapping on both solid and liquid butanol, and only a minor fraction scatters inelastically after an 80% loss of kinetic energy to surface modes. Trapped molecules either desorb within a few ms, or are taken up by the butanol phase during longer times. The water uptake and surface residence time increase with temperature above 180 K indicating melting of the butanol surface 4.5 K below the bulk melting temperature. Water uptake changes gradually across the melting point and trapped molecules are rapidly lost by diffusion into the liquid above 190 K. This indicates that liquid butanol maintains a surface phase with limited water permeability up to 5.5 K above the melting point. These surface observations are indicative of an incremental change from solid to liquid butanol over a range of 10 K straddling the bulk melting temperature, in contrast to the behavior of bulk butanol and previously studied materials.

physics.chem-ph↗

arXiv2013

Ice Formation via Deposition Mode Nucleation on Bare and Alcohol-covered Graphite Surfaces

Deposition of water on aerosol particles contributes to ice cloud formation in the atmosphere with implications for the water cycle and climate on Earth. The heterogeneous ice nucleation process is influenced by physico-chemical properties of the substrate, but the mechanisms remain incompletely understood. Here, we report on ice formation on bare and alcohol-covered graphite at temperatures from 175 to 213 K, probed by elastic helium and light scattering. Water has a low wettability on bare and butanol-covered graphite resulting in the growth of rough ice surfaces. In contrast, pre-adsorbed methanol provides hydrophilic surface sites and results in the formation of smooth crystalline ice; an effect that is pronounced also for sub-monolayer methanol coverages. The alcohols primarily reside at the ice surface and at the ice-graphite interface with a minor fraction being incorporated into the growing ice structures. Methanol has no observable effect on gas/solid water vapor exchange whereas butanol acts as a transport barrier for water resulting in a reduction in ice evaporation rate at 185 K. Implications for the description of deposition mode freezing are discussed.

physics.chem-ph↗

arXiv2011

Collision Dynamics and Solvation of Water Molecules in a Liquid Methanol Film

Environmental molecular beam experiments are used to examine water interactions with liquid methanol films at temperatures from 170 K to 190 K. We find that water molecules with 0.32 eV incident kinetic energy are efficiently trapped by the liquid methanol. The scattering process is characterized by an efficient loss of energy to surface modes with a minor component of the incident beam that is inelastically scattered. Thermal desorption of water molecules has a well characterized Arrhenius form with an activation energy of 0.47{\pm}0.11 eV and pre-exponential factor of 4.6 {\times} 10^(15{\pm}3) s^(-1). We also observe a temperature dependent incorporation of incident water into the methanol layer. The implication for fundamental studies and environmental applications is that even an alcohol as simple as methanol can exhibit complex and temperature dependent surfactant behavior.

physics.chem-ph↗