arXiv subjects

Xuan Xie

Publications and source records attributed to Xuan Xie.

32 records · Page 2Linked to original sources

arXiv2024

LeCov: Multi-level Testing Criteria for Large Language Models

Large Language Models (LLMs) are widely used in many different domains, but because of their limited interpretability, there are questions about how trustworthy they are in various perspectives, e.g., truthfulness and toxicity. Recent research has started developing testing methods for LLMs, aiming to uncover untrustworthy issues, i.e., defects, before deployment. However, systematic and formalized testing criteria are lacking, which hinders a comprehensive assessment of the extent and adequacy of testing exploration. To mitigate this threat, we propose a set of multi-level testing criteria, LeCov, for LLMs. The criteria consider three crucial LLM internal components, i.e., the attention mechanism, feed-forward neurons, and uncertainty, and contain nine types of testing criteria in total. We apply the criteria in two scenarios: test prioritization and coverage-guided testing. The experiment evaluation, on three models and four datasets, demonstrates the usefulness and effectiveness of LeCov.

cs.SE↗

arXiv2024

MORTAR: A Model-based Runtime Action Repair Framework for AI-enabled Cyber-Physical Systems

Cyber-Physical Systems (CPSs) are increasingly prevalent across various industrial and daily-life domains, with applications ranging from robotic operations to autonomous driving. With recent advancements in artificial intelligence (AI), learning-based components, especially AI controllers, have become essential in enhancing the functionality and efficiency of CPSs. However, the lack of interpretability in these AI controllers presents challenges to the safety and quality assurance of AI-enabled CPSs (AI-CPSs). Existing methods for improving the safety of AI controllers often involve neural network repair, which requires retraining with additional adversarial examples or access to detailed internal information of the neural network. Hence, these approaches have limited applicability for black-box policies, where only the inputs and outputs are accessible during operation. To overcome this, we propose MORTAR, a runtime action repair framework designed for AI-CPSs in this work. MORTAR begins by constructing a prediction model that forecasts the quality of actions proposed by the AI controller. If an unsafe action is detected, MORTAR then initiates a repair process to correct it. The generation of repaired actions is achieved through an optimization process guided by the safety estimates from the prediction model. We evaluate the effectiveness of MORTAR across various CPS tasks and AI controllers. The results demonstrate that MORTAR can efficiently improve task completion rates of AI controllers under specified safety specifications. Meanwhile, it also maintains minimal computational overhead, ensuring real-time operation of the AI-CPSs.

cs.SE↗

arXiv2024

LUNA: A Model-Based Universal Analysis Framework for Large Language Models

Over the past decade, Artificial Intelligence (AI) has had great success recently and is being used in a wide range of academic and industrial fields. More recently, LLMs have made rapid advancements that have propelled AI to a new level, enabling even more diverse applications and industrial domains with intelligence, particularly in areas like software engineering and natural language processing. Nevertheless, a number of emerging trustworthiness concerns and issues exhibited in LLMs have already recently received much attention, without properly solving which the widespread adoption of LLMs could be greatly hindered in practice. The distinctive characteristics of LLMs, such as the self-attention mechanism, extremely large model scale, and autoregressive generation schema, differ from classic AI software based on CNNs and RNNs and present new challenges for quality analysis. Up to the present, it still lacks universal and systematic analysis techniques for LLMs despite the urgent industrial demand. Towards bridging this gap, we initiate an early exploratory study and propose a universal analysis framework for LLMs, LUNA, designed to be general and extensible, to enable versatile analysis of LLMs from multiple quality perspectives in a human-interpretable manner. In particular, we first leverage the data from desired trustworthiness perspectives to construct an abstract model as an auxiliary analysis asset, which is empowered by various abstract model construction methods. To assess the quality of the abstract model, we collect and define a number of evaluation metrics, aiming at both abstract model level and the semantics level. Then, the semantics, which is the degree of satisfaction of the LLM w.r.t. the trustworthiness perspective, is bound to and enriches the abstract model with semantics, which enables more detailed analysis applications for diverse purposes.

cs.LG↗

arXiv2024

Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward

While Large Language Models (LLMs) have seen widespread applications across numerous fields, their limited interpretability poses concerns regarding their safe operations from multiple aspects, e.g., truthfulness, robustness, and fairness. Recent research has started developing quality assurance methods for LLMs, introducing techniques such as offline detector-based or uncertainty estimation methods. However, these approaches predominantly concentrate on post-generation analysis, leaving the online safety analysis for LLMs during the generation phase an unexplored area. To bridge this gap, we conduct in this work a comprehensive evaluation of the effectiveness of existing online safety analysis methods on LLMs. We begin with a pilot study that validates the feasibility of detecting unsafe outputs in the early generation process. Following this, we establish the first publicly available benchmark of online safety analysis for LLMs, including a broad spectrum of methods, models, tasks, datasets, and evaluation metrics. Utilizing this benchmark, we extensively analyze the performance of state-of-the-art online safety analysis methods on both open-source and closed-source LLMs. This analysis reveals the strengths and weaknesses of individual methods and offers valuable insights into selecting the most appropriate method based on specific application scenarios and task requirements. Furthermore, we also explore the potential of using hybridization methods, i.e., combining multiple methods to derive a collective safety conclusion, to enhance the efficacy of online safety analysis for LLMs. Our findings indicate a promising direction for the development of innovative and trustworthy quality assurance methodologies for LLMs, facilitating their reliable deployments across diverse domains.

cs.SE↗

arXiv2023

Towards Building AI-CPS with NVIDIA Isaac Sim: An Industrial Benchmark and Case Study for Robotics Manipulation

As a representative cyber-physical system (CPS), robotic manipulator has been widely adopted in various academic research and industrial processes, indicating its potential to act as a universal interface between the cyber and the physical worlds. Recent studies in robotics manipulation have started employing artificial intelligence (AI) approaches as controllers to achieve better adaptability and performance. However, the inherent challenge of explaining AI components introduces uncertainty and unreliability to these AI-enabled robotics systems, necessitating a reliable development platform for system design and performance assessment. As a foundational step towards building reliable AI-enabled robotics systems, we propose a public industrial benchmark for robotics manipulation in this paper. It leverages NVIDIA Omniverse Isaac Sim as the simulation platform, encompassing eight representative manipulation tasks and multiple AI software controllers. An extensive evaluation is conducted to analyze the performance of AI controllers in solving robotics manipulation tasks, enabling a thorough understanding of their effectiveness. To further demonstrate the applicability of our benchmark, we develop a falsification framework that is compatible with physical simulators and OpenAI Gym environments. This framework bridges the gap between traditional testing methods and modern physics engine-based simulations. The effectiveness of different optimization methods in falsifying AI-enabled robotics manipulation with physical simulators is examined via a falsification test. Our work not only establishes a foundation for the design and development of AI-enabled robotics systems but also provides practical experience and guidance to practitioners in this field, promoting further research in this critical academic and industrial domain.

cs.SE↗

arXiv2023

Mosaic: Model-based Safety Analysis Framework for AI-enabled Cyber-Physical Systems

Cyber-physical systems (CPSs) are now widely deployed in many industrial domains, e.g., manufacturing systems and autonomous vehicles. To further enhance the capability and applicability of CPSs, there comes a recent trend from both academia and industry to utilize learning-based AI controllers for the system control process, resulting in an emerging class of AI-enabled cyber-physical systems (AI-CPSs). Although such AI-CPSs could achieve obvious performance enhancement from the lens of some key industrial requirement indicators, due to the random exploration nature and lack of systematic explanations for their behavior, such AI-based techniques also bring uncertainties and safety risks to the controlled system, posing an urgent need for effective safety analysis techniques for AI-CPSs. Hence in this work, we propose Mosaic, a model-based safety analysis framework for AI-CPSs. Mosaic first constructs a Markov decision process (MDP) model as an abstract model of the AI-CPS, which tries to characterize the behaviors of the original AI-CPS. Then, based on the derived abstract model, safety analysis is designed in two aspects: online safety monitoring and offline model-guided falsification. The usefulness of Mosaic is evaluated on diverse and representative industry-level AI-CPSs, the results of which demonstrate that Mosaic is effective in providing safety monitoring to AI-CPSs and enables to outperform the state-of-the-art falsification techniques, providing the basis for advanced safety analysis of AI-CPSs.

cs.SE↗

arXiv2022

Neuro-Symbolic Verification of Deep Neural Networks

Formal verification has emerged as a powerful approach to ensure the safety and reliability of deep neural networks. However, current verification tools are limited to only a handful of properties that can be expressed as first-order constraints over the inputs and output of a network. While adversarial robustness and fairness fall under this category, many real-world properties (e.g., "an autonomous vehicle has to stop in front of a stop sign") remain outside the scope of existing verification technology. To mitigate this severe practical restriction, we introduce a novel framework for verifying neural networks, named neuro-symbolic verification. The key idea is to use neural networks as part of the otherwise logical specification, enabling the verification of a wide variety of complex, real-world properties, including the one above. Moreover, we demonstrate how neuro-symbolic verification can be implemented on top of existing verification infrastructure for neural networks, making our framework easily accessible to researchers and practitioners alike.

cs.AI↗

arXiv2022

DeepGalaxy: Testing Neural Network Verifiers via Two-Dimensional Input Space Exploration

Deep neural networks (DNNs) are widely developed and applied in many areas, and the quality assurance of DNNs is critical. Neural network verification (NNV) aims to provide formal guarantees to DNN models. Similar to traditional software, neural network verifiers could also contain bugs, which would have a critical and serious impact, especially in safety-critical areas. However, little work exists on validating neural network verifiers. In this work, we propose DeepGalaxy, an automated approach based on differential testing to tackle this problem. Specifically, we (1) propose a line of mutation rules, including model level mutation and specification level mutation, to effectively explore the two-dimensional input space of neural network verifiers; and (2) propose heuristic strategies to select test cases. We leveraged our implementation of DeepGalaxy to test three state-of-the-art neural network verifies, Marabou, Eran, and Neurify. The experimental results support the efficiency and effectiveness of DeepGalaxy. Moreover, five unique unknown bugs were discovered

cs.SE↗

arXiv2020

Sampling Policy Design for Tracking Time-Varying Graph Signals with Adaptive Budget Allocation

There have been many works that focus on the sampling set design for a static graph signal, but few for time-varying graph signals (GS). In this paper, we concentrate on how to select vertices to sample and how to allocate the sampling budget for a time-varying GS to achieve a minimal tracking error for the long-term. In the Kalman Filter (KF) framework, the problem of sampling policy design and budget allocation is formulated as an infinite horizon sequential decision process, in which the optimal sampling policy is obtained by Dynamic Programming (DP). Since the optimal policy is intractable, an approximate algorithm is proposed by truncating the infinite horizon. By introducing a new tool for analyzing the convexity or concavity of composite functions, we prove that the truncated problem is convex. Finally, we demonstrate the performance of the proposed approach through numerical experiments.

eess.SP↗

arXiv2019

On Critical Sampling of Time-Vertex Graph Signals

Joint time-vertex graph signals are pervasive in real-world. This paper focuses on the fundamental problem of sampling and reconstruction of joint time-vertex graph signals. We prove the existence and the necessary condition of a critical sampling set using minimum number of samples in time and graph domain respectively. The theory proposed in this paper suggests to assign heterogeneous sampling pattern for each node in a network under the constraint of minimum resources. An efficient algorithm is also provided to construct a critical sampling set.

eess.SP↗

arXiv2019

Bayesian Design of Sampling Set for Bandlimited Graph Signals

The design of sampling set (DoS) for bandlimited graph signals (GS) has been extensively studied in recent years, but few of them exploit the benefits of the stochastic prior of GS. In this work, we introduce the optimization framework for Bayesian DoS of bandlimited GS. We also illustrate how the choice of different sampling sets affects the estimation error and how the prior knowledge influences the result of DoS compared with the non-Bayesian DoS by the aid of analyzing Gershgorin discs of error metric matrix. Finally, based on our analysis, we propose a heuristic algorithm for DoS to avoid solving the optimization problem directly.

eess.SP↗

arXiv2019

Active Sampling for Approximately Bandlimited Graph Signals

This paper investigates the active sampling for estimation of approximately bandlimited graph signals. With the assistance of a graph filter, an approximately bandlimited graph signal can be formulated by a Gaussian random field over the graph. In contrast to offline sampling set design methods which usually rely on accurate prior knowledge about the model, unknown parameters in signal and noise distribution are allowed in the proposed active sampling algorithm. The active sampling process is divided into two alternating stages: unknown parameters are first estimated by Expectation Maximization (EM), with which the next node to sample is selected based on historical observations according to predictive uncertainty. Validated by simulations compared with related approaches, the proposed algorithm can reduce the sample size to reach a certain estimation accuracy.

eess.SP↗

arXiv2018

PPLS: A Privacy-Preserving Location-Sharing Scheme in Vehicular Social Networks

Recent advances in Socially Aware Networks (SANs) have allowed its use in many domains, out of which social Internet of vehicles (SIOV) is of prime importance. SANs can provide a promising routing and forwarding paradigm for SIOV by using interest-based communication. Though able to improve the forwarding performance, existing interest-based schemes fail to consider the important issue of protecting users' interest information. In this paper, we propose a PRivacy-preserving Interest-based Forwarding scheme (PRIF) for SIOV, which not only protects the interest information, but also improves the forwarding performance. We propose a privacy-preserving authentication protocol to recognize communities among mobile nodes. During data routing and forwarding, a node can know others' interests only if they are affiliated with the same community. Moreover, to improve forwarding performance, a new metric {\em community energy} is introduced to indicate vehicular social proximity. Community energy is generated when two nodes encounter one another and information is shared among them. PRIF considers this energy metric to select forwarders towards the destination node or the destination community. Security analysis indicates PRIF can protect nodes' interest information. In addition, extensive simulations have been conducted to demonstrate that PRIF outperforms the existing algorithms including the BEEINFO, Epidemic, and PRoPHET.

cs.CR↗

arXiv2017

Design of Sampling Set for Bandlimited Graph Signal Estimation

It is of particular interest to reconstruct or estimate bandlimited graph signals, which are smoothly varying signals defined over graphs, from partial noisy measurements. However, choosing an optimal subset of nodes to sample is NP-hard. We formularize the problem as the experimental design of a linear regression model if we allow multiple measurements on a single node. By relaxing it to a convex optimization problem, we get the proportion of sample for each node given the budget of total sample size. Then, we use a probabilistic quantization to get the number of each node to be sampled. Moreover, we analyze how the sample size influences whether our object function is well-defined by perturbation analysis. Finally, we demonstrate the performance of the proposed approach through various numerical experiments.

eess.SP↗