arXiv subjects

Shuo Liu

Publications and source records attributed to Shuo Liu.

At least 145 records · Page 8Linked to original sources

arXiv2024

MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

Large Vision-Language Models (LVLMs) show significant strides in general-purpose multimodal applications such as visual dialogue and embodied navigation. However, existing multimodal evaluation benchmarks cover a limited number of multimodal tasks testing rudimentary capabilities, falling short in tracking LVLM development. In this study, we present MMT-Bench, a comprehensive benchmark designed to assess LVLMs across massive multimodal tasks requiring expert knowledge and deliberate visual recognition, localization, reasoning, and planning. MMT-Bench comprises $31,325$ meticulously curated multi-choice visual questions from various multimodal scenarios such as vehicle driving and embodied navigation, covering $32$ core meta-tasks and $162$ subtasks in multimodal understanding. Due to its extensive task coverage, MMT-Bench enables the evaluation of LVLMs using a task map, facilitating the discovery of in- and out-of-domain tasks. Evaluation results involving $30$ LVLMs such as the proprietary GPT-4V, GeminiProVision, and open-sourced InternVL-Chat, underscore the significant challenges posed by MMT-Bench. We anticipate that MMT-Bench will inspire the community to develop next-generation multimodal foundation models aimed at achieving general-purpose multimodal intelligence.

cs.CV↗

arXiv2024

Auxiliary-Variable Adaptive Control Barrier Functions for Safety Critical Systems

This paper studies safety guarantees for systems with time-varying control bounds. It has been shown that optimizing quadratic costs subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) using Control Barrier Functions (CBFs). One of the main challenges in this method is that the CBF-based QP could easily become infeasible under tight control bounds, especially when the control bounds are time-varying. The recently proposed adaptive CBFs have addressed such infeasibility issues, but require extensive and non-trivial hyperparameter tuning for the CBF-based QP and may introduce overshooting control near the boundaries of safe sets. To address these issues, we propose a new type of adaptive CBFs called Auxiliary-Variable Adaptive CBFs (AVCBFs). Specifically, we introduce an auxiliary variable that multiplies each CBF itself, and define dynamics for the auxiliary variable to adapt it in constructing the corresponding CBF constraint. In this way, we can improve the feasibility of the CBF-based QP while avoiding extensive parameter tuning with non-overshooting control since the formulation is identical to classical CBF methods. We demonstrate the advantages of using AVCBFs and compare them with existing techniques on an Adaptive Cruise Control (ACC) problem with time-varying control bounds.

math.OC↗

arXiv2024

SOS-1K: A Fine-grained Suicide Risk Classification Dataset for Chinese Social Media Analysis

In the social media, users frequently express personal emotions, a subset of which may indicate potential suicidal tendencies. The implicit and varied forms of expression in internet language complicate accurate and rapid identification of suicidal intent on social media, thus creating challenges for timely intervention efforts. The development of deep learning models for suicide risk detection is a promising solution, but there is a notable lack of relevant datasets, especially in the Chinese context. To address this gap, this study presents a Chinese social media dataset designed for fine-grained suicide risk classification, focusing on indicators such as expressions of suicide intent, methods of suicide, and urgency of timing. Seven pre-trained models were evaluated in two tasks: high and low suicide risk, and fine-grained suicide risk classification on a level of 0 to 10. In our experiments, deep learning models show good performance in distinguishing between high and low suicide risk, with the best model achieving an F1 score of 88.39%. However, the results for fine-grained suicide risk classification were still unsatisfactory, with an weighted F1 score of 50.89%. To address the issues of data imbalance and limited dataset size, we investigated both traditional and advanced, large language model based data augmentation techniques, demonstrating that data augmentation can enhance model performance by up to 4.65% points in F1-score. Notably, the Chinese MentalBERT model, which was pre-trained on psychological domain data, shows superior performance in both tasks. This study provides valuable insights for automatic identification of suicidal individuals, facilitating timely psychological intervention on social media platforms. The source code and data are publicly available.

cs.CL↗

arXiv2024

Inductive Cognitive Diagnosis for Fast Student Learning in Web-Based Online Intelligent Education Systems

Cognitive diagnosis aims to gauge students' mastery levels based on their response logs. Serving as a pivotal module in web-based online intelligent education systems (WOIESs), it plays an upstream and fundamental role in downstream tasks like learning item recommendation and computerized adaptive testing. WOIESs are open learning environment where numerous new students constantly register and complete exercises. In WOIESs, efficient cognitive diagnosis is crucial to fast feedback and accelerating student learning. However, the existing cognitive diagnosis methods always employ intrinsically transductive student-specific embeddings, which become slow and costly due to retraining when dealing with new students who are unseen during training. To this end, this paper proposes an inductive cognitive diagnosis model (ICDM) for fast new students' mastery levels inference in WOIESs. Specifically, in ICDM, we propose a novel student-centered graph (SCG). Rather than inferring mastery levels through updating student-specific embedding, we derive the inductive mastery levels as the aggregated outcomes of students' neighbors in SCG. Namely, SCG enables to shift the task from finding the most suitable student-specific embedding that fits the response logs to finding the most suitable representations for different node types in SCG, and the latter is more efficient since it no longer requires retraining. To obtain this representation, ICDM consists of a construction-aggregation-generation-transformation process to learn the final representation of students, exercises and concepts. Extensive experiments across real-world datasets show that, compared with the existing cognitive diagnosis methods that are always transductive, ICDM is much more faster while maintains the competitive inference performance for new students.

cs.AI↗

arXiv2024

Absence of barren plateaus in finite local-depth circuits with long-range entanglement

Ground state preparation is classically intractable for general Hamiltonians. On quantum devices, shallow parameterized circuits can be effectively trained to obtain short-range entangled states under the paradigm of variational quantum eigensolver, while deep circuits are generally untrainable due to the barren plateau phenomenon. In this Letter, we give a general lower bound on the variance of circuit gradients for arbitrary quantum circuits composed of local 2-designs. Based on our unified framework, we prove the absence of barren plateaus in training finite local-depth circuits (FLDC) for the ground states of local Hamiltonians. FLDCs are allowed to be deep in the conventional circuit depth to generate long-range entangled ground states, such as topologically ordered states, but their local depths are finite, i.e., there is only a finite number of gates acting on individual qubits. This characteristic sets FLDC apart from shallow circuits: FLDC in general cannot be classically simulated to estimate local observables efficiently by existing tensor network methods in two and higher dimensions. We validate our analytical results with extensive numerical simulations and demonstrate the effectiveness of variational training using the generalized toric code model.

quant-ph↗

arXiv2024

Byzantine Fault-Tolerant Min-Max Optimization

In this paper, we consider a min-max optimization problem under adversarial manipulation, where there are $n$ cost functions, up to $f$ of which may be replaced by arbitrary faulty functions by an adversary. The goal is to minimize the maximum cost over $x$ among the $n$ functions despite the faulty functions. The problem formulation could naturally extend to Byzantine fault-tolerant distributed min-max optimization. We present a simple algorithm for Byzantine min-max optimization, and provide bounds on the output of the algorithm. We also present an approximate algorithm for this problem. We then extend the problem to a distributed setting and present a distributed algorithm. To the best of our knowledge, we are the first to consider this problem.

cs.DC↗

arXiv2024

Byzantine fault-tolerant distributed set intersection with redundancy

In this report, we study the problem of Byzantine fault-tolerant distributed set intersection and the importance of redundancy in solving this problem. Specifically, consider a distributed system with $n$ agents, each of which has a local set. There are up to $f$ agents that are Byzantine faulty. The goal is to find the intersection of the sets of the non-faulty agents. We derive the Byzantine set intersection problem from the Byzantine optimization problem. We present the definition of $2f$-redundancy, and identify the necessary and sufficient condition if the Byzantine set intersection problem can be solved if a certain redundancy property is satisfied, and then present an equivalent condition. We further extend our results to arbitrary communication graphs in a decentralized setting. Finally, we present solvability results for the Byzantine optimization problem, inspired by our findings on Byzantine set intersection. The results we provide are for synchronous and asynchronous systems both.

cs.DC↗

arXiv2024

Sensor Misalignment-tolerant AUV Navigation with Passive DoA and Doppler Measurements

We present a sensor misalignment-tolerant AUV navigation method that leverages measurements from an acoustic array and dead reckoned information. Recent studies have demonstrated the potential use of passive acoustic Direction of Arrival (DoA) measurements for AUV navigation without requiring ranging measurements. However, the sensor misalignment between the acoustic array and the attitude sensor was not accounted for. Such misalignment may deteriorate the navigation accuracy. This paper proposes a novel approach that allows simultaneous AUV navigation, beacon localization, and sensor alignment. An Unscented Kalman Filter (UKF) that enables the necessary calculations to be completed at an affordable computational load is developed. A Nonlinear Least Squares (NLS)-based technique is employed to find an initial solution for beacon localization and sensor alignment as early as possible using a short-term window of measurements. Experimental results demonstrate the performance of the proposed method.

cs.RO↗

arXiv2024

Delving into Parameter-Efficient Fine-Tuning in Code Change Learning: An Empirical Study

Compared to Full-Model Fine-Tuning (FMFT), Parameter Efficient Fine-Tuning (PEFT) has demonstrated superior performance and lower computational overhead in several code understanding tasks, such as code summarization and code search. This advantage can be attributed to PEFT's ability to alleviate the catastrophic forgetting issue of Pre-trained Language Models (PLMs) by updating only a small number of parameters. As a result, PEFT effectively harnesses the pre-trained general-purpose knowledge for downstream tasks. However, existing studies primarily involve static code comprehension, aligning with the pre-training paradigm of recent PLMs and facilitating knowledge transfer, but they do not account for dynamic code changes. Thus, it remains unclear whether PEFT outperforms FMFT in task-specific adaptation for code-change-related tasks. To address this question, we examine two prevalent PEFT methods, namely Adapter Tuning (AT) and Low-Rank Adaptation (LoRA), and compare their performance with FMFT on five popular PLMs. Specifically, we evaluate their performance on two widely-studied code-change-related tasks: Just-In-Time Defect Prediction (JIT-DP) and Commit Message Generation (CMG). The results demonstrate that both AT and LoRA achieve state-of-the-art (SOTA) results in JIT-DP and exhibit comparable performances in CMG when compared to FMFT and other SOTA approaches. Furthermore, AT and LoRA exhibit superiority in cross-lingual and low-resource scenarios. We also conduct three probing tasks to explain the efficacy of PEFT techniques on JIT-DP and CMG tasks from both static and dynamic perspectives. The study indicates that PEFT, particularly through the use of AT and LoRA, offers promising advantages in code-change-related tasks, surpassing FMFT in certain aspects.

cs.SE↗

arXiv2023

Impact of Redundancy on Resilience in Distributed Optimization and Learning

This report considers the problem of resilient distributed optimization and stochastic learning in a server-based architecture. The system comprises a server and multiple agents, where each agent has its own local cost function. The agents collaborate with the server to find a minimum of the aggregate of the local cost functions. In the context of stochastic learning, the local cost of an agent is the loss function computed over the data at that agent. In this report, we consider this problem in a system wherein some of the agents may be Byzantine faulty and some of the agents may be slow (also called stragglers). In this setting, we investigate the conditions under which it is possible to obtain an "approximate" solution to the above problem. In particular, we introduce the notion of $(f, r; ε)$-resilience to characterize how well the true solution is approximated in the presence of up to $f$ Byzantine faulty agents, and up to $r$ slow agents (or stragglers) -- smaller $ε$ represents a better approximation. We also introduce a measure named $(f, r; ε)$-redundancy to characterize the redundancy in the cost functions of the agents. Greater redundancy allows for a better approximation when solving the problem of aggregate cost minimization. In this report, we constructively show (both theoretically and empirically) that $(f, r; \mathcal{O}(ε))$-resilience can indeed be achieved in practice, given that the local cost functions are sufficiently redundant.

cs.DC↗

arXiv2023

Vignat: Vulnerability identification by learning code semantics via graph attention networks

Vulnerability identification is crucial to protect software systems from attacks for cyber-security. However, huge projects have more than millions of lines of code, and the complex dependencies make it hard to carry out traditional static and dynamic methods. Furthermore, the semantic structure of various types of vulnerabilities differs greatly and may occur simultaneously, making general rule-based methods difficult to extend. In this paper, we propose \textit{Vignat}, a novel attention-based framework for identifying vulnerabilities by learning graph-level semantic representations of code. We represent codes with code property graphs (CPGs) in fine grain and use graph attention networks (GATs) for vulnerability detection. The results show that Vignat is able to achieve $57.38\%$ accuracy on reliable datasets derived from popular C libraries. Furthermore, the interpretability of our GATs provides valuable insights into vulnerability patterns.

cs.CR↗

arXiv2023

FishMOT: A Simple and Effective Method for Fish Tracking Based on IoU Matching

Fish tracking plays a vital role in understanding fish behavior and ecology. However, existing tracking methods face challenges in accuracy and robustness dues to morphological change of fish, occlusion and complex environment. This paper proposes FishMOT(Multiple Object Tracking for Fish), a novel fish tracking approach combining object detection and IoU matching, including basic module, interaction module and refind module. Wherein, a basic module performs target association based on IoU of detection boxes between successive frames to deal with morphological change of fish; an interaction module combines IoU of detection boxes and IoU of fish entity to handle occlusions; a refind module use spatio-temporal information uses spatio-temporal information to overcome the tracking failure resulting from the missed detection by the detector under complex environment. FishMOT reduces the computational complexity and memory consumption since it does not require complex feature extraction or identity assignment per fish, and does not need Kalman filter to predict the detection boxes of successive frame. Experimental results demonstrate FishMOT outperforms state-of-the-art multi-object trackers and specialized fish tracking tools in terms of MOTA, accuracy, computation time, memory consumption, etc.. Furthermore, the method exhibits excellent robustness and generalizability for varying environments and fish numbers. The simplified workflow and strong performance make FishMOT as a highly effective fish tracking approach. The source codes and pre-trained models are available at: https://github.com/gakkistar/FishMOT

cs.CV↗

arXiv2023

Enhancing Psychological Counseling with Large Language Model: A Multifaceted Decision-Support System for Non-Professionals

In the contemporary landscape of social media, an alarming number of users express negative emotions, some of which manifest as strong suicidal intentions. This situation underscores a profound need for trained psychological counselors who can enact effective mental interventions. However, the development of these professionals is often an imperative but time-consuming task. Consequently, the mobilization of non-professionals or volunteers in this capacity emerges as a pressing concern. Leveraging the capabilities of artificial intelligence, and in particular, the recent advances in large language models, offers a viable solution to this challenge. This paper introduces a novel model constructed on the foundation of large language models to fully assist non-professionals in providing psychological interventions on online user discourses. This framework makes it plausible to harness the power of non-professional counselors in a meaningful way. A comprehensive study was conducted involving ten professional psychological counselors of varying expertise, evaluating the system across five critical dimensions. The findings affirm that our system is capable of analyzing patients' issues with relative accuracy and proffering professional-level strategies recommendations, thereby enhancing support for non-professionals. This research serves as a compelling validation of the application of large language models in the field of psychology and lays the groundwork for a new paradigm of community-based mental health support.

cs.AI↗

arXiv2023

A Method to Speed Up Convergence of Iterative Learning Control for High Precision Repetitive Motions

Various spacecraft have sensors that repeatedly perform a prescribed scanning maneuver, and one may want high precision. Iterative Learning Control (ILC) records previous run tracking error, adjusts the next run command, aiming for zero tracking error in the real world, not our model of the world. In response to a command, feedback control systems perform a convolution sum over all commands given since time zero, with a weighting factor getting smaller going further back in time. ILC learns to eliminate this error through iterations in hardware. It aims to find that command that causes the real world system to actually follow the desired command. The topic of this paper considers the possibility of learning to make our model of the world produce less error. This can be done easily and quickly numerically, and the result then used as a starting point for the ILC iterations performed in hardware. The point is to reduce the number of hardware iteration, converging more quickly to closely tracking the desired trajectory in the world. How to decide how many iterations with our model to make before switching to hardware iterations is presented, with numerical simulations performed to illustrate the approach.

eess.SY↗

arXiv2023

Iterative Convex Optimization for Model Predictive Control with Discrete-Time High-Order Control Barrier Functions

Safety is one of the fundamental challenges in control theory. Recently, multi-step optimal control problems for discrete-time dynamical systems were formulated to enforce stability, while subject to input constraints as well as safety-critical requirements using discrete-time control barrier functions within a model predictive control (MPC) framework. Existing work usually focus on the feasibility or the safety for the optimization problem, and the majority of the existing work restrict the discussions to relative-degree one control barrier functions. Additionally, the real-time computation is challenging when a large horizon is considered in the MPC problem for relative-degree one or high-order control barrier functions. In this paper, we propose a framework that solves the safety-critical MPC problem in an iterative optimization, which is applicable for any relative-degree control barrier functions. In the proposed formulation, the nonlinear system dynamics as well as the safety constraints modeled as discrete-time high-order control barrier functions (DHOCBF) are linearized at each time step. Our formulation is generally valid for any control barrier function with an arbitrary relative-degree. The advantages of fast computational performance with safety guarantee are analyzed and validated with numerical results.

math.OC↗

arXiv2023

Modifying and optimizing the inverse of the frequency response circulant matrix as an iterative learning control compensator

Feedback control systems do not do what you ask. The concept of bandwidth is defined to tell what components of a command are reasonably well handled. Iterative Learning Control (ILC) seeks to converge to zero error following any given finite time desired trajectory as iterations progress. The approach can be used to achieve high precision tracking in spacecraft sensors performing repeated highly accurate sensor scanning. ILC asks for zero error for a finite time tracking maneuver, containing initial transients each iteration. The purpose of this paper is to create a method of designing ILC compensators based on steady state frequency response, and have the ILC converge to zero error in spite of transients and bandwidth. In this work the inverse of the circulant matrix of Markov parameters is used as a learning gain matrix. One can show that this matrix gives the steady state frequency response of the system at the finite number of frequencies observable in the finite data sequence of an iteration or run. Methods are used to adjust the steady state frequency response gains to address the transient part of the error signal. Numerical simulations compare the design approach to common time domain ILC design approaches, and one observes much faster convergence.

eess.SY↗

arXiv2023

On designing finite time iterative learning control based on steady state frequency response

Iterative Learning Control (ILC) is useful in spacecraft application for repeated high precision scanning maneuvers. Repetitive Control (RC) produces effective active vibration isolation based on frequency response. This paper considers ILC designed from frequency response, comparing two methods recently developed. One adapts for ILC the FIR filter design in RC that mimics the systems steady state frequency response inverse, creating a filter designed for all frequencies from zero to Nyquist. Adjustment of gains must near the beginning of the matrix need to be made because FIR gains are truncated there. The other approach uses a circulant matrix obtained from the Toeplitz matrix of Markov parameters. It is shown to give steady state frequency response for the discrete frequencies that can be seen in the number of time steps in the ILC tracking problem. The main aim is to compare their performance and ease of use. A second aim is to learn from successful ILC designs to enhances stability robustness of the two methods. Finally, the use of an ILC frequency cutoff is studied as an alternative method to eliminate instability of the control action in the converged zero tracking error solution of many ILC problems.

eess.SY↗

arXiv2023

LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models

Large Vision-Language Models (LVLMs) have recently played a dominant role in multimodal vision-language learning. Despite the great success, it lacks a holistic evaluation of their efficacy. This paper presents a comprehensive evaluation of publicly available large multimodal models by building a LVLM evaluation Hub (LVLM-eHub). Our LVLM-eHub consists of $8$ representative LVLMs such as InstructBLIP and MiniGPT-4, which are thoroughly evaluated by a quantitative capability evaluation and an online arena platform. The former evaluates $6$ categories of multimodal capabilities of LVLMs such as visual question answering and embodied artificial intelligence on $47$ standard text-related visual benchmarks, while the latter provides the user-level evaluation of LVLMs in an open-world question-answering scenario. The study reveals several innovative findings. First, instruction-tuned LVLM with massive in-domain data such as InstructBLIP heavily overfits many existing tasks, generalizing poorly in the open-world scenario. Second, instruction-tuned LVLM with moderate instruction-following data may result in object hallucination issues (i.e., generate objects that are inconsistent with target images in the descriptions). It either makes the current evaluation metric such as CIDEr for image captioning ineffective or generates wrong answers. Third, employing a multi-turn reasoning evaluation framework can mitigate the issue of object hallucination, shedding light on developing an effective pipeline for LVLM evaluation. The findings provide a foundational framework for the conception and assessment of innovative strategies aimed at enhancing zero-shot multimodal techniques. Our LVLM-eHub will be available at https://github.com/OpenGVLab/Multi-Modality-Arena

cs.CV↗