arXiv subjects

Yao Liu

Publications and source records attributed to Yao Liu.

At least 109 records · Page 6Linked to original sources

arXiv2024

A Novel Convolutional Neural Network Architecture with a Continuous Symmetry

This paper introduces a new Convolutional Neural Network (ConvNet) architecture inspired by a class of partial differential equations (PDEs) called quasi-linear hyperbolic systems. With comparable performance on the image classification task, it allows for the modification of the weights via a continuous group of symmetry. This is a significant shift from traditional models where the architecture and weights are essentially fixed. We wish to promote the (internal) symmetry as a new desirable property for a neural network, and to draw attention to the PDE perspective in analyzing and interpreting ConvNets in the broader Deep Learning community.

cs.CV↗

arXiv2024

Two-stream Multi-level Dynamic Point Transformer for Two-person Interaction Recognition

As a fundamental aspect of human life, two-person interactions contain meaningful information about people's activities, relationships, and social settings. Human action recognition serves as the foundation for many smart applications, with a strong focus on personal privacy. However, recognizing two-person interactions poses more challenges due to increased body occlusion and overlap compared to single-person actions. In this paper, we propose a point cloud-based network named Two-stream Multi-level Dynamic Point Transformer for two-person interaction recognition. Our model addresses the challenge of recognizing two-person interactions by incorporating local-region spatial information, appearance information, and motion information. To achieve this, we introduce a designed frame selection method named Interval Frame Sampling (IFS), which efficiently samples frames from videos, capturing more discriminative information in a relatively short processing time. Subsequently, a frame features learning module and a two-stream multi-level feature aggregation module extract global and partial features from the sampled frames, effectively representing the local-region spatial information, appearance information, and motion information related to the interactions. Finally, we apply a transformer to perform self-attention on the learned features for the final classification. Extensive experiments are conducted on two large-scale datasets, the interaction subsets of NTU RGB+D 60 and NTU RGB+D 120. The results show that our network outperforms state-of-the-art approaches in most standard evaluation settings.

cs.CV↗

arXiv2024

Modeling Pedestrian Intrinsic Uncertainty for Multimodal Stochastic Trajectory Prediction via Energy Plan Denoising

Pedestrian trajectory prediction plays a pivotal role in the realms of autonomous driving and smart cities. Despite extensive prior research employing sequence and generative models, the unpredictable nature of pedestrians, influenced by their social interactions and individual preferences, presents challenges marked by uncertainty and multimodality. In response, we propose the Energy Plan Denoising (EPD) model for stochastic trajectory prediction. EPD initially provides a coarse estimation of the distribution of future trajectories, termed the Plan, utilizing the Langevin Energy Model. Subsequently, it refines this estimation through denoising via the Probabilistic Diffusion Model. By initiating denoising with the Plan, EPD effectively reduces the need for iterative steps, thereby enhancing efficiency. Furthermore, EPD differs from conventional approaches by modeling the distribution of trajectories instead of individual trajectories. This allows for the explicit modeling of pedestrian intrinsic uncertainties and eliminates the need for multiple denoising operations. A single denoising operation produces a distribution from which multiple samples can be drawn, significantly enhancing efficiency. Moreover, EPD's fine-tuning of the Plan contributes to improved model performance. We validate EPD on two publicly available datasets, where it achieves state-of-the-art results. Additionally, ablation experiments underscore the contributions of individual modules, affirming the efficacy of the proposed approach.

cs.CV↗

arXiv2024

Uncertainty-Aware Pedestrian Trajectory Prediction via Distributional Diffusion

Tremendous efforts have been put forth on predicting pedestrian trajectory with generative models to accommodate uncertainty and multi-modality in human behaviors. An individual's inherent uncertainty, e.g., change of destination, can be masked by complex patterns resulting from the movements of interacting pedestrians. However, latent variable-based generative models often entangle such uncertainty with complexity, leading to limited either latent expressivity or predictive diversity. In this work, we propose to separately model these two factors by implicitly deriving a flexible latent representation to capture intricate pedestrian movements, while integrating predictive uncertainty of individuals with explicit bivariate Gaussian mixture densities over their future locations. More specifically, we present a model-agnostic uncertainty-aware pedestrian trajectory prediction framework, parameterizing sufficient statistics for the mixture of Gaussians that jointly comprise the multi-modal trajectories. We further estimate these parameters of interest by approximating a denoising process that progressively recovers pedestrian movements from noise. Unlike previous studies, we translate the predictive stochasticity to explicit distributions, allowing it to readily generate plausible future trajectories indicating individuals' self-uncertainty. Moreover, our framework is compatible with different neural net architectures. We empirically show the performance gains over state-of-the-art even with lighter backbones, across most scenes on two public benchmarks.

cs.CV↗

arXiv2024

Attention-aware Social Graph Transformer Networks for Stochastic Trajectory Prediction

Trajectory prediction is fundamental to various intelligent technologies, such as autonomous driving and robotics. The motion prediction of pedestrians and vehicles helps emergency braking, reduces collisions, and improves traffic safety. Current trajectory prediction research faces problems of complex social interactions, high dynamics and multi-modality. Especially, it still has limitations in long-time prediction. We propose Attention-aware Social Graph Transformer Networks for multi-modal trajectory prediction. We combine Graph Convolutional Networks and Transformer Networks by generating stable resolution pseudo-images from Spatio-temporal graphs through a designed stacking and interception method. Furthermore, we design the attention-aware module to handle social interaction information in scenarios involving mixed pedestrian-vehicle traffic. Thus, we maintain the advantages of the Graph and Transformer, i.e., the ability to aggregate information over an arbitrary number of neighbors and the ability to perform complex time-dependent data processing. We conduct experiments on datasets involving pedestrian, vehicle, and mixed trajectories, respectively. Our results demonstrate that our model minimizes displacement errors across various metrics and significantly reduces the likelihood of collisions. It is worth noting that our model effectively reduces the final displacement error, illustrating the ability of our model to predict for a long time.

cs.CV↗

arXiv2024

Multi-agent Traffic Prediction via Denoised Endpoint Distribution

The exploration of high-speed movement by robots or road traffic agents is crucial for autonomous driving and navigation. Trajectory prediction at high speeds requires considering historical features and interactions with surrounding entities, a complexity not as pronounced in lower-speed environments. Prior methods have assessed the spatio-temporal dynamics of agents but often neglected intrinsic intent and uncertainty, thereby limiting their effectiveness. We present the Denoised Endpoint Distribution model for trajectory prediction, which distinctively models agents' spatio-temporal features alongside their intrinsic intentions and uncertainties. By employing Diffusion and Transformer models to focus on agent endpoints rather than entire trajectories, our approach significantly reduces model complexity and enhances performance through endpoint information. Our experiments on open datasets, coupled with comparison and ablation studies, demonstrate our model's efficacy and the importance of its components. This approach advances trajectory prediction in high-speed scenarios and lays groundwork for future developments.

cs.RO↗

arXiv2024

TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models

The full potential of large pretrained models remains largely untapped in control domains like robotics. This is mainly because of the scarcity of data and the computational challenges associated with training or fine-tuning these large models for such applications. Prior work mainly emphasizes either effective pretraining of large models for decision-making or single-task adaptation. But real-world problems will require data-efficient, continual adaptation for new control tasks. Recognizing these constraints, we introduce TAIL (Task-specific Adapters for Imitation Learning), a framework for efficient adaptation to new control tasks. Inspired by recent advancements in parameter-efficient fine-tuning in language domains, we explore efficient fine-tuning techniques -- e.g., Bottleneck Adapters, P-Tuning, and Low-Rank Adaptation (LoRA) -- in TAIL to adapt large pretrained models for new tasks with limited demonstration data. Our extensive experiments in large-scale language-conditioned manipulation tasks comparing prevalent parameter-efficient fine-tuning techniques and adaptation baselines suggest that TAIL with LoRA can achieve the best post-adaptation performance with only 1\% of the trainable parameters of full fine-tuning, while avoiding catastrophic forgetting and preserving adaptation plasticity in continual learning settings.

cs.LG↗

arXiv2024

Small and Large Dust Cavities in Disks around mid-M Stars in Taurus

High-angular resolution imaging by ALMA has revealed the near-universality and diversity of substructures in protoplanetary disks. However, disks around M-type pre-main-sequence stars are still poorly sampled, despite the prevalence of M-dwarfs in the galaxy. Here we present high-resolution (~50 mas, 8 au) ALMA Band 6 observations of six disks around mid-M stars in Taurus. We detect dust continuum emission in all six disks, 12CO in five disks, and 13CO line in two disks. The size ratios between gas and dust disks range from 1.6 to 5.1. The ratio of about 5 for 2M0436 and 2M0450 indicates efficient dust radial drift. Four disks show rings and cavities and two disks are smooth. The cavity sizes occupy a wide range: 60 au for 2M0412, and ~10 au for 2M0434, 2M0436 and 2M0508. Detailed visibility modeling indicates that small cavities of 1.7 and 5.7 au may hide in the two smooth disks 2M0450 and CIDA 12. We perform radiative transfer fitting of the infrared SEDs to constrain the cavity sizes, finding that micron-sized dust grains may have smaller cavities than millimeter grains. Planet-disk interactions are the preferred explanation to produce the large 60 au cavity, while other physics could be responsible for the three ~10 au cavities under current observations and theories. Currently, disks around mid-to-late M stars in Taurus show a higher detection frequency of cavities than earlier type stars, although a more complete sample is needed to evaluate any dependence of substructure on stellar mass.

astro-ph.EP↗

arXiv2024

Exponential contractivity and propagation of chaos for Langevin dynamics of McKean-Vlasov type with Lévy noises

By the probabilistic coupling approach which combines a new refined basic coupling with the synchronous coupling for Lévy processes, we obtain explicit exponential contraction rates in terms of the standard $L^1$-Wasserstein distance for the following Langevin dynamic $(X_t,Y_t)_{t\ge0}$ of McKean-Vlasov type on $\mathbb{R}^{2d}$: \begin{equation*}\left\{\begin{array}{l} dX_t=Y_tdt,\\ dY_t=\left(b(X_t)+\displaystyle\int_{\mathbb{R}^d}\tilde{b}(X_t,z)μ^X_t(dz)-γY_t\right)dt+dL_t,\quad μ^X_t={\rm Law}(X_t),\end{array}\right. \end{equation*} where $γ>0$, $b:\mathbb{R}^d\rightarrow\mathbb{R}^d$ and $\tilde{b}:\mathbb{R}^{2d}\rightarrow\mathbb{R}^d$ are two globally Lipschitz continuous functions, and $(L_t)_{t\ge0}$ is an $\mathbb{R}^d$-valued pure jump Lévy process. The proof is also based on a novel distance function, which is designed according to the distance of the marginals associated with the constructed coupling process. Furthermore, by applying the coupling technique above with some modifications, we also provide the propagation of chaos uniformly in time for the corresponding mean-field interacting particle systems with Lévy noises in the standard $L^1$-Wasserstein distance as well as with explicit bounds.

math.PR↗

arXiv2024

Detecting Adversarial Spectrum Attacks via Distance to Decision Boundary Statistics

Machine learning has been adopted for efficient cooperative spectrum sensing. However, it incurs an additional security risk due to attacks leveraging adversarial machine learning to create malicious spectrum sensing values to deceive the fusion center, called adversarial spectrum attacks. In this paper, we propose an efficient framework for detecting adversarial spectrum attacks. Our design leverages the concept of the distance to the decision boundary (DDB) observed at the fusion center and compares the training and testing DDB distributions to identify adversarial spectrum attacks. We create a computationally efficient way to compute the DDB for machine learning based spectrum sensing systems. Experimental results based on realistic spectrum data show that our method, under typical settings, achieves a high detection rate of up to 99\% and maintains a low false alarm rate of less than 1\%. In addition, our method to compute the DDB based on spectrum data achieves 54\%--64\% improvements in computational efficiency over existing distance calculation methods. The proposed DDB-based detection framework offers a practical and efficient solution for identifying malicious sensing values created by adversarial spectrum attacks.

cs.CR↗

arXiv2024

InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment

Do current large language models (LLMs) better solve graph reasoning and generation tasks with parameter updates? In this paper, we propose InstructGraph, a framework that empowers LLMs with the abilities of graph reasoning and generation by instruction tuning and preference alignment. Specifically, we first propose a structured format verbalizer to unify all graph data into a universal code-like format, which can simply represent the graph without any external graph-specific encoders. Furthermore, a graph instruction tuning stage is introduced to guide LLMs in solving graph reasoning and generation tasks. Finally, we identify potential hallucination problems in graph tasks and sample negative instances for preference alignment, the target of which is to enhance the output's reliability of the model. Extensive experiments across multiple graph-centric tasks exhibit that InstructGraph can achieve the best performance and outperform GPT-4 and LLaMA2 by more than 13\% and 38\%, respectively.

cs.CL↗

arXiv2024

JWST-MIRI Spectroscopy of Warm Molecular Emission and Variability in the AS 209 Disk

We present MIRI MRS observations of the large, multi-gapped protoplanetary disk around the T-Tauri star AS 209. The observations reveal hundreds of water vapor lines from 4.9 to 25.5 $μ$m towards the inner $\sim1$ au in the disk, including the first detection of ro-vibrational water emission in this disk. The spectrum is dominated by hot ($\sim800$ K) water vapor and OH gas, with only marginal detections of CO$_2$, HCN, and a possible colder water vapor component. Using slab models with a detailed treatment of opacities and line overlap, we retrieve the column density, emitting area, and excitation temperature of water vapor and OH, and provide upper limits for the observable mass of other molecules. Compared to MIRI spectra of other T-Tauri disks, the inner disk of AS 209 does not appear to be atypically depleted in CO$_2$ nor HCN. Based on \textit{Spitzer IRS} observations, we further find evidence for molecular emission variability over a 10-year baseline. Water, OH, and CO$_2$ line luminosities have decreased by factors 2-4 in the new MIRI epoch, yet there are minimal continuum emission variations. The origin of this variability is yet to be understood.

astro-ph.EP↗

arXiv2024

Ambush from All Sides: Understanding Security Threats in Open-Source Software CI/CD Pipelines

The continuous integration and continuous deployment (CI/CD) pipelines are widely adopted on Internet hosting platforms, such as GitHub. With the popularity, the CI/CD pipeline faces various security threats. However, current CI/CD pipelines suffer from malicious code and severe vulnerabilities. Even worse, people have not been fully aware of its attack surfaces and the corresponding impacts. Therefore, in this paper, we conduct a large-scale measurement and a systematic analysis to reveal the attack surfaces of the CI/CD pipeline and quantify their security impacts. Specifically, for the measurement, we collect a data set of 320,000+ CI/CD pipeline-configured GitHub repositories and build an analysis tool to parse the CI/CD pipelines and extract security-critical usages. Besides, current CI/CD ecosystem heavily relies on several core scripts, which may lead to a single point of failure. While the CI/CD pipelines contain sensitive information/operations, making them the attacker's favorite targets. Inspired by the measurement findings, we abstract the threat model and the attack approach toward CI/CD pipelines, followed by a systematic analysis of attack surfaces, attack strategies, and the corresponding impacts. We further launch case studies on five attacks in real-world CI/CD environments to validate the revealed attack surfaces. Finally, we give suggestions on mitigating attacks on CI/CD scripts, including securing CI/CD configurations, securing CI/CD scripts, and improving CI/CD infrastructure.

cs.CR↗

arXiv2023

IoTGAN: GAN Powered Camouflage Against Machine Learning Based IoT Device Identification

With the proliferation of IoT devices, researchers have developed a variety of IoT device identification methods with the assistance of machine learning. Nevertheless, the security of these identification methods mostly depends on collected training data. In this research, we propose a novel attack strategy named IoTGAN to manipulate an IoT device's traffic such that it can evade machine learning based IoT device identification. In the development of IoTGAN, we have two major technical challenges: (i) How to obtain the discriminative model in a black-box setting, and (ii) How to add perturbations to IoT traffic through the manipulative model, so as to evade the identification while not influencing the functionality of IoT devices. To address these challenges, a neural network based substitute model is used to fit the target model in black-box settings, it works as a discriminative model in IoTGAN. A manipulative model is trained to add adversarial perturbations into the IoT device's traffic to evade the substitute model. Experimental results show that IoTGAN can successfully achieve the attack goals. We also develop efficient countermeasures to protect machine learning based IoT device identification from been undermined by IoTGAN.

cs.CR↗

arXiv2023

Precipitation Prediction Using an Ensemble of Lightweight Learners

Precipitation prediction plays a crucial role in modern agriculture and industry. However, it poses significant challenges due to the diverse patterns and dynamics in time and space, as well as the scarcity of high precipitation events. To address this challenge, we propose an ensemble learning framework that leverages multiple learners to capture the diverse patterns of precipitation distribution. Specifically, the framework consists of a precipitation predictor with multiple lightweight heads (learners) and a controller that combines the outputs from these heads. The learners and the controller are separately optimized with a proposed 3-stage training scheme. By utilizing provided satellite images, the proposed approach can effectively model the intricate rainfall patterns, especially for high precipitation events. It achieved 1st place on the core test as well as the nowcasting leaderboards of the Weather4Cast 2023 competition. For detailed implementation, please refer to our GitHub repository at: https://github.com/lxz1217/weather4cast-2023-lxz.

physics.ao-ph↗

arXiv2023

Water-Rich Disks around Late M-stars Unveiled: Exploring the Remarkable Case of Sz114

We present an analysis of the JDISC JWST/MIRI-MRS spectrum of Sz~114, an accreting M5 star surrounded by a large dust disk with a shallow gap at $\sim 39$ au. The spectrum is molecular-rich: we report the detection of water, CO, CO$_2$, HCN, C$_2$H$_2$, and H$_2$. The only identified atomic/ionic transition is from [NeII] at 12.81 micron. A distinct feature of this spectrum is the forest of water lines with the 17.22 micron emission surpassing that of most mid-to-late M-star disks by an order of magnitude in flux and aligning instead with disks of earlier-type stars. Moreover, flux ratios of C$_2$H$_2$/H$_2$O and HCN/H$_2$O in Sz~114 also resemble those of earlier-type disks, with a slightly elevated CO$_2$/H$_2$O ratio. While accretional heating can boost all infrared lines, the unusual properties of Sz~114 could be explained by the young age of the source, its formation under unusual initial conditions (a large massive disk), and the presence of dust substructures. The latter delays the inward drift of icy pebbles and help preserve a lower C/O ratio over an extended period. In contrast, mid-to-late M-star disks--which are typically faint, small in size, and likely lack significant substructures--may have more quickly depleted the outer icy reservoir and already evolved out of a water-rich inner disk phase. Our findings underscore the unexpected diversity within mid-infrared spectra of mid-to-late M-star disks, highlighting the need to expand the observational sample for a comprehensive understanding of their variations and thoroughly test pebble drift and planet formation models.

astro-ph.EP↗

arXiv2023

Parrot-Trained Adversarial Examples: Pushing the Practicality of Black-Box Audio Attacks against Speaker Recognition Models

Audio adversarial examples (AEs) have posed significant security challenges to real-world speaker recognition systems. Most black-box attacks still require certain information from the speaker recognition model to be effective (e.g., keeping probing and requiring the knowledge of similarity scores). This work aims to push the practicality of the black-box attacks by minimizing the attacker's knowledge about a target speaker recognition model. Although it is not feasible for an attacker to succeed with completely zero knowledge, we assume that the attacker only knows a short (or a few seconds) speech sample of a target speaker. Without any probing to gain further knowledge about the target model, we propose a new mechanism, called parrot training, to generate AEs against the target model. Motivated by recent advancements in voice conversion (VC), we propose to use the one short sentence knowledge to generate more synthetic speech samples that sound like the target speaker, called parrot speech. Then, we use these parrot speech samples to train a parrot-trained(PT) surrogate model for the attacker. Under a joint transferability and perception framework, we investigate different ways to generate AEs on the PT model (called PT-AEs) to ensure the PT-AEs can be generated with high transferability to a black-box target model with good human perceptual quality. Real-world experiments show that the resultant PT-AEs achieve the attack success rates of 45.8% - 80.8% against the open-source models in the digital-line scenario and 47.9% - 58.3% against smart devices, including Apple HomePod (Siri), Amazon Echo, and Google Home, in the over-the-air scenario.

cs.SD↗

arXiv2023

MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluation

Curated datasets for healthcare are often limited due to the need of human annotations from experts. In this paper, we present MedEval, a multi-level, multi-task, and multi-domain medical benchmark to facilitate the development of language models for healthcare. MedEval is comprehensive and consists of data from several healthcare systems and spans 35 human body regions from 8 examination modalities. With 22,779 collected sentences and 21,228 reports, we provide expert annotations at multiple levels, offering a granular potential usage of the data and supporting a wide range of tasks. Moreover, we systematically evaluated 10 generic and domain-specific language models under zero-shot and finetuning settings, from domain-adapted baselines in healthcare to general-purposed state-of-the-art large language models (e.g., ChatGPT). Our evaluations reveal varying effectiveness of the two categories of language models across different tasks, from which we notice the importance of instruction tuning for few-shot usage of large language models. Our investigation paves the way toward benchmarking language models for healthcare and provides valuable insights into the strengths and limitations of adopting large language models in medical domains, informing their practical applications and future advancements.

cs.CL↗