arXiv subjects

Yihan Wang

Publications and source records attributed to Yihan Wang.

At least 109 records · Page 6Linked to original sources

arXiv2024

Game-Theoretic Unlearnable Example Generator

Unlearnable example attacks are data poisoning attacks aiming to degrade the clean test accuracy of deep learning by adding imperceptible perturbations to the training samples, which can be formulated as a bi-level optimization problem. However, directly solving this optimization problem is intractable for deep neural networks. In this paper, we investigate unlearnable example attacks from a game-theoretic perspective, by formulating the attack as a nonzero sum Stackelberg game. First, the existence of game equilibria is proved under the normal setting and the adversarial training setting. It is shown that the game equilibrium gives the most powerful poison attack in that the victim has the lowest test accuracy among all networks within the same hypothesis space, when certain loss functions are used. Second, we propose a novel attack method, called the Game Unlearnable Example (GUE), which has three main gradients. (1) The poisons are obtained by directly solving the equilibrium of the Stackelberg game with a first-order algorithm. (2) We employ an autoencoder-like generative network model as the poison attacker. (3) A novel payoff function is introduced to evaluate the performance of the poison. Comprehensive experiments demonstrate that GUE can effectively poison the model in various scenarios. Furthermore, the GUE still works by using a relatively small percentage of the training data to train the generator, and the poison generator can generalize to unseen data well. Our implementation code can be found at https://github.com/hong-xian/gue.

cs.LG↗

arXiv2024

Anisotropic Energy Injection from Magnetar Central Engines in Short GRBs

A long-lived magnetar, potentially originating from a binary neutron star system, has been proposed to explain the extended emission observed in certain short-duration gamma-ray bursts (sGRBs), and is posited as a potential central engine to power the engine-fed kilonovae. Previously, the process by which energy is injected into the surrounding ejecta/jet was widely believed to be nearly isotropic. In this study, we employ special relativity magnetohydrodynamic (SRMHD) simulations to investigate the wind injection process from a magnetar central engine. We explore the dynamics and energy distribution within the system and found that the parameter $α=u_{\rm A}/u_{\rm MWN}$ can be used to indicate the collimation of the magnetar wind energy injection, where $u_{\rm A}$ is the local Alfven four-speed and $u_{\rm MWN}$ is the four-speed of the magnetar wind nebular (MWN) formed from wind-ejecta collision. A significant portion of the injected energy from the magnetar spin-down wind will be channeled to the jet axis due to collimation within the MWN. Achieving isotropic energy injection requires a significantly small $α$ that necessitates either an ultra-relativistic expanding MWN or an extremely low magnetization MWN, both of which are challenging to attain in sGRBs. Consequently, a considerably reduced energy budget (i.e. energy per solid angle reduced by a factor of up to 10 with respect to the value under isotropic assumption) is anticipated to be injected into the ejecta for engine-fed kilonovae. Engine-fed kilonovae would appear fainter than originally anticipated.

astro-ph.HE↗

arXiv2024

Data-Dependent Stability Analysis of Adversarial Training

Stability analysis is an essential aspect of studying the generalization ability of deep learning, as it involves deriving generalization bounds for stochastic gradient descent-based training algorithms. Adversarial training is the most widely used defense against adversarial example attacks. However, previous generalization bounds for adversarial training have not included information regarding the data distribution. In this paper, we fill this gap by providing generalization bounds for stochastic gradient descent-based adversarial training that incorporate data distribution information. We utilize the concepts of on-average stability and high-order approximate Lipschitz conditions to examine how changes in data distribution and adversarial budget can affect robust generalization gaps. Our derived generalization bounds for both convex and non-convex losses are at least as good as the uniform stability-based counterparts which do not include data distribution information. Furthermore, our findings demonstrate how distribution shifts from data poisoning attacks can impact robust generalization.

cs.LG↗

arXiv2023

Universality and Limitations of Prompt Tuning

Despite the demonstrated empirical efficacy of prompt tuning to adapt a pretrained language model for a new task, the theoretical underpinnings of the difference between "tuning parameters before the input" against "the tuning of model weights" are limited. We thus take one of the first steps to understand the role of soft-prompt tuning for transformer-based architectures. By considering a general purpose architecture, we analyze prompt tuning from the lens of both: universal approximation and limitations with finite-depth fixed-weight pretrained transformers for continuous-valued functions. Our universality result guarantees the existence of a strong transformer with a prompt to approximate any sequence-to-sequence function in the set of Lipschitz functions. The limitations of prompt tuning for limited-depth transformers are first proved by constructing a set of datasets, that cannot be memorized by a prompt of any length for a given single encoder layer. We also provide a lower bound on the required number of tunable prompt parameters and compare the result with the number of parameters required for a low-rank update (based on LoRA) for a single-layer setting. We finally extend our analysis to multi-layer settings by providing sufficient conditions under which the transformer can at best learn datasets from invertible functions only. Our theoretical claims are also corroborated by empirical results.

cs.LG↗

arXiv2023

Improving the Generation Quality of Watermarked Large Language Models via Word Importance Scoring

The strong general capabilities of Large Language Models (LLMs) bring potential ethical risks if they are unrestrictedly accessible to malicious users. Token-level watermarking inserts watermarks in the generated texts by altering the token probability distributions with a private random number generator seeded by its prefix tokens. However, this watermarking algorithm alters the logits during generation, which can lead to a downgraded text quality if it chooses to promote tokens that are less relevant given the input. In this work, we propose to improve the quality of texts generated by a watermarked language model by Watermarking with Importance Scoring (WIS). At each generation step, we estimate the importance of the token to generate, and prevent it from being impacted by watermarking if it is important for the semantic correctness of the output. We further propose three methods to predict importance scoring, including a perturbation-based method and two model-based methods. Empirical experiments show that our method can generate texts with better quality with comparable level of detection rate.

cs.CL↗

arXiv2023

MVFAN: Multi-View Feature Assisted Network for 4D Radar Object Detection

4D radar is recognized for its resilience and cost-effectiveness under adverse weather conditions, thus playing a pivotal role in autonomous driving. While cameras and LiDAR are typically the primary sensors used in perception modules for autonomous vehicles, radar serves as a valuable supplementary sensor. Unlike LiDAR and cameras, radar remains unimpaired by harsh weather conditions, thereby offering a dependable alternative in challenging environments. Developing radar-based 3D object detection not only augments the competency of autonomous vehicles but also provides economic benefits. In response, we propose the Multi-View Feature Assisted Network (\textit{MVFAN}), an end-to-end, anchor-free, and single-stage framework for 4D-radar-based 3D object detection for autonomous vehicles. We tackle the issue of insufficient feature utilization by introducing a novel Position Map Generation module to enhance feature learning by reweighing foreground and background points, and their features, considering the irregular distribution of radar point clouds. Additionally, we propose a pioneering backbone, the Radar Feature Assisted backbone, explicitly crafted to fully exploit the valuable Doppler velocity and reflectivity data provided by the 4D radar sensor. Comprehensive experiments and ablation studies carried out on Astyx and VoD datasets attest to the efficacy of our framework. The incorporation of Doppler velocity and RCS reflectivity dramatically improves the detection performance for small moving objects such as pedestrians and cyclists. Consequently, our approach culminates in a highly optimized 4D-radar-based 3D object detection capability for autonomous driving systems, setting a new standard in the field.

cs.CV↗

arXiv2023

Red Teaming Language Model Detectors with Language Models

The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. To prevent the potentially deceptive usage of LLMs, recent works have proposed algorithms to detect LLM-generated text and protect LLMs. In this paper, we investigate the robustness and reliability of these LLM detectors under adversarial attacks. We study two types of attack strategies: 1) replacing certain words in an LLM's output with their synonyms given the context; 2) automatically searching for an instructional prompt to alter the writing style of the generation. In both strategies, we leverage an auxiliary LLM to generate the word replacements or the instructional prompt. Different from previous works, we consider a challenging setting where the auxiliary LLM can also be protected by a detector. Experiments reveal that our attacks effectively compromise the performance of all detectors in the study with plausible generations, underscoring the urgent need to improve the robustness of LLM-generated text detection systems.

cs.CL↗

arXiv2023

Changing-Look AGN Behaviour Induced by Disk-Captured Tidal Disruption Events

Recent observations of changing-look active galactic nuclei (AGN) hint at a frequency of accretion activity not fully explained by tidal disruption events (TDEs) stemming from relaxation processes in nucleus star clusters (NSCs), traditionally estimated to occur at rates of $10^{-4}$ to $10^{-5}$ yr$^{-1}$ per galaxy. In this letter, we propose an enhanced TDE rate through the AGN disk capture process, presenting a viable explanation for the frequent transitions observed in changing-look AGN. Specifically, we investigate the interaction between the accretion disk and retrograde stars within NSCs, resulting in the rapid occurrence of TDEs within a condensed time frame. Through detailed calculations, we derive the time-dependent TDE rates for both relaxation-induced TDE and disk-captured TDE. Our analysis reveals that TDEs triggered by the disk capture process can notably amplify the TDE rate by several orders of magnitude during the AGN phase. This mechanism offers a potential explanation for the enhanced high-energy variability characteristic of changing-look AGNs.

astro-ph.HE↗

arXiv2023

ThermRad: A Multi-modal Dataset for Robust 3D Object Detection under Challenging Conditions

Robust 3D object detection in extreme weather and illumination conditions is a challenging task. While radars and thermal cameras are known for their resilience to these conditions, few studies have been conducted on radar-thermal fusion due to the lack of corresponding datasets. To address this gap, we first present a new multi-modal dataset called ThermRad, which includes a 3D LiDAR, a 4D radar, an RGB camera and a thermal camera. This dataset is unique because it includes data from all four sensors in extreme weather conditions, providing a valuable resource for future research in this area. To validate the robustness of 4D radars and thermal cameras for 3D object detection in challenging weather conditions, we propose a new multi-modal fusion method called RTDF-RCNN, which leverages the complementary strengths of 4D radars and thermal cameras to boost object detection performance. To further prove the effectiveness of our proposed framework, we re-implement state-of-the-art (SOTA) 3D detectors on our dataset as benchmarks for evaluation. Our method achieves significant enhancements in detecting cars, pedestrians, and cyclists, with improvements of over 7.98%, 24.27%, and 27.15%, respectively, while achieving comparable results to LiDAR-based approaches. Our contributions in both the ThermRad dataset and the new multi-modal fusion method provide a new approach to robust 3D object detection in adverse weather and illumination conditions. The ThermRad dataset will be released.

cs.CV↗

arXiv2023

R2Det: Redemption from Range-view for Accurate 3D Object Detection

LiDAR-based 3D object detection is of paramount importance for autonomous driving. Recent trends show a remarkable improvement for bird's-eye-view (BEV) based and point-based methods as they demonstrate superior performance compared to range-view counterparts. This paper presents an insight that leverages range-view representation to enhance 3D points for accurate 3D object detection. Specifically, we introduce a Redemption from Range-view Module (R2M), a plug-and-play approach for 3D surface texture enhancement from the 2D range view to the 3D point view. R2M comprises BasicBlock for 2D feature extraction, Hierarchical-dilated (HD) Meta Kernel for expanding the 3D receptive field, and Feature Points Redemption (FPR) for recovering 3D surface texture information. R2M can be seamlessly integrated into state-of-the-art LiDAR-based 3D object detectors as preprocessing and achieve appealing improvement, e.g., 1.39%, 1.67%, and 1.97% mAP improvement on easy, moderate, and hard difficulty level of KITTI val set, respectively. Based on R2M, we further propose R2Detector (R2Det) with the Synchronous-Grid RoI Pooling for accurate box refinement. R2Det outperforms existing range-view-based methods by a significant margin on both the KITTI benchmark and the Waymo Open Dataset. Codes will be made publicly available.

cs.CV↗

arXiv2023

Democratizing Pathological Image Segmentation with Lay Annotators via Molecular-empowered Learning

Multi-class cell segmentation in high-resolution Giga-pixel whole slide images (WSI) is critical for various clinical applications. Training such an AI model typically requires labor-intensive pixel-wise manual annotation from experienced domain experts (e.g., pathologists). Moreover, such annotation is error-prone when differentiating fine-grained cell types (e.g., podocyte and mesangial cells) via the naked human eye. In this study, we assess the feasibility of democratizing pathological AI deployment by only using lay annotators (annotators without medical domain knowledge). The contribution of this paper is threefold: (1) We proposed a molecular-empowered learning scheme for multi-class cell segmentation using partial labels from lay annotators; (2) The proposed method integrated Giga-pixel level molecular-morphology cross-modality registration, molecular-informed annotation, and molecular-oriented segmentation model, so as to achieve significantly superior performance via 3 lay annotators as compared with 2 experienced pathologists; (3) A deep corrective learning (learning with imperfect label) method is proposed to further improve the segmentation performance using partially annotated noisy data. From the experimental results, our learning method achieved F1 = 0.8496 using molecular-informed annotations from lay annotators, which is better than conventional morphology-based annotations (F1 = 0.7015) from experienced pathologists. Our method democratizes the development of a pathological segmentation deep model to the lay annotator level, which consequently scales up the learning process similar to a non-medical computer vision task. The official implementation and cell annotations are publicly available at https://github.com/hrlblab/MolecularEL.

eess.IV↗

arXiv2023

Restore Translation Using Equivariant Neural Networks

Invariance to spatial transformations such as translations and rotations is a desirable property and a basic design principle for classification neural networks. However, the commonly used convolutional neural networks (CNNs) are actually very sensitive to even small translations. There exist vast works to achieve exact or approximate transformation invariance by designing transformation-invariant models or assessing the transformations. These works usually make changes to the standard CNNs and harm the performance on standard datasets. In this paper, rather than modifying the classifier, we propose a pre-classifier restorer to recover translated (or even rotated) inputs to the original ones which will be fed into any classifier for the same dataset. The restorer is based on a theoretical result which gives a sufficient and necessary condition for an affine operator to be translational equivariant on a tensor space.

cs.LG↗

arXiv2023

Infinite Photorealistic Worlds using Procedural Generation

We introduce Infinigen, a procedural generator of photorealistic 3D scenes of the natural world. Infinigen is entirely procedural: every asset, from shape to texture, is generated from scratch via randomized mathematical rules, using no external source and allowing infinite variation and composition. Infinigen offers broad coverage of objects and scenes in the natural world including plants, animals, terrains, and natural phenomena such as fire, cloud, rain, and snow. Infinigen can be used to generate unlimited, diverse training data for a wide range of computer vision tasks including object detection, semantic segmentation, optical flow, and 3D reconstruction. We expect Infinigen to be a useful resource for computer vision research and beyond. Please visit https://infinigen.org for videos, code and pre-generated data.

cs.CV↗

arXiv2023

On Periodic Points in Covering Systems

We study a system of intervals $I_1,\ldots,I_k$ on the real line and a continuous map $f$ with $f(I_1 \cup I_2 \cup \ldots \cup I_k)\supseteq I_1 \cup I_2 \cup \ldots \cup I_k$. It's conjectured that there exists a periodic point of period $\le k$ in $I_1\cup \ldots \cup I_k$. In this paper, we prove the conjecture by a discretization method and reduce the initial problem to an interesting combinatorial lemma concerning cyclic permutations. We also obtain a non-concentration property of periodic points of small periods in intervals.

math.DS↗

arXiv2023

Tidal Peeling Events: low-eccentricity tidal disruption of a star by a stellar-mass black hole

Close encounters between stellar-mass black holes (BHs) and stars occur frequently in dense star clusters and in the disks of active galactic nuclei (AGNs). Recent studies have shown that in highly eccentric close encounters, the star can be tidally disrupted by the BH (micro-tidal disruption event, or micro-TDE), resulting in rapid mass accretion and possibly bright electromagnetic signatures. Here we consider a scenario in which the star might approach the stellar-mass BH in a gradual, nearly circular inspiral, under the influence of dynamical friction on a circum-binary gas disk or three-body interactions in a star cluster. We perform hydro-dynamical simulations of this scenario using the smoothed particle hydrodynamics code PHANTOM. We find that the mass of the star is slowly stripped away by the BH. We call this gradual tidal disruption a "tidal-peeling event", or a TPE. Depending on the initial distance and eccentricity of the encounter, TPEs might exhibit significant accretion rates and orbital evolution distinct from those of a typical (eccentric) micro-TDE.

astro-ph.HE↗

arXiv2023

Effective two-body scatterings around a massive object

Two-body scatterings under the potential of a massive object are very common in astrophysics. If the massive body is far enough away that the two small bodies are in their own gravitational sphere of influence, the gravity of the massive body can be temporarily ignored. However, this requires the scattering process to be fast enough that the small objects do not spend too much time at distances near the surface of the sphere of influence. In this paper, we derive the validation criteria for effective two-body scattering and establish a simple analytical solution for this process, which we verify through numerical scattering experiments. We use this solution to study star-black hole scatterings in the disks of Active Galactic Nuclei and planet-planet scatterings in planetary systems, and calculate their one-dimensional cross-section analytically. Our solution will be valuable in reducing computational time when treating two-body scatterings under the potential of a much more massive third body, provided that the problem settings are in the valid parameter space region identified by our study.

astro-ph.SR↗

arXiv2022

RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning

Prompting has shown impressive success in enabling large pretrained language models (LMs) to perform diverse NLP tasks, especially when only few downstream data are available. Automatically finding the optimal prompt for each task, however, is challenging. Most existing work resorts to tuning soft prompt (e.g., embeddings) which falls short of interpretability, reusability across LMs, and applicability when gradients are not accessible. Discrete prompt, on the other hand, is difficult to optimize, and is often created by "enumeration (e.g., paraphrasing)-then-selection" heuristics that do not explore the prompt space systematically. This paper proposes RLPrompt, an efficient discrete prompt optimization approach with reinforcement learning (RL). RLPrompt formulates a parameter-efficient policy network that generates the desired discrete prompt after training with reward. To overcome the complexity and stochasticity of reward signals by the large LM environment, we incorporate effective reward stabilization that substantially enhances the training efficiency. RLPrompt is flexibly applicable to different types of LMs, such as masked (e.g., BERT) and left-to-right models (e.g., GPTs), for both classification and generation tasks. Experiments on few-shot classification and unsupervised text style transfer show superior performance over a wide range of existing finetuning or prompting methods. Interestingly, the resulting optimized prompts are often ungrammatical gibberish text; and surprisingly, those gibberish prompts are transferrable between different LMs to retain significant performance, indicating LM prompting may not follow human language patterns.

cs.CL↗

arXiv2022

Efficiently Computing Local Lipschitz Constants of Neural Networks via Bound Propagation

Lipschitz constants are connected to many properties of neural networks, such as robustness, fairness, and generalization. Existing methods for computing Lipschitz constants either produce relatively loose upper bounds or are limited to small networks. In this paper, we develop an efficient framework for computing the $\ell_\infty$ local Lipschitz constant of a neural network by tightly upper bounding the norm of Clarke Jacobian via linear bound propagation. We formulate the computation of local Lipschitz constants with a linear bound propagation process on a high-order backward graph induced by the chain rule of Clarke Jacobian. To enable linear bound propagation, we derive tight linear relaxations for specific nonlinearities in Clarke Jacobian. This formulate unifies existing ad-hoc approaches such as RecurJac, which can be seen as a special case of ours with weaker relaxations. The bound propagation framework also allows us to easily borrow the popular Branch-and-Bound (BaB) approach from neural network verification to further tighten Lipschitz constants. Experiments show that on tiny models, our method produces comparable bounds compared to exact methods that cannot scale to slightly larger models; on larger models, our method efficiently produces tighter results than existing relaxed or naive methods, and our method scales to much larger practical models that previous works could not handle. We also demonstrate an application on provable monotonicity analysis. Code is available at https://github.com/shizhouxing/Local-Lipschitz-Constants.

cs.LG↗