Searcharxiv⌕ Search

arXiv subjects

Han Wang

Publications and source records attributed to Han Wang.

At least 325 records · Page 18Linked to original sources
arXiv2024

MemeCraft: Contextual and Stance-Driven Multimodal Meme Generation

Online memes have emerged as powerful digital cultural artifacts in the age of social media, offering not only humor but also platforms for political discourse, social critique, and information dissemination. Their extensive reach and influence in shaping online communities' sentiments make them invaluable tools for campaigning and promoting ideologies. Despite the development of several meme-generation tools, there remains a gap in their systematic evaluation and their ability to effectively communicate ideologies. Addressing this, we introduce MemeCraft, an innovative meme generator that leverages large language models (LLMs) and visual language models (VLMs) to produce memes advocating specific social movements. MemeCraft presents an end-to-end pipeline, transforming user prompts into compelling multimodal memes without manual intervention. Conscious of the misuse potential in creating divisive content, an intrinsic safety mechanism is embedded to curb hateful meme production.

cs.CY↗
arXiv2024

Searching for singlet vector-like leptons via pair production at ILC

Vector-like leptons (VLLs) as one kind of the most intriguing particles, have been widely concerned in several extensions of the Standard Model (SM). In this work, we explore the discovery potential of VLLs via pair production in the context of models that satisfy asymptotic safety at the International Linear Collider (ILC). The expected sensitivities of the ILC with the center of mass energy $\sqrt{s} =$ 1 TeV and the integrated luminosity $\mathcal{L}$ = 1 ab$^{-1}$ to the parameter space of this kind of VLL models are derived. The results we obtained show that, for the VLL with mass $M_F$ in the region of $M_{F}\in$ [101GeV, 463GeV], the Yukawa coupling $κ$ can be as low as $κ\in$ [0.032, 0.098].

hep-ph↗
arXiv2024

MMAUD: A Comprehensive Multi-Modal Anti-UAV Dataset for Modern Miniature Drone Threats

In response to the evolving challenges posed by small unmanned aerial vehicles (UAVs), which possess the potential to transport harmful payloads or independently cause damage, we introduce MMAUD: a comprehensive Multi-Modal Anti-UAV Dataset. MMAUD addresses a critical gap in contemporary threat detection methodologies by focusing on drone detection, UAV-type classification, and trajectory estimation. MMAUD stands out by combining diverse sensory inputs, including stereo vision, various Lidars, Radars, and audio arrays. It offers a unique overhead aerial detection vital for addressing real-world scenarios with higher fidelity than datasets captured on specific vantage points using thermal and RGB. Additionally, MMAUD provides accurate Leica-generated ground truth data, enhancing credibility and enabling confident refinement of algorithms and models, which has never been seen in other datasets. Most existing works do not disclose their datasets, making MMAUD an invaluable resource for developing accurate and efficient solutions. Our proposed modalities are cost-effective and highly adaptable, allowing users to experiment and implement new UAV threat detection tools. Our dataset closely simulates real-world scenarios by incorporating ambient heavy machinery sounds. This approach enhances the dataset's applicability, capturing the exact challenges faced during proximate vehicular operations. It is expected that MMAUD can play a pivotal role in advancing UAV threat detection, classification, trajectory estimation capabilities, and beyond. Our dataset, codes, and designs will be available in https://github.com/ntu-aris/MMAUD.

cs.RO↗
arXiv2024

Few-Shot Character Understanding in Movies as an Assessment to Meta-Learning of Theory-of-Mind

When reading a story, humans can quickly understand new fictional characters with a few observations, mainly by drawing analogies to fictional and real people they already know. This reflects the few-shot and meta-learning essence of humans' inference of characters' mental states, i.e., theory-of-mind (ToM), which is largely ignored in existing research. We fill this gap with a novel NLP dataset, ToM-in-AMC, the first assessment of machines' meta-learning of ToM in a realistic narrative understanding scenario. Our dataset consists of ~1,000 parsed movie scripts, each corresponding to a few-shot character understanding task that requires models to mimic humans' ability of fast digesting characters with a few starting scenes in a new movie. We propose a novel ToM prompting approach designed to explicitly assess the influence of multiple ToM dimensions. It surpasses existing baseline models, underscoring the significance of modeling multiple ToM dimensions for our task. Our extensive human study verifies that humans are capable of solving our problem by inferring characters' mental states based on their previously seen movies. In comparison, our systems based on either state-of-the-art large language models (GPT-4) or meta-learning algorithms lags >20% behind, highlighting a notable limitation in existing approaches' ToM capabilities.

cs.CL↗
arXiv2024

Hierarchical Game for Coupled Power System with Energy Sharing and Transportation System

The wide deployment of distributed renewable energy sources and electric vehicles can help mitigate climate crisis. This necessitates new business models in the power sector to hedge against uncertainties while imposing a strong coupling between the connected power and transportation networks. To address these challenges, this paper first proposes an energy sharing mechanism considering AC power network constraints to encourage local energy exchange in the power system. Under the proposed mechanism, all prosumers play a generalized Nash game. We prove that the energy sharing equilibrium exists and is socially optimal. Furthermore, a hierarchical game is built to characterize the interactions both inside and between the power and transportation systems. Externally, the two systems are engaged in a generalized Nash game because traffic flows serve as electric demands by charging behaviors, and each driver pays the energy sharing price for charging. The hierarchical game is then converted into a mixed-integer linear program (MILP) with the help of optimality conditions and linearization techniques. Numerical experiments validate the theoretical results and show the mutual impact between the two systems.

eess.SY↗
arXiv2024

Nonlinear Optimal Guidance for Cooperatively Imposing Relative Intercept Angles

The optimal cooperative guidance in the nonlinear setting for intercepting a target by multiple pursuers is studied in the paper. As certain relative angles can improve observability, the guidance command is required to cooperatively control the pursuers to intercept the target with specific relative angles. By using the neural networks, an approach for real-time generation of the nonlinear cooperative optimal guidance command is developed. Specifically, the optimal control problem with constraints on relative intercepting angles is formulated. Then, Pontryagin's maximum principle is used to derive the necessary conditions for optimality, which are further employed to parameterize the nonlinear optimal guidance law. As a result, the dataset for the mapping from state to nonlinear optimal guidance command can be generated by a simple propagation. A simple feedforward neural network is trained by the dataset to generate the nonlinear optimal guidance command. Finally, numerical examples are presented, showing that a nonlinear optimal guidance command with specific relative angles can be generated within a faction of a millisecond.

math.OC↗
arXiv2024

Model-Free Verification for Neural Network Controlled Systems

Neural network controllers have shown potential in achieving superior performance in feedback control systems. Although a neural network can be trained efficiently using deep and reinforcement learning methods, providing formal guarantees for the closed-loop properties is challenging. The main difficulty comes from the nonlinear activation functions. One popular method is to use sector bounds on the activation functions resulting in a robust analysis. These methods work well under the assumption that the system dynamics are perfectly known, which is, however, impossible in practice. In this paper, we propose data-driven semi-definite programs to formally verify stability and safety for a neural network controlled linear system with unknown dynamics. The proposed method performs verification directly from end-to-end without identifying the dynamics. Through a numerical example, we validate the efficacy of our method on linear systems with controller trained by imitation learning.

math.OC↗
arXiv2024

GloTSFormer: Global Video Text Spotting Transformer

Video Text Spotting (VTS) is a fundamental visual task that aims to predict the trajectories and content of texts in a video. Previous works usually conduct local associations and apply IoU-based distance and complex post-processing procedures to boost performance, ignoring the abundant temporal information and the morphological characteristics in VTS. In this paper, we propose a novel Global Video Text Spotting Transformer GloTSFormer to model the tracking problem as global associations and utilize the Gaussian Wasserstein distance to guide the morphological correlation between frames. Our main contributions can be summarized as three folds. 1). We propose a Transformer-based global tracking method GloTSFormer for VTS and associate multiple frames simultaneously. 2). We introduce a Wasserstein distance-based method to conduct positional associations between frames. 3). We conduct extensive experiments on public datasets. On the ICDAR2015 video dataset, GloTSFormer achieves 56.0 MOTA with 4.6 absolute improvement compared with the previous SOTA method and outperforms the previous Transformer-based method by a significant 8.3 MOTA.

cs.CV↗
arXiv2023

Versatile LiDAR-Inertial Odometry With SE (2) Constraints for Ground Vehicles

LiDAR SLAM has become one of the major localization systems for ground vehicles since LiDAR Odometry And Mapping (LOAM). Many extension works on LOAM mainly leverage one specific constraint to improve the performance, e.g., information from on-board sensors such as loop closure and inertial state; prior conditions such as ground level and motion dynamics. In many robotic applications, these conditions are often known partially, hence a SLAM system can be a comprehensive problem due to the existence of numerous constraints. Therefore, we can achieve a better SLAM result by fusing them properly. In this paper, we propose a hybrid LiDAR-inertial SLAM framework that leverages both the on-board perception system and prior information such as motion dynamics to improve localization performance. In particular, we consider the case for ground vehicles, which are commonly used for autonomous driving and warehouse logistics. We present a computationally efficient LiDAR-inertial odometry method that directly parameterizes ground vehicle poses on SE(2). The out-of-SE(2) motion perturbations are not neglected but incorporated into an integrated noise term of a novel SE(2)-constraints model. For odometric measurement processing, we propose a versatile, tightly coupled LiDAR-inertial odometry to achieve better pose estimation than traditional LiDAR odometry. Thorough experiments are performed to evaluate our proposed method's performance in different scenarios, including localization for both indoor and outdoor environments. The proposed method achieves superior performance in accuracy and robustness.

cs.RO↗
arXiv2023

Prompting Large Language Models for Topic Modeling

Topic modeling is a widely used technique for revealing underlying thematic structures within textual data. However, existing models have certain limitations, particularly when dealing with short text datasets that lack co-occurring words. Moreover, these models often neglect sentence-level semantics, focusing primarily on token-level semantics. In this paper, we propose PromptTopic, a novel topic modeling approach that harnesses the advanced language understanding of large language models (LLMs) to address these challenges. It involves extracting topics at the sentence level from individual documents, then aggregating and condensing these topics into a predefined quantity, ultimately providing coherent topics for texts of varying lengths. This approach eliminates the need for manual parameter tuning and improves the quality of extracted topics. We benchmark PromptTopic against the state-of-the-art baselines on three vastly diverse datasets, establishing its proficiency in discovering meaningful topics. Furthermore, qualitative analysis showcases PromptTopic's ability to uncover relevant topics in multiple datasets.

cs.AI↗
arXiv2023

Generation of 3D Molecules in Pockets via Language Model

Generative models for molecules based on sequential line notation (e.g. SMILES) or graph representation have attracted an increasing interest in the field of structure-based drug design, but they struggle to capture important 3D spatial interactions and often produce undesirable molecular structures. To address these challenges, we introduce Lingo3DMol, a pocket-based 3D molecule generation method that combines language models and geometric deep learning technology. A new molecular representation, fragment-based SMILES with local and global coordinates, was developed to assist the model in learning molecular topologies and atomic spatial positions. Additionally, we trained a separate noncovalent interaction predictor to provide essential binding pattern information for the generative model. Lingo3DMol can efficiently traverse drug-like chemical spaces, preventing the formation of unusual structures. The Directory of Useful Decoys-Enhanced (DUD-E) dataset was used for evaluation. Lingo3DMol outperformed state-of-the-art methods in terms of drug-likeness, synthetic accessibility, pocket binding mode, and molecule generation speed.

cs.LG↗
arXiv2023

PromptMTopic: Unsupervised Multimodal Topic Modeling of Memes using Large Language Models

The proliferation of social media has given rise to a new form of communication: memes. Memes are multimodal and often contain a combination of text and visual elements that convey meaning, humor, and cultural significance. While meme analysis has been an active area of research, little work has been done on unsupervised multimodal topic modeling of memes, which is important for content moderation, social media analysis, and cultural studies. We propose \textsf{PromptMTopic}, a novel multimodal prompt-based model designed to learn topics from both text and visual modalities by leveraging the language modeling capabilities of large language models. Our model effectively extracts and clusters topics learned from memes, considering the semantic interaction between the text and visual modalities. We evaluate our proposed model through extensive experiments on three real-world meme datasets, which demonstrate its superiority over state-of-the-art topic modeling baselines in learning descriptive topics in memes. Additionally, our qualitative analysis shows that \textsf{PromptMTopic} can identify meaningful and culturally relevant topics from memes. Our work contributes to the understanding of the topics and themes of memes, a crucial form of communication in today's society.\\ \red{\textbf{Disclaimer: This paper contains sensitive content that may be disturbing to some readers.}}

cs.CL↗
arXiv2023

Detecting the coupling of axion-like particles with fermions at the ILC

New pseudoscalars, axion-like particles (ALPs), provide the exciting target for present and future collider-based experiments. Search for ALPs is performed in this paper via the $W^{+}W^{-}$ fusion process $e^{-}e^{+}\rightarrowν_{e}\overline{ν_{e}}a\rightarrowν_{e}\overline{ν_{e}}f\overline{f}$ at the $1$ TeV ILC corresponding to an integrated luminosity of $1$ ab$^{-1}$ and the beam polarization $P(e^{-}$, $e^{+}) = (-80\%$, $+20\%)$. Owing to the good capability of the ILC in performing b-tagging and the sufficiently large branching ratio of the ALP decaying into a pair of b quarks, the decay channel $a\rightarrow{b\overline{b}}$ is mainly concerned. The prospective sensitivities provided by the ILC on the ALP-fermion coupling as low as $1$ TeV$^{-1}$ and $1.75$ TeV$^{-1}$ are derived at $95\%$ confidence level in the ALP mass intervals $37-50$ GeV and $52-300$ GeV, respectively. Our results will help to probe significant parameter space in an unexplored region beyond the existing constraints.

hep-ph↗
arXiv2023

Parametrization Approach for Real-Time Generation of Minimum-Effort Trajectories via Neural Network

This paper is concerned with real-time generation of optimal flight trajectories for Minimum-Effort Control Problems (MECPs), which is fundamentally important for autonomous flight of aerospace vehicles. Although existing optimal control methods, such as indirect methods and direct methods, can be amended to solve MECPs, it is very challenging to obtain, in real time, the solution trajectories since those methods suffer the issue of convergence. As the artificial neural network can generate its output within a constant time, it has been alternative for real-time generation of optimal trajectories in the literature. The usual way is to train neural networks by solutions from indirect or direct methods, which, however, cannot ensure sufficient conditions for optimality to be met. As a result, the trained neural networks cannot be guaranteed to generate at least locally optimal trajectories. To address this issue, a parametrization approach is developed in the paper so that not only necessary but also sufficient conditions are embedded into a parameterized set of differential equations. This allows generating the data set of at least locally optimal trajectories through solving some initial value problems. Once a neural network is trained by the data set, it can not only generate optimal trajectories within milliseconds but also ensure the generated trajectories to be at least locally optimal, as finally demonstrated by two conventional MECPs in aerospace engineering.

math.OC↗
arXiv2023

Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL

Offline reinforcement learning (RL) offers an appealing approach to real-world tasks by learning policies from pre-collected datasets without interacting with the environment. However, the performance of existing offline RL algorithms heavily depends on the scale and state-action space coverage of datasets. Real-world data collection is often expensive and uncontrollable, leading to small and narrowly covered datasets and posing significant challenges for practical deployments of offline RL. In this paper, we provide a new insight that leveraging the fundamental symmetry of system dynamics can substantially enhance offline RL performance under small datasets. Specifically, we propose a Time-reversal symmetry (T-symmetry) enforced Dynamics Model (TDM), which establishes consistency between a pair of forward and reverse latent dynamics. TDM provides both well-behaved representations for small datasets and a new reliability measure for OOD samples based on compliance with the T-symmetry. These can be readily used to construct a new offline RL algorithm (TSRL) with less conservative policy constraints and a reliable latent space data augmentation procedure. Based on extensive experiments, we find TSRL achieves great performance on small benchmark datasets with as few as 1% of the original samples, which significantly outperforms the recent offline RL algorithms in terms of data efficiency and generalizability.Code is available at: https://github.com/pcheng2/TSRL

cs.LG↗
arXiv2023

Scattering of swell by currents

The refraction of surface gravity waves by currents leads to spatial modulations in the wave field and, in particular, in the significant wave height. We examine this phenomenon in the case of waves scattered by a localised current feature, assuming (i) the smallness of the ratio between current velocity and wave group speed, and (ii) a swell-like, highly directional wave spectrum. We apply matched asymptotics to the equation governing the conservation of wave action in the four-dimensional position--wavenumber space. The resulting explicit formulas show that the modulations in wave action and significant wave height past the localised current are controlled by the vorticity of the current integrated along the primary direction of the swell. We assess the asymptotic predictions against numerical simulations using WAVEWATCH III for a Gaussian vortex. We also consider vortex dipoles to demonstrate the possibility of `vortex cloaking' whereby certain currents have (asymptotically) no impact on the significant wave height. We discuss the role of the ratio of the two small parameters characterising assumptions (i) and (ii) above and show that caustics are only significant for unrealistically large values of this ratio, corresponding to unrealistically narrow directional spectra.

physics.flu-dyn↗
arXiv2023

Full-scale ab initio simulations of laser-driven atomistic dynamics

The coupling of excited states and ionic dynamics is the basic and challenging point for the materials response at extreme conditions. In laboratory, the intense laser produces transient nature and complexity with highly nonequilibrium states, making it extremely difficult and interesting for both experimental measurements and theoretical methods. With the inclusion of laser-excited states, we extended ab initio method into the direct simulations of whole laser-driven microscopic dynamics from solid to liquid. We constructed the framework of combining the electron-temperaturedependent deep neural network potential energy surface with hybrid atomistic-continuum approach, controlling non-adiabatic energy exchange and atomistic dynamics, which enables consistent interpretation of experimental data. By large scale ab inito simulations, we demonstrate that the nonthermal effects introduced by hot electrons play a dominant role in modulating the lattice dynamics, thermodynamic pathway, and structural transformation. We highlight that the present work provides a path to realistic computational studies of laser-driven processes, thus bridging the gap between experiments and simulations.

physics.comp-ph↗
arXiv2023

TLM: Token-Level Masking for Transformers

Structured dropout approaches, such as attention dropout and DropHead, have been investigated to regularize the multi-head attention mechanism in Transformers. In this paper, we propose a new regularization scheme based on token-level rather than structure-level to reduce overfitting. Specifically, we devise a novel Token-Level Masking (TLM) training strategy for Transformers to regularize the connections of self-attention, which consists of two masking techniques that are effective and easy to implement. The underlying idea is to manipulate the connections between tokens in the multi-head attention via masking, where the networks are forced to exploit partial neighbors' information to produce a meaningful representation. The generality and effectiveness of TLM are thoroughly evaluated via extensive experiments on 4 diversified NLP tasks across 18 datasets, including natural language understanding benchmark GLUE, ChineseGLUE, Chinese Grammatical Error Correction, and data-to-text generation. The results indicate that TLM can consistently outperform attention dropout and DropHead, e.g., it increases by 0.5 points relative to DropHead with BERT-large on GLUE. Moreover, TLM can establish a new record on the data-to-text benchmark Rotowire (18.93 BLEU). Our code will be publicly available at https://github.com/Young1993/tlm.

cs.CL↗