arXiv subjects

Kun Wang

Publications and source records attributed to Kun Wang.

At least 307 records · Page 17Linked to original sources

arXiv2024

A Physics-Informed Indirect Method for Trajectory Optimization

This work presents a Physics-Informed Indirect Method (PIIM) that propagates the dynamics of both states and co-states backward in time for trajectory optimization problems. In the case of a Time-Optimal Soft Landing Problem (TOSLP), based on the initial co-state vector normalization technique, we show that the initial guess of the mass co-state and the numerical factor can be eliminated from the shooting procedure. As a result, the initial guess of the unknown co-states can be constrained to lie on a unit 3-D hypersphere. Then, using the PIIM allows one to exploit the physical significance of the optimal control law, which further narrows down the solution space to a unit 3-D octant sphere. Meanwhile, the analytical estimations of the fuel consumption and final time are provided. Additionally, a usually overlooked issue that results in an infeasible solution with a negative final time, is fixed by a simple remedy strategy. Consequently, the reduced solution space becomes sufficiently small to ensure fast, robust, and guaranteed convergence for the TOSLP. Then, we extend the PIIM to solve the Fuel-Optimal Soft Landing Problem (FOSLP) with a homotopy approach. The numerical simulations show that compared with the conventional indirect method with a success rate of 89.35%, it takes a shorter time for the proposed method to find the feasible solution to the FOSLP with a success rate of 100%.

math.OC↗

arXiv2024

A Comprehensive Survey on EEG-Based Emotion Recognition: A Graph-Based Perspective

Compared to other modalities, electroencephalogram (EEG) based emotion recognition can intuitively respond to emotional patterns in the human brain and, therefore, has become one of the most focused tasks in affective computing. The nature of emotions is a physiological and psychological state change in response to brain region connectivity, making emotion recognition focus more on the dependency between brain regions instead of specific brain regions. A significant trend is the application of graphs to encapsulate such dependency as dynamic functional connections between nodes across temporal and spatial dimensions. Concurrently, the neuroscientific underpinnings behind this dependency endow the application of graphs in this field with a distinctive significance. However, there is neither a comprehensive review nor a tutorial for constructing emotion-relevant graphs in EEG-based emotion recognition. In this paper, we present a comprehensive survey of these studies, delivering a systematic review of graph-related methods in this field from a methodological perspective. We propose a unified framework for graph applications in this field and categorize these methods on this basis. Finally, based on previous studies, we also present several open challenges and future directions in this field.

eess.SP↗

arXiv2024

$μ$Drive: User-Controlled Autonomous Driving

Autonomous Vehicles (AVs) rely on sophisticated Autonomous Driving Systems (ADSs) to provide passengers a satisfying and safe journey. The individual preferences of riders plays a crucial role in shaping the perception of safety and comfort while they are in the car. Existing ADSs, however, lack mechanisms to systematically capture and integrate rider preferences into their planning modules. To bridge this gap, we propose $μ$Drive, an event-based Domain-Specific Language (DSL) designed for specifying autonomous vehicle behaviour. $μ$Drive enables users to express their preferences through rules triggered by contextual events, such as encountering obstacles or navigating complex traffic situations. These rules dynamically adjust the parameter settings of the ADS planning module, facilitating seamless integration of rider preferences into the driving plan. In our evaluation, we demonstrate the feasibility and efficacy of $μ$Drive by integrating it with the Apollo ADS framework. Our findings show that users can effectively influence Apollo's planning through $μ$Drive, assisting ADS in achieving improved compliance with traffic regulations. The response time for $μ$Drive commands remains consistently at the second or millisecond level. This suggests that $μ$Drive may help pave the way to more personalizsed and user-centric AV experiences.

cs.SE↗

arXiv2024

Investigation of discontinuous Galerkin methods in adjoint gradient-based aerodynamic shape optimization

This work develops a robust and efficient framework of the adjoint gradient-based aerodynamic shape optimization (ASO) using high-order discontinuous Galerkin methods (DGMs) as the CFD solver. The adjoint-enabled gradients based on different CFD solvers or solution representations are derived in detail, and the potential advantage of DG representations is discovered that the adjoint gradient computed by the DGMs contains a modification term which implies information of higher-order moments of the solution as compared with finite volume methods (FVMs). A number of numerical cases are tested for investigating the impact of different CFD solvers (including DGMs and FVMs) on the evaluation of the adjoint-enabled gradients. The numerical results demonstrate that the DGMs can provide more precise adjoint gradients even on a coarse mesh as compared with the FVMs under coequal computational costs, and extend the capability to explore the design space, further leading to acquiring the aerodynamic shapes with more superior aerodynamic performance.

math.NA↗

arXiv2024

Data is all you need: Finetuning LLMs for Chip Design via an Automated design-data augmentation framework

Recent advances in large language models have demonstrated their potential for automated generation of hardware description language (HDL) code from high-level prompts. Researchers have utilized fine-tuning to enhance the ability of these large language models (LLMs) in the field of Chip Design. However, the lack of Verilog data hinders further improvement in the quality of Verilog generation by LLMs. Additionally, the absence of a Verilog and Electronic Design Automation (EDA) script data augmentation framework significantly increases the time required to prepare the training dataset for LLM trainers. This paper proposes an automated design-data augmentation framework, which generates high-volume and high-quality natural language aligned with Verilog and EDA scripts. For Verilog generation, it translates Verilog files to an abstract syntax tree and then maps nodes to natural language with a predefined template. For Verilog repair, it uses predefined rules to generate the wrong verilog file and then pairs EDA Tool feedback with the right and wrong verilog file. For EDA Script generation, it uses existing LLM(GPT-3.5) to obtain the description of the Script. To evaluate the effectiveness of our data augmentation method, we finetune Llama2-13B and Llama2-7B models using the dataset generated by our augmentation framework. The results demonstrate a significant improvement in the Verilog generation tasks with LLMs. Moreover, the accuracy of Verilog generation surpasses that of the current state-of-the-art open-source Verilog generation model, increasing from 58.8% to 70.6% with the same benchmark. Our 13B model (ChipGPT-FT) has a pass rate improvement compared with GPT-3.5 in Verilog generation and outperforms in EDA script (i.e., SiliconCompiler) generation with only 200 EDA script data.

cs.AR↗

arXiv2024

CLEAR: Can Language Models Really Understand Causal Graphs?

Causal reasoning is a cornerstone of how humans interpret the world. To model and reason about causality, causal graphs offer a concise yet effective solution. Given the impressive advancements in language models, a crucial question arises: can they really understand causal graphs? To this end, we pioneer an investigation into language models' understanding of causal graphs. Specifically, we develop a framework to define causal graph understanding, by assessing language models' behaviors through four practical criteria derived from diverse disciplines (e.g., philosophy and psychology). We then develop CLEAR, a novel benchmark that defines three complexity levels and encompasses 20 causal graph-based tasks across these levels. Finally, based on our framework and benchmark, we conduct extensive experiments on six leading language models and summarize five empirical findings. Our results indicate that while language models demonstrate a preliminary understanding of causal graphs, significant potential for improvement remains. Our project website is at https://github.com/OpenCausaLab/CLEAR.

cs.CL↗

arXiv2024

Investigating higgsino dark matter in the semi-constrained NMSSM

In this study, we explore the characteristics of higgsino-dominated dark matter (DM) within the semi-constrained Next-to-Minimal Supersymmetric Standard Model (scNMSSM), covering a mass range from hundreds of GeV to several TeV. We carefully analyzed the parameter space under existing theoretical and experimental constraints to confirm the viability of higgsino-dominated lightest supersymmetric particles (LSPs) with masses between 100 GeV and 4 TeV. Our study examines various DM annihilation mechanisms, emphasizing the significant role of coannihilation with the next-to-lightest supersymmetric particle (NLSP), which includes other higgsino-dominated particles such as $\tildeχ^{0}_2$ and $\tildeχ^{\pm}_1$. We categorize the annihilation processes into three main classes: $\tildeχ_1^{\pm}$ coannihilation, Higgs funnel annihilation, and $\tildeτ_1$ coannihilation, each combines interactions with $\tildeχ_1^{\pm}$. Our results indicate that achieving the correct relic density in heavier higgsino LSPs requires a combination of coannihilation and Higgs funnel mechanisms. We also assess the potential of future experiments, such as XENONnT, LUX-ZEPLIN (LZ), PandaX-xT, and the Cherenkov Telescope Array (CTA), to probe these DM scenarios through direct and indirect detection. In particular, future spin-independent DM detection can cover all samples with the correct DM relic density for $μ\gtrsim 1300$ GeV. Furthermore, future colliders like the International Linear Collider (ILC) and the Compact Linear Collider (CLIC) are found to exceed the detection capabilities of current hadron colliders, especially for higher mass NLSPs. Notably, CLIC at 3000 GeV is anticipated to thoroughly investigate all samples with insufficient DM relic density for $μ\lesssim 1300$ GeV.

hep-ph↗

arXiv2024

The Heterophilic Snowflake Hypothesis: Training and Empowering GNNs for Heterophilic Graphs

Graph Neural Networks (GNNs) have become pivotal tools for a range of graph-based learning tasks. Notably, most current GNN architectures operate under the assumption of homophily, whether explicitly or implicitly. While this underlying assumption is frequently adopted, it is not universally applicable, which can result in potential shortcomings in learning effectiveness. In this paper, \textbf{for the first time}, we transfer the prevailing concept of ``one node one receptive field" to the heterophilic graph. By constructing a proxy label predictor, we enable each node to possess a latent prediction distribution, which assists connected nodes in determining whether they should aggregate their associated neighbors. Ultimately, every node can have its own unique aggregation hop and pattern, much like each snowflake is unique and possesses its own characteristics. Based on observations, we innovatively introduce the Heterophily Snowflake Hypothesis and provide an effective solution to guide and facilitate research on heterophilic graphs and beyond. We conduct comprehensive experiments including (1) main results on 10 graphs with varying heterophily ratios across 10 backbones; (2) scalability on various deep GNN backbones (SGC, JKNet, etc.) across various large number of layers (2,4,6,8,16,32 layers); (3) comparison with conventional snowflake hypothesis; (4) efficiency comparison with existing graph pruning algorithms. Our observations show that our framework acts as a versatile operator for diverse tasks. It can be integrated into various GNN frameworks, boosting performance in-depth and offering an explainable approach to choosing the optimal network depth. The source code is available at \url{https://github.com/bingreeky/HeteroSnoH}.

cs.LG↗

arXiv2024

MolTC: Towards Molecular Relational Modeling In Language Models

Molecular Relational Learning (MRL), aiming to understand interactions between molecular pairs, plays a pivotal role in advancing biochemical research. Recently, the adoption of large language models (LLMs), known for their vast knowledge repositories and advanced logical inference capabilities, has emerged as a promising way for efficient and effective MRL. Despite their potential, these methods predominantly rely on the textual data, thus not fully harnessing the wealth of structural information inherent in molecular graphs. Moreover, the absence of a unified framework exacerbates the issue of information underutilization, as it hinders the sharing of interaction mechanism learned across diverse datasets. To address these challenges, this work proposes a novel LLM-based multi-modal framework for Molecular inTeraction prediction following Chain-of-Thought (CoT) theory, termed MolTC, which effectively integrate graphical information of two molecules in pair. To train MolTC efficiently, we introduce a Multi-hierarchical CoT concept to refine its training paradigm, and conduct a comprehensive Molecular Interactive Instructions dataset for the development of biochemical LLMs involving MRL. Our experiments, conducted across various datasets involving over 4,000,000 molecular pairs, exhibit the superiority of our method over current GNN and LLM-based baselines. Code is available at https://github.com/MangoKiller/MolTC.

q-bio.QM↗

arXiv2024

Nonlinear Optimal Guidance with Constraints on Impact Time and Impact Angle

This paper aims to address the nonlinear optimal guidance problem with impact-time and impact-angle constraints, which is fundamentally important for multiple pursuers to collaboratively achieve a target. Addressing such a guidance problem is equivalent to solving a nonlinear minimum-effort control problem in real time. To this end, the Pontryagain's maximum principle is employed to convert extremal trajectories as the solutions of a parameterized differential system. The geometric property for the solution of the parameterized system is analyzed, leading to an additional optimality condition. By incorporating this optimality condition and the usual disconjugacy condition into the parameterized system, the dataset for optimal trajectories can be generated by propagating the parameterized system without using any optimization methods. In addition, a scaling invariance property is found for the solutions of the parameterized system. As a consequence of this scaling invariance property, a simple feedforward neural network trained by the solution of the parameterized system, selected at any fixed time, can be used to generate the nonlinear optimal guidance within milliseconds. Finally, numerical examples are presented, showing that the nonlinear optimal guidance command generated by the trained network can not only ensure the expected impact angle and impact time are precisely met but also requires less control effort compared with existing guidance methods.

math.OC↗

arXiv2024

95 GeV light Higgs in the top-pair-associated diphoton channel at the LHC in the minimal dilaton model

Motivated by experimental hints and theoretical frameworks indicating the existence of an extended Higgs sector, we explore the feasibility of detecting a 95 GeV light Higgs boson decaying into a diphoton within the minimal dilaton model at the 14 TeV LHC. Initially, we identify the correlations between the production cross section, decay branching ratios, and model parameters, e.g., the scalar mixing angle $\sinθ_S$. Subsequently, we utilize Monte Carlo simulations to generate the signal of the light Higgs boson via the $pp \to t\bar{t}(s\to γγ)$ process, along with the corresponding backgrounds. To effectively separate the signal from the dominant backgrounds $ttγγ$, we employ a meticulous cut-based selection process. Ultimately, we find that with an integrated luminosity of $L = 3000 {{~\rm fb}^{-1}}$, the regions of $|\sinθ_S|>0.2$ can be covered over the $3σ$ level.

hep-ph↗

arXiv2024

Buffered Asynchronous Secure Aggregation for Cross-Device Federated Learning

Asynchronous federated learning (AFL) is an effective method to address the challenge of device heterogeneity in cross-device federated learning. However, AFL is usually incompatible with existing secure aggregation protocols used to protect user privacy in federated learning because most existing secure aggregation protocols are based on synchronous aggregation. To address this problem, we propose a novel secure aggregation protocol named buffered asynchronous secure aggregation (BASA) in this paper. Compared with existing protocols, BASA is fully compatible with AFL and provides secure aggregation under the condition that each user only needs one round of communication with the server without relying on any synchronous interaction among users. Based on BASA, we propose the first AFL method which achieves secure aggregation without extra requirements on hardware. We empirically demonstrate that BASA outperforms existing secure aggregation protocols for cross-device federated learning in terms of training efficiency and scalability.

cs.CR↗

arXiv2024

FineDiffusion: Scaling up Diffusion Models for Fine-grained Image Generation with 10,000 Classes

The class-conditional image generation based on diffusion models is renowned for generating high-quality and diverse images. However, most prior efforts focus on generating images for general categories, e.g., 1000 classes in ImageNet-1k. A more challenging task, large-scale fine-grained image generation, remains the boundary to explore. In this work, we present a parameter-efficient strategy, called FineDiffusion, to fine-tune large pre-trained diffusion models scaling to large-scale fine-grained image generation with 10,000 categories. FineDiffusion significantly accelerates training and reduces storage overhead by only fine-tuning tiered class embedder, bias terms, and normalization layers' parameters. To further improve the image generation quality of fine-grained categories, we propose a novel sampling method for fine-grained image generation, which utilizes superclass-conditioned guidance, specifically tailored for fine-grained categories, to replace the conventional classifier-free guidance sampling. Compared to full fine-tuning, FineDiffusion achieves a remarkable 1.56x training speed-up and requires storing merely 1.77% of the total model parameters, while achieving state-of-the-art FID of 9.776 on image generation of 10,000 classes. Extensive qualitative and quantitative experiments demonstrate the superiority of our method compared to other parameter-efficient fine-tuning methods. The code and more generated results are available at our project website: https://finediffusion.github.io/.

cs.CV↗

arXiv2024

Earthfarseer: Versatile Spatio-Temporal Dynamical Systems Modeling in One Model

Efficiently modeling spatio-temporal (ST) physical processes and observations presents a challenging problem for the deep learning community. Many recent studies have concentrated on meticulously reconciling various advantages, leading to designed models that are neither simple nor practical. To address this issue, this paper presents a systematic study on existing shortcomings faced by off-the-shelf models, including lack of local fidelity, poor prediction performance over long time-steps,low scalability, and inefficiency. To systematically address the aforementioned problems, we propose an EarthFarseer, a concise framework that combines parallel local convolutions and global Fourier-based transformer architectures, enabling dynamically capture the local-global spatial interactions and dependencies. EarthFarseer also incorporates a multi-scale fully convolutional and Fourier architectures to efficiently and effectively capture the temporal evolution. Our proposal demonstrates strong adaptability across various tasks and datasets, with fast convergence and better local fidelity in long time-steps predictions. Extensive experiments and visualizations over eight human society physical and natural physical datasets demonstrates the state-of-the-art performance of EarthFarseer. We release our code at https://github.com/easylearningscores/EarthFarseer.

cs.AI↗

arXiv2024

Predicting Parking Availability in Singapore with Cross-Domain Data: A New Dataset and A Data-Driven Approach

The increasing number of vehicles highlights the need for efficient parking space management. Predicting real-time Parking Availability (PA) can help mitigate traffic congestion and the corresponding social problems, which is a pressing issue in densely populated cities like Singapore. In this study, we aim to collectively predict future PA across Singapore with complex factors from various domains. The contributions in this paper are listed as follows: (1) A New Dataset: We introduce the \texttt{SINPA} dataset, containing a year's worth of PA data from 1,687 parking lots in Singapore, enriched with various spatial and temporal factors. (2) A Data-Driven Approach: We present DeepPA, a novel deep-learning framework, to collectively and efficiently predict future PA across thousands of parking lots. (3) Extensive Experiments and Deployment: DeepPA demonstrates a 9.2% reduction in prediction error for up to 3-hour forecasts compared to existing advanced models. Furthermore, we implement DeepPA in a practical web-based platform to provide real-time PA predictions to aid drivers and inform urban planning for the governors in Singapore. We release the dataset and source code at https://github.com/yoshall/SINPA.

cs.AI↗

arXiv2024

ForecastGrapher: Redefining Multivariate Time Series Forecasting with Graph Neural Networks

The challenge of effectively learning inter-series correlations for multivariate time series forecasting remains a substantial and unresolved problem. Traditional deep learning models, which are largely dependent on the Transformer paradigm for modeling long sequences, often fail to integrate information from multiple time series into a coherent and universally applicable model. To bridge this gap, our paper presents ForecastGrapher, a framework reconceptualizes multivariate time series forecasting as a node regression task, providing a unique avenue for capturing the intricate temporal dynamics and inter-series correlations. Our approach is underpinned by three pivotal steps: firstly, generating custom node embeddings to reflect the temporal variations within each series; secondly, constructing an adaptive adjacency matrix to encode the inter-series correlations; and thirdly, augmenting the GNNs' expressive power by diversifying the node feature distribution. To enhance this expressive power, we introduce the Group Feature Convolution GNN (GFC-GNN). This model employs a learnable scaler to segment node features into multiple groups and applies one-dimensional convolutions with different kernel lengths to each group prior to the aggregation phase. Consequently, the GFC-GNN method enriches the diversity of node feature distribution in a fully end-to-end fashion. Through extensive experiments and ablation studies, we show that ForecastGrapher surpasses strong baselines and leading published techniques in the domain of multivariate time series forecasting.

cs.LG↗

arXiv2024

A Neighbor-Searching Discrepancy-based Drift Detection Scheme for Learning Evolving Data

Uncertain changes in data streams present challenges for machine learning models to dynamically adapt and uphold performance in real-time. Particularly, classification boundary change, also known as real concept drift, is the major cause of classification performance deterioration. However, accurately detecting real concept drift remains challenging because the theoretical foundations of existing drift detection methods - two-sample distribution tests and monitoring classification error rate, both suffer from inherent limitations such as the inability to distinguish virtual drift (changes not affecting the classification boundary, will introduce unnecessary model maintenance), limited statistical power, or high computational cost. Furthermore, no existing detection method can provide information on the trend of the drift, which could be invaluable for model maintenance. This work presents a novel real concept drift detection method based on Neighbor-Searching Discrepancy, a new statistic that measures the classification boundary difference between two samples. The proposed method is able to detect real concept drift with high accuracy while ignoring virtual drift. It can also indicate the direction of the classification boundary change by identifying the invasion or retreat of a certain class, which is also an indicator of separability change between classes. A comprehensive evaluation of 11 experiments is conducted, including empirical verification of the proposed theory using artificial datasets, and experimental comparisons with commonly used drift handling methods on real-world datasets. The results show that the proposed theory is robust against a range of distributions and dimensions, and the drift detection method outperforms state-of-the-art alternative methods.

cs.LG↗

arXiv2024

Retrieving non-linear features from noisy quantum states

Accurately estimating high-order moments of quantum states is an elementary precondition for many crucial tasks in quantum computing, such as entanglement spectroscopy, entropy estimation, spectrum estimation, and predicting non-linear features from quantum states. But in reality, inevitable quantum noise prevents us from accessing the desired value. In this paper, we address this issue by systematically analyzing the feasibility and efficiency of extracting high-order moments from noisy states. We first show that there exists a quantum protocol capable of accomplishing this task if and only if the underlying noise channel is invertible. We then establish a method for deriving protocols that attain optimal sample complexity using quantum operations and classical post-processing only. Our protocols, in contrast to conventional ones, incur lower overheads and avoid sampling different quantum operations due to a novel technique called observable shift, making the protocols strong candidates for practical usage on current quantum devices. The proposed method also indicates the power of entangled protocols in retrieving high-order information, whereas in the existing methods, entanglement does not help. We further construct the protocol for large quantum systems to retrieve the depolarizing channels, making the proposed method scalable. Our work contributes to a deeper understanding of how quantum noise could affect high-order information extraction and provides guidance on how to tackle it.

quant-ph↗