arXiv subjects

Di Wu

Publications and source records attributed to Di Wu.

At least 397 records · Page 22Linked to original sources

arXiv2024

Meta-Task Prompting Elicits Embeddings from Large Language Models

We introduce a new unsupervised text embedding method, Meta-Task Prompting with Explicit One-Word Limitation (MetaEOL), for generating high-quality sentence embeddings from Large Language Models (LLMs) without the need for model fine-tuning. Leveraging meta-task prompting, MetaEOL guides LLMs to produce embeddings through a series of carefully designed prompts that address multiple representational aspects. Our comprehensive experiments demonstrate that embeddings averaged from various meta-tasks are versatile embeddings that yield competitive performance on Semantic Textual Similarity (STS) benchmarks and excel in downstream tasks, surpassing contrastive-trained models. Our findings suggest a new scaling law, offering a versatile and resource-efficient approach for embedding generation across diverse scenarios.

cs.CL↗

arXiv2024

Kerr-MOG-(A)dS black hole and its shadow in scalar-tensor-vector gravity theory

The scalar-tensor-vector gravity (STVG) theory has attracted significant interest due to its ability to effectively address the issue of galaxy rotation curves and clusters of galaxies without considering the influence of dark matter. In this paper, we construct rotating black hole solutions with a cosmological constant in the STVG theory (i.e., Kerr-MOG-(A)dS black hole solutions), where the import of a gravitational charge as a source modifies the gravitational constant, determined by $ G=G_{\text{N}}(1+α) $. For Kerr-MOG-dS spacetime, the observer is situated at a specific location within the domain of outer communication, rather than being located infinitely far away. Since black hole shadows are shaped by light propagation in spacetime, the interaction between the MOG parameter and the cosmological constant is expected to produce novel effects on these shadows. As the cosmological constant $Λ$ increases, the apparent size of the black hole shadow decreases. Additionally, the shadow expands with an increase in the MOG parameter $α$, reaching a maximum at a certain value, and its shape becomes more rounded under an arbitrary rotation parameter, which leads to degeneracy between different black hole parameters. However, by employing numerical ray-tracing techniques, we have found that gravitational lensing and the frame-dragging effect effectively distinguish this degeneracy. Our work contributes to a deeper understanding of black holes in modified gravity, their observational signatures, and constraints.

gr-qc↗

arXiv2024

Shadow of slowly rotating Kalb-Ramond black holes

Real astronomical objects possess spin, yet deriving exact solutions for rotating black holes within gravitational theories is a formidable challenge. To understand the shadow of rotating black holes in Lorentz-violating spacetimes induced by antisymmetric tensor fields, known as Kalb-Ramond (KR) fields, we have focused on the slow-rotation approximation framework. Using this approach, we have obtained first-order rotation series solutions, which describe slowly rotating KR black holes. For this solutions, we have plotted the black hole shadow contours under various parameters using the numerical backward ray-tracing method. As the Lorentz-violating parameter increases, not only the apparent size of the black hole shadow decreases, but also the effects of rotation, such as the D-shaped structure and frame-dragging, are amplified. Furthermore, the KR field also enhances gravitational lensing, causing the shadow to occupy a larger area within the photon ring. This distinctive feature can differentiate KR gravity from general relativity. Additionally, using the latest observational data from EHT on M87* and Sgr A*, we have provided constraints on the Lorentz-violating parameter of rotating KR black holes. We found that, compared to static black holes, rotating black holes allow for the presence of stronger Lorentz violation effects.

gr-qc↗

arXiv2024

Interior transit orbits in the planar bicircular restricted four-body problem: classification and application

Low-energy transfers are advantageous for lunar exploration missions due to low fuel consumption and extended launch periods. This paper is devoted to the classification of interior transit orbits and their application on low-energy transfer in the Sun-Earth/Moon planar bicircular restricted four-body problem (PBCR4BP). First, the Lagrangian coherent structures (LCSs) are introduced to generate the interior transit orbits. The number of periapses about the Moon is selected as the classification parameter and mapped into the LCSs, achieving clear classification boundaries. Then, the evolution laws of the classifications with respect to energy and solar gravity perturbation are discussed and summarized. Construction strategies for low-energy transfer are proposed based on the classifications and their evolution laws. Numerical simulation of the transfer trajectories verifies the effectiveness of the proposed strategies. The dynamical behaviors and transfer characteristics of transit orbits and their families are revealed, and a direct link between transit orbit families and low-energy transfers is finally established.

astro-ph.EP↗

arXiv2024

Neuro-BERT: Rethinking Masked Autoencoding for Self-supervised Neurological Pretraining

Deep learning associated with neurological signals is poised to drive major advancements in diverse fields such as medical diagnostics, neurorehabilitation, and brain-computer interfaces. The challenge in harnessing the full potential of these signals lies in the dependency on extensive, high-quality annotated data, which is often scarce and expensive to acquire, requiring specialized infrastructure and domain expertise. To address the appetite for data in deep learning, we present Neuro-BERT, a self-supervised pre-training framework of neurological signals based on masked autoencoding in the Fourier domain. The intuition behind our approach is simple: frequency and phase distribution of neurological signals can reveal intricate neurological activities. We propose a novel pre-training task dubbed Fourier Inversion Prediction (FIP), which randomly masks out a portion of the input signal and then predicts the missing information using the Fourier inversion theorem. Pre-trained models can be potentially used for various downstream tasks such as sleep stage classification and gesture recognition. Unlike contrastive-based methods, which strongly rely on carefully hand-crafted augmentations and siamese structure, our approach works reasonably well with a simple transformer encoder with no augmentation requirements. By evaluating our method on several benchmark datasets, we show that Neuro-BERT improves downstream neurological-related tasks by a large margin.

eess.SP↗

arXiv2024

A robust three-way classifier with shadowed granular-balls based on justifiable granularity

The granular-ball (GB)-based classifier introduced by Xia, exhibits adaptability in creating coarse-grained information granules for input, thereby enhancing its generality and flexibility. Nevertheless, the current GB-based classifiers rigidly assign a specific class label to each data instance and lacks of the necessary strategies to address uncertain instances. These far-fetched certain classification approachs toward uncertain instances may suffer considerable risks. To solve this problem, we construct a robust three-way classifier with shadowed GBs for uncertain data. Firstly, combine with information entropy, we propose an enhanced GB generation method with the principle of justifiable granularity. Subsequently, based on minimum uncertainty, a shadowed mapping is utilized to partition a GB into Core region, Important region and Unessential region. Based on the constructed shadowed GBs, we establish a three-way classifier to categorize data instances into certain classes and uncertain case. Finally, extensive comparative experiments are conducted with 2 three-way classifiers, 3 state-of-the-art GB-based classifiers, and 3 classical machine learning classifiers on 12 public benchmark datasets. The results show that our model demonstrates robustness in managing uncertain data and effectively mitigates classification risks. Furthermore, our model almost outperforms the other comparison methods in both effectiveness and efficiency.

cs.LG↗

arXiv2024

Topological classes of thermodynamics of the static multi-charge AdS black holes in gauged supergravities: novel temperature-dependent thermodynamic topological phase transition

In this paper, we investigate, in the framework of the topological approach to black hole thermodynamics, using the generalized off-shell Helmholtz free energy, the topological numbers of the static multi-charge AdS black holes in four- and five-dimensional gauged supergravities. We find that the topological number of the static-charged AdS black holes in four-dimensional Kaluza-Klein (K-K) gauged supergravity theory is $W = 0$, while that of the static-charged AdS black holes in four-dimensional gauged $-iX^0X^1$-supergravity and STU gauged supergravity theories, and five-dimensional Einstein-Maxwell-dilaton-axion (EMDA) gauged supergravity and STU gauged supergravity, and five-dimensional static-charged AdS Horowitz-Sen black hole are both $W = 1$. Furthermore, we observe a novel temperature-dependent thermodynamic topological phase transition that can happen in the four-dimensional static-charged AdS black hole in EMDA gauged supergravity theory, the four-dimensional static-charged AdS Horowitz-Sen black hole, and the five-dimensional static-charged AdS black hole in K-K gauged supergravity theory. We believe that the novel temperature-dependent thermodynamic topological phase transition could help us better understand black hole thermodynamics and, further, shed new light on the fundamental nature of gauged supergravity theories.

hep-th↗

arXiv2024

FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learning

In this work, we investigate how to leverage pre-trained visual-language models (VLM) for online Reinforcement Learning (RL). In particular, we focus on sparse reward tasks with pre-defined textual task descriptions. We first identify the problem of reward misalignment when applying VLM as a reward in RL tasks. To address this issue, we introduce a lightweight fine-tuning method, named Fuzzy VLM reward-aided RL (FuRL), based on reward alignment and relay RL. Specifically, we enhance the performance of SAC/DrQ baseline agents on sparse reward tasks by fine-tuning VLM representations and using relay RL to avoid local minima. Extensive experiments on the Meta-world benchmark tasks demonstrate the efficacy of the proposed method. Code is available at: https://github.com/fuyw/FuRL.

cs.LG↗

arXiv2024

KPEval: Towards Fine-Grained Semantic-Based Keyphrase Evaluation

Despite the significant advancements in keyphrase extraction and keyphrase generation methods, the predominant approach for evaluation mainly relies on exact matching with human references. This scheme fails to recognize systems that generate keyphrases semantically equivalent to the references or diverse keyphrases that carry practical utility. To better assess the capability of keyphrase systems, we propose KPEval, a comprehensive evaluation framework consisting of four critical aspects: reference agreement, faithfulness, diversity, and utility. For each aspect, we design semantic-based metrics to reflect the evaluation objectives. Meta-evaluation studies demonstrate that our evaluation strategy correlates better with human preferences compared to a range of previously proposed metrics. Using KPEval, we re-evaluate 23 keyphrase systems and discover that (1) established model comparison results have blind-spots especially when considering reference-free evaluation; (2) large language models are underestimated by prior evaluation works; and (3) there is no single best model that can excel in all the aspects.

cs.CL↗

arXiv2024

Repoformer: Selective Retrieval for Repository-Level Code Completion

Recent advances in retrieval-augmented generation (RAG) have initiated a new era in repository-level code completion. However, the invariable use of retrieval in existing methods exposes issues in both efficiency and robustness, with a large proportion of the retrieved contexts proving unhelpful or harmful to code language models (code LMs). In this paper, we propose a selective RAG framework to avoid retrieval when unnecessary. To power this framework, we design a self-supervised learning approach to enable a code LM to accurately self-evaluate whether retrieval can improve its output quality and robustly leverage the potentially noisy retrieved contexts. Using this LM as both the selective RAG policy and the generation model, our framework achieves state-of-the-art repository-level code completion performance on diverse benchmarks including RepoEval, CrossCodeEval, and CrossCodeLongEval, a new long-form code completion benchmark. Meanwhile, our analyses show that selectively retrieving brings as much as 70% inference speedup in the online serving setting without harming the performance. We further demonstrate that our framework is able to accommodate different generation models, retrievers, and programming languages. These advancements position our framework as an important step towards more accurate and efficient repository-level code completion.

cs.SE↗

arXiv2024

VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence Modeling

Similar to natural language models, pre-trained genome language models are proposed to capture the underlying intricacies within genomes with unsupervised sequence modeling. They have become essential tools for researchers and practitioners in biology. However, the hand-crafted tokenization policies used in these models may not encode the most discriminative patterns from the limited vocabulary of genomic data. In this paper, we introduce VQDNA, a general-purpose framework that renovates genome tokenization from the perspective of genome vocabulary learning. By leveraging vector-quantized codebooks as learnable vocabulary, VQDNA can adaptively tokenize genomes into pattern-aware embeddings in an end-to-end manner. To further push its limits, we propose Hierarchical Residual Quantization (HRQ), where varying scales of codebooks are designed in a hierarchy to enrich the genome vocabulary in a coarse-to-fine manner. Extensive experiments on 32 genome datasets demonstrate VQDNA's superiority and favorable parameter efficiency compared to existing genome language models. Notably, empirical analysis of SARS-CoV-2 mutations reveals the fine-grained pattern awareness and biological significance of learned HRQ vocabulary, highlighting its untapped potential for broader applications in genomics.

q-bio.GN↗

arXiv2024

A Survey of Deep Learning Based Radar and Vision Fusion for 3D Object Detection in Autonomous Driving

With the rapid advancement of autonomous driving technology, there is a growing need for enhanced safety and efficiency in the automatic environmental perception of vehicles during their operation. In modern vehicle setups, cameras and mmWave radar (radar), being the most extensively employed sensors, demonstrate complementary characteristics, inherently rendering them conducive to fusion and facilitating the achievement of both robust performance and cost-effectiveness. This paper focuses on a comprehensive survey of radar-vision (RV) fusion based on deep learning methods for 3D object detection in autonomous driving. We offer a comprehensive overview of each RV fusion category, specifically those employing region of interest (ROI) fusion and end-to-end fusion strategies. As the most promising fusion strategy at present, we provide a deeper classification of end-to-end fusion methods, including those 3D bounding box prediction based and BEV based approaches. Moreover, aligning with recent advancements, we delineate the latest information on 4D radar and its cutting-edge applications in autonomous vehicles (AVs). Finally, we present the possible future trends of RV fusion and summarize this paper.

cs.CV↗

arXiv2024

Coloring some $(P_6,C_4)$-free graphs with $Δ-1$ colors

The Borodin-Kostochka Conjecture states that for a graph $G$, if $Δ(G)\geq9$, then $χ(G)\leq\max\{Δ(G)-1,ω(G)\}$. We use $P_t$ and $C_t$ to denote a path and a cycle on $t$ vertices, respectively. Let $C=v_1v_2v_3v_4v_5v_1$ be an induced $C_5$. A {\em $C_5^+$} is a graph obtained from $C$ by adding a $C_3=xyzx$ and a $P_2=t_1t_2$ such that (1) $x$ and $y$ are both exactly adjacent to $v_1,v_2,v_3$ in $V(C)$, $z$ is exactly adjacent to $v_2$ in $V(C)$, $t_1$ is exactly adjacent to $v_4,v_5$ in $V(C)$ and $t_2$ is exactly adjacent to $v_1,v_4,v_5$ in $V(C)$, (2) $t_1$ is exactly adjacent to $z$ in $\{x,y,z\}$ and $t_2$ has no neighbors in $\{x,y,z\}$. In this paper, we show that the Borodin-Kostochka Conjecture holds for ($P_6,C_4,H$)-free graphs, where $H\in \{K_7,C_5^+\}$. This generalizes some results of Gupta and Pradhan in \cite{GP21,GP24}.

math.CO↗

arXiv2024

CityGPT: Towards Urban IoT Learning, Analysis and Interaction with Multi-Agent System

The spatiotemporal data generated by massive sensors in the Internet of Things (IoT) is extremely dynamic, heterogeneous, large scale and time-dependent. It poses great challenges (e.g. accuracy, reliability, and stability) in real-time analysis and decision making for different IoT applications. The complexity of IoT data prevents the common people from gaining a deeper understanding of it. Agentized systems help address the lack of data insight for the common people. We propose a generic framework, namely CityGPT, to facilitate the learning and analysis of IoT time series with an end-to-end paradigm. CityGPT employs three agents to accomplish the spatiotemporal analysis of IoT data. The requirement agent facilitates user inputs based on natural language. Then, the analysis tasks are decomposed into temporal and spatial analysis processes, completed by corresponding data analysis agents (temporal and spatial agents). Finally, the spatiotemporal fusion agent visualizes the system's analysis results by receiving analysis results from data analysis agents and invoking sub-visualization agents, and can provide corresponding textual descriptions based on user demands. To increase the insight for common people using our framework, we have agnentized the framework, facilitated by a large language model (LLM), to increase the data comprehensibility. Our evaluation results on real-world data with different time dependencies show that the CityGPT framework can guarantee robust performance in IoT computing.

cs.AI↗

arXiv2024

Active learning of effective Hamiltonian for super-large-scale atomic structures

The first-principles-based effective Hamiltonian scheme provides one of the most accurate modeling technique for large-scale structures, especially for ferroelectrics. However, the parameterization of the effective Hamiltonian is complicated and can be difficult for some complex systems such as high-entropy perovskites. Here, we propose a general form of effective Hamiltonian and develop an active machine learning approach to parameterize the effective Hamiltonian based on Bayesian linear regression. The parameterization is employed in molecular dynamics simulations with the prediction of energy, forces, stress and their uncertainties at each step, which decides whether first-principles calculations are executed to retrain the parameters. Structures of BaTiO$_3$, Pb(Zr$_{0.75}$Ti$_{0.25}$)O$_3$ and (Pb,Sr)TiO$_3$ system are taken as examples to show the accuracy of this approach, as compared with conventional parametrization method and experiments. This machine learning approach provides a universal and automatic way to compute the effective Hamiltonian parameters for any considered complex systems with super-large-scale (more than $10^7$ atoms) atomic structures.

cond-mat.mtrl-sci↗

arXiv2024

Mack modes in supersonic boundary layer

Understanding the transition mechanism of boundary layer flows is of great significance in physics and engineering, especially due to the current development of supersonic and hypersonic aircraft. In this paper, we construct multiple unstable acoustic modes so-called Mack modes, which play a crucial role during the early stage of transition in the supersonic boundary layer. To this end, we develop an inner-outer gluing iteration to solve a hyperbolic-elliptic mixed type and singular system.

math.AP↗

arXiv2024

TrafficGPT: Towards Multi-Scale Traffic Analysis and Generation with Spatial-Temporal Agent Framework

The precise prediction of multi-scale traffic is a ubiquitous challenge in the urbanization process for car owners, road administrators, and governments. In the case of complex road networks, current and past traffic information from both upstream and downstream roads are crucial since various road networks have different semantic information about traffic. Rationalizing the utilization of semantic information can realize short-term, long-term, and unseen road traffic prediction. As the demands of multi-scale traffic analysis increase, on-demand interactions and visualizations are expected to be available for transportation participants. We have designed a multi-scale traffic generation system, namely TrafficGPT, using three AI agents to process multi-scale traffic data, conduct multi-scale traffic analysis, and present multi-scale visualization results. TrafficGPT consists of three essential AI agents: 1) a text-to-demand agent that is employed with Question & Answer AI to interact with users and extract prediction tasks through texts; 2) a traffic prediction agent that leverages multi-scale traffic data to generate temporal features and similarity, and fuse them with limited spatial features and similarity, to achieve accurate prediction of three tasks; and 3) a suggestion and visualization agent that uses the prediction results to generate suggestions and visualizations, providing users with a comprehensive understanding of traffic conditions. Our TrafficGPT system focuses on addressing concerns about traffic prediction from transportation participants, and conducted extensive experiments on five real-world road datasets to demonstrate its superior predictive and interactive performance

cs.LG↗

arXiv2024

Variable Selection in Ultra-high Dimensional Feature Space for the Cox Model with Interval-Censored Data

We develop a set of variable selection methods for the Cox model under interval censoring, in the ultra-high dimensional setting where the dimensionality can grow exponentially with the sample size. The methods select covariates via a penalized nonparametric maximum likelihood estimation with some popular penalty functions, including lasso, adaptive lasso, SCAD, and MCP. We prove that our penalized variable selection methods with folded concave penalties or adaptive lasso penalty enjoy the oracle property. Extensive numerical experiments show that the proposed methods have satisfactory empirical performance under various scenarios. The utility of the methods is illustrated through an application to a genome-wide association study of age to early childhood caries.

stat.ME↗