arXiv subjects

Wei Dai

Publications and source records attributed to Wei Dai.

At least 145 records · Page 8Linked to original sources

arXiv2024

Advancing Academic Knowledge Retrieval via LLM-enhanced Representation Similarity Fusion

In an era marked by robust technological growth and swift information renewal, furnishing researchers and the populace with top-tier, avant-garde academic insights spanning various domains has become an urgent necessity. The KDD Cup 2024 AQA Challenge is geared towards advancing retrieval models to identify pertinent academic terminologies from suitable papers for scientific inquiries. This paper introduces the LLM-KnowSimFuser proposed by Robo Space, which wins the 2nd place in the competition. With inspirations drawed from the superior performance of LLMs on multiple tasks, after careful analysis of the provided datasets, we firstly perform fine-tuning and inference using LLM-enhanced pre-trained retrieval models to introduce the tremendous language understanding and open-domain knowledge of LLMs into this task, followed by a weighted fusion based on the similarity matrix derived from the inference results. Finally, experiments conducted on the competition datasets show the superiority of our proposal, which achieved a score of 0.20726 on the final leaderboard.

cs.IR↗

arXiv2024

Unraveling the Energy-Energy Correlators for Heavy Flavor Tagged Jets in pp, p+Pb and Pb+Pb Collisions

In this study, energy-energy correlator (EEC) distributions for the $\rm D^0$, $\rm B^0$-tagged, inclusive, and the PYTHIA generated pure quark jets are computed in pp, p+Pb, and Pb+Pb collisions at \sqrts =5.02 TeV for a same jet transverse momentum interval 15-30 GeV. We find the number of particles per jet determines the height of the EEC distribution in pp baseline. The averaged energy weight distribution resulted in a shift of the originally larger angular distributed particle distribution to a smaller RL, thereby obtaining an EEC distribution. The EEC distributions for all quark-tagged jets in A+A exhibit a noticeable shift towards larger RL region, suggesting that the jets will be more widely distributed compared to those in pp collisions. The jet quenching effect will cause the pair angular distribution to shift towards larger values and increase the number of particles per jet. This redistribution of energy within the jets suggests that the already reduced jet energy is redistributed among a larger number of particles, leading to a reduced energy weight per pair. The enhancement of the number of particles per jet and the reduced averaged energy weight interplay with each other to form the medium modification of EEC.

hep-ph↗

arXiv2024

SoccerNet 2024 Challenges Results

The SoccerNet 2024 challenges represent the fourth annual video understanding challenges organized by the SoccerNet team. These challenges aim to advance research across multiple themes in football, including broadcast video understanding, field understanding, and player understanding. This year, the challenges encompass four vision-based tasks. (1) Ball Action Spotting, focusing on precisely localizing when and which soccer actions related to the ball occur, (2) Dense Video Captioning, focusing on describing the broadcast with natural language and anchored timestamps, (3) Multi-View Foul Recognition, a novel task focusing on analyzing multiple viewpoints of a potential foul incident to classify whether a foul occurred and assess its severity, (4) Game State Reconstruction, another novel task focusing on reconstructing the game state from broadcast videos onto a 2D top-view map of the field. Detailed information about the tasks, challenges, and leaderboards can be found at https://www.soccer-net.org, with baselines and development kits available at https://github.com/SoccerNet.

cs.CV↗

arXiv2024

DVRP-MHSI: Dynamic Visualization Research Platform for Multimodal Human-Swarm Interaction

In recent years, there has been a significant amount of research on algorithms and control methods for distributed collaborative robots. However, the emergence of collective behavior in a swarm is still difficult to predict and control. Nevertheless, human interaction with the swarm helps render the swarm more predictable and controllable, as human operators can utilize intuition or knowledge that is not always available to the swarm. Therefore, this paper designs the Dynamic Visualization Research Platform for Multimodal Human-Swarm Interaction (DVRP-MHSI), which is an innovative open system that can perform real-time dynamic visualization and is specifically designed to accommodate a multitude of interaction modalities (such as brain-computer, eye-tracking, electromyographic, and touch-based interfaces), thereby expediting progress in human-swarm interaction research. Specifically, the platform consists of custom-made low-cost omnidirectional wheeled mobile robots, multitouch screens and two workstations. In particular, the mutitouch screens can recognize human gestures and the shapes of objects placed on them, and they can also dynamically render diverse scenes. One of the workstations processes communication information within robots and the other one implements human-robot interaction methods. The development of DVRP-MHSI frees researchers from hardware or software details and allows them to focus on versatile swarm algorithms and human-swarm interaction methods without being limited to fixed scenarios, tasks, and interfaces. The effectiveness and potential of the platform for human-swarm interaction studies are validated by several demonstrative experiments.

cs.RO↗

arXiv2024

State-dependent Filtering of the Ring Model

Robustness is a measure of functional reliability of a system against perturbations. To achieve a good and robust performance, a system must filter out external perturbations by its internal priors. These priors are usually distilled in the structure and the states of the system. Biophysical neural network are known to be robust but the exact mechanisms are still elusive. In this paper, we probe how orientation-selective neurons organized on a 1-D ring network respond to perturbations in the hope of gaining some insights on the robustness of visual system in brain. We analyze the steady-state of the rate-based network and prove that the activation state of neurons, rather than their firing rates, determines how the model respond to perturbations. We then identify specific perturbation patterns that induce the largest responses for different configurations of activation states, and find them to be sinusoidal or sinusoidal-like while other patterns are largely attenuated. Similar results are observed in a spiking ring model. Finally, we remap the perturbations in orientation back into the 2-D image space using Gabor functions. The resulted optimal perturbation patterns mirror adversarial attacks in deep learning that exploit the priors of the system. Our results suggest that based on different state configurations, these priors could underlie some of the illusionary experiences as the cost of visual robustness.

q-bio.NC↗

arXiv2024

Harnessing Neural Unit Dynamics for Effective and Scalable Class-Incremental Learning

Class-incremental learning (CIL) aims to train a model to learn new classes from non-stationary data streams without forgetting old ones. In this paper, we propose a new kind of connectionist model by tailoring neural unit dynamics that adapt the behavior of neural networks for CIL. In each training session, it introduces a supervisory mechanism to guide network expansion whose growth size is compactly commensurate with the intrinsic complexity of a newly arriving task. This constructs a near-minimal network while allowing the model to expand its capacity when cannot sufficiently hold new classes. At inference time, it automatically reactivates the required neural units to retrieve knowledge and leaves the remaining inactivated to prevent interference. We name our model AutoActivator, which is effective and scalable. To gain insights into the neural unit dynamics, we theoretically analyze the model's convergence property via a universal approximation theorem on learning sequential mappings, which is under-explored in the CIL community. Experiments show that our method achieves strong CIL performance in rehearsal-free and minimal-expansion settings with different backbones.

cs.LG↗

arXiv2024

Robust Waveform Design for Integrated Sensing and Communication

Integrated sensing and communication (ISAC), which enables hardware, resources (e.g., spectra), and waveforms sharing, is becoming a key feature in future-generation communication systems. This paper investigates performance characterization and waveform design for ISAC systems when the underlying true communication channels are not accurately known. With uncertainty in a nominal communication channel, the nominal Pareto frontier of the sensing and communication performances cannot represent the true performance trade-off of a real-world operating ISAC system. Therefore, this paper portrays the robust (i.e., conservative) Pareto frontier considering the uncertainty in the communication channel. To be specific, the lower bound of the true (but unknown) Pareto frontier is investigated, technically by studying robust waveform design problems that find the best waveforms under the worst-case channels. The robust waveform design problems examined in this paper are shown to be non-convex and high-dimensional, which cannot be solved using existing optimization techniques. As such, we propose a computationally efficient solution framework to approximately solve them. Simulation results show that by solving the robust waveform design problems, the lower bound of the true but unknown Pareto frontier, which characterizes the sensing-communication performance trade-off under communication channel uncertainty, can be obtained.

eess.SP↗

arXiv2024

A Low-rank Projected Proximal Gradient Method for Spectral Compressed Sensing

This paper presents a new approach to the recovery of a spectrally sparse signal (SSS) from partially observed entries, focusing on challenges posed by large-scale data and heavy noise environments. The SSS reconstruction can be formulated as a non-convex low-rank Hankel recovery problem. Traditional formulations for SSS recovery often suffer from reconstruction inaccuracies due to unequally weighted norms and over-relaxation of the Hankel structure in noisy conditions. Moreover, a critical limitation of standard proximal gradient (PG) methods for solving the optimization problem is their slow convergence. We overcome this by introducing a more accurate formulation and a Low-rank Projected Proximal Gradient (LPPG) method, designed to efficiently converge to stationary points through a two-step process. The first step involves a modified PG approach, allowing for a constant step size independent of signal size, which significantly accelerates the gradient descent phase. The second step employs a subspace projection strategy, optimizing within a low-rank matrix space to further decrease the objective function. Both steps of the LPPG method are meticulously tailored to exploit the intrinsic low-rank and Hankel structures of the problem, thereby enhancing computational efficiency. Our numerical simulations reveal a substantial improvement in both the efficiency and recovery accuracy of the LPPG method compared to existing benchmark algorithms. This performance gain is particularly pronounced in scenarios with significant noise, demonstrating the method's robustness and applicability to large-scale SSS recovery tasks.

eess.SP↗

arXiv2024

Radial symmetry and sharp asymptotic behaviors of nonnegative solutions to $D^{1,p}$-critical quasi-linear static Schrödinger-Hartree equation involving $p$-Laplacian $-Δ_{p}$

In this paper, we mainly consider nonnegative weak solution to the $D^{1,p}(\R^{N})$-critical quasi-linear static Schrödinger-Hartree equation with $p$-Laplacian $-Δ_{p}$ and nonlocal nonlinearity: \begin{align*} -Δ_p u =\left(|x|^{-2p}\ast |u|^{p}\right)|u|^{p-2}u \qquad &\mbox{in} \,\, \mathbb{R}^N, \end{align*} where $1<p<\frac{N}{2}$, $N\geq3$ and $u\in D^{1,p}(\R^N)$. Being different to the $D^{1,p}(\R^{N})$-critical local nonlinear term $u^{p^{\star}-1}$ with $p^{\star}:=\frac{Np}{N-p}$ investigated in \cite{CFR,LDSMLMSB,GV,Ou,BS16,VJ16} etc., since the nonlocal convolution $|x|^{-2p}*u^p$ appears in the Hartree type nonlinearity, it is impossible for us to use the scaling arguments and the Doubling Lemma as in \cite{VJ16} to get preliminary estimates on upper bounds of asymptotic behaviors for any positive solutions $u$. Moreover, it is also quite difficult to obtain the boundedness of the quasi-norm $\|u \|_{L^{s,\infty}(\R^N)}$ and hence derive the sharp estimates on upper bounds of asymptotic behaviors from the preliminary estimates as in \cite{VJ16}. Fortunately, by showing a better preliminary estimates on upper bounds of asymptotic behaviors through the De Giorgi-Moser-Nash iteration method and combining the result from \cite{XCL}, we are able to overcome these difficulties and establish regularity and the sharp estimates on both upper and lower bounds of asymptotic behaviors for any positive solution $u$ to more general equation $-Δ_p u=V(x)u^{p-1}$ with $V\in L^{\frac{N}{p}}(\mathbb{R}^{N})$. Then, by using the arguments from \cite{BS16,VJ16}, we can deduce the sharp estimates on both upper and lower bounds for the decay rate of $|\nabla u|$. Finally, as a consequence, we can apply the method of moving planes to prove that all the nontrivial nonnegative solutions are radially symmetric and strictly decreasing about some point $x_0\in\R^N$.

math.AP↗

arXiv2024

Interpretable Neural Networks with Random Constructive Algorithm

This paper introduces an Interpretable Neural Network (INN) incorporating spatial information to tackle the opaque parameterization process of random weighted neural networks. The INN leverages spatial information to elucidate the connection between parameters and network residuals. Furthermore, it devises a geometric relationship strategy using a pool of candidate nodes and established relationships to select node parameters conducive to network convergence. Additionally, a lightweight version of INN tailored for large-scale data modeling tasks is proposed. The paper also showcases the infinite approximation property of INN. Experimental findings on various benchmark datasets and real-world industrial cases demonstrate INN's superiority over other neural networks of the same type in terms of modeling speed, accuracy, and network structure.

cs.LG↗

arXiv2024

Radio Frequency Interference Detection Using Efficient Multi-Scale Convolutional Attention UNet

Studying the universe through radio telescope observation is crucial. However, radio telescopes capture not only signals from the universe but also various interfering signals, known as Radio Frequency Interference (RFI). The presence of RFI can significantly impact data analysis. Ensuring the accuracy, reliability, and scientific integrity of research findings by detecting and mitigating or eliminating RFI in observational data, presents a persistent challenge in radio astronomy. In this study, we proposed a novel deep learning model called EMSCA-UNet for RFI detection. The model employs multi-scale convolutional operations to extract RFI features of various scale sizes. Additionally, an attention mechanism is utilized to assign different weights to the extracted RFI feature maps, enabling the model to focus on vital features for RFI detection. We evaluated the performance of the model using real data observed from the 40-meter radio telescope at Yunnan Observatory. Furthermore, we compared our results to other models, including U-Net, RFI-Net, and R-Net, using four commonly employed evaluation metrics: precision, recall, F1 score, and IoU. The results demonstrate that our model outperforms the other models on all evaluation metrics, achieving an average improvement of approximately 5\% compared to U-Net. Our model not only enhances the accuracy and comprehensiveness of RFI detection but also provides more detailed edge detection while minimizing the loss of useful signals.

astro-ph.IM↗

arXiv2024

P-TimeSync: A Precise Time Synchronization Simulation with Network Propagation Delays

Time serves as the foundation of modern society and will continue to grow in value in the future world. Unlike previous research papers, authors delve into various time sources, ranging from atomic time and GPS time to quartz time. Specifically, we explore the time uncertainty associated with the four major Global Navigation Satellite Systems. In existing time synchronization simulations provide partial usages. However, our research introduces a comprehensive and precise time synchronization simulation named P-TimeSync, leading to a better understanding of time synchronization in distributed environments. It is a state-of-the-art simulation tool for time because (1) it can simulate atomic clocks and quartz clocks with user-defined software clock algorithms, (2) the simulation provides nanosecond-level precision time across different network propagation paths and distances, (3) the tool offers a visualization platform with classic algorithms for distributed time synchronization, such as Cristian's algorithm and Berkeley algorithm. The simulation easily allows for the redefinition of configurations and functions, supporting advanced research and development. The simulation tool could be downloaded via the website: https://github.com/rui5097/purdue_timesync

cs.DC↗

arXiv2024

Towards Scalable Semidefinite Programming: Optimal Metric ADMM with A Worst-case Performance Guarantee

Despite the numerous uses of semidefinite programming (SDP) and its universal solvability via interior point methods (IPMs), it is rarely applied to practical large-scale problems. This mainly owes to the computational cost of IPMs that increases in a bad exponential way with the data size. While first-order algorithms such as ADMM can alleviate this issue, but the scalability improvement appears far not enough. In this work, we aim to achieve extra acceleration for ADMM by appealing to a non-Euclidean metric space, while maintaining everything in closed-form expressions. The efficiency gain comes from the extra degrees of freedom of a variable metric compared to a scalar step-size, which allows us to capture some additional ill-conditioning structures. On the application side, we consider the quadratically constrained quadratic program (QCQP), which naturally appears in an SDP form after a dualization procedure. This technique, known as semidefinite relaxation, has important uses across different fields, particularly in wireless communications. Numerically, we observe that the scalability property is significantly improved. Depending on the data generation process, the extra acceleration can easily surpass the scalar-parameter efficiency limit, and the advantage is rapidly increasing as the data conditioning becomes worse.

math.OC↗

arXiv2024

Proximal Dogleg Opportunistic Majorization for Nonconvex and Nonsmooth Optimization

We consider minimizing a function consisting of a quadratic term and a proximable term which is possibly nonconvex and nonsmooth. This problem is also known as scaled proximal operator. Despite its simple form, existing methods suffer from slow convergence or high implementation complexity or both. To overcome these limitations, we develop a fast and user-friendly second-order proximal algorithm. Key innovation involves building and solving a series of opportunistically majorized problems along a hybrid Newton direction. The approach directly uses the precise Hessian of the quadratic term, and calculates the inverse only once, eliminating the iterative numerical approximation of the Hessian, a common practice in quasi-Newton methods. The algorithm's convergence to a critical point is established, and local convergence rate is derived based on the Kurdyka-Lojasiewicz property of the objective function. Numerical comparisons are conducted on well-known optimization problems. The results demonstrate that the proposed algorithm not only achieves a faster convergence but also tends to converge to a better local optimum compare to benchmark algorithms.

math.OC↗

arXiv2024

ASI: Accuracy-Stability Index for Evaluating Deep Learning Models

In the context of deep learning research, where model introductions continually occur, the need for effective and efficient evaluation remains paramount. Existing methods often emphasize accuracy metrics, overlooking stability. To address this, the paper introduces the Accuracy-Stability Index (ASI), a quantitative measure incorporating both accuracy and stability for assessing deep learning models. Experimental results demonstrate the application of ASI, and a 3D surface model is presented for visualizing ASI, mean accuracy, and coefficient of variation. This paper addresses the important issue of quantitative benchmarking metrics for deep learning models, providing a new approach for accurately evaluating accuracy and stability of deep learning models. The paper concludes with discussions on potential weaknesses and outlines future research directions.

cs.LG↗

arXiv2024

TrojanPuzzle: Covertly Poisoning Code-Suggestion Models

With tools like GitHub Copilot, automatic code suggestion is no longer a dream in software engineering. These tools, based on large language models, are typically trained on massive corpora of code mined from unvetted public sources. As a result, these models are susceptible to data poisoning attacks where an adversary manipulates the model's training by injecting malicious data. Poisoning attacks could be designed to influence the model's suggestions at run time for chosen contexts, such as inducing the model into suggesting insecure code payloads. To achieve this, prior attacks explicitly inject the insecure code payload into the training data, making the poison data detectable by static analysis tools that can remove such malicious data from the training set. In this work, we demonstrate two novel attacks, COVERT and TROJANPUZZLE, that can bypass static analysis by planting malicious poison data in out-of-context regions such as docstrings. Our most novel attack, TROJANPUZZLE, goes one step further in generating less suspicious poison data by never explicitly including certain (suspicious) parts of the payload in the poison data, while still inducing a model that suggests the entire payload when completing code (i.e., outside docstrings). This makes TROJANPUZZLE robust against signature-based dataset-cleansing methods that can filter out suspicious sequences from the training data. Our evaluation against models of two sizes demonstrates that both COVERT and TROJANPUZZLE have significant implications for practitioners when selecting code used to train or tune code-suggestion models.

cs.CR↗

arXiv2024

Classification of solutions to $3$-D and $4$-D mixed order conformally invariant systems with critical and exponential growth

In this paper, without any assumption on $v$ and under the extremely mild assumption $u(x)= O(|x|^{K})$ as $|x|\rightarrow+\infty$ for some $K\gg1$ arbitrarily large, we classify solutions of the following conformally invariant system with mixed order and exponentially increasing nonlinearity in $\mathbb{R}^{3}$: $$ \begin{cases} \ (-Δ)^{\frac{1}{2}} u=v^{4} ,&x\in \mathbb{R}^{3},\\ \ -Δv=e^{pw} ,&x\in \mathbb{R}^{3},\\ \ (-Δ)^{\frac{3}{2}} w=u^{3} ,&x\in \mathbb{R}^{3}, \end{cases} $$ where $p>0$, $w(x)=o(|x|^{2})$ at $\infty$ and $u,v\geq0$ satisfies the finite total curvature condition $\int_{\mathbb{R}^{3}}u^{3}(x)\mathrm{d}x<+\infty$. Moreover, under the extremely mild assumption that \emph{either} $u(x)$ or $v(x)=O(|x|^{K})$ as $|x|\rightarrow+\infty$ for some $K\gg1$ arbitrarily large \emph{or} $\int_{\mathbb{R}^{4}}e^{Λpw(y)}\mathrm{d}y<+\infty$ for some $Λ\geq1$, we also prove classification of solutions to the conformally invariant system with mixed order and exponentially increasing nonlinearity in $\mathbb{R}^{4}$: \begin{align*} \begin{cases} \ (-Δ)^{\frac{1}{2}} u=e^{pw} ,&x\in \mathbb{R}^{4},\\ \ -Δv=u^2 ,&x\in \mathbb{R}^{4},\\ \ (-Δ)^{2} w=v^{4} ,&x\in \mathbb{R}^{4}, \end{cases} \end{align*} where $p>0$, and $w(x)=o(|x|^{2})$ at $\infty$ and $u,v\geq0$ satisfies the finite total curvature condition $\int_{\mathbb{R}^{4}}v^{4}(x)\mathrm{d}x<+\infty$. The key ingredients are deriving the integral representation formulae and crucial asymptotic behaviors of solutions $(u,v,w)$ and calculating the explicit value of the total curvature.

math.AP↗

arXiv2023

LightGCNet: A Lightweight Geometric Constructive Neural Network for Data-Driven Soft sensors

Data-driven soft sensors provide a potentially cost-effective and more accurate modeling approach to measure difficult-to-measure indices in industrial processes compared to mechanistic approaches. Artificial intelligence (AI) techniques, such as deep learning, have become a popular soft sensors modeling approach in the area of machine learning and big data. However, soft sensors models based deep learning potentially lead to complex model structures and excessive training time. In addition, industrial processes often rely on distributed control systems (DCS) characterized by resource constraints. Herein, guided by spatial geometric, a lightweight geometric constructive neural network, namely LightGCNet, is proposed, which utilizes compact angle constraint to assign the hidden parameters from dynamic intervals. At the same time, a node pool strategy and spatial geometric relationships are used to visualize and optimize the process of assigning hidden parameters, enhancing interpretability. In addition, the universal approximation property of LightGCNet is proved by spatial geometric analysis. Two versions algorithmic implementations of LightGCNet are presented in this article. Simulation results concerning both benchmark datasets and the ore grinding process indicate remarkable merits of LightGCNet in terms of small network size, fast learning speed, and sound generalization.

stat.ML↗