arXiv subjects

Rui Zhang

Publications and source records attributed to Rui Zhang.

At least 541 records · Page 30Linked to original sources

arXiv2024

Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network

Spiking neural networks (SNNs), known for their low-power, event-driven computation and intrinsic temporal dynamics, are emerging as promising solutions for processing dynamic, asynchronous signals from event-based sensors. Despite their potential, SNNs face challenges in training and architectural design, resulting in limited performance in challenging event-based dense prediction tasks compared to artificial neural networks (ANNs). In this work, we develop an efficient spiking encoder-decoder network (SpikingEDN) for large-scale event-based semantic segmentation tasks. To enhance the learning efficiency from dynamic event streams, we harness the adaptive threshold which improves network accuracy, sparsity and robustness in streaming inference. Moreover, we develop a dual-path Spiking Spatially-Adaptive Modulation module, which is specifically tailored to enhance the representation of sparse events and multi-modal inputs, thereby considerably improving network performance. Our SpikingEDN attains a mean intersection over union (MIoU) of 72.57\% on the DDD17 dataset and 58.32\% on the larger DSEC-Semantic dataset, showing competitive results to the state-of-the-art ANNs while requiring substantially fewer computational resources. Our results shed light on the untapped potential of SNNs in event-based vision applications. The source code will be made publicly available.

cs.CV↗

arXiv2024

EXACT-Net:EHR-guided lung tumor auto-segmentation for non-small cell lung cancer radiotherapy

Lung cancer is a devastating disease with the highest mortality rate among cancer types. Over 60% of non-small cell lung cancer (NSCLC) patients, which accounts for 87% of diagnoses, require radiation therapy. Rapid treatment initiation significantly increases the patient's survival rate and reduces the mortality rate. Accurate tumor segmentation is a critical step in the diagnosis and treatment of NSCLC. Manual segmentation is time and labor-consuming and causes delays in treatment initiation. Although many lung nodule detection methods, including deep learning-based models, have been proposed, there is still a long-standing problem of high false positives (FPs) with most of these methods. Here, we developed an electronic health record (EHR) guided lung tumor auto-segmentation called EXACT-Net (EHR-enhanced eXACtitude in Tumor segmentation), where the extracted information from EHRs using a pre-trained large language model (LLM), was used to remove the FPs and keep the TP nodules only. The auto-segmentation model was trained on NSCLC patients' computed tomography (CT), and the pre-trained LLM was used with the zero-shot learning approach. Our approach resulted in a 250% boost in successful nodule detection using the data from ten NSCLC patients treated in our institution.

eess.IV↗

arXiv2024

Relations between quantum metrology and criticality in general su(1, 1) systems

There is a prevalent effort to achieve quantum-enhanced metrology using criticality. However, the extent to which estimation precision is enhanced through criticality still needs further exploration under the constraint of finite time resources. We clarify relations between quantum metrology and criticality through a unitary parametrization process with a Hamiltonian governed by su(1, 1) Lie algebra. We demonstrate that the determination of the generator in the parameterization can be treated as an extended brachistochrone problem. Furthermore, the dynamic quantum Fisher information about the parameter exhibits a power-law dependence on the evolution time as the system approaches its critical point. By investigating the dynamic sensing proposals of three quantum critical systems, we show that the asymptotic behavior of sensitivity is consistent with our predictions. Our theory provides a deep understanding on the interplay of quantum metrology and criticality, providing insights into the underlying connections that involve both quantum phenomena and classical problems.

quant-ph↗

arXiv2024

Evaluating LLMs at Detecting Errors in LLM Responses

With Large Language Models (LLMs) being widely used across various tasks, detecting errors in their responses is increasingly crucial. However, little research has been conducted on error detection of LLM responses. Collecting error annotations on LLM responses is challenging due to the subjective nature of many NLP tasks, and thus previous research focuses on tasks of little practical value (e.g., word sorting) or limited error types (e.g., faithfulness in summarization). This work introduces ReaLMistake, the first error detection benchmark consisting of objective, realistic, and diverse errors made by LLMs. ReaLMistake contains three challenging and meaningful tasks that introduce objectively assessable errors in four categories (reasoning correctness, instruction-following, context-faithfulness, and parameterized knowledge), eliciting naturally observed and diverse errors in responses of GPT-4 and Llama 2 70B annotated by experts. We use ReaLMistake to evaluate error detectors based on 12 LLMs. Our findings show: 1) Top LLMs like GPT-4 and Claude 3 detect errors made by LLMs at very low recall, and all LLM-based error detectors perform much worse than humans. 2) Explanations by LLM-based error detectors lack reliability. 3) LLMs-based error detection is sensitive to small changes in prompts but remains challenging to improve. 4) Popular approaches to improving LLMs, including self-consistency and majority vote, do not improve the error detection performance. Our benchmark and code are provided at https://github.com/psunlpgroup/ReaLMistake.

cs.CL↗

arXiv2024

Near-field Integrated Sensing and Communication: Opportunities and Challenges

With the extremely large-scale array XL-array deployed in future wireless systems, wireless communication and sensing are expected to operate in the radiative near-field region, which needs to be characterized by the spherical rather than planar wavefronts. Unlike most existing works that considered far-field integrated sensing and communication (ISAC), we study in this article the new near-field ISAC, which integrates both functions of sensing and communication in the near-field region. To this end, we first discuss the appealing advantages of near-field communication and sensing over their far-field counterparts, respectively. Then, we introduce three approaches for near-field ISAC, including joint near-field communication and sensing, sensing-assisted near-field communication, and communication-assisted near-field sensing. We discuss their individual research opportunities, new design issues, as well as propose promising solutions. Finally, several important directions in near-field ISAC are also highlighted to motivate future work.

cs.IT↗

arXiv2024

LinguaLinker: Audio-Driven Portraits Animation with Implicit Facial Control Enhancement

This study delves into the intricacies of synchronizing facial dynamics with multilingual audio inputs, focusing on the creation of visually compelling, time-synchronized animations through diffusion-based techniques. Diverging from traditional parametric models for facial animation, our approach, termed LinguaLinker, adopts a holistic diffusion-based framework that integrates audio-driven visual synthesis to enhance the synergy between auditory stimuli and visual responses. We process audio features separately and derive the corresponding control gates, which implicitly govern the movements in the mouth, eyes, and head, irrespective of the portrait's origin. The advanced audio-driven visual synthesis mechanism provides nuanced control but keeps the compatibility of output video and input audio, allowing for a more tailored and effective portrayal of distinct personas across different languages. The significant improvements in the fidelity of animated portraits, the accuracy of lip-syncing, and the appropriate motion variations achieved by our method render it a versatile tool for animating any portrait in any language.

cs.CV↗

arXiv2024

Lattice QCD calculation of the pion distribution amplitude with domain wall fermions at physical pion mass

We present a direct lattice QCD calculation of the $x$-dependence of the pion distribution amplitude (DA), which is performed using the quasi-DA in large momentum effective theory on a domain-wall fermion ensemble at physical quark masses and spacing $a\approx 0.084$ fm. The bare quais-DA matrix elements are renormalized in the hybrid scheme and matched to $\overline{\rm MS}$ with a subtraction of the leading renormalon in the Wilson-line mass. For the first time, we include threshold resummation in the perturbative matching onto the light-cone DA, which resums the large logarithms in the soft gluon limit at next-to-next-to-leading log. The resummed results show controlled scale-variation uncertainty within the range of momentum fraction $x\in[0.25,0.75]$ at the largest pion momentum $P_z\approx 1.85$~GeV. In addition, we apply the same analysis to quasi-DAs from a highly-improved-staggered-quark ensemble at physical pion mass and $a=0.076$ fm. By comparison we find with $2σ$ confidence level that the DA obtained from chiral fermions is flatter and lower near $x=0.5$.

hep-lat↗

arXiv2024

Near-Field Integrated Sensing and Communication with Extremely Large-Scale Antenna Array

This paper studies a near-field integrated sensing and communication (ISAC) system with extremely large-scale antenna array (ELAA), in which a base station (BS) deployed with enormous number of antennas transmits wireless signals to communicate with multiple communication users (CUs) and simultaneously uses the echo signals to localize multiple point targets in the three-dimension (3D) space. To balance the performance tradeoff between communication and target localization, we design the transmit covariance matrix at the BS to optimize the localization performance while ensuring the signal-to-interference-plus-noise ratio (SINR) constraints at individual CUs. In particular, we formulate three design problems by considering different 3D localization performance metrics, including minimizing the sum Cramér-Rao bound (CRB), maximizing the minimum target illumination power, and maximizing the minimum target echo signal power. Although the three design problems are non-convex in general, we obtain their global optimal solutions via the technique of semi-definite relaxation (SDR). It is shown that the three problems have low-rank solution structures depending on the sensing and communication channel matrices, helping reduce the complexity of the SDR-based solutions. Interestingly, we find that in the special case with a single collocated target/CU present towards the middle of a symmetric uniform planar array (UPA), the optimal solutions to the three problems become identical to the SINR-maximization design and have a closed form, while in other cases they can be different in general. Besides, when the target/CU moves away from the transmitter/receiver, the CRB may first decrease and then increase. These two phenomena differ from those in the far-field. Numerical results show the benefits of the proposed designs for near-field ISAC, by exploiting the beam focusing capabilities of ELAA.

eess.SP↗

arXiv2024

MoveLight: Enhancing Traffic Signal Control through Movement-Centric Deep Reinforcement Learning

This paper introduces MoveLight, a novel traffic signal control system that enhances urban traffic management through movement-centric deep reinforcement learning. By leveraging detailed real-time data and advanced machine learning techniques, MoveLight overcomes the limitations of traditional traffic signal control methods. It employs a lane-level control approach using the FRAP algorithm to achieve dynamic and adaptive traffic signal control, optimizing traffic flow, reducing congestion, and improving overall efficiency. Our research demonstrates the scalability and effectiveness of MoveLight across single intersections, arterial roads, and network levels. Experimental results using real-world datasets from Cologne and Hangzhou show significant improvements in metrics such as queue length, delay, and throughput compared to existing methods. This study highlights the transformative potential of deep reinforcement learning in intelligent traffic signal control, setting a new standard for sustainable and efficient urban transportation systems.

cs.LG↗

arXiv2024

Backdoor Attacks against Hybrid Classical-Quantum Neural Networks

Hybrid Quantum Neural Networks (HQNNs) represent a promising advancement in Quantum Machine Learning (QML), yet their security has been rarely explored. In this paper, we present the first systematic study of backdoor attacks on HQNNs. We begin by proposing an attack framework and providing a theoretical analysis of the generalization bounds and minimum perturbation requirements for backdoor attacks on HQNNs. Next, we employ two classic backdoor attack methods on HQNNs and Convolutional Neural Networks (CNNs) to further investigate the robustness of HQNNs. Our experimental results demonstrate that HQNNs are more robust than CNNs, requiring more significant image modifications for successful attacks. Additionally, we introduce the Qcolor backdoor, which utilizes color shifts as triggers and employs the Non-dominated Sorting Genetic Algorithm II (NSGA-II) to optimize hyperparameters. Through extensive experiments, we demonstrate the effectiveness, stealthiness, and robustness of the Qcolor backdoor.

cs.CR↗

arXiv2024

vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving

Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests. This surge in demand poses significant challenges in optimizing throughput and latency while keeping costs manageable. The Key-Value (KV) cache, a standard method for retaining previous computations, makes LLM inference highly bounded by memory. While batching strategies can enhance performance, they frequently lead to significant memory fragmentation. Even though cutting-edge systems like vLLM mitigate KV cache fragmentation using paged Attention mechanisms, they still suffer from inefficient memory and computational operations due to the tightly coupled page management and computation kernels. This study introduces the vTensor, an innovative tensor structure for LLM inference based on GPU virtual memory management (VMM). vTensor addresses existing limitations by decoupling computation from memory defragmentation and offering dynamic extensibility. Our framework employs a CPU-GPU heterogeneous approach, ensuring efficient, fragmentation-free memory management while accommodating various computation kernels across different LLM architectures. Experimental results indicate that vTensor achieves an average speedup of 1.86x across different models, with up to 2.42x in multi-turn chat scenarios. Additionally, vTensor provides average speedups of 2.12x and 3.15x in kernel evaluation, reaching up to 3.92x and 3.27x compared to SGLang Triton prefix-prefilling kernels and vLLM paged Attention kernel, respectively. Furthermore, it frees approximately 71.25% (57GB) of memory on the NVIDIA A100 GPU compared to vLLM, enabling more memory-intensive workloads.

cs.DC↗

arXiv2024

Power Measurement Enabled Channel Autocorrelation Matrix Estimation for IRS-Assisted Wireless Communication

By reconfiguring wireless channels via passive signal reflection, intelligent reflecting surface (IRS) can bring significant performance enhancement for wireless communication systems. However, such performance improvement generally relies on the knowledge of channel state information (CSI) for IRS-involved links. Prior works on IRS CSI acquisition mainly estimate IRS-cascaded channels based on the extra pilot signals received at the users/base station (BS) with time-varying IRS reflections, which, however, needs to modify the existing channel training/estimation protocols of wireless systems. To address this issue, we propose in this paper a new channel estimation scheme for IRS-assisted communication systems based on the received signal power measured at the user terminal, which is practically attainable without the need of changing the current protocol. Due to the lack of signal phase information in measured power, the autocorrelation matrix of the BS-IRS-user cascaded channel is estimated by solving an equivalent rank-minimization problem. To this end, a low-rank-approaching (LRA) algorithm is proposed by employing the fractional programming and alternating optimization techniques. To reduce computational complexity, an approximate LRA (ALRA) algorithm is also developed. Furthermore, these two algorithms are extended to be robust against the receiver noise and quantization error in power measurement. Simulation results are provided to verify the effectiveness of the proposed channel estimation algorithms as well as the IRS passive reflection design based on the estimated channel autocorrelation matrix.

eess.SP↗

arXiv2024

Fine-grained Knowledge Graph-driven Video-Language Learning for Action Recognition

Recent work has explored video action recognition as a video-text matching problem and several effective methods have been proposed based on large-scale pre-trained vision-language models. However, these approaches primarily operate at a coarse-grained level without the detailed and semantic understanding of action concepts by exploiting fine-grained semantic connections between actions and body movements. To address this gap, we propose a contrastive video-language learning framework guided by a knowledge graph, termed KG-CLIP, which incorporates structured information into the CLIP model in the video domain. Specifically, we construct a multi-modal knowledge graph composed of multi-grained concepts by parsing actions based on compositional learning. By implementing a triplet encoder and deviation compensation to adaptively optimize the margin in the entity distance function, our model aims to improve alignment of entities in the knowledge graph to better suit complex relationship learning. This allows for enhanced video action recognition capabilities by accommodating nuanced associations between graph components. We comprehensively evaluate KG-CLIP on Kinetics-TPS, a large-scale action parsing dataset, demonstrating its effectiveness compared to competitive baselines. Especially, our method excels at action recognition with few sample frames or limited training data, which exhibits excellent data utilization and learning capabilities.

cs.MM↗

arXiv2024

KNOWNET: Guided Health Information Seeking from LLMs via Knowledge Graph Integration

The increasing reliance on Large Language Models (LLMs) for health information seeking can pose severe risks due to the potential for misinformation and the complexity of these topics. This paper introduces KNOWNET a visualization system that integrates LLMs with Knowledge Graphs (KG) to provide enhanced accuracy and structured exploration. Specifically, for enhanced accuracy, KNOWNET extracts triples (e.g., entities and their relations) from LLM outputs and maps them into the validated information and supported evidence in external KGs. For structured exploration, KNOWNET provides next-step recommendations based on the neighborhood of the currently explored entities in KGs, aiming to guide a comprehensive understanding without overlooking critical aspects. To enable reasoning with both the structured data in KGs and the unstructured outputs from LLMs, KNOWNET conceptualizes the understanding of a subject as the gradual construction of graph visualization. A progressive graph visualization is introduced to monitor past inquiries, and bridge the current query with the exploration history and next-step recommendations. We demonstrate the effectiveness of our system via use cases and expert interviews.

cs.HC↗

arXiv2024

Fixed and Movable Antenna Technology for 6G Integrated Sensing and Communication

By deploying antenna arrays at the transmitter/receiver to provide additional spatial-domain degrees of freedom (DoFs), multi-antenna technology greatly improves the reliability and efficiency of wireless communication. Meanwhile, the application of multi-antenna technology in the radar field has achieved spatial angle resolution and improved sensing DoF, thus significantly enhancing wireless sensing performance. However, wireless communication and radar sensing have undergone independent development over the past few decades. As a result, although multi-antenna technology has dramatically advanced in these two fields separately, it has not been deeply integrated by exploiting their synergy. A new opportunity to fill up this gap arises as the integration of sensing and communication has been identified as one of the typical usage scenarios of the 6G communication network. Motivated by the above, this article aims to explore the multi-antenna technology for 6G ISAC, with the focus on its future development trends such as continuous expansion of antenna array scale, more diverse array architectures, and more flexible antenna designs. First, we introduce several new and promising antenna architectures, including the centralized antenna architectures based on traditional compact arrays or emerging sparse arrays, the distributed antenna architectures exemplified by the cell-free massive MIMO, and the movable/fluid antennas with flexible positions and/or orientations in a given 3D space. Next, for each antenna architecture mentioned above, we present the corresponding far-field/near-field channel models and analyze the communication and sensing performance. Finally, we summarize the characteristics of different antenna architectures and look forward to new ideas for solving the difficulties in acquiring CSI caused by the continuous expansion of antenna array scale and flexible antenna designs.

cs.AR↗

arXiv2024

Backdoor Attacks against Image-to-Image Networks

Recently, deep learning-based Image-to-Image (I2I) networks have become the predominant choice for I2I tasks such as image super-resolution and denoising. Despite their remarkable performance, the backdoor vulnerability of I2I networks has not been explored. To fill this research gap, we conduct a comprehensive investigation on the susceptibility of I2I networks to backdoor attacks. Specifically, we propose a novel backdoor attack technique, where the compromised I2I network behaves normally on clean input images, yet outputs a predefined image of the adversary for malicious input images containing the trigger. To achieve this I2I backdoor attack, we propose a targeted universal adversarial perturbation (UAP) generation algorithm for I2I networks, where the generated UAP is used as the backdoor trigger. Additionally, in the backdoor training process that contains the main task and the backdoor task, multi-task learning (MTL) with dynamic weighting methods is employed to accelerate convergence rates. In addition to attacking I2I tasks, we extend our I2I backdoor to attack downstream tasks, including image classification and object detection. Extensive experiments demonstrate the effectiveness of the I2I backdoor on state-of-the-art I2I network architectures, as well as the robustness against different mainstream backdoor defenses.

cs.CV↗

arXiv2024

DDFAD: Dataset Distillation Framework for Audio Data

Deep neural networks (DNNs) have achieved significant success in numerous applications. The remarkable performance of DNNs is largely attributed to the availability of massive, high-quality training datasets. However, processing such massive training data requires huge computational and storage resources. Dataset distillation is a promising solution to this problem, offering the capability to compress a large dataset into a smaller distilled dataset. The model trained on the distilled dataset can achieve comparable performance to the model trained on the whole dataset. While dataset distillation has been demonstrated in image data, none have explored dataset distillation for audio data. In this work, for the first time, we propose a Dataset Distillation Framework for Audio Data (DDFAD). Specifically, we first propose the Fused Differential MFCC (FD-MFCC) as extracted features for audio data. After that, the FD-MFCC is distilled through the matching training trajectory distillation method. Finally, we propose an audio signal reconstruction algorithm based on the Griffin-Lim Algorithm to reconstruct the audio signal from the distilled FD-MFCC. Extensive experiments demonstrate the effectiveness of DDFAD on various audio datasets. In addition, we show that DDFAD has promising application prospects in many applications, such as continual learning and neural architecture search.

cs.SD↗

arXiv2024

Understanding the Importance of Evolutionary Search in Automated Heuristic Design with Large Language Models

Automated heuristic design (AHD) has gained considerable attention for its potential to automate the development of effective heuristics. The recent advent of large language models (LLMs) has paved a new avenue for AHD, with initial efforts focusing on framing AHD as an evolutionary program search (EPS) problem. However, inconsistent benchmark settings, inadequate baselines, and a lack of detailed component analysis have left the necessity of integrating LLMs with search strategies and the true progress achieved by existing LLM-based EPS methods to be inadequately justified. This work seeks to fulfill these research queries by conducting a large-scale benchmark comprising four LLM-based EPS methods and four AHD problems across nine LLMs and five independent runs. Our extensive experiments yield meaningful insights, providing empirical grounding for the importance of evolutionary search in LLM-based AHD approaches, while also contributing to the advancement of future EPS algorithmic development. To foster accessibility and reproducibility, we have fully open-sourced our benchmark and corresponding results.

cs.NE↗