arXiv subjects

Yu Wang

Publications and source records attributed to Yu Wang.

At least 667 records · Page 37Linked to original sources

arXiv2024

Ultrafast Spin Rotation of Relativistic Lepton Beams via Terahertz Wave in a Dielectric-Lined Waveguide

Spin rotation is central for the spin-manipulation of lepton beams which, in turn, plays an important role in investigation of the properties of spin-polarized lepton beams and the examination of spin-dependent interactions. However, realization of compact and ultrafast spin rotation of lepton beams, between longitudinal and transverse polarizations, still faces significant challenges. Here, we put forward a novel method for ultrafast (picosecond-timescale) spin rotation of a relativistic lepton beam via employing a moderate-intensity terahertz (THz) wave in a dielectric-lined waveguide (DLW). The lepton beam undergoes spin precession induced by the THz magnetic field. We find that optimizing the lepton velocity and THz phase velocity in the DLW can mitigate the impact of transverse Lorentz forces on the lepton beam and increase the precession frequency, thereby maintaining the beam quality and enhancing the efficiency of transverse-to-longitudinal spin rotation. The final polarization degree of the lepton beam exceeds $98\%$, and the energy spread can be improved significantly. Flexibility in adjusting the electromagnetic modes within the DLW adds further potential for spin-manipulation, and holds promise for advancing the development of spin-polarized particle beams, which have broad applications in materials science and atomic, nuclear, and high-energy physics.

physics.plasm-ph↗

arXiv2024

Spectral Convolutional Neural Network Chip for In-sensor Edge Computing of Incoherent Natural Light

Convolutional neural networks (CNNs) are representative models of artificial neural networks (ANNs). However, the considerable power consumption and limited computing speed of electrical computing platforms restrict further CNN development on edge devices. Optical neural networks are considered next-generation physical implementations of ANNs, but their capabilities are limited by on-chip integration scale and requirement for coherent light sources. This study proposes a spectral convolutional neural network (SCNN) of incoherent natural light by an optical convolutional layer (OCL) and a reconfigurable electrical backend. The OCL is implemented by integrating very large-scale, pixel-aligned spectral filters on a CMOS image sensor on a 12-inch wafer, facilitating highly parallel spectral vector-inner products of incident light. It accepts broadband incoherent natural light containing two spatial and one spectral dimension directly as input with the function of matter meta-imaging. This unique optoelectronic framework empowers in-sensor optical analog computing at extremely high energy efficiency because the OCL is driven by the energy of the information carrier, i. e. natural light. To the best of our knowledge, this is the first integrated optical computing utilizing natural light. We employ the same SCNN chip for completely different real-world complex tasks,and achieve accuracies of over 96% for pathological diagnosis and almost 100% for face anti-spoofing at video rates. The SCNN framework has an unprecedented new function of substance identification, provides a feasible optoelectronic and integrated optical CNN implementation for edge devices or cellphones, providing them with practical and powerful edge computing abilities and facilitating diverse applications, such as intelligent robotics, industrial automation, medical diagnosis, and remote sensing.

physics.optics↗

arXiv2024

A Novel Method of Estimating GRB Peak Energies Beyond the \emph{Swift}/BAT Limit

The \emph{Swift} Burst Alert Telescope (BAT), operating in the 15--150 keV energy band, struggles to detect the peak energy ($E_{\rm p}$) of gamma-ray bursts (GRBs), as most GRBs have $E_{\rm p}$ values typically distributed between 200-300 keV, exceeding BAT's upper limit. To address this, we develop an innovative method to robustly estimate the lower limit of $E_{\rm p}$ for GRBs with $E_{\rm p}>150$ keV. This approach relies on the intrinsic curvature of GRB spectra, which is already evident within the BAT energy range for such GRBs. By fitting BAT spectra with a cutoff power-law model and extrapolating the spectral curvature beyond BAT's range, we, therefore, can estimate the cutoff energy ($E^{'}_{\rm c}$) beyond 150 keV and the corresponding peak energy ($E^{'}_{\rm p}$). We applied this method to 17 GRBs, categorizing them into two main groups. Group I (10 bursts) maintains $α$ within a typical range (from $\sim$ -0.8 to $\sim$ -1.20) with increasing $E_{\rm c}$; Group II (2 bursts) maintains $E_{\rm c}$ within a typical range (300-500 keV) but with varying $α$. Our results show that for $E_{\rm c}\lesssim $1000 keV, the estimated $E^{'}_{\rm c}$ aligns well with observed values. Moreover, the reliability of $E^{'}_{\rm c}$ also depends on $α$: bursts with harder $α$ (e.g., $α\gtrsim -2/3$) show reduced accuracy, while bursts with softer $α$ (e.g., $α\lesssim -2/3$) yield more precise estimates. In conclusion, this method is well-suited for GRB spectra with moderately observed $E_{\rm c}$ ($E_{\rm p}$) values and $α$ indices that are not too hard.

astro-ph.HE↗

arXiv2024

A4-Unet: Deformable Multi-Scale Attention Network for Brain Tumor Segmentation

Brain tumor segmentation models have aided diagnosis in recent years. However, they face MRI complexity and variability challenges, including irregular shapes and unclear boundaries, leading to noise, misclassification, and incomplete segmentation, thereby limiting accuracy. To address these issues, we adhere to an outstanding Convolutional Neural Networks (CNNs) design paradigm and propose a novel network named A4-Unet. In A4-Unet, Deformable Large Kernel Attention (DLKA) is incorporated in the encoder, allowing for improved capture of multi-scale tumors. Swin Spatial Pyramid Pooling (SSPP) with cross-channel attention is employed in a bottleneck further to study long-distance dependencies within images and channel relationships. To enhance accuracy, a Combined Attention Module (CAM) with Discrete Cosine Transform (DCT) orthogonality for channel weighting and convolutional element-wise multiplication is introduced for spatial weighting in the decoder. Attention gates (AG) are added in the skip connection to highlight the foreground while suppressing irrelevant background information. The proposed network is evaluated on three authoritative MRI brain tumor benchmarks and a proprietary dataset, and it achieves a 94.4% Dice score on the BraTS 2020 dataset, thereby establishing multiple new state-of-the-art benchmarks. The code is available here: https://github.com/WendyWAAAAANG/A4-Unet.

cs.CV↗

arXiv2024

LaMoD: Latent Motion Diffusion Model For Myocardial Strain Generation

Motion and deformation analysis of cardiac magnetic resonance (CMR) imaging videos is crucial for assessing myocardial strain of patients with abnormal heart functions. Recent advances in deep learning-based image registration algorithms have shown promising results in predicting motion fields from routinely acquired CMR sequences. However, their accuracy often diminishes in regions with subtle appearance changes, with errors propagating over time. Advanced imaging techniques, such as displacement encoding with stimulated echoes (DENSE) CMR, offer highly accurate and reproducible motion data but require additional image acquisition, which poses challenges in busy clinical flows. In this paper, we introduce a novel Latent Motion Diffusion model (LaMoD) to predict highly accurate DENSE motions from standard CMR videos. More specifically, our method first employs an encoder from a pre-trained registration network that learns latent motion features (also considered as deformation-based shape features) from image sequences. Supervised by the ground-truth motion provided by DENSE, LaMoD then leverages a probabilistic latent diffusion model to reconstruct accurate motion from these extracted features. Experimental results demonstrate that our proposed method, LaMoD, significantly improves the accuracy of motion analysis in standard CMR images; hence improving myocardial strain analysis in clinical settings for cardiac patients. Our code is publicly available at https://github.com/jr-xing/LaMoD.

cs.CV↗

arXiv2024

GenMAC: Compositional Text-to-Video Generation with Multi-Agent Collaboration

Text-to-video generation models have shown significant progress in the recent years. However, they still struggle with generating complex dynamic scenes based on compositional text prompts, such as attribute binding for multiple objects, temporal dynamics associated with different objects, and interactions between objects. Our key motivation is that complex tasks can be decomposed into simpler ones, each handled by a role-specialized MLLM agent. Multiple agents can collaborate together to achieve collective intelligence for complex goals. We propose GenMAC, an iterative, multi-agent framework that enables compositional text-to-video generation. The collaborative workflow includes three stages: Design, Generation, and Redesign, with an iterative loop between the Generation and Redesign stages to progressively verify and refine the generated videos. The Redesign stage is the most challenging stage that aims to verify the generated videos, suggest corrections, and redesign the text prompts, frame-wise layouts, and guidance scales for the next iteration of generation. To avoid hallucination of a single MLLM agent, we decompose this stage to four sequentially-executed MLLM-based agents: verification agent, suggestion agent, correction agent, and output structuring agent. Furthermore, to tackle diverse scenarios of compositional text-to-video generation, we design a self-routing mechanism to adaptively select the proper correction agent from a collection of correction agents each specialized for one scenario. Extensive experiments demonstrate the effectiveness of GenMAC, achieving state-of-the art performance in compositional text-to-video generation.

cs.CV↗

arXiv2024

ASIGN: An Anatomy-aware Spatial Imputation Graphic Network for 3D Spatial Transcriptomics

Spatial transcriptomics (ST) is an emerging technology that enables medical computer vision scientists to automatically interpret the molecular profiles underlying morphological features. Currently, however, most deep learning-based ST analyses are limited to two-dimensional (2D) sections, which can introduce diagnostic errors due to the heterogeneity of pathological tissues across 3D sections. Expanding ST to three-dimensional (3D) volumes is challenging due to the prohibitive costs; a 2D ST acquisition already costs over 50 times more than whole slide imaging (WSI), and a full 3D volume with 10 sections can be an order of magnitude more expensive. To reduce costs, scientists have attempted to predict ST data directly from WSI without performing actual ST acquisition. However, these methods typically yield unsatisfying results. To address this, we introduce a novel problem setting: 3D ST imputation using 3D WSI histology sections combined with a single 2D ST slide. To do so, we present the Anatomy-aware Spatial Imputation Graph Network (ASIGN) for more precise, yet affordable, 3D ST modeling. The ASIGN architecture extends existing 2D spatial relationships into 3D by leveraging cross-layer overlap and similarity-based expansion. Moreover, a multi-level spatial attention graph network integrates features comprehensively across different data sources. We evaluated ASIGN on three public spatial transcriptomics datasets, with experimental results demonstrating that ASIGN achieves state-of-the-art performance on both 2D and 3D scenarios. Code is available at https://github.com/hrlblab/ASIGN.

cs.CV↗

arXiv2024

Personalized Multimodal Large Language Models: A Survey

Multimodal Large Language Models (MLLMs) have become increasingly important due to their state-of-the-art performance and ability to integrate multiple data modalities, such as text, images, and audio, to perform complex tasks with high accuracy. This paper presents a comprehensive survey on personalized multimodal large language models, focusing on their architecture, training methods, and applications. We propose an intuitive taxonomy for categorizing the techniques used to personalize MLLMs to individual users, and discuss the techniques accordingly. Furthermore, we discuss how such techniques can be combined or adapted when appropriate, highlighting their advantages and underlying rationale. We also provide a succinct summary of personalization tasks investigated in existing research, along with the evaluation metrics commonly used. Additionally, we summarize the datasets that are useful for benchmarking personalized MLLMs. Finally, we outline critical open challenges. This survey aims to serve as a valuable resource for researchers and practitioners seeking to understand and advance the development of personalized multimodal large language models.

cs.CV↗

arXiv2024

MASP: Scalable GNN-based Planning for Multi-Agent Navigation

We investigate multi-agent navigation tasks, where multiple agents need to reach initially unassigned goals in a limited time. Classical planning-based methods suffer from expensive computation overhead at each step and offer limited expressiveness for complex cooperation strategies. In contrast, reinforcement learning (RL) has recently become a popular approach for addressing this issue. However, RL struggles with low data efficiency and cooperation when directly exploring (nearly) optimal policies in a large exploration space, especially with an increased number of agents(e.g., 10+ agents) or in complex environments (e.g., 3-D simulators). In this paper, we propose the Multi-Agent Scalable Graph-based Planner (MASP), a goal-conditioned hierarchical planner for navigation tasks with a substantial number of agents in the decentralized setting. MASP employs a hierarchical framework to reduce space complexity by decomposing a large exploration space into multiple goal-conditioned subspaces, where a high-level policy assigns agents goals, and a low-level policy navigates agents toward designated goals. For agent cooperation and the adaptation to varying team sizes, we model agents and goals as graphs to better capture their relationship. The high-level policy, the Goal Matcher, leverages a graph-based Self-Encoder and Cross-Encoder to optimize goal assignment by updating the agent and the goal graphs. The low-level policy, the Coordinated Action Executor, introduces the Group Information Fusion to facilitate group division and extract agent relationships across groups, enhancing training efficiency for agent cooperation. The results demonstrate that MASP outperforms RL and planning-based baselines in task efficiency.

cs.LG↗

arXiv2024

Direct Measurement of Density Matrices via Dense Dual Bases

Efficient understanding of a quantum system fundamentally relies on the selection of observables. Pauli observables and mutually unbiased bases (MUBs) are widely used in practice and are often regarded as theoretically optimal for quantum state tomography (QST). However, Pauli observables require a large number of measurements for full-state tomography and do not permit direct measurement of density matrix elements with a constant number of observables. For MUBs, the existence of complete sets of $d+1$ bases in all dimensions remains unresolved, highlighting the need for alternative observables. In this work, we introduce Dense Dual Bases (DDB), a novel set of $2d$ observables specifically designed to enable the complete characterization of any $d$-dimensional quantum state. These observables offer two key advantages. First, they enable direct measurement of density matrix elements without auxiliary systems, allowing any element to be extracted using only three selected observables. Second, QST for unknown rank-$r$ density matrices--excluding only a negligible subset--can be achieved with $O(r \log d)$ observables, significantly improving measurement efficiency. As for circuit implementation, each observable is iteratively generated and can be efficiently decomposed into $O(n^4)$ elementary gates for an $n$-qubit system. These advances establish DDB as a practical and scalable alternative to traditional methods, offering promising opportunities to advance the efficiency and scalability of quantum system characterization.

quant-ph↗

arXiv2024

ΩSFormer: Dual-Modal Ω-like Super-Resolution Transformer Network for Cross-scale and High-accuracy Terraced Field Vectorization Extraction

Terraced field is a significant engineering practice for soil and water conservation (SWC). Terraced field extraction from remotely sensed imagery is the foundation for monitoring and evaluating SWC. This study is the first to propose a novel dual-modal Ω-like super-resolution Transformer network for intelligent TFVE, offering the following advantages: (1) reducing edge segmentation error from conventional multi-scale downsampling encoder, through fusing original high-resolution features with downsampling features at each step of encoder and leveraging a multi-head attention mechanism; (2) improving the accuracy of TFVE by proposing a Ω-like network structure, which fully integrates rich high-level features from both spectral and terrain data to form cross-scale super-resolution features; (3) validating an optimal fusion scheme for cross-modal and cross-scale (i.e., inconsistent spatial resolution between remotely sensed imagery and DEM) super-resolution feature extraction; (4) mitigating uncertainty between segmentation edge pixels by a coarse-to-fine and spatial topological semantic relationship optimization (STSRO) segmentation strategy; (5) leveraging contour vibration neural network to continuously optimize parameters and iteratively vectorize terraced fields from semantic segmentation results. Moreover, a DMRVD for deep-learning-based TFVE was created for the first time, which covers nine study areas in four provinces of China, with a total coverage area of 22441 square kilometers. To assess the performance of ΩSFormer, classic and SOTA networks were compared. The mIOU of ΩSFormer has improved by 0.165, 0.297 and 0.128 respectively, when compared with best accuracy single-modal remotely sensed imagery, single-modal DEM and dual-modal result.

cs.CV↗

arXiv2024

LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantization

Visual Autoregressive (VAR) has emerged as a promising approach in image generation, offering competitive potential and performance comparable to diffusion-based models. However, current AR-based visual generation models require substantial computational resources, limiting their applicability on resource-constrained devices. To address this issue, we conducted analysis and identified significant redundancy in three dimensions of the VAR model: (1) the attention map, (2) the attention outputs when using classifier free guidance, and (3) the data precision. Correspondingly, we proposed efficient attention mechanism and low-bit quantization method to enhance the efficiency of VAR models while maintaining performance. With negligible performance lost (less than 0.056 FID increase), we could achieve 85.2% reduction in attention computation, 50% reduction in overall memory and 1.5x latency reduction. To ensure deployment feasibility, we developed efficient training-free compression techniques and analyze the deployment feasibility and efficiency gain of each technique.

cs.CV↗

arXiv2024

Glo-In-One-v2: Holistic Identification of Glomerular Cells, Tissues, and Lesions in Human and Mouse Histopathology

Segmenting glomerular intraglomerular tissue and lesions traditionally depends on detailed morphological evaluations by expert nephropathologists, a labor-intensive process susceptible to interobserver variability. Our group previously developed the Glo-In-One toolkit for integrated detection and segmentation of glomeruli. In this study, we leverage the Glo-In-One toolkit to version 2 with fine-grained segmentation capabilities, curating 14 distinct labels for tissue regions, cells, and lesions across a dataset of 23,529 annotated glomeruli across human and mouse histopathology data. To our knowledge, this dataset is among the largest of its kind to date.In this study, we present a single dynamic head deep learning architecture designed to segment 14 classes within partially labeled images of human and mouse pathology data. Our model was trained using a training set derived from 368 annotated kidney whole-slide images (WSIs) to identify 5 key intraglomerular tissues covering Bowman's capsule, glomerular tuft, mesangium, mesangial cells, and podocytes. Additionally, the network segments 9 glomerular lesion classes including adhesion, capsular drop, global sclerosis, hyalinosis, mesangial lysis, microaneurysm, nodular sclerosis, mesangial expansion, and segmental sclerosis. The glomerulus segmentation model achieved a decent performance compared with baselines, and achieved a 76.5 % average Dice Similarity Coefficient (DSC). Additional, transfer learning from rodent to human for glomerular lesion segmentation model has enhanced the average segmentation accuracy across different types of lesions by more than 3 %, as measured by Dice scores. The Glo-In-One-v2 model and trained weight have been made publicly available at https: //github.com/hrlblab/Glo-In-One_v2.

eess.IV↗

arXiv2024

Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study

Teaching to improve student models (e.g., knowledge distillation) is an extensively studied methodology in LLMs. However, for humans, teaching improves not only students but also teachers, by fostering more rigorous and clear reasoning as well as knowledge building. We ask: Can LLMs also learn by teaching (LbT) for better reasoning? If the answer is yes, we can potentially unlock the possibility of continuously advancing the models without solely relying on human-produced data or stronger models. In this paper, we provide a preliminary exploration on this question. We show that LbT ideas can be incorporated into existing LLM training/prompting pipelines and bring improvements. Specifically, we design three methods, each mimicking one of the three levels of LbT: observing students' feedback, learning from the feedback, and learning iteratively, with the goals of improving answer accuracy without training or improving models' inherent capability with fine-tuning. We reveal some findings: (1) Teaching materials that make it easier for students to learn have clearer and more accurate logic when using in-context learning as the student's "learning" method; (2) Weak-to-strong generalization: LbT might help improve strong models by teaching weak models; (3) Diversity in students might help: teaching multiple students could be better than teaching one student or the teacher itself. We hope that our exploration can inspire future research on LbT and more broadly adopting the advanced techniques in education to improve LLMs. The code and website are at https://github.com/imagination-research/lbt and https://sites.google.com/view/llm-learning-by-teaching.

cs.CL↗

arXiv2024

Continual SFT Matches Multimodal RLHF with Negative Supervision

Multimodal RLHF usually happens after supervised finetuning (SFT) stage to continually improve vision-language models' (VLMs) comprehension. Conventional wisdom holds its superiority over continual SFT during this preference alignment stage. In this paper, we observe that the inherent value of multimodal RLHF lies in its negative supervision, the logit of the rejected responses. We thus propose a novel negative supervised finetuning (nSFT) approach that fully excavates these information resided. Our nSFT disentangles this negative supervision in RLHF paradigm, and continually aligns VLMs with a simple SFT loss. This is more memory efficient than multimodal RLHF where 2 (e.g., DPO) or 4 (e.g., PPO) large VLMs are strictly required. The effectiveness of nSFT is rigorously proved by comparing it with various multimodal RLHF approaches, across different dataset sources, base VLMs and evaluation metrics. Besides, fruitful of ablations are provided to support our hypothesis. We hope this paper will stimulate further research to properly align large vision language models.

cs.LG↗

arXiv2024

Towards Accurate and Efficient Sub-8-Bit Integer Training

Neural network training is a memory- and compute-intensive task. Quantization, which enables low-bitwidth formats in training, can significantly mitigate the workload. To reduce quantization error, recent methods have developed new data formats and additional pre-processing operations on quantizers. However, it remains quite challenging to achieve high accuracy and efficiency simultaneously. In this paper, we explore sub-8-bit integer training from its essence of gradient descent optimization. Our integer training framework includes two components: ShiftQuant to realize accurate gradient estimation, and L1 normalization to smoothen the loss landscape. ShiftQuant attains performance that approaches the theoretical upper bound of group quantization. Furthermore, it liberates group quantization from inefficient memory rearrangement. The L1 normalization facilitates the implementation of fully quantized normalization layers with impressive convergence accuracy. Our method frees sub-8-bit integer training from pre-processing and supports general devices. This framework achieves negligible accuracy loss across various neural networks and tasks ($0.92\%$ on 4-bit ResNets, $0.61\%$ on 6-bit Transformers). The prototypical implementation of ShiftQuant achieves more than $1.85\times/15.3\%$ performance improvement on CPU/GPU compared to its FP16 counterparts, and $33.9\%$ resource consumption reduction on FPGA than the FP16 counterparts. The proposed fully-quantized L1 normalization layers achieve more than $35.54\%$ improvement in throughout on CPU compared to traditional L2 normalization layers. Moreover, theoretical analysis verifies the advancement of our method.

cs.LG↗

arXiv2024

WelQrate: Defining the Gold Standard in Small Molecule Drug Discovery Benchmarking

While deep learning has revolutionized computer-aided drug discovery, the AI community has predominantly focused on model innovation and placed less emphasis on establishing best benchmarking practices. We posit that without a sound model evaluation framework, the AI community's efforts cannot reach their full potential, thereby slowing the progress and transfer of innovation into real-world drug discovery. Thus, in this paper, we seek to establish a new gold standard for small molecule drug discovery benchmarking, WelQrate. Specifically, our contributions are threefold: WelQrate Dataset Collection - we introduce a meticulously curated collection of 9 datasets spanning 5 therapeutic target classes. Our hierarchical curation pipelines, designed by drug discovery experts, go beyond the primary high-throughput screen by leveraging additional confirmatory and counter screens along with rigorous domain-driven preprocessing, such as Pan-Assay Interference Compounds (PAINS) filtering, to ensure the high-quality data in the datasets; WelQrate Evaluation Framework - we propose a standardized model evaluation framework considering high-quality datasets, featurization, 3D conformation generation, evaluation metrics, and data splits, which provides a reliable benchmarking for drug discovery experts conducting real-world virtual screening; Benchmarking - we evaluate model performance through various research questions using the WelQrate dataset collection, exploring the effects of different models, dataset quality, featurization methods, and data splitting strategies on the results. In summary, we recommend adopting our proposed WelQrate as the gold standard in small molecule drug discovery benchmarking. The WelQrate dataset collection, along with the curation codes, and experimental scripts are all publicly available at WelQrate.org.

cs.LG↗

arXiv2024

Dynamics of relativistic vortex electrons in external laser fields

Investigating the interactions of vortex electrons with electromagnetic fields is crucial for advancing particle acceleration techniques, scattering theory in background fields, and developing novel electron beams for material diagnostics. In this work, we systematically study the dynamics of relativistic vortex electrons during their head-on collisions with linearly polarized (LP) and circularly polarized (CP) laser pulses, as well as their superposition. We develop a theoretical framework using Volkov-Bessel wave functions to describe the spatiotemporal characteristics of vortex electrons in these external fields. We show that the beam center of the vortex electron follows the classical trajectory of a point-charge electron while maintaining the transverse structure of both vortex eigenstates and superposition states. Specifically, CP laser pulses cause the beam center to rotate, while LP laser pulses induce a lateral shift. The combined effect of LP and CP laser pulses in a two-mode field results in a twisted spiral pattern. Our findings demonstrate the potential for versatile control of vortex electron beams using various laser modes, providing a foundation for future experimental and theoretical studies. This work serves as a benchmark reference for investigations into the manipulation of vortex electron beams using more realistic laser or other types of external fields.

physics.optics↗