arXiv subjects

Le Zhang

Publications and source records attributed to Le Zhang.

At least 127 records · Page 7Linked to original sources

arXiv2025

WiFi CSI Based Temporal Activity Detection via Dual Pyramid Network

We address the challenge of WiFi-based temporal activity detection and propose an efficient Dual Pyramid Network that integrates Temporal Signal Semantic Encoders and Local Sensitive Response Encoders. The Temporal Signal Semantic Encoder splits feature learning into high and low-frequency components, using a novel Signed Mask-Attention mechanism to emphasize important areas and downplay unimportant ones, with the features fused using ContraNorm. The Local Sensitive Response Encoder captures fluctuations without learning. These feature pyramids are then combined using a new cross-attention fusion mechanism. We also introduce a dataset with over 2,114 activity segments across 553 WiFi CSI samples, each lasting around 85 seconds. Extensive experiments show our method outperforms challenging baselines.

cs.CV↗

arXiv2025

Electric-field switchable chirality in rhombohedral graphene Chern insulators stabilized by tungsten diselenide

Chern insulators host topologically protected chiral edge currents with quantized conductance characterized by their Chern number. Switching the chirality of a Chern insulator, namely, the direction of the edge current, is highly challenging due to topologically forbidden backscattering but is of considerable importance for the design of topological devices. Nevertheless, this can be achieved by reversing the sign of the Chern number through a topological phase transition. Here, we report electrically switchable chirality in rhombohedral multilayer graphene-based Chern insulators. By introducing moire superlattices in rhombohedral heptalayer graphene, we observed a cascade of topological phase transitions at quarter electron filling of a moire band with the Chern number tunable from -1, 1 to 2. Furthermore, integrating monolayer tungsten diselenide at the moireless interface of rhombohedral decalayer graphene/h-BN superlattices stabilizes the Chern insulators, enabling quantized anomalous Hall resistance of h/2e^2. Remarkably, the Chern number can be switched from -1 to 2 using displacement fields. Our work establishes rhombohedral multilayer graphene moire superlattices as a versatile platform for topological engineering, with switchable chirality offering significant promise for integrating chiral edge currents into topological electronic circuits.

cond-mat.mes-hall↗

arXiv2025

VisMin: Visual Minimal-Change Understanding

Fine-grained understanding of objects, attributes, and relationships between objects is crucial for visual-language models (VLMs). Existing benchmarks primarily focus on evaluating VLMs' capability to distinguish between two very similar captions given an image. In this paper, we introduce a new, challenging benchmark termed Visual Minimal-Change Understanding (VisMin), which requires models to predict the correct image-caption match given two images and two captions. The image pair and caption pair contain minimal changes, i.e., only one aspect changes at a time from among the following: object, attribute, count, and spatial relation. These changes test the models' understanding of objects, attributes (such as color, material, shape), counts, and spatial relationships between objects. We built an automatic framework using large language models and diffusion models, followed by a rigorous 4-step verification process by human annotators. Empirical experiments reveal that current VLMs exhibit notable deficiencies in understanding spatial relationships and counting abilities. We also generate a large-scale training dataset to finetune CLIP and Idefics2, showing significant improvements in fine-grained understanding across benchmarks and in CLIP's general image-text alignment. We release all resources, including the benchmark, training data, and finetuned model checkpoints, at https://vismin.net/.

cs.CV↗

arXiv2025

The diverse physical origins of stars in the dynamically hot bulge: CALIFA vs. IllustrisTNG

We compare the internal stellar structures of central galaxies in the TNG50 and TNG100 simulations and field galaxies in the CALIFA survey. The luminosity fractions of the dynamically cold, warm, and hot components in both TNG50 and TNG100 galaxies exhibit general consistency with those observed in CALIFA galaxies. For example, they all exhibit a minimum luminosity fraction of the dynamically hot component in galaxies with intermediate stellar masses, and the morphology of each orbital component in the TNG50 and TNG100 galaxies closely resembles that found in the CALIFA galaxies. We therefore use the simulations to quantify the physical origins of the different components, focusing on the dynamically hot component in TNG50. We identify three primary regimes and thus physical processes: (1) in low mass galaxies that have not experienced major mergers, stars are born with a wide range of circularity distributions and have remained relatively unchanged until the present day. Consequently, hot stars in such galaxies at redshift 0 are predominantly born hot. (2) In higher mass galaxies lacking major mergers, most stars are initially born cold but are subsequently heated through secular evolution. (3) In galaxies across the entire mass range, mergers, if they occurred, significantly increased the hot orbital fraction. As a result, the dynamically hot bulge within $R_e$ of present-day galaxies does not indicate their past merger histories; instead, the hot stars in the outer regions are mostly heated or accreted by mergers, thus indicating galaxy merger history. The massive galaxies are initially born with cold, rotationally supported structures, consistent with recent observations from the James Webb Space Telescope (JWST) regarding high-redshift galaxies.

astro-ph.GA↗

arXiv2025

From Dense to Sparse: Event Response for Enhanced Residential Load Forecasting

Residential load forecasting (RLF) is crucial for resource scheduling in power systems. Most existing methods utilize all given load records (dense data) to indiscriminately extract the dependencies between historical and future time series. However, there exist important regular patterns residing in the event-related associations among different appliances (sparse knowledge), which have yet been ignored. In this paper, we propose an Event-Response Knowledge Guided approach (ERKG) for RLF by incorporating the estimation of electricity usage events for different appliances, mining event-related sparse knowledge from the load series. With ERKG, the event-response estimation enables portraying the electricity consumption behaviors of residents, revealing regular variations in appliance operational states. To be specific, ERKG consists of knowledge extraction and guidance: i) a forecasting model is designed for the electricity usage events by estimating appliance operational states, aiming to extract the event-related sparse knowledge; ii) a novel knowledge-guided mechanism is established by fusing such state estimates of the appliance events into the RLF model, which can give particular focuses on the patterns of users' electricity consumption behaviors. Notably, ERKG can flexibly serve as a plug-in module to boost the capability of existing forecasting models by leveraging event response. In numerical experiments, extensive comparisons and ablation studies have verified the effectiveness of our ERKG, e.g., over 8% MAE can be reduced on the tested state-of-the-art forecasting models.

cs.LG↗

arXiv2025

CHAT: Beyond Contrastive Graph Transformer for Link Prediction in Heterogeneous Networks

Link prediction in heterogeneous networks is crucial for understanding the intricacies of network structures and forecasting their future developments. Traditional methodologies often face significant obstacles, including over-smoothing-wherein the excessive aggregation of node features leads to the loss of critical structural details-and a dependency on human-defined meta-paths, which necessitate extensive domain knowledge and can be inherently restrictive. These limitations hinder the effective prediction and analysis of complex heterogeneous networks. In response to these challenges, we propose the Contrastive Heterogeneous grAph Transformer (CHAT). CHAT introduces a novel sampling-based graph transformer technique that selectively retains nodes of interest, thereby obviating the need for predefined meta-paths. The method employs an innovative connection-aware transformer to encode node sequences and their interconnections with high fidelity, guided by a dual-faceted loss function specifically designed for heterogeneous network link prediction. Additionally, CHAT incorporates an ensemble link predictor that synthesizes multiple samplings to achieve enhanced prediction accuracy. We conducted comprehensive evaluations of CHAT using three distinct drug-target interaction (DTI) datasets. The empirical results underscore CHAT's superior performance, outperforming both general-task approaches and models specialized in DTI prediction. These findings substantiate the efficacy of CHAT in addressing the complex problem of link prediction in heterogeneous networks.

cs.CE↗

arXiv2025

DDEvENet: Evidence-based Ensemble Learning for Uncertainty-aware Brain Parcellation Using Diffusion MRI

In this study, we developed an Evidence-based Ensemble Neural Network, namely EVENet, for anatomical brain parcellation using diffusion MRI. The key innovation of EVENet is the design of an evidential deep learning framework to quantify predictive uncertainty at each voxel during a single inference. To do so, we design an evidence-based ensemble learning framework for uncertainty-aware parcellation to leverage the multiple dMRI parameters derived from diffusion MRI. Using EVENet, we obtained accurate parcellation and uncertainty estimates across different datasets from healthy and clinical populations and with different imaging acquisitions. The overall network includes five parallel subnetworks, where each is dedicated to learning the FreeSurfer parcellation for a certain diffusion MRI parameter. An evidence-based ensemble methodology is then proposed to fuse the individual outputs. We perform experimental evaluations on large-scale datasets from multiple imaging sources, including high-quality diffusion MRI data from healthy adults and clinically diffusion MRI data from participants with various brain diseases (schizophrenia, bipolar disorder, attention-deficit/hyperactivity disorder, Parkinson's disease, cerebral small vessel disease, and neurosurgical patients with brain tumors). Compared to several state-of-the-art methods, our experimental results demonstrate highly improved parcellation accuracy across the multiple testing datasets despite the differences in dMRI acquisition protocols and health conditions. Furthermore, thanks to the uncertainty estimation, our EVENet approach demonstrates a good ability to detect abnormal brain regions in patients with lesions, enhancing the interpretability and reliability of the segmentation results.

eess.IV↗

arXiv2024

CharacterBench: Benchmarking Character Customization of Large Language Models

Character-based dialogue (aka role-playing) enables users to freely customize characters for interaction, which often relies on LLMs, raising the need to evaluate LLMs' character customization capability. However, existing benchmarks fail to ensure a robust evaluation as they often only involve a single character category or evaluate limited dimensions. Moreover, the sparsity of character features in responses makes feature-focused generative evaluation both ineffective and inefficient. To address these issues, we propose CharacterBench, the largest bilingual generative benchmark, with 22,859 human-annotated samples covering 3,956 characters from 25 detailed character categories. We define 11 dimensions of 6 aspects, classified as sparse and dense dimensions based on whether character features evaluated by specific dimensions manifest in each response. We enable effective and efficient evaluation by crafting tailored queries for each dimension to induce characters' responses related to specific dimensions. Further, we develop CharacterJudge model for cost-effective and stable evaluations. Experiments show its superiority over SOTA automatic judges (e.g., GPT-4) and our benchmark's potential to optimize LLMs' character customization. Our repository is at https://github.com/thu-coai/CharacterBench.

cs.CL↗

arXiv2024

Efficient Quantization-Aware Training on Segment Anything Model in Medical Images and Its Deployment

Medical image segmentation is a critical component of clinical practice, and the state-of-the-art MedSAM model has significantly advanced this field. Nevertheless, critiques highlight that MedSAM demands substantial computational resources during inference. To address this issue, the CVPR 2024 MedSAM on Laptop Challenge was established to find an optimal balance between accuracy and processing speed. In this paper, we introduce a quantization-aware training pipeline designed to efficiently quantize the Segment Anything Model for medical images and deploy it using the OpenVINO inference engine. This pipeline optimizes both training time and disk storage. Our experimental results confirm that this approach considerably enhances processing speed over the baseline, while still achieving an acceptable accuracy level. The training script, inference script, and quantized model are publicly accessible at https://github.com/AVC2-UESTC/QMedSAM.

cs.CV↗

arXiv2024

Towards Open-Vocabulary Video Semantic Segmentation

Semantic segmentation in videos has been a focal point of recent research. However, existing models encounter challenges when faced with unfamiliar categories. To address this, we introduce the Open Vocabulary Video Semantic Segmentation (OV-VSS) task, designed to accurately segment every pixel across a wide range of open-vocabulary categories, including those that are novel or previously unexplored. To enhance OV-VSS performance, we propose a robust baseline, OV2VSS, which integrates a spatial-temporal fusion module, allowing the model to utilize temporal relationships across consecutive frames. Additionally, we incorporate a random frame enhancement module, broadening the model's understanding of semantic context throughout the entire video sequence. Our approach also includes video text encoding, which strengthens the model's capability to interpret textual information within the video context. Comprehensive evaluations on benchmark datasets such as VSPW and Cityscapes highlight OV-VSS's zero-shot generalization capabilities, especially in handling novel categories. The results validate OV2VSS's effectiveness, demonstrating improved performance in semantic segmentation tasks across diverse video datasets.

cs.MM↗

arXiv2024

Optimizing Redshift Distribution Inference through Joint Self-Calibration and Clustering-Redshift Synergy

Accurately characterizing the true redshift (true-$z$) distribution of a photometric redshift (photo-$z$) sample is critical for cosmological analyses in imaging surveys. Clustering-based techniques, which include clustering-redshift (CZ) and self-calibration (SC) methods--depending on whether external spectroscopic data are used--offer powerful tools for this purpose. In this study, we explore the joint inference of the true-$z$ distribution by combining SC and CZ (denoted as SC+CZ). We derive simple multiplicative update rules to perform the joint inference. By incorporating appropriate error weighting and an additional weighting function, our method shows significant improvement over previous algorithms. We validate our approach using a DES Y3 mock catalog. The true-$z$ distribution estimated through the combined SC+CZ method is generally more accurate than using SC or CZ alone. To account for the different constraining powers of these methods, we assign distinct weights to the SC and CZ contributions. The optimal weights, which minimize the distribution error, depend on the relative constraining strength of the SC and CZ data. Specifically, for a spectroscopic redshift sample that amounts to 1% of the photo-$z$ sample, the optimal combination reduces the total error by 20% (40%) compared to using CZ (SC) alone, and it keeps the bias in mean redshift [$Δ\bar{z} / (1 + z) $] at the level of 0.3%. Furthermore, when CZ data is only available in the low-$z$ range and the high-$z$ range relies solely on SC data, SC+CZ enables consistent estimation of the true-$z$ distribution across the entire redshift range. Our findings demonstrate that SC+CZ is an effective tool for constraining the true-$z$ distribution, paving the way for clustering-based methods to be applied at $z\gtrsim 1$.

astro-ph.CO↗

arXiv2024

Deep Learning Ensemble for Predicting Diabetic Macular Edema Onset Using Ultra-Wide Field Color Fundus Image

Diabetic macular edema (DME) is a severe complication of diabetes, characterized by thickening of the central portion of the retina due to accumulation of fluid. DME is a significant and common cause of visual impairment in diabetic patients. Center-involved DME (ci-DME) is the highest risk form of disease because fluid extends close to the fovea which is responsible for sharp central vision. Earlier diagnosis or prediction of ci-DME may improve treatment outcomes. Here, we propose an ensemble method to predict ci-DME onset within a year, after using synthetic ultra-wide field color fundus photography (UWF-CFP) images provided by the DIAMOND Challenge during development. We adopted a variety of baseline state-of-the-art classification networks including ResNet, DenseNet, EfficientNet, and VGG with the aim of enhancing model robustness. The best performing models were Densenet-121, Resnet-152 and EfficientNet-b7, and these were assembled into a definitive predictive model. The final ensemble model demonstrates a strong performance with an Area Under Curve (AUC) of 0.7017, an F1 score of 0.6512, and an Expected Calibration Error (ECE) of 0.2057 when deployed on the synthetic test dataset. Results from our ensemble model were superior/comparable to previous recorded results in highly curated settings using conventional fundus photography/ultra-wide field fundus photography. Optimal sensitivity in previous studies (using humans or computers to diagnose) ranges from 67.3%-98%, specificity from 47.8%-80%. Therefore, our method can be used safely and effectively in a range of settings may facilitate earlier diagnosis, better treatment decisions, and improved prognostication in ci-DME.

eess.IV↗

arXiv2024

Exploring Frequency-Inspired Optimization in Transformer for Efficient Single Image Super-Resolution

Transformer-based methods have exhibited remarkable potential in single image super-resolution (SISR) by effectively extracting long-range dependencies. However, most of the current research in this area has prioritized the design of transformer blocks to capture global information, while overlooking the importance of incorporating high-frequency priors, which we believe could be beneficial. In our study, we conducted a series of experiments and found that transformer structures are more adept at capturing low-frequency information, but have limited capacity in constructing high-frequency representations when compared to their convolutional counterparts. Our proposed solution, the cross-refinement adaptive feature modulation transformer (CRAFT), integrates the strengths of both convolutional and transformer structures. It comprises three key components: the high-frequency enhancement residual block (HFERB) for extracting high-frequency information, the shift rectangle window attention block (SRWAB) for capturing global information, and the hybrid fusion block (HFB) for refining the global representation. To tackle the inherent intricacies of transformer structures, we introduce a frequency-guided post-training quantization (PTQ) method aimed at enhancing CRAFT's efficiency. These strategies incorporate adaptive dual clipping and boundary refinement. To further amplify the versatility of our proposed approach, we extend our PTQ strategy to function as a general quantization method for transformer-based SISR techniques. Our experimental findings showcase CRAFT's superiority over current state-of-the-art methods, both in full-precision and quantization scenarios. These results underscore the efficacy and universality of our PTQ strategy. The source code is available at: https://github.com/AVC2-UESTC/Frequency-Inspired-Optimization-for-EfficientSR.git.

cs.CV↗

arXiv2024

Introducing Milabench: Benchmarking Accelerators for AI

AI workloads, particularly those driven by deep learning, are introducing novel usage patterns to high-performance computing (HPC) systems that are not comprehensively captured by standard HPC benchmarks. As one of the largest academic research centers dedicated to deep learning, Mila identified the need to develop a custom benchmarking suite to address the diverse requirements of its community, which consists of over 1,000 researchers. This report introduces Milabench, the resulting benchmarking suite. Its design was informed by an extensive literature review encompassing 867 papers, as well as surveys conducted with Mila researchers. This rigorous process led to the selection of 26 primary benchmarks tailored for procurement evaluations, alongside 16 optional benchmarks for in-depth analysis. We detail the design methodology, the structure of the benchmarking suite, and provide performance evaluations using GPUs from NVIDIA, AMD, and Intel. The Milabench suite is open source and can be accessed at github.com/mila-iqia/milabench.

cs.LG↗

arXiv2024

EEG-DIF: Early Warning of Epileptic Seizures through Generative Diffusion Model-based Multi-channel EEG Signals Forecasting

Multi-channel EEG signals are commonly used for the diagnosis and assessment of diseases such as epilepsy. Currently, various EEG diagnostic algorithms based on deep learning have been developed. However, most research efforts focus solely on diagnosing and classifying current signal data but do not consider the prediction of future trends for early warning. Additionally, since multi-channel EEG can be essentially regarded as the spatio-temporal signal data received by detectors at different locations in the brain, how to construct spatio-temporal information representations of EEG signals to facilitate future trend prediction for multi-channel EEG becomes an important problem. This study proposes a multi-signal prediction algorithm based on generative diffusion models (EEG-DIF), which transforms the multi-signal forecasting task into an image completion task, allowing for comprehensive representation and learning of the spatio-temporal correlations and future developmental patterns of multi-channel EEG signals. Here, we employ a publicly available epilepsy EEG dataset to construct and validate the EEG-DIF. The results demonstrate that our method can accurately predict future trends for multi-channel EEG signals simultaneously. Furthermore, the early warning accuracy for epilepsy seizures based on the generated EEG data reaches 0.89. In general, EEG-DIF provides a novel approach for characterizing multi-channel EEG signals and an innovative early warning algorithm for epilepsy seizures, aiding in optimizing and enhancing the clinical diagnosis process. The code is available at https://github.com/JZK00/EEG-DIF.

eess.SP↗

arXiv2024

TV-SAM: Increasing Zero-Shot Segmentation Performance on Multimodal Medical Images Using GPT-4 Generated Descriptive Prompts Without Human Annotation

This study presents a novel multimodal medical image zero-shot segmentation algorithm named the text-visual-prompt segment anything model (TV-SAM) without any manual annotations. The TV-SAM incorporates and integrates the large language model GPT-4, the vision language model GLIP, and the SAM to autonomously generate descriptive text prompts and visual bounding box prompts from medical images, thereby enhancing the SAM's capability for zero-shot segmentation. Comprehensive evaluations are implemented on seven public datasets encompassing eight imaging modalities to demonstrate that TV-SAM can effectively segment unseen targets across various modalities without additional training. TV-SAM significantly outperforms SAM AUTO and GSAM, closely matching the performance of SAM BBOX with gold standard bounding box prompts and surpasses the state-of-the-art methods on specific datasets such as ISIC and WBC. The study indicates that TV-SAM serves as an effective multimodal medical image zero-shot segmentation algorithm, highlighting the significant contribution of GPT-4 to zero-shot segmentation. By integrating foundational models such as GPT-4, GLIP, and SAM, the ability to address complex problems in specialized domains can be enhanced.

cs.CV↗

arXiv2024

ViT3D Alignment of LLaMA3: 3D Medical Image Report Generation

Automatic medical report generation (MRG), which aims to produce detailed text reports from medical images, has emerged as a critical task in this domain. MRG systems can enhance radiological workflows by reducing the time and effort required for report writing, thereby improving diagnostic efficiency. In this work, we present a novel approach for automatic MRG utilizing a multimodal large language model. Specifically, we employed the 3D Vision Transformer (ViT3D) image encoder introduced from M3D-CLIP to process 3D scans and use the Asclepius-Llama3-8B as the language model to generate the text reports by auto-regressive decoding. The experiment shows our model achieved an average Green score of 0.3 on the MRG task validation set and an average accuracy of 0.61 on the visual question answering (VQA) task validation set, outperforming the baseline model. Our approach demonstrates the effectiveness of the ViT3D alignment of LLaMA3 for automatic MRG and VQA tasks by tuning the model on a small dataset.

eess.IV↗

arXiv2024

Exotic hybrid pseudopotentials at finite temperature and chemical potential

Using gauge/gravity duality, we study the exotic hybrid pseudopotentials at finite temperature and chemical potential. The $Σ$ hybrid meson can be described by a model including an object called ``defect'' on a string linking the quark and antiquark. It was first proposed by Andreev and perfectly described the $Σ_u^-$ hybrid potential at zero temperature and chemical potential. In this paper, we would like to extend this model to finite chemical potential and compare the separate distance and pseudopotentials of $Σ_g^+ $ and $Σ_u^-$. Unlike $Σ_g^+$ ground state, the $Σ_u^-$ hybrid pseudopotentials no longer behave as Coulomb-like at short distances. In addition, temperature and chemical potential have a significant impact on the $Σ_u^-$ hybrid pseudopotentials. The screen distances and hybrid pseudopotentials of $Σ_u^-$ significantly decrease with the increase of temperature and chemical potential. At last, we draw the melting diagram of $Σ_g^+ $ and $Σ_u^-$ in the $T- μ$ plane, and confirm that the quark-antiquark pair in $Σ_u^-$ excited state is easier to melt than in $Σ_g^+$ ground state.

hep-ph↗