arXiv subjects

Le Zhang

Publications and source records attributed to Le Zhang.

At least 163 records · Page 9Linked to original sources

arXiv2024

NTIRE 2024 Challenge on Low Light Image Enhancement: Methods and Results

This paper reviews the NTIRE 2024 low light image enhancement challenge, highlighting the proposed solutions and results. The aim of this challenge is to discover an effective network design or solution capable of generating brighter, clearer, and visually appealing results when dealing with a variety of conditions, including ultra-high resolution (4K and beyond), non-uniform illumination, backlighting, extreme darkness, and night scenes. A notable total of 428 participants registered for the challenge, with 22 teams ultimately making valid submissions. This paper meticulously evaluates the state-of-the-art advancements in enhancing low-light images, reflecting the significant progress and creativity in this field.

cs.CV↗

arXiv2024

PASTA: Towards Flexible and Efficient HDR Imaging Via Progressively Aggregated Spatio-Temporal Alignment

Leveraging Transformer attention has led to great advancements in HDR deghosting. However, the intricate nature of self-attention introduces practical challenges, as existing state-of-the-art methods often demand high-end GPUs or exhibit slow inference speeds, especially for high-resolution images like 2K. Striking an optimal balance between performance and latency remains a critical concern. In response, this work presents PASTA, a novel Progressively Aggregated Spatio-Temporal Alignment framework for HDR deghosting. Our approach achieves effectiveness and efficiency by harnessing hierarchical representation during feature distanglement. Through the utilization of diverse granularities within the hierarchical structure, our method substantially boosts computational speed and optimizes the HDR imaging workflow. In addition, we explore within-scale feature modeling with local and global attention, gradually merging and refining them in a coarse-to-fine fashion. Experimental results showcase PASTA's superiority over current SOTA methods in both visual quality and performance metrics, accompanied by a substantial 3-fold (x3) increase in inference speed.

cs.CV↗

arXiv2024

Electronic ferroelectricity in monolayer graphene for multifunctional neuromorphic electronics

Ferroelectricity is intriguing for its spontaneous electric polarization, which is switchable by an external electric field. Expanding ferroelectric materials to two-dimensional limit will provide versatile applications for the development of next-generation nonvolatile devices. Conventional ferroelectricity requires the materials consisting of at least two constituent elements associated with polar crystalline structures. Monolayer graphene as an elementary two-dimensional material unlikely exhibits ferroelectric order due to its highly centrosymmetric hexagonal lattices. Nevertheless, two-dimensional moire superlattices offer a powerful way to engineer diverse electronic orders in non-polar materials. Here, we report the observations of electronic ferroelectricity in monolayer graphene by introducing asymmetric moire superlattice at the graphene/h-BN interface. Utilizing Hall measurements, the electric polarization is identified to stem from electron-hole dipoles, suggesting the electronic dynamics of the observed ferroelectricity. Standard polarization-electric field hysteresis loops, as well as unconventional multiple switchable polarization states, have been achieved. By in-situ comparing with control devices, we found that the electronic ferroelectricity in graphene moire systems is independent of layer number of graphene and the corresponding fine band structures. Furthermore, we demonstrate the applications of this ferroelectric moire structures in multi-state non-volatile data storage and the emulation of versatile synaptic behaviors, including short-term plasticity, long-term potentiation and long-term depression. This work not only enriches the fundamental understanding of ferroelectricity, but also demonstrates the promising applications of graphene in multi-state memories and neuromorphic computing.

cond-mat.mes-hall↗

arXiv2024

Vision Transformers with Hierarchical Attention

This paper tackles the high computational/space complexity associated with Multi-Head Self-Attention (MHSA) in vanilla vision transformers. To this end, we propose Hierarchical MHSA (H-MHSA), a novel approach that computes self-attention in a hierarchical fashion. Specifically, we first divide the input image into patches as commonly done, and each patch is viewed as a token. Then, the proposed H-MHSA learns token relationships within local patches, serving as local relationship modeling. Then, the small patches are merged into larger ones, and H-MHSA models the global dependencies for the small number of the merged tokens. At last, the local and global attentive features are aggregated to obtain features with powerful representation capacity. Since we only calculate attention for a limited number of tokens at each step, the computational load is reduced dramatically. Hence, H-MHSA can efficiently model global relationships among tokens without sacrificing fine-grained information. With the H-MHSA module incorporated, we build a family of Hierarchical-Attention-based Transformer Networks, namely HAT-Net. To demonstrate the superiority of HAT-Net in scene understanding, we conduct extensive experiments on fundamental vision tasks, including image classification, semantic segmentation, object detection, and instance segmentation. Therefore, HAT-Net provides a new perspective for vision transformers. Code and pretrained models are available at https://github.com/yun-liu/HAT-Net.

cs.CV↗

arXiv2024

RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection

Three-dimensional object detection is one of the key tasks in autonomous driving. To reduce costs in practice, low-cost multi-view cameras for 3D object detection are proposed to replace the expansive LiDAR sensors. However, relying solely on cameras is difficult to achieve highly accurate and robust 3D object detection. An effective solution to this issue is combining multi-view cameras with the economical millimeter-wave radar sensor to achieve more reliable multi-modal 3D object detection. In this paper, we introduce RCBEVDet, a radar-camera fusion 3D object detection method in the bird's eye view (BEV). Specifically, we first design RadarBEVNet for radar BEV feature extraction. RadarBEVNet consists of a dual-stream radar backbone and a Radar Cross-Section (RCS) aware BEV encoder. In the dual-stream radar backbone, a point-based encoder and a transformer-based encoder are proposed to extract radar features, with an injection and extraction module to facilitate communication between the two encoders. The RCS-aware BEV encoder takes RCS as the object size prior to scattering the point feature in BEV. Besides, we present the Cross-Attention Multi-layer Fusion module to automatically align the multi-modal BEV feature from radar and camera with the deformable attention mechanism, and then fuse the feature with channel and spatial fusion layers. Experimental results show that RCBEVDet achieves new state-of-the-art radar-camera fusion results on nuScenes and view-of-delft (VoD) 3D object detection benchmarks. Furthermore, RCBEVDet achieves better 3D detection results than all real-time camera-only and radar-camera 3D object detectors with a faster inference speed at 21~28 FPS. The source code will be released at https://github.com/VDIGPKU/RCBEVDet.

cs.CV↗

arXiv2024

Learning Triangular Distribution in Visual World

Convolution neural network is successful in pervasive vision tasks, including label distribution learning, which usually takes the form of learning an injection from the non-linear visual features to the well-defined labels. However, how the discrepancy between features is mapped to the label discrepancy is ambient, and its correctness is not guaranteed.To address these problems, we study the mathematical connection between feature and its label, presenting a general and simple framework for label distribution learning. We propose a so-called Triangular Distribution Transform (TDT) to build an injective function between feature and label, guaranteeing that any symmetric feature discrepancy linearly reflects the difference between labels. The proposed TDT can be used as a plug-in in mainstream backbone networks to address different label distribution learning tasks. Experiments on Facial Age Recognition, Illumination Chromaticity Estimation, and Aesthetics assessment show that TDT achieves on-par or better results than the prior arts.

cs.CV↗

arXiv2024

21-cm foreground removal using AI and frequency-difference technique

The deep learning technique has been employed in removing foreground contaminants from 21 cm intensity mapping, but its effectiveness is limited by the large dynamic range of the foreground amplitude. In this study, we develop a novel foreground removal technique grounded in U-Net networks. The essence of this technique lies in introducing an innovative data preprocessing step specifically, utilizing the temperature difference between neighboring frequency bands as input, which can substantially reduce the dynamic range of foreground amplitudes by approximately two orders of magnitude. This reduction proves to be highly advantageous for the U-Net foreground removal. We observe that the HI signal can be reliably recovered, as indicated by the cross-correlation power spectra showing unity agreement at the scale of $k < 0.3 h^{-1}$Mpc in the absence of instrumental effects. Moreover, accounting for the systematic beam effects, our reconstruction displays consistent auto-correlation and cross-correlation power spectrum ratios at the $1σ$ level across scales $k \lesssim 0.1 h^{-1}$Mpc, with only a 10% reduction observed in the cross-correlation power spectrum at $k\simeq0.2 h^{-1}$Mpc. The effects of redshift-space distortion are also reconstructed successfully, as evidenced by the quadrupole power spectra matching. In comparison, our method outperforms the traditional Principal Component Analysis method, which derived cross-correlation ratios are underestimated by around 60%. We simulated various white noise levels in the map and found that the mean cross-correlation ratio $\bar{R}_\mathrm{cross} \gtrsim 0.8$ when the level of the thermal noise is smaller than or equal to that of the HI signal. We conclude that the proposed frequency-difference technique can significantly enhance network performance by reducing the amplitude range of foregrounds and aiding in the prevention of HI loss.

astro-ph.CO↗

arXiv2024

Federated Learning Using Coupled Tensor Train Decomposition

Coupled tensor decomposition (CTD) can extract joint features from multimodal data in various applications. It can be employed for federated learning networks with data confidentiality. Federated CTD achieves data privacy protection by sharing common features and keeping individual features. However, traditional CTD schemes based on canonical polyadic decomposition (CPD) may suffer from low computational efficiency and heavy communication costs. Inspired by the efficient tensor train decomposition, we propose a coupled tensor train (CTT) decomposition for federated learning. The distributed coupled multi-way data are decomposed into a series of tensor trains with shared factors. In this way, we can extract common features of coupled modes while maintaining the different features of uncoupled modes. Thus the privacy preservation of information across different network nodes can be ensured. The proposed CTT approach is instantiated for two fundamental network structures, namely master-slave and decentralized networks. Experimental results on synthetic and real datasets demonstrate the superiority of the proposed schemes over existing methods in terms of both computational efficiency and communication rounds. In a classification task, experimental results show that the CTT-based federated learning achieves almost the same accuracy performance as that of the centralized counterpart.

cs.DC↗

arXiv2024

Holographic Schwinger Effect in Anisotropic Media

According to gauge/gravity correspondence, we study the holographic Schwinger effect within an anisotropic background. Firstly, the separate length of the particle-antiparticle pairs is computed within the context of an anisotropic background which is parameterized by dynamical exponent $ν$. It is found that the maximum separate length $x$ increases with the increase of dynamical exponent $ν$. By analyzing the potential energy, we find that the potential barrier increases with the dynamical exponent $ν$ at a small separate distance. This observation implies that the Schwinger effect within an anisotropic background is comparatively weaker when contrasted with its manifestation in an isotropic background.Finally, we also find that the Schwinger effect in the transverse direction is weakened compared to the parallel direction in the anisotropic background, which is consistent with the top-down model.

hep-th↗

arXiv2024

GSB: Group Superposition Binarization for Vision Transformer with Limited Training Samples

Vision Transformer (ViT) has performed remarkably in various computer vision tasks. Nonetheless, affected by the massive amount of parameters, ViT usually suffers from serious overfitting problems with a relatively limited number of training samples. In addition, ViT generally demands heavy computing resources, which limit its deployment on resource-constrained devices. As a type of model-compression method, model binarization is potentially a good choice to solve the above problems. Compared with the full-precision one, the model with the binarization method replaces complex tensor multiplication with simple bit-wise binary operations and represents full-precision model parameters and activations with only 1-bit ones, which potentially solves the problem of model size and computational complexity, respectively. In this paper, we investigate a binarized ViT model. Empirically, we observe that the existing binarization technology designed for Convolutional Neural Networks (CNN) cannot migrate well to a ViT's binarization task. We also find that the decline of the accuracy of the binary ViT model is mainly due to the information loss of the Attention module and the Value vector. Therefore, we propose a novel model binarization technique, called Group Superposition Binarization (GSB), to deal with these issues. Furthermore, in order to further improve the performance of the binarization model, we have investigated the gradient calculation procedure in the binarization process and derived more proper gradient calculation equations for GSB to reduce the influence of gradient mismatch. Then, the knowledge distillation technique is introduced to alleviate the performance degradation caused by model binarization. Analytically, model binarization can limit the parameters search space during parameter updates while training a model....

cs.CV↗

arXiv2023

ReliCD: A Reliable Cognitive Diagnosis Framework with Confidence Awareness

During the past few decades, cognitive diagnostics modeling has attracted increasing attention in computational education communities, which is capable of quantifying the learning status and knowledge mastery levels of students. Indeed, the recent advances in neural networks have greatly enhanced the performance of traditional cognitive diagnosis models through learning the deep representations of students and exercises. Nevertheless, existing approaches often suffer from the issue of overconfidence in predicting students' mastery levels, which is primarily caused by the unavoidable noise and sparsity in realistic student-exercise interaction data, severely hindering the educational application of diagnostic feedback. To address this, in this paper, we propose a novel Reliable Cognitive Diagnosis(ReliCD) framework, which can quantify the confidence of the diagnosis feedback and is flexible for different cognitive diagnostic functions. Specifically, we first propose a Bayesian method to explicitly estimate the state uncertainty of different knowledge concepts for students, which enables the confidence quantification of diagnostic feedback. In particular, to account for potential differences, we suggest modeling individual prior distributions for the latent variables of different ability concepts using a pre-trained model. Additionally, we introduce a logical hypothesis for ranking confidence levels. Along this line, we design a novel calibration loss to optimize the confidence parameters by modeling the process of student performance prediction. Finally, extensive experiments on four real-world datasets clearly demonstrate the effectiveness of our ReliCD framework.

cs.CY↗

arXiv2023

Learning to See Low-Light Images via Feature Domain Adaptation

Raw low light image enhancement (LLIE) has achieved much better performance than the sRGB domain enhancement methods due to the merits of raw data. However, the ambiguity between noisy to clean and raw to sRGB mappings may mislead the single-stage enhancement networks. The two-stage networks avoid ambiguity by decoupling the two mappings but usually have large computing complexity. To solve this problem, we propose a single-stage network empowered by Feature Domain Adaptation (FDA) to decouple the denoising and color mapping tasks in raw LLIE. The denoising encoder is supervised by the clean raw image, and then the denoised features are adapted for the color mapping task by an FDA module. We propose a Lineformer to serve as the FDA, which can well explore the global and local correlations with fewer line buffers (friendly to the line-based imaging process). During inference, the raw supervision branch is removed. In this way, our network combines the advantage of a two-stage enhancement process with the efficiency of single-stage inference. Experiments on four benchmark datasets demonstrate that our method achieves state-of-the-art performance with fewer computing costs (60% FLOPs of the two-stage method DNF). Our codes will be released after the acceptance of this work.

cs.CV↗

arXiv2023

CorrMatch: Label Propagation via Correlation Matching for Semi-Supervised Semantic Segmentation

This paper presents a simple but performant semi-supervised semantic segmentation approach, called CorrMatch. Previous approaches mostly employ complicated training strategies to leverage unlabeled data but overlook the role of correlation maps in modeling the relationships between pairs of locations. We observe that the correlation maps not only enable clustering pixels of the same category easily but also contain good shape information, which previous works have omitted. Motivated by these, we aim to improve the use efficiency of unlabeled data by designing two novel label propagation strategies. First, we propose to conduct pixel propagation by modeling the pairwise similarities of pixels to spread the high-confidence pixels and dig out more. Then, we perform region propagation to enhance the pseudo labels with accurate class-agnostic masks extracted from the correlation maps. CorrMatch achieves great performance on popular segmentation benchmarks. Taking the DeepLabV3+ with ResNet-101 backbone as our segmentation model, we receive a 76%+ mIoU score on the Pascal VOC 2012 dataset with only 92 annotated images. Code is available at https://github.com/BBBBchan/CorrMatch.

cs.CV↗

arXiv2023

A Novel Deep Clustering Framework for Fine-Scale Parcellation of Amygdala Using dMRI Tractography

The amygdala plays a vital role in emotional processing and exhibits structural diversity that necessitates fine-scale parcellation for a comprehensive understanding of its anatomico-functional correlations. Diffusion MRI tractography is an advanced imaging technique that can estimate the brain's white matter structural connectivity to potentially reveal the topography of the amygdala for studying its subdivisions. In this work, we present a deep clustering pipeline to perform automated, fine-scale parcellation of the amygdala using diffusion MRI tractography. First, we incorporate a newly proposed deep learning approach to enable accurate segmentation of the amygdala directly on the dMRI data. Next, we design a novel streamline clustering-based structural connectivity feature for a robust representation of voxels within the amygdala. Finally, we improve the popular joint dimensionality reduction and k-means clustering approach to enable amygdala parcellation at a finer scale. With the proposed method, we obtain nine unique amygdala parcels. Experiments show that these parcels can be consistently identified across subjects and have good correspondence to the widely used coarse-scale amygdala parcellation.

eess.IV↗

arXiv2023

Fast generation of mock galaxy catalogues with COLA

We investigate the feasibility of using COmoving Lagrangian Acceleration (COLA) technique to efficiently generate galaxy mock catalogues that can accurately reproduce the statistical properties of observed galaxies. Our proposed scheme combines the subhalo abundance matching (SHAM) procedure with COLA simulations, utilizing only three free parameters: the scatter magnitude ($σ_{\rm scat}$) in SHAM, the initial redshift ($z_{\rm init}$) of the COLA simulation, and the time stride ($da$) used by COLA. In this proof-of-concept study, we focus on a subset of BOSS CMASS NGC galaxies within the redshift range $z\in [0.45, 0.55]$. We perform $\mathtt{GADGET}$ simulation and low-resolution COLA simulations with various combinations of $(z_{\rm init}, da)$, each using $1024^{3}$ particles in an $800~h^{-1}{\rm Mpc}$ box. By minimizing the difference between COLA mock and CMASS NGC galaxies for the monopole of the two-point correlation function (2PCF), we obtain the optimal $σ_{\rm scat}$. We have found that by setting $z_{\rm init}=29$ and $da=1/30$, we achieve a good agreement between COLA mock and CMASS NGC galaxies within the range of 4 to $20~h^{-1}{\rm Mpc}$, with a computational cost two orders of magnitude lower than that of the N-body code. Moreover, a detailed verification is performed by comparing various statistical properties, such as anisotropic 2PCF, three-point clustering, and power spectrum multipoles, which shows similar performance between GADGET mock and COLA mock catalogues with the CMASS NGC galaxies. Furthermore, we assess the robustness of the COLA mock catalogues across different cosmological models, demonstrating consistent results in the resulting 2PCFs. Our findings suggest that COLA simulations are a promising tool for efficiently generating mock catalogues for emulators and machine learning analyses in exploring the large-scale structure of the Universe.

astro-ph.CO↗

arXiv2023

String breaking and running coupling of $\rm Q\bar{Q}$ in a rotating media from holography

Through gravity/gauge duality, the string breaking and running coupling constant of heavy quark-antiquark pair are investigated in the rotating background. For the meson decay mode $\rm Q\bar{Q} \rightarrow Q \bar{q}+\bar{Q} q$, we discuss the string breaking and running coupling in the parallel and transverse case. It is found that the parallel case has a more significant on string breaking and running coupling constant than the transverse case in the confined phase. The string-breaking distance and the maximum value of running coupling will decrease with the increase of angular velocity in the parallel case. Besides, we also investigate the running coupling and screening distance at finite angular velocity in the deconfined phase. It is found that the maximum values of the running coupling and screening distance are decreasing functions of angular velocity. The parallel case has a more significant influence on the running coupling and screening distance than the transverse case in the deconfined phase.

hep-ph↗

arXiv2023

MoqaGPT : Zero-Shot Multi-modal Open-domain Question Answering with Large Language Model

Multi-modal open-domain question answering typically requires evidence retrieval from databases across diverse modalities, such as images, tables, passages, etc. Even Large Language Models (LLMs) like GPT-4 fall short in this task. To enable LLMs to tackle the task in a zero-shot manner, we introduce MoqaGPT, a straightforward and flexible framework. Using a divide-and-conquer strategy that bypasses intricate multi-modality ranking, our framework can accommodate new modalities and seamlessly transition to new models for the task. Built upon LLMs, MoqaGPT retrieves and extracts answers from each modality separately, then fuses this multi-modal information using LLMs to produce a final answer. Our methodology boosts performance on the MMCoQA dataset, improving F1 by +37.91 points and EM by +34.07 points over the supervised baseline. On the MultiModalQA dataset, MoqaGPT surpasses the zero-shot baseline, improving F1 by 9.5 points and EM by 10.1 points, and significantly closes the gap with supervised methods. Our codebase is available at https://github.com/lezhang7/MOQAGPT.

cs.CL↗

arXiv2023

Holographic Schwinger Effect in Anisotropic Media

According to gauge/gravity correspondence, we study the holographic Schwinger effect within an anisotropic background. Firstly, the separate length of the particle-antiparticle pairs is computed within the context of an anisotropic background which is parameterized by dynamical exponent $ν$. It is found that the maximum separate length $x$ increases with the increase of dynamical exponent $ν$. By analyzing the potential energy, we find that the potential barrier increases with the dynamical exponent $ν$ at a small separate distance. This observation implies that the Schwinger effect within an anisotropic background is comparatively weaker when contrasted with its manifestation in an isotropic background. Finally, we also find that the Schwinger effect in the transverse direction is weakened, compared to the parallel direction in the anisotropic background, which is consistent with the top-down model.

hep-th↗