arXiv subjects

Renjing Xu

Publications and source records attributed to Renjing Xu.

139 records · Page 8Linked to original sources

arXiv2023

Spiking Denoising Diffusion Probabilistic Models

Spiking neural networks (SNNs) have ultra-low energy consumption and high biological plausibility due to their binary and bio-driven nature compared with artificial neural networks (ANNs). While previous research has primarily focused on enhancing the performance of SNNs in classification tasks, the generative potential of SNNs remains relatively unexplored. In our paper, we put forward Spiking Denoising Diffusion Probabilistic Models (SDDPM), a new class of SNN-based generative models that achieve high sample quality. To fully exploit the energy efficiency of SNNs, we propose a purely Spiking U-Net architecture, which achieves comparable performance to its ANN counterpart using only 4 time steps, resulting in significantly reduced energy consumption. Extensive experimental results reveal that our approach achieves state-of-the-art on the generative tasks and substantially outperforms other SNN-based generative models, achieving up to 12x and 6x improvement on the CIFAR-10 and the CelebA datasets, respectively. Moreover, we propose a threshold-guided strategy that can further improve the performances by 2.69% in a training-free manner. The SDDPM symbolizes a significant advancement in the field of SNN generation, injecting new perspectives and potential avenues of exploration. Our code is available at https://github.com/AndyCao1125/SDDPM.

cs.NE↗

arXiv2023

Pursing the Sparse Limitation of Spiking Deep Learning Structures

Spiking Neural Networks (SNNs), a novel brain-inspired algorithm, are garnering increased attention for their superior computation and energy efficiency over traditional artificial neural networks (ANNs). To facilitate deployment on memory-constrained devices, numerous studies have explored SNN pruning. However, these efforts are hindered by challenges such as scalability challenges in more complex architectures and accuracy degradation. Amidst these challenges, the Lottery Ticket Hypothesis (LTH) emerges as a promising pruning strategy. It posits that within dense neural networks, there exist winning tickets or subnetworks that are sparser but do not compromise performance. To explore a more structure-sparse and energy-saving model, we investigate the unique synergy of SNNs with LTH and design two novel spiking winning tickets to push the boundaries of sparsity within SNNs. Furthermore, we introduce an innovative algorithm capable of simultaneously identifying both weight and patch-level winning tickets, enabling the achievement of sparser structures without compromising on the final model's performance. Through comprehensive experiments on both RGB-based and event-based datasets, we demonstrate that our spiking lottery ticket achieves comparable or superior performance even when the model structure is extremely sparse.

cs.NE↗

arXiv2023

RBFormer: Improve Adversarial Robustness of Transformer by Robust Bias

Recently, there has been a surge of interest and attention in Transformer-based structures, such as Vision Transformer (ViT) and Vision Multilayer Perceptron (VMLP). Compared with the previous convolution-based structures, the Transformer-based structure under investigation showcases a comparable or superior performance under its distinctive attention-based input token mixer strategy. Introducing adversarial examples as a robustness consideration has had a profound and detrimental impact on the performance of well-established convolution-based structures. This inherent vulnerability to adversarial attacks has also been demonstrated in Transformer-based structures. In this paper, our emphasis lies on investigating the intrinsic robustness of the structure rather than introducing novel defense measures against adversarial attacks. To address the susceptibility to robustness issues, we employ a rational structure design approach to mitigate such vulnerabilities. Specifically, we enhance the adversarial robustness of the structure by increasing the proportion of high-frequency structural robust biases. As a result, we introduce a novel structure called Robust Bias Transformer-based Structure (RBFormer) that shows robust superiority compared to several existing baseline structures. Through a series of extensive experiments, RBFormer outperforms the original structures by a significant margin, achieving an impressive improvement of +16.12% and +5.04% across different evaluation criteria on CIFAR-10 and ImageNet-1k, respectively.

cs.CV↗

arXiv2023

TTPOINT: A Tensorized Point Cloud Network for Lightweight Action Recognition with Event Cameras

Event cameras have gained popularity in computer vision due to their data sparsity, high dynamic range, and low latency. As a bio-inspired sensor, event cameras generate sparse and asynchronous data, which is inherently incompatible with the traditional frame-based method. Alternatively, the point-based method can avoid additional modality transformation and naturally adapt to the sparsity of events. Still, it typically cannot reach a comparable accuracy as the frame-based method. We propose a lightweight and generalized point cloud network called TTPOINT which achieves competitive results even compared to the state-of-the-art (SOTA) frame-based method in action recognition tasks while only using 1.5 % of the computational resources. The model is adept at abstracting local and global geometry by hierarchy structure. By leveraging tensor-train compressed feature extractors, TTPOINT can be designed with minimal parameters and computational complexity. Additionally, we developed a straightforward downsampling algorithm to maintain the spatio-temporal feature. In the experiment, TTPOINT emerged as the SOTA method on three datasets while also attaining SOTA among point cloud methods on all five datasets. Moreover, by using the tensor-train decomposition method, the accuracy of the proposed TTPOINT is almost unaffected while compressing the parameter size by 55 % in all five datasets.

cs.CV↗

arXiv2023

Masked Spiking Transformer

The combination of Spiking Neural Networks (SNNs) and Transformers has attracted significant attention due to their potential for high energy efficiency and high-performance nature. However, existing works on this topic typically rely on direct training, which can lead to suboptimal performance. To address this issue, we propose to leverage the benefits of the ANN-to-SNN conversion method to combine SNNs and Transformers, resulting in significantly improved performance over existing state-of-the-art SNN models. Furthermore, inspired by the quantal synaptic failures observed in the nervous system, which reduces the number of spikes transmitted across synapses, we introduce a novel Masked Spiking Transformer (MST) framework that incorporates a Random Spike Masking (RSM) method to prune redundant spikes and reduce energy consumption without sacrificing performance. Our experimental results demonstrate that the proposed MST model achieves a significant reduction of 26.8% in power consumption when the masking ratio is 75% while maintaining the same level of performance as the unmasked model.

cs.NE↗

arXiv2016

Exciton Brightening in Monolayer Phosphorene via Dimensionality Modification

Two-dimensional (2D) monolayer phosphorene, a 2D system with quasi-one-dimensional (quasi-1D) excitons, provides a unique 2D platform for investigating the dynamics of excitons in reduced dimensions and fundamental many-body interactions. However, on the other hand, the quasi-1D excitonic nature can limit the luminescence quantum yield significantly. Here, we report exciton brightening in monolayer phosphorene achieved via the dimensionality modification of excitons from quasi-1D to zero-dimensional (0D), through the transference of monolayer phosphorene samples onto defect-rich oxide substrate deposited by plasma-enhanced chemical vapor deposition (PECVD). The resultant interfacial luminescent local states lead to exciton localization and trigger efficient photon emissions at a new wavelength of ~920 nm. The luminescence quantum yield of 0D-like localized excitons is measured to be at least 33.6 times larger than that of intrinsic quasi-1D free excitons in monolayer phosphorene. This is primarily due to the reduction of non-radiative decay rate and the possibly enhanced radiative recombination probability. Owing to the large trapping energy, this new photon emission from the localized excitons in monolayer phosphorene can be observed at elevated temperature, which contrasts markedly with defect-induced photon emission from transition metal dichalcogenide (TMD) semiconductor monolayers that can only be observed at cryogenic temperatures. Our findings introduce new avenues for the development of novel photonic devices based on monolayer phosphorene, such as near-infrared lighting devices that are operable at elevated temperature. More importantly, 2D phosphorene with quasi-1D free excitons and 0D-like localized excitons provides a unique platform to investigate the fundamental phenomena in the ideal 2D-1D-0D hybrid system.

cond-mat.mtrl-sci↗

arXiv2015

Layer-dependent surface potential of phosphorene and anisotropic/layer-dependent charge transfer in phosphorene-gold hybrid system

The surface potential and the efficiency of interfacial charge transfer are extremely important for designing future semiconductor devices based on the emerging two-dimensional (2D) phosphorene. Here, we directly measured the strongly layer-dependent surface potential of mono- and few-layer phosphorene on gold, which confirms with the reported theoretical prediction. At the same time, we used an optical way - photoluminescence (PL) spectroscopy to probe the charge transfer in phosphorene-gold hybrid system. We firstly observed highly anisotropic and layer-dependent PL quenching in the phosphorene-gold hybrid system, which is attributed to the highly anisotropic/layer-dependent interfacial charge transfer.

cond-mat.mtrl-sci↗

arXiv2015

Optical Tuning of Exciton and Trion Emissions in Monolayer Phosphorene

Monolayer phosphorene provides a unique two-dimensional (2D) platform to investigate the fundamental dynamics of excitons and trions (charged excitons) in reduced dimensions. However, owing to its high instability, unambiguous identification of monolayer phosphorene has been elusive. Consequently, many important fundamental properties, such as exciton dynamics, remain underexplored. We report a rapid, noninvasive, and highly accurate approach based on optical interferometry to determine the layer number of phosphorene, and confirm the results with reliable photoluminescence measurements. Furthermore, we successfully probed the dynamics of excitons and trions in monolayer phosphorene by controlling the photo-carrier injection in a relatively low excitation power range. Based on our measured optical gap and the previously measured electronic energy gap, we determined the exciton binding energy to be ~0.3 eV for the monolayer phosphorene on SiO2/Si substrate, which agrees well with theoretical predictions. A huge trion binding energy of ~100 meV was first observed in monolayer phosphorene, which is around five times higher than that in transition metal dichalcogenide (TMD) monolayer semiconductor, such as MoS2. The carrier lifetime of exciton emission in monolayer phosphorene was measured to be ~220 ps, which is comparable to those in other 2D TMD semiconductors. Our results open new avenues for exploring fundamental phenomena and novel optoelectronic applications using monolayer phosphorene.

cond-mat.mtrl-sci↗

arXiv2014

Unambiguous identification of monolayer phosphorene by phase-shifting interferometry

Monolayer phosphorene provides a unique two-dimensional (2D) platform to investigate the fundamental many-body interactions. However, owing to its high instability, unambiguous identification of monolayer phosphorene has been elusive. Consequently, many important fundamental properties, such as exciton dynamics, remain underexplored. We report a rapid, noninvasive, and highly accurate approach based on optical interferometry to determine the layer number of phosphorene, and confirm the results with reliable photoluminescence measurements. Based on the measured optical gap and the calculated electronic energy gap, we determined the exciton binding energy to be ~0.4 eV for the monolayer phosphorene on SiO2/Si substrate, which agrees well with theoretical predictions. Our results open new avenues for exploring fundamental phenomena and novel optoelectronic applications using monolayer phosphorene.

cond-mat.mtrl-sci↗

arXiv2014

Atomically Thin Optical Lenses and Gratings

Two-dimensional (2D) materials have emerged as promising candidates for miniaturized optoelectronic devices, due to their strong inelastic interactions with light. On the other hand, a miniaturized optical system also requires strong elastic light-matter interactions to control the flow of light. Here, we report giant optical path length (OPL) from a single-layer molybdenum disulfide (MoS2), which is around one order of magnitude larger than that from a single-layer graphene. Using such giant OPL to engineer the phase front of optical beams, we demonstrated, to the best of our knowledge, the world's thinnest optical lens consisting of a few layers of MoS2 less than 6.3 nm thick. Moreover, we show that MoS2 has much better dielectric response than good conductor (like gold) and other dielectric materials (like Si, SiO2 or graphene). By taking advantage of the giant elastic scattering efficiency in ultra-thin high-index 2D materials, we demonstrated high-efficiency gratings based on a single- or few-layers of MoS2. The capability of manipulating the flow of light in 2D materials opens an exciting avenue towards unprecedented miniaturization of optical components and the integration of advanced optical functionalities.

cond-mat.mtrl-sci↗

arXiv2014

Extraordinarily bound quasi-one-dimensional trions in two-dimensional phosphorene atomic semiconductors

The anisotropic nature of the new two-dimensional (2D) material phosphorene, in contrast to other 2D materials such as graphene and transition metal dichalcogenide (TMD) semiconductors, allows excitons to be confined in a quasi-one-dimensional (1D) space predicted in theory, leading to remarkable phenomena arising from the reduced dimensionality and screening. Here, we report a trion (charged exciton) binding energy of 190 meV in few-layer phosphorene at room temperature, which is nearly one to two orders of magnitude larger than those in 2D TMD semiconductors (20-30 meV) and quasi-2D quantum wells (1-5 meV). Such a large binding energy has only been observed in truly 1D materials such as carbon nanotubes, whose optoelectronic applications have been severely hurdled by their intrinsically small optical cross-sections. Phosphorene offers an elegant way to overcome this hurdle by enabling quasi-1D excitonic and trionic behaviors in a large 2D area, allowing optoelectronic integration. We experimentally validated the quasi-1D nature of excitonic and trionic dynamics in phospherene by demonstrating completely linearly polarized light emission from excitons and trions. The implications of the extraordinarily large trion binding energy in a higher-than-one-dimensional material are far-reaching. It provides a room-temperature 2D platform to observe the fundamental many-body interactions in the quasi-1D region. The strong photoluminescence emission in phosphorene has been electrically tuned over a large spectral range at room temperature, which opens a new route for tunable light sources.

cond-mat.mtrl-sci↗

arXiv2014

Review of Strong Field Approximation and Investigating Semiclassical Evolution Approach

This paper theoretically analyzes the behavior of an atom driven by a strong electro-magnetic field. Moreover, besides traditional quantum mechanics method, we also investigate semiclassical approaches to this problem. We first performed strong field approximation for system of an atom driven by a strong electromagnetic field in velocity gauge. Our simulation result is consistent with theories and close to experiments except some reasonable difference caused by different parameters and omitted bound and final states in the transition amplitude. Next, a new semiclassical approach is used to solve Volkov wave function. We prove that semiclassical approximation works well in predicting the particle evolution in quantum world, especially for system in a strong electromagnetic field with low frequency. Finally, we also briefly illustrated how to use semiclassical approximation to get the same results as strong field approximation and also partly tested the viability of the semiclassical approach to Teller Potential. This part needs future work to accomplish. But still, semiclassical approximation provides a potentially new method to solve complicated system, which might be more effective than traditional quantum mechanics recipe.

physics.atom-ph↗

arXiv2014

Extraordinary Photoluminescence and Strong Temperature/Angle-Dependent Raman Responses in Few-Layer Phosphorene

Phosphorene is a new family member of two-dimensional materials. We observed strong and highly layer-dependent photoluminescence in few-layer phosphorene (2 to 5 layers). The results confirmed the theoretical prediction that few-layer phosphorene has a direct and layer-sensitive band gap. We also demonstrated that few-layer phosphorene is more sensitive to temperature modulation than graphene and MoS2 in Raman scattering. The anisotropic Raman response in few-layer phosphorene has enabled us to use an optical method to quickly determine the crystalline orientation without tunneling electron microscope (TEM) or scanning tunneling microscope (STM). Our results provide much needed experimental information about the band structures and exciton nature in few-layer phosphorene.

cond-mat.mes-hall↗