arXiv subjects

Bin Lei

Publications and source records attributed to Bin Lei.

30 records · Page 2Linked to original sources

arXiv2023

Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought

Recent advancements in large-scale models, such as GPT-4, have showcased remarkable capabilities in addressing standard queries. However, when facing complex problems that require multi-step logical reasoning, their accuracy dramatically decreases. Current research has explored the realm of \textit{prompting engineering} to bolster the inferential capacities of these models. Our paper unveils a pioneering prompting technique, dubbed \textit{Graph of Thoughts (GoT)}. Through testing on a trio of escalating challenges: the 24-point game, resolution of high-degree polynomial equations, and derivation of formulas for recursive sequences, our method outperformed GPT-4, achieving accuracy improvements of $89.7\%$, $86\%$, and $56\%$ for each respective task. Moreover, when juxtaposed with the state-of-the-art (SOTA) prompting method, \textit{Tree of Thought (ToT)}, our approach registered an average accuracy boost of $23\%$, $24\%$, and $15\%$.

cs.LG↗

arXiv2023

Towards Zero Memory Footprint Spiking Neural Network Training

Biologically-inspired Spiking Neural Networks (SNNs), processing information using discrete-time events known as spikes rather than continuous values, have garnered significant attention due to their hardware-friendly and energy-efficient characteristics. However, the training of SNNs necessitates a considerably large memory footprint, given the additional storage requirements for spikes or events, leading to a complex structure and dynamic setup. In this paper, to address memory constraint in SNN training, we introduce an innovative framework, characterized by a remarkably low memory footprint. We \textbf{(i)} design a reversible SNN node that retains a high level of accuracy. Our design is able to achieve a $\mathbf{58.65\times}$ reduction in memory usage compared to the current SNN node. We \textbf{(ii)} propose a unique algorithm to streamline the backpropagation process of our reversible SNN node. This significantly trims the backward Floating Point Operations Per Second (FLOPs), thereby accelerating the training process in comparison to current reversible layer backpropagation method. By using our algorithm, the training time is able to be curtailed by $\mathbf{23.8\%}$ relative to existing reversible layer architectures.

cs.NE↗

arXiv2023

Neurogenesis Dynamics-inspired Spiking Neural Network Training Acceleration

Biologically inspired Spiking Neural Networks (SNNs) have attracted significant attention for their ability to provide extremely energy-efficient machine intelligence through event-driven operation and sparse activities. As artificial intelligence (AI) becomes ever more democratized, there is an increasing need to execute SNN models on edge devices. Existing works adopt weight pruning to reduce SNN model size and accelerate inference. However, these methods mainly focus on how to obtain a sparse model for efficient inference, rather than training efficiency. To overcome these drawbacks, in this paper, we propose a Neurogenesis Dynamics-inspired Spiking Neural Network training acceleration framework, NDSNN. Our framework is computational efficient and trains a model from scratch with dynamic sparsity without sacrificing model fidelity. Specifically, we design a new drop-and-grow strategy with decreasing number of non-zero weights, to maintain extreme high sparsity and high accuracy. We evaluate NDSNN using VGG-16 and ResNet-19 on CIFAR-10, CIFAR-100 and TinyImageNet. Experimental results show that NDSNN achieves up to 20.52\% improvement in accuracy on Tiny-ImageNet using ResNet-19 (with a sparsity of 99\%) as compared to other SOTA methods (e.g., Lottery Ticket Hypothesis (LTH), SET-SNN, RigL-SNN). In addition, the training cost of NDSNN is only 40.89\% of the LTH training cost on ResNet-19 and 31.35\% of the LTH training cost on VGG-16 on CIFAR-10.

cs.NE↗

arXiv2022

Efficient Traffic State Forecasting using Spatio-Temporal Network Dependencies: A Sparse Graph Neural Network Approach

Traffic state prediction in a transportation network is paramount for effective traffic operations and management, as well as informed user and system-level decision-making. However, long-term traffic prediction (beyond 30 minutes into the future) remains challenging in current research. In this work, we integrate the spatio-temporal dependencies in the transportation network from network modeling, together with the graph convolutional network (GCN) and graph attention network (GAT). To further tackle the dramatic computation and memory cost caused by the giant model size (i.e., number of weights) caused by multiple cascaded layers, we propose sparse training to mitigate the training cost, while preserving the prediction accuracy. It is a process of training using a fixed number of nonzero weights in each layer in each iteration. We consider the problem of long-term traffic speed forecasting for a real large-scale transportation network data from the California Department of Transportation (Caltrans) Performance Measurement System (PeMS). Experimental results show that the proposed GCN-STGT and GAT-STGT models achieve low prediction errors on short-, mid- and long-term prediction horizons, of 15, 30 and 45 minutes in duration, respectively. Using our sparse training, we could train from scratch with high sparsity (e.g., up to 90%), equivalent to 10 times floating point operations per second (FLOPs) reduction on computational cost using the same epochs as dense training, and arrive at a model with very small accuracy loss compared with the original dense training

cs.LG↗

arXiv2022

A new class of bilayer kagome lattice compounds with Dirac nodal lines and pressure-induced superconductivity

Kagome lattice composed of transition-metal ions provides a great opportunity to explore the intertwining between geometry, electronic orders and band topology. The discovery of multiple competing orders that connect intimately with the underlying topological band structure in nonmagnetic kagome metals $A$V$_3$Sb$_5$ ($A$ = K, Rb, Cs) further pushes this topic to the quantum frontier. Here we report the discovery and characterization of a new class of vanadium-based compounds with kagome bilayers, namely $A$V$_6$Sb$_6$ ($A$ = K, Rb, Cs) and V$_6$Sb$_4$, which, together with $A$V$_3$Sb$_5$, compose a series of kagome compounds with a generic chemical formula ($A_{m-1}$Sb$_{2m}$)(V$_3$Sb)$_n$ (m = 1, 2; n = 1, 2). Theoretical calculations combined with angle-resolved photoemission measurements reveal that these compounds feature Dirac nodal lines in close vicinity to the Fermi level. Pressure-induced superconductivity in $A$V$_6$Sb$_6$ further suggests promising emergent phenomena in these materials. The establishment of a new family of layered kagome materials paves the way for designer of fascinating kagome systems with diverse topological nontrivialities and collective ground states.

cond-mat.mtrl-sci↗

arXiv2020

Classification of Large-Scale High-Resolution SAR Images with Deep Transfer Learning

The classification of large-scale high-resolution SAR land cover images acquired by satellites is a challenging task, facing several difficulties such as semantic annotation with expertise, changing data characteristics due to varying imaging parameters or regional target area differences, and complex scattering mechanisms being different from optical imaging. Given a large-scale SAR land cover dataset collected from TerraSAR-X images with a hierarchical three-level annotation of 150 categories and comprising more than 100,000 patches, three main challenges in automatically interpreting SAR images of highly imbalanced classes, geographic diversity, and label noise are addressed. In this letter, a deep transfer learning method is proposed based on a similarly annotated optical land cover dataset (NWPU-RESISC45). Besides, a top-2 smooth loss function with cost-sensitive parameters was introduced to tackle the label noise and imbalanced classes' problems. The proposed method shows high efficiency in transferring information from a similarly annotated remote sensing dataset, a robust performance on highly imbalanced classes, and is alleviating the over-fitting problem caused by label noise. What's more, the learned deep model has a good generalization for other SAR-specific tasks, such as MSTAR target recognition with a state-of-the-art classification accuracy of 99.46%.

eess.SP↗

arXiv2019

Probing the direct factor for superconductivity in FeSe-Based Superconductors by Raman Scattering

The FeSe-based superconductors exhibit a wide range of critical temperature Tc under a variety of material and physical conditions, but extensive studies to date have yet to produce a consensus view on the underlying mechanism. Here we report on a systematic Raman scattering work on intercalated FeSe superconductors Lix(NH3)yFe2Se2 and (Li,Fe)OHFeSe compared to pristine FeSe. All three crystals show an anomalous power-law temperature dependence of phonon linewidths, deviating from the standard anharmonic behavior. This intriguing phenomenon is attributed to electron-phonon coupling effects enhanced by electron correlation, as evidenced by the evolution of the A1g Raman mode. Meanwhile, an analysis of the B1g mode, which probes the out-of-plane vibration of Fe, reveals a lack of influence by previously suggested structural parameters, and instead indicates a crucial role of the joint density of states in determining Tc. These findings identify carrier doping as the direct factor driving and modulating superconductivity in FeSe-based compounds.

cond-mat.supr-con↗

arXiv2019

What, Where and How to Transfer in SAR Target Recognition Based on Deep CNNs

Deep convolutional neural networks (DCNNs) have attracted much attention in remote sensing recently. Compared with the large-scale annotated dataset in natural images, the lack of labeled data in remote sensing becomes an obstacle to train a deep network very well, especially in SAR image interpretation. Transfer learning provides an effective way to solve this problem by borrowing the knowledge from the source task to the target task. In optical remote sensing application, a prevalent mechanism is to fine-tune on an existing model pre-trained with a large-scale natural image dataset, such as ImageNet. However, this scheme does not achieve satisfactory performance for SAR application because of the prominent discrepancy between SAR and optical images. In this paper, we attempt to discuss three issues that are seldom studied before in detail: (1) what network and source tasks are better to transfer to SAR targets, (2) in which layer are transferred features more generic to SAR targets and (3) how to transfer effectively to SAR targets recognition. Based on the analysis, a transitive transfer method via multi-source data with domain adaptation is proposed in this paper to decrease the discrepancy between the source data and SAR targets. Several experiments are conducted on OpenSARShip. The results indicate that the universal conclusions about transfer learning in natural images cannot be completely applied to SAR targets, and the analysis of what and where to transfer in SAR target recognition is helpful to decide how to transfer more effectively.

eess.SP↗

arXiv2018

Unveiling the Charge Density Wave Inhomogeneity and Pseudogap State in 1T-TiSe2

By using scanning tunneling microscopy (STM) / spectroscopy (STS), we systematically characterize the electronic structure of lightly doped 1T-TiSe2, and demonstrate the existence of the electronic inhomogeneity and the pseudogap state. It is found that the intercalation induced lattice distortion impacts the local band structure and reduce the size of the charge density wave (CDW) gap with the persisted 2x2 spatial modulation. On the other hand, the delocalized doping electrons promote the formation of pseudogap. Domination by either of the two effects results in the separation of two characteristic regions in real space, exhibiting rather different electronic structures. Further doping electrons to the surface confirms that the pseudogap may be the precursor for the superconducting gap. This study suggests that the competition of local lattice distortion and the delocalized doping effect contribute to the complicated relationship between charge density wave and superconductivity for intercalated 1T-TiSe2.

cond-mat.str-el↗

arXiv2018

Orbital Origin of Extremely Anisotropic Superconducting Gap in Nematic Phase of FeSe Superconductor

The iron-based superconductors are characterized by multiple-orbital physics where all the five Fe 3$d$ orbitals get involved. The multiple-orbital nature gives rise to various novel phenomena like orbital-selective Mott transition, nematicity and orbital fluctuation that provide a new route for realizing superconductivity. The complexity of multiple-orbital also asks to disentangle the relationship between orbital, spin and nematicity, and to identify dominant orbital ingredients that dictate superconductivity. The bulk FeSe superconductor provides an ideal platform to address these issues because of its simple crystal structure and unique coexistence of superconductivity and nematicity. However, the orbital nature of the low energy electronic excitations and its relation to the superconducting gap remain controversial. Here we report direct observation of highly anisotropic Fermi surface and extremely anisotropic superconducting gap in the nematic state of FeSe superconductor by high resolution laser-based angle-resolved photoemission measurements. We find that the low energy excitations of the entire hole pocket at the Brillouin zone center are dominated by the single $d_{xz}$ orbital. The superconducting gap exhibits an anti-correlation relation with the $d_{xz}$ spectral weight near the Fermi level, i.e., the gap size minimum (maximum) corresponds to the maximum (minimum) of the $d_{xz}$ spectral weight along the Fermi surface. These observations provide new insights in understanding the orbital origin of the extremely anisotropic superconducting gap in FeSe superconductor and the relation between nematicity and superconductivity in the iron-based superconductors.

cond-mat.supr-con↗

arXiv2017

Learning a Rotation Invariant Detector with Rotatable Bounding Box

Detection of arbitrarily rotated objects is a challenging task due to the difficulties of locating the multi-angle objects and separating them effectively from the background. The existing methods are not robust to angle varies of the objects because of the use of traditional bounding box, which is a rotation variant structure for locating rotated objects. In this article, a new detection method is proposed which applies the newly defined rotatable bounding box (RBox). The proposed detector (DRBox) can effectively handle the situation where the orientation angles of the objects are arbitrary. The training of DRBox forces the detection networks to learn the correct orientation angle of the objects, so that the rotation invariant property can be achieved. DRBox is tested to detect vehicles, ships and airplanes on satellite images, compared with Faster R-CNN and SSD, which are chosen as the benchmark of the traditional bounding box based methods. The results shows that DRBox performs much better than traditional bounding box based methods do on the given tasks, and is more robust against rotation of input image and target objects. Besides, results show that DRBox correctly outputs the orientation angles of the objects, which is very useful for locating multi-angle objects efficiently. The code and models are available at https://github.com/liulei01/DRBox.

cs.CV↗

arXiv2017

Repeat-pass SAR Interferometry Experiments with Gaofen-3: A Case Study of Ningbo Area

This paper reports the repeat-pass interferometric SAR results of Gaofen-3, a Chinese civil SAR satellite, acquired in November 2016 and March 2017 from Ningbo area. With the spatial baseline about 600 m and time baseline 116 days, the coherence of the two images still achieve good enough to generate the digital elevation model (DEM). During the InSAR processing, we compared several baseline estimating methods and obtained a good flat-earth phase removed interferogram map. By using the latest SAR interferogram filter and phase unwrapping method we proposed, we improved the coherence up to 0.88 in urban area and obtained a high quality DEM in Ningbo area. In addition, we evaluated the elevation model by comparing with the elevation values extracted from SRTM. And the result shows that accuracy of the elevation map is about 5 m (RMS) in plane area and 22m (RMS) in mountainous region, which demonstrated that Gaofen-3 has the powerful ability of repeat-pass SAR Interferometry.

physics.geo-ph↗