arXiv subjects

Shuai Li

Publications and source records attributed to Shuai Li.

At least 397 records · Page 22Linked to original sources

arXiv2018

Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN

Recurrent neural networks (RNNs) have been widely used for processing sequential data. However, RNNs are commonly difficult to train due to the well-known gradient vanishing and exploding problems and hard to learn long-term patterns. Long short-term memory (LSTM) and gated recurrent unit (GRU) were developed to address these problems, but the use of hyperbolic tangent and the sigmoid action functions results in gradient decay over layers. Consequently, construction of an efficiently trainable deep network is challenging. In addition, all the neurons in an RNN layer are entangled together and their behaviour is hard to interpret. To address these problems, a new type of RNN, referred to as independently recurrent neural network (IndRNN), is proposed in this paper, where neurons in the same layer are independent of each other and they are connected across layers. We have shown that an IndRNN can be easily regulated to prevent the gradient exploding and vanishing problems while allowing the network to learn long-term dependencies. Moreover, an IndRNN can work with non-saturated activation functions such as relu (rectified linear unit) and be still trained robustly. Multiple IndRNNs can be stacked to construct a network that is deeper than the existing RNNs. Experimental results have shown that the proposed IndRNN is able to process very long sequences (over 5000 time steps), can be used to construct very deep networks (21 layers used in the experiment) and still be trained robustly. Better performances have been achieved on various tasks by using IndRNNs compared with the traditional RNN and LSTM. The code is available at https://github.com/Sunnydreamrain/IndRNN_Theano_Lasagne.

cs.CV↗

arXiv2018

A Fusion Framework for Camouflaged Moving Foreground Detection in the Wavelet Domain

Detecting camouflaged moving foreground objects has been known to be difficult due to the similarity between the foreground objects and the background. Conventional methods cannot distinguish the foreground from background due to the small differences between them and thus suffer from under-detection of the camouflaged foreground objects. In this paper, we present a fusion framework to address this problem in the wavelet domain. We first show that the small differences in the image domain can be highlighted in certain wavelet bands. Then the likelihood of each wavelet coefficient being foreground is estimated by formulating foreground and background models for each wavelet band. The proposed framework effectively aggregates the likelihoods from different wavelet bands based on the characteristics of the wavelet transform. Experimental results demonstrated that the proposed method significantly outperformed existing methods in detecting camouflaged foreground objects. Specifically, the average F-measure for the proposed algorithm was 0.87, compared to 0.71 to 0.8 for the other state-of-the-art methods.

cs.CV↗

arXiv2018

Ground States of Two-Component Attractive Bose-Einstein Condensates II: Semi-trivial Limit Behavior

As a continuation of [14], we study new pattern formations of ground states $(u_1,u_2)$ for two-component Bose-Einstein condensates (BEC) with homogeneous trapping potentials in $R^2$, where the intraspecies interaction $(-a,-b)$ and the interspecies interaction $-β$ are both attractive, $i.e,$ $a$, $b$ and $β$ are all positive. If $0<b<a^*:=\|w\|^2_2$ and $0<β<a^*$ are fixed, where $w$ is the unique positive solution of $Δw-w+w^3=0$ in $R^2$, the semi-trivial behavior of $(u_1,u_2)$ as $a\nearrow a^*$ is proved in the sense that $u_1$ concentrates at a unique point and while $u_2\equiv 0$ in $R^2$. However, if $0<b<a^*$ and $a^*\leβ<β^*=a^*+\sqrt{(a^*-a)(a^*-b)}$, the refined spike profile and the uniqueness of $(u_1,u_2)$ as $a\nearrow a^*$ are analyzed, where $(u_1,u_2)$ must be unique, $u_1$ concentrates at a unique point, and meanwhile $u_2$ can either blow up or vanish, depending on how $β$ approaches to $a^*$.

math.AP↗

arXiv2018

Hole Filling with Multiple Reference Views in DIBR View Synthesis

Depth-image-based rendering (DIBR) oriented view synthesis has been widely employed in the current depth-based 3D video systems by synthesizing a virtual view from an arbitrary viewpoint. However, holes may appear in the synthesized view due to disocclusion, thus significantly degrading the quality. Consequently, efforts have been made on developing effective and efficient hole filling algorithms. Current hole filling techniques generally extrapolate/interpolate the hole regions with the neighboring information based on an assumption that the texture pattern in the holes is similar to that of the neighboring background information. However, in many scenarios especially of complex texture, the assumption may not hold. In other words, hole filling techniques can only provide an estimation for a hole which may not be good enough or may even be erroneous considering a wide variety of complex scene of images. In this paper, we first examine the view interpolation with multiple reference views, demonstrating that the problem of emerging holes in a target virtual view can be greatly alleviated by making good use of other neighboring complementary views in addition to its two (commonly used) most neighboring primary views. The effects of using multiple views for view extrapolation in reducing holes are also investigated in this paper. In view of the 3D Video and ongoing free-viewpoint TV standardization, we propose a new view synthesis framework which employs multiple views to synthesize output virtual views. Furthermore, a scheme of selective warping of complementary views is developed by efficiently locating a small number of useful pixels in the complementary views for hole reduction, to avoid a full warping of additional complementary views thus lowering greatly the warping complexity.

cs.CV↗

arXiv2017

2.6mJ/100Hz CEP stable near-single-cycle 4μm laser based on OPCPA and hollow-core-fiber compression

A carrier envelope phase stable near-single cycle mid-infrared laser based on optical parametric chirped pulse amplification and hollow-core-fiber compression is demonstrated. 4 μm laser pulses with 11.8 mJ energy are delivered from a KTA based OPCPA with 100 Hz repetition rate, and compressed to be ~105 fs by a two-grating compressor with efficiency over 50%. Subsequently, the pulse spectrum is broadened by employing a krypton gas-filled hollow-core-fiber (HCF). Then, the pulse duration is further compressed to 21.5 fs through a CaF2 bulk material with energy of 2.6 mJ and stability of 0.9% RMS, which is about 1.6 cycle for 4 μm laser pulse. The near-single cycle 4 μm laser pulse CEP is passively stabilized with ~370 mrad based on a CEP stable 4 μm OPA injection.

physics.optics↗

arXiv2017

Ground States of Two-Component Attractive Bose-Einstein Condensates I: Existence and Uniqueness

We study ground states of two-component Bose-Einstein condensates (BEC) with trapping potentials in $R^2$, where the intraspecies interaction $(-a_1,-a_2)$ and the interspecies interaction $-β$ are both attractive, $i.e,$ $a_1$, $a_2$ and $β$ are all positive. The existence and non-existence of ground states are classified completely by investigating equivalently the associated $L^2$-critical constraint variational problem. The uniqueness and symmetry-breaking of ground states are also analyzed under different types of trapping potentials as $β\nearrow β^*=a^*+\sqrt{(a^*-a_1)(a^*-a_2)}$, where $0<a_i<a^*:=\|w\|^2_2$ ($i=1,2$) is fixed and $w$ is the unique positive solution of $Δw-w+w^3=0$ in $R^2$. The semi-trivial limit behavior of ground states is tackled in the companion paper [12].

math.AP↗

arXiv2017

Imaging through glass diffusers using densely connected convolutional networks

Computational imaging through scatter generally is accomplished by first characterizing the scattering medium so that its forward operator is obtained; and then imposing additional priors in the form of regularizers on the reconstruction functional so as to improve the condition of the originally ill-posed inverse problem. In the functional, the forward operator and regularizer must be entered explicitly or parametrically (e.g. scattering matrices and dictionaries, respectively.) However, the process of determining these representations is often incomplete, prone to errors, or infeasible. Recently, deep learning architectures have been proposed to instead learn both the forward operator and regularizer through examples. Here, we propose for the first time, to our knowledge, a convolutional neural network architecture called "IDiffNet" for the problem of imaging through diffuse media and demonstrate that IDiffNet has superior generalization capability through extensive tests with well-calibrated diffusers. We found that the Negative Pearson Correlation Coefficient loss function for training is more appropriate for spatially sparse objects and strong scattering conditions. Our results show that the convolutional architecture is robust to the choice of prior, as demonstrated by the use of multiple training and testing object databases, and capable of achieving higher space-bandwidth product reconstructions than previously reported.

physics.optics↗

arXiv2017

Beetle Antennae Search without Parameter Tuning (BAS-WPT) for Multi-objective Optimization

Beetle antennae search (BAS) is an efficient meta-heuristic algorithm inspired by foraging behaviors of beetles. This algorithm includes several parameters for tuning and the existing results are limited to solve single objective optimization. This work pushes forward the research on BAS by providing one variant that releases the tuning parameters and is able to handle multi-objective optimization. This new approach applies normalization to simplify the original algorithm and uses a penalty function to exploit infeasible solutions with low constraint violation to solve the constraint optimization problem. Extensive experimental studies are carried out and the results reveal efficacy of the proposed approach to constraint handling.

cs.NE↗

arXiv2017

BAS: Beetle Antennae Search Algorithm for Optimization Problems

Meta-heuristic algorithms have become very popular because of powerful performance on the optimization problem. A new algorithm called beetle antennae search algorithm (BAS) is proposed in the paper inspired by the searching behavior of longhorn beetles. The BAS algorithm imitates the function of antennae and the random walking mechanism of beetles in nature, and then two main steps of detecting and searching are implemented. Finally, the algorithm is benchmarked on 2 well-known test functions, in which the numerical results validate the efficacy of the proposed BAS algorithm.

cs.NE↗

arXiv2017

Porcellio scaber algorithm (PSA) for solving constrained optimization problems

In this paper, we extend a bio-inspired algorithm called the porcellio scaber algorithm (PSA) to solve constrained optimization problems, including a constrained mixed discrete-continuous nonlinear optimization problem. Our extensive experiment results based on benchmark optimization problems show that the PSA has a better performance than many existing methods or algorithms. The results indicate that the PSA is a promising algorithm for constrained optimization.

cs.NE↗

arXiv2017

Foreground Detection in Camouflaged Scenes

Foreground detection has been widely studied for decades due to its importance in many practical applications. Most of the existing methods assume foreground and background show visually distinct characteristics and thus the foreground can be detected once a good background model is obtained. However, there are many situations where this is not the case. Of particular interest in video surveillance is the camouflage case. For example, an active attacker camouflages by intentionally wearing clothes that are visually similar to the background. In such cases, even given a decent background model, it is not trivial to detect foreground objects. This paper proposes a texture guided weighted voting (TGWV) method which can efficiently detect foreground objects in camouflaged scenes. The proposed method employs the stationary wavelet transform to decompose the image into frequency bands. We show that the small and hardly noticeable differences between foreground and background in the image domain can be effectively captured in certain wavelet frequency bands. To make the final foreground decision, a weighted voting scheme is developed based on intensity and texture of all the wavelet bands with weights carefully designed. Experimental results demonstrate that the proposed method achieves superior performance compared to the current state-of-the-art results.

cs.CV↗

arXiv2017

Lensless computational imaging through deep learning

Deep learning has been proven to yield reliably generalizable answers to numerous classification and decision tasks. Here, we demonstrate for the first time, to our knowledge, that deep neural networks (DNNs) can be trained to solve inverse problems in computational imaging. We experimentally demonstrate a lens-less imaging system where a DNN was trained to recover a phase object given a raw intensity image recorded some distance away.

cs.CV↗

arXiv2017

A Fully Trainable Network with RNN-based Pooling

Pooling is an important component in convolutional neural networks (CNNs) for aggregating features and reducing computational burden. Compared with other components such as convolutional layers and fully connected layers which are completely learned from data, the pooling component is still handcrafted such as max pooling and average pooling. This paper proposes a learnable pooling function using recurrent neural networks (RNN) so that the pooling can be fully adapted to data and other components of the network, leading to an improved performance. Such a network with learnable pooling function is referred to as a fully trainable network (FTN). Experimental results have demonstrated that the proposed RNN-based pooling can well approximate the existing pooling functions and improve the performance of the network. Especially for small networks, the proposed FTN can improve the performance by seven percentage points in terms of error rate on the CIFAR-10 dataset compared with the traditional CNN.

cs.CV↗

arXiv2017

Full-duplex Amplify-and-Forward Relaying: Power and Location Optimization

In this paper, we consider a full-duplex (FD) amplify-and-forward (AF) relay system and optimize its power allocation and relay location to minimize the system symbol error rate (SER). We first derive the asymptotic expressions of the outage probability and SER performance by taking into account the residual self interference (RSI) in FD systems. We then formulate the optimization problem based on the minimal SER criterion. Analytical and numerical results show that optimized relay location and power allocation can greatly improve system SER performance, and the performance floor caused by the RSI can be significantly reduced via optimizing relay location or power allocation.

cs.IT↗

arXiv2017

Deep LSTM for Large Vocabulary Continuous Speech Recognition

Recurrent neural networks (RNNs), especially long short-term memory (LSTM) RNNs, are effective network for sequential task like speech recognition. Deeper LSTM models perform well on large vocabulary continuous speech recognition, because of their impressive learning ability. However, it is more difficult to train a deeper network. We introduce a training framework with layer-wise training and exponential moving average methods for deeper LSTM models. It is a competitive framework that LSTM models of more than 7 layers are successfully trained on Shenma voice search data in Mandarin and they outperform the deep LSTM models trained by conventional approach. Moreover, in order for online streaming speech recognition applications, the shallow model with low real time factor is distilled from the very deep model. The recognition accuracy have little loss in the distillation process. Therefore, the model trained with the proposed training framework reduces relative 14\% character error rate, compared to original model which has the similar real-time capability. Furthermore, the novel transfer learning strategy with segmental Minimum Bayes-Risk is also introduced in the framework. The strategy makes it possible that training with only a small part of dataset could outperform full dataset training from the beginning.

cs.CL↗

arXiv2017

Fano-shaped impurity spectral density, electric-field-induced in-gap state and local magnetic moment of an adatom on trilayer graphene

Recently, the existence of local magnetic moment in a hydrogen adatom on graphene has been confirmed experimentally [González-Herrero et al., Science, 2016, 352, 437]. Inspired by this breakthrough, we theoretically investigate the top-site adatom on trilayer graphene (TLG) by solving the Anderson impurity model via self-consistent mean field method. The influence of the stacking order, the adsorption site and external electric field are carefully considered. We find that, due to its unique electronic structure, the situation of the TLG is drastically different from that of the monolayer graphene. Firstly, the adatom on rhombohedral stacked TLG (r-TLG) can have a Fano-shaped impurity spectral density, instead of the normal Lorentzian-like one, when the impurity level is around the Fermi level. Secondly, the impurity level of the adatom on r-TLG can be tuned into an in-gap state by an external electric field, which strongly depends on the direction of the applied electric field and can significantly affect the local magnetic moment formation. Finally, we systematically calculate the impurity magnetic phase diagrams, considering various stacking orders, adsorption sites, doping and electric field. We show that, because of the in-gap state, the impurity magnetic phase of r-TLG will obviously depend on the direction of the applied electric field as well. All our theoretical results can be readily tested in experiment, and may give a comprehensive understanding about the local magnetic moment of adatom on TLG.

cond-mat.mes-hall↗

arXiv2017

On Context-Dependent Clustering of Bandits

We investigate a novel cluster-of-bandit algorithm CAB for collaborative recommendation tasks that implements the underlying feedback sharing mechanism by estimating the neighborhood of users in a context-dependent manner. CAB makes sharp departures from the state of the art by incorporating collaborative effects into inference as well as learning processes in a manner that seamlessly interleaving explore-exploit tradeoffs and collaborative steps. We prove regret bounds under various assumptions on the data, which exhibit a crisp dependence on the expected number of clusters over the users, a natural measure of the statistical difficulty of the learning task. Experiments on production and real-world datasets show that CAB offers significantly increased prediction performance against a representative pool of state-of-the-art methods.

cs.LG↗

arXiv2017

Context-Aware Bandits

We propose an efficient Context-Aware clustering of Bandits (CAB) algorithm, which can capture collaborative effects. CAB can be easily deployed in a real-world recommendation system, where multi-armed bandits have been shown to perform well in particular with respect to the cold-start problem. CAB utilizes a context-aware clustering augmented by exploration-exploitation strategies. CAB dynamically clusters the users based on the content universe under consideration. We give a theoretical analysis in the standard stochastic multi-armed bandits setting. We show the efficiency of our approach on production and real-world datasets, demonstrate the scalability, and, more importantly, the significant increased prediction performance against several state-of-the-art methods.

cs.LG↗