arXiv subjects

Nan Jiang

Publications and source records attributed to Nan Jiang.

At least 253 records · Page 14Linked to original sources

arXiv2019

Online Supervised Learning for Traffic Load Prediction in Framed-ALOHA Networks

Predicting the current backlog, or traffic load, in framed-ALOHA networks enables the optimization of resource allocation, e.g., of the frame size. However, this prediction is made difficult by the lack of information about the cardinality of collisions and by possibly complex packet generation statistics. Assuming no prior information about the traffic model, apart from a bound on its temporal memory, this paper develops an online learning-based adaptive traffic load prediction method that is based on Recurrent Neural Networks (RNN) and specifically on the Long Short-Term Memory (LSTM) architecture. In order to enable online training in the absence of feedback on the exact cardinality of collisions, the proposed strategy leverages a novel approximate labeling technique that is inspired by Method of Moments (MOM) estimators. Numerical results show that the proposed online predictor considerably outperforms conventional methods and is able to adapt to changing traffic statistics.

cs.NI↗

arXiv2019

Model-based RL in Contextual Decision Processes: PAC bounds and Exponential Improvements over Model-free Approaches

We study the sample complexity of model-based reinforcement learning (henceforth RL) in general contextual decision processes that require strategic exploration to find a near-optimal policy. We design new algorithms for RL with a generic model class and analyze their statistical properties. Our algorithms have sample complexity governed by a new structural parameter called the witness rank, which we show to be small in several settings of interest, including factored MDPs. We also show that the witness rank is never larger than the recently proposed Bellman rank parameter governing the sample complexity of the model-free algorithm OLIVE (Jiang et al., 2017), the only other provably sample-efficient algorithm for global exploration at this level of generality. Focusing on the special case of factored MDPs, we prove an exponential lower bound for a general class of model-free approaches, including OLIVE, which, when combined with our algorithmic results, demonstrates exponential separation between model-based and model-free RL in some rich-observation settings.

cs.LG↗

arXiv2019

Information-Theoretic Considerations in Batch Reinforcement Learning

Value-function approximation methods that operate in batch mode have foundational importance to reinforcement learning (RL). Finite sample guarantees for these methods often crucially rely on two types of assumptions: (1) mild distribution shift, and (2) representation conditions that are stronger than realizability. However, the necessity ("why do we need them?") and the naturalness ("when do they hold?") of such assumptions have largely eluded the literature. In this paper, we revisit these assumptions and provide theoretical results towards answering the above questions, and make steps towards a deeper understanding of value-function approximation.

cs.LG↗

arXiv2019

The development and evaluation of the SmartAbility Android Application to detect users' abilities

The SmartAbility Android Application recommends Assistive Technology (AT) for people with reduced physical ability, by focusing on the actions (abilities) that can be performed independently. The Application utilises built-in sensor technologies in Android devices to detect user abilities, including head and limb movements, speech and blowing. The Application was evaluated by 18 participants with varying physical conditions and assessed through the System Usability Scale (SUS) and NASA Task Load Index (TLX). The Application achieved a SUS score of 72.5 (indicating 'Good Usability') with low levels of Temporal Demand and Frustration and medium levels of Mental Demand, Physical Demand and Effort. It is anticipated that the SmartAbility Application will be disseminated to the AT domain, to improve quality of life for people with reduced physical ability.

cs.HC↗

arXiv2019

On Oracle-Efficient PAC RL with Rich Observations

We study the computational tractability of PAC reinforcement learning with rich observations. We present new provably sample-efficient algorithms for environments with deterministic hidden state dynamics and stochastic rich observations. These methods operate in an oracle model of computation -- accessing policy and value function classes exclusively through standard optimization primitives -- and therefore represent computationally efficient alternatives to prior algorithms that require enumeration. With stochastic hidden state dynamics, we prove that the only known sample-efficient algorithm, OLIVE, cannot be implemented in the oracle model. We also present several examples that illustrate fundamental challenges of tractable PAC reinforcement learning in such general settings.

cs.LG↗

arXiv2018

Deep Reinforcement Learning for Real-Time Optimization in NB-IoT Networks

NarrowBand-Internet of Things (NB-IoT) is an emerging cellular-based technology that offers a range of flexible configurations for massive IoT radio access from groups of devices with heterogeneous requirements. A configuration specifies the amount of radio resource allocated to each group of devices for random access and for data transmission. Assuming no knowledge of the traffic statistics, there exists an important challenge in "how to determine the configuration that maximizes the long-term average number of served IoT devices at each Transmission Time Interval (TTI) in an online fashion". Given the complexity of searching for optimal configuration, we first develop real-time configuration selection based on the tabular Q-learning (tabular-Q), the Linear Approximation based Q-learning (LA-Q), and the Deep Neural Network based Q-learning (DQN) in the single-parameter single-group scenario. Our results show that the proposed reinforcement learning based approaches considerably outperform the conventional heuristic approaches based on load estimation (LE-URC) in terms of the number of served IoT devices. This result also indicates that LA-Q and DQN can be good alternatives for tabular-Q to achieve almost the same performance with much less training time. We further advance LA-Q and DQN via Actions Aggregation (AA-LA-Q and AA-DQN) and via Cooperative Multi-Agent learning (CMA-DQN) for the multi-parameter multi-group scenario, thereby solve the problem that Q-learning agents do not converge in high-dimensional configurations. In this scenario, the superiority of the proposed Q-learning approaches over the conventional LE-URC approach significantly improves with the increase of configuration dimensions, and the CMA-DQN approach outperforms the other approaches in both throughput and training efficiency.

cs.NI↗

arXiv2018

A note on Graphical Notation Reveals Topological Stability Criteria for Collective Dynamics in Complex Network

This paper clarifies the main research methods and ideas of the thesis [1,2,4]. The special calculation process is also realized by corresponding computer algorithm. Finally, we introduce zero rows sum case and give the corresponding algorithm, which greatly simplifies the relevant calculation of paper [1,4].

math.GM↗

arXiv2018

Cooperative Deep Reinforcement Learning for Multiple-Group NB-IoT Networks Optimization

cs.NI↗

arXiv2018

Taylor dispersion in two-dimensional bacterial turbulence

In this work, single particle dispersion was analyzed for a bacterial turbulence by retrieving the virtual Lagrangian trajectory via numerical integration of the Lagrangian equation. High-order displacement functions were calculated for cases with and without mean velocity effect. Two-regime power-law behavior for short and long time evolutions were identified experimentally, which were separated by the Lagrangian integral time. For the case with the mean velocity effect, the experimental Hurst numbers were determined to be $0.94$ and $0.97$ for short and long times evolutions, respectively. For the case without the mean velocity effect, the values were $0.88$ and $0.58$. Moreover, very weak intermittency correction was detected. All measured Hurst number were above $1/2$, the value of the normal diffusion, which verifies the super-diffusion behavior of living fluid. This behavior increases the efficiency of bacteria to obtain food.

physics.flu-dyn↗

arXiv2018

Extremal-point density of scaling processes from fractal Brownian motion to turbulence in one dimension

In recent years several local extrema based methodologies have been proposed to investigate either the nonlinear or the nonstationary time series for scaling analysis. In the present work we study systematically the distribution of the local extrema for both synthesized scaling processes and turbulent velocity data from experiments. The results show that for the fractional Brownian motion (fBm) without intermittency correction the measured extremal point density (EPD) agrees well with a theoretical prediction. For a multifractal random walk (MRW) with the lognormal statistics, the measured EPD is independent with the intermittency parameter $μ$, suggesting that the intermittency correction does not change the distribution of extremal points, but change the amplitude. By introducing a coarse-grained operator, the power-law behavior of these scaling processes is then revealed via the measured EPD for different scales. For fBm the scaling exponent $ξ(H)$ is found to be $ξ(H)=H$, where $H$ is Hurst number, while for MRW $ξ(H)$ shows a linear relation with the intermittency parameter $μ$. Such EPD approach is further applied to the turbulent velocity data obtained from a wind tunnel flow experiment with the Taylor scale $λ$ based Reynolds number $Re_λ= 720$, and a turbulent boundary layer with the momentum thickness $θ$ based Reynolds number $Re_θ= 810$. A scaling exponent $ξ\simeq 0.37$ is retrieved for the former case. For the latter one, the measured EPD shows clearly four regimes, which agree well with the four regimes of the turbulent boundary layer structures.

cond-mat.stat-mech↗

arXiv2018

Random Access Analysis for Massive IoT Networks under A New Spatio-Temporal Model: A Stochastic Geometry Approach

Massive Internet of Things (mIoT) has provided an auspicious opportunity to build powerful and ubiquitous connections that faces a plethora of new challenges, where cellular networks are potential solutions due to their high scalability, reliability, and efficiency. The Random Access CHannel (RACH) procedure is the first step of connection establishment between IoT devices and Base Stations (BSs) in the cellular-based mIoT network, where modelling the interactions between static properties of physical layer network and dynamic properties of queue evolving in each IoT device are challenging. To tackle this, we provide a novel traffic-aware spatio-temporal model to analyze RACH in cellular-based mIoT networks, where the physical layer network is modelled and analyzed based on stochastic geometry in the spatial domain, and the queue evolution is analyzed based on probability theory in the time domain. For performance evaluation, we derive the exact expressions for the preamble transmission success probabilities of a randomly chosen IoT device with different RACH schemes in each time slot, which offer insights into effectiveness of each RACH scheme. Our derived analytical results are verified by the realistic simulations capturing the evolution of packets in each IoT device. This mathematical model and analytical framework can be applied to evaluate the performance of other types of RACH schemes in the cellular-based networks by simply integrating its preamble transmission principle.

cs.IT↗

arXiv2018

The mechanism of nanoparticle precipitation induced by electron irradiation in transmission electron microscopy

Ru nanoparticles (NPs) can precipitate in Ru doped SiO2 amorphous thin films, triggered by electron irradiation in (scanning) transmission electron microscope ((S)TEM). A new mechanism was introduced to interpret the formation of metal NPs in (S)TEM. The induced electric field by electron irradiation, which originates from charging due to ionizations and excitations of atom electrons, can reduce the Gibbs free energy barrier for nucleation of metal particles. Furthermore, the directional ion drifting driven by the electric forces may accelerate the kinetic process of metal particle precipitation.

cond-mat.mes-hall↗

arXiv2018

Manipulation of Au nanoparticles using an electron probe: electron golf

A tool for manipulation of specific nanoparticles is essential for nanofabrication. This paper describes a new class of nanoscale techniques for manipulating individual nanoparticles on an insulating substrate using a focused electron beam. The controlled displacement of 20 nm Au nanoparticles is demonstrated, along with the extension to other systems, by exploiting the fundamental principles of particle motion under intense electron irradiation, which we elucidate. Excitation and ionization by the focused electron beam within the substrate are shown to produce an electric field on the particle in this new mechanism. The strength and direction of the electric field, and thus the motion of the nanoparticle, can be precisely controlled by the current density, exposure time and position of the electron beam.

cond-mat.mes-hall↗

arXiv2018

Hierarchical Imitation and Reinforcement Learning

We study how to effectively leverage expert feedback to learn sequential decision-making policies. We focus on problems with sparse rewards and long time horizons, which typically pose significant challenges in reinforcement learning. We propose an algorithmic framework, called hierarchical guidance, that leverages the hierarchical structure of the underlying problem to integrate different modes of expert interaction. Our framework can incorporate different combinations of imitation learning (IL) and reinforcement learning (RL) at different levels, leading to dramatic reductions in both expert effort and cost of exploration. Using long-horizon benchmarks, including Montezuma's Revenge, we demonstrate that our approach can learn significantly faster than hierarchical RL, and be significantly more label-efficient than standard IL. We also theoretically analyze labeling cost for certain instantiations of our framework.

cs.LG↗

arXiv2018

Image Classification Based on Quantum KNN Algorithm

Image classification is an important task in the field of machine learning and image processing. However, the usually used classification method --- the K Nearest-Neighbor algorithm has high complexity, because its two main processes: similarity computing and searching are time-consuming. Especially in the era of big data, the problem is prominent when the amount of images to be classified is large. In this paper, we try to use the powerful parallel computing ability of quantum computers to optimize the efficiency of image classification. The scheme is based on quantum K Nearest-Neighbor algorithm. Firstly, the feature vectors of images are extracted on classical computers. Then the feature vectors are inputted into a quantum superposition state, which is used to achieve parallel computing of similarity. Next, the quantum minimum search algorithm is used to speed up searching process for similarity. Finally, the image is classified by quantum measurement. The complexity of the quantum algorithm is only O((kM)^(1/2)), which is superior to the classical algorithms. Moreover, the measurement step is executed only once to ensure the validity of the scheme. The experimental results show that, the classification accuracy is 83.1% on Graz-01 dataset and 78% on Caltech-101 dataset, which is close to existing classical algorithms. Hence, our quantum scheme has a good classification performance while greatly improving the efficiency.

cs.CV↗

arXiv2018

Optimizing Recurrent Neural Networks Architectures under Time Constraints

Recurrent neural network (RNN)'s architecture is a key factor influencing its performance. We propose algorithms to optimize hidden sizes under running time constraint. We convert the discrete optimization into a subset selection problem. By novel transformations, the objective function becomes submodular and constraint becomes supermodular. A greedy algorithm with bounds is suggested to solve the transformed problem. And we show how transformations influence the bounds. To speed up optimization, surrogate functions are proposed which balance exploration and exploitation. Experiments show that our algorithms can find more accurate models or faster models than manually tuned state-of-the-art and random search. We also compare popular RNN architectures using our algorithms.

stat.ML↗

arXiv2018

Neural Network Architecture Optimization through Submodularity and Supermodularity

Deep learning models' architectures, including depth and width, are key factors influencing models' performance, such as test accuracy and computation time. This paper solves two problems: given computation time budget, choose an architecture to maximize accuracy, and given accuracy requirement, choose an architecture to minimize computation time. We convert this architecture optimization into a subset selection problem. With accuracy's submodularity and computation time's supermodularity, we propose efficient greedy optimization algorithms. The experiments demonstrate our algorithm's ability to find more accurate models or faster models. By analyzing architecture evolution with growing time budget, we discuss relationships among accuracy, time and architecture, and give suggestions on neural network architecture design.

stat.ML↗

arXiv2018

Hand Segmentation for Hand-Object Interaction from Depth map

Hand segmentation for hand-object interaction is a necessary preprocessing step in many applications such as augmented reality, medical application, and human-robot interaction. However, typical methods are based on color information which is not robust to objects with skin color, skin pigment difference, and light condition variations. Thus, we propose hand segmentation method for hand-object interaction using only a depth map. It is challenging because of the small depth difference between a hand and objects during an interaction. To overcome this challenge, we propose the two-stage random decision forest (RDF) method consisting of detecting hands and segmenting hands. To validate the proposed method, we demonstrate results on the publicly available dataset of hand segmentation for hand-object interaction. The proposed method achieves high accuracy in short processing time comparing to the other state-of-the-art methods.

cs.CV↗