arXiv subjects

Shuang Wu

Publications and source records attributed to Shuang Wu.

At least 127 records · Page 7Linked to original sources

arXiv2021

Deep Dual Consecutive Network for Human Pose Estimation

Multi-frame human pose estimation in complicated situations is challenging. Although state-of-the-art human joints detectors have demonstrated remarkable results for static images, their performances come short when we apply these models to video sequences. Prevalent shortcomings include the failure to handle motion blur, video defocus, or pose occlusions, arising from the inability in capturing the temporal dependency among video frames. On the other hand, directly employing conventional recurrent neural networks incurs empirical difficulties in modeling spatial contexts, especially for dealing with pose occlusions. In this paper, we propose a novel multi-frame human pose estimation framework, leveraging abundant temporal cues between video frames to facilitate keypoint detection. Three modular components are designed in our framework. A Pose Temporal Merger encodes keypoint spatiotemporal context to generate effective searching scopes while a Pose Residual Fusion module computes weighted pose residuals in dual directions. These are then processed via our Pose Correction Network for efficient refining of pose estimations. Our method ranks No.1 in the Multi-frame Person Pose Estimation Challenge on the large-scale benchmark datasets PoseTrack2017 and PoseTrack2018. We have released our code, hoping to inspire future research.

cs.CV↗

arXiv2021

Aggregated Multi-GANs for Controlled 3D Human Motion Prediction

Human motion prediction from historical pose sequence is at the core of many applications in machine intelligence. However, in current state-of-the-art methods, the predicted future motion is confined within the same activity. One can neither generate predictions that differ from the current activity, nor manipulate the body parts to explore various future possibilities. Undoubtedly, this greatly limits the usefulness and applicability of motion prediction. In this paper, we propose a generalization of the human motion prediction task in which control parameters can be readily incorporated to adjust the forecasted motion. Our method is compelling in that it enables manipulable motion prediction across activity types and allows customization of the human movement in a variety of fine-grained ways. To this aim, a simple yet effective composite GAN structure, consisting of local GANs for different body parts and aggregated via a global GAN is presented. The local GANs game in lower dimensions, while the global GAN adjusts in high dimensional space to avoid mode collapse. Extensive experiments show that our method outperforms state-of-the-art. The codes are available at https://github.com/herolvkd/AM-GAN.

cs.CV↗

arXiv2020

Chern Insulators and Topological Flat-bands in Magic-angle Twisted Bilayer Graphene

Magic-angle twisted bilayer graphene (MA-TBG) exhibits intriguing quantum phase transitions triggered by enhanced electron-electron interactions when its flat-bands are partially filled. However, the phases themselves and their connection to the putative non-trivial topology of the flat bands are largely unexplored. Here we report transport measurements revealing a succession of doping-induced Lifshitz transitions that are accompanied by van Hove singularities (VHS) which facilitate the emergence of correlation-induced gaps and topologically non-trivial sub-bands. In the presence of a magnetic field, well quantized Hall plateaus at filling of 1, 2, 3 carriers per moiré-cell reveal the sub-band topology and signal the emergence of Chern insulators with Chern-numbers, ! = !, !, !, respectively. Surprisingly, for magnetic fields exceeding 5T we observe a VHS at a filling of 3.5, suggesting the possibility of a fractional Chern insulator. This VHS is accompanied by a crossover from low-temperature metallic, to high-temperature insulating behavior, characteristic of entropically driven Pomeranchuk-like transitions,

cond-mat.mes-hall↗

arXiv2020

High-order minibands and interband Landau level reconstruction in graphene moire superlattice

The propagation of Dirac fermions in graphene through a long-period periodic potential would result in a band folding together with the emergence of a series of cloned Dirac points (DPs). In highly aligned graphene/hexagonal boron nitride (G/hBN) heterostructures, the lattice mismatch between the two atomic crystals generates a unique kind of periodic structure known as a moiré superlattice. Of particular interests is the emergent phenomena related to the reconstructed band-structure of graphene, such as the Hofstadter butterfly, topological currents, gate dependent pseudospin mixing, and ballistic miniband conduction. However, most studies so far have been limited to the lower-order minibands, e.g. the 1st and 2nd minibands counted from charge neutrality, and consequently the fundamental nature of the reconstructed higher-order miniband spectra still remains largely unknown. Here we report on probing the higher-order minibands of precisely aligned graphene moiré superlattices by transport spectroscopy. Using dual electrostatic gating, the edges of these high-order minibands, i.e. the 3rd and 4th minibands, can be reached. Interestingly, we have observed interband Landau level (LL) crossinginducing gap closures in a multiband magneto-transport regime, which originates from band overlap between the 2nd and 3rd minibands. As observed high-order minibands and LL reconstruction qualitatively match our simulated results. Our findings highlight the synergistic effect of minibands in transport, thus presenting a new opportunity for graphene electronic devices.

cond-mat.mes-hall↗

arXiv2020

Optimal Scheduling of Multiple Sensors over Lossy and Bandwidth Limited Channels

This work considers the sensor scheduling for multiple dynamic processes. We consider $n$ linear dynamic processes, the state of each process is measured by a sensor, which transmits their local state estimates over wireless channels to a remote estimator with certain communication costs. In each time step, only a portion of the sensors is allowed to transmit data to the remote estimator and the packet might be lost due to unreliability of the wireless channels. Our goal is to find a scheduling policy which coordinates the sensors in a centralized manner to minimize the total expected estimation error of the remote estimator and the communication costs. We formulate the problem as a Markov decision process. We develop an algorithm to check whether there exists a deterministic stationary optimal policy. We show the optimality of monotone policies, which saves computational effort of finding an optimal policy and facilitates practical implementation. Nevertheless, obtaining an exact optimal policy still suffers from curse of dimensionality when the number of processes are large. We further provide an index-based heuristics to avoid brute force computation. Numerical examples are presented to illustrate our theoretical results.

eess.SY↗

arXiv2019

Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integers

Deep neural network (DNN) quantization converting floating-point (FP) data in the network to integers (INT) is an effective way to shrink the model size for memory saving and simplify the operations for compute acceleration. Recently, researches on DNN quantization develop from inference to training, laying a foundation for the online training on accelerators. However, existing schemes leaving batch normalization (BN) untouched during training are mostly incomplete quantization that still adopts high precision FP in some parts of the data paths. Currently, there is no solution that can use only low bit-width INT data during the whole training process of large-scale DNNs with acceptable accuracy. In this work, through decomposing all the computation steps in DNNs and fusing three special quantization functions to satisfy the different precision requirements, we propose a unified complete quantization framework termed as ``WAGEUBN'' to quantize DNNs involving all data paths including W (Weights), A (Activation), G (Gradient), E (Error), U (Update), and BN. Moreover, the Momentum optimizer is also quantized to realize a completely quantized framework. Experiments on ResNet18/34/50 models demonstrate that WAGEUBN can achieve competitive accuracy on the ImageNet dataset. For the first time, the study of quantization in large-scale DNNs is advanced to the full 8-bit INT level. In this way, all the operations in the training and inference can be bit-wise operations, pushing towards faster processing speed, decreased memory cost, and higher energy efficiency. Our throughout quantization framework has great potential for future efficient portable devices with online learning ability.

cs.LG↗

arXiv2019

Electrostatic Imaging of Encapsulated Graphene

Devices made from two dimensional materials such as graphene and transition metal dichalcogenides exhibit remarkable electronic properties of interest to many subdisciplines of nanoscience. Owing to their 2D nature, their quality is highly susceptible to contamination and degradation when exposed to the ambient environment. Protecting the 2D layers by encapsulation between hexagonal boron nitride layers significantly improves their quality. Locating these samples within the encapsulant and assessing their integrity prior to further processing then becomes challenging. Here we show that conductive scanning probe techniques such as electrostatic force and Kelvin force microscopy makes it possible to visualize the encapsulated layers, their charge environment and local defects including cracks and bubbles on the sub-micrometer scale. Our techniques are employed without requiring electrical contact to the embedded layer, providing valuable feedback on the local electronic quality prior to any device etching or electrode deposition. We show that these measurement modes, which are simple extensions of atomic force microscopy, are perfectly suited for imaging encapsulated conductors and their local charge environments.

cond-mat.mes-hall↗

arXiv2019

Learning Optimal Scheduling Policy for Remote State Estimation under Uncertain Channel Condition

We consider optimal sensor scheduling with unknown communication channel statistics. We formulate two types of scheduling problems with the communication rate being a soft or hard constraint, respectively. We first present some structural results on the optimal scheduling policy using dynamic programming and assuming the channel statistics is known. We prove that the Q-factor is monotonic and submodular, which leads to the threshold-like structures in both types of problems. Then we develop a stochastic approximation and parameter learning frameworks to deal with the two scheduling problems with unknown channel statistics. We utilize their structures to design specialized learning algorithms. We prove the convergence of these algorithms. Performance improvement compared with the standard Q-learning algorithm is shown through numerical examples.

eess.SY↗

arXiv2019

Max-Min Fair Sensor Scheduling: Game-theoretic Perspective and Algorithmic Solution

We consider the design of a fair sensor schedule for a number of sensors monitoring different linear time-invariant processes. The largest average remote estimation error among all processes is to be minimized. We first consider a general setup for the max-min fair allocation problem. By reformulating the problem as its equivalent form, we transform the fair resource allocation problem into a zero-sum game between a "judge" and a resource allocator. We propose an equilibrium seeking procedure and show that there exists a unique Nash equilibrium in pure strategy for this game. We then apply the result to the sensor scheduling problem and show that the max-min fair sensor scheduling policy can be achieved.

eess.SY↗

arXiv2019

Stochastic Sparse Learning with Momentum Adaptation for Imprecise Memristor Networks

Memristor based neural networks have great potentials in on-chip neuromorphic computing systems due to the fast computation and low-energy consumption. However, the imprecise properties of existing memristor devices generally result in catastrophic failures for the network in-situ training, which significantly impedes their engineering applications. In this work, we design a novel learning scheme that integrates stochastic sparse updating with momentum adaption (SSM) to efficiently train the imprecise memristor networks with high classification accuracy. The SSM scheme consists of: (1) a stochastic and discrete learning method to make weight updates sparse; (2) a momentum based gradient algorithm to eliminate training noises and distill robust updates; (3) a network re-initialization method to mitigate the device-to-device variation; (4) an update compensation strategy to further stabilize the weight programming process. With the SSM scheme, experiments show that the classification accuracy on multilayer perceptron (MLP) and convolutional neural network (CNN) improves from 26.12% to 90.07% and from 65.98% to 92.38%, respectively. Meanwhile, the total numbers of weight updating pulses decrease 90% and 40% in MLP and CNN, respectively, and the convergence rates are both 3x faster. The SSM scheme provides a high-accuracy, low-power, and fast-convergence solution for the in-situ training of imprecise memristor networks, which is crucial to future neuromorphic intelligence systems.

cs.ET↗

arXiv2019

Convolution with even-sized kernels and symmetric padding

Compact convolutional neural networks gain efficiency mainly through depthwise convolutions, expanded channels and complex topologies, which contrarily aggravate the training process. Besides, 3x3 kernels dominate the spatial representation in these models, whereas even-sized kernels (2x2, 4x4) are rarely adopted. In this work, we quantify the shift problem occurs in even-sized kernel convolutions by an information erosion hypothesis, and eliminate it by proposing symmetric padding on four sides of the feature maps (C2sp, C4sp). Symmetric padding releases the generalization capabilities of even-sized kernels at little computational cost, making them outperform 3x3 kernels in image classification and generation tasks. Moreover, C2sp obtains comparable accuracy to emerging compact models with much less memory and time consumption during training. Symmetric padding coupled with even-sized convolutions can be neatly implemented into existing frameworks, providing effective elements for architecture designs, especially on online and continual learning occasions where training efforts are emphasized.

cs.CV↗

arXiv2019

Giant and nonreciprocal second harmonic generation from layered antiferromagnetism in bilayer CrI3

Layered antiferromagnetism is the spatial arrangement of ferromagnetic layers with antiferromagnetic interlayer coupling. Recently, the van der Waals magnet, chromium triiodide (CrI3), emerged as the first layered antiferromagnetic insulator in its few-layer form, opening up ample opportunities for novel device functionalities. Here, we discovered an emergent nonreciprocal second order nonlinear optical effect in bilayer CrI3. The observed second harmonic generation (SHG) is giant: several orders of magnitude larger than known magnetization induced SHG and comparable to SHG in the best 2D nonlinear optical materials studied so far (e.g. MoS2). We showed that while the parent lattice of bilayer CrI3 is centrosymmetric and thus does not contribute to the SHG signal, the observed nonreciprocal SHG originates purely from the layered antiferromagnetic order, which breaks both spatial inversion and time reversal symmetries. Furthermore, polarization-resolved measurements revealed the underlying C2h symmetry, and thus monoclinic stacking order in CrI3 bilayers, providing crucial structural information for the microscopic origin of layered antiferromagnetism. Our results highlight SHG as a highly sensitive probe that can reveal subtle magnetic order and open novel nonlinear and nonreciprocal optical device possibilities based on 2D magnets.

cond-mat.mes-hall↗

arXiv2019

The algebraic area of closed lattice random walks

We propose a formula for the enumeration of closed lattice random walks of length $n$ enclosing a given algebraic area. The information is contained in the Kreft coefficients which encode, in the commensurate case, the Hofstadter secular equation for a quantum particle hopping on a lattice coupled to a perpendicular magnetic field. The algebraic area enumeration is possible because it is split in $2^{n/2-1}$ pieces, each tractable in terms of explicit combinatorial expressions.

math-ph↗

arXiv2019

Evidence of Gate-Tunable Mott Insulator in Trilayer Graphene-Boron Nitride Moiré Superlattice

Mott insulator plays a central role in strongly correlated physics, where the repulsive Coulomb interaction dominates over the electron kinetic energy and leads to insulating states with one electron occupying each unit cell. Doped Mott insulator is often described by the Hubbard model3, which can give rise to other correlated phenomena such as unusual magnetism and even high-temperature superconductivity. A tunable Mott insulator, where the competition between the Coulomb interaction and the kinetic energy can be varied in situ, can provide an invaluable model system for the study of Mott physics. Here we report the realization of such a tunable Mott insulator in the ABC trilayer graphene (TLG) and hexagonal boron nitride (hBN) heterostructure with a moiré superlattice. Unlike massless Dirac electrons in monolayer graphene, electrons in pristine ABC TLG are characterized by quartic energy dispersion and large effective mass that are conducive for strongly correlated phenomena. The moiré superlattice in TLG/hBN heterostructures leads to narrow electronic minibands that are gate tunable. Each filled miniband contains 4 electrons in one moiré lattice site due to the spin and valley degeneracy of graphene. The Mott insulator states emerge at 1/4 and 1/2 fillings, corresponding to one electron and two electrons per site, respectively. Moreover, the Mott states in the ABC TLG/hBN heterostructure exhibit unprecedented tunability: the Mott gap can be modulated in situ by a vertical electrical field, and at the meantime, the electron doping can be gate-tuned to fill the band from one Mott insulating state to another. Our observation of a tunable Mott insulator opens up exciting opportunities to explore novel strongly correlated phenomena in two-dimensional moiré superlattice heterostructures.

cond-mat.mes-hall↗

arXiv2018

Robust and Efficient Graph Correspondence Transfer for Person Re-identification

Spatial misalignment caused by variations in poses and viewpoints is one of the most critical issues that hinders the performance improvement in existing person re-identification (Re-ID) algorithms. To address this problem, in this paper, we present a robust and efficient graph correspondence transfer (REGCT) approach for explicit spatial alignment in Re-ID. Specifically, we propose to establish the patch-wise correspondences of positive training pairs via graph matching. By exploiting both spatial and visual contexts of human appearance in graph matching, meaningful semantic correspondences can be obtained. To circumvent the cumbersome \emph{on-line} graph matching in testing phase, we propose to transfer the \emph{off-line} learned patch-wise correspondences from the positive training pairs to test pairs. In detail, for each test pair, the training pairs with similar pose-pair configurations are selected as references. The matching patterns (i.e., the correspondences) of the selected references are then utilized to calculate the patch-wise feature distances of this test pair. To enhance the robustness of correspondence transfer, we design a novel pose context descriptor to accurately model human body configurations, and present an approach to measure the similarity between a pair of pose context descriptors. Meanwhile, to improve testing efficiency, we propose a correspondence template ensemble method using the voting mechanism, which significantly reduces the amount of patch-wise matchings involved in distance calculation. With aforementioned strategies, the REGCT model can effectively and efficiently handle the spatial misalignment problem in Re-ID. Extensive experiments on five challenging benchmarks, including VIPeR, Road, PRID450S, 3DPES and CUHK01, evidence the superior performance of REGCT over other state-of-the-art approaches.

cs.CV↗

arXiv2018

Graph Correspondence Transfer for Person Re-identification

In this paper, we propose a graph correspondence transfer (GCT) approach for person re-identification. Unlike existing methods, the GCT model formulates person re-identification as an off-line graph matching and on-line correspondence transferring problem. In specific, during training, the GCT model aims to learn off-line a set of correspondence templates from positive training pairs with various pose-pair configurations via patch-wise graph matching. During testing, for each pair of test samples, we select a few training pairs with the most similar pose-pair configurations as references, and transfer the correspondences of these references to test pair for feature distance calculation. The matching score is derived by aggregating distances from different references. For each probe image, the gallery image with the highest matching score is the re-identifying result. Compared to existing algorithms, our GCT can handle spatial misalignment caused by large variations in view angles and human poses owing to the benefits of patch-wise graph matching. Extensive experiments on five benchmarks including VIPeR, Road, PRID450S, 3DPES and CUHK01 evidence the superior performance of GCT model over other state-of-the-art methods.

cs.CV↗

arXiv2018

L1-Norm Batch Normalization for Efficient Training of Deep Neural Networks

Batch Normalization (BN) has been proven to be quite effective at accelerating and improving the training of deep neural networks (DNNs). However, BN brings additional computation, consumes more memory and generally slows down the training process by a large margin, which aggravates the training effort. Furthermore, the nonlinear square and root operations in BN also impede the low bit-width quantization techniques, which draws much attention in deep learning hardware community. In this work, we propose an L1-norm BN (L1BN) with only linear operations in both the forward and the backward propagations during training. L1BN is shown to be approximately equivalent to the original L2-norm BN (L2BN) by multiplying a scaling factor. Experiments on various convolutional neural networks (CNNs) and generative adversarial networks (GANs) reveal that L1BN maintains almost the same accuracies and convergence rates compared to L2BN but with higher computational efficiency. On FPGA platform, the proposed signum and absolute operations in L1BN can achieve 1.5$\times$ speedup and save 50\% power consumption, compared with the original costly square and root operations, respectively. This hardware-friendly normalization method not only surpasses L2BN in speed, but also simplify the hardware design of ASIC accelerators with higher energy efficiency. Last but not the least, L1BN promises a fully quantized training of DNNs, which is crucial to future adaptive terminal devices.

cs.LG↗

arXiv2018

Training and Inference with Integers in Deep Neural Networks

Researches on deep neural networks with discrete parameters and their deployment in embedded systems have been active and promising topics. Although previous works have successfully reduced precision in inference, transferring both training and inference processes to low-bitwidth integers has not been demonstrated simultaneously. In this work, we develop a new method termed as "WAGE" to discretize both training and inference, where weights (W), activations (A), gradients (G) and errors (E) among layers are shifted and linearly constrained to low-bitwidth integers. To perform pure discrete dataflow for fixed-point devices, we further replace batch normalization by a constant scaling layer and simplify other components that are arduous for integer implementation. Improved accuracies can be obtained on multiple datasets, which indicates that WAGE somehow acts as a type of regularization. Empirically, we demonstrate the potential to deploy training in hardware systems such as integer-based deep learning accelerators and neuromorphic chips with comparable accuracy and higher energy efficiency, which is crucial to future AI applications in variable scenarios with transfer and continual learning demands.

cs.LG↗