arXiv subjects

Limin Wang

Publications and source records attributed to Limin Wang.

At least 253 records · Page 14Linked to original sources

arXiv2018

StNet: Local and Global Spatial-Temporal Modeling for Action Recognition

Despite the success of deep learning for static image understanding, it remains unclear what are the most effective network architectures for the spatial-temporal modeling in videos. In this paper, in contrast to the existing CNN+RNN or pure 3D convolution based approaches, we explore a novel spatial temporal network (StNet) architecture for both local and global spatial-temporal modeling in videos. Particularly, StNet stacks N successive video frames into a \emph{super-image} which has 3N channels and applies 2D convolution on super-images to capture local spatial-temporal relationship. To model global spatial-temporal relationship, we apply temporal convolution on the local spatial-temporal feature maps. Specifically, a novel temporal Xception block is proposed in StNet. It employs a separate channel-wise and temporal-wise convolution over the feature sequence of video. Extensive experiments on the Kinetics dataset demonstrate that our framework outperforms several state-of-the-art approaches in action recognition and can strike a satisfying trade-off between recognition accuracy and model complexity. We further demonstrate the generalization performance of the leaned video representations on the UCF101 dataset.

cs.CV↗

arXiv2018

CoAs: The line of 3d demarcation

Transition metal-pnictide compounds have received attention for their tendency to combine magnetism and unconventional superconductivity. Binary CoAs lies on the border of paramagnetism and the more complex behavior seen in isostructural CrAs, MnP, FeAs, and FeP. Here we report the properties of CoAs single crystals grown with two distinct techniques along with density functional theory calculations of its electronic structure and magnetic ground state. While all indications are that CoAs is paramagnetic, both experiment and theory suggest proximity to a ferromagnetic instability. Quantum oscillations are seen in torque measurements up to 31.5~T, and support the calculated paramagnetic Fermiology.

cond-mat.str-el↗

arXiv2018

Appearance-and-Relation Networks for Video Classification

Spatiotemporal feature learning in videos is a fundamental problem in computer vision. This paper presents a new architecture, termed as Appearance-and-Relation Network (ARTNet), to learn video representation in an end-to-end manner. ARTNets are constructed by stacking multiple generic building blocks, called as SMART, whose goal is to simultaneously model appearance and relation from RGB input in a separate and explicit manner. Specifically, SMART blocks decouple the spatiotemporal learning module into an appearance branch for spatial modeling and a relation branch for temporal modeling. The appearance branch is implemented based on the linear combination of pixels or filter responses in each frame, while the relation branch is designed based on the multiplicative interactions between pixels or filter responses across multiple frames. We perform experiments on three action recognition benchmarks: Kinetics, UCF101, and HMDB51, demonstrating that SMART blocks obtain an evident improvement over 3D convolutions for spatiotemporal feature learning. Under the same training setting, ARTNets achieve superior performance on these three datasets to the existing state-of-the-art methods.

cs.CV↗

arXiv2018

Evolution of Structure and Superconductivity in Ba(Ni$_{1-x}$Co$_x$)$_2$As$_2$

The effects of Co-substitution on Ba(Ni$_{1-x}$Co$_x$)$_2$As$_2$ ($0\leq x\leq 0.251$) single crystals grown out of Pb flux are investigated via transport, magnetic, and thermodynamic measurements. BaNi$_2$As$_2$ exhibits a first order tetragonal to triclinic structural phase transition at $T_s=137 K$ upon cooling, and enters a superconducting phase below $T_c=0.7 K$. The structural phase transition is sensitive to cobalt content and is suppressed completely by $x\geq0.133$. The superconducting critical temperature, $T_c$, increases continuously with $x$, reaching a maximum of $T_c=2.3 K$ at the structural critical point $x=0.083$ and then decreases monotonically until superconductivity is no longer observable well into the tetragonal phase. In contrast to similar BaNi$_2$As$_2$ substitutional studies, which show an abrupt change in $T_c$ at the triclinic-tetragonal boundary that extends far into the tetragonal phase, Ba(Ni$_{1-x}$Co$_x$)$_2$As$_2$ exhibits a dome-like phase diagram centered around the first-order critical point. Together with an anomalously large heat capacity jump $ΔC_e/γT\sim 2.2$ at optimal doping, the smooth evolution of $T_c$ in the Ba(Ni$_{1-x}$Co$_x$)$_2$As$_2$ system suggests a mechanism for pairing enhancement other than phonon softening.

cond-mat.supr-con↗

arXiv2018

Structured Triplet Learning with POS-tag Guided Attention for Visual Question Answering

Visual question answering (VQA) is of significant interest due to its potential to be a strong test of image understanding systems and to probe the connection between language and vision. Despite much recent progress, general VQA is far from a solved problem. In this paper, we focus on the VQA multiple-choice task, and provide some good practices for designing an effective VQA model that can capture language-vision interactions and perform joint reasoning. We explore mechanisms of incorporating part-of-speech (POS) tag guided attention, convolutional n-grams, triplet attention interactions between the image, question and candidate answer, and structured learning for triplets based on image-question pairs. We evaluate our models on two popular datasets: Visual7W and VQA Real Multiple Choice. Our final model achieves the state-of-the-art performance of 68.2% on Visual7W, and a very competitive performance of 69.6% on the test-standard split of VQA Real Multiple Choice.

cs.CV↗

arXiv2017

Temporal Action Detection with Structured Segment Networks

Detecting actions in untrimmed videos is an important yet challenging task. In this paper, we present the structured segment network (SSN), a novel framework which models the temporal structure of each action instance via a structured temporal pyramid. On top of the pyramid, we further introduce a decomposed discriminative model comprising two classifiers, respectively for classifying actions and determining completeness. This allows the framework to effectively distinguish positive proposals from background or incomplete ones, thus leading to both accurate recognition and localization. These components are integrated into a unified network that can be efficiently trained in an end-to-end fashion. Additionally, a simple yet effective temporal action proposal scheme, dubbed temporal actionness grouping (TAG) is devised to generate high quality action proposals. On two challenging benchmarks, THUMOS14 and ActivityNet, our method remarkably outperforms previous state-of-the-art methods, demonstrating superior accuracy and strong adaptivity in handling actions with various temporal structures.

cs.CV↗

arXiv2017

WebVision Database: Visual Learning and Understanding from Web Data

In this paper, we present a study on learning visual recognition models from large scale noisy web data. We build a new database called WebVision, which contains more than $2.4$ million web images crawled from the Internet by using queries generated from the 1,000 semantic concepts of the benchmark ILSVRC 2012 dataset. Meta information along with those web images (e.g., title, description, tags, etc.) are also crawled. A validation set and test set containing human annotated images are also provided to facilitate algorithmic development. Based on our new database, we obtain a few interesting observations: 1) the noisy web images are sufficient for training a good deep CNN model for visual recognition; 2) the model learnt from our WebVision database exhibits comparable or even better generalization ability than the one trained from the ILSVRC 2012 dataset when being transferred to new datasets and tasks; 3) a domain adaptation issue (a.k.a., dataset bias) is observed, which means the dataset can be used as the largest benchmark dataset for visual domain adaptation. Our new WebVision database and relevant studies in this work would benefit the advance of learning state-of-the-art visual models with minimum supervision based on web data.

cs.CV↗

arXiv2017

Beyond Triplet: Unconventional Superconductivity in a Spin-3/2 Topological Semimetal

In all known fermionic superfluids, Cooper pairs are composed of spin-1/2 quasi-particles that pair to form either spin-singlet or spin-triplet bound states. The "spin" of a Bloch electron, however, is fixed by the symmetries of the crystal and the atomic orbitals from which it is derived, and in some cases can behave as if it were a spin-3/2 particle. The superconducting state of such a system allows pairing beyond spin-triplet, with higher spin quasi-particles combining to form quintet or septet pairs. Here, we report evidence of unconventional superconductivity emerging from a spin-3/2 quasiparticle electronic structure in the half-Heusler semimetal YPtBi, a low-carrier density noncentrosymmetric cubic material with a high symmetry that preserves the $p$-like $j=3/2$ manifold in the Bi-based $Γ_8$ band in the presence of strong spin-orbit coupling. With a striking linear temperature dependence of the London penetration depth, the existence of line nodes in the superconducting order parameter $Δ$ is directly explained by a mixed-parity Cooper pairing model with high total angular momentum, consistent with a high-spin fermionic superfluid state. We propose a $\mathbf{k\cdot p}$ model of the $j=3/2$ fermions to explain how a dominant $J$=3 septet pairing state is the simplest solution that naturally produces nodes in the mixed even-odd parity gap. Together with the underlying topologically non-trivial band structure, the unconventional pairing in this system represents a truly novel form of superfluidity that has strong potential for leading the development of a new generation of topological superconductors.

cond-mat.supr-con↗

arXiv2017

Phase Transition of Na3SbS4 Superionic Conductor and Its Impact on Ion Transport

In this work, a comprehensive research coupling experiment and computation has been performed to understand the phase transition of Na3SbS4 and to synthesize cubic Na3SbS4 (c-Na3SbS4), a high temperature phase of Na3SbS4 that is difficult to be preserved from high temperature. The formation of c-Na3SbS4 is verified by Rietveld refinement and electrochemical impedance spectroscopy. Both experiment and theoretical calculation reveal that the ionic conductivity of c-Na3SbS4 is higher than that of t-Na3SbS4, though the values are in the same order of magnitude. Both structures allow fast ion transport.

cond-mat.mtrl-sci↗

arXiv2017

Quantum Oscillations in the Anomalous Spin Density Wave State of FeAs

Quantum oscillations in the binary antiferromagnetic metal FeAs are presented and compared to theoretical predictions for the electronic band structure in the anomalous spin density wave state of this material. Demonstrating a new method for growing single crystals out of Bi flux, we utilize the highest quality FeAs to perform torque magnetometry experiments up to 35 T, using rotations of field angle in two planes to provide evidence for one electron and one hole band in the magnetically ordered state. The resulting picture agrees with previous experimental evidence for multiple carriers at low temperatures, but the exact Fermi surface shape differs from predictions, suggesting that correlations play a role in deviation from ab initio theory and cause up to a four-fold enhancement in the effective carrier mass.

cond-mat.str-el↗

arXiv2017

UntrimmedNets for Weakly Supervised Action Recognition and Detection

Current action recognition methods heavily rely on trimmed videos for model training. However, it is expensive and time-consuming to acquire a large-scale trimmed video dataset. This paper presents a new weakly supervised architecture, called UntrimmedNet, which is able to directly learn action recognition models from untrimmed videos without the requirement of temporal annotations of action instances. Our UntrimmedNet couples two important components, the classification module and the selection module, to learn the action models and reason about the temporal duration of action instances, respectively. These two components are implemented with feed-forward networks, and UntrimmedNet is therefore an end-to-end trainable architecture. We exploit the learned models for action recognition (WSR) and detection (WSD) on the untrimmed video datasets of THUMOS14 and ActivityNet. Although our UntrimmedNet only employs weak supervision, our method achieves performance superior or comparable to that of those strongly supervised approaches on these two datasets.

cs.CV↗

arXiv2017

WebVision Challenge: Visual Learning and Understanding With Web Data

We present the 2017 WebVision Challenge, a public image recognition challenge designed for deep learning based on web images without instance-level human annotation. Following the spirit of previous vision challenges, such as ILSVRC, Places2 and PASCAL VOC, which have played critical roles in the development of computer vision by contributing to the community with large scale annotated data for model designing and standardized benchmarking, we contribute with this challenge a large scale web images dataset, and a public competition with a workshop co-located with CVPR 2017. The WebVision dataset contains more than $2.4$ million web images crawled from the Internet by using queries generated from the $1,000$ semantic concepts of the benchmark ILSVRC 2012 dataset. Meta information is also included. A validation set and test set containing human annotated images are also provided to facilitate algorithmic development. The 2017 WebVision challenge consists of two tracks, the image classification task on WebVision test set, and the transfer learning task on PASCAL VOC 2012 dataset. In this paper, we describe the details of data collection and annotation, highlight the characteristics of the dataset, and introduce the evaluation metrics.

cs.CV↗

arXiv2017

Temporal Segment Networks for Action Recognition in Videos

Deep convolutional networks have achieved great success for image recognition. However, for action recognition in videos, their advantage over traditional methods is not so evident. We present a general and flexible video-level framework for learning action models in videos. This method, called temporal segment network (TSN), aims to model long-range temporal structures with a new segment-based sampling and aggregation module. This unique design enables our TSN to efficiently learn action models by using the whole action videos. The learned models could be easily adapted for action recognition in both trimmed and untrimmed videos with simple average pooling and multi-scale temporal window integration, respectively. We also study a series of good practices for the instantiation of TSN framework given limited training samples. Our approach obtains the state-the-of-art performance on four challenging action recognition benchmarks: HMDB51 (71.0%), UCF101 (94.9%), THUMOS14 (80.1%), and ActivityNet v1.2 (89.6%). Using the proposed RGB difference for motion models, our method can still achieve competitive accuracy on UCF101 (91.0%) while running at 340 FPS. Furthermore, based on the temporal segment networks, we won the video classification track at the ActivityNet challenge 2016 among 24 teams, which demonstrates the effectiveness of TSN and the proposed good practices.

cs.CV↗

arXiv2017

Thin-Slicing Network: A Deep Structured Model for Pose Estimation in Videos

Deep ConvNets have been shown to be effective for the task of human pose estimation from single images. However, several challenging issues arise in the video-based case such as self-occlusion, motion blur, and uncommon poses with few or no examples in training data sets. Temporal information can provide additional cues about the location of body joints and help to alleviate these issues. In this paper, we propose a deep structured model to estimate a sequence of human poses in unconstrained videos. This model can be efficiently trained in an end-to-end manner and is capable of representing appearance of body joints and their spatio-temporal relationships simultaneously. Domain knowledge about the human body is explicitly incorporated into the network providing effective priors to regularize the skeletal structure and to enforce temporal consistency. The proposed end-to-end architecture is evaluated on two widely used benchmarks (Penn Action dataset and JHMDB dataset) for video-based pose estimation. Our approach significantly outperforms the existing state-of-the-art methods.

cs.CV↗

arXiv2017

A Pursuit of Temporal Accuracy in General Activity Detection

Detecting activities in untrimmed videos is an important but challenging task. The performance of existing methods remains unsatisfactory, e.g., they often meet difficulties in locating the beginning and end of a long complex action. In this paper, we propose a generic framework that can accurately detect a wide variety of activities from untrimmed videos. Our first contribution is a novel proposal scheme that can efficiently generate candidates with accurate temporal boundaries. The other contribution is a cascaded classification pipeline that explicitly distinguishes between relevance and completeness of a candidate instance. On two challenging temporal activity detection datasets, THUMOS14 and ActivityNet, the proposed framework significantly outperforms the existing state-of-the-art methods, demonstrating superior accuracy and strong adaptivity in handling activities with various temporal structures.

cs.CV↗

arXiv2017

Weakly Supervised PatchNets: Describing and Aggregating Local Patches for Scene Recognition

Traditional feature encoding scheme (e.g., Fisher vector) with local descriptors (e.g., SIFT) and recent convolutional neural networks (CNNs) are two classes of successful methods for image recognition. In this paper, we propose a hybrid representation, which leverages the discriminative capacity of CNNs and the simplicity of descriptor encoding schema for image recognition, with a focus on scene recognition. To this end, we make three main contributions from the following aspects. First, we propose a patch-level and end-to-end architecture to model the appearance of local patches, called {\em PatchNet}. PatchNet is essentially a customized network trained in a weakly supervised manner, which uses the image-level supervision to guide the patch-level feature extraction. Second, we present a hybrid visual representation, called {\em VSAD}, by utilizing the robust feature representations of PatchNet to describe local patches and exploiting the semantic probabilities of PatchNet to aggregate these local patches into a global representation. Third, based on the proposed VSAD representation, we propose a new state-of-the-art scene recognition approach, which achieves an excellent performance on two standard benchmarks: MIT Indoor67 (86.2\%) and SUN397 (73.0\%).

cs.CV↗

arXiv2017

Knowledge Guided Disambiguation for Large-Scale Scene Classification with Multi-Resolution CNNs

Convolutional Neural Networks (CNNs) have made remarkable progress on scene recognition, partially due to these recent large-scale scene datasets, such as the Places and Places2. Scene categories are often defined by multi-level information, including local objects, global layout, and background environment, thus leading to large intra-class variations. In addition, with the increasing number of scene categories, label ambiguity has become another crucial issue in large-scale classification. This paper focuses on large-scale scene recognition and makes two major contributions to tackle these issues. First, we propose a multi-resolution CNN architecture that captures visual content and structure at multiple levels. The multi-resolution CNNs are composed of coarse resolution CNNs and fine resolution CNNs, which are complementary to each other. Second, we design two knowledge guided disambiguation techniques to deal with the problem of label ambiguity. (i) We exploit the knowledge from the confusion matrix computed on validation data to merge ambiguous classes into a super category. (ii) We utilize the knowledge of extra networks to produce a soft label for each image. Then the super categories or soft labels are employed to guide CNN training on the Places2. We conduct extensive experiments on three large-scale image datasets (ImageNet, Places, and Places2), demonstrating the effectiveness of our approach. Furthermore, our method takes part in two major scene recognition challenges, and achieves the second place at the Places2 challenge in ILSVRC 2015, and the first place at the LSUN challenge in CVPR 2016. Finally, we directly test the learned representations on other scene benchmarks, and obtain the new state-of-the-art results on the MIT Indoor67 (86.7\%) and SUN397 (72.0\%). We release the code and models at~\url{https://github.com/wanglimin/MRCNN-Scene-Recognition}.

cs.CV↗

arXiv2016

Locally-Supervised Deep Hybrid Model for Scene Recognition

Convolutional neural networks (CNN) have recently achieved remarkable successes in various image classification and understanding tasks. The deep features obtained at the top fully-connected layer of the CNN (FC-features) exhibit rich global semantic information and are extremely effective in image classification. On the other hand, the convolutional features in the middle layers of the CNN also contain meaningful local information, but are not fully explored for image representation. In this paper, we propose a novel Locally-Supervised Deep Hybrid Model (LS-DHM) that effectively enhances and explores the convolutional features for scene recognition. Firstly, we notice that the convolutional features capture local objects and fine structures of scene images, which yield important cues for discriminating ambiguous scenes, whereas these features are significantly eliminated in the highly-compressed FC representation. Secondly, we propose a new Local Convolutional Supervision (LCS) layer to enhance the local structure of the image by directly propagating the label information to the convolutional layers. Thirdly, we propose an efficient Fisher Convolutional Vector (FCV) that successfully rescues the orderless mid-level semantic information (e.g. objects and textures) of scene image. The FCV encodes the large-sized convolutional maps into a fixed-length mid-level representation, and is demonstrated to be strongly complementary to the high-level FC-features. Finally, both the FCV and FC-features are collaboratively employed in the LSDHM representation, which achieves outstanding performance in our experiments. It obtains 83.75% and 67.56% accuracies respectively on the heavily benchmarked MIT Indoor67 and SUN397 datasets, advancing the stat-of-the-art substantially.

cs.CV↗