arXiv subjects

Jianfu Zhang

Publications and source records attributed to Jianfu Zhang.

59 records · Page 4Linked to original sources

arXiv2019

Exploiting Motion Information from Unlabeled Videos for Static Image Action Recognition

Static image action recognition, which aims to recognize action based on a single image, usually relies on expensive human labeling effort such as adequate labeled action images and large-scale labeled image dataset. In contrast, abundant unlabeled videos can be economically obtained. Therefore, several works have explored using unlabeled videos to facilitate image action recognition, which can be categorized into the following two groups: (a) enhance visual representations of action images with a designed proxy task on unlabeled videos, which falls into the scope of self-supervised learning; (b) generate auxiliary representations for action images with the generator learned from unlabeled videos. In this paper, we integrate the above two strategies in a unified framework, which consists of Visual Representation Enhancement (VRE) module and Motion Representation Augmentation (MRA) module. Specifically, the VRE module includes a proxy task which imposes pseudo motion label constraint and temporal coherence constraint on unlabeled videos, while the MRA module could predict the motion information of a static action image by exploiting unlabeled videos. We demonstrate the superiority of our framework based on four benchmark human action datasets with limited labeled data.

cs.CV↗

arXiv2019

A Proposal-based Approach for Activity Image-to-Video Retrieval

Activity image-to-video retrieval task aims to retrieve videos containing the similar activity as the query image, which is a challenging task because videos generally have many background segments irrelevant to the activity. In this paper, we utilize R-C3D model to represent a video by a bag of activity proposals, which can filter out background segments to some extent. However, there are still noisy proposals in each bag. Thus, we propose an Activity Proposal-based Image-to-Video Retrieval (APIVR) approach, which incorporates multi-instance learning into cross-modal retrieval framework to address the proposal noise issue. Specifically, we propose a Graph Multi-Instance Learning (GMIL) module with graph convolutional layer, and integrate this module with classification loss, adversarial loss, and triplet loss in our cross-modal retrieval framework. Moreover, we propose geometry-aware triplet loss based on point-to-subspace distance to preserve the structural information of activity proposals. Extensive experiments on three widely-used datasets verify the effectiveness of our approach.

cs.CV↗

arXiv2019

Inductive Guided Filter: Real-time Deep Image Matting with Weakly Annotated Masks on Mobile Devices

Recently, significant progress has been achieved in deep image matting. Most of the classical image matting methods are time-consuming and require an ideal trimap which is difficult to attain in practice. A high efficient image matting method based on a weakly annotated mask is in demand for mobile applications. In this paper, we propose a novel method based on Deep Learning and Guided Filter, called Inductive Guided Filter, which can tackle the real-time general image matting task on mobile devices. We design a lightweight hourglass network to parameterize the original Guided Filter method that takes an image and a weakly annotated mask as input. Further, the use of Gabor loss is proposed for training networks for complicated textures in image matting. Moreover, we create an image matting dataset MAT-2793 with a variety of foreground objects. Experimental results demonstrate that our proposed method massively reduces running time with robust accuracy.

cs.CV↗

arXiv2018

Multi-shot Pedestrian Re-identification via Sequential Decision Making

Multi-shot pedestrian re-identification problem is at the core of surveillance video analysis. It matches two tracks of pedestrians from different cameras. In contrary to existing works that aggregate single frames features by time series model such as recurrent neural network, in this paper, we propose an interpretable reinforcement learning based approach to this problem. Particularly, we train an agent to verify a pair of images at each time. The agent could choose to output the result (same or different) or request another pair of images to verify (unsure). By this way, our model implicitly learns the difficulty of image pairs, and postpone the decision when the model does not accumulate enough evidence. Moreover, by adjusting the reward for unsure action, we can easily trade off between speed and accuracy. In three open benchmarks, our method are competitive with the state-of-the-art methods while only using 3% to 6% images. These promising results demonstrate that our method is favorable in both efficiency and performance.

cs.CV↗

arXiv2014

Origin of Multi-band Emission from the Microquasar Cygnus X--1

We study the origin of non-thermal emissions from the Galactic black hole X-ray binary Cygnus X--1 which is a confirmed high mass microquasar. By analogy with those methods used in studies of active galactic nuclei, we propose a two-dimensional, time-dependent radiation model from the microquasar Cygnus X--1. In this model, the evolution equation for relativistic electrons in a conical jet are numerically solved by including escape, adiabatic and various radiative losses. The radiative processes involved are synchrotron emission, its self-Compton scattering, and inverse Compton scatterings of an accretion disk and its surrounding stellar companion. This model also includes an electromagnetic cascade process of an anisotropic $γ$-$γ$ interaction. We study the spectral properties of electron evolution and its emission spectral characteristic at different heights of the emission region located in the jet. We find that radio data from Cygnus X--1 are reproduced by the synchrotron emission, the \emph{Fermi} LAT measurements by the synchrotron emission and Comptonization of photons of the stellar companion, the TeV band emission fluxes by the Comptonization of the stellar photons. Our results show that: (1) Radio emission region extends from the binary system scales to the termination of the jet. (2) The GeV band emissions should originate from the distance close to the binary system scales. (3) The TeV band emissions could be inside the binary system, and these emissions could be probed by the upcoming CTA telescope. (4) The MeV tail emissions, which produces a strongly linearly polarized signal, are emitted inside the binary system. The location of the emissions is very close to the inner region of the jet.

astro-ph.HE↗