arXiv subjects

Jingyuan Chen

Publications and source records attributed to Jingyuan Chen.

76 records · Page 5Linked to original sources

arXiv2018

Real-Time Referring Expression Comprehension by Single-Stage Grounding Network

In this paper, we propose a novel end-to-end model, namely Single-Stage Grounding network (SSG), to localize the referent given a referring expression within an image. Different from previous multi-stage models which rely on object proposals or detected regions, our proposed model aims to comprehend a referring expression through one single stage without resorting to region proposals as well as the subsequent region-wise feature extraction. Specifically, a multimodal interactor is proposed to summarize the local region features regarding the referring expression attentively. Subsequently, a grounder is proposed to localize the referring expression within the given image directly. For further improving the localization accuracy, a guided attention mechanism is proposed to enforce the grounder to focus on the central region of the referent. Moreover, by exploiting and predicting visual attribute information, the grounder can further distinguish the referent objects within an image and thereby improve the model performance. Experiments on RefCOCO, RefCOCO+, and RefCOCOg datasets demonstrate that our proposed SSG without relying on any region proposals can achieve comparable performance with other advanced models. Furthermore, our SSG outperforms the previous models and achieves the state-of-art performance on the ReferItGame dataset. More importantly, our SSG is time efficient and can ground a referring expression in a 416*416 image from the RefCOCO dataset in 25ms (40 referents per second) on average with a Nvidia Tesla P40, accomplishing more than 9* speedups over the existing multi-stage models.

cs.CV↗

arXiv2018

Fine-grained Video Attractiveness Prediction Using Multimodal Deep Learning on a Large Real-world Dataset

Nowadays, billions of videos are online ready to be viewed and shared. Among an enormous volume of videos, some popular ones are widely viewed by online users while the majority attract little attention. Furthermore, within each video, different segments may attract significantly different numbers of views. This phenomenon leads to a challenging yet important problem, namely fine-grained video attractiveness prediction. However, one major obstacle for such a challenging problem is that no suitable benchmark dataset currently exists. To this end, we construct the first fine-grained video attractiveness dataset, which is collected from one of the most popular video websites in the world. In total, the constructed FVAD consists of 1,019 drama episodes with 780.6 hours covering different categories and a wide variety of video contents. Apart from the large amount of videos, hundreds of millions of user behaviors during watching videos are also included, such as "view counts", "fast-forward", "fast-rewind", and so on, where "view counts" reflects the video attractiveness while other engagements capture the interactions between the viewers and videos. First, we demonstrate that video attractiveness and different engagements present different relationships. Second, FVAD provides us an opportunity to study the fine-grained video attractiveness prediction problem. We design different sequential models to perform video attractiveness prediction by relying solely on video contents. The sequential models exploit the multimodal relationships between visual and audio components of the video contents at different levels. Experimental results demonstrate the effectiveness of our proposed sequential models with different visual and audio representations, the necessity of incorporating the two modalities, and the complementary behaviors of the sequential prediction models at different levels.

cs.CV↗

arXiv2018

Depletion zones and crystallography on pinched spheres

Understanding the interplay between ordered structures and substrate curvature is an interesting problem with versatile applications, including functionalization of charged supramolecular surfaces and modern microfluidic technologies. In this work, we investigate the two-dimensional packing structures of charged particles confined on a pinched sphere. By continuously pinching the sphere, we observe cleavage of elongated scars into pleats, proliferation of disclinations, and subsequently, emergence of a depletion zone at the negatively curved waist that is completely void of particles. We systematically study the geometrics and energetics of the depletion zone, and reveal its physical origin as a finite size effect, due to the interplay between Coulomb repulsion and concave geometry of pinched sphere. These results further our understanding of crystallography on curved surfaces, and have implications in design and manipulation of charged, deformable interfaces in various applications.

cond-mat.soft↗

arXiv2016

Combinatorial Constructions of Optimal $(m, n,4,2)$ Optical Orthogonal Signature Pattern Codes

Optical orthogonal signature pattern codes (OOSPCs) play an important role in a novel type of optical code-division multiple-access (CDMA) network for 2-dimensional image transmission. There is a one-to-one correspondence between an $(m, n, w, λ)$-OOSPC and a $(λ+1)$-$(mn,w,1)$ packing design admitting an automorphism group isomorphic to $\mathbb{Z}_m\times \mathbb{Z}_n$. In 2010, Sawa gave the first infinite class of $(m, n, 4, 2)$-OOSPCs by using $S$-cyclic Steiner quadruple systems. In this paper, we use various combinatorial designs such as strictly $\mathbb{Z}_m\times \mathbb{Z}_n$-invariant $s$-fan designs, strictly $\mathbb{Z}_m\times \mathbb{Z}_n$-invariant $G$-designs and rotational Steiner quadruple systems to present some constructions for $(m, n, 4, 2)$-OOSPCs. As a consequence, our new constructions yield more infinite families of optimal $(m, n, 4, 2)$-OOSPCs. Especially, we shall see that in some cases an optimal $(m, n, 4, 2)$-OOSPC can not achieve the Johnson bound.

cs.DM↗