arXiv subjects

Zhou Yu

Publications and source records attributed to Zhou Yu.

268 records · Page 15Linked to original sources

arXiv2019

Beyond Bilinear: Generalized Multimodal Factorized High-order Pooling for Visual Question Answering

Visual question answering (VQA) is challenging because it requires a simultaneous understanding of both visual content of images and textual content of questions. To support the VQA task, we need to find good solutions for the following three issues: 1) fine-grained feature representations for both the image and the question; 2) multi-modal feature fusion that is able to capture the complex interactions between multi-modal features; 3) automatic answer prediction that is able to consider the complex correlations between multiple diverse answers for the same question. For fine-grained image and question representations, a `co-attention' mechanism is developed by using a deep neural network architecture to jointly learn the attentions for both the image and the question, which can allow us to reduce the irrelevant features effectively and obtain more discriminative features for image and question representations. For multi-modal feature fusion, a generalized Multi-modal Factorized High-order pooling approach (MFH) is developed to achieve more effective fusion of multi-modal features by exploiting their correlations sufficiently, which can further result in superior VQA performance as compared with the state-of-the-art approaches. For answer prediction, the KL (Kullback-Leibler) divergence is used as the loss function to achieve precise characterization of the complex correlations between multiple diverse answers with the same or similar meaning, which can allow us to achieve faster convergence rate and obtain slightly better accuracy on answer prediction. A deep neural network architecture is designed to integrate all these aforementioned modules into a unified model for achieving superior VQA performance. With an ensemble of our MFH models, we achieve the state-of-the-art performance on the large-scale VQA datasets and win the runner-up in VQA Challenge 2017.

cs.CV↗

arXiv2019

Double Helical Conformation and Extreme Rigidity in a Rodlike Polyelectrolyte

The ubiquitous biomacromolecule DNA has an axial rigidity persistence length of ~50 nm, driven by its elegant double helical structure. While double and multiple helix structures appear widely in nature, only rarely are these found in synthetic non-chiral macromolecules. Here we describe a double helical conformation in the densely charged aromatic polyamide poly(2,2'-disulfonyl-4,4'-benzidine terephthalamide) or PBDT. This double helix macromolecule represents one of the most rigid simple molecular structures known, exhibiting an extremely high axial persistence length (~1 micrometer). We present X-ray diffraction, NMR spectroscopy, and molecular dynamics (MD) simulations that reveal and confirm the double helical conformation. The discovery of this extreme rigidity in combination with high charge density gives insight into the self-assembly of molecular ionic composites with high mechanical modulus (~1 GPa) yet with liquid-like ion motions inside, and provides fodder for formation of new 1D-reinforced composites.

cond-mat.soft↗

arXiv2018

Incorporating Structured Commonsense Knowledge in Story Completion

The ability to select an appropriate story ending is the first step towards perfect narrative comprehension. Story ending prediction requires not only the explicit clues within the context, but also the implicit knowledge (such as commonsense) to construct a reasonable and consistent story. However, most previous approaches do not explicitly use background commonsense knowledge. We present a neural story ending selection model that integrates three types of information: narrative sequence, sentiment evolution and commonsense knowledge. Experiments show that our model outperforms state-of-the-art approaches on a public dataset, ROCStory Cloze Task , and the performance gain from adding the additional commonsense knowledge is significant.

cs.CL↗

arXiv2018

Structured Content Preservation for Unsupervised Text Style Transfer

Text style transfer aims to modify the style of a sentence while keeping its content unchanged. Recent style transfer systems often fail to faithfully preserve the content after changing the style. This paper proposes a structured content preserving model that leverages linguistic information in the structured fine-grained supervisions to better preserve the style-independent content during style transfer. In particular, we achieve the goal by devising rich model objectives based on both the sentence's lexical information and a language model that conditions on content. The resulting model therefore is encouraged to retain the semantic meaning of the target sentences. We perform extensive experiments that compare our model to other existing approaches in the tasks of sentiment and political slant transfer. Our model achieves significant improvement in terms of both content preservation and style transfer in automatic and human evaluation.

cs.CL↗

arXiv2018

Anti-Stokes excitation of solid-state quantum emitters for nanoscale thermometry

Color centers in solids are the fundamental constituents of a plethora of applications such as lasers, light emitting diodes and sensors, as well as the foundation of advanced quantum information and communication technologies. Their photoluminescence properties are usually studied under Stokes excitation, in which the emitted photons are at a lower energy than the excitation ones. In this work, we explore the opposite Anti-Stokes process, where excitation is performed with lower energy photons. We report that the process is sufficiently efficient to excite even a single quantum system, namely the germanium-vacancy center in diamond. Consequently, we leverage the temperature-dependent, phonon-assisted mechanism to realize an all-optical nanoscale thermometry scheme that outperforms any homologous optical method employed to date. Our results frame a promising approach for exploring fundamental light-matter interactions in isolated quantum systems, and harness it towards the realization of practical nanoscale thermometry and sensing.

physics.app-ph↗

arXiv2018

Cross-Lingual Cross-Platform Rumor Verification Pivoting on Multimedia Content

With the increasing popularity of smart devices, rumors with multimedia content become more and more common on social networks. The multimedia information usually makes rumors look more convincing. Therefore, finding an automatic approach to verify rumors with multimedia content is a pressing task. Previous rumor verification research only utilizes multimedia as input features. We propose not to use the multimedia content but to find external information in other news platforms pivoting on it. We introduce a new features set, cross-lingual cross-platform features that leverage the semantic similarity between the rumors and the external information. When implemented, machine learning methods utilizing such features achieved the state-of-the-art rumor verification results.

cs.CL↗

arXiv2018

A Visual Attention Grounding Neural Model for Multimodal Machine Translation

We introduce a novel multimodal machine translation model that utilizes parallel visual and textual information. Our model jointly optimizes the learning of a shared visual-language embedding and a translator. The model leverages a visual attention grounding mechanism that links the visual semantics with the corresponding textual semantics. Our approach achieves competitive state-of-the-art results on the Multi30K and the Ambiguous COCO datasets. We also collected a new multilingual multimodal product description dataset to simulate a real-world international online shopping scenario. On this dataset, our visual attention grounding model outperforms other methods by a large margin.

cs.CL↗

arXiv2018

Overlapping Sliced Inverse Regression for Dimension Reduction

Sliced inverse regression (SIR) is a pioneer tool for supervised dimension reduction. It identifies the effective dimension reduction space, the subspace of significant factors with intrinsic lower dimensionality. In this paper, we propose to refine the SIR algorithm through an overlapping slicing scheme. The new algorithm, called overlapping sliced inverse regression (OSIR), is able to estimate the effective dimension reduction space and determine the number of effective factors more accurately. We show that such overlapping procedure has the potential to identify the information contained in the derivatives of the inverse regression curve, which helps to explain the superiority of OSIR. We also prove that OSIR algorithm is $\sqrt n $-consistent and verify its effectiveness by simulations and real applications.

stat.ML↗

arXiv2018

Rethinking Diversified and Discriminative Proposal Generation for Visual Grounding

Visual grounding aims to localize an object in an image referred to by a textual query phrase. Various visual grounding approaches have been proposed, and the problem can be modularized into a general framework: proposal generation, multi-modal feature representation, and proposal ranking. Of these three modules, most existing approaches focus on the latter two, with the importance of proposal generation generally neglected. In this paper, we rethink the problem of what properties make a good proposal generator. We introduce the diversity and discrimination simultaneously when generating proposals, and in doing so propose Diversified and Discriminative Proposal Networks model (DDPN). Based on the proposals generated by DDPN, we propose a high performance baseline model for visual grounding and evaluate it on four benchmark datasets. Experimental results demonstrate that our model delivers significant improvements on all the tested data-sets (e.g., 18.8\% improvement on ReferItGame and 8.2\% improvement on Flickr30k Entities over the existing state-of-the-arts respectively)

cs.CV↗

arXiv2018

Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog

Creating an intelligent conversational system that understands vision and language is one of the ultimate goals in Artificial Intelligence (AI)~\cite{winograd1972understanding}. Extensive research has focused on vision-to-language generation, however, limited research has touched on combining these two modalities in a goal-driven dialog context. We propose a multimodal hierarchical reinforcement learning framework that dynamically integrates vision and language for task-oriented visual dialog. The framework jointly learns the multimodal dialog state representation and the hierarchical dialog policy to improve both dialog task success and efficiency. We also propose a new technique, state adaptation, to integrate context awareness in the dialog state representation. We evaluate the proposed framework and the state adaptation technique in an image guessing game and achieve promising results.

cs.CL↗

arXiv2018

Sufficient variable screening via directional regression with censored response

We in this paper propose a directional regression based approach for ultrahigh dimensional sufficient variable screening with censored responses. The new method is designed in a model-free manner and thus can be adapted to various complex model structures. Under some commonly used assumptions, we show that the proposed method enjoys the sure screening property when the dimension p diverges at an exponential rate of the sample size n. To improve the marginal screening method, the corresponding iterative screening algorithm and stability screening algorithm are further equipped. We demonstrate the effectiveness of the proposed method through simulation studies and a real data analysis.

stat.ME↗

arXiv2017

Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering

Visual question answering (VQA) is challenging because it requires a simultaneous understanding of both the visual content of images and the textual content of questions. The approaches used to represent the images and questions in a fine-grained manner and questions and to fuse these multi-modal features play key roles in performance. Bilinear pooling based models have been shown to outperform traditional linear models for VQA, but their high-dimensional representations and high computational complexity may seriously limit their applicability in practice. For multi-modal feature fusion, here we develop a Multi-modal Factorized Bilinear (MFB) pooling approach to efficiently and effectively combine multi-modal features, which results in superior performance for VQA compared with other bilinear pooling approaches. For fine-grained image and question representation, we develop a co-attention mechanism using an end-to-end deep network architecture to jointly learn both the image and question attentions. Combining the proposed MFB approach with co-attention learning in a new network architecture provides a unified model for VQA. Our experimental results demonstrate that the single MFB with co-attention model achieves new state-of-the-art performance on the real-world VQA dataset. Code available at https://github.com/yuzcccc/mfb.

cs.CV↗

arXiv2017

Learning Conversational Systems that Interleave Task and Non-Task Content

Task-oriented dialog systems have been applied in various tasks, such as automated personal assistants, customer service providers and tutors. These systems work well when users have clear and explicit intentions that are well-aligned to the systems' capabilities. However, they fail if users intentions are not explicit. To address this shortcoming, we propose a framework to interleave non-task content (i.e. everyday social conversation) into task conversations. When the task content fails, the system can still keep the user engaged with the non-task content. We trained a policy using reinforcement learning algorithms to promote long-turn conversation coherence and consistency, so that the system can have smooth transitions between task and non-task content. To test the effectiveness of the proposed framework, we developed a movie promotion dialog system. Experiments with human users indicate that a system that interleaves social and task content achieves a better task success rate and is also rated as more engaging compared to a pure task-oriented system.

cs.CL↗

arXiv2015

A progressively reduced pretension method to fabricate Bradbury-Nielsen gates with uniform tension

A Bradbury-Nielsen gate (BNG) is often used to modulate ion beams. It consists of two interleaved and electrically isolated sets of wires with uniform tension, which can keep parallel, equidistant, and coplanar over a wide temperature range, making the BNG reliable and robust. We have previously analyzed the non-uniform problem of wire tensions with sequentially winding method, and developed a template-based transfer method to solve this problem. In this paper, we introduced a progressively reduced pretension method, which allows directly and sequentially winding wires on the substrate without using a template. Theoretical analysis shows that by applying proper pretension to each wire when fixing it onto the substrate, the final wire tensions of all wires can be uniform. The algorithm and flowchart to calculate the pretension sequence are given, and the fabrication process is introduced in detail. Pretensions are generated by weights combination with a homebuilt weaving device. A BNG with stainless steel wire and a printed circuit board substrate is constructed with this method. The non-uniformity of the final wire tensions is less than 2.5% in theory. The BNG is successfully employed in our homemade ion mobility spectrometer, and the measured resolution is 33.5 at a gate opening time of 350 μs. Compared to the template-based method, this method is simpler, faster and more flexible when making BNGs with different configurations.

physics.ins-det↗

arXiv2014

Trace Pursuit: A General Framework for Model-Free Variable Selection

We propose trace pursuit for model-free variable selection under the sufficient dimension reduction paradigm. Two distinct algorithms are proposed: stepwise trace pursuit and forward trace pursuit. Stepwise trace pursuit achieves selection consistency with fixed p, and is readily applicable in the challenging setting with p>n. Forward trace pursuit can serve as an initial screening step to speed up the computation in the case of ultrahigh dimensionality. The screening consistency property of forward trace pursuit based on sliced inverse regression is established. Finite sample performances of trace pursuit and other model-free variable selection methods are compared through numerical studies.

stat.ME↗

arXiv2013

Dual-target function validation of single-particle selection from low-contrast cryo-electron micrographs

Weak-signal detection and single-particle selection from low-contrast micrographs of frozen hydrated biomolecules by cryo-electron microscopy (cryo-EM) presents a practical challenge. Cryo-EM image contrast degrades as the size of biomolecules of structural interest decreases. When the image contrast falls into a range where the location or presence of single particles becomes ambiguous, a need arises for objective computational approaches to detect weak signal and to select and verify particles from these low-contrast micrographs. Here we propose an objective validation scheme for low-contrast particle selection using a combination of two different target functions. In an implementation of this dual-target function (DTF) validation, a first target function of fast local correlation was used to select particles through template matching, followed by signal validation through a second target function of maximum likelihood. By a systematic study of simulated data, we found that such an implementation of DTF validation is capable of selecting and verifying particles from cryo-EM micrographs with a signal-to-noise ratio as low as 0.002. Importantly, we demonstrated that DTF validation can robustly evade over-fitting or reference bias from the particle-picking template, allowing true signal to emerge from amidst heavy noise in an objective fashion. The DTF approach allows efficient assembly of a large number of single-particle cryo-EM images of smaller biomolecules or specimens containing contrast-degrading agents like detergents in a semi-automatic manner.

q-bio.QM↗