arXiv subjects

Yi Zhu

Publications and source records attributed to Yi Zhu.

At least 199 records · Page 11Linked to original sources

arXiv2020

Generating Semantically Valid Adversarial Questions for TableQA

Adversarial attack on question answering systems over tabular data (TableQA) can help evaluate to what extent they can understand natural language questions and reason with tables. However, generating natural language adversarial questions is difficult, because even a single character swap could lead to huge semantic difference in human perception. In this paper, we propose SAGE (Semantically valid Adversarial GEnerator), a Wasserstein sequence-to-sequence model for TableQA white-box attack. To preserve meaning of original questions, we apply minimum risk training with SIMILE and entity delexicalization. We use Gumbel-Softmax to incorporate adversarial loss for end-to-end training. Our experiments show that SAGE outperforms existing local attack models on semantic validity and fluency while achieving a good attack success rate. Finally, we demonstrate that adversarial training with SAGE augmented data can improve performance and robustness of TableQA systems.

cs.CL↗

arXiv2020

NUTA: Non-uniform Temporal Aggregation for Action Recognition

In the world of action recognition research, one primary focus has been on how to construct and train networks to model the spatial-temporal volume of an input video. These methods typically uniformly sample a segment of an input clip (along the temporal dimension). However, not all parts of a video are equally important to determine the action in the clip. In this work, we focus instead on learning where to extract features, so as to focus on the most informative parts of the video. We propose a method called the non-uniform temporal aggregation (NUTA), which aggregates features only from informative temporal segments. We also introduce a synchronization method that allows our NUTA features to be temporally aligned with traditional uniformly sampled video features, so that both local and clip-level features can be combined. Our model has achieved state-of-the-art performance on four widely used large-scale action-recognition datasets (Kinetics400, Kinetics700, Something-something V2 and Charades). In addition, we have created a visualization to illustrate how the proposed NUTA method selects only the most relevant parts of a video clip.

cs.CV↗

arXiv2020

Global Regularity to Incompressible Viscoelastic System With a Class of Large Initial Data

The global existence of solutions to incompressible viscoelastic flows has been a longstanding open problem, even for the global weak solution. Under some special structure ("div-curl" condition) the global small smooth solution was obtained in \cite{llzhou, cz}. However, the result with large initial data remains unknown up to now, and it is studied in this paper. We shall put forward a new structure: the cone-condition and then derive the global smooth solution to 3D incompressible viscoelastic system with a class of large initial data. The key is to gain an angle quantity in the estimate of nonlinear terms by restricting the solution to a cone.

math.AP↗

arXiv2020

A Comprehensive Study of Deep Video Action Recognition

Video action recognition is one of the representative tasks for video understanding. Over the last decade, we have witnessed great advancements in video action recognition thanks to the emergence of deep learning. But we also encountered new challenges, including modeling long-range temporal information in videos, high computation costs, and incomparable results due to datasets and evaluation protocol variances. In this paper, we provide a comprehensive survey of over 200 existing papers on deep learning for video action recognition. We first introduce the 17 video action recognition datasets that influenced the design of models. Then we present video action recognition models in chronological order: starting with early attempts at adapting deep learning, then to the two-stream networks, followed by the adoption of 3D convolutional kernels, and finally to the recent compute-efficient models. In addition, we benchmark popular methods on several representative datasets and release code for reproducibility. In the end, we discuss open problems and shed light on opportunities for video action recognition to facilitate new research ideas.

cs.CV↗

arXiv2020

Scale Aware Adaptation for Land-Cover Classification in Remote Sensing Imagery

Land-cover classification using remote sensing imagery is an important Earth observation task. Recently, land cover classification has benefited from the development of fully connected neural networks for semantic segmentation. The benchmark datasets available for training deep segmentation models in remote sensing imagery tend to be small, however, often consisting of only a handful of images from a single location with a single scale. This limits the models' ability to generalize to other datasets. Domain adaptation has been proposed to improve the models' generalization but we find these approaches are not effective for dealing with the scale variation commonly found between remote sensing image collections. We therefore propose a scale aware adversarial learning framework to perform joint cross-location and cross-scale land-cover classification. The framework has a dual discriminator architecture with a standard feature discriminator as well as a novel scale discriminator. We also introduce a scale attention module which produces scale-enhanced features. Experimental results show that the proposed framework outperforms state-of-the-art domain adaptation methods by a large margin.

cs.CV↗

arXiv2020

Towards Good Practices in Self-supervised Representation Learning

Self-supervised representation learning has seen remarkable progress in the last few years. More recently, contrastive instance learning has shown impressive results compared to its supervised learning counterparts. However, even with the ever increased interest in contrastive instance learning, it is still largely unclear why these methods work so well. In this paper, we aim to unravel some of the mysteries behind their success, which are the good practices. Through an extensive empirical analysis, we hope to not only provide insights but also lay out a set of best practices that led to the success of recent work in self-supervised representation learning.

cs.CV↗

arXiv2020

Lexical Simplification with Pretrained Encoders

Lexical simplification (LS) aims to replace complex words in a given sentence with their simpler alternatives of equivalent meaning. Recently unsupervised lexical simplification approaches only rely on the complex word itself regardless of the given sentence to generate candidate substitutions, which will inevitably produce a large number of spurious candidates. We present a simple LS approach that makes use of the Bidirectional Encoder Representations from Transformers (BERT) which can consider both the given sentence and the complex word during generating candidate substitutions for the complex word. Specifically, we mask the complex word of the original sentence for feeding into the BERT to predict the masked token. The predicted results will be used as candidate substitutions. Despite being entirely unsupervised, experimental results show that our approach obtains obvious improvement compared with these baselines leveraging linguistic databases and parallel corpus, outperforming the state-of-the-art by more than 12 Accuracy points on three well-known benchmarks.

cs.CL↗

arXiv2020

Revealing hidden dynamics from time-series data by ODENet

To derive the hidden dynamics from observed data is one of the fundamental but also challenging problems in many different fields. In this study, we propose a new type of interpretable network called the ordinary differential equation network (ODENet), in which the numerical integration of explicit ordinary differential equations (ODEs) are embedded into the machine learning scheme to build a general framework for revealing the hidden dynamics buried in massive time-series data efficiently and reliably. ODENet takes full advantage of both machine learning algorithms and ODE modeling. On one hand, the embedding of ODEs makes the framework more interpretable benefiting from the mature theories of ODEs. On the other hand, the schemes of machine learning enable data handling, paralleling, and optimization to be easily and efficiently implemented. From classical Lotka-Volterra equations to chaotic Lorenz equations, the ODENet exhibits its remarkable capability in handling time-series data even in the presence of large noise. We further apply the ODENet to real actin aggregation data, which shows an impressive performance as well. These results demonstrate the superiority of ODENet in dealing with noisy data, data with either non-equal spacing or large sampling time steps over other traditional machine learning algorithms.

math.DS↗

arXiv2020

Motion-Excited Sampler: Video Adversarial Attack with Sparked Prior

Deep neural networks are known to be susceptible to adversarial noise, which are tiny and imperceptible perturbations. Most of previous work on adversarial attack mainly focus on image models, while the vulnerability of video models is less explored. In this paper, we aim to attack video models by utilizing intrinsic movement pattern and regional relative motion among video frames. We propose an effective motion-excited sampler to obtain motion-aware noise prior, which we term as sparked prior. Our sparked prior underlines frame correlations and utilizes video dynamics via relative motion. By using the sparked prior in gradient estimation, we can successfully attack a variety of video classification models with fewer number of queries. Extensive experimental results on four benchmark datasets validate the efficacy of our proposed method.

cs.CV↗

arXiv2020

Unfitted Nitsche's method for computing band structures in phononic crystals with impurities

In this paper, we propose an unfitted Nitsche's method to compute the band structures of phononic crystal with impurities of general geometry. The proposed method does not require the background mesh to fit the interfaces of impurities, and thus avoids the expensive cost of generating body-fitted meshes and simplifies the inclusion of interface conditions in the formulation. The quasi-periodic boundary conditions are handled by the Floquet-Bloch transform, which converts the computation of band structures into an eigenvalue problem with periodic boundary conditions. More importantly, we show the well-posedness of the proposed method using a delicate argument based on the trace inequality, and further prove the convergence by the Babuška-Osborn theory. We achieve the optimal convergence rate at the presence of the impurities of general geometry. We confirm the theoretical results by two numerical examples, and show the capability of the proposed methods for computing the band structures without fitting the interfaces of impurities.

math.NA↗

arXiv2020

LSBert: A Simple Framework for Lexical Simplification

Lexical simplification (LS) aims to replace complex words in a given sentence with their simpler alternatives of equivalent meaning, to simplify the sentence. Recently unsupervised lexical simplification approaches only rely on the complex word itself regardless of the given sentence to generate candidate substitutions, which will inevitably produce a large number of spurious candidates. In this paper, we propose a lexical simplification framework LSBert based on pretrained representation model Bert, that is capable of (1) making use of the wider context when both detecting the words in need of simplification and generating substitue candidates, and (2) taking five high-quality features into account for ranking candidates, including Bert prediction order, Bert-based language model, and the paraphrase database PPDB, in addition to the word frequency and word similarity commonly used in other LS methods. We show that our system outputs lexical simplifications that are grammatically correct and semantically appropriate, and obtains obvious improvement compared with these baselines, outperforming the state-of-the-art by 29.8 Accuracy points on three well-known benchmarks.

cs.CL↗

arXiv2020

Unfitted Nitsche's method for computing wave modes in topological materials

In this paper, we propose an unfitted Nitsche's method for computing wave modes in topological materials. The proposed method is based on Nitsche's technique to study the performance-enhanced topological materials which have strongly heterogeneous structures (e.g., the refractive index is piecewise constant with high contrasts). For periodic bulk materials, we use Floquet-Bloch theory and solve an eigenvalue problem on a torus with unfitted meshes. For the materials with a line defect, a sufficiently large domain with zero boundary conditions is used to compute the localized eigenfunctions corresponding to the edge modes. The interfaces are handled by Nitsche's method on an unfitted uniform mesh. We prove the proposed methods converge optimally, and present numerical examples to validate the theoretical results and demonstrate the capability of simulating topological materials.

math.NA↗

arXiv2020

Wave packets in the fractional nonlinear Schrödinger equation with a honeycomb potential

In this article, we study wave dynamics in the fractional nonlinear Schrödinger equation with a modulated honeycomb potential. This problem arises from recent research interests in the interplay between topological materials and nonlocal governing equations. Both are current focuses in scientific research fields. We first develop the Floquet-Bloch spectral theory of the linear fractional Schrödinger operator with a honeycomb potential. Especially, we prove the existence of conical degenerate points, i.e., Dirac points, at which two dispersion band functions intersect. We then investigate the dynamics of wave packets spectrally localized at a Dirac point and derive the leading effective envelope equation. It turns out the envelope can be described by a nonlinear Dirac equation with a varying mass. With rigorous error estimates, we demonstrate that the asymptotic solution based on the effective envelope equation approximates the true solution well in the weighted-$H^s$ space.

math.AP↗

arXiv2020

Identification of hydrodynamic instability by convolutional neural networks

The onset of hydrodynamic instabilities is of great importance in both industry and daily life, due to the dramatic mechanical and thermodynamic changes for different types of flow motions. In this paper, modern machine learning techniques, especially the convolutional neural networks (CNN), are applied to identify the transition between different flow motions raised by hydrodynamic instability, as well as critical non-dimensionalized parameters for characterizing this transit. CNN not only correctly predicts the critical transition values for both Taylor-Couette (TC) flow and Rayleigh- Bénard (RB) convection under various setups and conditions, but also shows an outstanding performance on robustness and noise-tolerance. In addition, key spatial features used for classifying different flow patterns are revealed by the principal component analysis.

physics.comp-ph↗

arXiv2020

When Machine Learning Meets Multiscale Modeling in Chemical Reactions

Due to the intrinsic complexity and nonlinearity of chemical reactions, direct applications of traditional machine learning algorithms may face with many difficulties. In this study, through two concrete examples with biological background, we illustrate how the key ideas of multiscale modeling can help to reduce the computational cost of machine learning a lot, as well as how machine learning algorithms perform model reduction automatically in a time-scale separated system. Our study highlights the necessity and effectiveness of an integration of machine learning algorithms and multiscale modeling during the study of chemical reactions.

q-bio.MN↗

arXiv2020

Improving Semantic Segmentation via Self-Training

Deep learning usually achieves the best results with complete supervision. In the case of semantic segmentation, this means that large amounts of pixelwise annotations are required to learn accurate models. In this paper, we show that we can obtain state-of-the-art results using a semi-supervised approach, specifically a self-training paradigm. We first train a teacher model on labeled data, and then generate pseudo labels on a large set of unlabeled data. Our robust training framework can digest human-annotated and pseudo labels jointly and achieve top performances on Cityscapes, CamVid and KITTI datasets while requiring significantly less supervision. We also demonstrate the effectiveness of self-training on a challenging cross-domain generalization task, outperforming conventional finetuning method by a large margin. Lastly, to alleviate the computational burden caused by the large amount of pseudo labels, we propose a fast training schedule to accelerate the training of segmentation models by up to 2x without performance degradation.

cs.CV↗

arXiv2020

Testing kinetically coupled inflation models with CMB distortions

Inflation scenarios kinetically coupled with the Einstein tensor have been widely studied. They can be consistent with current observational data. Future experiments on the measurement on CMB distortions will potentially extend information about the scalar spectrum to small scales $1 \Mpc^{-1} \lesssim k \lesssim 2 \times 10^4 \Mpc^{-1}$. By taking the sensitivity of the PIXIE experiment as the criterion, we perform a model-oriented analysis of the observational prospects of spectral distortions for kinetically coupled inflation. There are five models that possibly generate a detectable level of distortions, among the 49 single-field inflation models listed in Ref. \cite{Martin2013a}. These models are: hybrid inflation in the valley (VHI), non-canonical Kähler inflation (NCKI), generalized MSSM inflation (GMSSMI), generalized renormalization point inflation (GRIPI), and running-mass inflation (RMI). Each of these models can satisfy the Planck constraints on spectral tilt and lead to increase power on scales relevant for CMB distortions in a tuned region of their parameter space. The existence of kinetic coupling suppresses the value of the model parameters with mass dimension for VHI, GMSSMI, and GRIPI, such that these three models can be in agreement with their theoretical considerations. However, the tuned regions for all these models fail to satisfy the constraints on tensor modes.

astro-ph.CO↗

arXiv2020

Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks

Vision-Language Navigation (VLN) is a task where agents learn to navigate following natural language instructions. The key to this task is to perceive both the visual scene and natural language sequentially. Conventional approaches exploit the vision and language features in cross-modal grounding. However, the VLN task remains challenging, since previous works have neglected the rich semantic information contained in the environment (such as implicit navigation graphs or sub-trajectory semantics). In this paper, we introduce Auxiliary Reasoning Navigation (AuxRN), a framework with four self-supervised auxiliary reasoning tasks to take advantage of the additional training signals derived from the semantic information. The auxiliary tasks have four reasoning objectives: explaining the previous actions, estimating the navigation progress, predicting the next orientation, and evaluating the trajectory consistency. As a result, these additional training signals help the agent to acquire knowledge of semantic representations in order to reason about its activity and build a thorough perception of the environment. Our experiments indicate that auxiliary reasoning tasks improve both the performance of the main task and the model generalizability by a large margin. Empirically, we demonstrate that an agent trained with self-supervised auxiliary reasoning tasks substantially outperforms the previous state-of-the-art method, being the best existing approach on the standard benchmark.

cs.CV↗