arXiv subjects

Yi Zhu

Publications and source records attributed to Yi Zhu.

At least 181 records · Page 10Linked to original sources

arXiv2021

Progressive Coordinate Transforms for Monocular 3D Object Detection

Recognizing and localizing objects in the 3D space is a crucial ability for an AI agent to perceive its surrounding environment. While significant progress has been achieved with expensive LiDAR point clouds, it poses a great challenge for 3D object detection given only a monocular image. While there exist different alternatives for tackling this problem, it is found that they are either equipped with heavy networks to fuse RGB and depth information or empirically ineffective to process millions of pseudo-LiDAR points. With in-depth examination, we realize that these limitations are rooted in inaccurate object localization. In this paper, we propose a novel and lightweight approach, dubbed {\em Progressive Coordinate Transforms} (PCT) to facilitate learning coordinate representations. Specifically, a localization boosting mechanism with confidence-aware loss is introduced to progressively refine the localization prediction. In addition, semantic image representation is also exploited to compensate for the usage of patch proposals. Despite being lightweight and simple, our strategy leads to superior improvements on the KITTI and Waymo Open Dataset monocular 3D detection benchmarks. At the same time, our proposed PCT shows great generalization to most coordinate-based 3D detection frameworks. The code is available at: https://github.com/amazon-research/progressive-coordinate-transforms .

cs.CV↗

arXiv2021

Video Contrastive Learning with Global Context

Contrastive learning has revolutionized self-supervised image representation learning field, and recently been adapted to video domain. One of the greatest advantages of contrastive learning is that it allows us to flexibly define powerful loss objectives as long as we can find a reasonable way to formulate positive and negative samples to contrast. However, existing approaches rely heavily on the short-range spatiotemporal salience to form clip-level contrastive signals, thus limit themselves from using global context. In this paper, we propose a new video-level contrastive learning method based on segments to formulate positive pairs. Our formulation is able to capture global context in a video, thus robust to temporal content change. We also incorporate a temporal order regularization term to enforce the inherent sequential structure of videos. Extensive experiments show that our video-level contrastive learning framework (VCLR) is able to outperform previous state-of-the-arts on five video datasets for downstream action classification, action localization and video retrieval. Code is available at https://github.com/amazon-research/video-contrastive-learning.

cs.CV↗

arXiv2021

A Unified Efficient Pyramid Transformer for Semantic Segmentation

Semantic segmentation is a challenging problem due to difficulties in modeling context in complex scenes and class confusions along boundaries. Most literature either focuses on context modeling or boundary refinement, which is less generalizable in open-world scenarios. In this work, we advocate a unified framework(UN-EPT) to segment objects by considering both context information and boundary artifacts. We first adapt a sparse sampling strategy to incorporate the transformer-based attention mechanism for efficient context modeling. In addition, a separate spatial branch is introduced to capture image details for boundary refinement. The whole model can be trained in an end-to-end manner. We demonstrate promising performance on three popular benchmarks for semantic segmentation with low memory footprint. Code will be released soon.

cs.CV↗

arXiv2021

Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation

Language instruction plays an essential role in the natural language grounded navigation tasks. However, navigators trained with limited human-annotated instructions may have difficulties in accurately capturing key information from the complicated instruction at different timesteps, leading to poor navigation performance. In this paper, we exploit to train a more robust navigator which is capable of dynamically extracting crucial factors from the long instruction, by using an adversarial attacking paradigm. Specifically, we propose a Dynamic Reinforced Instruction Attacker (DR-Attacker), which learns to mislead the navigator to move to the wrong target by destroying the most instructive information in instructions at different timesteps. By formulating the perturbation generation as a Markov Decision Process, DR-Attacker is optimized by the reinforcement learning algorithm to generate perturbed instructions sequentially during the navigation, according to a learnable attack score. Then, the perturbed instructions, which serve as hard samples, are used for improving the robustness of the navigator with an effective adversarial training strategy and an auxiliary self-supervised reasoning task. Experimental results on both Vision-and-Language Navigation (VLN) and Navigation from Dialog History (NDH) tasks show the superiority of our proposed method over state-of-the-art methods. Moreover, the visualization analysis shows the effectiveness of the proposed DR-Attacker, which can successfully attack crucial information in the instructions at different timesteps. Code is available at https://github.com/expectorlin/DR-Attacker.

cs.CV↗

arXiv2021

Optimal Resource Allocation for Serverless Queries

Optimizing resource allocation for analytical workloads is vital for reducing costs of cloud-data services. At the same time, it is incredibly hard for users to allocate resources per query in serverless processing systems, and they frequently misallocate by orders of magnitude. Unfortunately, prior work focused on predicting peak allocation while ignoring aggressive trade-offs between resource allocation and run-time. Additionally, these methods fail to predict allocation for queries that have not been observed in the past. In this paper, we tackle both these problems. We introduce a system for optimal resource allocation that can predict performance with aggressive trade-offs, for both new and past observed queries. We introduce the notion of a performance characteristic curve (PCC) as a parameterized representation that can compactly capture the relationship between resources and performance. To tackle training data sparsity, we introduce a novel data augmentation technique to efficiently synthesize the entire PCC using a single run of the query. Lastly, we demonstrate the advantages of a constrained loss function coupled with GNNs, over traditional ML methods, for capturing the domain specific behavior through an extensive experimental evaluation over SCOPE big data workloads at Microsoft.

cs.DB↗

arXiv2021

AutoAdapt: Automated Segmentation Network Search for Unsupervised Domain Adaptation

Neural network-based semantic segmentation has achieved remarkable results when large amounts of annotated data are available, that is, in the supervised case. However, such data is expensive to collect and so methods have been developed to adapt models trained on related, often synthetic data for which labels are readily available. Current adaptation approaches do not consider the dependence of the generalization/transferability of these models on network architecture. In this paper, we perform neural architecture search (NAS) to provide architecture-level perspective and analysis for domain adaptation. We identify the optimization gap that exists when searching architectures for unsupervised domain adaptation which makes this NAS problem uniquely difficult. We propose bridging this gap by using maximum mean discrepancy and regional weighted entropy to estimate the accuracy metric. Experimental results on several widely adopted benchmarks show that our proposed AutoAdapt framework indeed discovers architectures that improve the performance of a number of existing adaptation techniques.

cs.CV↗

arXiv2021

Analyzing Adversarial Robustness of Deep Neural Networks in Pixel Space: a Semantic Perspective

The vulnerability of deep neural networks to adversarial examples, which are crafted maliciously by modifying the inputs with imperceptible perturbations to misled the network produce incorrect outputs, reveals the lack of robustness and poses security concerns. Previous works study the adversarial robustness of image classifiers on image level and use all the pixel information in an image indiscriminately, lacking of exploration of regions with different semantic meanings in the pixel space of an image. In this work, we fill this gap and explore the pixel space of the adversarial image by proposing an algorithm to looking for possible perturbations pixel by pixel in different regions of the segmented image. The extensive experimental results on CIFAR-10 and ImageNet verify that searching for the modified pixel in only some pixels of an image can successfully launch the one-pixel adversarial attacks without requiring all the pixels of the entire image, and there exist multiple vulnerable points scattered in different regions of an image. We also demonstrate that the adversarial robustness of different regions on the image varies with the amount of semantic information contained.

cs.CV↗

arXiv2021

Improving adversarial robustness of deep neural networks by using semantic information

The vulnerability of deep neural networks (DNNs) to adversarial attack, which is an attack that can mislead state-of-the-art classifiers into making an incorrect classification with high confidence by deliberately perturbing the original inputs, raises concerns about the robustness of DNNs to such attacks. Adversarial training, which is the main heuristic method for improving adversarial robustness and the first line of defense against adversarial attacks, requires many sample-by-sample calculations to increase training size and is usually insufficiently strong for an entire network. This paper provides a new perspective on the issue of adversarial robustness, one that shifts the focus from the network as a whole to the critical part of the region close to the decision boundary corresponding to a given class. From this perspective, we propose a method to generate a single but image-agnostic adversarial perturbation that carries the semantic information implying the directions to the fragile parts on the decision boundary and causes inputs to be misclassified as a specified target. We call the adversarial training based on such perturbations "region adversarial training" (RAT), which resembles classical adversarial training but is distinguished in that it reinforces the semantic information missing in the relevant regions. Experimental results on the MNIST and CIFAR-10 datasets show that this approach greatly improves adversarial robustness even using a very small dataset from the training data; moreover, it can defend against FGSM adversarial attacks that have a completely different pattern from the model seen during retraining.

cs.LG↗

arXiv2021

A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters

Few-shot crosslingual transfer has been shown to outperform its zero-shot counterpart with pretrained encoders like multilingual BERT. Despite its growing popularity, little to no attention has been paid to standardizing and analyzing the design of few-shot experiments. In this work, we highlight a fundamental risk posed by this shortcoming, illustrating that the model exhibits a high degree of sensitivity to the selection of few shots. We conduct a large-scale experimental study on 40 sets of sampled few shots for six diverse NLP tasks across up to 40 languages. We provide an analysis of success and failure cases of few-shot transfer, which highlights the role of lexical features. Additionally, we show that a straightforward full model finetuning approach is quite effective for few-shot transfer, outperforming several state-of-the-art few-shot approaches. As a step towards standardizing few-shot crosslingual experimental designs, we make our sampled few shots publicly available.

cs.CL↗

arXiv2021

Membranes in Optic Nerve Models

Membranes are present in all cells and tissues. Mathematical models of cells and tissues need a compact mathematical description of membranes with a resolution of about 1 nm. Membranes isolate cells because ions have difficulty penetrating the dielectric barrier they create. Here we introduce a dielectric mathematical membrane condition to replace a condition that did not include dielectric properties. Our mathematical membrane condition includes a dielectric lipid bilayer punctured by channels that conduct ions selectively.

math.DS↗

arXiv2021

A Tridomain Model for Potassium Clearance in Optic Nerve of Necturus

The accumulation of potassium in the narrow space outside nerve cells is a classical subject of biophysics that has received much attention recently. It may be involved in potassium accumulation \textcolor{black}{including} spreading depression, perhaps migraine and some kinds of epilepsy, even (speculatively) learning. Quantitative analysis is likely to help evaluate the role of potassium clearance from the extracellular space after a train of action potentials. Clearance involves three structures that extend down the length of the nerve: glia, extracellular space, and axon and so need to be described as systems distributed in space in the tradition used for electrical potential in the `cable equations' of nerve since the work of Hodgkin in 1937. A three-compartment model is proposed here for the optic nerve and is used to study the accumulation of potassium and its clearance. The model allows the convection, diffusion, and electrical migration of water and ions. We depend on the data of Orkand et al to ensure the relevance of our model and align its parameters with the anatomy and properties of membranes, channels, and transporters: our model fits their experimental data quite well. The aligned model shows that glia has an important role in buffering potassium, as expected. The model shows that potassium is cleared mostly by convective flow through the syncytia of glia driven by osmotic pressure differences. A simplified model might be possible, but it must involve flow down the length of the optic nerve. It is easy for compartment models to neglect this flow. Our model can be used for structures quite different from the optic nerve that might have different distributions of channels and transporters in its three compartments. It can be generalized to include a fourth (distributed) compartment representing blood vessels to deal with the glymphatic flow into the circulatory system.

physics.bio-ph↗

arXiv2021

Rapid Multi-Physics Simulation for Electro-Thermal Origami Systems

Electro-thermally actuated origami provides a novel method for creating 3-D systems with advanced morphing and functional capabilities. However, it is currently difficult to simulate the multi-physical behavior of such systems because the electro-thermal actuation and large folding deformations are highly interdependent. In this work, we introduce a rapid multi-physics simulation framework for electro-thermally actuated origami systems that can simultaneously capture: thermo-mechancially coupled actuation, inter panel contact, heat transfer, large deformation folding, and other complex loading applied onto the origami. Comparisons with finite element models validate the proposed framework for simulating origami heat transfer with different system geometries, materials, and surrounding environments. Verification of the simulated folding behaviors against physical electro-thermal micro-origami further demonstrates the validity of the proposed model. Simulations of more complex origami patterns and a case study for origami optimization are provided as application examples to show the capability and efficiency of the model. The framework provides a novel simulation tool for analysis, design, control, and optimization of active origami systems, pushing the boundary for feasible shape morphing and functional capability.

cs.RO↗

arXiv2021

Optic Nerve Microcirculation: Fluid Flow and Electrodiffusion

Complex fluids flow in complex ways in complex structures. Transport of water and various organic and inorganic molecules in the central nervous system are important in a wide range of biological and medical processes [C. Nicholson, and S. Hrabětová, Biophysical Journal, 113(10), 2133(2017)]. However, the exact driving mechanisms are often not known. In this paper, we investigate flows induced by action potentials in an optic nerve as a prototype of the central nervous system (CNS). Different from traditional fluid dynamics problems, flows in biological tissues such as the CNS are coupled with ion transport. It is driven by osmosis created by concentration gradient of ionic solutions, which in term influence the transport of ions. Our mathematical model is based on the known structural and biophysical properties of the experimental system used by the Harvard group Orkand et al [R.K. Orkand, J.G. Nicholls, S.W. Kuffler, Journal of Neurophysiology, 29(4), 788(1966)]. Asymptotic analysis and numerical computation show the significant role of water in convective ion transport. The full model (including water) and the electrodiffusion model (excluding water) are compared in detail to reveal an interesting interplay between water and ion transport. In the full model, convection due to water flow dominates inside the glial domain. This water flow in the glia contributes significantly to the spatial buffering of potassium in the extracellular space. Convection in the extracellular domain does not contribute significantly to spatial buffering. Electrodiffusion is the dominant mechanism for flows confined to the extracellular domain.

physics.bio-ph↗

arXiv2021

A phase field model for mass transport with semi-permeable interfaces

In this paper, a thermal-dynamical consistent model for mass transfer across permeable moving interfaces is proposed by using the energy variation method. We consider a restricted diffusion problem where the flux across the interface depends on its conductance and the difference of the concentration on each side. The diffusive interface phase-field framework used here has several advantages over the sharp interface method. First of all, explicit tracking of the interface is no longer necessary. Secondly, the interfacial condition can be incorporated with a variable diffusion coefficient. A detailed asymptotic analysis confirms the diffusive interface model converges to the existing sharp interface model as the interface thickness goes to zero. A decoupled energy stable numerical scheme is developed to solve this system efficiently. Numerical simulations first illustrate the consistency of theoretical results on the sharp interface limit. Then a convergence study and energy decay test are conducted to ensure the efficiency and stability of the numerical scheme. To illustrate the effectiveness of our phase-field approach, several examples are provided, including a study of a two-phase mass transfer problem where drops with deformable interfaces are suspended in a moving fluid.

math.NA↗

arXiv2021

Three-fold Weyl points in the Schrödinger operator with periodic potentials

Weyl points are degenerate points on the spectral bands at which energy bands intersect conically. They are the origins of many novel physical phenomena and have attracted much attention recently. In this paper, we investigate the existence of such points in the spectrum of the 3-dimensional Schrödinger operator $H = - Δ+V(\textbf{x})$ with $V(\textbf{x})$ being in a large class of periodic potentials. Specifically, we give very general conditions on the potentials which ensure the existence of 3-fold Weyl points on the associated energy bands. Different from 2-dimensional honeycomb structures which possess Dirac points where two adjacent band surfaces touch each other conically, the 3-fold Weyl points are conically intersection points of two energy bands with an extra band sandwiched in between. To ensure the 3-fold and 3-dimensional conical structures, more delicate, new symmetries are required. As a consequence, new techniques combining more symmetries are used to justify the existence of such conical points under the conditions proposed. This paper provides comprehensive proof of such 3-fold Weyl points. In particular, the role of each symmetry endowed to the potential is carefully analyzed. Our proof extends the analysis on the conical spectral points to a higher dimension and higher multiplicities. We also provide some numerical simulations on typical potentials to demonstrate our analysis.

math-ph↗

arXiv2021

Subterahertz collective dynamics of polar vortices

The collective dynamics of topological structures have been of great interest from both fundamental and applied perspectives. For example, the studies of dynamical properties of magnetic vortices and skyrmions not only deepened the understanding of many-body physics but also led to potential applications in data processing and storage. Topological structures constructed from electrical polarization rather than spin have recently been realized in ferroelectric superlattices, promising for ultrafast electric-field control of topological orders. However, little is known about the dynamics of such complex extended nanostructures which in turn underlies their functionalities. Using terahertz-field excitation and femtosecond x-ray diffraction measurements, we observe ultrafast collective polarization dynamics that are unique to polar vortices, with orders of magnitude higher frequencies and smaller lateral size than those of experimentally realized magnetic vortices. A previously unseen soft mode, hereafter referred to as a vortexon, emerges as transient arrays of nanoscale circular patterns of atomic displacements, which reverse their vorticity on picosecond time scales. Its frequency is significantly reduced at a critical strain, indicating a condensation of structural dynamics. First-principles-based atomistic calculations and phase-field modeling reveal the microscopic atomic arrangements and frequencies of the vortex modes. The discovery of subterahertz collective dynamics in polar vortices opens up opportunities for applications of electric-field driven data processing in topological structures with ultrahigh speed and density.

cond-mat.mes-hall↗

arXiv2021

Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification

Semi-supervised learning through deep generative models and multi-lingual pretraining techniques have orchestrated tremendous success across different areas of NLP. Nonetheless, their development has happened in isolation, while the combination of both could potentially be effective for tackling task-specific labelled data shortage. To bridge this gap, we combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task. Compared to strong supervised learning baselines, our semi-supervised classification framework is highly competitive and outperforms the state-of-the-art counterparts in low-resource settings across several languages.

cs.CL↗

arXiv2020

ResNeSt: Split-Attention Networks

It is well known that featuremap attention and multi-path representation are important for visual recognition. In this paper, we present a modularized architecture, which applies the channel-wise attention on different network branches to leverage their success in capturing cross-feature interactions and learning diverse representations. Our design results in a simple and unified computation block, which can be parameterized using only a few variables. Our model, named ResNeSt, outperforms EfficientNet in accuracy and latency trade-off on image classification. In addition, ResNeSt has achieved superior transfer learning results on several public benchmarks serving as the backbone, and has been adopted by the winning entries of COCO-LVIS challenge. The source code for complete system and pretrained models are publicly available.

cs.CV↗