arXiv subjects

Liu Liu

Publications and source records attributed to Liu Liu.

At least 235 records · Page 13Linked to original sources

arXiv2021

Transcribing Natural Languages for The Deaf via Neural Editing Programs

This work studies the task of glossification, of which the aim is to em transcribe natural spoken language sentences for the Deaf (hard-of-hearing) community to ordered sign language glosses. Previous sequence-to-sequence language models trained with paired sentence-gloss data often fail to capture the rich connections between the two distinct languages, leading to unsatisfactory transcriptions. We observe that despite different grammars, glosses effectively simplify sentences for the ease of deaf communication, while sharing a large portion of vocabulary with sentences. This has motivated us to implement glossification by executing a collection of editing actions, e.g. word addition, deletion, and copying, called editing programs, on their natural spoken language counterparts. Specifically, we design a new neural agent that learns to synthesize and execute editing programs, conditioned on sentence contexts and partial editing results. The agent is trained to imitate minimal editing programs, while exploring more widely the program space via policy gradients to optimize sequence-wise transcription quality. Results show that our approach outperforms previous glossification models by a large margin.

cs.CL↗

arXiv2021

OMAD: Object Model with Articulated Deformations for Pose Estimation and Retrieval

Articulated objects are pervasive in daily life. However, due to the intrinsic high-DoF structure, the joint states of the articulated objects are hard to be estimated. To model articulated objects, two kinds of shape deformations namely the geometric and the pose deformation should be considered. In this work, we present a novel category-specific parametric representation called Object Model with Articulated Deformations (OMAD) to explicitly model the articulated objects. In OMAD, a category is associated with a linear shape function with shared shape basis and a non-linear joint function. Both functions can be learned from a large-scale object model dataset and fixed as category-specific priors. Then we propose an OMADNet to predict the shape parameters and the joint states from an object's single observation. With the full representation of the object shape and joint states, we can address several tasks including category-level object pose estimation and the articulated object retrieval. To evaluate these tasks, we create a synthetic dataset based on PartNet-Mobility. Extensive experiments show that our simple OMADNet can serve as a strong baseline for both tasks.

cs.CV↗

arXiv2021

Multi-fidelity methods for uncertainty propagation in kinetic equations

The construction of efficient methods for uncertainty quantification in kinetic equations represents a challenge due to the high dimensionality of the models: often the computational costs involved become prohibitive. On the other hand, precisely because of the curse of dimensionality, the construction of simplified models capable of providing approximate solutions at a computationally reduced cost has always represented one of the main research strands in the field of kinetic equations. Approximations based on suitable closures of the moment equations or on simplified collisional models have been studied by many authors. In the context of uncertainty quantification, it is therefore natural to take advantage of such models in a multi-fidelity setting where the original kinetic equation represents the high-fidelity model, and the simplified models define the low-fidelity surrogate models. The scope of this article is to survey some recent results about multi-fidelity methods for kinetic equations that are able to accelerate the solution of the uncertainty quantification process by combining high-fidelity and low-fidelity model evaluations with particular attention to the case of compressible and incompressible hydrodynamic limits. We will focus essentially on two classes of strategies: multi-fidelity control variates methods and bi-fidelity stochastic collocation methods. The various approaches considered are analyzed in light of the different surrogate models used and the different numerical techniques adopted. Given the relevance of the specific choice of the surrogate model, an application-oriented approach has been chosen in the presentation.

math.NA↗

arXiv2021

Bi-fidelity stochastic collocation methods for epidemic transport models with uncertainties

Uncertainty in data is certainly one of the main problems in epidemiology, as shown by the recent COVID-19 pandemic. The need for efficient methods capable of quantifying uncertainty in the mathematical model is essential in order to produce realistic scenarios of the spread of infection. In this paper, we introduce a bi-fidelity approach to quantify uncertainty in spatially dependent epidemic models. The approach is based on evaluating a high-fidelity model on a small number of samples properly selected from a large number of evaluations of a low-fidelity model. In particular, we will consider the class of multiscale transport models recently introduced in Bertaglia, Boscheri, Dimarco & Pareschi, Math. Biosci. Eng. (2021) and Boscheri, Dimarco & Pareschi, Math. Mod. Meth. App. Scie. (2021) as the high-fidelity reference and use simple two-velocity discrete models for low-fidelity evaluations. Both models share the same diffusive behavior and are solved with ad-hoc asymptotic-preserving numerical discretizations. A series of numerical experiments confirm the validity of the approach.

math.NA↗

arXiv2021

Weak-shot Fine-grained Classification via Similarity Transfer

Recognizing fine-grained categories remains a challenging task, due to the subtle distinctions among different subordinate categories, which results in the need of abundant annotated samples. To alleviate the data-hungry problem, we consider the problem of learning novel categories from web data with the support of a clean set of base categories, which is referred to as weak-shot learning. In this setting, we propose a method called SimTrans to transfer pairwise semantic similarity from base categories to novel categories. Specifically, we firstly train a similarity net on clean data, and then leverage the transferred similarity to denoise web training data using two simple yet effective strategies. In addition, we apply adversarial loss on similarity net to enhance the transferability of similarity. Comprehensive experiments demonstrate the effectiveness of our weak-shot setting and our SimTrans method. Datasets and codes are available at https://github.com/bcmi/SimTrans-Weak-Shot-Classification.

cs.CV↗

arXiv2021

Transformer Acceleration with Dynamic Sparse Attention

Transformers are the mainstream of NLP applications and are becoming increasingly popular in other domains such as Computer Vision. Despite the improvements in model quality, the enormous computation costs make Transformers difficult at deployment, especially when the sequence length is large in emerging applications. Processing attention mechanism as the essential component of Transformer is the bottleneck of execution due to the quadratic complexity. Prior art explores sparse patterns in attention to support long sequence modeling, but those pieces of work are on static or fixed patterns. We demonstrate that the sparse patterns are dynamic, depending on input sequences. Thus, we propose the Dynamic Sparse Attention (DSA) that can efficiently exploit the dynamic sparsity in the attention of Transformers. Compared with other methods, our approach can achieve better trade-offs between accuracy and model complexity. Moving forward, we identify challenges and provide solutions to implement DSA on existing hardware (GPUs) and specialized hardware in order to achieve practical speedup and efficiency improvements for Transformer execution.

cs.LG↗

arXiv2021

Channelized Axial Attention for Semantic Segmentation -- Considering Channel Relation within Spatial Attention for Semantic Segmentation

Spatial and channel attentions, modelling the semantic interdependencies in spatial and channel dimensions respectively, have recently been widely used for semantic segmentation. However, computing spatial and channel attentions separately sometimes causes errors, especially for those difficult cases. In this paper, we propose Channelized Axial Attention (CAA) to seamlessly integrate channel attention and spatial attention into a single operation with negligible computation overhead. Specifically, we break down the dot-product operation of the spatial attention into two parts and insert channel relation in between, allowing for independently optimized channel attention on each spatial location. We further develop grouped vectorization, which allows our model to run with very little memory consumption without slowing down the running speed. Comparative experiments conducted on multiple benchmark datasets, including Cityscapes, PASCAL Context, and COCO-Stuff, demonstrate that our CAA outperforms many state-of-the-art segmentation models (including dual attention) on all tested datasets.

cs.CV↗

arXiv2021

A bi-fidelity stochastic collocation method for transport equations with diffusive scaling and multi-dimensional random inputs

In this paper, we consider the development of efficient numerical methods for linear transport equations with random parameters and under the diffusive scaling. We extend to the present case the bi-fidelity stochastic collocation method introduced in [33,50,51]. For the high-fidelity transport model, the asymptotic-preserving scheme [29] is used for each stochastic sample. We employ the simple two-velocity Goldstein-Taylor equation as low-fidelity model to accelerate the convergence of the uncertainty quantification process. The choice is motivated by the fact that both models, high fidelity and low fidelity, share the same diffusion limit. Speed-up is achieved by proper selection of the collocation points and relative approximation of the high-fidelity solution. Extensive numerical experiments are conducted to show the efficiency and accuracy of the proposed method, even in non diffusive regimes, with empirical error bound estimations as studied in [16].

math.NA↗

arXiv2021

Multi-modal Affect Analysis using standardized data within subjects in the Wild

Human affective recognition is an important factor in human-computer interaction. However, the method development with in-the-wild data is not yet accurate enough for practical usage. In this paper, we introduce the affective recognition method focusing on facial expression (EXP) and valence-arousal calculation that was submitted to the Affective Behavior Analysis in-the-wild (ABAW) 2021 Contest. When annotating facial expressions from a video, we thought that it would be judged not only from the features common to all people, but also from the relative changes in the time series of individuals. Therefore, after learning the common features for each frame, we constructed a facial expression estimation model and valence-arousal model using time-series data after combining the common features and the standardized features for each video. Furthermore, the above features were learned using multi-modal data such as image features, AU, Head pose, and Gaze. In the validation set, our model achieved a facial expression score of 0.546. These verification results reveal that our proposed framework can improve estimation accuracy and robustness effectively.

cs.CV↗

arXiv2021

Robust Compressed Sensing using Generative Models

The goal of compressed sensing is to estimate a high dimensional vector from an underdetermined system of noisy linear equations. In analogy to classical compressed sensing, here we assume a generative model as a prior, that is, we assume the vector is represented by a deep generative model $G: \mathbb{R}^k \rightarrow \mathbb{R}^n$. Classical recovery approaches such as empirical risk minimization (ERM) are guaranteed to succeed when the measurement matrix is sub-Gaussian. However, when the measurement matrix and measurements are heavy-tailed or have outliers, recovery may fail dramatically. In this paper we propose an algorithm inspired by the Median-of-Means (MOM). Our algorithm guarantees recovery for heavy-tailed data, even in the presence of outliers. Theoretically, our results show our novel MOM-based algorithm enjoys the same sample complexity guarantees as ERM under sub-Gaussian assumptions. Our experiments validate both aspects of our claims: other algorithms are indeed fragile and fail under heavy-tailed and/or corrupted data, while our approach exhibits the predicted robustness.

stat.ML↗

arXiv2021

On the multi-species Boltzmann equation with uncertainty and its stochastic Galerkin approximation

In this paper the nonlinear multi-species Boltzmann equation with random uncertainty coming from the initial data and collision kernel is studied. Well-posedness and long-time behavior - exponential decay to the global equilibrium - of the analytical solution, and spectral gap estimate for the corresponding linearized gPC-based stochastic Galerkin system are obtained, by using and extending the analytical tools provided in [M. Briant and E. S. Daus, {\it Arch. Ration. Mech. Anal.}, 3, 1367-1443, 2016] for the deterministic problem in the perturbative regime, and in [E. S. Daus, S. Jin and L. Liu, {\it Kinet. Relat. Models}, 12, 909-922, 2019] for the single-species problem with uncertainty. The well-posedness result of the sensitivity system presented here has not been obtained so far even for the single-species case.

math.AP↗

arXiv2021

Towards Real-World Category-level Articulation Pose Estimation

Human life is populated with articulated objects. Current Category-level Articulation Pose Estimation (CAPE) methods are studied under the single-instance setting with a fixed kinematic structure for each category. Considering these limitations, we reform this problem setting for real-world environments and suggest a CAPE-Real (CAPER) task setting. This setting allows varied kinematic structures within a semantic category, and multiple instances to co-exist in an observation of real world. To support this task, we build an articulated model repository ReArt-48 and present an efficient dataset generation pipeline, which contains Fast Articulated Object Modeling (FAOM) and Semi-Authentic MixEd Reality Technique (SAMERT). Accompanying the pipeline, we build a large-scale mixed reality dataset ReArtMix and a real world dataset ReArtVal. We also propose an effective framework ReArtNOCS that exploits RGB-D input to estimate part-level pose for multiple instances in a single forward pass. Extensive experiments demonstrate that the proposed ReArtNOCS can achieve good performance on both CAPER and CAPE settings. We believe it could serve as a strong baseline for future research on the CAPER task.

cs.CV↗

arXiv2021

HiCOMEX: Facial Action Unit Recognition Based on Hierarchy Intensity Distribution and COMEX Relation Learning

The detection of facial action units (AUs) has been studied as it has the competition due to the wide-ranging applications thereof. In this paper, we propose a novel framework for the AU detection from a single input image by grasping the \textbf{c}o-\textbf{o}ccurrence and \textbf{m}utual \textbf{ex}clusion (COMEX) as well as the intensity distribution among AUs. Our algorithm uses facial landmarks to detect the features of local AUs. The features are input to a bidirectional long short-term memory (BiLSTM) layer for learning the intensity distribution. Afterwards, the new AU feature continuously passed through a self-attention encoding layer and a continuous-state modern Hopfield layer for learning the COMEX relationships. Our experiments on the challenging BP4D and DISFA benchmarks without any external data or pre-trained models yield F1-scores of 63.7\% and 61.8\% respectively, which shows our proposed networks can lead to performance improvement in the AU detection task.

cs.CV↗

arXiv2021

Finite Difference Nets: A Deep Recurrent Framework for Solving Evolution PDEs

There has been an arising trend of adopting deep learning methods to study partial differential equations (PDEs). In this paper, we introduce a deep recurrent framework for solving time-dependent PDEs without generating large scale data sets. We provide a new perspective, that is, a different type of architecture through exploring the possible connections between traditional numerical methods (such as finite difference schemes) and deep neural networks, particularly convolutional and fully-connected neural networks. Our proposed approach will show its effectiveness and efficiency in solving PDE models with an integral form, in particular, we test on one-way wave equations and system of conservation laws.

math.NA↗

arXiv2021

New-generation silicon photonics beyond the singlemode regime

The singlemode condition is one of the most important design rules for optical waveguides in guided-wave optics. The reason following the singlemode condition is that higher-order modes might be excited and thus introduce some undesired mode-mismatching loss as well as inter-mode crosstalk when light propagates along an optical waveguide beyond the singlemode regime. As a result, multimode photonic waveguides are usually not allowed. In this paper, we propose the concept of silicon photonics beyond the singlemode regime, developed with low-loss and low-crosstalk light propagation in multimode photonic waveguides with broadened silicon cores. In particular, silicon photonic waveguides with a broadened core region have shown an ultra-low-loss of ~0.1 dB/cm for the fundamental mode even without any special fabrication process. A micro-racetrack resonator fabricated with standard 220-nm-SOI MPW-foundry processes shows a record intrinsic Q-factor as high as 1.02*107 for the first time, corresponding to ultra-low waveguide propagation loss of only 0.065 dB/cm. A high-performance microwave photonic filter on silicon is then realized with an ultra-narrow 3-dB bandwidth of 20.6 MHz as well as a tuning range of ~20 GHz for the first time. An on-chip 100-cm-long delayline is also demonstrated by using the present broadened SOI photonic waveguides with compact Euler-curve bends, the measured propagation loss is ~0.14 dB/cm. The proposed concept of silicon photonics beyond the singlemode regime helps solve the issue of high propagation loss and also significantly reduces the random phase errors of light due to the random variations of waveguide dimensions. In particularity it enables silicon photonic devices with enhanced performances, which paves the way for new-generation silicon photonics realizing the large-scale photonic integration.

physics.optics↗

arXiv2021

Error estimate of a bi-fidelity method for kinetic equations with random parameters and multiple scales

In this paper, we conduct uniform error estimates of the bi-fidelity method for multi-scale kinetic equations. We take the Boltzmann and the linear transport equations as important examples. The main analytic tool is the hypocoercivity analysis for kinetic equations, considering solutions in a perturbative setting close to the global equilibrium. This allows us to obtain the error estimates in both kinetic and hydrodynamic regimes.

math.NA↗

arXiv2021

A deep neural network approach on solving the linear transport model under diffusive scaling

In this work, we propose a learning method for solving the linear transport equation under the diffusive scaling. Due to the multiscale nature of our model equation, the model is challenging to solve by using conventional methods. We employ the physical informed neural network (PINN) framework, a mesh-free learning method that can numerically solve partial differential equations. Compared to conventional methods (such as finite difference or finite element type), our proposed learning method is able to obtain the solution at any given points in the chosen domain accurately and efficiently, which enables us to better understand the physics underlying the model. In our framework, the solution is approximated by a neural network that satisfies both the governing equation and other constraints. The network is then trained with a combination of different loss terms. Using the approximation theory and energy estimates for kinetic models, we prove theoretically that the total loss vanishes as the neural network converges, upon which the neural network approximated solution converges pointwisely to the analytic solution of the linear transport model. Numerical experiments for two benchmark examples are conducted to verify the effectiveness and accuracy of our proposed method.

math.NA↗

arXiv2021

Solving the linear transport equation by a deep neural network approach

In this paper, we study the linear transport model by adopting the deep learning method, in particular the deep neural network (DNN) approach. While the interest of using DNN to study partial differential equations is arising, here we adapt it to study kinetic models, in particular the linear transport model. Moreover, theoretical analysis on the convergence of the neural network and its approximated solution towards the analytic solution is shown. We demonstrate the accuracy and effectiveness of the proposed DNN method in the numerical experiments.

math.NA↗