arXiv subjects

Xiaoli Li

Publications and source records attributed to Xiaoli Li.

At least 145 records · Page 8Linked to original sources

arXiv2021

Time-Series Representation Learning via Temporal and Contextual Contrasting

Learning decent representations from unlabeled time-series data with temporal dynamics is a very challenging task. In this paper, we propose an unsupervised Time-Series representation learning framework via Temporal and Contextual Contrasting (TS-TCC), to learn time-series representation from unlabeled data. First, the raw time-series data are transformed into two different yet correlated views by using weak and strong augmentations. Second, we propose a novel temporal contrasting module to learn robust temporal representations by designing a tough cross-view prediction task. Last, to further learn discriminative representations, we propose a contextual contrasting module built upon the contexts from the temporal contrasting module. It attempts to maximize the similarity among different contexts of the same sample while minimizing similarity among contexts of different samples. Experiments have been carried out on three real-world time-series datasets. The results manifest that training a linear classifier on top of the features learned by our proposed TS-TCC performs comparably with the supervised training. Additionally, our proposed TS-TCC shows high efficiency in few-labeled data and transfer learning scenarios. The code is publicly available at https://github.com/emadeldeen24/TS-TCC.

cs.LG↗

arXiv2021

Multilinear Dirichlet Processes

Dependent Dirichlet processes (DDP) have been widely applied to model data from distributions over collections of measures which are correlated in some way. On the other hand, in recent years, increasing research efforts in machine learning and data mining have been dedicated to dealing with data involving interactions from two or more factors. However, few researchers have addressed the heterogeneous relationship in data brought by modulation of multiple factors using techniques of DDP. In this paper, we propose a novel technique, MultiLinear Dirichlet Processes (MLDP), to constructing DDPs by combining DP with a state-of-the-art factor analysis technique, multilinear factor analyzers (MLFA). We have evaluated MLDP on real-word data sets for different applications and have achieved state-of-the-art performance.

cs.LG↗

arXiv2021

CODA: Constructivism Learning for Instance-Dependent Dropout Architecture Construction

Dropout is attracting intensive research interest in deep learning as an efficient approach to prevent overfitting. Recently incorporating structural information when deciding which units to drop out produced promising results comparing to methods that ignore the structural information. However, a major issue of the existing work is that it failed to differentiate among instances when constructing the dropout architecture. This can be a significant deficiency for many applications. To solve this issue, we propose Constructivism learning for instance-dependent Dropout Architecture (CODA), which is inspired from a philosophical theory, constructivism learning. Specially, based on the theory we have designed a better drop out technique, Uniform Process Mixture Models, using a Bayesian nonparametric method Uniform process. We have evaluated our proposed method on 5 real-world datasets and compared the performance with other state-of-the-art dropout techniques. The experimental results demonstrated the effectiveness of CODA.

cs.LG↗

arXiv2021

RST Parsing from Scratch

We introduce a novel top-down end-to-end formulation of document-level discourse parsing in the Rhetorical Structure Theory (RST) framework. In this formulation, we consider discourse parsing as a sequence of splitting decisions at token boundaries and use a seq2seq network to model the splitting decisions. Our framework facilitates discourse parsing from scratch without requiring discourse segmentation as a prerequisite; rather, it yields segmentation as part of the parsing process. Our unified parsing model adopts a beam search to decode the best tree structure by searching through a space of high-scoring trees. With extensive experiments on the standard English RST discourse treebank, we demonstrate that our parser outperforms existing methods by a good margin in both end-to-end parsing and parsing with gold segmentation. More importantly, it does so without using any handcrafted features, making it faster and easily adaptable to new languages and domains.

cs.CL↗

arXiv2021

DocOIE: A Document-level Context-Aware Dataset for OpenIE

Open Information Extraction (OpenIE) aims to extract structured relational tuples (subject, relation, object) from sentences and plays critical roles for many downstream NLP applications. Existing solutions perform extraction at sentence level, without referring to any additional contextual information. In reality, however, a sentence typically exists as part of a document rather than standalone; we often need to access relevant contextual information around the sentence before we can accurately interpret it. As there is no document-level context-aware OpenIE dataset available, we manually annotate 800 sentences from 80 documents in two domains (Healthcare and Transportation) to form a DocOIE dataset for evaluation. In addition, we propose DocIE, a novel document-level context-aware OpenIE model. Our experimental results based on DocIE demonstrate that incorporating document-level context is helpful in improving OpenIE performance. Both DocOIE dataset and DocIE model are released for public.

cs.CL↗

arXiv2021

Stability and error analysis of IMEX SAV schemes for the magneto-hydrodynamic equations

We construct and analyze first- and second-order implicit-explicit (IMEX) schemes based on the scalar auxiliary variable (SAV) approach for the magneto-hydrodynamic equations. These schemes are linear, only require solving a sequence of linear differential equations with constant coefficients at each time step, and are unconditionally energy stable. We derive rigorous error estimates for the velocity, pressure and magnetic field of the first-order scheme in the two dimensional case without any condition on the time step. Numerical examples are presented to validate the proposed schemes.

math.NA↗

arXiv2020

Multi-View Collaborative Network Embedding

Real-world networks often exist with multiple views, where each view describes one type of interaction among a common set of nodes. For example, on a video-sharing network, while two user nodes are linked if they have common favorite videos in one view, they can also be linked in another view if they share common subscribers. Unlike traditional single-view networks, multiple views maintain different semantics to complement each other. In this paper, we propose MANE, a multi-view network embedding approach to learn low-dimensional representations. Similar to existing studies, MANE hinges on diversity and collaboration - while diversity enables views to maintain their individual semantics, collaboration enables views to work together. However, we also discover a novel form of second-order collaboration that has not been explored previously, and further unify it into our framework to attain superior node representations. Furthermore, as each view often has varying importance w.r.t. different nodes, we propose MANE+, an attention-based extension of MANE to model node-wise view importance. Finally, we conduct comprehensive experiments on three public, real-world multi-view networks, and the results demonstrate that our models consistently outperform state-of-the-art approaches.

cs.LG↗

arXiv2020

Sea-Net: Squeeze-And-Excitation Attention Net For Diabetic Retinopathy Grading

Diabetes is one of the most common disease in individuals. \textit{Diabetic retinopathy} (DR) is a complication of diabetes, which could lead to blindness. Automatic DR grading based on retinal images provides a great diagnostic and prognostic value for treatment planning. However, the subtle differences among severity levels make it difficult to capture important features using conventional methods. To alleviate the problems, a new deep learning architecture for robust DR grading is proposed, referred to as SEA-Net, in which, spatial attention and channel attention are alternatively carried out and boosted with each other, improving the classification performance. In addition, a hybrid loss function is proposed to further maximize the inter-class distance and reduce the intra-class variability. Experimental results have shown the effectiveness of the proposed architecture.

cs.CV↗

arXiv2020

A highly efficient and accurate exponential semi-implicit scalar auxiliary variable (ESI-SAV) approach for dissipative system

The scalar auxiliary variable (SAV) approach is a very popular and efficient method to simulate various phase field models. To save the computational cost, a new SAV approach is given by introducing a new variable $θ$. The new SAV approach can be proved to save nearly half CPU time of the original SAV approach while keeping all its other advantages. In this paper, we propose a novel technique to construct an exponential semi-implicit scalar auxiliary variable (ESI-SAV) approach without introducing any extra variables. The new proposed method also only needs to solve one linear equation with constant coefficients at each time step. Furthermore, the constructed ESI-SAV method does not need the bounded below restriction of nonlinear free energy potential which is more reasonable and effective for various phase field models. Meanwhile it is easy to construct first-order, second-order and higher-order unconditionally energy stable time-stepping schemes. Other than that, the ESI-SAV approach can be proved to be effective to solve the non-gradient but dissipative system such as Navier-Stokes equations. Several numerical examples are provided to demonstrate the improved efficiency and accuracy of the proposed method.

math.NA↗

arXiv2020

On fully decoupled MSAV schemes for the Cahn-Hilliard-Navier-Stokes model of Two-Phase Incompressible Flows

We construct first- and second-order time discretization schemes for the Cahn-Hilliard-Navier-Stokes system based on the multiple scalar auxiliary variables approach (MSAV) approach for gradient systems and (rotational) pressure-correction for Navier-Stokes equations. These schemes are linear, fully decoupled, unconditionally energy stable, and only require solving a sequence of elliptic equations with constant coefficients at each time step. We carry out a rigorous error analysis for the first-order scheme, establishing optimal convergence rate for all relevant functions in different norms. We also provide numerical experiments to verify our theoretical results.

math.NA↗

arXiv2020

Attention Sequence to Sequence Model for Machine Remaining Useful Life Prediction

Accurate estimation of remaining useful life (RUL) of industrial equipment can enable advanced maintenance schedules, increase equipment availability and reduce operational costs. However, existing deep learning methods for RUL prediction are not completely successful due to the following two reasons. First, relying on a single objective function to estimate the RUL will limit the learned representations and thus affect the prediction accuracy. Second, while longer sequences are more informative for modelling the sensor dynamics of equipment, existing methods are less effective to deal with very long sequences, as they mainly focus on the latest information. To address these two problems, we develop a novel attention-based sequence to sequence with auxiliary task (ATS2S) model. In particular, our model jointly optimizes both reconstruction loss to empower our model with predictive capabilities (by predicting next input sequence given current input sequence) and RUL prediction loss to minimize the difference between the predicted RUL and actual RUL. Furthermore, to better handle longer sequence, we employ the attention mechanism to focus on all the important input information during training process. Finally, we propose a new dual-latent feature representation to integrate the encoder features and decoder hidden states, to capture rich semantic information in data. We conduct extensive experiments on four real datasets to evaluate the efficacy of the proposed method. Experimental results show that our proposed method can achieve superior performance over 13 state-of-the-art methods consistently.

cs.LG↗

arXiv2020

Efficient Constituency Parsing by Pointing

We propose a novel constituency parsing model that casts the parsing problem into a series of pointing tasks. Specifically, our model estimates the likelihood of a span being a legitimate tree constituent via the pointing score corresponding to the boundary words of the span. Our parsing model supports efficient top-down decoding and our learning objective is able to enforce structural consistency without resorting to the expensive CKY inference. The experiments on the standard English Penn Treebank parsing task show that our method achieves 92.78 F1 without using pre-trained models, which is higher than all the existing methods with similar time complexity. Using pre-trained BERT, our model achieves 95.48 F1, which is competitive with the state-of-the-art while being faster. Our approach also establishes new state-of-the-art in Basque and Swedish in the SPMRL shared tasks on multilingual constituency parsing.

cs.CL↗

arXiv2020

Differentiable Window for Dynamic Local Attention

We propose Differentiable Window, a new neural module and general purpose component for dynamic window selection. While universally applicable, we demonstrate a compelling use case of utilizing Differentiable Window to improve standard attention modules by enabling more focused attentions over the input regions. We propose two variants of Differentiable Window, and integrate them within the Transformer architecture in two novel ways. We evaluate our proposed approach on a myriad of NLP tasks, including machine translation, sentiment analysis, subject-verb agreement and language modeling. Our experimental results demonstrate consistent and sizable improvements across all tasks.

cs.LG↗

arXiv2020

Efficient Linear and Unconditionally Energy Stable Schemes for the Modified Phase Field Crystal Equation

In this paper, we construct efficient schemes based on the scalar auxiliary variable (SAV) block-centered finite difference method for the modified phase field crystal (MPFC) equation, which is a sixth-order nonlinear damped wave equation. The schemes are linear, conserve mass and unconditionally dissipate a pseudo energy. We prove rigorously second-order error estimates in both time and space for the phase field variable in discrete norms. We also present some numerical experiments to verify our theoretical results and demonstrate the robustness and accuracy.

math.NA↗

arXiv2020

StereoNeuroBayesSLAM: A Neurobiologically Inspired Stereo Visual SLAM System Based on Direct Sparse Method

We propose a neurobiologically inspired visual simultaneous localization and mapping (SLAM) system based on direction sparse method to real-time build cognitive maps of large-scale environments from a moving stereo camera. The core SLAM system mainly comprises a Bayesian attractor network, which utilizes neural responses of head direction (HD) cells in the hippocampus and grid cells in the medial entorhinal cortex (MEC) to represent the head direction and the position of the robot in the environment, respectively. Direct sparse method is employed to accurately and robustly estimate velocity information from a stereo camera. Input rotational and translational velocities are integrated by the HD cell and grid cell networks, respectively. We demonstrated our neurobiologically inspired stereo visual SLAM system on the KITTI odometry benchmark datasets. Our proposed SLAM system is robust to real-time build a coherent semi-metric topological map from a stereo camera. Qualitative evaluation on cognitive maps shows that our proposed neurobiologically inspired stereo visual SLAM system outperforms our previous brain-inspired algorithms and the neurobiologically inspired monocular visual SLAM system both in terms of tracking accuracy and robustness, which is closer to the traditional state-of-the-art one.

cs.RO↗

arXiv2020

New SAV-pressure correction methods for the Navier-Stokes equations: stability and error analysis

We construct new first- and second-order pressure correction schemes using the scalar auxiliary variable (SAV) approach for the Navier-Stokes equations. These schemes are linear, decoupled and only require a sequence of solving Poisson type equations at each time step. Furthermore, they are unconditionally energy stable. We also establish rigorous error estimates for the velocity and pressure approximation of the first-order scheme without any condition on the time step.

math.NA↗

arXiv2019

Step-by-step solving schemes based on scalar auxiliary variable and invariant energy quadratization approaches for gradient flows

In this paper, we propose several novel numerical techniques to deal with nonlinear terms in gradient flows. These step-by-step solving schemes, termed 3S-SAV and 3S-IEQ schemes, are based on recently popular scalar auxiliary variable (SAV) and invariant energy quadratization (IEQ) approaches. In these constructed numerical methods, the phase function $ϕ$ and auxiliary variable can be calculated step-by-step. Compared with the traditional SAV/IEQ approaches, there are many advantages for the novel 3S-SAV/3S-IEQ schemes. Firstly, we do not need the restriction of the bounded from below of the nonlinear free energy potential/density function. Secondly, the auxiliary variable combined with nonlinear function can be treated totally explicitly in the 3S-SAV/3S-IEQ approaches. Specially, for solving the discrete scheme based on IEQ approach, the linear system usually involves variable coefficients which change at each time step. However, the discrete scheme based on 3S-IEQ approach leads to linear equation with constant coefficients. Two comparative studies of traditional SAV/IEQ and 3S-SAV/3S-IEQ approaches are considered to show the accuracy and efficiency. Finally, we present various 2D numerical simulations to demonstrate the stability and accuracy.

math.NA↗

arXiv2019

The exponential scalar auxiliary variable (E-SAV) approach for phase field models and its explicit computing

In this paper, we consider an exponential scalar auxiliary variable (E-SAV) approach to obtain energy stable schemes for a class of phase field models. This novel auxiliary variable method based on exponential form of nonlinear free energy potential is more effective and applicable than the traditional SAV method which is very popular to construct energy stable schemes. The first contribution is that the auxiliary variable without square root removes the bounded from below restriction of the nonlinear free energy potential. Then, we prove the unconditional energy stability for the semi-discrete schemes carefully and rigorously. Another contribution is that we can discrete the auxiliary variable combined with the nonlinear term totally explicitly. Such modification is very efficient for fast calculation. Furthermore, the positive property of $r$ can be guaranteed which is very important and reasonable for the models' equivalence. Besides, for complex phase field models with two or more unknown variables and nonlinear terms, we construct a multiple E-SAV (ME-SAV) approach to enhance the applicability of the proposed E-SAV approach. A comparative study of classical SAV and E-SAV approaches is considered to show the accuracy and efficiency. Finally, we present various 2D numerical simulations to demonstrate the stability and accuracy.

math.NA↗