arXiv subjects

Zequn Liu

Publications and source records attributed to Zequn Liu.

23 records · Page 2Linked to original sources

arXiv2020

Learning to Customize Model Structures for Few-shot Dialogue Generation Tasks

Training the generative models with minimal corpus is one of the critical challenges for building open-domain dialogue systems. Existing methods tend to use the meta-learning framework which pre-trains the parameters on all non-target tasks then fine-tunes on the target task. However, fine-tuning distinguishes tasks from the parameter perspective but ignores the model-structure perspective, resulting in similar dialogue models for different tasks. In this paper, we propose an algorithm that can customize a unique dialogue model for each task in the few-shot setting. In our approach, each dialogue model consists of a shared module, a gating module, and a private module. The first two modules are shared among all the tasks, while the third one will differentiate into different network structures to better capture the characteristics of the corresponding task. The extensive experiments on two datasets show that our method outperforms all the baselines in terms of task consistency, response quality, and diversity.

cs.CL↗

arXiv2020

Multi-task Learning via Adaptation to Similar Tasks for Mortality Prediction of Diverse Rare Diseases

Mortality prediction of diverse rare diseases using electronic health record (EHR) data is a crucial task for intelligent healthcare. However, data insufficiency and the clinical diversity of rare diseases make it hard for directly training deep learning models on individual disease data or all the data from different diseases. Mortality prediction for these patients with different diseases can be viewed as a multi-task learning problem with insufficient data and large task number. But the tasks with little training data also make it hard to train task-specific modules in multi-task learning models. To address the challenges of data insufficiency and task diversity, we propose an initialization-sharing multi-task learning method (Ada-Sit) which learns the parameter initialization for fast adaptation to dynamically measured similar tasks. We use Ada-Sit to train long short-term memory networks (LSTM) based prediction models on longitudinal EHR data. And experimental results demonstrate that the proposed model is effective for mortality prediction of diverse rare diseases.

cs.LG↗

arXiv2019

Early Prediction of Sepsis From Clinical Datavia Heterogeneous Event Aggregation

Sepsis is a life-threatening condition that seriously endangers millions of people over the world. Hopefully, with the widespread availability of electronic health records (EHR), predictive models that can effectively deal with clinical sequential data increase the possibility to predict sepsis and take early preventive treatment. However, the early prediction is challenging because patients' sequential data in EHR contains temporal interactions of multiple clinical events. And capturing temporal interactions in the long event sequence is hard for traditional LSTM. Rather than directly applying the LSTM model to the event sequences, our proposed model firstly aggregates heterogeneous clinical events in a short period and then captures temporal interactions of the aggregated representations with LSTM. Our proposed Heterogeneous Event Aggregation can not only shorten the length of clinical event sequence but also help to retain temporal interactions of both categorical and numerical features of clinical events in the multiple heads of the aggregation representations. In the PhysioNet/Computing in Cardiology Challenge 2019, with the team named PKU_DLIB, our proposed model, in high efficiency, achieved utility score (0.321) in the full test set.

cs.LG↗

arXiv2019

Topologies on Quotient Space of Matrices via Semi-tensor Product

An equivalence of matrices via semi-tensor product (STP) is proposed. Using this equivalence, the quotient space is obtained. Parallel and sequential arrangements of the natural projection on different shapes of matrices leads to the product topology and quotient topology respectively. Then the Frobenious inner product of matrices is extended to equivalence classes, which produces a metric on the quotient space. This metric leads to a metric topology. A comparison for these three topologies is presented. Some topological properties are revealed.

math.OC↗

arXiv2018

Cross-Dimensional Linear Systems

Semi-tensor product(STP) or matrix (M-) product of matrices turns the set of matrices with arbitrary dimensions into a monoid $({\cal M},\ltimes)$. A matrix (M-) addition is defined over subsets of a partition of ${\cal M}$, and a matrix (M-) equivalence is proposed. Eventually, some quotient spaces are obtained as vector spaces of matrices. Furthermore, a set of formal polynomials is constructed, which makes the quotient space of $({\cal M},\ltimes)$, denoted by $(Σ, \ltimes)$, a vector space and a monoid. Similarly, a vector addition (V-addition) and a vector equivalence (V-equivalence) are defined on ${\cal V}$, the set of vectors of arbitrary dimensions. Then the quotient space of vectors, $Ω$, is also obtained as a vector space. The action of monoid $({\cal M},\ltimes)$ on ${\cal V}$ (or $(Σ, \ltimes)$ on $Ω$) is defined as a vector (V-) product, which becomes a pseudo-dynamic system, called the cross-dimensional linear system (CDLS). Both the discrete time and the continuous time CDLSs have been investigated. For certain time-invariant case, the solutions (trajectories) are presented. Furthermore, the corresponding cross-dimensional linear control systems are also proposed and the controllability and observability are discussed. Both M-product and V-product are generalizations of the conventional matrix product, that is, when the dimension matching condition required by the conventional matrix product is satisfied they coincide with the conventional matrix product. Both M-addition and V-addition are generalizations of conventional matrix addition. Hence, the dynamics discussed in this paper is a generalization of conventional linear system theory.

math.OC↗