arXiv subjects

Shikun Zhang

Publications and source records attributed to Shikun Zhang.

120 records · Page 7Linked to original sources

arXiv2021

A Data-Centric Framework for Composable NLP Workflows

Empirical natural language processing (NLP) systems in application domains (e.g., healthcare, finance, education) involve interoperation among multiple components, ranging from data ingestion, human annotation, to text retrieval, analysis, generation, and visualization. We establish a unified open-source framework to support fast development of such sophisticated NLP workflows in a composable manner. The framework introduces a uniform data representation to encode heterogeneous results by a wide range of NLP tasks. It offers a large repository of processors for NLP tasks, visualization, and annotation, which can be easily assembled with full interoperability under the unified representation. The highly extensible framework allows plugging in custom processors from external off-the-shelf NLP and deep learning libraries. The whole framework is delivered through two modularized yet integratable open-source projects, namely Forte (for workflow infrastructure and NLP function processors) and Stave (for user interaction, visualization, and annotation).

cs.CL↗

arXiv2021

Legal Judgment Prediction with Multi-Stage CaseRepresentation Learning in the Real Court Setting

Legal judgment prediction(LJP) is an essential task for legal AI. While prior methods studied on this topic in a pseudo setting by employing the judge-summarized case narrative as the input to predict the judgment, neglecting critical case life-cycle information in real court setting could threaten the case logic representation quality and prediction correctness. In this paper, we introduce a novel challenging dataset from real courtrooms to predict the legal judgment in a reasonably encyclopedic manner by leveraging the genuine input of the case -- plaintiff's claims and court debate data, from which the case's facts are automatically recognized by comprehensively understanding the multi-role dialogues of the court debate, and then learnt to discriminate the claims so as to reach the final judgment through multi-task learning. An extensive set of experiments with a large civil trial data set shows that the proposed model can more accurately characterize the interactions among claims, fact and debate for legal judgment prediction, achieving significant improvements over strong state-of-the-art baselines. Moreover, the user study conducted with real judges and law school students shows the neural predictions can also be interpretable and easily observed, and thus enhancing the trial efficiency and judgment quality.

cs.CL↗

arXiv2021

Capturing Event Argument Interaction via A Bi-Directional Entity-Level Recurrent Decoder

Capturing interactions among event arguments is an essential step towards robust event argument extraction (EAE). However, existing efforts in this direction suffer from two limitations: 1) The argument role type information of contextual entities is mainly utilized as training signals, ignoring the potential merits of directly adopting it as semantically rich input features; 2) The argument-level sequential semantics, which implies the overall distribution pattern of argument roles over an event mention, is not well characterized. To tackle the above two bottlenecks, we formalize EAE as a Seq2Seq-like learning problem for the first time, where a sentence with a specific event trigger is mapped to a sequence of event argument roles. A neural architecture with a novel Bi-directional Entity-level Recurrent Decoder (BERD) is proposed to generate argument roles by incorporating contextual entities' argument role predictions, like a word-by-word text generation process, thereby distinguishing implicit argument distribution patterns within an event more accurately.

cs.CL↗

arXiv2021

Multi-view Inference for Relation Extraction with Uncertain Knowledge

Knowledge graphs (KGs) are widely used to facilitate relation extraction (RE) tasks. While most previous RE methods focus on leveraging deterministic KGs, uncertain KGs, which assign a confidence score for each relation instance, can provide prior probability distributions of relational facts as valuable external knowledge for RE models. This paper proposes to exploit uncertain knowledge to improve relation extraction. Specifically, we introduce ProBase, an uncertain KG that indicates to what extent a target entity belongs to a concept, into our RE architecture. We then design a novel multi-view inference framework to systematically integrate local context and global knowledge across three views: mention-, entity- and concept-view. The experimental results show that our model achieves competitive performances on both sentence- and document-level relation extraction, which verifies the effectiveness of introducing uncertain knowledge and the multi-view inference framework that we design.

cs.CL↗

arXiv2021

Dark Modes in Non-Markovian Linear Quantum Systems

In this note, we are concerned with dark modes in a class of non-Markovian open quantum systems. Based on a microscopic model, a time-convoluted linear quantum stochastic differential equation and an output equation are derived to describe the system dynamics. The definition of dark modes is given building on the input-output structure of the system. Then, we present a necessary and sufficient condition for the existence of dark modes. Also, the problem of dark mode synthesis via Hamiltonian engineering is constructively solved and an example is presented to illustrate our results.

quant-ph↗

arXiv2021

Exploiting Method Names to Improve Code Summarization: A Deliberation Multi-Task Learning Approach

Code summaries are brief natural language descriptions of source code pieces. The main purpose of code summarization is to assist developers in understanding code and to reduce documentation workload. In this paper, we design a novel multi-task learning (MTL) approach for code summarization through mining the relationship between method code summaries and method names. More specifically, since a method's name can be considered as a shorter version of its code summary, we first introduce the tasks of generation and informativeness prediction of method names as two auxiliary training objectives for code summarization. A novel two-pass deliberation mechanism is then incorporated into our MTL architecture to generate more consistent intermediate states fed into a summary decoder, especially when informative method names do not exist. To evaluate our deliberation MTL approach, we carried out a large-scale experiment on two existing datasets for Java and Python. The experiment results show that our technique can be easily applied to many state-of-the-art neural models for code summarization and improve their performance. Meanwhile, our approach shows significant superiority when generating summaries for methods with non-informative names.

cs.CL↗

arXiv2021

Expectation Synchronization Synthesis in Non-Markovian Open Quantum Systems

In this article, we investigate the problem of engineering synchronization in non-Markovian quantum systems. First, a time-convoluted linear quantum stochastic differential equation is derived which describes the Heisenberg evolution of a localized quantum system driven by multiple colored noise inputs. Then, we define quantum expectation synchronization in an augmented system consisting of two subsystems. We prove that, for two homogenous subsystems, synchronization can always be synthesized without designing direct Hamiltonian coupling given that the degree of non-Markovianity is below a certain threshold. System parameters are explicitly designed to achieve quantum synchronization. Also, a numerical example is presented to illustrate our results.

quant-ph↗

arXiv2020

SongMASS: Automatic Song Writing with Pre-training and Alignment Constraint

Automatic song writing aims to compose a song (lyric and/or melody) by machine, which is an interesting topic in both academia and industry. In automatic song writing, lyric-to-melody generation and melody-to-lyric generation are two important tasks, both of which usually suffer from the following challenges: 1) the paired lyric and melody data are limited, which affects the generation quality of the two tasks, considering a lot of paired training data are needed due to the weak correlation between lyric and melody; 2) Strict alignments are required between lyric and melody, which relies on specific alignment modeling. In this paper, we propose SongMASS to address the above challenges, which leverages masked sequence to sequence (MASS) pre-training and attention based alignment modeling for lyric-to-melody and melody-to-lyric generation. Specifically, 1) we extend the original sentence-level MASS pre-training to song level to better capture long contextual information in music, and use a separate encoder and decoder for each modality (lyric or melody); 2) we leverage sentence-level attention mask and token-level attention constraint during training to enhance the alignment between lyric and melody. During inference, we use a dynamic programming strategy to obtain the alignment between each word/syllable in lyric and note in melody. We pre-train SongMASS on unpaired lyric and melody datasets, and both objective and subjective evaluations demonstrate that SongMASS generates lyric and melody with significantly better quality than the baseline method without pre-training or alignment constraint.

cs.SD↗

arXiv2020

Leveraging Code Generation to Improve Code Retrieval and Summarization via Dual Learning

Code summarization generates brief natural language description given a source code snippet, while code retrieval fetches relevant source code given a natural language query. Since both tasks aim to model the association between natural language and programming language, recent studies have combined these two tasks to improve their performance. However, researchers have yet been able to effectively leverage the intrinsic connection between the two tasks as they train these tasks in a separate or pipeline manner, which means their performance can not be well balanced. In this paper, we propose a novel end-to-end model for the two tasks by introducing an additional code generation task. More specifically, we explicitly exploit the probabilistic correlation between code summarization and code generation with dual learning, and utilize the two encoders for code summarization and code generation to train the code retrieval task via multi-task learning. We have carried out extensive experiments on an existing dataset of SQL and Python, and results show that our model can significantly improve the results of the code retrieval task over the-state-of-art models, as well as achieve competitive performance in terms of BLEU score for the code summarization task.

cs.IR↗

arXiv2019

Floquet Engineering with Particle Swarm Optimization: Maximizing Topological Invariants

It is of theoretical and experimental interest to engineer topological phases with very large topological invariants via periodic driving. As advocated by this work, such Floquet engineering can be elegantly achieved by the particle swarm optimization (PSO) technique from the swarm intelligence family. With the recognition that conventional gradient-based optimization approaches are not suitable for directly optimizing topological invariants as integers, the highly effective PSO route yields new promises in the search for exotic topological phases, requiring limited physical resource. Our results are especially timely in view of two important insights from literature: low-frequency driving may be beneficial in creating large topological invariants, but an open-ended low-frequency driving often leads to drastic fluctuations in the obtained topological invariants. Indeed, using a simple continuously driven Harper model with three quasi-energy bands, we show that the Floquet-band Chern numbers can enjoy many-fold increase compared with that using a simple harmonic driving of the same period, without demanding more energy cost of the driving field. It is also found that the resulting Floquet insulator bands are still well-gapped, with the maximized topological invariants in agreement with physical observations from Thouless pumping. The emergence of many edge modes under the open boundary condition is also consistent with the bulk-edge correspondence. Our results are expected to be highly useful towards the optimization of many different types of topological invariants in Floquet topological matter.

cond-mat.mes-hall↗

arXiv2019

Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data

In practical scenario, relation extraction needs to first identify entity pairs that have relation and then assign a correct relation class. However, the number of non-relation entity pairs in context (negative instances) usually far exceeds the others (positive instances), which negatively affects a model's performance. To mitigate this problem, we propose a multi-task architecture which jointly trains a model to perform relation identification with cross-entropy loss and relation classification with ranking loss. Meanwhile, we observe that a sentence may have multiple entities and relation mentions, and the patterns in which the entities appear in a sentence may contain useful semantic information that can be utilized to distinguish between positive and negative instances. Thus we further incorporate the embeddings of character-wise/word-wise BIO tag from the named entity recognition task into character/word embeddings to enrich the input representation. Experiment results show that our proposed approach can significantly improve the performance of a baseline model with more than 10% absolute increase in F1-score, and outperform the state-of-the-art models on ACE 2005 Chinese and English corpus. Moreover, BIO tag embeddings are particularly effective and can be used to improve other models as well.

cs.CL↗

arXiv2019

Subspace Stabilization Analysis for Non-Markovian Open Quantum Systems

Studied in this article is non-Markovian open quantum systems parametrized by Hamiltonian H, coupling operator L, and memory kernel function γ, which is a proper candidate for describing the dynamics of various solid-state quantum information processing devices. We look into the subspace stabilization problem of the system from the perspective of dynamical systems and control. The problem translates itself into finding analytic conditions that characterize invariant and attractive subspaces. Necessary and sufficient conditions are found for subspace invariance based on algebraic computations, and sufficient conditions are derived for subspace attractivity by applying a double integral Lyapunov functional. Mathematical proof is given for those conditions and a numerical example is provided to illustrate the theoretical result.

quant-ph↗