Searcharxiv⌕ Search

arXiv subjects

Yiyang Li

Publications and source records attributed to Yiyang Li.

At least 55 records · Page 3Linked to original sources
arXiv2024

CIER: A Novel Experience Replay Approach with Causal Inference in Deep Reinforcement Learning

In the training process of Deep Reinforcement Learning (DRL), agents require repetitive interactions with the environment. With an increase in training volume and model complexity, it is still a challenging problem to enhance data utilization and explainability of DRL training. This paper addresses these challenges by focusing on the temporal correlations within the time dimension of time series. We propose a novel approach to segment multivariate time series into meaningful subsequences and represent the time series based on these subsequences. Furthermore, the subsequences are employed for causal inference to identify fundamental causal factors that significantly impact training outcomes. We design a module to provide feedback on the causality during DRL training. Several experiments demonstrate the feasibility of our approach in common environments, confirming its ability to enhance the effectiveness of DRL training and impart a certain level of explainability to the training process. Additionally, we extended our approach with priority experience replay algorithm, and experimental results demonstrate the continued effectiveness of our approach.

cs.LG↗
arXiv2024

Improving Factual Error Correction for Abstractive Summarization via Data Distillation and Conditional-generation Cloze

Improving factual consistency in abstractive summarization has been a focus of current research. One promising approach is the post-editing method. However, previous works have yet to make sufficient use of factual factors in summaries and suffers from the negative effect of the training datasets. In this paper, we first propose a novel factual error correction model FactCloze based on a conditional-generation cloze task. FactCloze can construct the causality among factual factors while being able to determine whether the blank can be answered or not. Then, we propose a data distillation method to generate a more faithful summarization dataset SummDSC via multiple-dimensional evaluation. We experimentally validate the effectiveness of our approach, which leads to an improvement in multiple factual consistency metrics compared to baselines.

cs.CL↗
arXiv2023

Just ClozE! A Novel Framework for Evaluating the Factual Consistency Faster in Abstractive Summarization

The issue of factual consistency in abstractive summarization has received extensive attention in recent years, and the evaluation of factual consistency between summary and document has become an important and urgent task. Most of the current evaluation metrics are adopted from the question answering (QA) or natural language inference (NLI) task. However, the application of QA-based metrics is extremely time-consuming in practice while NLI-based metrics are lack of interpretability. In this paper, we propose a cloze-based evaluation framework called ClozE and show the great potential of the cloze-based metric. It inherits strong interpretability from QA, while maintaining the speed of NLI- level reasoning. We demonstrate that ClozE can reduce the evaluation time by nearly 96% relative to QA-based metrics while retaining their interpretability and performance through experiments on six human-annotated datasets and a meta-evaluation benchmark GO FIGURE (Gabriel et al., 2021). Finally, we discuss three important facets of ClozE in practice, which further shows better overall performance of ClozE compared to other metrics.

cs.CL↗
arXiv2023

MCNS: Mining Causal Natural Structures Inside Time Series via A Novel Internal Causality Scheme

Causal inference permits us to discover covert relationships of various variables in time series. However, in most existing works, the variables mentioned above are the dimensions. The causality between dimensions could be cursory, which hinders the comprehension of the internal relationship and the benefit of the causal graph to the neural networks (NNs). In this paper, we find that causality exists not only outside but also inside the time series because it reflects a succession of events in the real world. It inspires us to seek the relationship between internal subsequences. However, the challenges are the hardship of discovering causality from subsequences and utilizing the causal natural structures to improve NNs. To address these challenges, we propose a novel framework called Mining Causal Natural Structure (MCNS), which is automatic and domain-agnostic and helps to find the causal natural structures inside time series via the internal causality scheme. We evaluate the MCNS framework and impregnation NN with MCNS on time series classification tasks. Experimental results illustrate that our impregnation, by refining attention, shape selection classification, and pruning datasets, drives NN, even the data itself preferable accuracy and interpretability. Besides, MCNS provides an in-depth, solid summary of the time series and datasets.

cs.LG↗
arXiv2023

Pre-training Multi-party Dialogue Models with Latent Discourse Inference

Multi-party dialogues are more difficult for models to understand than one-to-one two-party dialogues, since they involve multiple interlocutors, resulting in interweaving reply-to relations and information flows. To step over these obstacles, an effective way is to pre-train a model that understands the discourse structure of multi-party dialogues, namely, to whom each utterance is replying. However, due to the lack of explicitly annotated discourse labels in multi-party dialogue corpora, previous works fail to scale up the pre-training process by putting aside the unlabeled multi-party conversational data for nothing. To fully utilize the unlabeled data, we propose to treat the discourse structures as latent variables, then jointly infer them and pre-train the discourse-aware model by unsupervised latent variable inference methods. Experiments on multiple downstream tasks show that our pre-trained model outperforms strong baselines by large margins and achieves state-of-the-art (SOTA) results, justifying the effectiveness of our method. The official implementation of this paper is available at https://github.com/EricLee8/MPD_EMVI.

cs.CL↗
arXiv2023

EM Pre-training for Multi-party Dialogue Response Generation

Dialogue response generation requires an agent to generate a response according to the current dialogue history, in terms of which two-party dialogues have been well studied, but leaving a great gap for multi-party dialogues at the same time. Different from two-party dialogues where each response is a direct reply to its previous utterance, the addressee of a response utterance should be specified before it is generated in the multi-party scenario. Thanks to the huge amount of two-party conversational data, various pre-trained language models for two-party dialogue response generation have been proposed. However, due to the lack of annotated addressee labels in multi-party dialogue datasets, it is hard to use them to pre-train a response generation model for multi-party dialogues. To tackle this obstacle, we propose an Expectation-Maximization (EM) approach that iteratively performs the expectation steps to generate addressee labels, and the maximization steps to optimize a response generation model. Theoretical analyses and extensive experiments have justified the feasibility and effectiveness of our proposed method.

cs.CL↗
arXiv2023

Jantzen filtration of Weyl modules for general linear supergroups

Let $G=GL(m|n)$ be a general linear supergroup over an algebraically closed field $k$ of odd characteristic $p$. In this paper we construct Jantzen filtration of Weyl modules $V(λ)$ of $G$ when $λ$ is a typical weight in the sense of Kac's definition, and consequently obtain a sum formula for their characters. By Steinberg's tensor product theorem, it is enough for us to study typical weights with aim to formulate irreducible characters. As an application, it turns out that an irreducible $G$-module $L(λ)$ can be realized as a Kac module if and only if $λ$ is $p$-typical.

math.RT↗
arXiv2023

Comfortable Priority Handling with Predictive Velocity Optimization for Intersection Crossings

We address the problem of motion planning for four-way intersection crossings with right-of-ways. Road safety typically assigns liability to the follower in rear-end collisions and to the approaching vehicle required to yield in side crashes. As an alternative to previous models based on heuristic state machines, we propose a planning framework which changes the prediction model of other cars (e.g. their prototypical accelerations and decelerations) depending on the given longitudinal or lateral priority rules. Combined with a state-of-the-art trajectory optimization approach ROPT (Risk Optimization Method) this allows to find ego velocity profiles minimizing risks from curves and all involved vehicles while maximizing utility (needed time to arrive at a goal) and comfort (change and duration of acceleration) under the presence of regulatory conditions. Analytical and statistical evaluations show that our method is able to follow right-of-ways for a wide range of other vehicle behaviors and path geometries. Even when the other cars drive in a non-priority-compliant way, ROPT achieves good risk-comfort tradeoffs.

cs.RO↗
arXiv2023

Optimization of Velocity Ramps with Survival Analysis for Intersection Merge-Ins

We consider the problem of correct motion planning for T-intersection merge-ins of arbitrary geometry and vehicle density. A merge-in support system has to estimate the chances that a gap between two consecutive vehicles can be taken successfully. In contrast to previous models based on heuristic gap size rules, we present an approach which optimizes the integral risk of the situation using parametrized velocity ramps. It accounts for the risks from curves and all involved vehicles (front and rear on all paths) with a so-called survival analysis. For comparison, we also introduce a specially designed extension of the Intelligent Driver Model (IDM) for entering intersections. We show in a quantitative statistical evaluation that the survival method provides advantages in terms of lower absolute risk (i.e., no crash happens) and better risk-utility tradeoff (i.e., making better use of appearing gaps). Furthermore, our approach generalizes to more complex situations with additional risk sources.

cs.AI↗
arXiv2022

Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling

Multi-turn dialogue modeling as a challenging branch of natural language understanding (NLU), aims to build representations for machines to understand human dialogues, which provides a solid foundation for multiple downstream tasks. Recent studies of dialogue modeling commonly employ pre-trained language models (PrLMs) to encode the dialogue history as successive tokens, which is insufficient in capturing the temporal characteristics of dialogues. Therefore, we propose Bidirectional Information Decoupling Network (BiDeN) as a universal dialogue encoder, which explicitly incorporates both the past and future contexts and can be generalized to a wide range of dialogue-related tasks. Experimental results on datasets of different downstream tasks demonstrate the universality and effectiveness of our BiDeN.

cs.CL↗
arXiv2022

Nonvolatile Electrochemical Random-Access Memory Under Short Circuit

Electrochemical random-access memory (ECRAM) is a recently developed and highly promising analog resistive memory element for in-memory computing. One longstanding challenge of ECRAM is attaining retention time beyond a few hours. This short retention has precluded ECRAM from being considered for inference classification in deep neural networks, which is likely the largest opportunity for in-memory computing. In this work, we develop an ECRAM cell with orders of magnitude longer retention than previously achieved, and which we anticipate to exceed 10 years at 85C. We hypothesize that the origin of this exceptional retention is phase separation, which enables the formation of multiple effectively equilibrium resistance states. This work highlights the promises and opportunities to use phase separation to yield ECRAM cells with exceptionally long, and potentially permanent, retention times.

cs.ET↗
arXiv2022

Semantic-Preserving Adversarial Code Comprehension

Based on the tremendous success of pre-trained language models (PrLMs) for source code comprehension tasks, current literature studies either ways to further improve the performance (generalization) of PrLMs, or their robustness against adversarial attacks. However, they have to compromise on the trade-off between the two aspects and none of them consider improving both sides in an effective and practical way. To fill this gap, we propose Semantic-Preserving Adversarial Code Embeddings (SPACE) to find the worst-case semantic-preserving attacks while forcing the model to predict the correct labels under these worst cases. Experiments and analysis demonstrate that SPACE can stay robust against state-of-the-art attacks while boosting the performance of PrLMs for code.

cs.CL↗
arXiv2021

Self- and Pseudo-self-supervised Prediction of Speaker and Key-utterance for Multi-party Dialogue Reading Comprehension

Multi-party dialogue machine reading comprehension (MRC) brings tremendous challenge since it involves multiple speakers at one dialogue, resulting in intricate speaker information flows and noisy dialogue contexts. To alleviate such difficulties, previous models focus on how to incorporate these information using complex graph-based modules and additional manually labeled data, which is usually rare in real scenarios. In this paper, we design two labour-free self- and pseudo-self-supervised prediction tasks on speaker and key-utterance to implicitly model the speaker information flows, and capture salient clues in a long dialogue. Experimental results on two benchmark datasets have justified the effectiveness of our method over competitive baselines and current state-of-the-art models.

cs.CL↗
arXiv2021

Recurrent multiple shared layers in Depth for Neural Machine Translation

Learning deeper models is usually a simple and effective approach to improve model performance, but deeper models have larger model parameters and are more difficult to train. To get a deeper model, simply stacking more layers of the model seems to work well, but previous works have claimed that it cannot benefit the model. We propose to train a deeper model with recurrent mechanism, which loops the encoder and decoder blocks of Transformer in the depth direction. To address the increasing of model parameters, we choose to share parameters in different recursive moments. We conduct our experiments on WMT16 English-to-German and WMT14 English-to-France translation tasks, our model outperforms the shallow Transformer-Base/Big baseline by 0.35, 1.45 BLEU points, which is 27.23% of Transformer-Big model parameters. Compared to the deep Transformer(20-layer encoder, 6-layer decoder), our model has similar model performance and infer speed, but our model parameters are 54.72% of the former.

cs.CL↗
arXiv2021

Residual Tree Aggregation of Layers for Neural Machine Translation

Although attention-based Neural Machine Translation has achieved remarkable progress in recent layers, it still suffers from issue of making insufficient use of the output of each layer. In transformer, it only uses the top layer of encoder and decoder in the subsequent process, which makes it impossible to take advantage of the useful information in other layers. To address this issue, we propose a residual tree aggregation of layers for Transformer(RTAL), which helps to fuse information across layers. Specifically, we try to fuse the information across layers by constructing a post-order binary tree. In additional to the last node, we add the residual connection to the process of generating child nodes. Our model is based on the Neural Machine Translation model Transformer and we conduct our experiments on WMT14 English-to-German and WMT17 English-to-France translation tasks. Experimental results across language pairs show that the proposed approach outperforms the strong baseline model significantly

cs.CL↗
arXiv2020

Asymptotic values of four Laplacian-type energies for matrices with degree-distance-based entries of random graphs

Let $f(D(i, j), d_i, d_j)$ be a real function symmetric in $i$ and $j$ with the property that $f(d, (1+o(1))np, (1+o(1))np)=(1+o(1))f(d, np, np)$ for $d=1,2$. Let $G$ be a graph, $d_i$ denote the degree of a vertex $i$ of $G$ and $D(i, j)$ denote the distance between vertices $i$ and $j$ in $G$. In this paper, we define the $f$-weighted Laplacian matrix for random graphs in the Erd$\ddot{o}$s-R$\acute{e}$nyi random graph model $\mathcal{G}_{n, p}$, where $p\in (0, 1)$ is fixed. Four weighted Laplacian type energies: the weighted Laplacian energy $\mathscr{LE}_f(G)$, weighted signless Laplacian energy $\mathscr{LE}^{+}_f(G)$, weighted incidence energy $\mathscr{IE}_f(G)$ and the weighted Laplacian-energy like invariant $\mathscr{LEL}_f(G)$ are introduced and studied. We obtain the asymptotic values of $\mathscr{IE}_f(G)$ and $\mathscr{LEL}_f(G)$, and the values of $\mathscr{LE}_f(G)$ and $\mathscr{LE}_f^{+}(G)$ under the condition that $f(D(i, j), d_i, d_j)$ is a function dependent only on $D(i, j)$. As a consequence, we get that for almost all graphs $G_p\in \mathcal{G}_{n, p}$, the energy for the matrix with degree-distance-based entries of $G_p$, $\mathscr{E}(W_f(G_p)) < \mathscr{LE}_f(G_p),$ the Laplacian energy of the matrix, which is a generalization of a conjecture by Gutman et al.

math.CO↗
arXiv2020

The asymptotic value of energy for matrices with degree-distance-based entries of random graphs

For a graph $G=(V, E)$ and $i, j\in V$, denote the distance between $i$ and $j$ in $G$ by $D(i, j)$ and the degrees of $i$, $j$ by $d_i$, $d_j$, respectively. Let $f(D(i, j), d_{i}, d_{j})$ be a function symmetric in $i$ and $j$. Define a matrix $W_f(G)$, called the weighted distance matrix, of $G$, with the $ij$-entry $W_f(G)(i, j)=f(D(i, j), d_{i}, d_{j})$ if $i\neq j$ and $W_f(G)(i, j)=0$ if $i=j$. In this paper, we prove that if the symmetric function $f$ satisfies that $f(D(i, j), (1+o(1))np, (1+o(1))np)=(1+o(1))f(D(i, j), np, np)$, then for almost all graphs $G_p$ in the $Erd\ddot{o}s$-$R\acute{e}nyi$ random graph model $\mathcal{G}_{n, p}$, the energy of $W_f(G_p)$ is $\{(\frac{8}{3π}\sqrt{p(1-p)}+o(1))\cdot|f(1, np, np)-f(2, np, np)|+o(|f(2, np, np)|)\}\cdot n^{3/2}$. As a consequence, we give the asymptotic values of energies of a variety of weighted distance matrices with function $f$ from distance-based only and mixed with degree-distance-based topological indices of chemical use. This generalizes our former result with only degree-based weights.

math.CO↗
arXiv2020

The asymptotic value of graph energy for random graphs with degree-based weights

In this paper, we investigate the energy of a weighted random graph $G_p(f)$ in $G_{n,p}(f)$, in which each edge $ij$ takes the weight $f(d_i,d_j)$, where $d_v$ is a random variable, the degree of vertex $v$ in the random graph $G_p$ of the Erdös--Rényi random graph model $G_{n,p}$, and $f$ is a symmetric real function on two variables. Suppose $|f(d_i,d_j)|\leq C n^m$ for some constants $C, m>0$, and $f((1+o(1))np,(1+o(1))np)=(1+o(1))f(np,np)$. Then, for almost all graphs $G_p(f)$ in $G_{n,p}(f)$, the energy of $G_p(f)$ is $(1+o(1))f(np,np)\frac{8}{3π}\sqrt{p(1-p)}\cdot n^{3/2},$ where $p\in(0,1)$ is any fixed and independent of $n$. Consequently, with this one basket we can get the asymptotic values of various kinds of graph energies of chemical use, such as Randić energy, ABC energy, and energies of random matrices obtained from various kinds of degree-based chemical indices.

math.CO↗