Searcharxiv⌕ Search

arXiv subjects

Aaron Chan

Publications and source records attributed to Aaron Chan.

At least 55 records · Page 3Linked to original sources
arXiv2023

ER-Test: Evaluating Explanation Regularization Methods for Language Models

By explaining how humans would solve a given task, human rationales can provide strong learning signal for neural language models (LMs). Explanation regularization (ER) aims to improve LM generalization by pushing the LM's machine rationales (Which input tokens did the LM focus on?) to align with human rationales (Which input tokens would humans focus on?). Though prior works primarily study ER via in-distribution (ID) evaluation, out-of-distribution (OOD) generalization is often more critical in real-world scenarios, yet ER's effect on OOD generalization has been underexplored. In this paper, we introduce ER-Test, a framework for evaluating ER models' OOD generalization along three dimensions: unseen dataset tests, contrast set tests, and functional tests. Using ER-Test, we extensively analyze how ER models' OOD generalization varies with different ER design choices. Across two tasks and six datasets, ER-Test shows that ER has little impact on ID performance but can yield large OOD performance gains. Also, we find that ER can improve OOD performance even with limited rationale supervision. ER-Test's results help demonstrate ER's utility and establish best practices for using ER effectively.

cs.CL↗
arXiv2023

UNIREX: A Unified Learning Framework for Language Model Rationale Extraction

An extractive rationale explains a language model's (LM's) prediction on a given task instance by highlighting the text inputs that most influenced the prediction. Ideally, rationale extraction should be faithful (reflective of LM's actual behavior) and plausible (convincing to humans), without compromising the LM's (i.e., task model's) task performance. Although attribution algorithms and select-predict pipelines are commonly used in rationale extraction, they both rely on certain heuristics that hinder them from satisfying all three desiderata. In light of this, we propose UNIREX, a flexible learning framework that generalizes rationale extractor optimization as follows: (1) specify architecture for a learned rationale extractor; (2) select explainability objectives (i.e., faithfulness and plausibility criteria); and (3) jointly the train task model and rationale extractor on the task using the selected objectives. UNIREX enables replacing prior works' heuristic design choices with a generic learned rationale extractor in (1) and optimizing it for all three desiderata in (2)-(3). To facilitate comparison between methods with respect to multiple desiderata, we introduce the Normalized Relative Gain (NRG) metric. Across five text classification datasets, our best UNIREX configuration outperforms baselines by an average of 32.9% NRG. Plus, we find that UNIREX-trained rationale extractors can even generalize to unseen datasets and tasks.

cs.CL↗
arXiv2022

FRAME: Evaluating Rationale-Label Consistency Metrics for Free-Text Rationales

Following how humans communicate, free-text rationales aim to use natural language to explain neural language model (LM) behavior. However, free-text rationales' unconstrained nature makes them prone to hallucination, so it is important to have metrics for free-text rationale quality. Existing free-text rationale metrics measure how consistent the rationale is with the LM's predicted label, but there is no protocol for assessing such metrics' reliability. Thus, we propose FRAME, a framework for evaluating rationale-label consistency (RLC) metrics for free-text rationales. FRAME is based on three axioms: (1) good metrics should yield highest scores for reference rationales, which maximize RLC by construction; (2) good metrics should be appropriately sensitive to semantic perturbation of rationales; and (3) good metrics should be robust to variation in the LM's task performance. Across three text classification datasets, we show that existing RLC metrics cannot satisfy all three FRAME axioms, since they are implemented via model pretraining which muddles the metric's signal. Then, we introduce a non-pretraining RLC metric that greatly outperforms baselines on (1) and (3), while performing competitively on (2). Finally, we discuss the limitations of using RLC to evaluate free-text rationales.

cs.CL↗
arXiv2022

XMD: An End-to-End Framework for Interactive Explanation-Based Debugging of NLP Models

NLP models are susceptible to learning spurious biases (i.e., bugs) that work on some datasets but do not properly reflect the underlying task. Explanation-based model debugging aims to resolve spurious biases by showing human users explanations of model behavior, asking users to give feedback on the behavior, then using the feedback to update the model. While existing model debugging methods have shown promise, their prototype-level implementations provide limited practical utility. Thus, we propose XMD: the first open-source, end-to-end framework for explanation-based model debugging. Given task- or instance-level explanations, users can flexibly provide various forms of feedback via an intuitive, web-based UI. After receiving user feedback, XMD automatically updates the model in real time, by regularizing the model so that its explanations align with the user feedback. The new model can then be easily deployed into real-world applications via Hugging Face. Using XMD, we can improve the model's OOD performance on text classification tasks by up to 18%.

cs.CL↗
arXiv2022

SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

Augmenting pre-trained language models with knowledge graphs (KGs) has achieved success on various commonsense reasoning tasks. However, for a given task instance, the KG, or certain parts of the KG, may not be useful. Although KG-augmented models often use attention to focus on specific KG components, the KG is still always used, and the attention mechanism is never explicitly taught which KG components should be used. Meanwhile, saliency methods can measure how much a KG feature (e.g., graph, node, path) influences the model to make the correct prediction, thus explaining which KG features are useful. This paper explores how saliency explanations can be used to improve KG-augmented models' performance. First, we propose to create coarse (Is the KG useful?) and fine (Which nodes/paths in the KG are useful?) saliency explanations. Second, to motivate saliency-based supervision, we analyze oracle KG-augmented models which directly use saliency explanations as extra inputs for guiding their attention. Third, we propose SalKG, a framework for KG-augmented models to learn from coarse and/or fine saliency explanations. Given saliency explanations created from a task's training set, SalKG jointly trains the model to predict the explanations, then solve the task by attending to KG features highlighted by the predicted explanations. On three commonsense QA benchmarks (CSQA, OBQA, CODAH) and a range of KG-augmented models, we show that SalKG can yield considerable performance gains -- up to 2.76% absolute improvement on CSQA.

cs.CL↗
arXiv2021

Learning Contextualized Knowledge Structures for Commonsense Reasoning

Recently, knowledge graph (KG) augmented models have achieved noteworthy success on various commonsense reasoning tasks. However, KG edge (fact) sparsity and noisy edge extraction/generation often hinder models from obtaining useful knowledge to reason over. To address these issues, we propose a new KG-augmented model: Hybrid Graph Network (HGN). Unlike prior methods, HGN learns to jointly contextualize extracted and generated knowledge by reasoning over both within a unified graph structure. Given the task input context and an extracted KG subgraph, HGN is trained to generate embeddings for the subgraph's missing edges to form a "hybrid" graph, then reason over the hybrid graph while filtering out context-irrelevant edges. We demonstrate HGN's effectiveness through considerable performance gains across four commonsense reasoning benchmarks, plus a user study on edge validness and helpfulness.

cs.CL↗
arXiv2021

TRACE: A Differentiable Approach to Line-level Stroke Recovery for Offline Handwritten Text

Stroke order and velocity are helpful features in the fields of signature verification, handwriting recognition, and handwriting synthesis. Recovering these features from offline handwritten text is a challenging and well-studied problem. We propose a new model called TRACE (Trajectory Recovery by an Adaptively-trained Convolutional Encoder). TRACE is a differentiable approach that uses a convolutional recurrent neural network (CRNN) to infer temporal stroke information from long lines of offline handwritten text with many characters and dynamic time warping (DTW) to align predictions and ground truth points. TRACE is perhaps the first system to be trained end-to-end on entire lines of text of arbitrary width and does not require the use of dynamic exemplars. Moreover, the system does not require images to undergo any pre-processing, nor do the predictions require any post-processing. Consequently, the recovered trajectory is differentiable and can be used as a loss function for other tasks, including synthesizing offline handwritten text. We demonstrate that temporal stroke information recovered by TRACE from offline data can be used for handwriting synthesis and establish the first benchmarks for a stroke trajectory recovery system trained on the IAM online handwriting dataset.

cs.CV↗
arXiv2021

Learning to Deceive Knowledge Graph Augmented Models via Targeted Perturbation

Knowledge graphs (KGs) have helped neural models improve performance on various knowledge-intensive tasks, like question answering and item recommendation. By using attention over the KG, such KG-augmented models can also "explain" which KG information was most relevant for making a given prediction. In this paper, we question whether these models are really behaving as we expect. We show that, through a reinforcement learning policy (or even simple heuristics), one can produce deceptively perturbed KGs, which maintain the downstream performance of the original KG while significantly deviating from the original KG's semantics and structure. Our findings raise doubts about KG-augmented models' ability to reason about KG information and give sensible explanations.

cs.CL↗
arXiv2020

On representation-finite gendo-symmetric algebras with only one non-injective projective module

Motivated by the relation between Schur algebra and the group algebra of a symmetric group, along with other similar examples in algebraic Lie theory, Min Fang and Steffen Koenig addressed some behaviour of the endomorphism algebra of a generator over a symmetric algebra, which they called gendo-symmetric algebra. Continuing this line of works, we classify in this article the representation-finite gendo-symmetric algebras that have at most one isomorphism class of indecomposable non-injective projective module. We also determine their almost ν-stable derived equivalence classes in the sense of Wei Hu and Changchang Xi. It turns out that a representative can be chosen as the quotient of a representation-finite symmetric algebra by the socle of a certain indecomposable projective module.

math.RT↗
arXiv2020

Classifying torsion classes of gentle algebras

For a finite-dimensional gentle algebra, it is already known that the functorially finite torsion classes of its category of finite-dimensional modules can be classified using a combinatorial interpretation, called maximal non-crossing sets of strings, of the corresponding support $τ$-tilting module (or equivalently, two-term silting complexes). In the topological interpretation of gentle algebras via marked surfaces, such a set can be interpreted as a dissection (or partial triangulation), or equivalently, a lamination that does not contain a closed curve. We will refine this combinatorics, which gives us a classification of torsion classes in the category of finite length modules over a (possibly infinite-dimensional) gentle algebra. As a consequence, our result also unifies the functorially finite torsion class classification of finite-dimensional gentle algebras with certain classes of special biserial algebras - such as Brauer graph algebras.

math.RT↗
arXiv2020

Irreducible representations of the symmetric groups from slash homologies of p-complexes

In the 40s, Mayer introduced a construction of (simplicial) $p$-complex by using the unsigned boundary map and taking coefficients of chains modulo $p$. We look at such a $p$-complex associated to an $(n-1)$-simplex; in which case, this is also a $p$-complex of representations of the symmetric group of rank $n$ - specifically, of permutation modules associated to two-row compositions. In this article, we calculate the so-called slash homology - a homology theory introduced by Khovanov and Qi - of such a $p$-complex. We show that every non-trivial slash homology group appears as an irreducible representation associated to two-row partitions, and how this calculation leads to a basis of these irreducible representations given by the so-called $p$-standard tableaux.

math.RT↗
arXiv2019

On simple-minded systems and $τ$-periodic modules of self-injective algebras

Let $A$ be a finite-dimensional self-injective algebra over an algebraically closed field, $\mathcal{C}$ a stably quasi-serial component (i.e. its stable part is a tube) of rank $n$ of the Auslander-Reiten quiver of $A$, and $\mathcal{S}$ be a simple-minded system of the stable module category $\stmod{A}$. We show that the intersection $\mathcal{S}\cap\mathcal{C}$ is of size strictly less than $n$, and consists only of modules with quasi-length strictly less than $n$. In particular, all modules in the homogeneous tubes of the Auslander-Reiten quiver of $A$ cannot be in any simple-minded system.

math.RT↗
arXiv2019

Coidempotent subcoalgebras and short exact sequences of finitary 2-representations

In this article, we study short exact sequences of finitary 2-representations of a weakly fiat 2-category. We provide a correspondence between such short exact sequences with fixed middle term and coidempotent subcoalgebras of a coalgebra 1-morphism defining this middle term. We additionally relate these to recollements of the underlying abelian 2-representations.

math.RT↗
arXiv2018

Egocentric Basketball Motion Planning from a Single First-Person Image

We present a model that uses a single first-person image to generate an egocentric basketball motion sequence in the form of a 12D camera configuration trajectory, which encodes a player's 3D location and 3D head orientation throughout the sequence. To do this, we first introduce a future convolutional neural network (CNN) that predicts an initial sequence of 12D camera configurations, aiming to capture how real players move during a one-on-one basketball game. We also introduce a goal verifier network, which is trained to verify that a given camera configuration is consistent with the final goals of real one-on-one basketball players. Next, we propose an inverse synthesis procedure to synthesize a refined sequence of 12D camera configurations that (1) sufficiently matches the initial configurations predicted by the future CNN, while (2) maximizing the output of the goal verifier network. Finally, by following the trajectory resulting from the refined camera configuration sequence, we obtain the complete 12D motion sequence. Our model generates realistic basketball motion sequences that capture the goals of real players, outperforming standard deep learning approaches such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), and generative adversarial networks (GANs).

cs.CV↗
arXiv2017

Auslander-Gorenstein algebras from Serre-formal algebras via replication

We introduce a new family of algebras, called Serre-formal algebras. They are Iwanaga-Gorenstein algebras for which applying any power of the Serre functor on any indecomposable projective module, the result remains a stalk complex. Typical examples are given by (higher) hereditary algebras and self-injective algebras; it turns out that other interesting algebras such as (higher) canonical algebras are also Serre-formal. Starting from a Serre-formal algebra, we consider a series of algebras - called the replicated algebras - given by certain subquotients of its repetitive algebra. We calculate the self-injective dimension and dominant dimension of all such replicated algebras and determine which of them are minimal Auslander-Gorenstein, i.e. when the two dimensions are finite and equal to each other. In particular, we show that there exist infinitely many minimal Auslander-Gorenstien algebras in such a series if, and only if, the Serre-formal algebra is twisted fractionally Calabi-Yau. We apply these results to a construction of algebras from Yamagata, called SGC extensions, given by iteratively taking the endomorphism ring of the smallest generator-cogenerator. We give a sufficient condition so that the SGC extensions and replicated algebras coincide. Consequently, in such a case, we obtain explicit formulae for the self-injective dimension and dominant dimension of the SGC extension algebras.

math.RT↗
arXiv2017

6-DoF Object Pose from Semantic Keypoints

This paper presents a novel approach to estimating the continuous six degree of freedom (6-DoF) pose (3D translation and rotation) of an object from a single RGB image. The approach combines semantic keypoints predicted by a convolutional network (convnet) with a deformable shape model. Unlike prior work, we are agnostic to whether the object is textured or textureless, as the convnet learns the optimal representation from the available training image data. Furthermore, the approach can be applied to instance- and class-based pose recovery. Empirically, we show that the proposed approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios with a cluttered background. For class-based object pose estimation, state-of-the-art accuracy is shown on the large-scale PASCAL3D+ dataset.

cs.CV↗