arXiv subjects

Vaclav Cvicek

Publications and source records attributed to Vaclav Cvicek.

5 recordsLinked to original sources

arXiv2025

SPIRe: Boosting LLM Inference Throughput with Speculative Decoding

Speculative decoding (SD) has been shown to reduce the latency of autoregressive decoding (AD) by 2-3x for small batch sizes. However, increasing throughput and therefore reducing the cost per token requires decoding with large batch sizes. Recent work shows that SD can accelerate decoding with large batch sizes too if the context is sufficiently long and the draft model's KV cache is sparse. We introduce SPIRe, a draft model that combines static sparse attention, pruned initialization, and feedback memory to increase the modeled throughput of speculative decoding by over 100% compared to speculation with a much smaller draft model and by over 35% compared to the strong baseline of sparse self-speculation. Our approach is particularly effective when context lengths vary significantly across requests.

cs.LG↗

arXiv2022

LogicInference: A New Dataset for Teaching Logical Inference to seq2seq Models

Machine learning models such as Transformers or LSTMs struggle with tasks that are compositional in nature such as those involving reasoning/inference. Although many datasets exist to evaluate compositional generalization, when it comes to evaluating inference abilities, options are more limited. This paper presents LogicInference, a new dataset to evaluate the ability of models to perform logical inference. The dataset focuses on inference using propositional logic and a small subset of first-order logic, represented both in semi-formal logical notation, as well as in natural language. We also report initial results using a collection of machine learning models to establish an initial baseline in this dataset.

cs.AI↗

arXiv2022

Making Transformers Solve Compositional Tasks

Several studies have reported the inability of Transformer models to generalize compositionally, a key type of generalization in many NLP tasks such as semantic parsing. In this paper we explore the design space of Transformer models showing that the inductive biases given to the model by several design decisions significantly impact compositional generalization. Through this exploration, we identified Transformer configurations that generalize compositionally significantly better than previously reported in the literature in a diverse set of compositional tasks, and that achieve state-of-the-art results in a semantic parsing compositional generalization benchmark (COGS), and a string edit operation composition benchmark (PCFG).

cs.AI↗

arXiv2020

ETC: Encoding Long and Structured Inputs in Transformers

Transformer models have advanced the state of the art in many Natural Language Processing (NLP) tasks. In this paper, we present a new Transformer architecture, Extended Transformer Construction (ETC), that addresses two key challenges of standard Transformer architectures, namely scaling input length and encoding structured inputs. To scale attention to longer inputs, we introduce a novel global-local attention mechanism between global tokens and regular input tokens. We also show that combining global-local attention with relative position encodings and a Contrastive Predictive Coding (CPC) pre-training objective allows ETC to encode structured inputs. We achieve state-of-the-art results on four natural language datasets requiring long and/or structured inputs.

cs.LG↗

arXiv2010

Tilt-Induced Anisotropic to Isotropic Phase Transition at $ν= 5/2$

A modest in-plane magnetic field \Bpar\ is sufficient to destroy the fractional quantized Hall states at $ν= 5/2$ and 7/2 and replace them with anisotropic compressible phases. Remarkably, we find that at larger \Bpar\ these anisotropic phases can themselves be replaced by isotropic compressible phases reminiscent of the composite fermion fluid at $ν= 1/2$. We present strong evidence that this transition is a consequence of the mixing of Landau levels from different electric subbands. We also report surprising dependences of the energy gaps at $ν= 5/2$ and 7/3 on the width of the confinement potential.

cond-mat.str-el↗