Searcharxiv⌕ Search

arXiv subjects

Tolunay Yilmaz

Publications and source records attributed to Tolunay Yilmaz.

3 recordsLinked to original sources

A Computational Brainsphere Model for the Simulation of Alzheimer's Disease

Nowadays, there are different medical imaging techniques to collect data about the progression of Alzhei\-mer's disease in a patient's brain. These data describe different phenomena which are still not understood from a biological point of view. How can these data sets be combined in a mathematical model to simulate the evolution of such a neurodegenerative disease in a computer? In this snapshot, we present one possible approach to address this task with the help of graph theory and partial differential equations.

math.NA↗

LAMP: Look-Ahead Mixed-Precision Inference of Large Language Models

Mixed-precision computations are a hallmark of the current stage of AI, driving the progress in large language models towards efficient, locally deployable solutions. This article addresses the floating-point computation of compositionally-rich functions, concentrating on transformer inference. Based on the rounding error analysis of a composition $f(g(\mathrm{x}))$, we provide an adaptive strategy that selects a small subset of components of $g(\mathrm{x})$ to be computed more accurately while all other computations can be carried out with lower accuracy. We then explain how this strategy can be applied to different compositions within a transformer and illustrate its overall effect on transformer inference. We study the effectiveness of this algorithm numerically on GPT-2 models and demonstrate that already very low recomputation rates allow for improvements of up to two orders of magnitude in accuracy.

cs.LG↗

LampAttention: Look-Ahead Mixed-Precision FlashAttention for Dedicated Accelerators

While most attention logits can be computed in low precision without degrading numerical stability, current attention kernels fail to exploit this phenomenon. We introduce a novel hardware-algorithm co-design in the form of mixed-precision FlashAttention. Our method accumulates key-query products and evaluates their exponentials in 8-bit formats, then adaptively identifies sensitive sub-blocks and recomputes them in 16-bit formats. We propose the specifications for a dedicated accelerator capable of executing this pipeline efficiently. Simulated experiments with Qwen3 and Gemma 3 show that rerouting a selective minority of sub-blocks to high precision is sufficient to recover the baseline model performance.

cs.LG↗