Searcharxiv⌕ Search

arXiv subjects

Jianjun Ou

Publications and source records attributed to Jianjun Ou.

3 recordsLinked to original sources

Callo: Serving Cache-Miss Requests with Speculative Decoding on Invalidated KV under Prefill-Decode Disaggregation

Agent sessions may involve frequent context edits, which effectively invalidate cached KV state and force requests to wait for re-prefill before decoding. We present Callo, a disaggregated serving system that leverages invalidated KV for speculative drafts to reduce this latency, while verifying drafts against the exact KV after prefill before committing tokens. Callo makes effective use of decode resources by scheduling speculations during periods of underutilization and preempting speculations when drafts are likely to not be accepted. We evaluated Callo on a two-GPU testbed using SWE-agent sessions across three model families, observing all-request speedups of 1.05-1.19x over vLLM at moderate load. We demonstrate that these gains are also realized at scale through cluster-level simulations driven by real-world traces.

cs.DC↗

Trellis: Bringing Compiler-Guided Repair into Agentic Code Generation

Coding agents regularly generate programs that do not compile and require substantial tokens to repair. Yet most repairs are small and local to the edit that caused them. We present Trellis, a system that integrates incremental compiler feedback into code generation to improve code correctness while controlling repair costs. Trellis uses rollback-based search for targeted repair and asynchronous checking and checkpointing for efficient execution. We adapt Clang and tsc to Trellis's checker interface and integrate Trellis transparently into OpenCode. On C/C++ and TypeScript software engineering tasks, Trellis improves compilation rate by up to 13% and test pass rate by up to 11% with comparable token consumption to baseline OpenCode.

cs.SE↗

HAFFormer: A Hierarchical Attention-Free Framework for Alzheimer's Disease Detection From Spontaneous Speech

Automatically detecting Alzheimer's Disease (AD) from spontaneous speech plays an important role in its early diagnosis. Recent approaches highly rely on the Transformer architectures due to its efficiency in modelling long-range context dependencies. However, the quadratic increase in computational complexity associated with self-attention and the length of audio poses a challenge when deploying such models on edge devices. In this context, we construct a novel framework, namely Hierarchical Attention-Free Transformer (HAFFormer), to better deal with long speech for AD detection. Specifically, we employ an attention-free module of Multi-Scale Depthwise Convolution to replace the self-attention and thus avoid the expensive computation, and a GELU-based Gated Linear Unit to replace the feedforward layer, aiming to automatically filter out the redundant information. Moreover, we design a hierarchical structure to force it to learn a variety of information grains, from the frame level to the dialogue level. By conducting extensive experiments on the ADReSS-M dataset, the introduced HAFFormer can achieve competitive results (82.6% accuracy) with other recent work, but with significant computational complexity and model size reduction compared to the standard Transformer. This shows the efficiency of HAFFormer in dealing with long audio for AD detection.

cs.SD↗