Searcharxiv⌕ Search

arXiv subjects

Koushik Kumar Dey

Publications and source records attributed to Koushik Kumar Dey.

2 recordsLinked to original sources

VelServe: Taming Decode Jitters in LLM Inference with Tail-Calibrated Scheduling

Interactive LLM services must meet latency service-level objectives (SLOs) for the time to first token (TTFT) and the time between tokens (TBT), as streaming and soft real-time applications require predictable token delivery. We identify decode jitters in both aggregated and disaggregated serving. A jitter delays every request in the batch, and long responses span hundreds of steps, so even rare overruns cause many SLO violations. In aggregated serving, SLO-aware schedulers set each step's prefill budget until its predicted latency just meets the TBT target, leaving no room for mispredictions or host stalls. Disaggregation meets average TBT targets at a cost in capacity, yet the steps that admit new requests still run late. We present \sysname, a scheduler that calibrates how much prefill work each decode step carries using the measured tail of its own iteration latency. The SLO determines how rare a late step must be, and \sysname leaves enough headroom in each iteration to prevent overruns and meet the target decode latency. This headroom slows prefill, so \sysname serves the shortest waiting prompt first. When TTFT deadlines grow with prompt length, the long prompts it delays have the most time to spare. Under disaggregation, it spaces admissions so that no request waits on more slow admission steps than its P99 target allows. On three attention families and varying workloads, \sysname sustains 2.4--2.9$\times$ the SLO goodput of the strongest baseline on Qwen3-8B and DeepSeek-V4-Flash, and at least 4.4$\times$ on DeepSeek-V2-Lite. With prefill and decode disaggregated on eight GPUs, \sysname meets the SLO for more than twice as many requests as state-of-the-art baselines.

cs.AI↗

Counting the minimum number of arcs in an oriented graph having weak diameter 2

An oriented graph has weak diameter at most $d$ if every non-adjacent pair of vertices are connected by a directed $d$-path. The function $f_d(n)$ denotes the minimum number of arcs in an oriented graph on $n$ vertices having weak diameter $d$. Finding the exact value of $f_d(n)$ is a challenging problem even for $d = 2$. This function was introduced by Katona and Szemeredi (1967), and after that several attempts were made to find its exact value by Znam (1970), Dawes and Meijer (1987), Furedi, Horak, Pareek and Zhu (1998), and Kostochka, Luczak, Simonyi and Sopena (1999) through improving its best known bounds. In that process, it was proved that this function is asymptotically equal to $n\log_2 n$ and hence, is an asymptotically increasing function. However, the exact value and behaviour of this function was not known. In this article, we observe that the oriented graphs with weak diameter at most $2$ are precisely the absolute oriented cliques, that is, analogues of cliques for oriented graphs in the context of oriented coloring. Through studying arc-minimal absolute oriented cliques we prove that $f_2(n)$ is a strictly increasing function. Furthermore, we improve the best known upper bound of $f_2(n)$ and conjecture that our upper bound is tight. This improvement of the upper bound improves known bounds involving the oriented achromatic number.

cs.DM↗