arXiv subjects

Akash Haridas

Publications and source records attributed to Akash Haridas.

7 recordsLinked to original sources

arXiv2026

Long-Context Aware Upcycling: A New Frontier for Hybrid LLM Scaling

Hybrid sequence models that combine efficient Transformer components with linear sequence modeling blocks are a promising alternative to pure Transformers, but most are still pretrained from scratch and therefore fail to reuse existing Transformer checkpoints. We study upcycling as a practical path to convert pretrained Transformer LLMs into hybrid architectures while preserving short-context quality and improving long-context capability. We call our solution \emph{HyLo} (HYbrid LOng-context): a long-context upcycling recipe that combines architectural adaptation with efficient Transformer blocks, Multi-Head Latent Attention (MLA), and linear blocks (Mamba2 or Gated DeltaNet), together with staged long-context training and teacher-guided distillation for stable optimization. HyLo extends usable context length by up to $32\times$ through efficient post-training and reduces KV-cache memory by more than $90\%$, enabling up to 2M-token prefill and decoding in our \texttt{vLLM} inference stack, while comparable Llama baselines run out of memory beyond 64K context. Across 1B- and 3B-scale settings (Llama- and Qwen-based variants), HyLo delivers consistently strong short- and long-context performance and significantly outperforms state-of-the-art upcycled hybrid baselines on long-context evaluations such as RULER. Notably, at similar scale, HyLo-Qwen-1.7B trained on only 10B tokens significantly outperforms JetNemotron (trained on 400B tokens) on GSM8K, Lm-Harness common sense reasoning and RULER-64K.

cs.CL↗

arXiv2026

DC-DiT: Adaptive Compute and Elastic Inference for Visual Generation via Dynamic Chunking

Diffusion Transformers rely on static patchify tokenization, assigning the same token budget to smooth backgrounds, detailed object regions, noisy early timesteps, and late-stage refinements. We introduce the Dynamic Chunking Diffusion Transformer (DC-DiT), which replaces fixed patchification with a learned encoder-router-decoder scaffold that adaptively compresses the 2D input into a shorter token sequence through a chunking mechanism learned end-to-end with diffusion training. DC-DiT allocates fewer tokens to predictable regions and noisy timesteps, and more tokens to detailed regions and later refinement stages, yielding meaningful spatial segmentations and timestep-adaptive compression schedules without supervision. Furthermore, the router provides an importance ordering over retained tokens, enabling elastic inference: a single checkpoint can be evaluated at flexible compute budgets with a smooth quality-compute tradeoff. Additionally, DC-DiT can be upcycled from pretrained DiT checkpoints and is also compatible with orthogonal dynamic computation approaches. On class-conditional ImageNet generation, DC-DiT reduces inference FLOPs by up to 36.8% and improves FID by up to 37.8% over DiT baselines, yielding a stronger quality--compute Pareto frontier across model scales, resolutions, and guidance settings. More broadly, these results suggest that adaptive tokenization is a general mechanism for making visual generation both more efficient and more flexible at inference time.

cs.CV↗

arXiv2025

Verifier Threshold: An Efficient Test-Time Scaling Approach for Image Generation

Image generation has emerged as a mainstream application of large generative models. Just as test-time compute and reasoning have improved language model capabilities, similar benefits have been observed for image generation models. In particular, searching over noise samples for diffusion and flow models has been shown to scale well with test-time compute. While recent works explore allocating non-uniform inference-compute budgets across denoising steps, existing approaches rely on greedy heuristics and often allocate the compute budget ineffectively. In this work, we study this problem and propose a simple fix. We propose Verifier-Threshold, which automatically reallocates test-time compute and delivers substantial efficiency improvements. For the same performance on the GenEval benchmark, we achieve a 2-4x reduction in computational time over the state-of-the-art method.

cs.CV↗

arXiv2025

High-Fidelity Human Avatars from Laptop Webcams using Edge Compute

Photo-realistic human avatars have broad applications, yet high-fidelity avatar generation has traditionally required expensive professional camera rigs and extensive artistic labor. Recent research has enabled constructing them automatically from smartphones with RGB and IR sensors, however, these new methods still rely on high-resolution cameras on modern smartphones and often require offloading the processing to powerful servers with GPUs. Modern applications such as video conferencing call for the ability to generate these avatars from consumer-grade laptop webcams using limited compute available on-device. In this work, we develop a novel method based on 3D morphable models, landmark detection, photorealistic texture GANs, and differentiable rendering to tackle the problem of low webcam image quality and edge computation. We build an automatic system to generate high-fidelity animatable avatars under these limitations, leveraging the compute capabilities of AMD mobile processors.

cs.CV↗

arXiv2023

Artificial Neural Networks and Guided Gene Expression Programming to Predict Wall Pressure Spectra Beneath Turbulent Boundary Layers

This study evaluates the efficacy of two machine learning (ML) techniques, namely artificial neural networks (ANN) and gene expression programming (GEP) that use data-driven modeling to predict wall pressure spectra (WPS) underneath turbulent boundary layers. Different datasets of WPS from experiments and high-fidelity numerical simulations covering a wide range of pressure gradients and Reynolds numbers are considered. For both ML methods, an optimal hyperparameter environment is identified that yields accurate predictions. ANN is observed to be faster and more accurate than GEP with an order of magnitude lower training time and logarithmic mean squared error ($lMSE$), despite a higher memory consumption. Novel training schemes are devised to address the shortcomings of GEP. These include (a) ANN-assisted GEP to reduce the noise in the training data, (b) exploiting the low and high-frequency trends to guide the GEP search, and (c) a stepped training strategy where the chromosomes are first trained on the canonical datasets followed by the datasets with complex features. When compared to the baseline scheme, these training strategies accelerated convergence and resulted in models with superior accuracy ($\approx 30\%$ reduction in the median $lMSE$) and higher reliability ($\approx 75\%$ reduction in the spread of $lMSE$ in the interquartile range). The final GEP models captured the complex trends of WPS across varying flow conditions and pressure gradients, surpassing the accuracy of Goody's model.

physics.flu-dyn↗

arXiv2023

DADAgger: Disagreement-Augmented Dataset Aggregation

DAgger is an imitation algorithm that aggregates its original datasets by querying the expert on all samples encountered during training. In order to reduce the number of samples queried, we propose a modification to DAgger, known as DADAgger, which only queries the expert for state-action pairs that are out of distribution (OOD). OOD states are identified by measuring the variance of the action predictions of an ensemble of models on each state, which we simulate using dropout. Testing on the Car Racing and Half Cheetah environments achieves comparable performance to DAgger but with reduced expert queries, and better performance than a random sampling baseline. We also show that our algorithm may be used to build efficient, well-balanced training datasets by running with no initial data and only querying the expert to resolve uncertainty.

cs.LG↗

arXiv2022

Deep Neural Networks to Correct Sub-Precision Errors in CFD

Information loss in numerical physics simulations can arise from various sources when solving discretized partial differential equations. In particular, errors related to numerical precision ("sub-precision errors") can accumulate in the quantities of interest when the simulations are performed using low-precision 16-bit floating-point arithmetic compared to an equivalent 64-bit simulation. On the other hand, low-precision computation is less resource intensive than high-precision computation. Several machine learning techniques proposed recently have been successful in correcting errors due to coarse spatial discretization. In this work, we extend these techniques to improve CFD simulations performed with low numerical precision. We quantify the precision-related errors accumulated in a Kolmogorov forced turbulence test case. Subsequently, we employ a Convolutional Neural Network together with a fully differentiable numerical solver performing 16-bit arithmetic to learn a tightly-coupled ML-CFD hybrid solver. Compared to the 16-bit solver, we demonstrate the efficacy of the hybrid solver towards improving various metrics pertaining to the statistical and pointwise accuracy of the simulation.

physics.flu-dyn↗