arXiv subjects

Reet Barik

Publications and source records attributed to Reet Barik.

3 recordsLinked to original sources

arXiv2025

PreLoRA: Hybrid Pre-training of Vision Transformers with Full Training and Low-Rank Adapters

Training large models ranging from millions to billions of parameters is highly resource-intensive, requiring significant time, compute, and memory. It is observed that most of the learning (higher change in weights) takes place in the earlier stage of the training loop. As training progresses, these changes stabilize, suggesting that the resulting updates may be amenable to approximation using low intrinsic-rank matrices. Therefore, we propose an approach to identify such states of partial convergence and dynamically switch from full parameter training to Low Rank Adaptation (LoRA) on the ViT-Large model. We introduce a flexible approach that leverages user-defined hyperparameters to determine the switching point and assign a rank specific to each module layer based on its level of convergence. Experimental results show that this approach preserves model accuracy while reducing the number of trainable parameters to 10% of its original size, resulting in a 3x improvement in throughput, and a 1.5x reduction in average training time per epoch while also reducing GPU memory consumption by 20%.

cs.LG↗

arXiv2024

An Integrated Epidemic Simulation Workflow for Submodular Intervention Strategies

Owing to the ongoing COVID-19 pandemic and other recent global epidemics, epidemic simulation frameworks are gaining rapid significance. In this work, we present a workflow that will allow researchers to simulate the spread of an infectious disease under different intervention schemes. Our workflow is built using the Covasim simulator for COVID-19 alongside a network-based PREEMPT tool for vaccination. The Covasim simulator is a stochastic agent-based simulator with the capacity to test the efficacy of different intervention schemes. PREEMPT is a graph-theoretic approach that models epidemic intervention on a network using submodular optimization. By integrating the PREEMPT tool with the Covasim simulator, users will be able to test network diffusion based interventions for vaccination. The paper presents a description of this integrated workflow alongside preliminary results of our empirical evaluation for COVID-19.

cs.SI↗

arXiv2024

GreediRIS: Scalable Influence Maximization using Distributed Streaming Maximum Cover

Influence maximization--the problem of identifying a subset of k influential seeds (vertices) in a network--is a classical problem in network science with numerous applications. The problem is NP-hard, but there exist efficient polynomial time approximations. However, scaling these algorithms still remain a daunting task due to the complexities associated with steps involving stochastic sampling and large-scale aggregations. In this paper, we present a new parallel distributed approximation algorithm for influence maximization with provable approximation guarantees. Our approach, which we call GreediRIS, leverages the RandGreedi framework--a state-of-the-art approach for distributed submodular optimization--for solving a step that computes a maximum k cover. GreediRIS combines distributed and streaming models of computations, along with pruning techniques, to effectively address the communication bottlenecks of the algorithm. Experimental results on up to 512 nodes (32K cores) of the NERSC Perlmutter supercomputer show that GreediRIS can achieve good strong scaling performance, preserve quality, and significantly outperform the other state-of-the-art distributed implementations. For instance, on 512 nodes, the most performant variant of GreediRIS achieves geometric mean speedups of 28.99x and 36.35x for two different diffusion models, over a state-of-the-art parallel implementation. We also present a communication-optimized version of GreediRIS that further improves the speedups by two orders of magnitude.

cs.DC↗