arXiv subjects

Nishant Sharma

Publications and source records attributed to Nishant Sharma.

4 recordsLinked to original sources

arXiv2025

Adaptive Blockwise Search: Inference-Time Alignment for Large Language Models

LLM alignment remains a critical challenge. Inference-time methods provide a flexible alternative to fine-tuning, but their uniform computational effort often yields suboptimal alignment. We hypothesize that for many alignment tasks, the initial tokens of a response are disproportionately more critical. To leverage this principle, we introduce AdaSearch, a novel blockwise search strategy. It adaptively allocates a fixed computational budget using a sampling schedule, focusing search effort on these critical tokens. We apply AdaSearch to sequential decoding and introduce its tree-search counterpart, AdaBeam. Our comprehensive evaluation across eight LLMs demonstrates that AdaSearch outperforms strong Best-of-N and fine-tuning baselines. Specifically, win-rates improve by over 10% for harmlessness generation, controlled sentiment generation, and for mathematical reasoning tasks relative to Best-of-N.

cs.CL↗

arXiv2025

Pretraining Large Language Models with NVFP4

Large Language Models (LLMs) today are powerful problem solvers across many domains, and they continue to get stronger as they scale in model size, training set size, and training set quality, as shown by extensive research and experimentation across the industry. Training a frontier model today requires on the order of tens to hundreds of yottaflops, which is a massive investment of time, compute, and energy. Improving pretraining efficiency is therefore essential to enable the next generation of even more capable LLMs. While 8-bit floating point (FP8) training is now widely adopted, transitioning to even narrower precision, such as 4-bit floating point (FP4), could unlock additional improvements in computational speed and resource utilization. However, quantization at this level poses challenges to training stability, convergence, and implementation, notably for large-scale models trained on long token horizons. In this study, we introduce a novel approach for stable and accurate training of large language models (LLMs) using the NVFP4 format. Our method integrates Random Hadamard transforms (RHT) to bound block-level outliers, employs a two-dimensional quantization scheme for consistent representations across both the forward and backward passes, utilizes stochastic rounding for unbiased gradient estimation, and incorporates selective high-precision layers. We validate our approach by training a 12-billion-parameter model on 10 trillion tokens -- the longest publicly documented training run in 4-bit precision to date. Our results show that the model trained with our NVFP4-based pretraining technique achieves training loss and downstream task accuracies comparable to an FP8 baseline. These findings highlight that NVFP4, when combined with our training approach, represents a major step forward in narrow-precision LLM training algorithms.

cs.CL↗

arXiv2023

Near 6 GHz Sezawa Mode Surface Acoustic Wave Resonators using AlScN on SiC

Surface Acoustic Wave (SAW) devices featuring Aluminum Scandium Nitride (AlScN) on a 4H-Silicon Carbide (SiC) substrate, offer a unique blend of high sound velocity, low thermal resistance, substantial piezoelectric response, simplified fabrication, as well as suitability for high-temperature and harsh environment operation. This study presents high-frequency SAW resonators employing AlScN thin films on SiC substrates, utilizing the second SAW mode (referred to as the Sezawa mode). The resonators achieve remarkable performance, boasting a K2 value of 5.5% and a maximum Q-factor (Qmax) of 1048 at 4.7 GHz, outperforming previous benchmarks. Additionally, a SAW resonator with a 960 nm wavelength attains 5.9 GHz frequency with record K2 (4.0%) and Qmax (887). Our study underscores the potential of the AlScN on SiC platform for advanced radio-frequency applications.

physics.app-ph↗

arXiv2023

Scalable and Stable Ferroelectric Non-Volatile Memory at > 500 $^\circ$C

Non-volatile memory (NVM) devices that reliably operate at temperatures above 300 $^\circ$C are currently non-existent and remains a critically unmet challenge in the development of high-temperature (T) resilient electronics, necessary for many emerging, complex computing and sensing in harsh environments. Ferroelectric Al$_x$Sc$_{1-x}$N exhibits strong potential for utilization in NVM devices operating at very high temperatures (> 500 $^\circ$C) given its stable and high remnant polarization (PR) above 100 $μ$C/cm$^2$ with demonstrated ferroelectric transition temperature (TC) > 1000 $^\circ$C. Here, we demonstrate an Al$_{0.68}$Sc$_{0.32}$N ferroelectric diode based NVM device that can reliably operate with clear ferroelectric switching up to 600 $^\circ$C with distinguishable On and Off states. The coercive field (EC) from the Pulsed I-V measurements is found to be -5.84 (EC-) and +5.98 (EC+) (+/- 0.1) MV/cm at room temperature (RT) and found to decrease with increasing temperature up to 600 $^\circ$C. The devices exhibit high remnant polarizations (> 100 $μ$C/cm$^2$) which are stable at high temperatures. At 500 $^\circ$C, our devices show 1 million read cycles and stable On-Off ratio above 1 for > 6 hours. Finally, the operating voltages of our AlScN ferrodiodes are < 15 V at 600 $^\circ$C which is well matched and compatible with Silicon Carbide (SiC) based high temperature logic technology, thereby making our demonstration a major step towards commercialization of NVM integrated high-T computers.

physics.app-ph↗