arXiv subjects

Aniket Deshmukh

Publications and source records attributed to Aniket Deshmukh.

5 recordsLinked to original sources

arXiv2026

Optimal Design for Human Preference Elicitation

Learning of preference models from human feedback has been central to recent advances in artificial intelligence. Motivated by the cost of obtaining high-quality human annotations, we study efficient human preference elicitation for learning preference models. The key idea in our work is to generalize optimal designs, an approach to computing optimal information-gathering policies, to lists of items that represent potential questions with answers. The policy is a distribution over the lists and we elicit preferences from them proportionally to their probabilities. To show the generality of our ideas, we study both absolute and ranking feedback models on items in the list. We design efficient algorithms for both and analyze them. Finally, we demonstrate that our algorithms are practical by evaluating them on existing question-answering problems.

cs.LG↗

arXiv2026

Online Posterior Sampling with a Diffusion Prior

Posterior sampling in contextual bandits with a Gaussian prior can be implemented exactly or approximately using the Laplace approximation. The Gaussian prior is computationally efficient but it cannot describe complex distributions. In this work, we propose approximate posterior sampling algorithms for contextual bandits with a diffusion model prior. The key idea is to sample from a chain of approximate conditional posteriors, one for each stage of the reverse diffusion process, which are obtained by the Laplace approximation. Our approximations are motivated by posterior sampling with a Gaussian prior, and inherit its simplicity and efficiency. They are asymptotically consistent and perform well empirically on a variety of contextual bandit problems.

cs.LG↗

arXiv2024

Multi-Objective Alignment of Large Language Models Through Hypervolume Maximization

Multi-objective alignment from human feedback (MOAHF) in large language models (LLMs) is a challenging problem as human preferences are complex, multifaceted, and often conflicting. Recent works on MOAHF considered a-priori multi-objective optimization (MOO), where human preferences are known at training or inference time. In contrast, when human preferences are unknown or difficult to quantify, a natural approach is to cover the Pareto front by multiple diverse solutions. We propose an algorithm HaM for learning diverse LLM policies that maximizes their hypervolume. This is the first application of a-posteriori MOO to MOAHF. HaM is computationally and space efficient, and empirically superior across objectives such as harmlessness, helpfulness, humor, faithfulness, and hallucination, on various datasets.

cs.LG↗

arXiv2024

Experimental Design for Active Transductive Inference in Large Language Models

One emergent ability of large language models (LLMs) is that query-specific examples can be included in the prompt at inference time. In this work, we use active learning for adaptive prompt design and call it Active In-context Prompt Design (AIPD). We design the LLM prompt by adaptively choosing few-shot examples from a training set to optimize performance on a test set. The training examples are initially unlabeled and we obtain the label of the most informative ones, which maximally reduces uncertainty in the LLM prediction. We propose two algorithms, GO and SAL, which differ in how the few-shot examples are chosen. We analyze these algorithms in linear models: first GO and then use its equivalence with SAL. We experiment with many different tasks in small, medium-sized, and large language models; and show that GO and SAL outperform other methods for choosing few-shot examples in the LLM prompt at inference time.

cs.LG↗

arXiv2021

An efficient reverse-lookup table based strategy for solving the synonym and cache coherence problem in virtually indexed, virtually tagged caches

Virtually indexed and virtually tagged (VIVT) caches are an attractive option for micro-processor level-1 caches, because of their fast response time and because they are cheaper to implement than more complex caches such as virtually-indexed physical-tagged (VIPT) caches. The level-1 VIVT cache becomes even simpler to construct if it is implemented as a direct-mapped cache (VIVT-DM cache). However, VIVT and VIVT-DM caches have some drawbacks. When the number of sets in the cache is larger than the smallest page size, there is a possibility of synonyms (two or more virtual addresses mapped to the same physical address) existing in the cache. Further, maintenance of cache coherence across multiple processors requires a physical to virtual translation mechanism in the hardware. We describe a simple, efficient reverse lookup table based approach to address the synonym and the coherence problems in VIVT (both set associative and direct-mapped) caches. In particular, the proposed scheme does not disturb the critical memory access paths in a typical micro-processor, and requires a low overhead for its implementation. We have implemented and validated the scheme in the AJIT 32-bit microprocessor core (an implementation of the SPARC-V8 ISA) and the implementation uses approximately 2% of the gates and 5.3% of the memory bits in the processor core.

cs.AR↗