arXiv subjects

Pramod Khargonekar

Publications and source records attributed to Pramod Khargonekar.

9 recordsLinked to original sources

arXiv2026

CRUISE: Vision-Language Model-Guided Uncertainty-Aware Cross-Modal Sensor Fusion for Robust Autonomous Driving

Modern autonomous vehicles are equipped with multiple sensors, such as cameras, LiDAR, and radar, for comprehensive environmental perception. However, robust cross-modal feature fusion remains a critical challenge, as the reliability of each sensor varies significantly across diverse real-world driving conditions, including poor visibility and adverse weather. While uncertainty quantification (UQ) mitigates this issue by allowing models to prioritize reliable signals, existing uncertainty-aware fusion methods typically rely on simple feature-level uncertainty estimates and thus often fail to generalize effectively in complex, out-of-distribution scenarios. To address this limitation, we propose CRUISE, a novel uncertainty-aware cross-modal sensor fusion framework. CRUISE integrates a vision-language model (VLM)-guided UQ module that generates fine-grained, pixel-level uncertainty estimates. By leveraging the VLM's rich prior knowledge and superior contextual reasoning, our approach provides a highly informative guide for the fusion process. Furthermore, we introduce a dynamic adaptive mechanism that explicitly models and captures cross-modal dependencies, ensuring the framework fully exploits the inherent complementary nature of multi-sensor inputs.

cs.AI↗

arXiv2026

HydraCollab: Adaptive Collaborative-Perception for Distributed Autonomous Systems

Collaborative-perception enables multi-robot systems to enhance situational awareness by sharing perceptual information. Existing collaborative-perception systems face an inherent trade-off between communication bandwidth requirements and perception accuracy, where methods that exchange more information achieve better perception results at the cost of increased communication overhead. However, real-world communication networks impose bandwidth constraints that require minimizing communication overhead without sacrificing perception performance. To address this challenge, we propose HydraCollab, an adaptive collaborative-perception framework that (i) selectively transmits the most informative sensor features and (ii) dynamically employs collaboration strategies (intermediate or late) based on spatial confidence maps. Extensive evaluations on the V2X-R, V2X-Radar and UAV3D-mini datasets demonstrate that HydraCollab achieves the best overall trade-off between accuracy and communication cost among existing collaborative-perception methods. Relative to SOTA Where2comm, HydraCollab uses only 41% of the bandwidth on V2X-R and 26% on V2X-Radar while improving performance by 0.78% and 0.75% respectively. Our code and models are available at https://github.com/AICPS/HydraCollab.

cs.RO↗

arXiv2025

Hyperdimensional Uncertainty Quantification for Multimodal Uncertainty Fusion in Autonomous Vehicles Perception

Uncertainty Quantification (UQ) is crucial for ensuring the reliability of machine learning models deployed in real-world autonomous systems. However, existing approaches typically quantify task-level output prediction uncertainty without considering epistemic uncertainty at the multimodal feature fusion level, leading to sub-optimal outcomes. Additionally, popular uncertainty quantification methods, e.g., Bayesian approximations, remain challenging to deploy in practice due to high computational costs in training and inference. In this paper, we propose HyperDUM, a novel deterministic uncertainty method (DUM) that efficiently quantifies feature-level epistemic uncertainty by leveraging hyperdimensional computing. Our method captures the channel and spatial uncertainties through channel and patch -wise projection and bundling techniques respectively. Multimodal sensor features are then adaptively weighted to mitigate uncertainty propagation and improve feature fusion. Our evaluations show that HyperDUM on average outperforms the state-of-the-art (SOTA) algorithms by up to 2.01%/1.27% in 3D Object Detection and up to 1.29% improvement over baselines in semantic segmentation tasks under various types of uncertainties. Notably, HyperDUM requires 2.36x less Floating Point Operations and up to 38.30x less parameters than SOTA methods, providing an efficient solution for real-world autonomous systems.

cs.CV↗

arXiv2022

Mixed-Precision Neural Networks: A Survey

Mixed-precision Deep Neural Networks achieve the energy efficiency and throughput needed for hardware deployment, particularly when the resources are limited, without sacrificing accuracy. However, the optimal per-layer bit precision that preserves accuracy is not easily found, especially with the abundance of models, datasets, and quantization techniques that creates an enormous search space. In order to tackle this difficulty, a body of literature has emerged recently, and several frameworks that achieved promising accuracy results have been proposed. In this paper, we start by summarizing the quantization techniques used generally in literature. Then, we present a thorough survey of the mixed-precision frameworks, categorized according to their optimization techniques such as reinforcement learning and quantization techniques like deterministic rounding. Furthermore, the advantages and shortcomings of each framework are discussed, where we present a juxtaposition. We finally give guidelines for future mixed-precision frameworks.

cs.LG↗

arXiv2022

A Meta-Learning Control Algorithm with Provable Finite-Time Guarantees

In this work we provide provable regret guarantees for an online meta-learning control algorithm in an iterative control setting, where in each iteration the system to be controlled is a linear deterministic system that is different and unknown, the cost for the controller in an iteration is a general additive cost function and the control input is required to be constrained, which if violated incurs an additional cost. We prove (i) that the algorithm achieves a regret for the controller cost and constraint violation that are $O(T^{3/4})$ for an episode of duration $T$ with respect to the best policy that satisfies the control input control constraints and (ii) that the average of the regret for the controller cost and constraint violation with respect to the same policy vary as $O((1+\log(N)/N)T^{3/4})$ with the number of iterations $N$, showing that the worst regret for the learning within an iteration continuously improves with experience of more iterations.

cs.LG↗

arXiv2021

Generative Adversarial Imitation Learning for Empathy-based AI

Generative adversarial imitation learning (GAIL) is a model-free algorithm that has been shown to provide strong results in imitating complex behaviors in high-dimensional environments. In this paper, we utilize the GAIL model for text generation to develop empathy-based context-aware conversational AI. Our model uses an expert trajectory of empathetic prompt-response dialogues which can accurately exhibit the correct empathetic emotion when generating a response. The Generator of the GAIL model uses the GPT-2 sequential pre-trained language model trained on 117 million parameters from 40 GB of internet data. We propose a novel application of an approach used in transfer learning to fine tune the GPT-2 model in order to generate concise, user-specific empathetic responses validated against the Discriminator. Our novel GAIL model utilizes a sentiment analysis history-based reinforcement learning approach to empathetically respond to human interactions in a personalized manner. We find that our model's response scores on various human-generated prompts collected from the Facebook Empathetic Dialogues dataset outperform baseline counterparts. Moreover, our model improves upon various history-based conversational AI models developed recently, as our model's performance over a sustained conversation of 3 or more interactions outperform similar conversational AI models.

cs.CL↗

arXiv2019

Reduced-rank Analysis of the Total Least Squares

The reduced-rank method exploits the distortion-variance tradeoff to yield superior solutions for classic problems in statistical signal processing such as parameter estimation and filtering. The central idea is to reduce the variance of the solution at the expense of introducing a little distortion. In the context of parameter estimation, this yields an estimator whose sum of distortion plus variance is smaller than the variance of its undistorted counterpart. The method intrinsically results in an ordering mechanism for the singular vectors of the system matrix in the measurement model used for estimating the parameter of interest. According to this ordering rule, only a few \emph{dominant} singular vectors need to be selected to construct the reduced-rank solution while the rest can be discarded. The reduced-rank estimator is less sensitive to measurement errors. In this paper, we attempt to derive the reduced-rank estimator for the total least squares (TLS) problem, including the order selection rule. It will be shown that, due to the inherent structure of the problem, it is not possible to exploit the distortion-variance tradeoff in TLS formulations using existing techniques, except in some special cases. This result has not been reported previously and warrants a new viewpoint to achieve rank reduction for the TLS estimation problem. The work is motivated by the problems arising in practical applications such as channel estimation in wireless communication systems and time synchronization in wireless sensor networks.

cs.IT↗

arXiv2018

On the Marginal Value of Electricity Storage

We investigate the problem of characterizing the economic value of energy storage capacity to a wind power producer (WPP) that sells its energy in a conventional two-settlement electricity market. The WPP can offer a forward contract to supply power in the day-ahead market, subject to financial penalties for imbalances between the contracted power and the power that is delivered in real-time. We consider the setting in which the WPP has access to a co-located energy storage system, and can thus reshape its wind power production subject to its storage capacity constraints. Modeling wind power as a random process, we show that the problem of determining optimal forward contract offerings--given recourse with storage--is convex. We further establish that the maximum expected profit is concave and non-decreasing in the energy storage capacity, which reveals that the greatest marginal benefit from energy storage is derived from initial investment in small storage capacity. We provide a characterization of the marginal value of small energy storage capacity to the WPP. The formulae we derive shed light on the relationship between the value of storage and certain statistical measures of variability in the underlying wind power process.

math.OC↗

arXiv2016

An Approximation Algorithm for a Shortest Dubins Path Problem

The problem of finding the shortest path for a vehicle visiting a given sequence of target points subject to the motion constraints of the vehicle is an important problem that arises in several monitoring and surveillance applications involving unmanned aerial vehicles. There is currently no algorithm that can find an optimal solution to this problem. Therefore, heuristics that can find approximate solutions with guarantees on the quality of the solutions are useful. The best approximation algorithm currently available for the case when the distance between any two adjacent target points in the sequence is at least equal to twice the minimum radius of the vehicle has a guarantee of 3.04. This article provides a new approximation algorithm which improves this guarantee to 2.04. The developed algorithm is also implemented for hundreds of typical instances involving at most 30 points to corroborate the performance of the proposed approach.

cs.RO↗