arXiv subjects

Siddartha Khastgir

Publications and source records attributed to Siddartha Khastgir.

At least 19 recordsLinked to original sources

arXiv2026

Reusing Operational Evidence After Context Changes: A Conservative Bayesian Framework for Autonomous Vehicle Safety

Operational evidence, i.e., evidence of operation without failure is an important component of confidence in the safety or reliability of a system in service, but it is costly to collect. When the context of operation changes, the relevance of previously collected operational evidence becomes unclear. This problem arises, for example, when an autonomous vehicle that has operated safely in one operational environment (the Target Operational Domain, TOD) is deployed in a different but related TOD. Existing practice often treats such evidence in an all-or-nothing manner: either it is fully reused, or it is discarded. Neither position is satisfactory when there are good reasons to believe that the new context is no worse than the previous one, but that belief is itself uncertain. This paper studies how to make post-change reliability claims by combining evidence from two TODs using Conservative Bayesian Inference (CBI), which combines the evidence from the new context with a weighted amount of previous context and partial prior knowledge through constraints on a set of admissible priors. This yields conservative posterior bounds on quantities of interest. A numerical example illustrates how pre-existing evidence from a previous TOD can be transferred, conservatively and transparently, to support reliability claims in the changed TOD.

stat.ME↗

arXiv2026

Modeling Memory-Dependent Reliability of LLMs: A Hidden Markov Model

Reliability assessment of large language models (LLMs) seeks to estimate the probability that a model produces correct responses under a specified operational profile. Conventional benchmark-based evaluation, often summarized by aggregate accuracy, provides a point estimate of performance but does not characterize the uncertainty associated with reliability claims. Currently, statistical inference methods for LLM reliability assessment are emerging. However, a key assumption underlying these models is that test outcomes can be treated as independent repeated trials. This assumption may be inappropriate in sequential settings, where later responses depend on earlier interactions through retained context, error propagation, or an evolving interaction state. We extend a hierarchical Bayesian framework for LLM reliability assessment by relaxing the assumption of independent task outcomes and introducing a Hidden Markov Model to capture sequential dependence in benchmark-constructed interaction sessions. In this formulation, outcomes are generated from a latent interaction state evolving according to a first-order Markov process, capturing changes in interaction context. Through experiments using Anthropic Claude and OpenAI on four datasets, we demonstrate the potential impact of sequential dependence on reliability assessment. The results suggest that ignoring sequential dependence may lead to overconfident reliability estimates.

stat.ML↗

arXiv2026

Probabilistic Robustness in Medical Image Classification

Deep learning (DL) has shown strong performance in medical image classification, but its trustworthy deployment remains challenging in safety-critical clinical settings, where prediction errors under perturbations may lead to severe consequences. Existing studies mainly focus on adversarial robustness (AR) from a worst-case perspective; however, such settings may be less representative of real medical applications. In this work, we investigate probabilistic robustness (PR) as a more practical measure of model trustworthiness. To this end, we construct a set of natural corruption settings for medical image classification and systematically evaluate commonly used DL models on MedMNIST v2 dataset. Our study provides a statistically grounded perspective on assessing the trustworthiness of DL models, thereby supporting their more trustworthy deployment in medical imaging applications.

cs.CV↗

arXiv2026

Non-Parametric Probabilistic Robustness: A Conservative Risk Estimator under Unknown Perturbation Distributions

Deep learning (DL) models, despite their remarkable success, remain vulnerable to small input perturbations that can cause erroneous outputs, motivating the recent proposal of probabilistic robustness (PR) as a complementary alternative to adversarial robustness (AR). However, existing PR formulations assume a fixed and known perturbation distribution, an unrealistic expectation in practice. To address this limitation, we propose non-parametric probabilistic robustness (NPPR), a more practical PR metric that does not rely on any predefined perturbation distribution. Following the non-parametric paradigm in statistical modeling, NPPR learns an optimized perturbation distribution directly from data, enabling conservative PR evaluation under distributional uncertainty. We further develop an NPPR estimator based on a Gaussian Mixture Model (GMM), covering various input-dependent and input-independent perturbation scenarios. Theoretical analyses establish the relationships among AR, PR, and NPPR. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet across ResNet18/50, WideResNet50 and VGG16 validate NPPR as a more practical robustness metric, showing conservative (lower) PR estimates compared to assuming those common perturbation distributions used in state-of-the-arts.

cs.CV↗

arXiv2026

PRBench: A Standardized Probabilistic Robustness Benchmark

Deep learning models are notoriously vulnerable to imperceptible perturbations. Most existing research centers on adversarial robustness (AR), which evaluates models under worst-case scenarios by examining the existence of deterministic adversarial examples (AEs). In contrast, probabilistic robustness (PR) adopts a statistical perspective, measuring the probability that predictions remain correct under stochastic perturbations. While PR is widely regarded as a practical complement to AR, dedicated training methods for improving PR are still relatively underexplored, albeit with emerging progress. Among the few PR-targeted training methods, we identify three limitations: i non-comparable evaluation protocols; ii limited comparisons to strong AT baselines despite anecdotal PR gains from AT; and iii no unified framework to compare the generalization of these methods. Thus, we introduce PRBench, the first benchmark dedicated to evaluating improvements in PR achieved by different robustness training methods. PRBench empirically compares most common AT and PR-targeted training methods using a comprehensive set of metrics, including clean accuracy, PR and AR performance, training efficiency, and generalization error (GE). We also provide theoretical analysis on the GE of PR performance across different training methods. Main findings revealed by PRBench include: AT methods are more versatile than PR-targeted training methods in terms of improving both AR and PR performance across diverse hyperparameter settings, while PR-targeted training methods consistently yield lower GE and higher clean accuracy. A leaderboard comprising 229 trained models across 7 datasets and 10 model architectures is publicly available at https://wellzline.github.io/PRBenchLeaderboard/.

cs.CV↗

arXiv2026

Toward Safe and Energy-Efficient 5G NR V2X Communications in Rural Environments

Connected braking can reduce fatal collisions in connected and autonomous vehicles (CAVs) by using reliable, low-latency 5G New Radio (NR) links, especially NR Sidelink Vehicle-to-Everything (V2X). In rural areas, road side units are sparse and power-constrained, so energy efficiency must be considered alongside safety. This paper studies how three communication control factors including subcarrier spacing ($\mathrm{SCS}$), modulation and coding scheme ($\mathrm{MCS}$), and transmit power ($P_{\mathrm{t}}$) should be configured to balance safety and energy consumption in rural scenarios in light and heavy traffic scenarios. Safety is quantified by the packet receive ratio ($\mathrm{PRR}$) against the minimum communication distance $D_{\mathrm{comm}}$, defined as the distance that the vehicle travels during the transmission of the safety message. Results show that, under heavy traffic, increasing $P_{\mathrm{t}}$ and selecting a low-rate $\mathrm{MCS}$ at $\mathrm{SCS} = 30$ kHz sustains high $\mathrm{PRR}$ at $D_{\mathrm{comm}}$, albeit with higher energy cost. In light traffic, maintaining lower $P_\mathrm{t}$ with low $\mathrm{MCS}$ levels achieves a favorable reliability-energy trade-off while preserving acceptable $\mathrm{PRR}$ at $D_{\mathrm{comm}}$. These findings demonstrate the necessity of adaptive, energy-aware strategy to guarantee both safety and energy efficiency in rural V2X systems.

eess.SY↗

arXiv2026

Uncertainty-Aware Measurement of Scenario Suite Representativeness for Autonomous Systems

Assuring the trustworthiness and safety of AI systems, e.g., autonomous vehicles (AV), depends critically on the data-related safety properties, e.g., representativeness, completeness, etc., of the datasets used for their training and testing. Among these properties, this paper focuses on representativeness-the extent to which the scenario-based data used for training and testing, reflect the operational conditions that the system is designed to operate safely in, i.e., Operational Design Domain (ODD) or expected to encounter, i.e., Target Operational Domain (TOD). We propose a probabilistic method that quantifies representativeness by comparing the statistical distribution of features encoded by the scenario suites with the corresponding distribution of features representing the TOD, acknowledging that the true TOD distribution is unknown, as it can only be inferred from limited data. We apply an imprecise Bayesian method to handle limited data and uncertain priors. The imprecise Bayesian formulation produces interval-valued, uncertainty-aware estimates of representativeness, rather than a single value. We present a numerical example comparing the distributions of the scenario suite and the inferred TOD across operational categories-weather, road type, time of day, etc., under dependencies and prior uncertainty. We estimate representativeness locally (between categories) and globally as an interval.

cs.AI↗

arXiv2026

A Hierarchical Imprecise Probability Approach to Reliability Assessment of Large Language Models

Large Language Models (LLMs) are increasingly deployed across diverse domains, raising the need for rigorous reliability assessment methods. Existing benchmark-based evaluations primarily offer descriptive statistics of model accuracy over datasets, providing limited insight into the probabilistic behavior of LLMs under real operational conditions. This paper introduces HIP-LLM, a Hierarchical Imprecise Probability framework for modeling and inferring LLM reliability. Building upon the foundations of software reliability engineering, HIP-LLM defines LLM reliability as the probability of failure-free operation over a specified number of future tasks under a given Operational Profile (OP). HIP-LLM represents dependencies across (sub-)domains hierarchically, enabling multi-level inference from subdomain to system-level reliability. HIP-LLM embeds imprecise priors to capture epistemic uncertainty and incorporates OPs to reflect usage contexts. It derives posterior reliability envelopes that quantify uncertainty across priors and data. Experiments on multiple benchmark datasets demonstrate that HIP-LLM offers a more accurate and standardized reliability characterization than existing benchmark and state-of-the-art approaches. A publicly accessible repository of HIP-LLM is provided.

cs.SE↗

arXiv2026

Quantifying Fidelity: A Decisive Feature Approach to Comparing Synthetic and Real Imagery

Virtual testing using synthetic data has become a cornerstone of autonomous vehicle (AV) safety assurance. Despite progress in improving visual realism through advanced simulators and generative AI, recent studies reveal that pixel-level fidelity alone does not ensure reliable transfer from simulation to the real world. What truly matters is whether the system-under-test (SUT) bases its decisions on consistent decision evidence in both real and simulated environments, not just whether images "look real" to humans. To this end this paper proposes a behavior-grounded fidelity measure by introducing Decisive Feature Fidelity (DFF), a new SUT-specific metric that extends the existing fidelity spectrum to capture mechanism parity, that is, agreement in the model-specific decisive evidence that drives the SUT's decisions across domains. DFF leverages explainable-AI methods to identify and compare the decisive features driving the SUT's outputs for matched real-synthetic pairs. We further propose estimators based on counterfactual explanations, along with a DFF-guided calibration scheme to enhance simulator fidelity. Experiments on 2126 matched KITTI-VirtualKITTI2 pairs demonstrate that DFF reveals discrepancies overlooked by conventional output-value fidelity. Furthermore, results show that DFF-guided calibration improves decisive-feature and input-level fidelity without sacrificing output value fidelity across diverse SUTs.

cs.AI↗

arXiv2026

Introducing a Novel Systems Thinking approach inspired by STPA: Road Safety Intervention design case study

According to the latest provisional statistics released by the UK Department for Transport, Great Britain recorded 1,633 road deaths in 2024, representing a slight increase from 2023 and raising concerns about safety progress, which indicates that preventable fatalities remain a challenge. The deployment of advanced mobility systems, even certified and safety-assessed, is not sufficient to deliver improved safety outcomes, and existing road infrastructure is not sufficiently equipped to prevent severe collisions. Successful application of the ``Safe System'' approach demands systems thinking in an integrated and holistic manner, encompassing all aspects of road safety. This paper argues that road safety must be managed as a complex socio-technical system where risk evolves dynamically and must be continuously monitored. To address these safety gaps, we propose a systems thinking approach that identifies factors contributing to fatal outcomes and mitigates them. The framework consists of four steps: 1) List stakeholders who influence road safety, 2) Model the interactions between these stakeholders, 3) List assumptions that might be identified as factors for fatalities, and 4) Monitor these assumptions throughout the system lifecycle. The approach is applied to the United Kingdom (UK) road network to demonstrate feasibility. The study provides actionable guidance and new KPIs categories for stakeholders to implement road safety monitoring and eliminate any unreasonable road safety risks.

physics.soc-ph↗

arXiv2025

Safety Analysis of eVTOL Operations based on STPA

Electric Vertical Take-Off and Landing (eVTOL) aircraft are expected to be quieter and more cost-effective than helicopters, offering major economic and social benefits through improved connectivity. Their adoption will require new ground infrastructure and airspace redesign, introducing risks involving multiple stakeholders (Regulators, eVTOL operators, Air navigation service providers, Vertiport operators, OEMs, Pilots, etc.). To assess these risks for the UK airspace, systems-thinking based System Theoretic Process Analysis (STPA) was conducted. To manage the large number of Unsafe Control Actions (UCAs) and requirements generated due to the complexity of the analysis, a novel extension to STPA for the prioritization of results was applied. 317 UCAs were identified in total out of which 110 high-priority UCAs were analyzed (Step-4), resulting in 377 causal factors and 432 requirements. These were prioritized to produce a targeted list of 124 distinct high-priority requirements, 56 of which were identified as gaps in existing aviation regulations, policies, or procedures.. These highlight opportunities for regulatory updates in areas such as organizational performance, certification processes, training, collision avoidance, energy management, and automation. The findings provide regulators with safety considerations that could shape new or updated regulations, compliance methods, and guidance materials for the safe deployment of eVTOLs.

eess.SY↗

arXiv2025

A Scalable Framework for Safety Assurance of Self-Driving Vehicles based on Assurance 2.0

Assurance 2.0 is a modern framework developed to address the assurance challenges of increasingly complex, adaptive, and autonomous systems. Building on the traditional Claims-Argument-Evidence (CAE) model, it introduces reusable assurance theories and explicit counterarguments (defeaters) to enhance rigor, transparency, and adaptability. It supports continuous, incremental assurance, enabling innovation without compromising safety. However, limitations persist in confidence measurement, residual doubt management, automation support, and the practical handling of defeaters and confirmation bias. This paper presents \textcolor{black}{a set of decomposition frameworks to identify a complete set of safety arguments and measure their corresponding evidence.} Grounded in the Assurance 2.0 paradigm, the framework is instantiated through a structured template and employs a three-tiered decomposition strategy. \textcolor{black}{A case study regarding the application of the decomposition framework in the end-to-end (E2E) AI-based Self-Driving Vehicle (SDV) development is also presented in this paper.} At the top level, the SDV development is divided into three critical phases: Requirements Engineering (RE), Verification and Validation (VnV), and Post-Deployment (PD). Each phase is further decomposed according to its Product Development Lifecycle (PDLC). To ensure comprehensive coverage, each PDLC is analyzed using an adapted 5M1E model (Man, Machine, Method, Material, Measurement, and Environment). Originally developed for manufacturing quality control, the 5M1E model is reinterpreted and contextually mapped to the assurance domain. This enables a multi-dimensional decomposition that supports fine-grained traceability of safety claims, evidence, and potential defeaters.

cs.SE↗

arXiv2025

A Scalable Framework for the Management of STPA Requirements: a Case Study on eVTOL Operations

System-Theoretic Process Analysis (STPA) is a recommended method for analysing complex systems, capable of identifying thousands of safety requirements often missed by traditional techniques such as Failure Mode and Effects Analysis (FMEA) and Fault Tree Analysis (FTA). However, the absence of a structured framework for managing and prioritising these requirements presents challenges, particularly in fast-paced development environments. This paper introduces a scalable framework for prioritising STPA-derived requirements. The framework integrates outputs from each STPA step and incorporates expert evaluations based on four key factors: implementation time, cost, requirement type, and regulatory coverage. To reduce subjectivity, Monte-Carlo Simulation (MCS) is employed to calculate and stabilise requirement rankings. An automation toolchain supports the framework, enabling dynamic mapping of prioritised requirements in a scaling matrix. This visualisation aids decision-making and ensures traceability across development phases. The framework is applicable from early conceptualisation to more advanced stages, enhancing its utility in iterative system development. The framework was validated through a real-world case study focused on Electric Vertical Take-off and Landing (eVTOL) operations, conducted in collaboration with the UK Civil Aviation Authority. The findings contributed directly to CAP3141, a Civil Aviation Publication that identifies systemic operational risks and safety mitigations for regulators, operators, and vertiports. The prioritisation process supported decision-making by helping stakeholders identify and manage high-impact requirements efficiently. This work contributes a practical solution for managing STPA outputs, bridging gaps in requirement prioritisation and supporting safety-critical development in emerging technologies.

cs.SE↗

arXiv2025

Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Survey

Text-to-Image (T2I) Diffusion Models (DMs) have garnered widespread attention for their impressive advancements in image generation. However, their growing popularity has raised ethical and social concerns related to key non-functional properties of trustworthiness, such as robustness, fairness, security, privacy, factuality, and explainability, similar to those in traditional deep learning (DL) tasks. Conventional approaches for studying trustworthiness in DL tasks often fall short due to the unique characteristics of T2I DMs, e.g., the multi-modal nature. Given the challenge, recent efforts have been made to develop new methods for investigating trustworthiness in T2I DMs via various means, including falsification, enhancement, verification \& validation and assessment. However, there is a notable lack of in-depth analysis concerning those non-functional properties and means. In this survey, we provide a timely and focused review of the literature on trustworthy T2I DMs, covering a concise-structured taxonomy from the perspectives of property, means, benchmarks and applications. Our review begins with an introduction to essential preliminaries of T2I DMs, and then we summarise key definitions/metrics specific to T2I tasks and analyses the means proposed in recent literature based on these definitions/metrics. Additionally, we review benchmarks and domain applications of T2I DMs. Finally, we highlight the gaps in current research, discuss the limitations of existing methods, and propose future research directions to advance the development of trustworthy T2I DMs. Furthermore, we keep up-to-date updates in this field to track the latest developments and maintain our GitHub repository at: https://github.com/wellzline/Trustworthy_T2I_DMs

cs.LG↗

arXiv2025

Self-driving technologies need the help of the public: A narrative review of the evidence

If public trust is lost in a new technology early in its life cycle it can take much more time for the benefits of that technology to be realised. Eventually tens-of-millions of people will collectively have the power to determine self-driving technology success of failure driven by their perception of risk, data handling, safety, governance, accountability, benefits to their life and more. This paper reviews the evidence on safety critical technology covering trust, engagement, and acceptance. The paper takes a narrative review approach concluding with a scalable model for self-driving technology education and engagement. The paper find that if a mismatch between the publics perception and expectations about self driving systems emerge it can lead to misuse, disuse, or abuse of the system. Furthermore we find from the evidence that industrial experts often misunderstand what matters to the public, users, and stakeholders. However we find that engagement programmes that develop approaches to defining the right information at the right time, in the right format orientated around what matters to the public creates the potential for ever more sophisticated conversations, greater trust, and moving the public into a progressive more active role of critique and advocacy. This work has been undertaken as part of the Partners for Automated Vehicle Education (PAVE) United Kingdom programme.

cs.HC↗

arXiv2024

ProTIP: Probabilistic Robustness Verification on Text-to-Image Diffusion Models against Stochastic Perturbation

Text-to-Image (T2I) Diffusion Models (DMs) have shown impressive abilities in generating high-quality images based on simple text descriptions. However, as is common with many Deep Learning (DL) models, DMs are subject to a lack of robustness. While there are attempts to evaluate the robustness of T2I DMs as a binary or worst-case problem, they cannot answer how robust in general the model is whenever an adversarial example (AE) can be found. In this study, we first introduce a probabilistic notion of T2I DMs' robustness; and then establish an efficient framework, ProTIP, to evaluate it with statistical guarantees. The main challenges stem from: i) the high computational cost of the generation process; and ii) determining if a perturbed input is an AE involves comparing two output distributions, which is fundamentally harder compared to other DL tasks like classification where an AE is identified upon misprediction of labels. To tackle the challenges, we employ sequential analysis with efficacy and futility early stopping rules in the statistical testing for identifying AEs, and adaptive concentration inequalities to dynamically determine the "just-right" number of stochastic perturbations whenever the verification target is met. Empirical experiments validate the effectiveness and efficiency of ProTIP over common T2I DMs. Finally, we demonstrate an application of ProTIP to rank commonly used defence methods.

cs.CV↗

arXiv2023

Safety Analysis in the Era of Large Language Models: A Case Study of STPA using ChatGPT

Can safety analysis make use of Large Language Models (LLMs)? A case study explores Systems Theoretic Process Analysis (STPA) applied to Automatic Emergency Brake (AEB) and Electricity Demand Side Management (DSM) systems using ChatGPT. We investigate how collaboration schemes, input semantic complexity, and prompt guidelines influence STPA results. Comparative results show that using ChatGPT without human intervention may be inadequate due to reliability related issues, but with careful design, it may outperform human experts. No statistically significant differences are found when varying the input semantic complexity or using common prompt guidelines, which suggests the necessity for developing domain-specific prompt engineering. We also highlight future challenges, including concerns about LLM trustworthiness and the necessity for standardisation and regulation in this domain.

cs.CL↗

arXiv2023

STPA for Learning-Enabled Systems: A Survey and A New Practice

Systems Theoretic Process Analysis (STPA) is a systematic approach for hazard analysis that has been used across many industrial sectors including transportation, energy, and defense. The unstoppable trend of using Machine Learning (ML) in safety-critical systems has led to the pressing need of extending STPA to Learning-Enabled Systems (LESs). Although works have been carried out on various example LESs, without a systematic review, it is unclear how effective and generalisable the extended STPA methods are, and whether further improvements can be made. To this end, we present a systematic survey of 31 papers, summarising them from five perspectives (attributes of concern, objects under study, modifications, derivatives and processes being modelled). Furthermore, we identify room for improvement and accordingly introduce DeepSTPA, which enhances STPA from two aspects that are missing from the state-of-the-practice: (i) Control loop structures are explicitly extended to identify hazards from the data-driven development process spanning the ML lifecycle; (ii) Fine-grained functionalities are modelled at the layer-wise levels of ML models to detect root causes. We demonstrate and compare DeepSTPA and STPA through a case study on an autonomous emergency braking system.

cs.SE↗