arXiv subjects

Abhinav Palia

Publications and source records attributed to Abhinav Palia.

7 recordsLinked to original sources

arXiv2026

ASSERT: A Measurement Pipeline for GenAI Audits

Audits of generative AI (GenAI) systems often summarize behavior as a reported rate: how often the audited system complies with policy. Researchers and stakeholders use that rate to compare systems, track regressions, and gate deployment. A reported rate reflects both the system under audit and the measurement choices behind it, so a change in the rate can leave it unclear whether the system or those choices moved. We introduce ASSERT, a specification-driven measurement pipeline for GenAI audits that ties each reported rate to a written specification of the measurement choices used to produce it. ASSERT helps draft a behavioral rubric and test cases, then runs the audit against a GenAI system and returns a reported rate. In a case study on conversational deception, we observe that the reported rate moves substantially with the dialogue setup, the simulated user, the judge, and the evidence bar for non-compliance. These measurement choices substantially change the reported rate and can reorder GenAI system rankings. Because each reported rate is tied to an explicit specification, differences across audits are easier to attribute and interpret.

cs.CL↗

arXiv2026

Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming

We argue that conclusions drawn about relative system safety or attack method efficacy via AI red teaming are often not supported by evidence provided by attack success rate (ASR) comparisons. We show, through conceptual, theoretical, and empirical contributions, that many conclusions are founded on apples-to-oranges comparisons or low-validity measurements. Our arguments are grounded in asking a simple question: When can attack success rates be meaningfully compared? To answer this question, we draw on ideas from social science measurement theory and inferential statistics, which, taken together, provide a conceptual grounding for understanding when numerical values obtained through the quantification of system attributes can be meaningfully compared. Through this lens, we articulate conditions under which ASRs can and cannot be meaningfully compared. Using jailbreaking as a running example, we provide examples and extensive discussion of apples-to-oranges ASR comparisons and measurement validity challenges.

cs.LG↗

arXiv2022

A Trust Framework for Government Use of Artificial Intelligence and Automated Decision Making

This paper identifies the current challenges of the mechanisation, digitisation and automation of public sector systems and processes, and proposes a modern and practical framework to ensure and assure ethical and high veracity Artificial Intelligence (AI) or Automated Decision Making (ADM) systems in public institutions. This framework is designed for the specific context of the public sector, in the jurisdictional and constitutional context of Australia, but is extendable to other jurisdictions and private sectors. The goals of the framework are to: 1) earn public trust and grow public confidence in government systems; 2) to ensure the unique responsibilities and accountabilities (including to the public) of public institutions under Administrative Law are met effectively; and 3) to assure a positive human, societal and ethical impact from the adoption of such systems. The framework could be extended to assure positive environmental or other impacts, but this paper focuses on human/societal outcomes and public trust. This paper is meant to complement principles-based frameworks like Australia's Artificial Intelligence Ethics Framework and the EU Assessment List for Trustworthy AI. In many countries, COVID created a bubble of improved trust, a bubble which has arguably already popped, and in an era of unprecedented mistrust of public institutions (but even in times of high trust) it is not enough that a service is faster, or more cost-effective. This paper proposes recommendations for government systems (technology platforms, operations, culture, governance, engagement, etc.) that would help to improve public confidence and trust in public institutions, policies and services, whilst meeting the special obligations and responsibilities of the public sector.

cs.CY↗

arXiv2021

Utilizing Shannon's Entropy to Create Privacy Aware Architectures

Privacy is an individual choice to determine which personal details can be collected, used and shared. Individual consent and transparency are the core tenets for earning customers trust and this motivates the organizations to adopt privacy enhancing practices while creating the systems. The goal of a privacy-aware design is to protect information in a way that does not increase an adversary's existing knowledge about an individual beyond what is permissible. This becomes critical when these data elements can be linked with the wealth of auxiliary information available outside the system to identify an individual. Privacy regulations around the world provide directives to protect individual privacy but are generally complex and vague, making their translation into actionable and technical privacy-friendly architectures challenging. In this paper, we utilize Shannon's Entropy to create an objective metric that can help simplify the state-of-the-art Privacy Design Strategies proposed in the literature and aid our key technical design decisions to create privacy aware architectures.

cs.CR↗

arXiv2020

Program Controls Effectiveness Measurement Framework & Metrics

Any program that is designed to accomplish certain objectives, needs to establish program level controls pertaining to the overall goal. A critical aspect that determines the success of a program is the quality of the controls and their effectiveness in accomplishing the goal. Traditional Control Maturity Models primarily focus on the efficiency, management, and optimization of controls, while only indirectly measuring control effectiveness which neglects an essential aspect of control efficacy. In this paper, we highlight the ineffectiveness of these models, outline an adaptable program controls framework, and provide an approach to define measurable attributes (metrics) that enable a zero-defect program. To the best of our knowledge, we believe this is the first paper that provides a structured approach to defining a controls measurement framework and creation of effectiveness metrics that can be adopted by a variety of program use cases.

cs.SE↗

arXiv2018

A Scalable Permission Management System With Support of Conditional and Customized Attributes

Along with the classical problem of managing multiple identities, actions, devices, APIs etc. in different businesses, there has been an escalating need for having the capability of flexible attribute based access control~(ABAC) mechanisms. In order to fill this gap, several variations of ABAC model have been proposed such as \textit{Amazon's AWS IAM}, which uses JSON as their underlying storage data structure and adds policies/constraints as fields over the regular ABAC. However, these systems still do not provide the capability to have customized permissions and to perform various operations (such as comparison/aggregation) on them. In this paper, we introduce a string based resource naming strategy that supports the customized and conditional permissions for resource access. Further, we propose the basic architecture of our system which, along with our naming scheme, makes the system scalable, secure, efficient, flexible and customizable. Finally, we present the proof of concept for our algorithm as well as the experimental set up and the future trajectory for this work.

cs.CR↗

arXiv2017

Optimizing noise level for perturbing geo-location data

With the tremendous increase in the number of smart phones, app stores have been overwhelmed with applications requiring geo-location access in order to provide their users better services through personalization. Revealing a user's location to these third party apps, no matter at what frequency, is a severe privacy breach which can have unpleasant social consequences. In order to prevent inference attacks derived from geo-location data, a number of location obfuscation techniques have been proposed in the literature. However, none of them provides any objective measure of privacy guarantee. Some work has been done to define differential privacy for geo-location data in the form of geo-indistinguishability with l privacy guarantee. These techniques do not utilize any prior background information about the Points of Interest (PoIs) of a user and apply Laplacian noise to perturb all the location coordinates. Intuitively, the utility of such a mechanism can be improved if the noise distribution is derived after considering some prior information about PoIs. In this paper, we apply the standard definition of differential privacy on geo-location data. We use first principles to model various privacy and utility constraints, prior background information available about the PoIs (distribution of PoI locations in a 1D plane) and the granularity of the input required by different types of apps, in order to produce a more accurate and a utility maximizing differentially private algorithm for geo-location data at the OS level. We investigate this for a particular category of apps and for some specific scenarios. This will also help us to verify that whether Laplacian noise is still the optimal perturbation when we have such prior information.

cs.CR↗