Hemanth Reddy Samidi | Searcharxiv

arXiv2023

GenAIPABench: A Benchmark for Generative AI-based Privacy Assistants

Privacy policies of websites are often lengthy and intricate. Privacy assistants assist in simplifying policies and making them more accessible and user friendly. The emergence of generative AI (genAI) offers new opportunities to build privacy assistants that can answer users questions about privacy policies. However, genAIs reliability is a concern due to its potential for producing inaccurate information. This study introduces GenAIPABench, a benchmark for evaluating Generative AI-based Privacy Assistants (GenAIPAs). GenAIPABench includes: 1) A set of questions about privacy policies and data protection regulations, with annotated answers for various organizations and regulations; 2) Metrics to assess the accuracy, relevance, and consistency of responses; and 3) A tool for generating prompts to introduce privacy documents and varied privacy questions to test system robustness. We evaluated three leading genAI systems ChatGPT-4, Bard, and Bing AI using GenAIPABench to gauge their effectiveness as GenAIPAs. Our results demonstrate significant promise in genAI capabilities in the privacy domain while also highlighting challenges in managing complex queries, ensuring consistency, and verifying source accuracy.

cs.CR↗

arXiv2023

A Study of the Landscape of Privacy Policies of Smart Devices

As the adoption of smart devices continues to permeate all aspects of our lives, user privacy concerns have become more pertinent than ever. Privacy policies outline the data handling practices of these devices. Prior work in the domains of websites and mobile apps has shown that privacy policies are rarely read and understood by users. In these domains, automatic analysis of privacy policies has been shown to help give users appropriate insights. However, there is a lack of such an analysis in the domain of smart device privacy policies. This paper presents a comprehensive study of the landscape of privacy policies of smart devices. We introduce a methodology that addresses the unique challenges of smart devices, by finding information about them, their manufacturers, and their privacy policies on the Web. Our methodology utilizes state-of-the-art analysis techniques to assess readability and privacy of smart device policies and compares it policies of e-commerce websites and mobile applications. Overall, we analyzed 4,556 smart devices, 2,211 manufacturers, and 819 privacy policies. Despite smart devices having access to more intrusive data about their users (using sensors such as cameras and microphones), more than 1,167 of the analyzed manufacturers did not have policies available. The study highlights that significant improvement is required on communicating the data management practices of smart devices.

cs.CY↗