arXiv subjects

Heather Mattie

Publications and source records attributed to Heather Mattie.

5 recordsLinked to original sources

arXiv2025

Red Teaming for Generative AI, Report on a Copyright-Focused Exercise Completed in an Academic Medical Center

Background: Generative artificial intelligence (AI) deployment in academic medical settings raises copyright compliance concerns. Dana-Farber Cancer Institute implemented GPT4DFCI, an internal generative AI tool utilizing OpenAI models, that is approved for enterprise use in research and operations. Given (1) the exceptionally broad adoption of the tool in our organization, (2) our research mission, and (3) the shared responsibility model required to benefit from Customer Copyright Commitment in Azure OpenAI Service products, we deemed rigorous copyright compliance testing necessary. Case Description: We conducted a structured red teaming exercise in Nov. 2024, with 42 participants from academic, industry, and government institutions. Four teams attempted to extract copyrighted content from GPT4DFCI across four domains: literary works, news articles, scientific publications, and access-restricted clinical notes. Teams successfully extracted verbatim book dedications and near-exact passages through various strategies. News article extraction failed despite jailbreak attempts. Scientific article reproduction yielded only high-level summaries. Clinical note testing revealed appropriate privacy safeguards. Discussion: The successful extraction of literary content indicates potential copyrighted material presence in training data, necessitating inference-time filtering. Differential success rates across content types suggest varying protective mechanisms. The event led to implementation of a copyright-specific meta-prompt in GPT4DFCI; this mitigation has been in production since Jan. 2025. Conclusion: Systematic red teaming revealed specific vulnerabilities in generative AI copyright compliance, leading to concrete mitigation strategies. Academic medical institutions deploying generative AI should implement continuous testing protocols to ensure legal and ethical compliance.

cs.CY↗

arXiv2022

Data science in public health: building next generation capacity

Rapidly evolving technology, data and analytic landscapes are permeating many fields and professions. In public health, the need for data science skills including data literacy is particularly prominent given both the potential of novel data types and analysis methods to fill gaps in existing public health research and intervention practices, as well as the potential of such data or methods to perpetuate or augment health disparities. Through a review of public health courses and programs at the top 10 U.S. and globally ranked schools of public health, this article summarizes existing educational efforts in public health data science. These existing practices serve to inform efforts for broadening such curricula to further schools and populations. Data science ethics course offerings are also examined in context of assessing how population health principles can be blended into training across levels of data involvement to augment the traditional core of public health curricula. Parallel findings from domestic and international 'outside the classroom' training programs are also synthesized to advance approaches for increasing diversity in public health data science. Based on these program reviews and their synthesis, a four-point formula is distilled for furthering public health data science education efforts, toward development of a critical and inclusive mass of practitioners with fluency to leverage data to advance goals of public health and improve quality of life in the digital age.

cs.CY↗

arXiv2020

Generalizations of Edge Overlap to Weighted and Directed Networks

With the increasing availability of behavioral data from diverse digital sources, such as social media sites and cell phones, it is now possible to obtain detailed information about the structure, strength, and directionality of social interactions in varied settings. While most metrics of network structure have traditionally been defined for unweighted and undirected networks only, the richness of current network data calls for extending these metrics to weighted and directed networks. One fundamental metric in social networks is edge overlap, the proportion of friends shared by two connected individuals. Here we extend definitions of edge overlap to weighted and directed networks, and present closed-form expressions for the mean and variance of each version for the Erdos-Renyi random graph and its weighted and directed counterparts. We apply these results to social network data collected in rural villages in southern Karnataka, India. We use our analytical results to quantify the extent to which the average overlap of the empirical social network deviates from that of corresponding random graphs and compare the values of overlap across networks. Our novel definitions allow the calculation of edge overlap for more complex networks and our derivations provide a statistically rigorous way for comparing edge overlap across networks.

stat.ME↗

arXiv2019

A Framework for Predicting Impactability of Healthcare Interventions Using Machine Learning Methods, Administrative Claims, Sociodemographic and App Generated Data

It is not clear how to target patients who are most likely to benefit from digital care management programs ex-ante, a shortcoming of current risk score based approaches. This study focuses on defining impactability by identifying those patients most likely to benefit from technology enabled care management, delivered through a digital health platform, including a mobile app and clinician web dashboard. Anonymized insurance claims data were used from a commercially insured population across several U.S. states and combined with inferred sociodemographic data and data derived from the patient-held mobile application itself. Our approach involves the creation of two models and the comparative analysis of the methodologies and performances therein. We first train a cost prediction model to calculate the differences in predicted (without intervention) versus actual (with onboarding onto digital health platform) healthcare expenditure for patients (N = 1,242). This enables the classification of impactability if differences in predicted versus actual costs meet a predetermined threshold. A random forest machine learning model was then trained to accurately categorize new patients as impactable versus not impactable, reaching an overall accuracy of 71.9%. We then modify these parameters through grid search to define the parameters that deliver optimal performance. A roadmap is proposed to iteratively improve the performance of the model. As the number of newly onboarded patients and length of use continues to increase, the accuracy of predicting impactability will improve commensurately as more advanced machine learning techniques such as deep learning become relevant. This approach is generalizable to analyzing the impactability of any intervention and is a key component of realising closed loop feedback systems for continuous improvement in healthcare.

q-bio.QM↗

arXiv2017

The Social Bow Tie

Understanding tie strength in social networks, and the factors that influence it, have received much attention in a myriad of disciplines for decades. Several models incorporating indicators of tie strength have been proposed and used to quantify relationships in social networks, and a standard set of structural network metrics have been applied to predominantly online social media sites to predict tie strength. Here, we introduce the concept of the "social bow tie" framework, a small subgraph of the network that consists of a collection of nodes and ties that surround a tie of interest, forming a topological structure that resembles a bow tie. We also define several intuitive and interpretable metrics that quantify properties of the bow tie. We use random forests and regression models to predict categorical and continuous measures of tie strength from different properties of the bow tie, including nodal attributes. We also investigate what aspects of the bow tie are most predictive of tie strength in two distinct social networks: a collection of 75 rural villages in India and a nationwide call network of European mobile phone users. Our results indicate several of the bow tie metrics are highly predictive of tie strength, and we find the more the social circles of two individuals overlap, the stronger their tie, consistent with previous findings. However, we also find that the more tightly-knit their non-overlapping social circles, the weaker the tie. This new finding complements our current understanding of what drives the strength of ties in social networks.

cs.SI↗