arXiv subjects

Caroline Jay

Publications and source records attributed to Caroline Jay.

At least 19 recordsLinked to original sources

arXiv2026

Does social identity matter in software engineering? Assessing the case of research software engineers

Social identity is a concept from psychology that refers to the part of an individual's identity that derives from their group membership(s). In this paper, we explore social identity in members of the professional community of Research Software Engineers (RSEs). Using a mixed-methods approach, our study combined computational linguistic analysis and inferential statistics to examine over 28,000 social media posts, 1,700 blogs, and survey responses from 381 professional RSEs. The findings highlight the emergence of a collective RSE identity and demonstrate its role in shaping professional wellbeing. This study contributes an interdisciplinary perspective by integrating social psychology and software engineering to show how a professional identity evolves and why it matters.

cs.SE↗

arXiv2026

Comparing how Large Language Models perform against keyword-based searches for social science research data discovery

This paper evaluates the performance of a large language model (LLM) based semantic search tool relative to a traditional keyword-based search for data discovery. Using real-world search behaviour, we compare outputs from a bespoke semantic search system applied to UKRI data services with the Consumer Data Research Centre (CDRC) keyword search. Analysis is based on 131 of the most frequently used search terms extracted from CDRC search logs between December 2023 and October 2024. We assess differences in the volume, overlap, ranking, and relevance of returned datasets using descriptive statistics, qualitative inspection, and quantitative similarity measures, including exact dataset overlap, Jaccard similarity, and cosine similarity derived from BERT embeddings. Results show that the semantic search consistently returns a larger number of results than the keyword search and performs particularly well for place based, misspelled, obscure, or complex queries. While the semantic search does not capture all keyword based results, the datasets returned are overwhelmingly semantically similar, with high cosine similarity scores despite lower exact overlap. Rankings of the most relevant results differ substantially between tools, reflecting contrasting prioritisation strategies. Case studies demonstrate that the LLM based tool is robust to spelling errors, interprets geographic and contextual relevance effectively, and supports natural-language queries that keyword search fails to resolve. Overall, the findings suggest that LLM driven semantic search offers a substantial improvement for data discovery, complementing rather than fully replacing traditional keyword-based approaches.

cs.IR↗

arXiv2025

From keywords to semantics: Perceptions of large language models in data discovery

Current approaches to data discovery match keywords between metadata and queries. This matching requires researchers to know the exact wording that other researchers previously used, creating a challenging process that could lead to missing relevant data. Large Language Models (LLMs) could enhance data discovery by removing this requirement and allowing researchers to ask questions with natural language. However, we do not currently know if researchers would accept LLMs for data discovery. Using a human-centered artificial intelligence (HCAI) focus, we ran focus groups (N = 27) to understand researchers' perspectives towards LLMs for data discovery. Our conceptual model shows that the potential benefits are not enough for researchers to use LLMs instead of current technology. Barriers prevent researchers from fully accepting LLMs, but features around transparency could overcome them. Using our model will allow developers to incorporate features that result in an increased acceptance of LLMs for data discovery.

cs.HC↗

arXiv2025

The Impostor is Among Us: Can Large Language Models Capture the Complexity of Human Personas?

Large Language Models (LLMs) created new opportunities for generating personas, expected to streamline and accelerate the human-centered design process. Yet, AI-generated personas may not accurately represent actual user experiences, as they can miss contextual and emotional insights critical to understanding real users' needs and behaviors. This introduces a potential threat to quality, especially for novices. This paper examines the differences in how users perceive personas created by LLMs compared to those crafted by humans regarding their credibility for design. We gathered ten human-crafted personas developed by HCI experts according to relevant attributes established in related work. Then, we systematically generated ten personas with an LLM and compared them with human-crafted ones in a survey. The results showed that participants differentiated between human-created and AI-generated personas, with the latter perceived as more informative and consistent. However, participants noted that the AI-generated personas tended to follow stereotypes, highlighting the need for a greater emphasis on diversity when utilizing LLMs for persona creation.

cs.HC↗

arXiv2024

Contrasting Attitudes Towards Current and Future AI Applications for Computerised Interpretation of ECG: A Clinical Stakeholder Interview Study

Objectives: To investigate clinicians' attitudes towards current automated interpretation of ECG and novel AI technologies and their perception of computer-assisted interpretation. Materials and Methods: We conducted a series of interviews with clinicians in the UK. Our study: (i) explores the potential for AI, specifically future 'human-like' computing approaches, to facilitate ECG interpretation and support clinical decision making, and (ii) elicits their opinions about the importance of explainability and trustworthiness of AI algorithms. Results: We performed inductive thematic analysis on interview transcriptions from 23 clinicians and identified the following themes: (i) a lack of trust in current systems, (ii) positive attitudes towards future AI applications and requirements for these, (iii) the relationship between the accuracy and explainability of algorithms, and (iv) opinions on education, possible deskilling, and the impact of AI on clinical competencies. Discussion: Clinicians do not trust current computerised methods, but welcome future 'AI' technologies. Where clinicians trust future AI interpretation to be accurate, they are less concerned that it is explainable. They also preferred ECG interpretation that demonstrated the results of the algorithm visually. Whilst clinicians do not fear job losses, they are concerned about deskilling and the need to educate the workforce to use AI responsibly. Conclusion: Clinicians are positive about the future application of AI in clinical decision-making. Accuracy is a key factor of uptake and visualisations are preferred over current computerised methods. This is viewed as a potential means of training and upskilling, in contrast to the deskilling that automation might be perceived to bring.

cs.CY↗

arXiv2023

Individual context-free online community health indicators fail to identify open source software sustainability

The global value of open source software is estimated to be in the billions or trillions worldwide1, but despite this, it is often under-resourced and subject to high-impact security vulnerabilities and stability failures2,3. In order to investigate factors contributing to open source community longevity, we monitored thirty-eight open source projects over the period of a year, focusing primarily, but not exclusively, on open science-related online code-oriented communities. We measured performance indicators, using both subjective and qualitative measures (participant surveys), as well as using computational scripts to retrieve and analyse indicators associated with these projects' online source control codebases. None of the projects were abandoned during this period, and only one project entered a planned shutdown. Project ages spanned from under one year to over forty years old at the start of the study, and results were highly heterogeneous, showing little commonality across documentation, mean response times for issues and code contributions, and available funding/staffing resources. Whilst source code-based indicators were able to offer some insights into project activity, we observed that similar indicators across different projects often had very different meanings when context was taken into account. We conclude that the individual context-free metrics we studied were not sufficient or essential for project longevity and sustainability, and might even become detrimental if used to support high-stakes decision making. When attempting to understand an online open community's longer-term sustainability, we recommend that researchers avoid cross-project quantitative comparisons, and advise instead that they use single-project-level assessments which combine quantitative measures with contextualising qualitative data.

cs.SE↗

arXiv2023

Tracking the Structure and Sentiment of Vaccination Discussions on Mumsnet

Vaccination is one of the most impactful healthcare interventions in terms of lives saved at a given cost, leading the anti-vaccination movement to be identified as one of the top 10 threats to global health in 2019 by the World Health Organization. This issue increased in importance during the COVID-19 pandemic where, despite good overall adherence to vaccination, specific communities still showed high rates of refusal. Online social media has been identified as a breeding ground for anti-vaccination discussions. In this work, we study how vaccination discussions are conducted in the discussion forum of Mumsnet, a United Kingdom based website aimed at parents. By representing vaccination discussions as networks of social interactions, we can apply techniques from network analysis to characterize these discussions, namely network comparison, a task aimed at quantifying similarities and differences between networks. Using network comparison based on graphlets -- small connected network subgraphs -- we show how the topological structure vaccination discussions on Mumsnet differs over time, in particular before and after COVID-19. We also perform sentiment analysis on the content of the discussions and show how the sentiment towards vaccinations changes over time. Our results highlight an association between differences in network structure and changes to sentiment, demonstrating how network comparison can be used as a tool to guide and enhance the conclusions from sentiment analysis.

cs.SI↗

arXiv2023

Novice programmers strategies for online resource use and their impact on source code

Websites are frequently used by programmers to support the development process. This paper investigates programmer-Web interactions when coding, and combines observations of behaviour with assessments of the resulting source code. We report on an online observational study with ten undergraduate student programmers as they engaged in programming tasks of varying complexity. Screens were recorded of participants' activities, and each participated in an interview. Videos and interviews were thematically analysed. Novice programmers employed various strategies for seeking and utilising online knowledge. The resulting source code was examined to determine the extent to which it met requirements and whether it contained errors. The source code analysis revealed that coding with the websites involved more coding time and effort, but increased the possibility of producing correct code. However, coding with websites also introduced instances of either incorrect or non-executable source code.

cs.SE↗

arXiv2022

Subjective data models in bioinformatics: Do wet-lab and computational biologists comprehend data differently?

Biological science produces large amounts of data in a variety of formats, which necessitates the use of computational tools to process, integrate, analyse, and glean insights from the data. Researchers who use computational biology tools range from those who use computers primarily for communication and data lookup, to those who write complex software programs in order to analyse data or make it easier for others to do so. This research examines how people differ in how they conceptualise the same data, for which we coin the term "subjective data models". We interviewed 22 people with biological experience and varied levels of computational experience to elicit their perceptions of the same subset of biological data entities. The results suggest that many people had fluid subjective data models that would change depending on the circumstance or tool they were using. Surprisingly, results generally did not seem to cluster around a participant's computational experience/education levels, or the lack thereof. We further found that people did not consistently map entities from an abstract data model to the same identifiers in real-world files, and found that certain data identifier formats were easier for participants to infer meaning from than others. Real-world implications of these findings suggests that 1) software engineers should design interfaces for task performance and emulate other related popular user interfaces, rather than targeting a person's professional background; 2) when insufficient context is provided, people may guess what data means, whether or not their guesses are correct, emphasising the importance of providing contextual metadata when preparing data for re-use by other, to remove the need for potentially erroneous guesswork.

cs.HC↗

arXiv2022

Comparing directed networks via denoising graphlet distributions

Network comparison is a widely-used tool for analyzing complex systems, with applications in varied domains including comparison of protein interactions or highlighting changes in structure of trade networks. In recent years, a number of network comparison methodologies based on the distribution of graphlets (small connected network subgraphs) have been introduced. In particular, NetEmd has recently achieved state of the art performance in undirected networks. In this work, we propose an extension of NetEmd to directed networks and deal with the significant increase in complexity of graphlet structure in the directed case by denoising through linear projections. Simulation results show that our framework is able to improve on the performance of a simple translation of the undirected NetEmd algorithm to the directed case, especially when networks differ in size and density.

cs.SI↗

arXiv2022

The role of regular asymptomatic testing in reducing the impact of a COVID-19 wave

Testing for infection with SARS-CoV-2 is an important intervention in reducing onwards transmission of COVID-19, particularly when combined with the isolation and contact-tracing of positive cases. Many countries with the capacity to do so have made use of lab-processed Polymerase Chain Reaction (PCR) testing targeted at individuals with symptoms and the contacts of confirmed cases. Alternatively, Lateral Flow Tests (LFTs) are able to deliver a result quickly, without lab-processing and at a relatively low cost. Their adoption can support regular mass asymptomatic testing, allowing earlier detection of infection and isolation of infectious individuals. In this paper we extend and apply the agent-based epidemic modelling framework Covasim to explore the impact of regular asymptomatic testing on the peak and total number of infections in an emerging COVID-19 wave. We explore testing with LFTs at different frequency levels within a population with high levels of immunity and with background symptomatic PCR testing, case isolation and contact tracing for testing. The effectiveness of regular asymptomatic testing was compared with `lockdown' interventions seeking to reduce the number of non-household contacts across the whole population through measures such as mandating working from home and restrictions on gatherings. Since regular asymptomatic testing requires only those with a positive result to reduce contact, while lockdown measures require the whole population to reduce contact, any policy decision that seeks to trade off harms from infection against other harms will not automatically favour one over the other. Our results demonstrate that, where such a trade off is being made, at moderate rates of early exponential growth regular asymptomatic testing has the potential to achieve significant infection control without the wider harms associated with additional lockdown measures.

q-bio.PE↗

arXiv2022

COVID-19: An exploration of consecutive systemic barriers to pathogen-related data sharing during a pandemic

In 2020, the COVID-19 pandemic resulted in a rapid response from governments and researchers worldwide. As of late 2023, over millions have died as a result of COVID-19, with many COVID-19 survivors going on to experience long-term effects weeks, months, or years after their illness. Despite this staggering toll, those who work with pandemic-relevant data often face significant systemic barriers to accessing, sharing or re-using this data. In this paper we report results of a study, where we interviewed data professionals working with COVID-19-relevant data types including social media, mobility, viral genome, testing, infection, hospital admission, and deaths. These data types are variously used for pandemic spread modelling, healthcare system strain awareness, and devising therapeutic treatments for COVID-19. Barriers to data access, sharing and re-use include the cost of access to data (primarily certain healthcare sources and mobility data from mobile phone carriers), human throughput bottlenecks, unclear pathways to request access to data, unnecessarily strict access controls and data re-use policies, unclear data provenance, inability to link separate data sources that could collectively create a more complete picture, poor adherence to metadata standards, and a lack of computer-suitable data formats.

cs.CY↗

arXiv2021

Diversity of symptom phenotypes in SARS-CoV-2 community infections observed in multiple large datasets

Through the use of cutting-edge unsupervised classification techniques from statistics and machine learning, we characterise symptom phenotypes among symptomatic SARS-CoV-2 PCR-positive community cases. We first analyse each dataset in isolation and across age bands, before using methods that allow us to compare multiple datasets. While we observe separation due to the total number of symptoms experienced by cases, we also see a separation of symptoms into gastrointestinal, respiratory and other types, and different symptom co-occurrence patterns at the extremes of age. In this way, we are able to demonstrate the deep structure of symptoms of COVID-19 without usual biases due to study design. This is expected to have implications for the identification and management of community SARS-CoV-2 cases and could be further applied to symptom-based management of other diseases and syndromes.

stat.AP↗

arXiv2021

Using Interaction Data to Predict Engagement with Interactive Media

Media is evolving from traditional linear narratives to personalised experiences, where control over information (or how it is presented) is given to individual audience members. Measuring and understanding audience engagement with this media is important in at least two ways: (1) a post-hoc understanding of how engaged audiences are with the content will help production teams learn from experience and improve future productions; (2), this type of media has potential for real-time measures of engagement to be used to enhance the user experience by adapting content on-the-fly. Engagement is typically measured by asking samples of users to self-report, which is time consuming and expensive. In some domains, however, interaction data have been used to infer engagement. Fortuitously, the nature of interactive media facilitates a much richer set of interaction data than traditional media; our research aims to understand if these data can be used to infer audience engagement. In this paper, we report a study using data captured from audience interactions with an interactive TV show to model and predict engagement. We find that temporal metrics, including overall time spent on the experience and the interval between events, are predictive of engagement. The results demonstrate that interaction data can be used to infer users' engagement during and after an experience, and the proposed techniques are relevant to better understand audience preference and responses.

cs.HC↗

arXiv2021

Structuralist analysis for neural network system diagrams

This short paper examines diagrams describing neural network systems in academic conference proceedings. Many aspects of scholarly communication are controlled, particularly with relation to text and formatting, but often diagrams are not centrally curated beyond a peer review. Using a corpus-based approach, we argue that the heterogeneous diagrammatic notations used for neural network systems has implications for signification in this domain. We divide this into (i) what content is being represented and (ii) how relations are encoded. Using a novel structuralist framework, we use a corpus analysis to quantitatively cluster diagrams according to the author's representational choices. This quantitative diagram classification in a heterogeneous domain may provide a foundation for further analysis.

cs.HC↗

arXiv2021

Why scholars are diagramming neural network models

Complex models, such as neural networks (NNs), are comprised of many interrelated components. In order to represent these models, eliciting and characterising the relations between components is essential. Perhaps because of this, diagrams, as "icons of relation", are a prevalent medium for signifying complex models. Diagrams used to communicate NN architectures are currently extremely varied. The diversity in diagrammatic choices provides an opportunity to gain insight into the aspects which are being prioritised for communication. In this philosophical exploration of NN diagrams, we integrate theories of conceptual models, communication theory, and semiotics.

cs.HC↗

arXiv2021

Number and quality of diagrams in scholarly publications is associated with number of citations

Diagrams are often used in scholarly communication. We analyse a corpus of diagrams found in scholarly computational linguistics conference proceedings (ACL 2017), and find inclusion of a system diagram to be correlated with higher numbers of citations after 3 years. Inclusion of over three diagrams in this 8-page limit conference was found to correlate with a lower citation count. Focusing on neural network system diagrams, we find a correlation between highly cited papers and "good diagramming practice" quantified by level of compliance with a set of diagramming guidelines. Two diagram classification types (one visually based, one mental model based) were not found to correlate with number of citations, but enabled quantification of heterogeneity in those dimensions. Exploring scholarly paper-writing guides, we find diagrams to be a neglected media. This study suggests that diagrams may be a useful source of quality data for predicting citations, and that "graphicacy" is a key skill for scholars with insufficient support at present.

cs.DL↗

arXiv2021

Understanding Equity, Diversity and Inclusion Challenges Within the Research Software Community

Research software -- specialist software used to support or undertake research -- is of huge importance to researchers. It contributes to significant advances in the wider world and requires collaboration between people with diverse skills and backgrounds. Analysis of recent survey data provides evidence for a lack of diversity in the Research Software Engineer community. We identify interventions which could address challenges in the wider research software community and highlight areas where the community is becoming more diverse. There are also lessons that are applicable, more generally, to the field of software development around recruitment from other disciplines and the importance of welcoming communities.

cs.SE↗