arXiv subjects

Rob Procter

Publications and source records attributed to Rob Procter.

At least 37 records · Page 2Linked to original sources

arXiv2022

Disentangled Learning of Stance and Aspect Topics for Vaccine Attitude Detection in Social Media

Building models to detect vaccine attitudes on social media is challenging because of the composite, often intricate aspects involved, and the limited availability of annotated data. Existing approaches have relied heavily on supervised training that requires abundant annotations and pre-defined aspect categories. Instead, with the aim of leveraging the large amount of unannotated data now available on vaccination, we propose a novel semi-supervised approach for vaccine attitude detection, called VADet. A variational autoencoding architecture based on language models is employed to learn from unlabelled data the topical information of the domain. Then, the model is fine-tuned with a few manually annotated examples of user attitudes. We validate the effectiveness of VADet on our annotated data and also on an existing vaccination corpus annotated with opinions on vaccines. Our results show that VADet is able to learn disentangled stance and aspect topics, and outperforms existing aspect-based sentiment analysis models on both stance detection and tweet clustering.

cs.CL↗

arXiv2022

Trust, Professional Vision and Diagnostic Work

In this paper we consider some empirical materials from our ongoing research into forms of everyday detection and diagnosis work in healthcare settings, and how these relate to issues of trust; trust in people, in technology, processes and in data.

cs.HC↗

arXiv2022

Natural Language Inference with Self-Attention for Veracity Assessment of Pandemic Claims

We present a comprehensive work on automated veracity assessment from dataset creation to developing novel methods based on Natural Language Inference (NLI), focusing on misinformation related to the COVID-19 pandemic. We first describe the construction of the novel PANACEA dataset consisting of heterogeneous claims on COVID-19 and their respective information sources. The dataset construction includes work on retrieval techniques and similarity measurements to ensure a unique set of claims. We then propose novel techniques for automated veracity assessment based on Natural Language Inference including graph convolutional networks and attention based approaches. We have carried out experiments on evidence retrieval and veracity assessment on the dataset using the proposed techniques and found them competitive with SOTA methods, and provided a detailed discussion.

cs.CL↗

arXiv2022

Using Computational Grounded Theory to Understand Tutors' Experiences in the Gig Economy

The introduction of online marketplace platforms has led to the advent of new forms of flexible, on-demand (or 'gig') work. Yet, most prior research concerning the experience of gig workers examines delivery or crowdsourcing platforms, while the experience of the large numbers of workers who undertake educational labour in the form of tutoring gigs remains understudied. To address this, we use a computational grounded theory approach to analyse tutors' discussions on Reddit. This approach consists of three phases including data exploration, modelling and human-centred interpretation. We use both validation and human evaluation to increase the trustworthiness and reliability of the computational methods. This paper is a work in progress and reports on the first of the three phases of this approach.

cs.HC↗

arXiv2021

Evaluating the application of NLP tools in mainstream participatory budgeting processes in Scotland

In recent years participatory budgeting (PB) in Scotland has grown from a handful of community-led processes to a movement supported by local and national government. This is epitomized by an agreement between the Scottish Government and the Convention of Scottish Local Authorities (COSLA) that at least 1% of local authority budgets will be subject to PB. This ongoing research paper explores the challenges that emerge from this 'scaling up' or 'mainstreaming' across the 32 local authorities that make up Scotland. The main objective is to evaluate local authority use of the digital platform Consul, which applies Natural Language Processing (NLP) to address these challenges. This project adopts a qualitative longitudinal design with interviews, observations of PB processes, and analysis of the digital platform data. Thematic analysis is employed to capture the major issues and themes which emerge. Longitudinal analysis then explores how these evolve over time. The potential for 32 live study sites provides a unique opportunity to explore discrete political and social contexts which materialize and allow for a deeper dive into the challenges and issues that may exist, something a wider cross-sectional study would miss. Initial results show that issues and challenges which come from scaling up may be tackled using NLP technology which, in a previous controlled use case-based evaluation, has shown to improve the effectiveness of citizen participation.

cs.CL↗

arXiv2021

Evaluation of Abstractive Summarisation Models with Machine Translation in Deliberative Processes

We present work on summarising deliberative processes for non-English languages. Unlike commonly studied datasets, such as news articles, this deliberation dataset reflects difficulties of combining multiple narratives, mostly of poor grammatical quality, in a single text. We report an extensive evaluation of a wide range of abstractive summarisation models in combination with an off-the-shelf machine translation model. Texts are translated into English, summarised, and translated back to the original language. We obtain promising results regarding the fluency, consistency and relevance of the summaries produced. Our approach is easy to implement for many languages for production purposes by simply changing the translation model.

cs.CL↗

arXiv2021

A mixed-methods ethnographic approach to participatory budgeting in Scotland

Participatory budgeting (PB) is already well established in Scotland in the form of community led grant-making yet has recently transformed from a grass-roots activity to a mainstream process or embedded 'policy instrument'. An integral part of this turn is the use of the Consul digital platform as the primary means of citizen participation. Using a mixed method approach, this ongoing research paper explores how each of the 32 local authorities that make up Scotland utilise the Consul platform to engage their citizens in the PB process and how they then make sense of citizens' contributions. In particular, we focus on whether natural language processing (NLP) tools can facilitate both citizen engagement, and the processes by which citizens' contributions are analysed and translated into policies.

cs.CL↗

arXiv2021

Machine Learning for Mediation in Armed Conflicts

Today's conflicts are becoming increasingly complex, fluid and fragmented, often involving a host of national and international actors with multiple and often divergent interests. This development poses significant challenges for conflict mediation, as mediators struggle to make sense of conflict dynamics, such as the range of conflict parties and the evolution of their political positions, the distinction between relevant and less relevant actors in peace making, or the identification of key conflict issues and their interdependence. International peace efforts appear increasingly ill-equipped to successfully address these challenges. While technology is being increasingly used in a range of conflict related fields, such as conflict predicting or information gathering, less attention has been given to how technology can contribute to conflict mediation. This case study is the first to apply state-of-the-art machine learning technologies to data from an ongoing mediation process. Using dialogue transcripts from peace negotiations in Yemen, this study shows how machine-learning tools can effectively support international mediators by managing knowledge and offering additional conflict analysis tools to assess complex information. Apart from illustrating the potential of machine learning tools in conflict mediation, the paper also emphasises the importance of interdisciplinary and participatory research design for the development of context-sensitive and targeted tools and to ensure meaningful and responsible implementation.

cs.CL↗

arXiv2021

Evaluation of Thematic Coherence in Microblogs

Collecting together microblogs representing opinions about the same topics within the same timeframe is useful to a number of different tasks and practitioners. A major question is how to evaluate the quality of such thematic clusters. Here we create a corpus of microblog clusters from three different domains and time windows and define the task of evaluating thematic coherence. We provide annotation guidelines and human annotations of thematic coherence by journalist experts. We subsequently investigate the efficacy of different automated evaluation metrics for the task. We consider a range of metrics including surface level metrics, ones for topic model coherence and text generation metrics (TGMs). While surface level metrics perform well, outperforming topic coherence metrics, they are not as consistent as TGMs. TGMs are more reliable than all other metrics considered for capturing thematic coherence in microblog clusters due to being less sensitive to the effect of time windows.

cs.CL↗

arXiv2021

A Query-Driven Topic Model

Topic modeling is an unsupervised method for revealing the hidden semantic structure of a corpus. It has been increasingly widely adopted as a tool in the social sciences, including political science, digital humanities and sociological research in general. One desirable property of topic models is to allow users to find topics describing a specific aspect of the corpus. A possible solution is to incorporate domain-specific knowledge into topic modeling, but this requires a specification from domain experts. We propose a novel query-driven topic model that allows users to specify a simple query in words or phrases and return query-related topics, thus avoiding tedious work from domain experts. Our proposed approach is particularly attractive when the user-specified query has a low occurrence in a text corpus, making it difficult for traditional topic models built on word cooccurrence patterns to identify relevant topics. Experimental results demonstrate the effectiveness of our model in comparison with both classical topic models and neural topic models.

cs.IR↗

arXiv2021

Citizen Participation and Machine Learning for a Better Democracy

The development of democratic systems is a crucial task as confirmed by its selection as one of the Millennium Sustainable Development Goals by the United Nations. In this article, we report on the progress of a project that aims to address barriers, one of which is information overload, to achieving effective direct citizen participation in democratic decision-making processes. The main objectives are to explore if the application of Natural Language Processing (NLP) and machine learning can improve citizens' experience of digital citizen participation platforms. Taking as a case study the "Decide Madrid" Consul platform, which enables citizens to post proposals for policies they would like to see adopted by the city council, we used NLP and machine learning to provide new ways to (a) suggest to citizens proposals they might wish to support; (b) group citizens by interests so that they can more easily interact with each other; (c) summarise comments posted in response to proposals; (d) assist citizens in aggregating and developing proposals. Evaluation of the results confirms that NLP and machine learning have a role to play in addressing some of the barriers users of platforms such as Consul currently experience.

cs.CL↗

arXiv2020

Accounts, Accountability and Agency for Safe and Ethical AI

We examine the problem of explainable AI (xAI) and explore what delivering xAI means in practice, particularly in contexts that involve formal or informal and ad-hoc collaboration where agency and accountability in decision-making are achieved and sustained interactionally. We use an example from an earlier study of collaborative decision-making in screening mammography and the difficulties users faced when trying to interpret the behavior of an AI tool to illustrate the challenges of delivering usable and effective xAI. We conclude by setting out a study programme for future research to help advance our understanding of xAI requirements for safe and ethical AI.

cs.HC↗

arXiv2020

Online platforms of public participation -- a deliberative democracy or a delusion?

Trust and confidence in democratic institutions is at an all-time low. At the same time, many of the complex issues faced by city administrators and politicians remain unresolved. To tackle these concerns, many argue that citizens should, through the use of digital platforms, have greater involvement in decision-making processes. This paper describes research into two such platforms, 'Decide Madrid' and 'Better Reykjavik'. Through the use of interviews, questionnaires, ethnographic observation, and analysis of platform data, the study will determine if these platforms provide greater participation or simply replicate what is already offered by numerous other digital tools. The findings so far suggest that to be successful platforms must take on a form of deliberative democracy, allowing for knowledge co-production and the emergence of collective intelligence. Based on this, we aim to identify key features of sustainable models of online participation.

cs.SI↗

arXiv2019

A Study of Cyber Hate on Twitter with Implications for Social Media Governance Strategies

This paper explores ways in which the harmful effects of cyber hate may be mitigated through mechanisms for enhancing the self governance of new digital spaces. We report findings from a mixed methods study of responses to cyber hate posts, which aimed to: (i) understand how people interact in this context by undertaking qualitative interaction analysis and developing a statistical model to explain the volume of responses to cyber hate posted to Twitter, and (ii) explore use of machine learning techniques to assist in identifying cyber hate counter-speech.

cs.CY↗

arXiv2018

A Longitudinal Analysis of the Public Perception of the Opportunities and Challenges of the Internet of Things

The Internet of Things (or IoT), which enables the networked interconnection of everyday objects, is becoming increasingly popular in many aspects of our lives ranging from entertainment to health care. While the IoT brings a set of invaluable advantages and opportunities with it, there is also evidence of numerous challenges that are yet to be resolved. This is certainly the case with regard to ensuring the cyber security of the IoT, and there are various examples of devices being hacked. Despite this evidence, little is known about the public perceptions of the opportunities and challenges presented by the IoT. To advance research in this direction, we mined the social media platform Twitter to learn about public opinion about the IoT. Analysing a longitudinal dataset of more than 6.7 million tweets, we reveal insights into public perceptions of the IoT, identifying big data analytics as the most positive aspect, whereas security issues are the main public concern on the negative side. Our study serves to highlight the importance of keeping IoT devices secure, and remind manufacturers that it is a concern that remains unresolved, at least insofar as the public believes.

cs.SI↗

arXiv2018

Detection and Resolution of Rumours in Social Media: A Survey

Despite the increasing use of social media platforms for information and news gathering, its unmoderated nature often leads to the emergence and spread of rumours, i.e. pieces of information that are unverified at the time of posting. At the same time, the openness of social media platforms provides opportunities to study how users share and discuss rumours, and to explore how natural language processing and data mining techniques may be used to find ways of determining their veracity. In this survey we introduce and discuss two types of rumours that circulate on social media; long-standing rumours that circulate for long periods of time, and newly-emerging rumours spawned during fast-paced events such as breaking news, where reports are released piecemeal and often with an unverified status in their early stages. We provide an overview of research into social media rumours with the ultimate goal of developing a rumour classification system that consists of four components: rumour detection, rumour tracking, rumour stance classification and rumour veracity classification. We delve into the approaches presented in the scientific literature for the development of each of these four components. We summarise the efforts and achievements so far towards the development of rumour classification systems and conclude with suggestions for avenues for future research in social media mining for detection and resolution of rumours.

cs.CL↗

arXiv2018

Political Homophily in Independence Movements: Analysing and Classifying Social Media Users by National Identity

Social media and data mining are increasingly being used to analyse political and societal issues. Here we undertake the classification of social media users as supporting or opposing ongoing independence movements in their territories. Independence movements occur in territories whose citizens have conflicting national identities; users with opposing national identities will then support or oppose the sense of being part of an independent nation that differs from the officially recognised country. We describe a methodology that relies on users' self-reported location to build large-scale datasets for three territories -- Catalonia, the Basque Country and Scotland. An analysis of these datasets shows that homophily plays an important role in determining who people connect with, as users predominantly choose to follow and interact with others from the same national identity. We show that a classifier relying on users' follow networks can achieve accurate, language-independent classification performances ranging from 85% to 97% for the three territories.

cs.CL↗

arXiv2017

Discourse-Aware Rumour Stance Classification in Social Media Using Sequential Classifiers

Rumour stance classification, defined as classifying the stance of specific social media posts into one of supporting, denying, querying or commenting on an earlier post, is becoming of increasing interest to researchers. While most previous work has focused on using individual tweets as classifier inputs, here we report on the performance of sequential classifiers that exploit the discourse features inherent in social media interactions or 'conversational threads'. Testing the effectiveness of four sequential classifiers -- Hawkes Processes, Linear-Chain Conditional Random Fields (Linear CRF), Tree-Structured Conditional Random Fields (Tree CRF) and Long Short Term Memory networks (LSTM) -- on eight datasets associated with breaking news stories, and looking at different types of local and contextual features, our work sheds new light on the development of accurate stance classifiers. We show that sequential classifiers that exploit the use of discourse properties in social media conversations while using only local features, outperform non-sequential classifiers. Furthermore, we show that LSTM using a reduced set of features can outperform the other sequential classifiers; this performance is consistent across datasets and across types of stances. To conclude, our work also analyses the different features under study, identifying those that best help characterise and distinguish between stances, such as supporting tweets being more likely to be accompanied by evidence than denying tweets. We also set forth a number of directions for future research.

cs.CL↗