SearcharxivSearch

arXiv subjects

Marcos Fernández-Pichel

Publications and source records attributed to Marcos Fernández-Pichel.

4 recordsLinked to original sources

Overview of ROMCIR 2026: The 6th Workshop on Reducing Online Misinformation through Credible Information Retrieval

In the digital online ecosystem, we are surrounded by distinct forms of information pollution, posing significant threats to both individuals and society. Fake news, for instance, wields power to sway public opinion on matters of politics and finance. Deceptive reviews can either bolster or tarnish the reputation of businesses, while unverified medical advice may steer people toward harmful health practices. In light of this challenging landscape, it has become imperative to ensure that users have access to both topically relevant and factually accurate information that does not warp their perception of reality, and there has been a surge of interest in various strategies to combat misinformation through different contexts and multiple tasks. The purpose of the ROMCIR Workshop, for some years now, is precisely that of engaging the Information Retrieval community to explore potential solutions that extend beyond conventional misinformation detection approaches. Key objectives include identifying subjective and objective factors associated with information credibility and truthfulness, respectively, and integrating such factors as fundamental dimensions of relevance within IR Systems (IRSs), achieving early detection of misinformation, and ensuring that the search results retrieved are not only truthful but also explainable to the users of IRSs. Moreover, it is essential to evaluate the role of generative models such as Large Language Models (LLMs) in inadvertently amplifying misinformation problems, and how they can be used to support IRSs, together with the contribution that the human-in-the-loop paradigm can have in this context.

cs.IR

ChatGPT as a Solver and Grader of Programming Exams written in Spanish

Evaluating the capabilities of Large Language Models (LLMs) to assist teachers and students in educational tasks is receiving increasing attention. In this paper, we assess ChatGPT's capacities to solve and grade real programming exams, from an accredited BSc degree in Computer Science, written in Spanish. Our findings suggest that this AI model is only effective for solving simple coding tasks. Its proficiency in tackling complex problems or evaluating solutions authored by others are far from effective. As part of this research, we also release a new corpus of programming tasks and the corresponding prompts for solving the problems or grading the solutions. This resource can be further exploited by other research teams.

cs.AI

Evaluating Search Engines and Large Language Models for Answering Health Questions

Search engines (SEs) have traditionally been primary tools for information seeking, but the new Large Language Models (LLMs) are emerging as powerful alternatives, particularly for question-answering tasks. This study compares the performance of four popular SEs, seven LLMs, and retrieval-augmented (RAG) variants in answering 150 health-related questions from the TREC Health Misinformation (HM) Track. Results reveal SEs correctly answer between 50 and 70% of questions, often hindered by many retrieval results not responding to the health question. LLMs deliver higher accuracy, correctly answering about 80% of questions, though their performance is sensitive to input prompts. RAG methods significantly enhance smaller LLMs' effectiveness, improving accuracy by up to 30% by integrating retrieval evidence.

cs.IR

DepreSym: A Depression Symptom Annotated Corpus and the Role of LLMs as Assessors of Psychological Markers

Computational methods for depression detection aim to mine traces of depression from online publications posted by Internet users. However, solutions trained on existing collections exhibit limited generalisation and interpretability. To tackle these issues, recent studies have shown that identifying depressive symptoms can lead to more robust models. The eRisk initiative fosters research on this area and has recently proposed a new ranking task focused on developing search methods to find sentences related to depressive symptoms. This search challenge relies on the symptoms specified by the Beck Depression Inventory-II (BDI-II), a questionnaire widely used in clinical practice. Based on the participant systems' results, we present the DepreSym dataset, consisting of 21580 sentences annotated according to their relevance to the 21 BDI-II symptoms. The labelled sentences come from a pool of diverse ranking methods, and the final dataset serves as a valuable resource for advancing the development of models that incorporate depressive markers such as clinical symptoms. Due to the complex nature of this relevance annotation, we designed a robust assessment methodology carried out by three expert assessors (including an expert psychologist). Additionally, we explore here the feasibility of employing recent Large Language Models (ChatGPT and GPT4) as potential assessors in this complex task. We undertake a comprehensive examination of their performance, determine their main limitations and analyze their role as a complement or replacement for human annotators.

cs.CL