arXiv subjects

Sergio Morales

Publications and source records attributed to Sergio Morales.

4 recordsLinked to original sources

arXiv2025

Mind the Language Gap: Automated and Augmented Evaluation of Bias in LLMs for High- and Low-Resource Languages

Large Language Models (LLMs) have exhibited impressive natural language processing capabilities but often perpetuate social biases inherent in their training data. To address this, we introduce MultiLingual Augmented Bias Testing (MLA-BiTe), a framework that improves prior bias evaluation methods by enabling systematic multilingual bias testing. MLA-BiTe leverages automated translation and paraphrasing techniques to support comprehensive assessments across diverse linguistic settings. In this study, we evaluate the effectiveness of MLA-BiTe by testing four state-of-the-art LLMs in six languages -- including two low-resource languages -- focusing on seven sensitive categories of discrimination.

cs.CL↗

arXiv2025

The Software Diversity Card: A Framework for Reporting Diversity in Software Projects

Context: Interest in diversity in software development has significantly increased in recent years. Reporting on diversity in software projects can enhance user trust and assist regulators in evaluating adoption. Recent AI directives include clauses that mandate diversity information during development, highlighting the growing interest of public regulators. However, current documentation often neglects diversity in favor of technical features, partly due to a lack of tools for its description and annotation. Objectives: This work introduces the Software Diversity Card, a structured approach for documenting and sharing diversity-related aspects within software projects. It aims to profile the various teams involved in software development and governance, including user groups in testing and software adaptations for diverse social groups. Methods: We conducted a literature review on diversity and inclusion in software development and analyzed 1,000 top-starred Open Source Software (OSS) repositories on GitHub to identify diversity-related information. Moreover, we present a diversity modeling language, a toolkit for generating cards using it, and a study of its application in two real-world software projects. Results: Despite the growing awareness of diversity in the research community, our analysis found a notable lack of diversity reporting in OSS projects. Applying the card to real-world examples highlighted challenges such as balancing anonymity and transparency, managing sensitive data, and ensuring authenticity. Conclusion: Our proposal can enhance diversity practices in software development, support public administrations in software assessment, and help businesses promote diversity as a key asset.

cs.SE↗

arXiv2024

A Framework to Model ML Engineering Processes

The development of Machine Learning (ML) based systems is complex and requires multidisciplinary teams with diverse skill sets. This may lead to communication issues or misapplication of best practices. Process models can alleviate these challenges by standardizing task orchestration, providing a common language to facilitate communication, and nurturing a collaborative environment. Unfortunately, current process modeling languages are not suitable for describing the development of such systems. In this paper, we introduce a framework for modeling ML-based software development processes, built around a domain-specific language and derived from an analysis of scientific and gray literature. A supporting toolkit is also available.

cs.SE↗

arXiv2024

LangBiTe: A Platform for Testing Bias in Large Language Models

The integration of Large Language Models (LLMs) into various software applications raises concerns about their potential biases. Typically, those models are trained on a vast amount of data scrapped from forums, websites, social media and other internet sources, which may instill harmful and discriminating behavior into the model. To address this issue, we present LangBiTe, a testing platform to systematically assess the presence of biases within an LLM. LangBiTe enables development teams to tailor their test scenarios, and automatically generate and execute the test cases according to a set of user-defined ethical requirements. Each test consists of a prompt fed into the LLM and a corresponding test oracle that scrutinizes the LLM's response for the identification of biases. LangBite provides users with the bias evaluation of LLMs, and end-to-end traceability between the initial ethical requirements and the insights obtained.

cs.SE↗