arXiv subjects

Oya Deniz Beyan

Publications and source records attributed to Oya Deniz Beyan.

6 recordsLinked to original sources

arXiv2026

Enhancing RL Generalizability in Robotics through SHAP Analysis of Algorithms and Hyperparameters

Despite significant advances in Reinforcement Learning (RL), model performance remains highly sensitive to algorithm and hyperparameter configurations, while generalization gaps across environments complicate real-world deployment. Although prior work has studied RL generalization, the relative contribution of specific configurations to the generalization gap has not been quantitatively decomposed and systematically leveraged for configuration selection. To address this limitation, we propose an explainable framework that evaluates RL performance across robotic environments using SHapley Additive exPlanations (SHAP) to quantify configuration impacts. We establish a theoretical foundation connecting Shapley values to generalizability, empirically analyze configuration impact patterns, and introduce SHAP-guided configuration selection to enhance generalization. Our results reveal distinct patterns across algorithms and hyperparameters, with consistent configuration impacts across diverse tasks and environments. By applying these insights to configuration selection, we achieve improved RL generalizability and provide actionable guidance for practitioners.

cs.LG↗

arXiv2025

Multi-Objective Reinforcement Learning for Large Language Model Optimization: Visionary Perspective

Multi-Objective Reinforcement Learning (MORL) presents significant challenges and opportunities for optimizing multiple objectives in Large Language Models (LLMs). We introduce a MORL taxonomy and examine the advantages and limitations of various MORL methods when applied to LLM optimization, identifying the need for efficient and flexible approaches that accommodate personalization functionality and inherent complexities in LLMs and RL. We propose a vision for a MORL benchmarking framework that addresses the effects of different methods on diverse objective relationships. As future research directions, we focus on meta-policy MORL development that can improve efficiency and flexibility through its bi-level learning paradigm, highlighting key research questions and potential solutions for improving LLM performance.

cs.CL↗

arXiv2025

EMORL: Ensemble Multi-Objective Reinforcement Learning for Efficient and Flexible LLM Fine-Tuning

Recent advances in reinforcement learning (RL) for large language model (LLM) fine-tuning show promise in addressing multi-objective tasks but still face significant challenges, including competing objective balancing, low training efficiency, poor scalability, and limited explainability. Leveraging ensemble learning principles, we introduce an Ensemble Multi-Objective RL (EMORL) framework that fine-tunes multiple models with individual objectives while optimizing their aggregation after the fine-tuning to improve efficiency and flexibility. Our method is the first to aggregate the hidden states of individual models, incorporating contextual information from multiple objectives. This approach is supported by a hierarchical grid search algorithm that identifies optimal weighted combinations. We evaluate EMORL on counselor reflection generation tasks, using text classification models to score the generations and provide rewards during RL fine-tuning. Through comprehensive experiments on the PAIR and Psych8k datasets, we demonstrate the advantages of EMORL against existing baselines: significantly lower and more stable training consumption ($17,529\pm 1,650$ data points and $6,573\pm 147.43$ seconds), improved scalability and explainability, and comparable performance across multiple objectives.

cs.CL↗

arXiv2023

From Large Language Models to Knowledge Graphs for Biomarker Discovery in Cancer

Domain experts often rely on most recent knowledge for apprehending and disseminating specific biological processes that help them design strategies for developing prevention and therapeutic decision-making in various disease scenarios. A challenging scenarios for artificial intelligence (AI) is using biomedical data (e.g., texts, imaging, omics, and clinical) to provide diagnosis and treatment recommendations for cancerous conditions.~Data and knowledge about biomedical entities like cancer, drugs, genes, proteins, and their mechanism is spread across structured (knowledge bases (KBs)) and unstructured (e.g., scientific articles) sources. A large-scale knowledge graph (KG) can be constructed by integrating and extracting facts about semantically interrelated entities and relations. Such a KG not only allows exploration and question answering (QA) but also enables domain experts to deduce new knowledge. However, exploring and querying large-scale KGs is tedious for non-domain users due to their lack of understanding of the data assets and semantic technologies. In this paper, we develop a domain KG to leverage cancer-specific biomarker discovery and interactive QA. For this, we constructed a domain ontology called OncoNet Ontology (ONO), which enables semantic reasoning for validating gene-disease (different types of cancer) relations. The KG is further enriched by harmonizing the ONO, metadata, controlled vocabularies, and biomedical concepts from scientific articles by employing BioBERT- and SciBERT-based information extractors. Further, since the biomedical domain is evolving, where new findings often replace old ones, without having access to up-to-date scientific findings, there is a high chance an AI system exhibits concept drift while providing diagnosis and treatment. Therefore, we fine-tune the KG using large language models (LLMs) based on more recent articles and KBs.

cs.CL↗

arXiv2023

Privacy-Preserving Linkage of Distributed Datasets using the Personal Health Train

With the generation of personal and medical data at several locations, medical data science faces unique challenges when working on distributed datasets. Growing data protection requirements in recent years drastically limit the use of personally identifiable information. Distributed data analysis aims to provide solutions for securely working on highly sensitive data while minimizing the risk of information leaks, which would not be possible to the same degree in a centralized approach. A novel concept in this field is the Personal Health Train (PHT), which encapsulates the idea of bringing the analysis to the data, not vice versa. Data sources are represented as train stations. Trains containing analysis tasks move between stations and aggregate results. Train executions are coordinated by a central station which data analysts can interact with. Data remains at their respective stations and analysis results are only stored inside the train, providing a safe and secure environment for distributed data analysis. Duplicate records across multiple locations can skew results in a distributed data analysis. On the other hand, merging information from several datasets referring to the same real-world entities may improve data completeness and therefore data quality. In this paper, we present an approach for record linkage on distributed datasets using the Personal Health Train. We verify this approach and evaluate its effectiveness by applying it to two datasets based on real-world data and outline its possible applications in the context of distributed data analysis tasks.

cs.DC↗

arXiv2021

Distributed Learning for Melanoma Classification using Personal Health Train

Skin cancer is the most common cancer type. Usually, patients with suspicion of cancer are treated by doctors without any aided visual inspection. At this point, dermoscopy has become a suitable tool to support physicians in their decision-making. However, clinicians need years of expertise to classify possibly malicious skin lesions correctly. Therefore, research has applied image processing and analysis tools to improve the treatment process. In order to perform image analysis and train a model on dermoscopic images data needs to be centralized. Nevertheless, data centralization does not often comply with local data protection regulations due to its sensitive nature and due to the loss of sovereignty if data providers allow unlimited access to the data. A method to circumvent all privacy-related challenges of data centralization is Distributed Analytics (DA) approaches, which bring the analysis to the data instead of vice versa. This paradigm shift enables data analyses - in our case, image analysis - with data remaining inside institutional borders, i.e., the origin. In this documentation, we describe a straightforward use case including a model training for skin lesion classification based on decentralised data.

cs.DC↗