arXiv subjects

Djallel Bouneffouf

Publications and source records attributed to Djallel Bouneffouf.

At least 73 records · Page 4Linked to original sources

arXiv2018

Incorporating Behavioral Constraints in Online AI Systems

AI systems that learn through reward feedback about the actions they take are increasingly deployed in domains that have significant impact on our daily life. However, in many cases the online rewards should not be the only guiding criteria, as there are additional constraints and/or priorities imposed by regulations, values, preferences, or ethical principles. We detail a novel online agent that learns a set of behavioral constraints by observation and uses these learned constraints as a guide when making decisions in an online setting while still being reactive to reward feedback. To define this agent, we propose to adopt a novel extension to the classical contextual multi-armed bandit setting and we provide a new algorithm called Behavior Constrained Thompson Sampling (BCTS) that allows for online learning while obeying exogenous constraints. Our agent learns a constrained policy that implements the observed behavioral constraints demonstrated by a teacher agent, and then uses this constrained policy to guide the reward-based online exploration and exploitation. We characterize the upper bound on the expected regret of the contextual bandit algorithm that underlies our agent and provide a case study with real world data in two application domains. Our experiments show that the designed agent is able to act within the set of behavior constraints without significantly degrading its overall reward performance.

cs.AI↗

arXiv2017

Context Attentive Bandits: Contextual Bandit with Restricted Context

We consider a novel formulation of the multi-armed bandit model, which we call the contextual bandit with restricted context, where only a limited number of features can be accessed by the learner at every iteration. This novel formulation is motivated by different online problems arising in clinical trials, recommender systems and attention modeling. Herein, we adapt the standard multi-armed bandit algorithm known as Thompson Sampling to take advantage of our restricted context setting, and propose two novel algorithms, called the Thompson Sampling with Restricted Context(TSRC) and the Windows Thompson Sampling with Restricted Context(WTSRC), for handling stationary and nonstationary environments, respectively. Our empirical results demonstrate advantages of the proposed approaches on several real-life datasets

cs.AI↗

arXiv2017

Bandit Models of Human Behavior: Reward Processing in Mental Disorders

Drawing an inspiration from behavioral studies of human decision making, we propose here a general parametric framework for multi-armed bandit problem, which extends the standard Thompson Sampling approach to incorporate reward processing biases associated with several neurological and psychiatric conditions, including Parkinson's and Alzheimer's diseases, attention-deficit/hyperactivity disorder (ADHD), addiction, and chronic pain. We demonstrate empirically that the proposed parametric approach can often outperform the baseline Thompson Sampling on a variety of datasets. Moreover, from the behavioral modeling perspective, our parametric framework can be viewed as a first step towards a unifying computational model capturing reward processing abnormalities across multiple mental conditions.

cs.AI↗

arXiv2017

Multi-armed Bandit Problem with Known Trend

We consider a variant of the multi-armed bandit model, which we call multi-armed bandit problem with known trend, where the gambler knows the shape of the reward function of each arm but not its distribution. This new problem is motivated by different online problems like active learning, music and interface recommendation applications, where when an arm is sampled by the model the received reward change according to a known trend. By adapting the standard multi-armed bandit algorithm UCB1 to take advantage of this setting, we propose the new algorithm named A-UCB that assumes a stochastic model. We provide upper bounds of the regret which compare favourably with the ones of UCB1. We also confirm that experimentally with different simulations

cs.LG↗

arXiv2014

A Neural Networks Committee for the Contextual Bandit Problem

This paper presents a new contextual bandit algorithm, NeuralBandit, which does not need hypothesis on stationarity of contexts and rewards. Several neural networks are trained to modelize the value of rewards knowing the context. Two variants, based on multi-experts approach, are proposed to choose online the parameters of multi-layer perceptrons. The proposed algorithms are successfully tested on a large dataset with and without stationarity of rewards.

cs.NE↗

arXiv2014

Freshness-Aware Thompson Sampling

To follow the dynamicity of the user's content, researchers have recently started to model interactions between users and the Context-Aware Recommender Systems (CARS) as a bandit problem where the system needs to deal with exploration and exploitation dilemma. In this sense, we propose to study the freshness of the user's content in CARS through the bandit problem. We introduce in this paper an algorithm named Freshness-Aware Thompson Sampling (FA-TS) that manages the recommendation of fresh document according to the user's risk of the situation. The intensive evaluation and the detailed analysis of the experimental results reveals several important discoveries in the exploration/exploitation (exr/exp) behaviour.

cs.IR↗

arXiv2014

R-UCB: a Contextual Bandit Algorithm for Risk-Aware Recommender Systems

Mobile Context-Aware Recommender Systems can be naturally modelled as an exploration/exploitation trade-off (exr/exp) problem, where the system has to choose between maximizing its expected rewards dealing with its current knowledge (exploitation) and learning more about the unknown user's preferences to improve its knowledge (exploration). This problem has been addressed by the reinforcement learning community but they do not consider the risk level of the current user's situation, where it may be dangerous to recommend items the user may not desire in her current situation if the risk level is high. We introduce in this paper an algorithm named R-UCB that considers the risk level of the user's situation to adaptively balance between exr and exp. The detailed analysis of the experimental results reveals several important discoveries in the exr/exp behaviour.

cs.IR↗

arXiv2014

Exponentiated Gradient Exploration for Active Learning

Active learning strategies respond to the costly labelling task in a supervised classification by selecting the most useful unlabelled examples in training a predictive model. Many conventional active learning algorithms focus on refining the decision boundary, rather than exploring new regions that can be more informative. In this setting, we propose a sequential algorithm named EG-Active that can improve any Active learning algorithm by an optimal random exploration. Experimental results show a statistically significant and appreciable improvement in the performance of our new approach over the existing active feedback methods.

cs.LG↗

arXiv2014

Context-Based Information Retrieval in Risky Environment

Context-Based Information Retrieval is recently modelled as an exploration/ exploitation trade-off (exr/exp) problem, where the system has to choose between maximizing its expected rewards dealing with its current knowledge (exploitation) and learning more about the unknown user's preferences to improve its knowledge (exploration). This problem has been addressed by the reinforcement learning community but they do not consider the risk level of the current user's situation, where it may be dangerous to explore the non-top-ranked documents the user may not desire in his/her current situation if the risk level is high. We introduce in this paper an algorithm named CBIR-R-greedy that considers the risk level of the user's situation to adaptively balance between exr and exp.

cs.IR↗

arXiv2014

Étude des dimensions spécifiques du contexte dans un système de filtrage d'informations

In the context of business information systems, e-commerce and access to knowledge, the relevance of the information provided to use is a key fact to the success of information systems. Therefore the quality of access is determined by access to the right information at the right time, at the right place. In this context, it is important to consider the users needs when access to information and his contextual situation in order to provide relevant information, tailored to their needs and context use. In what follows we describe the prelude to a project that tries to combine all of these needs to improve information systems.

cs.IR↗

arXiv2014

Optimizing an Utility Function for Exploration / Exploitation Trade-off in Context-Aware Recommender System

In this paper, we develop a dynamic exploration/ exploitation (exr/exp) strategy for contextual recommender systems (CRS). Specifically, our methods can adaptively balance the two aspects of exr/exp by automatically learning the optimal tradeoff. This consists of optimizing a utility function represented by a linearized form of the probability distributions of the rewards of the clicked and the non-clicked documents already recommended. Within an offline simulation framework we apply our algorithms to a CRS and conduct an evaluation with real event log data. The experimental results and detailed analysis demonstrate that our algorithms outperform existing algorithms in terms of click-through-rate (CTR).

cs.IR↗

arXiv2014

Improving adaptation of ubiquitous recommander systems by using reinforcement learning and collaborative filtering

The wide development of mobile applications provides a considerable amount of data of all types (images, texts, sounds, videos, etc.). Thus, two main issues have to be considered: assist users in finding information and reduce search and navigation time. In this sense, context-based recommender systems (CBRS) propose the user the adequate information depending on her/his situation. Our work consists in applying machine learning techniques and reasoning process in order to bring a solution to some of the problems concerning the acceptance of recommender systems by users, namely avoiding the intervention of experts, reducing cold start problem, speeding learning process and adapting to the user's interest. To achieve this goal, we propose a fundamental modification in terms of how we model the learning of the CBRS. Inspired by models of human reasoning developed in robotic, we combine reinforcement learning and case-based reasoning to define a contextual recommendation process based on different context dimensions (cognitive, social, temporal, geographic). This paper describes an ongoing work on the implementation of a CBRS based on a hybrid Q-learning (HyQL) algorithm which combines Q-learning, collaborative filtering and case-based reasoning techniques. It also presents preliminary results by comparing HyQL and the standard Q-Learning w.r.t. solving the cold start problem.

cs.IR↗

arXiv2014

Situation-Aware Approach to Improve Context-based Recommender System

In this paper, we introduce a novel situation aware approach to improve a context based recommender system. To build situation aware user profiles, we rely on evidence issued from retrieval situations. A retrieval situation refers to the social spatio temporal context of the user when he interacts with the recommender system. A situation is represented as a combination of social spatio temporal concepts inferred from ontological knowledge given social group, location and time information. User's interests are inferred from past user's interaction with the recommender system related to the identified situations. They are represented using concepts issued from a domain ontology. We also propose a method to dynamically adapt the system to the user's interest's evolution.

cs.IR↗

arXiv2014

Hybrid Q-Learning Applied to Ubiquitous recommender system

Ubiquitous information access becomes more and more important nowadays and research is aimed at making it adapted to users. Our work consists in applying machine learning techniques in order to bring a solution to some of the problems concerning the acceptance of the system by users. To achieve this, we propose a fundamental shift in terms of how we model the learning of recommender system: inspired by models of human reasoning developed in robotic, we combine reinforcement learning and case-base reasoning to define a recommendation process that uses these two approaches for generating recommendations on different context dimensions (social, temporal, geographic). We describe an implementation of the recommender system based on this framework. We also present preliminary results from experiments with the system and show how our approach increases the recommendation quality.

cs.LG↗

arXiv2014

The Impact of Situation Clustering in Contextual-Bandit Algorithm for Context-Aware Recommender Systems

Most existing approaches in Context-Aware Recommender Systems (CRS) focus on recommending relevant items to users taking into account contextual information, such as time, location, or social aspects. However, few of them have considered the problem of user's content dynamicity. We introduce in this paper an algorithm that tackles the user's content dynamicity by modeling the CRS as a contextual bandit algorithm and by including a situation clustering algorithm to improve the precision of the CRS. Within a deliberately designed offline simulation framework, we conduct evaluations with real online event log data. The experimental results and detailed analysis reveal several important discoveries in context aware recommender system.

cs.IR↗

arXiv2014

Recommandation mobile, sensible au contexte de contenus évolutifs: Contextuel-E-Greedy

We introduce in this paper an algorithm named Contextuel-E-Greedy that tackles the dynamicity of the user's content. It is based on dynamic exploration/exploitation tradeoff and can adaptively balance the two aspects by deciding which situation is most relevant for exploration or exploitation. The experimental results demonstrate that our algorithm outperforms surveyed algorithms.

cs.AI↗

arXiv2013

Exponentiated Gradient LINUCB for Contextual Multi-Armed Bandits

We present Exponentiated Gradient LINUCB, an algorithm for con-textual multi-armed bandits. This algorithm uses Exponentiated Gradient to find the optimal exploration of the LINUCB. Within a deliberately designed offline simulation framework we conduct evaluations with real online event log data. The experimental results demonstrate that our algorithm outperforms surveyed algorithms.

cs.AI↗

arXiv2013

Mobile Recommender Systems Methods: An Overview

The information that mobiles can access becomes very wide nowadays, and the user is faced with a dilemma: there is an unlimited pool of information available to him but he is unable to find the exact information he is looking for. This is why the current research aims to design Recommender Systems (RS) able to continually send information that matches the user's interests in order to reduce his navigation time. In this paper, we treat the different approaches to recommend.

cs.IR↗