arXiv subjects

Vesa Muhonen

Publications and source records attributed to Vesa Muhonen.

5 recordsLinked to original sources

arXiv2021

pH-RL: A personalization architecture to bring reinforcement learning to health practice

While reinforcement learning (RL) has proven to be the approach of choice for tackling many complex problems, it remains challenging to develop and deploy RL agents in real-life scenarios successfully. This paper presents pH-RL (personalization in e-Health with RL) a general RL architecture for personalization to bring RL to health practice. pH-RL allows for various levels of personalization in health applications and allows for online and batch learning. Furthermore, we provide a general-purpose implementation framework that can be integrated with various healthcare applications. We describe a step-by-step guideline for the successful deployment of RL policies in a mobile application. We implemented our open-source RL architecture and integrated it with the MoodBuster mobile application for mental health to provide messages to increase daily adherence to the online therapeutic modules. We then performed a comprehensive study with human participants over a sustained period. Our experimental results show that the developed policies learn to select appropriate actions consistently using only a few days' worth of data. Furthermore, we empirically demonstrate the stability of the learned policies during the study.

cs.AI↗

arXiv2020

A clustering-based reinforcement learning approach for tailored personalization of e-Health interventions

Personalization is very powerful in improving the effectiveness of health interventions. Reinforcement learning (RL) algorithms are suitable for learning these tailored interventions from sequential data collected about individuals. However, learning can be very fragile. The time to learn intervention policies is limited as disengagement from the user can occur quickly. Also, in e-Health intervention timing can be crucial before the optimal window passes. We present an approach that learns tailored personalization policies for groups of users by combining RL and clustering. The benefits are two-fold: speeding up the learning to prevent disengagement while maintaining a high level of personalization. Our clustering approach utilizes dynamic time warping to compare user trajectories consisting of states and rewards. We apply online and batch RL to learn policies over clusters of individuals and introduce our self-developed and publicly available simulator for e-Health interventions to evaluate our approach. We compare our methods with an e-Health intervention benchmark. We demonstrate that batch learning outperforms online learning for our setting. Furthermore, our proposed clustering approach for RL finds near-optimal clusterings which lead to significantly better policies in terms of cumulative reward compared to learning a policy per individual or learning one non-personalized policy across all individuals. Our findings also indicate that the learned policies accurately learn to send interventions at the right moments and that the users workout more and at the right times of the day.

cs.AI↗

arXiv2007

Hints of Isocurvature Perturbations in the Cosmic Microwave Background?

The improved data on the cosmic microwave background (CMB) anisotropy allow a better determination of the adiabaticity of the primordial perturbation. Interestingly, we find that recent CMB data seem to favor a contribution of a primordial isocurvature mode where the entropy perturbation is positively correlated with the primordial curvature perturbation and has a large spectral index (niso ~ 3). With 4 additional parameters we obtain a better fit to the CMB data by Delta chi^2 = 9.7 compared to an adiabatic model. For this best-fit model the nonadiabatic contribution to the CMB temperature variance is 4%. According to a Markov Chain Monte Carlo analysis the nonadiabatic contribution is positive at more than 95% C.L. The exact C.L. depends somewhat on the choice of priors, and we discuss the effect of different priors as well as additional cosmological data.

astro-ph↗

arXiv2004

Correlated Primordial Perturbations in Light of CMB and LSS Data

We use cosmic microwave background (CMB) and large-scale structure data to constrain cosmological models where the primordial perturbations have both an adiabatic and a cold dark matter (CDM) isocurvature component. We allow for a possible correlation between the adiabatic and isocurvature modes, and for different spectral indices for the power in each mode and for their correlation. We do a likelihood analysis with 11 independent parameters. We discuss the effect of choosing the pivot scale for the definition of amplitude parameters. The upper limit for the isocurvature fraction is 18% around a pivot scale k = 0.01 Mpc^{-1}. For smaller pivot wavenumbers the limit stays about the same. For larger pivot wavenumbers, very large values of the isocurvature spectral index are favored, which makes the analysis problematic, but larger isocurvature fractions seem to be allowed. For large isocurvature spectral indices n_iso > 2 a positive correlation between the adiabatic and isocurvature mode is favored, and for n_iso < 2 a negative correlation is favored. The upper limit to the nonadiabatic contribution to the CMB temperature variance is 7.5%. Of the standard cosmological parameters, determination of the CDM density $ω_c$ and the sound horizon angle $θ$ (or the Hubble constant $H_0$) are affected most by a possible presence of a correlated isocurvature contribution. The baryon density $ω_b$ nearly retains its ``adiabatic value''.

astro-ph↗

arXiv2003

Correlated adiabatic and isocurvature CMB fluctuations in the wake of the WMAP

In general correlated models, in addition to the usual adiabatic component with a spectral index n_ad1 there is another adiabatic component with a spectral index n_ad2 generated by entropy perturbation during inflation. We extend the analysis of a correlated mixture of adiabatic and isocurvature CMB fluctuations of the WMAP group, who set the two adiabatic spectral indices equal. Allowing n_ad1 and n_ad2 to vary independently we find that the WMAP data favor models where the two adiabatic components have opposite spectral tilts. Using the WMAP data only, the 2-sigma upper bound for the isocurvature fraction f_iso of the initial power spectrum at k_0=0.05 Mpc^{-1} increases somewhat, e.g., from 0.76 of n_ad2 = n_ad1 models to 0.84 with a prior n_iso < 1.84 for the isocurvature spectral index. We also comment on a possible degeneration between the correlation component and the optical depth tau. Moreover, the measured low quadrupole in the TT angular power could be achieved by a strong negative correlation, but then one needs a large tau to fit the TE spectrum.

astro-ph↗