Searcharxiv⌕ Search

arXiv subjects

Piotr Miłoś

Publications and source records attributed to Piotr Miłoś.

At least 55 records · Page 3Linked to original sources
arXiv2021

Phase transition for the interchange and quantum Heisenberg models on the Hamming graph

We study a family of random permutation models on the Hamming graph $H(2,n)$ (i.e., the $2$-fold Cartesian product of complete graphs), containing the interchange process and the cycle-weighted interchange process with parameter $θ> 0$. This family contains the random walk representation of the quantum Heisenberg ferromagnet. We show that in these models the cycle structure of permutations undergoes a \textit{phase transition} -- when the number of transpositions defining the permutation is $\leq c n^2$, for small enough $c > 0$, all cycles are microscopic, while for more than $\geq C n^2$ transpositions, for large enough $C > 0$, macroscopic cycles emerge with high probability. We provide bounds on values $C,c$ depending on the parameter $θ$ of the model, in particular for the interchange process we pinpoint exactly the critical time of the phase transition. Our results imply also the existence of a phase transition in the quantum Heisenberg ferromagnet on $H(2,n)$, namely for low enough temperatures spontaneous magnetization occurs, while it is not the case for high temperatures. At the core of our approach is a novel application of the cyclic random walk, which might be of independent interest. By analyzing explorations of the cyclic random walk, we show that sufficiently long cycles of a random permutation are uniformly spread on the graph, which makes it possible to compare our models to the mean-field case, i.e., the interchange process on the complete graph, extending the approach used earlier by Schramm.

math.PR↗
arXiv2020

Uncertainty-sensitive Learning and Planning with Ensembles

We propose a reinforcement learning framework for discrete environments in which an agent makes both strategic and tactical decisions. The former manifests itself through the use of value function, while the latter is powered by a tree search planner. These tools complement each other. The planning module performs a local \textit{what-if} analysis, which allows to avoid tactical pitfalls and boost backups of the value function. The value function, being global in nature, compensates for inherent locality of the planner. In order to further solidify this synergy, we introduce an exploration mechanism with two distinctive components: uncertainty modelling and risk measurement. To model the uncertainty we use value function ensembles, and to reflect risk we use propose several functionals that summarize the implied by the ensemble. We show that our method performs well on hard exploration environments: Deep-sea, toy Montezuma's Revenge, and Sokoban. In all the cases, we obtain speed-up in learning and boost in performance.

cs.LG↗
arXiv2019

The interchange process with reversals on the complete graph

We consider an extension of the interchange process on the complete graph, in which a fraction of the transpositions are replaced by `reversals'. The model is motivated by statistical physics, where it plays a role in stochastic representations of $XXZ$-models. We prove convergence to PD($\tfrac12$) of the rescaled cycle sizes, above the critical point for the appearance of macroscopic cycles. This extends a result of Schramm on convergence to PD(1) for the usual interchange process.

math.PR↗
arXiv2019

Developmentally motivated emergence of compositional communication via template transfer

This paper explores a novel approach to achieving emergent compositional communication in multi-agent systems. We propose a training regime implementing template transfer, the idea of carrying over learned biases across contexts. In our method, a sender-receiver pair is first trained with disentangled loss functions and then the receiver is transferred to train a new sender with a standard loss. Unlike other methods (e.g. the obverter algorithm), our approach does not require imposing inductive biases on the architecture of the agents. We experimentally show the emergence of compositional communication using topographical similarity, zero-shot generalization and context independence as evaluation metrics. The presented approach is connected to an important line of work in semiotics and developmental psycholinguistics: it supports a conjecture that compositional communication is scaffolded on simpler communication protocols.

cs.LG↗
arXiv2018

Brownian motion and Random Walk above Quenched Random Wall

We study the persistence exponent for the first passage time of a random walk below the trajectory of another random walk. More precisely, let $\{B_n\}$ and $\{W_n\}$ be two centered, weakly dependent random walks. We establish that $\mathbb{P}(\forall_{n\leq N} B_n \geq W_n|W) = N^{-γ+ o(1)}$ for a non-random $γ\geq 1/2$. In the classical setting, $W_n \equiv 0$, it is well-known that $γ= 1/2$. We prove that for any non-trivial $W$ one has $γ>1/2$ and the exponent $γ$ depends only on $\text{Var}(B_1)/\text{Var}(W_1)$. Our result holds also in the continuous setting, when $B$ and $W$ are independent and possibly perturbed Brownian motions or Ornstein-Uhlenbeck processes. In the latter case the probability decays at exponential rate.

math.PR↗
arXiv2018

Expert-augmented actor-critic for ViZDoom and Montezumas Revenge

We propose an expert-augmented actor-critic algorithm, which we evaluate on two environments with sparse rewards: Montezumas Revenge and a demanding maze from the ViZDoom suite. In the case of Montezumas Revenge, an agent trained with our method achieves very good results consistently scoring above 27,000 points (in many experiments beating the first world). With an appropriate choice of hyperparameters, our algorithm surpasses the performance of the expert data. In a number of experiments, we have observed an unreported bug in Montezumas Revenge which allowed the agent to score more than 800,000 points.

cs.LG↗
arXiv2018

Learning to Run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments

In the NIPS 2017 Learning to Run challenge, participants were tasked with building a controller for a musculoskeletal model to make it run as fast as possible through an obstacle course. Top participants were invited to describe their algorithms. In this work, we present eight solutions that used deep reinforcement learning approaches, based on algorithms such as Deep Deterministic Policy Gradient, Proximal Policy Optimization, and Trust Region Policy Optimization. Many solutions use similar relaxations and heuristics, such as reward shaping, frame skipping, discretization of the action space, symmetry, and policy blending. However, each of the eight teams implemented different modifications of the known algorithms.

cs.LG↗
arXiv2018

CLT for supercritical branching processes with heavy-tailed branching law

Consider a branching system with particles moving according to an Ornstein-Uhlenbeck process with drift $μ>0$ and branching according to a law in the domain of attraction of the $(1+β)$-stable distribution. The mean of the branching law is strictly larger than $1$ implying that the system is supercritical and the total number of particles grows exponentially at some rate $λ>0$. It is known that the system obeys a law of large numbers. In the paper we study its rate of convergence. We discover an interesting interplay between the branching rate $λ$ and the drift parameter $μ$. There are three regimes of the second order behavior: $\cdot$ small branching, $λ<(1+1/β) μ$, then the speed of convergence is the same as in the stable central limit theorem but the limit is affected by the dependence between particles. $\cdot$ critical branching, $λ=(1+1/β) μ$, then the dependence becomes strong enough to make the rate of convergence slightly smaller, yet the qualitative behaviour still resembles the stable central limit theorem $\cdot$ large branching, $λ> (1+1/β) μ$, then the dependence manifests much more profoundly, the rate of convergence is substantially smaller and strangely the limit holds a.s.

math.PR↗
arXiv2017

Note on Bolthausen-Deuschel-Zeitouni's paper on the absence of a wetting transition for a pinned harmonic crystal in dimensions three and larger

The article [Bolthausen et al., 2000] provides a proof of the absence of a wetting transition for the discrete Gaussian free field conditioned to stay positive, and undergoing a weak delta-pinning at height 0. The proof is generalized to the case of a square pinning-potential replacing the delta-pinning, but it relies on a lower bound on the probability for the field to stay above the support of the potential, the proof of which appears to be incorrect. We provide a modified proof of the absence of a wetting transition in the square-potential case, which does not require the aforementioned lower bound. An alternative approach is given in a recent paper by Giacomin and Lacoin.

math.PR↗
arXiv2016

Branching Brownian motion with absorption and the all-time minimum of branching Brownian motion with drift

We study a dyadic branching Brownian motion on the real line with absorption at 0, drift $μ\in \mathbb{R}$ and started from a single particle at position $x>0.$ When $μ$ is large enough so that the process has a positive probability of survival, we consider $K(t),$ the number of individuals absorbed at 0 by time $t$ and for $s\ge 0$ the functions $ω_s(x):= \mathbb{E}^x[s^{K(\infty)}].$ We show that $ω_s<\infty$ if and only of $s\in[0,s_0]$ for some $s_0>1$ and we study the properties of these functions. Furthermore, for $s=0, ω(x) := ω_0(x) =\mathbb{P}^x(K(\infty)=0)$ is the cumulative distribution function of the all time minimum of the branching Brownian motion with drift started at 0 without absorption. We give three descriptions of the family $ω_s, s\in [0,s_0]$ through a single pair of functions, as the two extremal solutions of the Kolmogorov-Petrovskii-Piskunov (KPP) traveling wave equation on the half-line, through a martingale representation and as an explicit series expansion. We also obtain a precise result concerning the tail behavior of $K(\infty)$. In addition, in the regime where $K(\infty)>0$ almost surely, we show that $u(x,t) := \mathbb{P}^x(K(t)=0)$ suitably centered converges to the KPP critical travelling wave on the whole real line.

math.PR↗
arXiv2015

Delocalization of two-dimensional random surfaces with hard-core constraints

We study the fluctuations of random surfaces on a two-dimensional discrete torus. The random surfaces we consider are defined via a nearest-neighbor pair potential which we require to be twice continuously differentiable on a (possibly infinite) interval and infinity outside of this interval. No convexity assumption is made and we include the case of the so-called hammock potential, when the random surface is uniformly chosen from the set of all surfaces satisfying a Lipschitz constraint. Our main result is that these surfaces delocalize, having fluctuations whose variance is at least of order $\log n$, where $n$ is the side length of the torus. We also show that the expected maximum of such surfaces is of order at least $\log n$. The main tool in our analysis is an adaptation to the lattice setting of an algorithm of Richthammer, who developed a variant of a Mermin-Wagner-type argument applicable to hard-core constraints. We rely also on the reflection positivity of the random surface model. The result answers a question mentioned by Brascamp, Lieb and Lebowitz 1975 on the hammock potential and a question of Velenik 2006.

math.PR↗
arXiv2013

Exact representation of truncated variation of Brownian motion

In the recent papers [Lochowski:2011fk, Lochowski:2013yq, Lochowski:2013lr] the truncated variation has been introduced, characterized and studied in various stochastic settings. In this note we uncover an intimate link to the Skorokhod problem. Further, we exploit it to give an explicit representation of the truncated variation of a Brownian motion. More precisely, we prove that the inverse of this process is, up to a minor time shift, a Lévy subordinator with the exponent \sqrt{2q}\tanh(c\sqrt{q/2}) . This also gives a representation of a solution of the two-sided Skorokhod problem for a Brownian motion.

math.PR↗
arXiv2013

A second note on the discrete Gaussian Free Field with disordered pinning on Z^d, d\geq 2

We study the discrete massless Gaussian Free Field on Z^d, d \geq 2, in the presence of a disordered square-well potential supported on a finite strip around zero. The disorder is introduced by reward/penalty interaction coefficients, which are given by i.i.d. random variables. In the previous note, we proved under minimal assumptions on the law of the environment, that the quenched free energy associated to this model exists in R^+, is deterministic, and strictly smaller than the annealed free energy whenever the latter is strictly positive. Here we consider Bernoulli reward/penalty coefficients b e_x + h with P(e_x=-1)=P(e_x=+1)=1/2 for all x in Z^d, and b > 0, h in R. We prove that in the plane (b,h), the quenched critical line (separating the phases of positive and zero free energy) lies strictly below the line h = 0, showing in particular that there exists a non trivial region where the field is localized though repulsed on average by the environment.

math.PR↗
arXiv2013

A note on the discrete Gaussian Free Field with disordered pinning on Z^d, d\geq 2

We study the discrete massless Gaussian Free Field on $\Z^d$, $d\geq2$, in the presence of a disordered square-well potential supported on a finite strip around zero. The disorder is introduced by reward/penalty interaction coefficients, which are given by i.i.d. random variables. Under minimal assumptions on the law of the environment, we prove that the quenched free energy associated to this model exists in $\R^+$, is deterministic, and strictly smaller than the annealed free energy whenever the latter is strictly positive.

math.PR↗
arXiv2012

On limit distributions of normalized truncated variation, upward truncated variation and downward truncated variation processes

In the paper we introduce the truncated variation, upward truncated variation and downward truncated variation. These are closely related to the total variation but are well-defined even if the latter is infinite. Our aim is to explore their feasibility to studies of stochastic processes. We concentrate on a Brownian motion with drift for which we prove the convergence of the above- mentioned quantities. For example, we study the truncated variation when the truncation parameter c tends to 0. We prove in this case that for "small" c's it is well-approximated by a deterministic process. Moreover we prove that error in this approximation converges weakly (in functional sense) to a Brownian motion. We prove also similar result for truncated variation processes when time parameter is rescaled to infinity. We stress that our methodology is robust. A key to the proofs was a decomposition of the truncated variation (see Lemmas 11 and 12). It can be used for studies of any continuous processes. Some additional results like an analog of the Anscombe-Donsker theorem and the Laplace transform of time to given drawdown by c (and analogously drawup till time) are presented.

math.PR↗
arXiv2012

On truncated variation, upward truncated variation and downward truncated variation for diffusions

The truncated variation, $TV^c$, is a fairly new concept introduced in [5]. Roughly speaking, given a càdlàg function $f$, its truncated variation is "the total variation which does not pay attention to small changes of $f$, below some threshold $c>0$". The very basic consequence of such approach is that contrary to the total variation, $TV^c$ is always finite. This is appealing to the stochastic analysis where so-far large classes of processes, like semimartingales or diffusions, could not be studied with the total variation. Recently in [6], another characterization of $TV^c$ was found. Namely $TV^c$ is the smallest total variation of a function which approximates $f$ uniformly with accuracy $c/2$. Due to these properties we envisage that $TV^c$ might be a useful concept to the theory of processes. For this reason we determine some properties of $TV^c$ for some well-known processes. In course of our research we discover intimate connections with already known concepts of the stochastic processes theory. Firstly, for semimartingales we proved that $TV^c$ is of order $c^{-1}$ and the normalized truncated variation converges almost surely to the quadratic variation of the semimartingale as $c\searrow0$. Secondly, we studied the rate of this convergence. As this task was much more demanding we narrowed to the class of diffusions (with some mild additional assumptions). We obtained the weak convergence to a so-called Ocone martingale. These results can be viewed as some kind of large numbers theorem and the corresponding central limit theorem. All the results above were obtained in a functional setting, viz. we worked with processes describing the growth of the truncated variation in time. Moreover, in the same respect we also treated two closely related quantities - the so-called upward truncated variation and downward truncated variation.

math.PR↗
arXiv2012

Spatial CLT for the supercritical Ornstein-Uhlenbeck superprocess

In this paper we consider a superprocess being a measure-valued diffusion corresponding to the equation $u_{t}=Lu+αu-βu^{2}$, where $L$ is the infinitesimal operator of the \emph{Ornstein-Uhlenbeck process} and $β>0,\:α>0$. The latter condition implies that the process is \emph{supercritical,} i.e. its total mass grows exponentially. This system is known to fulfill a law of large numbers. In the paper we prove the corresponding \emph{central limit theorem}. The limit and the CLT normalization fall into three qualitatively different classes. In what we call the small growth rate case the situation resembles the classical CLT. The weak limit is Gaussian and the normalization is the square root of the size of the system. In the critical case the limit is still Gaussian, however the normalization requires an additional term. Finally, when the growth rate is large the situation is completely different. The limit is no longer Gaussian, the normalization is substantially larger than the classical one and the convergence holds in probability. These different regimes arise as a result of "competition" between spatial smoothing due to the particles' movement and the system's growth which is local.

math.PR↗