arXiv subjects

Hanbaek Lyu

Publications and source records attributed to Hanbaek Lyu.

At least 55 records · Page 3Linked to original sources

arXiv2020

Applications of Online Nonnegative Matrix Factorization to Image and Time-Series Data

Online nonnegative matrix factorization (ONMF) is a matrix factorization technique in the online setting where data are acquired in a streaming fashion and the matrix factors are updated each time. This enables factor analysis to be performed concurrently with the arrival of new data samples. In this article, we demonstrate how one can use online nonnegative matrix factorization algorithms to learn joint dictionary atoms from an ensemble of correlated data sets. We propose a temporal dictionary learning scheme for time-series data sets, based on ONMF algorithms. We demonstrate our dictionary learning technique in the application contexts of historical temperature data, video frames, and color images.

cs.LG↗

arXiv2020

Online matrix factorization for Markovian data and applications to Network Dictionary Learning

Online Matrix Factorization (OMF) is a fundamental tool for dictionary learning problems, giving an approximate representation of complex data sets in terms of a reduced number of extracted features. Convergence guarantees for most of the OMF algorithms in the literature assume independence between data matrices, and the case of dependent data streams remains largely unexplored. In this paper, we show that a non-convex generalization of the well-known OMF algorithm for i.i.d. stream of data in \citep{mairal2010online} converges almost surely to the set of critical points of the expected loss function, even when the data matrices are functions of some underlying Markov chain satisfying a mild mixing condition. This allows one to extract features more efficiently from dependent data streams, as there is no need to subsample the data sequence to approximately satisfy the independence assumption. As the main application, by combining online non-negative matrix factorization and a recent MCMC algorithm for sampling motifs from networks, we propose a novel framework of Network Dictionary Learning, which extracts ``network dictionary patches' from a given network in an online manner that encodes main features of the network. We demonstrate this technique and its application to network denoising problems on real-world network data.

cs.LG↗

arXiv2020

On the number of contingency tables and the independence heuristic

We obtain sharp asymptotic estimates on the number of $n \times n$ contingency tables with two linear margins $Cn$ and $BCn$. The results imply a second order phase transition on the number of such contingency tables, with a critical value at \ts $B_{c}:=1 + \sqrt{1+1/C}$. As a consequence, for \ts $B>B_{c}$, we prove that the classical \emph{independence heuristic} leads to a large undercounting.

math.CO↗

arXiv2020

Phase transition in random contingency tables with non-uniform margins

For parameters $n,δ,B,$ and $C$, let $X=(X_{k\ell})$ be the random uniform contingency table whose first $\lfloor n^δ \rfloor $ rows and columns have margin $\lfloor BCn \rfloor$ and the last $n$ rows and columns have margin $\lfloor Cn \rfloor$. For every $0<δ<1$, we establish a sharp phase transition of the limiting distribution of each entry of $X$ at the critical value $B_{c}=1+\sqrt{1+1/C}$. In particular, for $1/2<δ<1$, we show that the distribution of each entry converges to a geometric distribution in total variation distance, whose mean depends sensitively on whether $B B_{c}$. Our main result shows that $\mathbb{E}[X_{11}]$ is uniformly bounded for $B B_{c}$. We also establish a strong law of large numbers for the row sums in top right and top left blocks.

math.PR↗

arXiv2020

Double jump phase transition in a soliton cellular automaton

In this paper, we consider the soliton cellular automaton introduced in [Takahashi 1990] with a random initial configuration. We give multiple constructions of a Young diagram describing various statistics of the system in terms of familiar objects like birth-and-death chains and Galton-Watson forests. Using these ideas, we establish limit theorems showing that if the first $n$ boxes are occupied independently with probability $p\in(0,1)$, then the number of solitons is of order $n$ for all $p$, and the length of the longest soliton is of order $\log n$ for $p<1/2$, order $\sqrt{n}$ for $p=1/2$, and order $n$ for $p>1/2$. Additionally, we uncover a condensation phenomenon in the supercritical regime: For each fixed $j\geq 1$, the top $j$ soliton lengths have the same order as the longest for $p\leq 1/2$, whereas all but the longest have order at most $\log n$ for $p>1/2$. As an application, we obtain scaling limits for the lengths of the $k^{\text{th}}$ longest increasing and decreasing subsequences in a random stack-sortable permutation of length $n$ in terms of random walks and Brownian excursions.

math.PR↗

arXiv2020

Stretched exponential decay for subcritical parking times on $\mathbb{Z}^d$

In the parking model on $\mathbb{Z}^d$, each vertex is initially occupied by a car (with probability $p$) or by a vacant parking spot (with probability $1-p$). Cars perform independent random walks and when they enter a vacant spot, they park there, thereby rendering the spot occupied. Cars visiting occupied spots simply keep driving (continuing their random walk). It is known that $p=1/2$ is a critical value in the sense that the origin is a.s. visited by finitely many distinct cars when $p<1/2$, and by infinitely many distinct cars when $p\geq 1/2$. Furthermore, any given car a.s. eventually parks for $p \leq 1/2$ and with positive probability does not park for $p > 1/2$. We study the subcritical phase and prove that the tail of the parking time $τ$ of the car initially at the origin obeys the bounds \[ \exp\left( - C_1 t^{\frac{d}{d+2}}\right) \leq \mathbb{P}_p(τ> t) \leq \exp\left( - c_2 t^{\frac{d}{d+2}}\right) \] for $p>0$ sufficiently small. For $d=1$, we prove these inequalities for all $p \in [0,1/2)$. This result presents an asymmetry with the supercritical phase ($p>1/2$), where methods of Bramson--Lebowitz imply that for $d=1$ the corresponding tail of the parking time of the parking spot of the origin decays like $e^{-c\sqrt{t}}$. Our exponent $d/(d+2)$ also differs from those previously obtained in the case of moving obstacles.

math.PR↗

arXiv2020

COVID-19 Time-series Prediction by Joint Dictionary Learning and Online NMF

Predicting the spread and containment of COVID-19 is a challenge of utmost importance that the broader scientific community is currently facing. One of the main sources of difficulty is that a very limited amount of daily COVID-19 case data is available, and with few exceptions, the majority of countries are currently in the "exponential spread stage," and thus there is scarce information available which would enable one to predict the phase transition between spread and containment. In this paper, we propose a novel approach to predicting the spread of COVID-19 based on dictionary learning and online nonnegative matrix factorization (online NMF). The key idea is to learn dictionary patterns of short evolution instances of the new daily cases in multiple countries at the same time, so that their latent correlation structures are captured in the dictionary patterns. We first learn such patterns by minibatch learning from the entire time-series and then further adapt them to the time-series by online NMF. As we progressively adapt and improve the learned dictionary patterns to the more recent observations, we also use them to make one-step predictions by the partial fitting. Lastly, by recursively applying the one-step predictions, we can extrapolate our predictions into the near future. Our prediction results can be directly attributed to the learned dictionary patterns due to their interpretability.

cs.LG↗

arXiv2019

Topic-aware chatbot using Recurrent Neural Networks and Nonnegative Matrix Factorization

We propose a novel model for a topic-aware chatbot by combining the traditional Recurrent Neural Network (RNN) encoder-decoder model with a topic attention layer based on Nonnegative Matrix Factorization (NMF). After learning topic vectors from an auxiliary text corpus via NMF, the decoder is trained so that it is more likely to sample response words from the most correlated topic vectors. One of the main advantages in our architecture is that the user can easily switch the NMF-learned topic vectors so that the chatbot obtains desired topic-awareness. We demonstrate our model by training on a single conversational data set which is then augmented with topic matrices learned from different auxiliary data sets. We show that our topic-aware chatbot not only outperforms the non-topic counterpart, but also that each topic-aware model qualitatively and contextually gives the most relevant answer depending on the topic of question.

cs.CL↗

arXiv2019

Large deviations and one-sided scaling limit of randomized multicolor box-ball system

The basic $κ$-color box-ball (BBS) system is an integrable cellular automaton on one dimensional lattice whose local states take $\{0,1,\cdots,κ\}$ with $0$ regarded as an empty box. The time evolution is defined by a combinatorial rule of quantum group theoretical origin, and the complete set of conserved quantities is given by a $κ$-tuple of Young diagrams. In the randomized BBS, a probability distribution on $\{0,1,\cdots,κ\}$ to independently fill the consecutive $n$ sites in the initial state induces a highly nontrivial probability measure on the $κ$-tuple of those invariant Young diagrams. In a recent work \cite{kuniba2018randomized}, their large $n$ `equilibrium shape' has been determined in terms of Schur polynomials by a Markov chain method and also by a very different approach of Thermodynamic Bethe Ansatz (TBA). In this paper, we establish a large deviations principle for the row lengths of the invariant Young diagrams. As a corollary, they are shown to converge almost surely to the equilibrium shape at an exponential rate. We also refine the TBA analysis and obtain the exact scaling form of the vacancy, the row length and the column multiplicity, which exhibit nontrivial factorization in a one-parameter specialization.

math.PR↗

arXiv2018

Randomized box-ball systems, limit shape of rigged configurations and Thermodynamic Bethe ansatz

We introduce a probability distribution on the set of states in a generalized box-ball system associated with Kirillov-Reshetikhin (KR) crystals of type $A^{(1)}_n$. Their conserved quantities induce $n$-tuple of random Young diagrams in the rigged configurations. We determine their limit shape as the system gets large by analyzing the Fermionic formula by thermodynamic Bethe ansatz. The result is expressed as a logarithmic derivative of a deformed character of the KR modules and agrees with the stationary local energy of the associated Markov process of carriers.

math-ph↗

arXiv2018

Parking on transitive unimodular graphs

Place a car independently with probability $p$ at each site of a graph. Each initially vacant site is a parking spot that can fit one car. Cars simultaneously perform independent random walks. When a car encounters an available parking spot it parks there. Other cars can still drive over the site, but cannot park there. For a large class of transitive and unimodular graphs, we show that the root is almost surely visited infinitely many times when $p \geq 1/2$, and only finitely many times otherwise.

math.PR↗

arXiv2018

Chromatic number, induced cycles, and non-separating cycles

We study two parameters obtained from the Euler characteristic by replacing the number of faces with that of induced and induced non-separating cycles. By establishing monotonicity of such parameters under certain homomorphism and edge contraction, we obtain new upper bounds on the chromatic number in terms of the number of induced cycles and the Hadwiger number in terms of the number of induced non-separating cycles. As an application, we show that a 3-connected graph with average degree $k\ge 2$ have at least $(k-1)|V|+Ck^{3}\log^{3/2}k$ induced non-separating cycles for some explicit constant $C>0$. This improves the previous best lower bound $(k-1)|V|+1$, which follows from Tutte's cycle space theorem. We also give a short proof of this theorem of Tutte.

math.CO↗

arXiv2018

Clustering in the three and four color cyclic particle systems in one dimension

We study the $κ$-color cyclic particle system on the one-dimensional integer lattice $\mathbb{Z}$, first introduced by Bramson and Griffeath in \cite{bramson1989flux}. In that paper they show that almost surely, every site changes its color infinitely often if $κ\in \{3,4\}$ and only finitely many times if $κ\ge 5$. In addition, they conjecture that for $κ\in \{3,4\}$ the system clusters, that is, for any pair of sites $x,y$, with probability tending to 1 as $t\to\infty$, $x$ and $y$ have the same color at time $t$. Here we prove that conjecture.

math.PR↗

arXiv2017

Global synchronization of pulse-coupled oscillators on trees

Consider a distributed network on a finite simple graph $G=(V,E)$ with diameter $d$ and maximum degree $Δ$, where each node has a phase oscillator revolving on $S^{1}=\mathbb{R}/\mathbb{Z}$ with unit speed. Pulse-coupling is a class of distributed time evolution rule for such networked phase oscillators inspired by biological oscillators, which depends only upon event-triggered local pulse communications. In this paper, we propose a novel inhibitory pulse-coupling and prove that arbitrary phase configuration on $G$ synchronizes by time $51d$ if $G$ is a tree and $Δ\le 3$. We extend this pulse-coupling by letting each oscillator throttle the input according to an auxiliary state variable. We show that the resulting adaptive pulse-coupling synchronizes arbitrary initial configuration on $G$ by time $83d$ if $G$ is a tree. As an application, we obtain a universal randomized distributed clock synchronization algorithm, which uses $O(\log Δ)$ memory per node and converges on any $G$ with expected worst case running time of $O(|V|+(d^{5}+Δ^{2})\log |V|)$.

math.OC↗

arXiv2017

Persistence of sums of correlated increments and clustering in cellular automata

We consider sums of increments given by a functional of a stationary Markov chain. Letting $T$ be the first return time of the partial sums process to $(-\infty,0]$, under general assumptions, we determine the asymptotic behavior of the survival probability, $\mathbb{P}(T\ge t)\sim Ct^{-1/2}$ for an explicit constant $C$. Our analysis is based on a connection between the survival probability and the running maximum of the time-reversed process, and relies on a functional central limit theorem for Markov chains. Our result extends the classic theorem of Sparre Anderson on sums of mean zero and independent increments to the case of correlated increments. As applications, we recover known clustering results for the 3-color cyclic cellular automaton and the Greenberg-Hastings model in one dimension, and we prove a new clustering result for the 3-color firefly cellular automaton.

math.PR↗

arXiv2016

Limiting behavior of 3-color excitable media on arbitrary graphs

Fix a simple graph $G=(V,E)$ and choose a random initial 3-coloring of vertices drawn from a uniform product measure. The 3-color cycle cellular automaton is a process in which at each discrete time step in parallel, every vertex with color $i$ advances to the successor color $(i+1)$ mod 3 if in contact with a neighbor with the successor color, and otherwise retains the same color. In the Greenberg-Hastings Model, the same update rule applies only to color 0, while other two colors automatically advance. The limiting behavior of these processes has been studied mainly on the integer lattices. In this paper, we introduce a monotone comparison process defined on the universal covering space of the underlying graph, and characterize the limiting behavior of these processes on arbitrary connected graphs. In particular, we establish a phase transition on the Erdös-Rényi random graph. On infinite trees, we connect the rate of color change to the cloud speed of an associated tree-indexed walk. We give estimates of the cloud speed by generalizing known results to trees with leaves.

math.PR↗

arXiv2015

Synchronization of finite-state pulse-coupled oscillators

We propose a novel generalized cellular automaton(GCA) model for discrete-time pulse-coupled oscillators and study the emergence of synchrony. Given a finite simple graph and an integer $n\ge 3$, each vertex is an identical oscillator of period $n$ with the following weak coupling along the edges: each oscillator inhibits its phase update if it has at least one neighboring oscillator at a particular "blinking" state and if its state is ahead of this blinking state. We obtain conditions on initial configurations and on network topologies for which states of all vertices eventually synchronize. We show that our GCA model synchronizes arbitrary initial configurations on paths, trees, and with random perturbation, any connected graph. In particular, our main result is the following local-global principle for tree networks: for $n\in \{3,4,5,6\}$, any $n$-periodic network on a tree synchronizes arbitrary initial configuration if and only if the maximum degree of the tree is less than the period $n$.

eess.SY↗

arXiv2012

A Note on Graph Characteristics and Hadwiger's Conjecture

This is a note on three graph parameters motivated by the Euler-Poincare characteristic for simplicial complex. We show those three graph parameters of a given connected graph $G$ is greater than or equal to that of the complete graph with $\max(h(G),χ(G))$ vertices. This will yield three different simultaneous upperbounds of both the hadwiger number and chromatic number by means of the number of particular types of induced subgraphs. Some applications to Hadwiger's Conjecture is also discussed.

math.CO↗