arXiv subjects

Jason Li

Publications and source records attributed to Jason Li.

At least 127 records · Page 7Linked to original sources

arXiv2020

A Lightweight Algorithm to Uncover Deep Relationships in Data Tables

Many data we collect today are in tabular form, with rows as records and columns as attributes associated with each record. Understanding the structural relationship in tabular data can greatly facilitate the data science process. Traditionally, much of this relational information is stored in table schema and maintained by its creators, usually domain experts. In this paper, we develop automated methods to uncover deep relationships in a single data table without expert or domain knowledge. Our method can decompose a data table into layers of smaller tables, revealing its deep structure. The key to our approach is a computationally lightweight forward addition algorithm that we developed to recursively extract the functional dependencies between table columns that are scalable to tables with many columns. With our solution, data scientists will be provided with automatically generated, data-driven insights when exploring new data sets.

cs.DS↗

arXiv2020

Cross-Language Transfer Learning, Continuous Learning, and Domain Adaptation for End-to-End Automatic Speech Recognition

In this paper, we demonstrate the efficacy of transfer learning and continuous learning for various automatic speech recognition (ASR) tasks. We start with a pre-trained English ASR model and show that transfer learning can be effectively and easily performed on: (1) different English accents, (2) different languages (German, Spanish and Russian) and (3) application-specific domains. Our experiments demonstrate that in all three cases, transfer learning from a good base model has higher accuracy than a model trained from scratch. It is preferred to fine-tune large models than small pre-trained models, even if the dataset for fine-tuning is small. Moreover, transfer learning significantly speeds up convergence for both very small and very large target datasets.

eess.AS↗

arXiv2020

A Deterministic Algorithm for Balanced Cut with Applications to Dynamic Connectivity, Flows, and Beyond

We consider the classical Minimum Balanced Cut problem: given a graph $G$, compute a partition of its vertices into two subsets of roughly equal volume, while minimizing the number of edges connecting the subsets. We present the first {\em deterministic, almost-linear time} approximation algorithm for this problem. Specifically, our algorithm, given an $n$-vertex $m$-edge graph $G$ and any parameter $1\leq r\leq O(\log n)$, computes a $(\log m)^{r^2}$-approximation for Minimum Balanced Cut on $G$, in time $O\left ( m^{1+O(1/r)+o(1)}\cdot (\log m)^{O(r^2)}\right )$. In particular, we obtain a $(\log m)^{1/ε}$-approximation in time $m^{1+O(1/\sqrtε)}$ for any constant $ε$, and a $(\log m)^{f(m)}$-approximation in time $m^{1+o(1)}$, for any slowly growing function $m$. We obtain deterministic algorithms with similar guarantees for the Sparsest Cut and the Lowest-Conductance Cut problems. Our algorithm for the Minimum Balanced Cut problem in fact provides a stronger guarantee: it either returns a balanced cut whose value is close to a given target value, or it certifies that such a cut does not exist by exhibiting a large subgraph of $G$ that has high conductance. We use this algorithm to obtain deterministic algorithms for dynamic connectivity and minimum spanning forest, whose worst-case update time on an $n$-vertex graph is $n^{o(1)}$, thus resolving a major open problem in the area of dynamic graph algorithms. Our work also implies deterministic algorithms for a host of additional problems, whose time complexities match, up to subpolynomial in $n$ factors, those of known randomized algorithms. The implications include almost-linear time deterministic algorithms for solving Laplacian systems and for approximating maximum flows in undirected graphs.

cs.DS↗

arXiv2020

Cycle Text-To-Image GAN with BERT

We explore novel approaches to the task of image generation from their respective captions, building on state-of-the-art GAN architectures. Particularly, we baseline our models with the Attention-based GANs that learn attention mappings from words to image features. To better capture the features of the descriptions, we then built a novel cyclic design that learns an inverse function to maps the image back to original caption. Additionally, we incorporated recently developed BERT pretrained word embeddings as our initial text featurizer and observe a noticeable improvement in qualitative and quantitative performance compared to the Attention GAN baseline.

cs.CV↗

arXiv2020

Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep Networks

We propose NovoGrad, an adaptive stochastic gradient descent method with layer-wise gradient normalization and decoupled weight decay. In our experiments on neural networks for image classification, speech recognition, machine translation, and language modeling, it performs on par or better than well tuned SGD with momentum and Adam or AdamW. Additionally, NovoGrad (1) is robust to the choice of learning rate and weight initialization, (2) works well in a large batch setting, and (3) has two times smaller memory footprint than Adam.

cs.LG↗

arXiv2019

Planar Diameter via Metric Compression

We develop a new approach for distributed distance computation in planar graphs that is based on a variant of the metric compression problem recently introduced by Abboud et al. [SODA'18]. One of our key technical contributions is in providing a compression scheme that encodes all $S \times T$ distances using $\widetilde{O}(|S|\cdot poly(D)+|T|)$ bits for unweighted graphs with diameter $D$. This significantly improves the state of the art of $\widetilde{O}(|S|\cdot 2^{D}+|T| \cdot D)$ bits. We also consider an approximate version of the problem for \emph{weighted} graphs, where the goal is to encode $(1+ε)$ approximation of the $S \times T$ distances. At the heart of this compact compression scheme lies a VC-dimension type argument on planar graphs. This efficient compression scheme leads to several improvements and simplifications in the setting of diameter computation, most notably in the distributed setting: - There is an $\widetilde{O}(D^5)$-round randomized distributed algorithm for computing the diameter in planar graphs, w.h.p. - There is an $\widetilde{O}(D^3)+ poly(\log n/ε)\cdot D^2$-round randomized distributed algorithm for computing an $(1+ε)$ approximation of the diameter in weighted graphs with polynomially bounded weights, w.h.p. No sublinear round algorithms were known for these problems before. These distributed constructions are based on a new recursive graph decomposition that preserves the (unweighted) diameter of each of the subgraphs up to a logarithmic term. Using this decomposition, we also get an \emph{exact} SSSP tree computation within $\widetilde{O}(D^2)$ rounds.

cs.DS↗

arXiv2019

The Karger-Stein Algorithm is Optimal for $k$-cut

In the $k$-cut problem, we are given an edge-weighted graph and want to find the least-weight set of edges whose deletion breaks the graph into $k$ connected components. Algorithms due to Karger-Stein and Thorup showed how to find such a minimum $k$-cut in time approximately $O(n^{2k-2})$. The best lower bounds come from conjectures about the solvability of the $k$-clique problem and a reduction from $k$-clique to $k$-cut, and show that solving $k$-cut is likely to require time $Ω(n^k)$. Our recent results have given special-purpose algorithms that solve the problem in time $n^{1.98k + O(1)}$, and ones that have better performance for special classes of graphs (e.g., for small integer weights). In this work, we resolve the problem for general graphs, by showing that for any fixed $k \geq 2$, the Karger-Stein algorithm outputs any fixed minimum $k$-cut with probability at least $\hat{O}(n^{-k})$, where $\hat{O}(\cdot)$ hides a $2^{O(\ln \ln n)^2}$ factor. This also gives an extremal bound of $\hat{O}(n^k)$ on the number of minimum $k$-cuts in an $n$-vertex graph and an algorithm to compute a minimum $k$-cut in similar runtime. Both are tight up to $\hat{O}(1)$ factors. The first main ingredient in our result is a fine-grained analysis of how the graph shrinks---and how the average degree evolves---under the Karger-Stein process. The second ingredient is an extremal result bounding the number of cuts of size at most $(2-δ) OPT/k$, using the Sunflower lemma.

cs.DS↗

arXiv2019

Detecting Feedback Vertex Sets of Size $k$ in $O^\star(2.7^k)$ Time

In the Feedback Vertex Set problem, one is given an undirected graph $G$ and an integer $k$, and one needs to determine whether there exists a set of $k$ vertices that intersects all cycles of $G$ (a so-called feedback vertex set). Feedback Vertex Set is one of the most central problems in parameterized complexity: It served as an excellent test bed for many important algorithmic techniques in the field such as Iterative Compression~[Guo et al. (JCSS'06)], Randomized Branching~[Becker et al. (J. Artif. Intell. Res'00)] and Cut\&Count~[Cygan et al. (FOCS'11)]. In particular, there has been a long race for the smallest dependence $f(k)$ in run times of the type $O^\star(f(k))$, where the $O^\star$ notation omits factors polynomial in $n$. This race seemed to be run in 2011, when a randomized algorithm $O^\star(3^k)$ time algorithm based on Cut\&Count was introduced. In this work, we show the contrary and give a $O^\star(2.7^k)$ time randomized algorithm. Our algorithm combines all mentioned techniques with substantial new ideas: First, we show that, given a feedback vertex set of size $k$ of bounded average degree, a tree decomposition of width $(1-Ω(1))k$ can be found in polynomial time. Second, we give a randomized branching strategy inspired by the one from~[Becker et al. (J. Artif. Intell. Res'00)] to reduce to the aforementioned bounded average degree setting. Third, we obtain significant run time improvements by employing fast matrix multiplication.

cs.DS↗

arXiv2019

Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens

Mellotron is a multispeaker voice synthesis model based on Tacotron 2 GST that can make a voice emote and sing without emotive or singing training data. By explicitly conditioning on rhythm and continuous pitch contours from an audio signal or music score, Mellotron is able to generate speech in a variety of styles ranging from read speech to expressive speech, from slow drawls to rap and from monotonous voice to singing voice. Unlike other methods, we train Mellotron using only read speech data without alignments between text and audio. We evaluate our models using the LJSpeech and LibriTTS datasets. We provide F0 Frame Errors and synthesized samples that include style transfer from other speakers, singers and styles not seen during training, procedural manipulation of rhythm and pitch and choir synthesis.

cs.SD↗

arXiv2019

QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions

We propose a new end-to-end neural acoustic model for automatic speech recognition. The model is composed of multiple blocks with residual connections between them. Each block consists of one or more modules with 1D time-channel separable convolutional layers, batch normalization, and ReLU layers. It is trained with CTC loss. The proposed network achieves near state-of-the-art accuracy on LibriSpeech and Wall Street Journal, while having fewer parameters than all competing models. We also demonstrate that this model can be effectively fine-tuned on new datasets.

eess.AS↗

arXiv2019

Deterministic Graph Cuts in Subquadratic Time: Sparse, Balanced, and k-Vertex

We study deterministic algorithms for computing graph cuts, with focus on two fundamental problems: balanced sparse cut and $k$-vertex connectivity for small $k$ ($k=O(\polylog n)$). Both problems can be solved in near-linear time with randomized algorithms, but their previous deterministic counterparts take at least quadratic time. In this paper, we break this bound for both problems. Interestingly, achieving this for one problem crucially relies on doing so for the other. In particular, via a divide-and-conquer argument, a variant of the cut-matching game by [Khandekar et al.`07], and the local vertex connectivity algorithm of [Nanongkai et al. STOC'19], we give a subquadratic time algorithm for $k$-vertex connectivity using a subquadratic time algorithm for computing balanced sparse cuts on sparse graphs. To achieve the latter, we improve the previously best $mn$ bound for approximating balanced sparse cut for the whole range of $m$. This starts from (1) breaking the $n^3$ barrier on dense graphs to $n^{ω+ o(1)}$ (where $ω< 2.372$) using the the PageRank matrix, but without explicitly sweeping to find sparse cuts; to (2) getting the $\tilde O(m^{1.58})$ bound by combining the $J$-trees by [Madry FOCS `10] with the $n^{ω+ o(1)}$ bound above, and finally; to (3) getting the $m^{1.5 + o(1)}$ bound by recursively invoking the second bound in conjunction with expander-based graph sparsification. Interestingly, our final $m^{1.5 + o(1)}$ bound lands at a natural stopping point in the sense that polynomially breaking it would lead to a breakthrough for the dynamic connectivity problem.

cs.DS↗

arXiv2019

Faster Minimum k-cut of a Simple Graph

We consider the (exact, minimum) $k$-cut problem: given a graph and an integer $k$, delete a minimum-weight set of edges so that the remaining graph has at least $k$ connected components. This problem is a natural generalization of the global minimum cut problem, where the goal is to break the graph into $k=2$ pieces. Our main result is a (combinatorial) $k$-cut algorithm on simple graphs that runs in $n^{(1+o(1))k}$ time for any constant $k$, improving upon the previously best $n^{(2ω/3+o(1))k}$ time algorithm of Gupta et al.~[FOCS'18] and the previously best $n^{(1.981+o(1))k}$ time combinatorial algorithm of Gupta et al.~[STOC'19]. For combinatorial algorithms, this algorithm is optimal up to $o(1)$ factors assuming recent hardness conjectures: we show by a straightforward reduction that $k$-cut on even a simple graph is as hard as $(k-1)$-clique, establishing a lower bound of $n^{(1-o(1))k}$ for $k$-cut. This settles, up to lower-order factors, the complexity of $k$-cut on a simple graph for combinatorial algorithms.

cs.DS↗

arXiv2019

NeMo: a toolkit for building AI applications using Neural Modules

NeMo (Neural Modules) is a Python framework-agnostic toolkit for creating AI applications through re-usability, abstraction, and composition. NeMo is built around neural modules, conceptual blocks of neural networks that take typed inputs and produce typed outputs. Such modules typically represent data layers, encoders, decoders, language models, loss functions, or methods of combining activations. NeMo makes it easy to combine and re-use these building blocks while providing a level of semantic correctness checking via its neural type system. The toolkit comes with extendable collections of pre-built modules for automatic speech recognition and natural language processing. Furthermore, NeMo provides built-in support for distributed training and mixed precision on latest NVIDIA GPUs. NeMo is open-source https://github.com/NVIDIA/NeMo

cs.LG↗

arXiv2019

Jasper: An End-to-End Convolutional Neural Acoustic Model

In this paper, we report state-of-the-art results on LibriSpeech among end-to-end speech recognition models without any external training data. Our model, Jasper, uses only 1D convolutions, batch normalization, ReLU, dropout, and residual connections. To improve training, we further introduce a new layer-wise optimizer called NovoGrad. Through experiments, we demonstrate that the proposed deep architecture performs as well or better than more complex choices. Our deepest Jasper variant uses 54 convolutional layers. With this architecture, we achieve 2.95% WER using a beam-search decoder with an external neural language model and 3.86% WER with a greedy decoder on LibriSpeech test-clean. We also report competitive results on the Wall Street Journal and the Hub5'00 conversational evaluation datasets.

eess.AS↗

arXiv2019

Localized Triggering of the Insulator-Metal Transition in VO2 using a Single Carbon Nanotube

Vanadium dioxide (VO2) has been widely studied for its rich physics and potential applications, undergoing a prominent insulator-metal transition (IMT) near room temperature. The transition mechanism remains highly debated, and little is known about the IMT at nanoscale dimensions. To shed light on this problem, here we use ~1 nm wide carbon nanotube (CNT) heaters to trigger the IMT in VO2. Single metallic CNTs switch the adjacent VO2 at less than half the voltage and power required by control devices without a CNT, with switching power as low as ~85 $μW$ at 300 nm device lengths. We also obtain potential and temperature maps of devices during operation using Kelvin Probe Microscopy (KPM) and Scanning Thermal Microscopy (SThM). Comparing these with three-dimensional electrothermal simulations, we find that the local heating of the VO2 by the CNT play a key role in the IMT. These results demonstrate the ability to trigger IMT in VO2 using nanoscale heaters, and highlight the significance of thermal engineering to improve device behaviour.

cond-mat.mtrl-sci↗

arXiv2019

The Number of Minimum $k$-Cuts: Improving the Karger-Stein Bound

Given an edge-weighted graph, how many minimum $k$-cuts can it have? This is a fundamental question in the intersection of algorithms, extremal combinatorics, and graph theory. It is particularly interesting in that the best known bounds are algorithmic: they stem from algorithms that compute the minimum $k$-cut. In 1994, Karger and Stein obtained a randomized contraction algorithm that finds a minimum $k$-cut in $O(n^{(2-o(1))k})$ time. It can also enumerate all such $k$-cuts in the same running time, establishing a corresponding extremal bound of $O(n^{(2-o(1))k})$. Since then, the algorithmic side of the minimum $k$-cut problem has seen much progress, leading to a deterministic algorithm based on a tree packing result of Thorup, which enumerates all minimum $k$-cuts in the same asymptotic running time, and gives an alternate proof of the $O(n^{(2-o(1))k})$ bound. However, beating the Karger--Stein bound, even for computing a single minimum $k$-cut, has remained out of reach. In this paper, we give an algorithm to enumerate all minimum $k$-cuts in $O(n^{(1.981+o(1))k})$ time, breaking the algorithmic and extremal barriers for enumerating minimum $k$-cuts. To obtain our result, we combine ideas from both the Karger--Stein and Thorup results, and draw a novel connection between minimum $k$-cut and extremal set theory. In particular, we give and use tighter bounds on the size of set systems with bounded dual VC-dimension, which may be of independent interest.

cs.DS↗

arXiv2019

Tight FPT Approximations for $k$-Median and $k$-Means

We investigate the fine-grained complexity of approximating the classical $k$-median / $k$-means clustering problems in general metric spaces. We show how to improve the approximation factors to $(1+2/e+\varepsilon)$ and $(1+8/e+\varepsilon)$ respectively, using algorithms that run in fixed-parameter time. Moreover, we show that we cannot do better in FPT time, modulo recent complexity-theoretic conjectures.

cs.DS↗

arXiv2019

Distributed Computation in Node-Capacitated Networks

In this paper, we study distributed graph algorithms in networks in which the nodes have a limited communication capacity. Many distributed systems are built on top of an underlying networking infrastructure, for example by using a virtual communication topology known as an overlay network. Although this underlying network might allow each node to directly communicate with a large number of other nodes, the amount of communication that a node can perform in a fixed amount of time is typically much more limited. We introduce the Node-Capacitated Clique model as an abstract communication model, which allows us to study the effect of nodes having limited communication capacity on the complexity of distributed graph computations. In this model, the $n$ nodes of a network are connected as a clique and communicate in synchronous rounds. In each round, every node can exchange messages of $O(\log n)$ bits with at most $O(\log n)$ other nodes. When solving a graph problem, the input graph $G$ is defined on the same set of $n$ nodes, where each node knows which other nodes are its neighbors in $G$. To initiate research on the Node-Capacitated Clique model, we present distributed algorithms for the Minimum Spanning Tree (MST), BFS Tree, Maximal Independent Set, Maximal Matching, and Vertex Coloring problems. We show that even with only $O(\log n)$ concurrent interactions per node, the MST problem can still be solved in polylogarithmic time. In all other cases, the runtime of our algorithms depends linearly on the arboricity of $G$, which is a constant for many important graph families such as planar graphs.

cs.DC↗