arXiv subjects

Rasul Tutunov

Publications and source records attributed to Rasul Tutunov.

33 records · Page 2Linked to original sources

arXiv2021

BOiLS: Bayesian Optimisation for Logic Synthesis

Optimising the quality-of-results (QoR) of circuits during logic synthesis is a formidable challenge necessitating the exploration of exponentially sized search spaces. While expert-designed operations aid in uncovering effective sequences, the increase in complexity of logic circuits favours automated procedures. Inspired by the successes of machine learning, researchers adapted deep learning and reinforcement learning to logic synthesis applications. However successful, those techniques suffer from high sample complexities preventing widespread adoption. To enable efficient and scalable solutions, we propose BOiLS, the first algorithm adapting modern Bayesian optimisation to navigate the space of synthesis operations. BOiLS requires no human intervention and effectively trades-off exploration versus exploitation through novel Gaussian process kernels and trust-region constrained acquisitions. In a set of experiments on EPFL benchmarks, we demonstrate BOiLS's superior performance compared to state-of-the-art in terms of both sample efficiency and QoR values.

cs.LG↗

arXiv2021

High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning

We introduce a method combining variational autoencoders (VAEs) and deep metric learning to perform Bayesian optimisation (BO) over high-dimensional and structured input spaces. By adapting ideas from deep metric learning, we use label guidance from the blackbox function to structure the VAE latent space, facilitating the Gaussian process fit and yielding improved BO performance. Importantly for BO problem settings, our method operates in semi-supervised regimes where only few labelled data points are available. We run experiments on three real-world tasks, achieving state-of-the-art results on the penalised logP molecule generation benchmark using just 3% of the labelled data required by previous approaches. As a theoretical contribution, we present a proof of vanishing regret for VAE BO.

cs.LG↗

arXiv2021

Efficient Semi-Implicit Variational Inference

In this paper, we propose CI-VI an efficient and scalable solver for semi-implicit variational inference (SIVI). Our method, first, maps SIVI's evidence lower bound (ELBO) to a form involving a nonlinear functional nesting of expected values and then develops a rigorous optimiser capable of correctly handling bias inherent to nonlinear nested expectations using an extrapolation-smoothing mechanism coupled with gradient sketching. Our theoretical results demonstrate convergence to a stationary point of the ELBO in general non-convex settings typically arising when using deep network models and an order of $O(t^{-\frac{4}{5}})$ gradient-bias-vanishing rate. We believe these results generalise beyond the specific nesting arising from SIVI to other forms. Finally, in a set of experiments, we demonstrate the effectiveness of our algorithm in approximating complex posteriors on various data-sets including those from natural language processing.

cs.LG↗

arXiv2020

Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?

Bayesian optimisation presents a sample-efficient methodology for global optimisation. Within this framework, a crucial performance-determining subroutine is the maximisation of the acquisition function, a task complicated by the fact that acquisition functions tend to be non-convex and thus nontrivial to optimise. In this paper, we undertake a comprehensive empirical study of approaches to maximise the acquisition function. Additionally, by deriving novel, yet mathematically equivalent, compositional forms for popular acquisition functions, we recast the maximisation task as a compositional optimisation problem, allowing us to benefit from the extensive literature in this field. We highlight the empirical advantages of the compositional approach to acquisition function maximisation across 3958 individual experiments comprising synthetic optimisation tasks as well as tasks from Bayesmark. Given the generality of the acquisition function maximisation subroutine, we posit that the adoption of compositional optimisers has the potential to yield performance improvements across all domains in which Bayesian optimisation is currently being applied.

cs.LG↗

arXiv2020

Compositional ADAM: An Adaptive Compositional Solver

In this paper, we present C-ADAM, the first adaptive solver for compositional problems involving a non-linear functional nesting of expected values. We proof that C-ADAM converges to a stationary point in $\mathcal{O}(δ^{-2.25})$ with $δ$ being a precision parameter. Moreover, we demonstrate the importance of our results by bridging, for the first time, model-agnostic meta-learning (MAML) and compositional optimisation showing fastest known rates for deep network adaptation to-date. Finally, we validate our findings in a set of experiments from portfolio optimisation and meta-learning. Our results manifest significant sample complexity reductions compared to both standard and compositional solvers.

cs.LG↗

arXiv2020

$α^α$-Rank: Practically Scaling $α$-Rank through Stochastic Optimisation

Recently, $α$-Rank, a graph-based algorithm, has been proposed as a solution to ranking joint policy profiles in large scale multi-agent systems. $α$-Rank claimed tractability through a polynomial time implementation with respect to the total number of pure strategy profiles. Here, we note that inputs to the algorithm were not clearly specified in the original presentation; as such, we deem complexity claims as not grounded, and conjecture solving $α$-Rank is NP-hard. The authors of $α$-Rank suggested that the input to $α$-Rank can be an exponentially-sized payoff matrix; a claim promised to be clarified in subsequent manuscripts. Even though $α$-Rank exhibits a polynomial-time solution with respect to such an input, we further reflect additional critical problems. We demonstrate that due to the need of constructing an exponentially large Markov chain, $α$-Rank is infeasible beyond a small finite number of agents. We ground these claims by adopting amount of dollars spent as a non-refutable evaluation metric. Realising such scalability issue, we present a stochastic implementation of $α$-Rank with a double oracle mechanism allowing for reductions in joint strategy spaces. Our method, $α^α$-Rank, does not need to save exponentially-large transition matrix, and can terminate early under required precision. Although theoretically our method exhibits similar worst-case complexity guarantees compared to $α$-Rank, it allows us, for the first time, to practically conduct large-scale multi-agent evaluations. On $10^4 \times 10^4$ random matrices, we achieve $1000x$ speed reduction. Furthermore, we also show successful results on large joint strategy profiles with a maximum size in the order of $\mathcal{O}(2^{25})$ ($\approx 33$ million joint strategies) -- a setting not evaluable using $α$-Rank with reasonable computational budget.

cs.MA↗

arXiv2019

Derivative-Free & Order-Robust Optimisation

In this paper, we formalise order-robust optimisation as an instance of online learning minimising simple regret, and propose Vroom, a zero'th order optimisation algorithm capable of achieving vanishing regret in non-stationary environments, while recovering favorable rates under stochastic reward-generating processes. Our results are the first to target simple regret definitions in adversarial scenarios unveiling a challenge that has been rarely considered in prior work.

cs.LG↗

arXiv2019

Graph Attention Memory for Visual Navigation

Visual navigation in complex environments is inefficient with traditional reactive policy or general-purposed recurrent policy. To address the long-term memory issue, this paper proposes a graph attention memory (GAM) architecture consisting of memory construction module, graph attention module and control module. The memory construction module builds the topological graph based on supervised learning by taking the exploration prior. Then, guided attention features are extracted with the graph attention module. Finally, the deep reinforcement learning based control module makes decisions based on visual observations and guided attention features. Detailed convergence analysis of GAM is presented in this paper. We evaluate GAM-based navigation system in two complex 3D environments. Experimental results show that the GAM-based navigation system significantly improves learning efficiency and outperforms all baselines in average success rate.

cs.CV↗

arXiv2016

An Exact Distributed Newton Method for Reinforcement Learning

In this paper, we propose a distributed second- order method for reinforcement learning. Our approach is the fastest in literature so-far as it outperforms state-of-the-art methods, including ADMM, by significant margins. We achieve this by exploiting the sparsity pattern of the dual Hessian and transforming the problem of computing the Newton direction to one of solving a sequence of symmetric diagonally dominant system of equations. We validate the above claim both theoretically and empirically. On the theoretical side, we prove that similar to exact Newton, our algorithm exhibits super-linear convergence within a neighborhood of the optimal solution. Empirically, we demonstrate the superiority of this new method on a set of benchmark reinforcement learning tasks.

math.OC↗

arXiv2016

A Distributed Newton Method for Large Scale Consensus Optimization

In this paper, we propose a distributed Newton method for consensus optimization. Our approach outperforms state-of-the-art methods, including ADMM. The key idea is to exploit the sparsity of the dual Hessian and recast the computation of the Newton step as one of efficiently solving symmetric diagonally dominant linear equations. We validate our algorithm both theoretically and empirically. On the theory side, we demonstrate that our algorithm exhibits superlinear convergence within a neighborhood of optimality. Empirically, we show the superiority of this new method on a variety of machine learning problems. The proposed approach is scalable to very large problems and has a low communication overhead.

cs.DC↗

arXiv2015

Distributed SDDM Solvers: Theory & Applications

In this paper, we propose distributed solvers for systems of linear equations given by symmetric diagonally dominant M-matrices based on the parallel solver of Spielman and Peng. We propose two versions of the solvers, where in the first, full communication in the network is required, while in the second communication is restricted to the R-Hop neighborhood between nodes for some $R \geq 1$. We rigorously analyze the convergence and convergence rates of our solvers, showing that our methods are capable of outperforming state-of-the-art techniques. Having developed such solvers, we then contribute by proposing an accurate distributed Newton method for network flow optimization. Exploiting the sparsity pattern of the dual Hessian, we propose a Newton method for network flow optimization that is both faster and more accurate than state-of-the-art techniques. Our method utilizes the distributed SDDM solvers for determining the Newton direction up to any arbitrary precision $ε>0$. We analyze the properties of our algorithm and show superlinear convergence within a neighborhood of the optimal. Finally, in a set of experiments conducted on randomly generated and barbell networks, we demonstrate that our approach is capable of significantly outperforming state-of-the-art techniques.

math.OC↗

arXiv2015

Safe Policy Search for Lifelong Reinforcement Learning with Sublinear Regret

Lifelong reinforcement learning provides a promising framework for developing versatile agents that can accumulate knowledge over a lifetime of experience and rapidly learn new tasks by building upon prior knowledge. However, current lifelong learning methods exhibit non-vanishing regret as the amount of experience increases and include limitations that can lead to suboptimal or unsafe control policies. To address these issues, we develop a lifelong policy gradient learner that operates in an adversarial set- ting to learn multiple tasks online while enforcing safety constraints on the learned policies. We demonstrate, for the first time, sublinear regret for lifelong policy search, and validate our algorithm on several benchmark dynamical systems and an application to quadrotor control.

cs.LG↗

arXiv2015

Fast, Accurate Second Order Methods for Network Optimization

Dual descent methods are commonly used to solve network flow optimization problems, since their implementation can be distributed over the network. These algorithms, however, often exhibit slow convergence rates. Approximate Newton methods which compute descent directions locally have been proposed as alternatives to accelerate the convergence rates of conventional dual descent. The effectiveness of these methods, is limited by the accuracy of such approximations. In this paper, we propose an efficient and accurate distributed second order method for network flow problems. The proposed approach utilizes the sparsity pattern of the dual Hessian to approximate the the Newton direction using a novel distributed solver for symmetric diagonally dominant linear equations. Our solver is based on a distributed implementation of a recent parallel solver of Spielman and Peng (2014). We analyze the properties of the proposed algorithm and show that, similar to conventional Newton methods, superlinear convergence within a neighbor- hood of the optimal value is attained. We finally demonstrate the effectiveness of the approach in a set of experiments on randomly generated networks.

math.OC↗

arXiv2015

A Fast Distributed Solver for Symmetric Diagonally Dominant Linear Equations

In this paper, we propose a fast distributed solver for linear equations given by symmetric diagonally dominant M-Matrices. Our approach is based on a distributed implementation of the parallel solver of Spielman and Peng by considering a specific approximated inverse chain which can be computed efficiently in a distributed fashion. Representing the system of equations by a graph $\mathbb{G}$, the proposed distributed algorithm is capable of attaining $ε$-close solutions (for arbitrary $ε$) in time proportional to $n^{3}$ (number of nodes in $\mathbb{G}$), $α$ (upper bound on the size of the R-Hop neighborhood), and $\frac{{W}_{max}}{{W}_{min}}$ (maximum and minimum weight of edges in $\mathbb{G}$).

cs.DC↗

arXiv2014

On the Degree Distribution of Pólya Urn Graph Processes

This paper presents a tighter bound on the degree distribution of arbitrary Pólya urn graph processes, proving that the proportion of vertices with degree $d$ obeys a power-law distribution $P(d) \propto d^{-γ}$ for $d \leq n^{\frac{1}{6}-ε}$ for any $ε> 0$, where $n$ represents the number of vertices in the network. Previous work by Bollobás et al. formalized the well-known preferential attachment model of Barabási and Albert, and showed that the power-law distribution held for $d \leq n^{\frac{1}{15}}$ with $γ= 3$. Our revised bound represents a significant improvement over existing models of degree distribution in scale-free networks, where its tightness is restricted by the Azuma-Hoeffding concentration inequality for martingales. We achieve this tighter bound through a careful analysis of the first set of vertices in the network generation process, and show that the newly acquired is at the edge of exhausting Bollobás model in the sense that the degree expectation breaks down for other powers.

math.PR↗