arXiv subjects

Ji Liu

Publications and source records attributed to Ji Liu.

At least 253 records · Page 14Linked to original sources

arXiv2019

Distributed Learning over Unreliable Networks

Most of today's distributed machine learning systems assume {\em reliable networks}: whenever two machines exchange information (e.g., gradients or models), the network should guarantee the delivery of the message. At the same time, recent work exhibits the impressive tolerance of machine learning algorithms to errors or noise arising from relaxed communication or synchronization. In this paper, we connect these two trends, and consider the following question: {\em Can we design machine learning systems that are tolerant to network unreliability during training?} With this motivation, we focus on a theoretical problem of independent interest---given a standard distributed parameter server architecture, if every communication between the worker and the server has a non-zero probability $p$ of being dropped, does there exist an algorithm that still converges, and at what speed? The technical contribution of this paper is a novel theoretical analysis proving that distributed learning over unreliable network can achieve comparable convergence rate to centralized or distributed learning over reliable networks. Further, we prove that the influence of the packet drop rate diminishes with the growth of the number of \textcolor{black}{parameter servers}. We map this theoretical result onto a real-world scenario, training deep neural networks over an unreliable network layer, and conduct network simulation to validate the system improvement by allowing the networks to be unreliable.

cs.DC↗

arXiv2019

Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization

Asynchronous parallel implementations of stochastic gradient (SG) have been broadly used in solving deep neural network and received many successes in practice recently. However, existing theories cannot explain their convergence and speedup properties, mainly due to the nonconvexity of most deep learning formulations and the asynchronous parallel mechanism. To fill the gaps in theory and provide theoretical supports, this paper studies two asynchronous parallel implementations of SG: one is on the computer network and the other is on the shared memory system. We establish an ergodic convergence rate $O(1/\sqrt{K})$ for both algorithms and prove that the linear speedup is achievable if the number of workers is bounded by $\sqrt{K}$ ($K$ is the total number of iterations). Our results generalize and improve existing analysis for convex minimization.

math.OC↗

arXiv2019

ECC: Platform-Independent Energy-Constrained Deep Neural Network Compression via a Bilinear Regression Model

Many DNN-enabled vision applications constantly operate under severe energy constraints such as unmanned aerial vehicles, Augmented Reality headsets, and smartphones. Designing DNNs that can meet a stringent energy budget is becoming increasingly important. This paper proposes ECC, a framework that compresses DNNs to meet a given energy constraint while minimizing accuracy loss. The key idea of ECC is to model the DNN energy consumption via a novel bilinear regression function. The energy estimate model allows us to formulate DNN compression as a constrained optimization that minimizes the DNN loss function over the energy constraint. The optimization problem, however, has nontrivial constraints. Therefore, existing deep learning solvers do not apply directly. We propose an optimization algorithm that combines the essence of the Alternating Direction Method of Multipliers (ADMM) framework with gradient-based learning algorithms. The algorithm decomposes the original constrained optimization into several subproblems that are solved iteratively and efficiently. ECC is also portable across different hardware platforms without requiring hardware knowledge. Experiments show that ECC achieves higher accuracy under the same or lower energy budget compared to state-of-the-art resource-constrained DNN compression techniques.

cs.LG↗

arXiv2019

A Distributed Observer for a Discrete-Time Linear System

A simply structured distributed observer is described for estimating the state of a discrete-time, jointly observable, input-free, linear system whose sensed outputs are distributed across a time-varying network. It is explained how to construct the local estimators which comprise the observer so that their state estimation errors all converge exponentially fast to zero at a fixed, but arbitrarily chosen rate provided the network's graph is strongly connected for all time. This is accomplished by exploiting several well-known properties of invariant subspaces plus several kinds of suitably defined matrix norms.

eess.SY↗

arXiv2019

Optimal Projection Guided Transfer Hashing for Image Retrieval

Recently, learning to hash has been widely studied for image retrieval thanks to the computation and storage efficiency of binary codes. For most existing learning to hash methods, sufficient training images are required and used to learn precise hashing codes. However, in some real-world applications, there are not always sufficient training images in the domain of interest. In addition, some existing supervised approaches need a amount of labeled data, which is an expensive process in term of time, label and human expertise. To handle such problems, inspired by transfer learning, we propose a simple yet effective unsupervised hashing method named Optimal Projection Guided Transfer Hashing (GTH) where we borrow the images of other different but related domain i.e., source domain to help learn precise hashing codes for the domain of interest i.e., target domain. Besides, we propose to seek for the maximum likelihood estimation (MLE) solution of the hashing functions of target and source domains due to the domain gap. Furthermore,an alternating optimization method is adopted to obtain the two projections of target and source domains such that the domain hashing disparity is reduced gradually. Extensive experiments on various benchmark databases verify that our method outperforms many state-of-the-art learning to hash methods. The implementation details are available at https://github.com/liuji93/GTH.

cs.IR↗

arXiv2019

Inexact Proximal Cubic Regularized Newton Methods for Convex Optimization

In this paper, we use Proximal Cubic regularized Newton Methods (PCNM) to optimize the sum of a smooth convex function and a non-smooth convex function, where we use inexact gradient and Hessian, and an inexact subsolver for the cubic regularized second-order subproblem. We propose inexact variants of PCNM and accelerated PCNM respectively, and show that both variants can achieve the same convergence rate as in the exact case, provided that the errors in the inexact gradient, Hessian and subsolver decrease at appropriate rates. Meanwhile, in the online stochastic setting where data comes endlessly, we give the overall complexity of the proposed algorithms and show that they are as competitive as the stochastic gradient descent. Moreover, we give the overall complexity of the proposed algorithms in the finite-sum setting and show that it is as competitive as the state of the art variance reduced algorithms. Finally, we propose an efficient algorithm for the cubic regularized second-order subproblem, which can converge to an enough small neighborhood of the optimal solution in a superlinear rate.

math.OC↗

arXiv2019

Learning Simple Thresholded Features with Sparse Support Recovery

The thresholded feature has recently emerged as an extremely efficient, yet rough empirical approximation, of the time-consuming sparse coding inference process. Such an approximation has not yet been rigorously examined, and standard dictionaries often lead to non-optimal performance when used for computing thresholded features. In this paper, we first present two theoretical recovery guarantees for the thresholded feature to exactly recover the nonzero support of the sparse code. Motivated by them, we then formulate the Dictionary Learning for Thresholded Features (DLTF) model, which learns an optimized dictionary for applying the thresholded feature. In particular, for the $(k, 2)$ norm involved, a novel proximal operator with log-linear time complexity $O(m\log m)$ is derived. We evaluate the performance of DLTF on a vast range of synthetic and real-data tasks, where DLTF demonstrates remarkable efficiency, effectiveness and robustness in all experiments. In addition, we briefly discuss the potential link between DLTF and deep learning building blocks.

cs.CV↗

arXiv2019

Marginal Policy Gradients: A Unified Family of Estimators for Bounded Action Spaces with Applications

Many complex domains, such as robotics control and real-time strategy (RTS) games, require an agent to learn a continuous control. In the former, an agent learns a policy over $\mathbb{R}^d$ and in the latter, over a discrete set of actions each of which is parametrized by a continuous parameter. Such problems are naturally solved using policy based reinforcement learning (RL) methods, but unfortunately these often suffer from high variance leading to instability and slow convergence. Unnecessary variance is introduced whenever policies over bounded action spaces are modeled using distributions with unbounded support by applying a transformation $T$ to the sampled action before execution in the environment. Recently, the variance reduced clipped action policy gradient (CAPG) was introduced for actions in bounded intervals, but to date no variance reduced methods exist when the action is a direction, something often seen in RTS games. To this end we introduce the angular policy gradient (APG), a stochastic policy gradient method for directional control. With the marginal policy gradients family of estimators we present a unified analysis of the variance reduction properties of APG and CAPG; our results provide a stronger guarantee than existing analyses for CAPG. Experimental results on a popular RTS game and a navigation task show that the APG estimator offers a substantial improvement over the standard policy gradient.

cs.LG↗

arXiv2019

Fully Implicit Online Learning

Regularized online learning is widely used in machine learning applications. In online learning, performing exact minimization ($i.e.,$ implicit update) is known to be beneficial to the numerical stability and structure of solution. In this paper we study a class of regularized online algorithms without linearizing the loss function or the regularizer, which we call \emph{fully implicit online learning} (FIOL). We show that for arbitrary Bregman divergence, FIOL has the $O(\sqrt{T})$ regret for general convex setting and $O(\log T)$ regret for strongly convex setting, and the regret has an one-step improvement effect because it avoids the approximation error of linearization. Then we propose efficient algorithms to solve the subproblem of FIOL. We show that even if the solution of the subproblem has no closed form, it can be solved with complexity comparable to the linearized online algoritms. Experiments validate the proposed approaches.

cs.LG↗

arXiv2019

Communication Compression for Decentralized Training

Optimizing distributed learning systems is an art of balancing between computation and communication. There have been two lines of research that try to deal with slower networks: {\em communication compression} for low bandwidth networks, and {\em decentralization} for high latency networks. In this paper, We explore a natural question: {\em can the combination of both techniques lead to a system that is robust to both bandwidth and latency?} Although the system implication of such combination is trivial, the underlying theoretical principle and algorithm design is challenging: unlike centralized algorithms, simply compressing exchanged information, even in an unbiased stochastic way, within the decentralized network would accumulate the error and fail to converge. In this paper, we develop a framework of compressed, decentralized training and propose two different strategies, which we call {\em extrapolation compression} and {\em difference compression}. We analyze both algorithms and prove both converge at the rate of $O(1/\sqrt{nT})$ where $n$ is the number of workers and $T$ is the number of iterations, matching the convergence rate for full precision, centralized training. We validate our algorithms and find that our proposed algorithm outperforms the best of merely decentralized and merely quantized algorithm significantly for networks with {\em both} high latency and low bandwidth.

cs.LG↗

arXiv2019

Observation of parity-time symmetry breaking transitions in a dissipative Floquet system of ultracold atoms

Open physical systems with balanced loss and gain, described by non-Hermitian parity-time ($\mathcal{PT}$) reflection symmetric Hamiltonians, exhibit a transition which could engenders modes that exponentially decay or grow with time and thus spontaneously breaks the $\mathcal{PT}$-symmetry. Such $\mathcal{PT}$-symmetry breaking transitions have attracted many interests because of their extraordinary behaviors and functionalities absent in closed systems. Here we report on the observation of $\mathcal{PT}$-symmetry breaking transitions by engineering time-periodic dissipation and coupling, which are realized through state-dependent atom loss in an optical dipole trap of ultracold $^6$Li atoms. Comparing with a single transition appearing for static dissipation, the time-periodic counterpart undergoes $\mathcal{PT}$-symmetry breaking and restoring transitions at vanishingly small dissipation strength in both single and multiphoton transition domains, revealing rich phase structures associated to a Floquet open system. The results enable ultracold atoms to be a versatile tool for studying $\mathcal{PT}$-symmetric quantum systems.

cond-mat.quant-gas↗

arXiv2019

Cooperative opinion dynamics on multiple interdependent topics: Modeling and analysis

To model the interdependent couplings of multiple topics, we develop a set of rules for opinion updates of a group of agents. The rules are used to design or assign values to the elements of interdependent weighting matrices. The cooperative and anti-cooperative couplings are modeled in both the inverse-proportional and proportional feedbacks. The behaviors of cooperative opinion dynamics are analyzed using a null space property of state-dependent matrix-weighted Laplacian matrices and a Lyapunov candidate. Various consensus properties of state-dependent matrix-weighted Laplacian matrices are predicted according to the intra-agent network topology and inter-dependency topical coupling topologies.

eess.SY↗

arXiv2019

Analysis and Control of a Continuous-Time Bi-Virus Model

This paper studies a distributed continuous-time bi-virus model in which two competing viruses spread over a network consisting of multiple groups of individuals. Limiting behaviors of the network are characterized by analyzing the equilibria of the system and their stability. Specifically, when the two viruses spread over possibly different directed infection graphs, the system may have (1) a unique equilibrium, the healthy state, which is globally stable, implying that both viruses will eventually be eradicated, (2) two equilibria including the healthy state and a dominant virus state, which is almost globally stable, implying that one virus will pervade the entire network causing a single-virus epidemic while the other virus will be eradicated, or (3) at least three equilibria including the healthy state and two dominant virus states, depending on certain conditions on the healing and infection rates. When the two viruses spread over the same directed infection graph, the system may have zero or infinitely many coexisting epidemic equilibria, which represents the pervasion of the two viruses. Sensitivity properties of some nontrivial equilibria are investigated in the context of a decentralized control technique, and an impossibility result is given for a certain type of distributed feedback controller.

math.OC↗

arXiv2018

Monocular 3D Pose Recovery via Nonconvex Sparsity with Theoretical Analysis

For recovering 3D object poses from 2D images, a prevalent method is to pre-train an over-complete dictionary $\mathcal D=\{B_i\}_i^D$ of 3D basis poses. During testing, the detected 2D pose $Y$ is matched to dictionary by $Y \approx \sum_i M_i B_i$ where $\{M_i\}_i^D=\{c_i ΠR_i\}$, by estimating the rotation $R_i$, projection $Π$ and sparse combination coefficients $c \in \mathbb R_{+}^D$. In this paper, we propose non-convex regularization $H(c)$ to learn coefficients $c$, including novel leaky capped $\ell_1$-norm regularization (LCNR), \begin{align*} H(c)=α\sum_{i } \min(|c_i|,τ)+ β\sum_{i } \max(| c_i|,τ), \end{align*} where $0\leq β\leq α$ and $0<τ$ is a certain threshold, so the invalid components smaller than $τ$ are composed with larger regularization and other valid components with smaller regularization. We propose a multi-stage optimizer with convex relaxation and ADMM. We prove that the estimation error $\mathcal L(l)$ decays w.r.t. the stages $l$, \begin{align*} Pr\left(\mathcal L(l) < ρ^{l-1} \mathcal L(0) + δ\right) \geq 1- ε, \end{align*} where $0< ρ<1, 0<δ, 0<ε\ll 1$. Experiments on large 3D human datasets like H36M are conducted to support our improvement upon previous approaches. To the best of our knowledge, this is the first theoretical analysis in this line of research, to understand how the recovery error is affected by fundamental factors, e.g. dictionary size, observation noises, optimization times. We characterize the trade-off between speed and accuracy towards real-time inference in applications.

cs.CV↗

arXiv2018

TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game

Starcraft II (SC2) is widely considered as the most challenging Real Time Strategy (RTS) game. The underlying challenges include a large observation space, a huge (continuous and infinite) action space, partial observations, simultaneous move for all players, and long horizon delayed rewards for local decisions. To push the frontier of AI research, Deepmind and Blizzard jointly developed the StarCraft II Learning Environment (SC2LE) as a testbench of complex decision making systems. SC2LE provides a few mini games such as MoveToBeacon, CollectMineralShards, and DefeatRoaches, where some AI agents have achieved the performance level of human professional players. However, for full games, the current AI agents are still far from achieving human professional level performance. To bridge this gap, we present two full game AI agents in this paper - the AI agent TStarBot1 is based on deep reinforcement learning over a flat action structure, and the AI agent TStarBot2 is based on hard-coded rules over a hierarchical action structure. Both TStarBot1 and TStarBot2 are able to defeat the built-in AI agents from level 1 to level 10 in a full game (1v1 Zerg-vs-Zerg game on the AbyssalReef map), noting that level 8, level 9, and level 10 are cheating agents with unfair advantages such as full vision on the whole map and resource harvest boosting. To the best of our knowledge, this is the first public work to investigate AI agents that can defeat the built-in AI in the StarCraft II full game.

cs.AI↗

arXiv2018

Anti-fatigue-fracture hydrogels

The emerging applications of hydrogels in devices and machines require these soft materials to maintain robustness under cyclic mechanical loads. Whereas hydrogels have been made tough to resist fracture under a single cycle of mechanical load, these toughened gels still suffer from fatigue fracture under multiple cycles of loads. The reported fatigue threshold (i.e., the minimal fracture energy at which crack propagation occurs under cyclic loads) for synthetic hydrogels is on the order of 1-100 J/m2, which is primarily associated with the energy required to fracture a single layer of polymer chains per unit area. Here, we demonstrate that the controlled introduction of crystallinity in hydrogels can significantly enhance their fatigue thresholds, since the process of fracturing crystalline domains for fatigue-crack propagation requires much higher energy than fracturing a single layer of polymer chains. The fatigue threshold of polyvinyl alcohol (PVA) with a crystallinity of 18.9 wt.% in the swollen state can exceed 1,000 J/m2. We further develop a strategy to enhance the anti-fatigue-fracture properties of PVA hydrogels, but still maintain their high water contents and low moduli by patterning highly-crystalline regions in the hydrogels. The current work not only reveals an anti-fatigue-fracture mechanism in hydrogels but also provides a practical method to design anti-fatigue-fracture hydrogels for diverse applications.

physics.app-ph↗

arXiv2018

Consensus and Disagreement of Heterogeneous Belief Systems in Influence Networks

Recently, an opinion dynamics model has been proposed to describe a network of individuals discussing a set of logically interdependent topics. For each individual, the set of topics and the logical interdependencies between the topics (captured by a logic matrix) form a belief system. We investigate the role the logic matrix and its structure plays in determining the final opinions, including existence of the limiting opinions, of a strongly connected network of individuals. We provide a set of results that, given a set of individuals' belief systems, allow a systematic determination of which topics will reach a consensus, and which topics will disagreement in arise. For irreducible logic matrices, each topic reaches a consensus. For reducible logic matrices, which indicates a cascade interdependence relationship, conditions are given on whether a topic will reach a consensus or not. It turns out that heterogeneity among the individuals' logic matrices, including especially differences in the signs of the off-diagonal entries, can be a key determining factor. This paper thus attributes, for the first time, a strong diversity of limiting opinions to heterogeneity of belief systems in influence networks, in addition to the more typical explanation that strong diversity arises from individual stubbornness.

cs.SI↗

arXiv2018

Distributed Learning of Average Belief Over Networks Using Sequential Observations

This paper addresses the problem of distributed learning of average belief with sequential observations, in which a network of $n>1$ agents aim to reach a consensus on the average value of their beliefs, by exchanging information only with their neighbors. Each agent has sequentially arriving samples of its belief in an online manner. The neighbor relationships among the $n$ agents are described by a graph which is possibly time-varying, whose vertices correspond to agents and whose edges depict neighbor relationships. Two distributed online algorithms are introduced for undirected and directed graphs, which are both shown to converge to the average belief almost surely. Moreover, the sequences generated by both algorithms are shown to reach consensus with an $O(1/t)$ rate with high probability, where $t$ is the number of iterations. For undirected graphs, the corresponding algorithm is modified for the case with quantized communication and limited precision of the division operation. It is shown that the modified algorithm causes all $n$ agents to either reach a quantized consensus or enter a small neighborhood around the average of their beliefs. Numerical simulations are then provided to corroborate the theoretical results.

cs.MA↗