arXiv subjects

Han Cai

Publications and source records attributed to Han Cai.

At least 91 records · Page 5Linked to original sources

arXiv2020

Optimal Locally Repairable Codes: An Improved Bound and Constructions

We study the Singleton-type bound that provides an upper limit on the minimum distance of locally repairable codes. We present an improved bound by carefully analyzing the combinatorial structure of the repair sets. Thus, we show the previous bound is unachievable for certain parameters. We then also provide explicit constructions of optimal codes that show that for certain parameters the new bound is sharp. Additionally, as a byproduct, some previously known codes are shown to attain the new bound and are thus proved to be optimal.

cs.IT↗

arXiv2020

Flat-band localization in Creutz superradiance lattices

Flat bands play an important role in diffraction-free photonics and attract fundamental interest in many-body physics. Here we report the engineering of flat-band localization of collective excited states of atoms in Creutz superradiance lattices with tunable synthetic gauge fields. Magnitudes and phases of the lattice hopping coefficients can be independently tuned to control the state components of the flat band and the Aharonov-Bohm phases. We can selectively excite the flat band and control the flat-band localization with the synthetic gauge field. Our study provides a room-temperature platform for flat bands of atoms and holds promising applications in exploring correlated topological materials.

quant-ph↗

arXiv2020

APQ: Joint Search for Network Architecture, Pruning and Quantization Policy

We present APQ for efficient deep learning inference on resource-constrained hardware. Unlike previous methods that separately search the neural architecture, pruning policy, and quantization policy, we optimize them in a joint manner. To deal with the larger design space it brings, a promising approach is to train a quantization-aware accuracy predictor to quickly get the accuracy of the quantized model and feed it to the search engine to select the best fit. However, training this quantization-aware accuracy predictor requires collecting a large number of quantized pairs, which involves quantization-aware finetuning and thus is highly time-consuming. To tackle this challenge, we propose to transfer the knowledge from a full-precision (i.e., fp32) accuracy predictor to the quantization-aware (i.e., int8) accuracy predictor, which greatly improves the sample efficiency. Besides, collecting the dataset for the fp32 accuracy predictor only requires to evaluate neural networks without any training cost by sampling from a pretrained once-for-all network, which is highly efficient. Extensive experiments on ImageNet demonstrate the benefits of our joint optimization approach. With the same accuracy, APQ reduces the latency/energy by 2x/1.3x over MobileNetV2+HAQ. Compared to the separate optimization approach (ProxylessNAS+AMC+HAQ), APQ achieves 2.3% higher ImageNet accuracy while reducing orders of magnitude GPU hours and CO2 emission, pushing the frontier for green AI that is environmental-friendly. The code and video are publicly available.

cs.LG↗

arXiv2020

HAT: Hardware-Aware Transformers for Efficient Natural Language Processing

Transformers are ubiquitous in Natural Language Processing (NLP) tasks, but they are difficult to be deployed on hardware due to the intensive computation. To enable low-latency inference on resource-constrained hardware platforms, we propose to design Hardware-Aware Transformers (HAT) with neural architecture search. We first construct a large design space with $\textit{arbitrary encoder-decoder attention}$ and $\textit{heterogeneous layers}$. Then we train a $\textit{SuperTransformer}$ that covers all candidates in the design space, and efficiently produces many $\textit{SubTransformers}$ with weight sharing. Finally, we perform an evolutionary search with a hardware latency constraint to find a specialized $\textit{SubTransformer}$ dedicated to run fast on the target hardware. Extensive experiments on four machine translation tasks demonstrate that HAT can discover efficient models for different hardware (CPU, GPU, IoT device). When running WMT'14 translation task on Raspberry Pi-4, HAT can achieve $\textbf{3}\times$ speedup, $\textbf{3.7}\times$ smaller size over baseline Transformer; $\textbf{2.7}\times$ speedup, $\textbf{3.6}\times$ smaller size over Evolved Transformer with $\textbf{12,041}\times$ less search cost and no performance loss. HAT code is https://github.com/mit-han-lab/hardware-aware-transformers.git

cs.CL↗

arXiv2020

Once-for-All: Train One Network and Specialize it for Efficient Deployment

We address the challenging problem of efficient inference across many devices and resource constraints, especially on edge devices. Conventional approaches either manually design or use neural architecture search (NAS) to find a specialized neural network and train it from scratch for each case, which is computationally prohibitive (causing $CO_2$ emission as much as 5 cars' lifetime) thus unscalable. In this work, we propose to train a once-for-all (OFA) network that supports diverse architectural settings by decoupling training and search, to reduce the cost. We can quickly get a specialized sub-network by selecting from the OFA network without additional training. To efficiently train OFA networks, we also propose a novel progressive shrinking algorithm, a generalized pruning method that reduces the model size across many more dimensions than pruning (depth, width, kernel size, and resolution). It can obtain a surprisingly large number of sub-networks ($> 10^{19}$) that can fit different hardware platforms and latency constraints while maintaining the same level of accuracy as training independently. On diverse edge devices, OFA consistently outperforms state-of-the-art (SOTA) NAS methods (up to 4.0% ImageNet top1 accuracy improvement over MobileNetV3, or same accuracy but 1.5x faster than MobileNetV3, 2.6x faster than EfficientNet w.r.t measured latency) while reducing many orders of magnitude GPU hours and $CO_2$ emission. In particular, OFA achieves a new SOTA 80.0% ImageNet top-1 accuracy under the mobile setting ($<$600M MACs). OFA is the winning solution for the 3rd Low Power Computer Vision Challenge (LPCVC), DSP classification track and the 4th LPCVC, both classification track and detection track. Code and 50 pre-trained models (for many devices & many latency constraints) are released at https://github.com/mit-han-lab/once-for-all.

cs.LG↗

arXiv2020

Many-body chiral edge currents and sliding phases of atomic spinwaves in momentum-space lattice

Collective excitations (spinwaves) of long-lived atomic hyperfine states can be synthesized into a Bose-Hubbard model in momentum space. We explore many-body ground states and dynamics of a two-leg momentum-space lattice formed by two coupled hyperfine states. Essential ingredients of this setting are a staggered artificial magnetic field engineered by lasers that couple the spinwave states, and a state-dependent long-range interaction, which is induced by laser-dressing a hyperfine state to a Rydberg state. The Rydberg dressed two-body interaction gives rise to a state-dependent blockade in momentum space, and can amplify staggered flux induced anti-chiral edge currents in the many-body ground state in the presence of magnetic flux. When the Rydberg dressing is applied to both hyperfine states, exotic sliding insulating and superfluid/supersolid phases emerge. Due to the Rydberg dressed long-range interaction, spinwaves slide along a leg of the momentum-space lattice without costing energy. Our study paves a route to the quantum simulation of topological phases and exotic dynamics with interacting spinwaves of atomic hyperfine states in momentum-space lattice.

cond-mat.quant-gas↗

arXiv2020

On Optimal Locally Repairable Codes and Generalized Sector-Disk Codes

Optimal locally repairable codes with information locality are considered. Optimal codes are constructed, whose length is also order-optimal with respect to a new bound on the code length derived in this paper. The length of the constructed codes is super-linear in the alphabet size, which improves upon the well known pyramid codes, whose length is only linear in the alphabet size. The recoverable erasure patterns are also analyzed for the new codes. Based on the recoverable erasure patterns, we construct generalized sector-disk (GSD) codes, which can recover from disk erasures mixed with sector erasures in a more general setting than known sector-disk (SD) codes. Additionally, the number of sectors in the constructed GSD codes is super-linear in the alphabet size, compared with known SD codes, whose number of sectors is only linear in the alphabet size.

cs.IT↗

arXiv2020

Topological phases of quantized light

Topological photonics is an emerging research area that focuses on the topological states of classical light. Here we reveal the topological phases that are intrinsic to the particle nature of light, i.e., solely related to the quantized Fock states and the inhomogeneous coupling between them. The Hamiltonian of two cavities coupled with a two-level atom is an intrinsic one-dimensional Su-Schriefer-Heeger model of Fock states. By adding another cavity, the Fock-state lattice is extended to two dimensions with a honeycomb structure, where the strain due to the inhomogeneity of the coupling strengths induces a Lifshitz topological phase transition between a semimetal and a band insulator. In the semimetallic phase, the strain is equivalent to a pseudomagnetic field, which results in the quantization of the Landau levels and the valley Hall effect. We further construct a Haldane model where the topological phases can be characterized by the topological markers. This study demonstrates a fundamental distinction between the topological phases of bosons and fermions and provides a novel platform for studying topological physics in dimensions higher than three.

cond-mat.mes-hall↗

arXiv2019

Nanoscale High Transition Temperature Superconducting Quantum Interference Device Transimpedance Amplifier

As the quantum generation of electronics takes the stage, a cast of important support electronics is needed to connect these novel devices to our classical worlds. In the case of superconducting electronics, this is a challenge because the Josephson junction devices they are based upon require tiny current pulses to create and manipulate the single flux quanta which guide their operation. Difficulty arises in transitioning these signals through large temperature gradients for connection to semiconductor components. In this work, we present nano superconducting quantum interference devices (SQUID) with critical dimensions as small as 10 nm from the high-transition-temperature superconductor YBa$_2$Cu$_3$O$_{7-δ}$ (YBCO). We integrate these nano-SQUIDs with nano-isolated inductively coupled control lines to create a low power superconducting output driver capable of transimpedance conversion over a very wide temperature range.

physics.app-ph↗

arXiv2019

Network-Coding Solutions for Minimal Combination Networks and Their Sub-networks

Minimal multicast networks are fascinating and efficient combinatorial objects, where the removal of a single link makes it impossible for all receivers to obtain all messages. We study the structure of such networks, and prove some constraints on their possible solutions. We then focus on the combination network, which is one of the simplest and most insightful network in network-coding theory. Of particular interest are minimal combination networks. We study the gap in alphabet size between vector-linear and scalar-linear network-coding solutions for such minimal combination networks and some of their sub-networks. For minimal multicast networks with two source messages we find the maximum possible gap. We define and study sub-networks of the combination network, which we call Kneser networks, and prove that they attain the upper bound on the gap with equality. We also prove that the study of this gap may be limited to the study of sub-networks of minimal combination networks, by using graph homomorphisms connected with the $q$-analog of Kneser graphs. Additionally, we prove a gap for minimal multicast networks with three or more source messages by studying Kneser networks. Finally, an upper bound on the gap for full minimal combination networks shows nearly no gap, or none in some cases. This is obtained using an MDS-like bound for subspaces over a finite field.

cs.IT↗

arXiv2019

Design Automation for Efficient Deep Learning Computing

Efficient deep learning computing requires algorithm and hardware co-design to enable specialization: we usually need to change the algorithm to reduce memory footprint and improve energy efficiency. However, the extra degree of freedom from the algorithm makes the design space much larger: it's not only about designing the hardware but also about how to tweak the algorithm to best fit the hardware. Human engineers can hardly exhaust the design space by heuristics. It's labor consuming and sub-optimal. We propose design automation techniques for efficient neural networks. We investigate automatically designing specialized fast models, auto channel pruning, and auto mixed-precision quantization. We demonstrate such learning-based, automated design achieves superior performance and efficiency than rule-based human design. Moreover, we shorten the design cycle by 200x than previous work, so that we can afford to design specialized neural network models for different hardware platforms.

cs.LG↗

arXiv2019

On Optimal Locally Repairable Codes with Super-Linear Length

Locally repairable codes which are optimal with respect to the bound presented by Prakash et al. are considered. New upper bounds on the length of such optimal codes are derived. The new bounds both improve and generalize previously known bounds. Optimal codes are constructed, whose length is order-optimal when compared with the new upper bounds. The length of the codes is super-linear in the alphabet size.

cs.IT↗

arXiv2019

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

Neural architecture search (NAS) has a great impact by automatically designing effective neural network architectures. However, the prohibitive computational demand of conventional NAS algorithms (e.g. $10^4$ GPU hours) makes it difficult to \emph{directly} search the architectures on large-scale tasks (e.g. ImageNet). Differentiable NAS can reduce the cost of GPU hours via a continuous representation of network architecture but suffers from the high GPU memory consumption issue (grow linearly w.r.t. candidate set size). As a result, they need to utilize~\emph{proxy} tasks, such as training on a smaller dataset, or learning with only a few blocks, or training just for a few epochs. These architectures optimized on proxy tasks are not guaranteed to be optimal on the target task. In this paper, we present \emph{ProxylessNAS} that can \emph{directly} learn the architectures for large-scale target tasks and target hardware platforms. We address the high memory consumption issue of differentiable NAS and reduce the computational cost (GPU hours and GPU memory) to the same level of regular training while still allowing a large candidate set. Experiments on CIFAR-10 and ImageNet demonstrate the effectiveness of directness and specialization. On CIFAR-10, our model achieves 2.08\% test error with only 5.7M parameters, better than the previous state-of-the-art architecture AmoebaNet-B, while using 6$\times$ fewer parameters. On ImageNet, our model achieves 3.1\% better top-1 accuracy than MobileNetV2, while being 1.2$\times$ faster with measured GPU latency. We also apply ProxylessNAS to specialize neural architectures for hardware with direct hardware metrics (e.g. latency) and provide insights for efficient CNN architecture design.

cs.LG↗

arXiv2018

Activation Maximization Generative Adversarial Nets

Class labels have been empirically shown useful in improving the sample quality of generative adversarial nets (GANs). In this paper, we mathematically study the properties of the current variants of GANs that make use of class label information. With class aware gradient and cross-entropy decomposition, we reveal how class labels and associated losses influence GAN's training. Based on that, we propose Activation Maximization Generative Adversarial Networks (AM-GAN) as an advanced solution. Comprehensive experiments have been conducted to validate our analysis and evaluate the effectiveness of our solution, where AM-GAN outperforms other strong baselines and achieves state-of-the-art Inception Score (8.91) on CIFAR-10. In addition, we demonstrate that, with the Inception ImageNet classifier, Inception Score mainly tracks the diversity of the generator, and there is, however, no reliable evidence that it can reflect the true sample quality. We thus propose a new metric, called AM Score, to provide a more accurate estimation of the sample quality. Our proposed model also outperforms the baseline methods in the new metric.

cs.LG↗

arXiv2018

Layout Design for Intelligent Warehouse by Evolution with Fitness Approximation

With the rapid growth of the express industry, intelligent warehouses that employ autonomous robots for carrying parcels have been widely used to handle the vast express volume. For such warehouses, the warehouse layout design plays a key role in improving the transportation efficiency. However, this work is still done by human experts, which is expensive and leads to suboptimal results. In this paper, we aim to automate the warehouse layout designing process. We propose a two-layer evolutionary algorithm to efficiently explore the warehouse layout space, where an auxiliary objective fitness approximation model is introduced to predict the outcome of the designed warehouse layout and a two-layer population structure is proposed to incorporate the approximation model into the ordinary evolution framework. Empirical experiments show that our method can efficiently design effective warehouse layouts that outperform both heuristic-designed and vanilla evolution-designed warehouse layouts.

cs.AI↗

arXiv2018

Large-scale Interactive Recommendation with Tree-structured Policy Gradient

Reinforcement learning (RL) has recently been introduced to interactive recommender systems (IRS) because of its nature of learning from dynamic interactions and planning for long-run performance. As IRS is always with thousands of items to recommend (i.e., thousands of actions), most existing RL-based methods, however, fail to handle such a large discrete action space problem and thus become inefficient. The existing work that tries to deal with the large discrete action space problem by utilizing the deep deterministic policy gradient framework suffers from the inconsistency between the continuous action representation (the output of the actor network) and the real discrete action. To avoid such inconsistency and achieve high efficiency and recommendation effectiveness, in this paper, we propose a Tree-structured Policy Gradient Recommendation (TPGR) framework, where a balanced hierarchical clustering tree is built over the items and picking an item is formulated as seeking a path from the root to a certain leaf of the tree. Extensive experiments on carefully-designed environments based on two real-world datasets demonstrate that our model provides superior recommendation performance and significant efficiency improvement over state-of-the-art methods.

cs.LG↗

arXiv2018

Observation of momentum-space chiral edge currents in room-temperature atoms

Chiral edge currents play an important role in characterizing topological matter. In atoms, they have been observed at such a low temperature that the atomic motion can be measured. Here we report the first experimental observation of chiral edge currents in atoms at room temperature. Staggered magnetic fluxes are induced by the spatial phase difference between two standing-wave light fields, which couple atoms to form a momentum-space zigzag superradiance lattice. The chiral edge currents have been measured by comparing the directional superradiant emissions of two timed Dicke states in the lattice. This work paves the way for quantum simulation of topological matter with hot atoms and facilitates the application of topological physics in real devices.

physics.optics↗

arXiv2018

Path-Level Network Transformation for Efficient Architecture Search

We introduce a new function-preserving transformation for efficient neural architecture search. This network transformation allows reusing previously trained networks and existing successful architectures that improves sample efficiency. We aim to address the limitation of current network transformation operations that can only perform layer-level architecture modifications, such as adding (pruning) filters or inserting (removing) a layer, which fails to change the topology of connection paths. Our proposed path-level transformation operations enable the meta-controller to modify the path topology of the given network while keeping the merits of reusing weights, and thus allow efficiently designing effective structures with complex path topologies like Inception models. We further propose a bidirectional tree-structured reinforcement learning meta-controller to explore a simple yet highly expressive tree-structured architecture space that can be viewed as a generalization of multi-branch architectures. We experimented on the image classification datasets with limited computational resources (about 200 GPU-hours), where we observed improved parameter efficiency and better test results (97.70% test accuracy on CIFAR-10 with 14.3M parameters and 74.6% top-1 accuracy on ImageNet in the mobile setting), demonstrating the effectiveness and transferability of our designed architectures.

cs.LG↗