arXiv subjects

Andrew Howard

Publications and source records attributed to Andrew Howard.

At least 37 records · Page 2Linked to original sources

arXiv2021

BasisNet: Two-stage Model Synthesis for Efficient Inference

In this work, we present BasisNet which combines recent advancements in efficient neural network architectures, conditional computation, and early termination in a simple new form. Our approach incorporates a lightweight model to preview the input and generate input-dependent combination coefficients, which later controls the synthesis of a more accurate specialist model to make final prediction. The two-stage model synthesis strategy can be applied to any network architectures and both stages are jointly trained. We also show that proper training recipes are critical for increasing generalizability for such high capacity neural networks. On ImageNet classification benchmark, our BasisNet with MobileNets as backbone demonstrated clear advantage on accuracy-efficiency trade-off over several strong baselines. Specifically, BasisNet-MobileNetV3 obtained 80.3% top-1 accuracy with only 290M Multiply-Add operations, halving the computational cost of previous state-of-the-art without sacrificing accuracy. With early termination, the average cost can be further reduced to 198M MAdds while maintaining accuracy of 80.0% on ImageNet.

cs.CV↗

arXiv2021

Discovering Multi-Hardware Mobile Models via Architecture Search

Hardware-aware neural architecture designs have been predominantly focusing on optimizing model performance on single hardware and model development complexity, where another important factor, model deployment complexity, has been largely ignored. In this paper, we argue that, for applications that may be deployed on multiple hardware, having different single-hardware models across the deployed hardware makes it hard to guarantee consistent outputs across hardware and duplicates engineering work for debugging and fixing. To minimize such deployment cost, we propose an alternative solution, multi-hardware models, where a single architecture is developed for multiple hardware. With thoughtful search space design and incorporating the proposed multi-hardware metrics in neural architecture search, we discover multi-hardware models that give state-of-the-art (SoTA) performance across multiple hardware in both average and worse case scenarios. For performance on individual hardware, the single multi-hardware model yields similar or better results than SoTA performance on accelerators like GPU, DSP and EdgeTPU which was achieved by different models, while having similar performance with MobilenetV3 Large Minimalistic model on mobile CPU.

cs.CV↗

arXiv2021

Multi-path Neural Networks for On-device Multi-domain Visual Classification

Learning multiple domains/tasks with a single model is important for improving data efficiency and lowering inference cost for numerous vision tasks, especially on resource-constrained mobile devices. However, hand-crafting a multi-domain/task model can be both tedious and challenging. This paper proposes a novel approach to automatically learn a multi-path network for multi-domain visual classification on mobile devices. The proposed multi-path network is learned from neural architecture search by applying one reinforcement learning controller for each domain to select the best path in the super-network created from a MobileNetV3-like search space. An adaptive balanced domain prioritization algorithm is proposed to balance optimizing the joint model on multiple domains simultaneously. The determined multi-path model selectively shares parameters across domains in shared nodes while keeping domain-specific parameters within non-shared nodes in individual domain paths. This approach effectively reduces the total number of parameters and FLOPS, encouraging positive knowledge transfer while mitigating negative interference across domains. Extensive evaluations on the Visual Decathlon dataset demonstrate that the proposed multi-path model achieves state-of-the-art performance in terms of accuracy, model size, and FLOPS against other approaches using MobileNetV3-like architectures. Furthermore, the proposed method improves average accuracy over learning single-domain models individually, and reduces the total number of parameters and FLOPS by 78% and 32% respectively, compared to the approach that simply bundles single-domain models for multi-domain learning.

cs.CV↗

arXiv2021

SpotPatch: Parameter-Efficient Transfer Learning for Mobile Object Detection

Deep learning based object detectors are commonly deployed on mobile devices to solve a variety of tasks. For maximum accuracy, each detector is usually trained to solve one single specific task, and comes with a completely independent set of parameters. While this guarantees high performance, it is also highly inefficient, as each model has to be separately downloaded and stored. In this paper we address the question: can task-specific detectors be trained and represented as a shared set of weights, plus a very small set of additional weights for each task? The main contributions of this paper are the following: 1) we perform the first systematic study of parameter-efficient transfer learning techniques for object detection problems; 2) we propose a technique to learn a model patch with a size that is dependent on the difficulty of the task to be learned, and validate our approach on 10 different object detection tasks. Our approach achieves similar accuracy as previously proposed approaches, while being significantly more compact.

cs.CV↗

arXiv2020

Large-Scale Generative Data-Free Distillation

Knowledge distillation is one of the most popular and effective techniques for knowledge transfer, model compression and semi-supervised learning. Most existing distillation approaches require the access to original or augmented training samples. But this can be problematic in practice due to privacy, proprietary and availability concerns. Recent work has put forward some methods to tackle this problem, but they are either highly time-consuming or unable to scale to large datasets. To this end, we propose a new method to train a generative image model by leveraging the intrinsic normalization layers' statistics of the trained teacher network. This enables us to build an ensemble of generators without training data that can efficiently produce substitute inputs for subsequent distillation. The proposed method pushes forward the data-free distillation performance on CIFAR-10 and CIFAR-100 to 95.02% and 77.02% respectively. Furthermore, we are able to scale it to ImageNet dataset, which to the best of our knowledge, has never been done using generative models in a data-free setting.

cs.LG↗

arXiv2020

The Occurrence of Rocky Habitable Zone Planets Around Solar-Like Stars from Kepler Data

We present occurrence rates for rocky planets in the habitable zones (HZ) of main-sequence dwarf stars based on the Kepler DR25 planet candidate catalog and Gaia-based stellar properties. We provide the first analysis in terms of star-dependent instellation flux, which allows us to track HZ planets. We define $η_\oplus$ as the HZ occurrence of planets with radius between 0.5 and 1.5 $R_\oplus$ orbiting stars with effective temperatures between 4800 K and 6300 K. We find that $η_\oplus$ for the conservative HZ is between $0.37^{+0.48}_{-0.21}$ (errors reflect 68\% credible intervals) and $0.60^{+0.90}_{-0.36}$ planets per star, while the optimistic HZ occurrence is between $0.58^{+0.73}_{-0.33}$ and $0.88^{+1.28}_{-0.51}$ planets per star. These bounds reflect two extreme assumptions about the extrapolation of completeness beyond orbital periods where DR25 completeness data are available. The large uncertainties are due to the small number of detected small HZ planets. We find similar occurrence rates using both a Poisson likelihood Bayesian analysis and Approximate Bayesian Computation. Our results are corrected for catalog completeness and reliability. Both completeness and the planet occurrence rate are dependent on stellar effective temperature. We also present occurrence rates for various stellar populations and planet size ranges. We estimate with $95\%$ confidence that, on average, the nearest HZ planet around G and K dwarfs is about 6 pc away, and there are about 4 HZ rocky planets around G and K dwarfs within 10 pc of the Sun.

astro-ph.EP↗

arXiv2020

An Asymmetric Eclipse Seen Towards the Pre-Main Sequence Binary System V928 Tau

K2 observations of the weak-lined T Tauri binary V928 Tau A+B show the detection of a single, asymmetric eclipse which may be due to a previously unknown substellar companion eclipsing one component of the binary with an orbital period $>$ 66 days. Over an interval of about 9 hours, one component of the binary dims by around 60%, returning to its normal brightness about 5 hours later. From modeling of the eclipse shape we find evidence that the eclipsing companion may be surrounded by a disk or a vast ring system. The modeled disk has a radius of $0.9923\,\pm\,0.0005\,R_*$, with an inclination of $56.78\,\pm\, 0.03^\circ$, a tilt of $41.22\,\pm\,0.05^\circ$, an impact parameter of $-0.2506\,\pm\,0.0002\,R_*$ and an opacity of 1.00. The occulting disk must also move at a transverse velocity of $6.637\,\pm\,0.002\,R_*\,\mathrm{day}^{-1}$, which depending on whether it orbits V928 Tau A or B, corresponds to approximately 73.53 or 69.26 $\mathrm{km s}^{-1}$. A search in ground based archival data reveals additional dimming events, some of which suggest periodicity, but no unambiguous period associated with the eclipse observed by K2. We present a new epoch of astrometry which is used to further refine the orbit of the binary, presenting a new lower bound of 67 years, and constraints on the possible orbital periods of the eclipsing companion. The binary is also separated by 18" ($\sim$2250 au) from the lower mass CFHT-BD-Tau 7, which is likely associated with V928 Tau A+B. We also present new high dispersion optical spectroscopy that we use to characterize the unresolved stellar binary.

astro-ph.SR↗

arXiv2020

EarthFinder Probe Mission Concept Study: Characterizing nearby stellar exoplanet systems with Earth-mass analogs for future direct imaging

EarthFinder is a NASA Astrophysics Probe mission concept selected for study as input to the 2020 Astrophysics National Academies Decadal Survey. The EarthFinder concept is based on a dramatic shift in our understanding of how PRV measurements should be made. We propose a new paradigm which brings the high precision, high cadence domain of transit photometry as demonstrated by Kepler and TESS to the challenges of PRV measurements at the cm/s level. This new paradigm takes advantage of: 1) broad wavelength coverage from the UV to NIR which is only possible from space to minimize the effects of stellar activity; 2) extremely compact, highly stable, highly efficient spectrometers (R>150,000) which require the diffraction-limited imaging possible only from space over a broad wavelength range; 3) the revolution in laser-based wavelength standards to ensure cm/s precision over many years; 4) a high cadence observing program which minimizes sampling-induced period aliases; 5) exploiting the absolute flux stability from space for continuum normalization for unprecedented line-by-line analysis not possible from the ground; and 6) focusing on the bright stars which will be the targets of future imaging missions so that EarthFinder can use a ~1.5 m telescope.

astro-ph.IM↗

arXiv2019

Searching for MobileNetV3

We present the next generation of MobileNets based on a combination of complementary search techniques as well as a novel architecture design. MobileNetV3 is tuned to mobile phone CPUs through a combination of hardware-aware network architecture search (NAS) complemented by the NetAdapt algorithm and then subsequently improved through novel architecture advances. This paper starts the exploration of how automated search algorithms and network design can work together to harness complementary approaches improving the overall state of the art. Through this process we create two new MobileNet models for release: MobileNetV3-Large and MobileNetV3-Small which are targeted for high and low resource use cases. These models are then adapted and applied to the tasks of object detection and semantic segmentation. For the task of semantic segmentation (or any dense pixel prediction), we propose a new efficient segmentation decoder Lite Reduced Atrous Spatial Pyramid Pooling (LR-ASPP). We achieve new state of the art results for mobile classification, detection and segmentation. MobileNetV3-Large is 3.2\% more accurate on ImageNet classification while reducing latency by 15\% compared to MobileNetV2. MobileNetV3-Small is 4.6\% more accurate while reducing latency by 5\% compared to MobileNetV2. MobileNetV3-Large detection is 25\% faster at roughly the same accuracy as MobileNetV2 on COCO detection. MobileNetV3-Large LR-ASPP is 30\% faster than MobileNetV2 R-ASPP at similar accuracy for Cityscapes segmentation.

cs.CV↗

arXiv2019

Non-discriminative data or weak model? On the relative importance of data and model resolution

We explore the question of how the resolution of the input image ("input resolution") affects the performance of a neural network when compared to the resolution of the hidden layers ("internal resolution"). Adjusting these characteristics is frequently used as a hyperparameter providing a trade-off between model performance and accuracy. An intuitive interpretation is that the reduced information content in the low-resolution input causes decay in the accuracy. In this paper, we show that up to a point, the input resolution alone plays little role in the network performance, and it is the internal resolution that is the critical driver of model quality. We then build on these insights to develop novel neural network architectures that we call \emph{Isometric Neural Networks}. These models maintain a fixed internal resolution throughout their entire depth. We demonstrate that they lead to high accuracy models with low activation footprint and parameter count.

cs.CV↗

arXiv2019

Geo-Aware Networks for Fine-Grained Recognition

Fine-grained recognition distinguishes among categories with subtle visual differences. In order to differentiate between these challenging visual categories, it is helpful to leverage additional information. Geolocation is a rich source of additional information that can be used to improve fine-grained classification accuracy, but has been understudied. Our contributions to this field are twofold. First, to the best of our knowledge, this is the first paper which systematically examined various ways of incorporating geolocation information into fine-grained image classification through the use of geolocation priors, post-processing or feature modulation. Secondly, to overcome the situation where no fine-grained dataset has complete geolocation information, we release two fine-grained datasets with geolocation by providing complementary information to existing popular datasets - iNaturalist and YFCC100M. By leveraging geolocation information we improve top-1 accuracy in iNaturalist from 70.1% to 79.0% for a strong baseline image-only model. Comparing several models, we found that best performance was achieved by a post-processing model that consumed the output of the image-only baseline alongside geolocation. However, for a resource-constrained model (MobileNetV2), performance was better with a feature modulation model that trains jointly over pixels and geolocation: accuracy increased from 59.6% to 72.2%. Our work makes a strong case for incorporating geolocation information in fine-grained recognition models for both server and on-device.

cs.CV↗

arXiv2019

High-resolution Infrared Spectrograph for Exoplanet Characterization with the Keck and Thirty Meter Telescopes

HISPEC (High-resolution Infrared Spectrograph for Exoplanet Characterization) is a proposed diffraction-limited spectrograph for the W.M. Keck Observatory, and a pathfinder for the MODHIS facility project (Multi-Object Diffraction-limited High-resolution Infrared Spectrograph) on the Thirty Meter Telescope. HISPEC/MODHIS builds on diffraction-limited spectrograph designs which rely on adaptively corrected single-mode fiber feeds. Seeing-limited high-resolution spectrographs, by virtue of the conservation of beam etendue, grow in volume following a D^3 power law (D is the telescope diameter), and are subject to daunting challenges associated with their large size. Diffraction-limited spectrographs fed by single mode fibers are decoupled from the telescope input, and are orders of magnitude more compact and have intrinsically stable line spread functions. Their efficiency is directly proportional to the performance of the adaptive optics (AO) system. AO technologies have matured rapidly over the past two decades and are baselined for future extremely large telescopes. HISPEC/MODHIS will take R>100,000 spectra of a few objects in a 10" field-of-view sampled at the diffraction limit (~10-50 mas), simultaneously from 0.95 to 2.4 microns (y-K). The scientific scope ranges from exoplanet infrared precision radial velocities, spectroscopy of transiting, close-in, and directly imaged exoplanets (atmospheric composition and dynamics, RM effect, spin measurements, Doppler imaging), brown dwarf characterization, stellar physics/chemistry, proto-planetary disk kinematics/composition, Solar system, extragalactic science, and cosmology. HISPEC/MODHIS features a compact, cost-effective design optimized to fully exploit the existing Keck-AO and future TMT-NFIRAOS infrastructures and boost the scientific reach of Keck Observatory and TMT soon after first light.

astro-ph.IM↗

arXiv2019

Visual Wake Words Dataset

The emergence of Internet of Things (IoT) applications requires intelligence on the edge. Microcontrollers provide a low-cost compute platform to deploy intelligent IoT applications using machine learning at scale, but have extremely limited on-chip memory and compute capability. To deploy computer vision on such devices, we need tiny vision models that fit within a few hundred kilobytes of memory footprint in terms of peak usage and model size on device storage. To facilitate the development of microcontroller friendly models, we present a new dataset, Visual Wake Words, that represents a common microcontroller vision use-case of identifying whether a person is present in the image or not, and provides a realistic benchmark for tiny vision models. Within a limited memory footprint of 250 KB, several state-of-the-art mobile models achieve accuracy of 85-90% on the Visual Wake Words dataset. We anticipate the proposed dataset will advance the research on tiny vision models that can push the pareto-optimal boundary in terms of accuracy versus memory usage for microcontroller applications.

cs.CV↗

arXiv2019

MnasNet: Platform-Aware Neural Architecture Search for Mobile

Designing convolutional neural networks (CNN) for mobile devices is challenging because mobile models need to be small and fast, yet still accurate. Although significant efforts have been dedicated to design and improve mobile CNNs on all dimensions, it is very difficult to manually balance these trade-offs when there are so many architectural possibilities to consider. In this paper, we propose an automated mobile neural architecture search (MNAS) approach, which explicitly incorporate model latency into the main objective so that the search can identify a model that achieves a good trade-off between accuracy and latency. Unlike previous work, where latency is considered via another, often inaccurate proxy (e.g., FLOPS), our approach directly measures real-world inference latency by executing the model on mobile phones. To further strike the right balance between flexibility and search space size, we propose a novel factorized hierarchical search space that encourages layer diversity throughout the network. Experimental results show that our approach consistently outperforms state-of-the-art mobile CNN models across multiple vision tasks. On the ImageNet classification task, our MnasNet achieves 75.2% top-1 accuracy with 78ms latency on a Pixel phone, which is 1.8x faster than MobileNetV2 [29] with 0.5% higher accuracy and 2.3x faster than NASNet [36] with 1.2% higher accuracy. Our MnasNet also achieves better mAP quality than MobileNets for COCO object detection. Code is at https://github.com/tensorflow/tpu/tree/master/models/official/mnasnet

cs.CV↗

arXiv2019

Low-Power Computer Vision: Status, Challenges, Opportunities

Computer vision has achieved impressive progress in recent years. Meanwhile, mobile phones have become the primary computing platforms for millions of people. In addition to mobile phones, many autonomous systems rely on visual data for making decisions and some of these systems have limited energy (such as unmanned aerial vehicles also called drones and mobile robots). These systems rely on batteries and energy efficiency is critical. This article serves two main purposes: (1) Examine the state-of-the-art for low-power solutions to detect objects in images. Since 2015, the IEEE Annual International Low-Power Image Recognition Challenge (LPIRC) has been held to identify the most energy-efficient computer vision solutions. This article summarizes 2018 winners' solutions. (2) Suggest directions for research as well as opportunities for low-power computer vision.

cs.CV↗

arXiv2019

MobileNetV2: Inverted Residuals and Linear Bottlenecks

In this paper we describe a new mobile architecture, MobileNetV2, that improves the state of the art performance of mobile models on multiple tasks and benchmarks as well as across a spectrum of different model sizes. We also describe efficient ways of applying these mobile models to object detection in a novel framework we call SSDLite. Additionally, we demonstrate how to build mobile semantic segmentation models through a reduced form of DeepLabv3 which we call Mobile DeepLabv3. The MobileNetV2 architecture is based on an inverted residual structure where the input and output of the residual block are thin bottleneck layers opposite to traditional residual models which use expanded representations in the input an MobileNetV2 uses lightweight depthwise convolutions to filter features in the intermediate expansion layer. Additionally, we find that it is important to remove non-linearities in the narrow layers in order to maintain representational power. We demonstrate that this improves performance and provide an intuition that led to this design. Finally, our approach allows decoupling of the input/output domains from the expressiveness of the transformation, which provides a convenient framework for further analysis. We measure our performance on Imagenet classification, COCO object detection, VOC image segmentation. We evaluate the trade-offs between accuracy, and number of operations measured by multiply-adds (MAdd), as well as the number of parameters

cs.CV↗

arXiv2019

The Super-Earth Opportunity - Search for Habitable Exoplanets in the 2020s

The recent discovery of a staggering diversity of planets beyond the Solar System has brought with it a greatly expanded search space for habitable worlds. The Kepler exoplanet survey has revealed that most planets in our interstellar neighborhood are larger than Earth and smaller than Neptune. Collectively termed super-Earths and mini-Neptunes, some of these planets may have the conditions to support liquid water oceans, and thus Earth-like biology, despite differing in many ways from our own planet. In addition to their quantitative abundance, super-Earths are relatively large and are thus more easily detected than true Earth twins. As a result, super-Earths represent a uniquely powerful opportunity to discover and explore a panoply of fascinating and potentially habitable planets in 2020 - 2030 and beyond.

astro-ph.EP↗

arXiv2019

K for the Price of 1: Parameter-efficient Multi-task and Transfer Learning

We introduce a novel method that enables parameter-efficient transfer and multi-task learning with deep neural networks. The basic approach is to learn a model patch - a small set of parameters - that will specialize to each task, instead of fine-tuning the last layer or the entire network. For instance, we show that learning a set of scales and biases is sufficient to convert a pretrained network to perform well on qualitatively different problems (e.g. converting a Single Shot MultiBox Detection (SSD) model into a 1000-class image classification model while reusing 98% of parameters of the SSD feature extractor). Similarly, we show that re-learning existing low-parameter layers (such as depth-wise convolutions) while keeping the rest of the network frozen also improves transfer-learning accuracy significantly. Our approach allows both simultaneous (multi-task) as well as sequential transfer learning. In several multi-task learning problems, despite using much fewer parameters than traditional logits-only fine-tuning, we match single-task performance.

cs.LG↗