arXiv subjects

Jiayi Liu

Publications and source records attributed to Jiayi Liu.

100 records · Page 6Linked to original sources

arXiv2020

Improving Model Training by Periodic Sampling over Weight Distributions

In this paper, we explore techniques centered around periodic sampling of model weights that provide convergence improvements on gradient update methods (vanilla \acs{SGD}, Momentum, Adam) for a variety of vision problems (classification, detection, segmentation). Importantly, our algorithms provide better, faster and more robust convergence and training performance with only a slight increase in computation time. Our techniques are independent of the neural network model, gradient optimization methods or existing optimal training policies and converge in a less volatile fashion with performance improvements that are approximately monotonic. We conduct a variety of experiments to quantify these improvements and identify scenarios where these techniques could be more useful.

cs.LG↗

arXiv2019

Auptimizer -- an Extensible, Open-Source Framework for Hyperparameter Tuning

Tuning machine learning models at scale, especially finding the right hyperparameter values, can be difficult and time-consuming. In addition to the computational effort required, this process also requires some ancillary efforts including engineering tasks (e.g., job scheduling) as well as more mundane tasks (e.g., keeping track of the various parameters and associated results). We present Auptimizer, a general Hyperparameter Optimization (HPO) framework to help data scientists speed up model tuning and bookkeeping. With Auptimizer, users can use all available computing resources in distributed settings for model training. The user-friendly system design simplifies creating, controlling, and tracking of a typical machine learning project. The design also allows researchers to integrate new HPO algorithms. To demonstrate its flexibility, we show how Auptimizer integrates a few major HPO techniques (from random search to neural architecture search). The code is available at https://github.com/LGE-ARC-AdvancedAI/auptimizer.

cs.LG↗

arXiv2019

Adaptive Wavelet Clustering for Highly Noisy Data

In this paper we make progress on the unsupervised task of mining arbitrarily shaped clusters in highly noisy datasets, which is a task present in many real-world applications. Based on the fundamental work that first applies a wavelet transform to data clustering, we propose an adaptive clustering algorithm, denoted as AdaWave, which exhibits favorable characteristics for clustering. By a self-adaptive thresholding technique, AdaWave is parameter free and can handle data in various situations. It is deterministic, fast in linear time, order-insensitive, shape-insensitive, robust to highly noisy data, and requires no pre-knowledge on data models. Moreover, AdaWave inherits the ability from the wavelet transform to cluster data in different resolutions. We adopt the "grid labeling" data structure to drastically reduce the memory consumption of the wavelet transform so that AdaWave can be used for relatively high dimensional data. Experiments on synthetic as well as natural datasets demonstrate the effectiveness and efficiency of our proposed method.

cs.DB↗

arXiv2018

Make (Nearly) Every Neural Network Better: Generating Neural Network Ensembles by Weight Parameter Resampling

Deep Neural Networks (DNNs) have become increasingly popular in computer vision, natural language processing, and other areas. However, training and fine-tuning a deep learning model is computationally intensive and time-consuming. We propose a new method to improve the performance of nearly every model including pre-trained models. The proposed method uses an ensemble approach where the networks in the ensemble are constructed by reassigning model parameter values based on the probabilistic distribution of these parameters, calculated towards the end of the training process. For pre-trained models, this approach results in an additional training step (usually less than one epoch). We perform a variety of analysis using the MNIST dataset and validate the approach with a number of DNN models using pre-trained models on the ImageNet dataset.

cs.LG↗

arXiv2018

Effective Building Block Design for Deep Convolutional Neural Networks using Search

Deep learning has shown promising results on many machine learning tasks but DL models are often complex networks with large number of neurons and layers, and recently, complex layer structures known as building blocks. Finding the best deep model requires a combination of finding both the right architecture and the correct set of parameters appropriate for that architecture. In addition, this complexity (in terms of layer types, number of neurons, and number of layers) also present problems with generalization since larger networks are easier to overfit to the data. In this paper, we propose a search framework for finding effective architectural building blocks for convolutional neural networks (CNN). Our approach is much faster at finding models that are close to state-of-the-art in performance. In addition, the models discovered by our approach are also smaller than models discovered by similar techniques. We achieve these twin advantages by designing our search space in such a way that it searches over a reduced set of state-of-the-art building blocks for CNNs including residual block, inception block, inception-residual block, ResNeXt block and many others. We apply this technique to generate models for multiple image datasets and show that these models achieve performance comparable to state-of-the-art (and even surpassing the state-of-the-art in one case). We also show that learned models are transferable between datasets.

cs.LG↗

arXiv2017

LADDER: A Human-Level Bidding Agent for Large-Scale Real-Time Online Auctions

We present LADDER, the first deep reinforcement learning agent that can successfully learn control policies for large-scale real-world problems directly from raw inputs composed of high-level semantic information. The agent is based on an asynchronous stochastic variant of DQN (Deep Q Network) named DASQN. The inputs of the agent are plain-text descriptions of states of a game of incomplete information, i.e. real-time large scale online auctions, and the rewards are auction profits of very large scale. We apply the agent to an essential portion of JD's online RTB (real-time bidding) advertising business and find that it easily beats the former state-of-the-art bidding policy that had been carefully engineered and calibrated by human experts: during JD.com's June 18th anniversary sale, the agent increased the company's ads revenue from the portion by more than 50%, while the advertisers' ROI (return on investment) also improved significantly.

cs.LG↗

arXiv2017

Negative differential resistance and magnetoresistance in zigzag borophene nanoribbons

We investigate the transport properties of pristine zigzag-edged borophene nanoribbons (ZBNRs) of different widths, using the fist-principles calculations. We choose ZBNRs with widths of 5 and 6 as odd and even widths. The differences of the quantum transport properties are found, where even-N BNRs and odd-N BNRs have different current-voltage relationships. Moreover, the negative differential resistance (NDR) can be observed within certain bias range in 5-ZBNR, while 6-ZBNR behaves as metal whose current rises with the increase of the voltage. The spin filter effect of 36% can be revealed when the two electrodes have opposite magnetization direction. Furthermore, the magnetoresistance effect appears to be in even-N ZBNRs, and the maximum value can reach 70%.

cond-mat.mtrl-sci↗

arXiv2016

SZE Observables, Pressure Profiles and Center Offsets in Magneticum Simulation Galaxy Clusters

We present a detailed study of the galaxy cluster thermal \ac{sze} signal $Y$ and pressure profiles using {\it Magneticum} Pathfinder hydrodynamical simulations. With a sample of 50,000 galaxy clusters ($M_{\rm 500c}>1.4\times10^{14} \rm M_{\odot}$) out to $z=2$, we find significant variations in the shape of the pressure profile with mass and redshift and present a new generalized NFW model that follows these trends. We show that the thermal pressure at $R_{\rm 500c}$ accounts for only 80~percent of the pressure required to maintain hydrostatic equilibrium, and therefore even idealized hydrostatic mass estimates would be biased at the 20~percent level. We compare the cluster \ac{sze} signal extracted from a sphere with different virial-like radii, a virial cylinder within a narrow redshift slice and the full light cone, confirming small scatter ($σ_{\ln Y}\simeq 0.087$) in the sphere and showing that structure immediately surrounding clusters increases the scatter and strengthens non self-similar redshift evolution in the cylinder. Uncorrelated large scale structure along the line of sight leads to an increase in the \ac{sze} signal and scatter that is more pronounced for low mass clusters, resulting in non self-similar trends in both mass and redshift and a mass dependent scatter that is $\sim0.16$ at low masses. The scatter distribution is consistent with log-normal in all cases. We present a model of the offsets between the center of the gravitational potential and the \ac{sze} center that follows the variations with cluster mass and redshift.

astro-ph.CO↗

arXiv2016

$\mathtt{ComEst}$: a Completeness Estimator of Source Extraction on Astronomical Imaging

The completeness of source detection is critical for analyzing the photometric and spatial properties of the population of interest observed by astronomical imaging. We present a software package $\mathtt{ComEst}$, which calculates the completeness of source detection on charge-coupled device (CCD) images of astronomical observations, especially for the optical and near-infrared (NIR) imaging of galaxies and point sources. The completeness estimator $\mathtt{ComEst}$ is designed for the source finder $\mathtt{SExtractor}$ used on the CCD images saved in the Flexible Image Transport System (FITS) format. Specifically, $\mathtt{ComEst}$ estimates the completeness of the source detection by deriving the detection rate of synthetic point sources and galaxies simulated on the observed CCD images. In order to capture any observational artifacts or noise properties while deriving the completeness, $\mathtt{ComEst}$ directly carries out the detection of simulated sources on the observed images. Given an observed CCD image saved in FITS format, $\mathtt{ComEst}$ derives the completeness of the source detection from end to end as a function of source flux (or magnitude) and CCD position. In addition, $\mathtt{ComEst}$ can also estimate the purity of the source detection by comparing the catalog of the detected sources to the input catalogs of the simulated sources. We run ComEst on the images from Blanco Cosmology Survey (BCS) and compare the derived completeness as a function of magnitude to the limiting magnitudes derived by using the Signal-to-Noise ratio (SNR) and number count histogram of the detected sources. $\mathtt{ComEst}$ is released as a Python package with an easy-to-use syntax and is publicly available at https://github.com/inonchiu/ComEst

astro-ph.IM↗

arXiv2010

Noisy weak-lensing convergence peak statistics near clusters of galaxies and beyond

Taking into account noise from intrinsic ellipticities of source galaxies, in this paper, we study the peak statistics in weak-lensing convergence maps around clusters of galaxies and beyond. We emphasize how the noise peak statistics is affected by the density distribution of nearby clusters, and also how cluster-peak signals are changed by the existence of noise. These are the important aspects to be understood thoroughly in weak-lensing analyses for individual clusters as well as in cosmological applications of weak-lensing cluster statistics. We adopt Gaussian smoothing with the smoothing scale $θ_G=0.5\hbox{ arcmin}$ in our analyses. It is found that the noise peak distribution near a cluster of galaxies depends sensitively on the density profile of the cluster. For a cored isothermal cluster with the core radius $R_c$, the inner region with $R\le R_c$ appears noisy containing on average $\sim 2.4$ peaks with $ν\ge 5$ for $R_c= 1.7\hbox{ arcmin}$ and the true peak height of the cluster $ν=5.6$, where $ν$ denotes the convergence signal to noise ratio. For a NFW cluster of the same mass and the same central $ν$, the average number of peaks with $ν\ge 5$ within $R\le R_c$ is $\sim 1.6$. Thus a high peak corresponding to the main cluster can be identified more cleanly in the NFW case. In the outer region with $R_c<R\le 5R_c$, the number of high noise peaks is considerably enhanced in comparison with that of the pure noise case without the nearby cluster. (abridged)

astro-ph.CO↗