arXiv subjects

Ke Li

Publications and source records attributed to Ke Li.

At least 397 records · Page 22Linked to original sources

arXiv2020

Filter Grafting for Deep Neural Networks

This paper proposes a new learning paradigm called filter grafting, which aims to improve the representation capability of Deep Neural Networks (DNNs). The motivation is that DNNs have unimportant (invalid) filters (e.g., l1 norm close to 0). These filters limit the potential of DNNs since they are identified as having little effect on the network. While filter pruning removes these invalid filters for efficiency consideration, filter grafting re-activates them from an accuracy boosting perspective. The activation is processed by grafting external information (weights) into invalid filters. To better perform the grafting process, we develop an entropy-based criterion to measure the information of filters and an adaptive weighting strategy for balancing the grafted information among networks. After the grafting operation, the network has very few invalid filters compared with its untouched state, enpowering the model with more representation capacity. We also perform extensive experiments on the classification and recognition tasks to show the superiority of our method. For example, the grafted MobileNetV2 outperforms the non-grafted MobileNetV2 by about 7 percent on CIFAR-100 dataset. Code is available at https://github.com/fxmeng/filter-grafting.git.

cs.CV↗

arXiv2020

Understanding the Automated Parameter Optimization on Transfer Learning for CPDP: An Empirical Study

Data-driven defect prediction has become increasingly important in software engineering process. Since it is not uncommon that data from a software project is insufficient for training a reliable defect prediction model, transfer learning that borrows data/knowledge from other projects to facilitate the model building at the current project, namely cross-project defect prediction (CPDP), is naturally plausible. Most CPDP techniques involve two major steps, i.e., transfer learning and classification, each of which has at least one parameter to be tuned to achieve their optimal performance. This practice fits well with the purpose of automated parameter optimization. However, there is a lack of thorough understanding about what are the impacts of automated parameter optimization on various CPDP techniques. In this paper, we present the first empirical study that looks into such impacts on 62 CPDP techniques, 13 of which are chosen from the existing CPDP literature while the other 49 ones have not been explored before. We build defect prediction models over 20 real-world software projects that are of different scales and characteristics. Our findings demonstrate that: (1) Automated parameter optimization substantially improves the defect prediction performance of 77\% CPDP techniques with a manageable computational cost. Thus more efforts on this aspect are required in future CPDP studies. (2) Transfer learning is of ultimate importance in CPDP. Given a tight computational budget, it is more cost-effective to focus on optimizing the parameter configuration of transfer learning algorithms (3) The research on CPDP is far from mature where it is "not difficult" to find a better alternative by making a combination of existing transfer learning and classification techniques. This finding provides important insights about the future design of CPDP techniques.

cs.NE↗

arXiv2020

Surrogate Assisted Evolutionary Algorithm for Medium Scale Expensive Multi-Objective Optimisation Problems

Building a surrogate model of an objective function has shown to be effective to assist evolutionary algorithms (EAs) to solve real-world complex optimisation problems which involve either computationally expensive numerical simulations or costly physical experiments. However, their effectiveness mostly focuses on small-scale problems with less than 10 decision variables. The scalability of surrogate assisted EAs (SAEAs) have not been well studied yet. In this paper, we propose a Gaussian process surrogate model assisted EA for medium-scale expensive multi-objective optimisation problems with up to 50 decision variables. There are three distinctive features of our proposed SAEA. First, instead of using all decision variables in surrogate model building, we only use those correlated ones to build the surrogate model for each objective function. Second, rather than directly optimising the surrogate objective functions, the original multi-objective optimisation problem is transformed to a new one based on the surrogate models. Last but not the least, a subset selection method is developed to choose a couple of promising candidate solutions for actual objective function evaluations thus to update the training dataset. The effectiveness of our proposed algorithm is validated on benchmark problems with 10, 20, 50 variables, comparing with three state-of-the-art SAEAs.

cs.NE↗

arXiv2020

Routing-Led Placement of VNFs in Arbitrary Networks

The ever increasing demand for computing resources has led to the creation of hyperscale datacentres with tens of thousands of servers. As demand continues to rise, new technologies must be incorporated to ensure high quality services can be provided without the damaging environmental impact of high energy consumption. Virtualisation technology such as network function virtualisation (NFV) allows for the creation of services by connecting component parts known as virtual network functions (VNFs). VNFs cam be used to maximally utilise available datacentre resources by optimising the placement and routes of VNFs, to maintain a high quality of service whilst minimising energy costs. Current research on this problem has focussed on placing VNFs and considered routing as a secondary concern. In this work we argue that the opposite approach, a routing-led approach is preferable. We propose a novel routing-led algorithm and analyse each of the component parts over a range of different topologies on problems with up to 16000 variables and compare its performance against a traditional placement based algorithm. Empirical results show that our routing-led algorithm can produce significantly better, faster solutions to large problem instances on a range of datacentre topologies.

cs.NI↗

arXiv2020

Covert Communication in Continuous-Time Systems

Recent works have considered the ability of transmitter Alice to communicate reliably to receiver Bob without being detected by warden Willie. These works generally assume a standard discrete-time model. But the assumption of a discrete-time model in standard communication scenarios is often predicated on its equivalence to a continuous-time model, which has not been established for the covert communications problem. Here, we consider the continuous-time channel directly and study if efficient covert communication can still be achieved. We assume that an uninformed jammer is present to assist Alice, and we consider additive white Gaussian noise (AWGN) channels between all parties. For a channel with approximate bandwidth W, we establish constructions such that O(WT) information bits can be transmitted covertly and reliably from Alice to Bob in T seconds for two separate scenarios: 1) when the path-loss between Alice and Willie is known; and 2) when the path-loss between Alice and Willie is unknown.

cs.CR↗

arXiv2020

Technical Proposal: FASERnu

FASERnu is a proposed small and inexpensive emulsion detector designed to detect collider neutrinos for the first time and study their properties. FASERnu will be located directly in front of FASER, 480 m from the ATLAS interaction point along the beam collision axis in the unused service tunnel TI12. From 2021-23 during Run 3 of the 14 TeV LHC, roughly 1,300 electron neutrinos, 20,000 muon neutrinos, and 20 tau neutrinos will interact in FASERnu with TeV-scale energies. With the ability to observe these interactions, reconstruct their energies, and distinguish flavors, FASERnu will probe the production, propagation, and interactions of neutrinos at the highest human-made energies ever recorded. The FASERnu detector will be composed of 1000 emulsion layers interleaved with tungsten plates. The total volume of the emulsion and tungsten is 25cm x 25cm x 1.35m, and the tungsten target mass is 1.2 tonnes. From 2021-23, 7 sets of emulsion layers will be installed, with replacement roughly every 20-50 1/fb in planned Technical Stops. In this document, we summarize FASERnu's physics goals and discuss the estimates of neutrino flux and interaction rates. We then describe the FASERnu detector in detail, including plans for assembly, transport, installation, and emulsion replacement, and procedures for emulsion readout and analyzing the data. We close with cost estimates for the detector components and infrastructure work and a timeline for the experiment.

physics.ins-det↗

arXiv2019

Asymmetric Co-Teaching for Unsupervised Cross Domain Person Re-Identification

Person re-identification (re-ID), is a challenging task due to the high variance within identity samples and imaging conditions. Although recent advances in deep learning have achieved remarkable accuracy in settled scenes, i.e., source domain, few works can generalize well on the unseen target domain. One popular solution is assigning unlabeled target images with pseudo labels by clustering, and then retraining the model. However, clustering methods tend to introduce noisy labels and discard low confidence samples as outliers, which may hinder the retraining process and thus limit the generalization ability. In this study, we argue that by explicitly adding a sample filtering procedure after the clustering, the mined examples can be much more efficiently used. To this end, we design an asymmetric co-teaching framework, which resists noisy labels by cooperating two models to select data with possibly clean labels for each other. Meanwhile, one of the models receives samples as pure as possible, while the other takes in samples as diverse as possible. This procedure encourages that the selected training samples can be both clean and miscellaneous, and that the two models can promote each other iteratively. Extensive experiments show that the proposed framework can consistently benefit most clustering-based methods, and boost the state-of-the-art adaptation accuracy. Our code is available at https://github.com/FlyingRoastDuck/ACT_AAAI20.

cs.CV↗

arXiv2019

Model Adaption Object Detection System for Robot

Object detection for robot guidance is a crucial mission for autonomous robots, which has provoked extensive attention for researchers. However, the changing view of robot movement and limited available data hinder the research in this area. To address these matters, we proposed a new vision system for robots, the model adaptation object detection system. Instead of using a single one to solve problems, We made use of different object detection neural networks to guide the robot in accordance with various situations, with the help of a meta neural network to allocate the object detection neural networks. Furthermore, taking advantage of transfer learning technology and depthwise separable convolutions, our model is easy to train and can address small dataset problems.

cs.CV↗

arXiv2019

sPortfolio: Stratified Visual Analysis of Stock Portfolios

Quantitative Investment, built on the solid foundation of robust financial theories, is at the center stage in investment industry today. The essence of quantitative investment is the multi-factor model, which explains the relationship between the risk and return of equities. However, the multi-factor model generates enormous quantities of factor data, through which even experienced portfolio managers find it difficult to navigate. This has led to portfolio analysis and factor research being limited by a lack of intuitive visual analytics tools. Previous portfolio visualization systems have mainly focused on the relationship between the portfolio return and stock holdings, which is insufficient for making actionable insights or understanding market trends. In this paper, we present sPortfolio, which, to the best of our knowledge, is the first visualization that attempts to explore the factor investment area. In particular, sPortfolio provides a holistic overview of the factor data and aims to facilitate the analysis at three different levels: a Risk-Factor level, for a general market situation analysis; a Multiple-Portfolio level, for understanding the portfolio strategies; and a Single-Portfolio level, for investigating detailed operations. The system's effectiveness and usability are demonstrated through three case studies. The system has passed its pilot study and is soon to be deployed in industry.

cs.HC↗

arXiv2019

A New Deep Learning Method for Image Deblurring in Optical Microscopic Systems

Deconvolution is the most commonly used image processing method to remove the blur caused by the point-spread-function (PSF) in optical imaging systems. While this method has been successful in deblurring, it suffers from several disadvantages including being slow, since it takes many iterations, suboptimal, in cases where experimental operator chosen to represent PSF is not optimal. In this paper, we are proposing a deep-learning-based deblurring method applicable to optical microscopic imaging systems. We tested the proposed method in database data, simulated data, and experimental data (include 2D optical microscopic data and 3D photoacoustic microscopic data), all of which showed much improved deblurred results compared to deconvolution. To quantify the improved performance, we compared our results against several deconvolution methods. Our results are better than conventional techniques and do not require multiple iterations or pre-determined experimental operator. Our method has the advantages of simple operation, short time to compute, good deblur results and wide application in all types of optical microscopic imaging systems. The deep learning approach opens up a new path for deblurring and can be applied in various biomedical imaging fields.

eess.IV↗

arXiv2019

Does Preference Always Help? A Holistic Study on Preference-Based Evolutionary Multi-Objective Optimisation Using Reference Points

The ultimate goal of multi-objective optimisation is to help a decision maker (DM) identify solution(s) of interest (SOI) achieving satisfactory trade-offs among multiple conflicting criteria. This can be realised by leveraging DM's preference information in evolutionary multi-objective optimisation (EMO). No consensus has been reached on the effectiveness brought by incorporating preference in EMO (either a priori or interactively) versus a posteriori decision making after a complete run of an EMO algorithm. Bearing this consideration in mind, this paper i) provides a pragmatic overview of the existing developments of preference-based EMO; and ii) conducts a series of experiments to investigate the effectiveness brought by preference incorporation in EMO for approximating various SOI. In particular, the DM's preference information is elicited as a reference point, which represents her/his aspirations for different objectives. Experimental results demonstrate that preference incorporation in EMO does not always lead to a desirable approximation of SOI if the DM's preference information is not well utilised, nor does the DM elicit invalid preference information, which is not uncommon when encountering a black-box system. To a certain extent, this issue can be remedied through an interactive preference elicitation. Last but not the least, we find that a preference-based EMO algorithm is able to be generalised to approximate the whole PF given an appropriate setup of preference information.

cs.NE↗

arXiv2019

D3M: A deep domain decomposition method for partial differential equations

A state-of-the-art deep domain decomposition method (D3M) based on the variational principle is proposed for partial differential equations (PDEs). The solution of PDEs can be formulated as the solution of a constrained optimization problem, and we design a multi-fidelity neural network framework to solve this optimization problem. Our contribution is to develop a systematical computational procedure for the underlying problem in parallel with domain decomposition. Our analysis shows that the D3M approximation solution converges to the exact solution of underlying PDEs. Our proposed framework establishes a foundation to use variational deep learning in large-scale engineering problems and designs. We present a general mathematical framework of D3M, validate its accuracy and demonstrate its efficiency with numerical experiments.

cs.LG↗

arXiv2019

Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark

Substantial efforts have been devoted more recently to presenting various methods for object detection in optical remote sensing images. However, the current survey of datasets and deep learning based methods for object detection in optical remote sensing images is not adequate. Moreover, most of the existing datasets have some shortcomings, for example, the numbers of images and object categories are small scale, and the image diversity and variations are insufficient. These limitations greatly affect the development of deep learning based object detection methods. In the paper, we provide a comprehensive review of the recent deep learning based object detection progress in both the computer vision and earth observation communities. Then, we propose a large-scale, publicly available benchmark for object DetectIon in Optical Remote sensing images, which we name as DIOR. The dataset contains 23463 images and 192472 instances, covering 20 object classes. The proposed DIOR dataset 1) is large-scale on the object categories, on the object instance number, and on the total image number; 2) has a large range of object size variations, not only in terms of spatial resolutions, but also in the aspect of inter- and intra-class size variability across objects; 3) holds big variations as the images are obtained with different imaging conditions, weathers, seasons, and image quality; and 4) has high inter-class similarity and intra-class diversity. The proposed benchmark can help the researchers to develop and validate their data-driven methods. Finally, we evaluate several state-of-the-art approaches on our DIOR dataset to establish a baseline for future research.

cs.CV↗

arXiv2019

Diverse Image Synthesis from Semantic Layouts via Conditional IMLE

Most existing methods for conditional image synthesis are only able to generate a single plausible image for any given input, or at best a fixed number of plausible images. In this paper, we focus on the problem of generating images from semantic segmentation maps and present a simple new method that can generate an arbitrary number of images with diverse appearance for the same semantic layout. Unlike most existing approaches which adopt the GAN framework, our method is based on the recently introduced Implicit Maximum Likelihood Estimation (IMLE) framework. Compared to the leading approach, our method is able to generate more diverse images while producing fewer artifacts despite using the same architecture. The learned latent space also has sensible structure despite the lack of supervision that encourages such behaviour. Videos and code are available at https://people.eecs.berkeley.edu/~ke.li/projects/imle/scene_layouts/.

cs.CV↗

arXiv2019

Semi-Supervised Adversarial Monocular Depth Estimation

In this paper, we address the problem of monocular depth estimation when only a limited number of training image-depth pairs are available. To achieve a high regression accuracy, the state-of-the-art estimation methods rely on CNNs trained with a large number of image-depth pairs, which are prohibitively costly or even infeasible to acquire. Aiming to break the curse of such expensive data collections, we propose a semi-supervised adversarial learning framework that only utilizes a small number of image-depth pairs in conjunction with a large number of easily-available monocular images to achieve high performance. In particular, we use one generator to regress the depth and two discriminators to evaluate the predicted depth , i.e., one inspects the image-depth pair while the other inspects the depth channel alone. These two discriminators provide their feedbacks to the generator as the loss to generate more realistic and accurate depth predictions. Experiments show that the proposed approach can (1) improve most state-of-the-art models on the NYUD v2 dataset by effectively leveraging additional unlabeled data sources; (2) reach state-of-the-art accuracy when the training set is small, e.g., on the Make3D dataset; (3) adapt well to an unseen new dataset (Make3D in our case) after training on an annotated dataset (KITTI in our case).

cs.CV↗

arXiv2019

Bayesian Network Based Label Correlation Analysis For Multi-label Classifier Chain

Classifier chain (CC) is a multi-label learning approach that constructs a sequence of binary classifiers according to a label order. Each classifier in the sequence is responsible for predicting the relevance of one label. When training the classifier for a label, proceeding labels will be taken as extended features. If the extended features are highly correlated to the label, the performance will be improved, otherwise, the performance will not be influenced or even degraded. How to discover label correlation and determine the label order is critical for CC approach. This paper employs Bayesian network (BN) to model the label correlations and proposes a new BN-based CC method (BNCC). First, conditional entropy is used to describe the dependency relations among labels. Then, a BN is built up by taking nodes as labels and weights of edges as their dependency relations. A new scoring function is proposed to evaluate a BN structure, and a heuristic algorithm is introduced to optimize the BN. At last, by applying topological sorting on the nodes of the optimized BN, the label order for constructing CC model is derived. Experimental comparisons demonstrate the feasibility and effectiveness of the proposed method.

cs.LG↗

arXiv2019

A hierarchical neural hybrid method for failure probability estimation

Failure probability evaluation for complex physical and engineering systems governed by partial differential equations (PDEs) are computationally intensive, especially when high-dimensional random parameters are involved. Since standard numerical schemes for solving these complex PDEs are expensive, traditional Monte Carlo methods which require repeatedly solving PDEs are infeasible. Alternative approaches which are typically the surrogate based methods suffer from the so-called ``curse of dimensionality'', which limits their application to problems with high-dimensional parameters. For this purpose, we develop a novel hierarchical neural hybrid (HNH) method to efficiently compute failure probabilities of these challenging high-dimensional problems. Especially, multifidelity surrogates are constructed based on neural networks with different levels of layers, such that expensive highfidelity surrogates are adapted only when the parameters are in the suspicious domain. The efficiency of our new HNH method is theoretically analyzed and is demonstrated with numerical experiments. From numerical results, we show that to achieve an accuracy in estimating the rare failure probability (e.g., $10^{-5}$), the traditional Monte Carlo method needs to solve PDEs more than a million times, while our HNH only requires solving them a few thousand times.

math.NA↗

arXiv2019

GRID: a Student Project to Monitor the Transient Gamma-Ray Sky in the Multi-Messenger Astronomy Era

The Gamma-Ray Integrated Detectors (GRID) is a space mission concept dedicated to monitoring the transient gamma-ray sky in the energy range from 10 keV to 2 MeV using scintillation detectors onboard CubeSats in low Earth orbits. The primary targets of GRID are the gamma-ray bursts (GRBs) in the local universe. The scientific goal of GRID is, in synergy with ground-based gravitational wave (GW) detectors such as LIGO and VIRGO, to accumulate a sample of GRBs associated with the merger of two compact stars and study jets and related physics of those objects. It also involves observing and studying other gamma-ray transients such as long GRBs, soft gamma-ray repeaters, terrestrial gamma-ray flashes, and solar flares. With multiple CubeSats in various orbits, GRID is unaffected by the Earth occultation and serves as a full-time and all-sky monitor. Assuming a horizon of 200 Mpc for ground-based GW detectors, we expect to see a few associated GW-GRB events per year. With about 10 CubeSats in operation, GRID is capable of localizing a faint GRB like 170817A with a 90% error radius of about 10 degrees, through triangulation and flux modulation. GRID is proposed and developed by students, with considerable contribution from undergraduate students, and will remain operated as a student project in the future. The current GRID collaboration involves more than 20 institutes and keeps growing. On August 29th, the first GRID detector onboard a CubeSat was launched into a Sun-synchronous orbit and is currently under test.

astro-ph.IM↗