arXiv subjects

Pengfei Chen

Publications and source records attributed to Pengfei Chen.

At least 73 records · Page 4Linked to original sources

arXiv2021

Function Delivery Network: Extending Serverless Computing for Heterogeneous Platforms

Serverless computing has rapidly grown following the launch of Amazon's Lambda platform. Function-as-a-Service (FaaS) a key enabler of serverless computing allows an application to be decomposed into simple, standalone functions that are executed on a FaaS platform. The FaaS platform is responsible for deploying and facilitating resources to the functions. Several of today's cloud applications spread over heterogeneous connected computing resources and are highly dynamic in their structure and resource requirements. However, FaaS platforms are limited to homogeneous clusters and homogeneous functions and do not account for the data access behavior of functions before scheduling. We introduce an extension of FaaS to heterogeneous clusters and to support heterogeneous functions through a network of distributed heterogeneous target platforms called Function Delivery Network (FDN). A target platform is a combination of a cluster of homogeneous nodes and a FaaS platform on top of it. FDN provides Function-Delivery-as-a-Service (FDaaS), delivering the function to the right target platform. We showcase the opportunities such as varied target platform's characteristics, possibility of collaborative execution between multiple target platforms, and localization of data that the FDN offers in fulfilling two objectives: Service Level Objective (SLO) requirements and energy efficiency when scheduling functions by evaluating over five distributed target platforms using the FDNInspector, a tool developed by us for benchmarking distributed target platforms. Scheduling functions on an edge target platform in our evaluation reduced the overall energy consumption by 17x without violating the SLO requirements in comparison to scheduling on a high-end target platform.

cs.DC↗

arXiv2020

Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise

Supervised learning under label noise has seen numerous advances recently, while existing theoretical findings and empirical results broadly build up on the class-conditional noise (CCN) assumption that the noise is independent of input features given the true label. In this work, we present a theoretical hypothesis testing and prove that noise in real-world dataset is unlikely to be CCN, which confirms that label noise should depend on the instance and justifies the urgent need to go beyond the CCN assumption.The theoretical results motivate us to study the more general and practical-relevant instance-dependent noise (IDN). To stimulate the development of theory and methodology on IDN, we formalize an algorithm to generate controllable IDN and present both theoretical and empirical evidence to show that IDN is semantically meaningful and challenging. As a primary attempt to combat IDN, we present a tiny algorithm termed self-evolution average label (SEAL), which not only stands out under IDN with various noise fractions, but also improves the generalization on real-world noise benchmark Clothing1M. Our code is released. Notably, our theoretical analysis in Section 2 provides rigorous motivations for studying IDN, which is an important topic that deserves more research attention in future.

cs.LG↗

arXiv2020

Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels

For multi-class classification under class-conditional label noise, we prove that the accuracy metric itself can be robust. We concretize this finding's inspiration in two essential aspects: training and validation, with which we address critical issues in learning with noisy labels. For training, we show that maximizing training accuracy on sufficiently many noisy samples yields an approximately optimal classifier. For validation, we prove that a noisy validation set is reliable, addressing the critical demand of model selection in scenarios like hyperparameter-tuning and early stopping. Previously, model selection using noisy validation samples has not been theoretically justified. We verify our theoretical results and additional claims with extensive experiments. We show characterizations of models trained with noisy labels, motivated by our theoretical results, and verify the utility of a noisy validation set by showing the impressive performance of a framework termed noisy best teacher and student (NTS). Our code is released.

cs.LG↗

arXiv2020

Integrated Traffic Simulation-Prediction System using Neural Networks with Application to the Los Angeles International Airport Road Network

Transportation networks are highly complex and the design of efficient traffic management systems is difficult due to lack of adequate measured data and accurate predictions of the traffic states. Traffic simulation models can capture the complex dynamics of transportation networks by using limited available traffic data and can help central traffic authorities in their decision-making, if appropriate input is fed into the simulator. In this paper, we design an integrated simulation-prediction system which estimates the Origin-Destination (OD) matrix of a road network using only flow rate information and predicts the behavior of the road network in different simulation scenarios. The proposed system includes an optimization-based OD matrix generation method, a Neural Network (NN) model trained to predict OD matrices via the pattern of traffic flow and a microscopic traffic simulator with a Dynamic Traffic Assignment (DTA) scheme to predict the behavior of the transportation system. We test the proposed system on the road network of the central terminal area (CTA) of the Los Angeles International Airport (LAX), which demonstrates that the integrated traffic simulation-prediction system can be used to simulate the effects of several real world scenarios such as lane closures, curbside parking and other changes. The model is an effective tool for learning the impact and possible benefits of changes in the network and for analyzing scenarios at a very low cost without disrupting the network.

eess.SP↗

arXiv2020

Inviscid Limit for the Free-Boundary problems of MHD Equations with or without Surface Tension

In this paper, we investigate the convergence rates of inviscid limits for the free-boundary problems of the incompressible magnetohydrodynamics (MHD) with or without surface tension in $\mathbb{R}^3$, where the magnetic field is identically constant on the surface and outside of the domain. First, we establish the vorticity, the normal derivatives and the regularity structure of the solutions, and develop a priori co-norm estimates including time derivatives by the vorticity system. Second, we obtain two independent sufficient conditions for the existence of strong vorticity layers: (I) the limit of the difference between the initial MHD vorticity of velocity or magnetic field and that of the ideal MHD equations is nonzero. (II) The cross product of tangential projection on the free surface of the ideal MHD strain tensor of velocity or magnetic field with the normal vector of the free surface is nonzero. Otherwise, the vorticity layer is weak. Third, we prove high order convergence rates of tangential derivatives and the first order normal derivative in standard Sobolev space, where the convergence rates depend on the ideal MHD boundary value.

math.AP↗

arXiv2019

Alchemy: A Quantum Chemistry Dataset for Benchmarking AI Models

We introduce a new molecular dataset, named Alchemy, for developing machine learning models useful in chemistry and material science. As of June 20th 2019, the dataset comprises of 12 quantum mechanical properties of 119,487 organic molecules with up to 14 heavy atoms, sampled from the GDB MedChem database. The Alchemy dataset expands the volume and diversity of existing molecular datasets. Our extensive benchmarks of the state-of-the-art graph neural network models on Alchemy clearly manifest the usefulness of new data in validating and developing machine learning models for chemistry and material science. We further launch a contest to attract attentions from researchers in the related fields. More details can be found on the contest website \footnote{https://alchemy.tencent.com}. At the time of benchamrking experiment, we have generated 119,487 molecules in our Alchemy dataset. More molecular samples are generated since then. Hence, we provide a list of molecules used in the reported benchmarks.

cs.LG↗

arXiv2019

Utilizing Edge Features in Graph Neural Networks via Variational Information Maximization

Graph Neural Networks (GNNs) achieve an impressive performance on structured graphs by recursively updating the representation vector of each node based on its neighbors, during which parameterized transformation matrices should be learned for the node feature updating. However, existing propagation schemes are far from being optimal since they do not fully utilize the relational information between nodes. We propose the information maximizing graph neural networks (IGNN), which maximizes the mutual information between edge states and transform parameters. We reformulate the mutual information as a differentiable objective via a variational approach. We compare our model against several recent variants of GNNs and show that our model achieves the state-of-the-art performance on multiple tasks including quantum chemistry regression on QM9 dataset, generalization capability from QM9 to larger molecular graphs, and prediction of molecular bioactivities relevant for drug discovery. The IGNN model is based on an elegant and fundamental idea in information theory as explained in the main text, and it could be easily generalized beyond the contexts of molecular graphs considered in this work. To encourage more future work in this area, all datasets and codes used in this paper will be released for public access.

cs.LG↗

arXiv2019

A Meta Approach to Defend Noisy Labels by the Manifold Regularizer PSDR

Noisy labels are ubiquitous in real-world datasets, which poses a challenge for robustly training deep neural networks (DNNs) since DNNs can easily overfit to the noisy labels. Most recent efforts have been devoted to defending noisy labels by discarding noisy samples from the training set or assigning weights to training samples, where the weight associated with a noisy sample is expected to be small. Thereby, these previous efforts result in a waste of samples, especially those assigned with small weights. The input $x$ is always useful regardless of whether its observed label $y$ is clean. To make full use of all samples, we introduce a manifold regularizer, named as Paired Softmax Divergence Regularization (PSDR), to penalize the Kullback-Leibler (KL) divergence between softmax outputs of similar inputs. In particular, similar inputs can be effectively generated by data augmentation. PSDR can be easily implemented on any type of DNNs to improve the robustness against noisy labels. As empirically demonstrated on benchmark datasets, our PSDR impressively improve state-of-the-art results by a significant margin.

cs.LG↗

arXiv2019

Disentangling Dynamics and Returns: Value Function Decomposition with Future Prediction

Value functions are crucial for model-free Reinforcement Learning (RL) to obtain a policy implicitly or guide the policy updates. Value estimation heavily depends on the stochasticity of environmental dynamics and the quality of reward signals. In this paper, we propose a two-step understanding of value estimation from the perspective of future prediction, through decomposing the value function into a reward-independent future dynamics part and a policy-independent trajectory return part. We then derive a practical deep RL algorithm from the above decomposition, consisting of a convolutional trajectory representation model, a conditional variational dynamics model to predict the expected representation of future trajectory and a convex trajectory return model that maps a trajectory representation to its return. Our algorithm is evaluated in MuJoCo continuous control tasks and shows superior results under both common settings and delayed reward settings.

cs.LG↗

arXiv2019

Rethinking the Usage of Batch Normalization and Dropout in the Training of Deep Neural Networks

In this work, we propose a novel technique to boost training efficiency of a neural network. Our work is based on an excellent idea that whitening the inputs of neural networks can achieve a fast convergence speed. Given the well-known fact that independent components must be whitened, we introduce a novel Independent-Component (IC) layer before each weight layer, whose inputs would be made more independent. However, determining independent components is a computationally intensive task. To overcome this challenge, we propose to implement an IC layer by combining two popular techniques, Batch Normalization and Dropout, in a new manner that we can rigorously prove that Dropout can quadratically reduce the mutual information and linearly reduce the correlation between any pair of neurons with respect to the dropout layer parameter $p$. As demonstrated experimentally, the IC layer consistently outperforms the baseline approaches with more stable training process, faster convergence speed and better convergence limit on CIFAR10/100 and ILSVRC2012 datasets. The implementation of our IC layer makes us rethink the common practices in the design of neural networks. For example, we should not place Batch Normalization before ReLU since the non-negative responses of ReLU will make the weight layer updated in a suboptimal way, and we can achieve better performance by combining Batch Normalization and Dropout together as an IC layer.

cs.LG↗

arXiv2019

Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels

Noisy labels are ubiquitous in real-world datasets, which poses a challenge for robustly training deep neural networks (DNNs) as DNNs usually have the high capacity to memorize the noisy labels. In this paper, we find that the test accuracy can be quantitatively characterized in terms of the noise ratio in datasets. In particular, the test accuracy is a quadratic function of the noise ratio in the case of symmetric noise, which explains the experimental findings previously published. Based on our analysis, we apply cross-validation to randomly split noisy datasets, which identifies most samples that have correct labels. Then we adopt the Co-teaching strategy which takes full advantage of the identified samples to train DNNs robustly against noisy labels. Compared with extensive state-of-the-art methods, our strategy consistently improves the generalization performance of DNNs under both synthetic and real-world training noise.

cs.LG↗

arXiv2018

Fully Coupled Simulation of Cosmic Reionization. III. Stochastic Early Reionization by the Smallest Galaxies

Previously we identified a new class of early galaxy that we estimate contributes up to 30\% of the ionizing photons responsible for reionization. These are low mass halos in the range $M_h =10^{6.5}-10^{8} M_{\odot}$ that have been chemically enriched by supernova ejecta from prior Pop III star formation. Despite their low star formation rates, these Metal Cooling halos (MCs) are significant sources of ionizing radiation, especially at the onset of reionization, due to their high number density and ionizing escape fractions. Here we present a fully-coupled radiation hydrodynamic simulation of reionization that includes these MCs as well the more massive hydrogen atomic line cooling halos. Our method is novel: we perform halo finding inline with the radiation hydrodynamical simulation, and assign escaping ionizing fluxes to halos using a probability distribution function (PDF) measured from the galaxy-resolving Renaissance Simulations. The PDF captures the mass dependence of the ionizing escape fraction as well as the probability that a halo is actively forming stars. With MCs, reionization starts earlier than if only halos of $10^8 M_{\odot}$ and above are included, however the redshift when reionization completes is only marginally affected as this is driven by more massive galaxies. Because star formation is intermittent in MCs, the earliest phase of reionization exhibits a stochastic nature, with small H II regions forming and recombining. Only later, once halos of mass $\sim 10^9 M_{\odot}$ and above begin to dominate the ionizing emissivity, does reionization proceed smoothly in the usual manner deduced from previous studies. This occurs at $z\approx 10$ in our simulation.

astro-ph.CO↗

arXiv2018

Understanding Convolution for Semantic Segmentation

Recent advances in deep learning, especially deep convolutional neural networks (CNNs), have led to significant improvement over previous semantic segmentation systems. Here we show how to improve pixel-wise semantic segmentation by manipulating convolution-related operations that are of both theoretical and practical value. First, we design dense upsampling convolution (DUC) to generate pixel-level prediction, which is able to capture and decode more detailed information that is generally missing in bilinear upsampling. Second, we propose a hybrid dilated convolution (HDC) framework in the encoding phase. This framework 1) effectively enlarges the receptive fields (RF) of the network to aggregate global information; 2) alleviates what we call the "gridding issue" caused by the standard dilated convolution operation. We evaluate our approaches thoroughly on the Cityscapes dataset, and achieve a state-of-art result of 80.1% mIOU in the test set at the time of submission. We also have achieved state-of-the-art overall on the KITTI road estimation benchmark and the PASCAL VOC2012 segmentation task. Our source code can be found at https://github.com/TuSimple/TuSimple-DUC .

cs.CV↗

arXiv2017

Vanishing Viscosity Limit For the 3D Nonhomogeneous Incompressible Navier-Stokes Equations With a Slip Boundary Condition

In this paper, we investigate the vanishing viscosity limit for the 3D nonhomogeneous incompressible Navier-Stokes equations with a slip boundary condition. We establish the local well-posedness of the strong solutions for initial boundary value problems for such systems. Furthermore, the vanishing viscosity limit process is established and a strong rate of convergence is obtained as the boundary of the domain is flat. In addition, it is needed to add some additional condition for density to match well the boundary condition.

math.AP↗

arXiv2015

Making Availability as a Service in the Clouds

Cloud computing has achieved great success in modern IT industry as an excellent computing paradigm due to its flexible management and elastic resource sharing. To date, cloud computing takes an irrepalceable position in our socioeconomic system and influences almost every aspect of our daily life. However, it is still in its infancy, many problems still exist.Besides the hotly-debated security problem, availability is also an urgent issue.With the limited power of availability mechanisms provided in present cloud platform, we can hardly get detailed availability information of current applications such as the root causes of availability problem,mean time to failure, etc. Thus a new mechanism based on deep avaliability analysis is neccessary and benificial.Following the prevalent terminology 'XaaS',this paper proposes a new win-win concept for cloud users and providers in term of 'Availability as a Service' (abbreviated as 'AaaS').The aim of 'AaaS' is to provide comprehensive and aimspecific runtime avaliabilty analysis services for cloud users by integrating plent of data-driven and modeldriven approaches. To illustrate this concept, we realize a prototype named 'EagleEye' with all features of 'AaaS'. By subscribing corresponding services in 'EagleEye', cloud users could get specific availability information of their applications deployed in cloud platform. We envision this new kind of service will be merged into the cloud management mechanism in the near future.

cs.DC↗

arXiv2015

CHAOS: Accurate and Realtime Detection of Aging-Oriented Failure Using Entropy

Even well-designed software systems suffer from chronic performance degradation, also named "software aging", due to internal (e.g. software bugs) and external (e.g. resource exhaustion) impairments. These chronic problems often fly under the radar of software monitoring systems before causing severe impacts (e.g. system failure). Therefore it's a challenging issue how to timely detect these problems to prevent system crash. Although a large quantity of approaches have been proposed to solve this issue, the accuracy and effectiveness of these approaches are still far from satisfactory due to the insufficiency of aging indicators adopted by them. In this paper, we present a novel entropy-based aging indicator, Multidimensional Multi-scale Entropy (MMSE). MMSE employs the complexity embedded in runtime performance metrics to indicate software aging and leverages multi-scale and multi-dimension integration to tolerate system fluctuations. Via theoretical proof and experimental evaluation, we demonstrate that MMSE satisfies Stability, Monotonicity and Integration which we conjecture that an ideal aging indicator should have. Based upon MMSE, we develop three failure detection approaches encapsulated in a proof-of-concept named CHAOS. The experimental evaluations in a Video on Demand (VoD) system and in a real-world production system, AntVision, show that CHAOS can detect the failure-prone state in an extraordinarily high accuracy and a near 0 Ahead-Time-To-Failure (ATTF). Compared to previous approaches, CHAOS improves the detection accuracy by about 5 times and reduces the ATTF even by 3 orders of magnitude. In addition, CHAOS is light-weight enough to satisfy the realtime requirement.

cs.OH↗

arXiv2014

Scaling Relations for Galaxies Prior to Reionization

The first galaxies in the Universe are the building blocks of all observed galaxies. We present scaling relations for galaxies forming at redshifts $z \ge 15$ when reionization is just beginning. We utilize the ``Rarepeak' cosmological radiation hydrodynamics simulation that captures the complete star formation history in over 3,300 galaxies, starting with massive Population III stars that form in dark matter halos as small as ~$10^6 M_\odot$. We make various correlations between the bulk halo quantities, such as virial, gas, and stellar masses and metallicities and their respective accretion rates, quantifying a variety of properties of the first galaxies up to halo masses of $10^9 M_\odot$. Galaxy formation is not solely relegated to atomic cooling halos with virial temperatures greater than $10^4$ K, where we find a dichotomy in galaxy properties between halos above and below this critical mass scale. Halos below the atomic cooling limit have a stellar mass -- halo mass relationship $\log M_\star \simeq 3.5 + 1.3\log(M_{\rm vir} / 10^7 M_\odot)$. We find a non-monotonic relationship between metallicity and halo mass for the smallest galaxies. Their initial star formation events enrich the interstellar medium and subsequent star formation to a median of $10^{-2} Z_\odot$ and $10^{-1.5} Z_\odot$, respectively, in halos of total mass $10^7 M_\odot$ that is then diluted by metal-poor inflows, well beyond Population III pre-enrichment levels of $10^{-3.5} Z_\odot$. The scaling relations presented here can be employed in models of reionization, galaxy formation and chemical evolution in order to consider these galaxies forming prior to reionization.

astro-ph.GA↗

arXiv2014

Bio-inspired Mechanism and Model Exploration of Software Aging

Software systems situated in network environment may experience performance degradation, availability decrease and even crash during long time running, which is called software aging. This phenomenon has been studied for more than 15 years, but most of the literatures studied software as a black box, none of them uncovered the fundamental and widely accepted mechanism of software aging as far as we know. Through analyzing the characteristics between biological aging and software aging, we find some interesting common points and bridge the gap between these two seemingly unrelated phenomena. The free radical aging theory in biological studies is also applicative to explore the mechanism and model of software aging. This paper finds an equivalent concept named `software free radical' in software aging to free radical in biological aging. In our study, the accumulation of `software free radical' is a root cause of software aging. Using the free radical modeling methodology in biological aging, we give a model for describing the kinetic of software aging based on feedback loops. Although this paper doesn't give enough theoretical proof of the modeling method, the practical results show that the feedback loop model can describe the kinetic of software aging precisely. To further validate the aging mechanism, we propose several software rejuvenation strategies focusing on cleaning the `software free radical'. The results show that software aging can be mitigated effectively by strengthening negative feedback loop or weakening positive feedback loop. This paper is the first try to answer the question `How software ages' through interdisciplinary studies. Leveraging the conclusions in this paper, people can design better software systems or keep their systems at a high performance level during long time running.

cs.SE↗