arXiv subjects

Wei Wei

Publications and source records attributed to Wei Wei.

At least 433 records · Page 24Linked to original sources

arXiv2020

XSP: Across-Stack Profiling and Analysis of Machine Learning Models on GPUs

There has been a rapid proliferation of machine learning/deep learning (ML) models and wide adoption of them in many application domains. This has made profiling and characterization of ML model performance an increasingly pressing task for both hardware designers and system providers, as they would like to offer the best possible system to serve ML models with the target latency, throughput, cost, and energy requirements while maximizing resource utilization. Such an endeavor is challenging as the characteristics of an ML model depend on the interplay between the model, framework, system libraries, and the hardware (or the HW/SW stack). Existing profiling tools are disjoint, however, and only focus on profiling within a particular level of the stack, which limits the thoroughness and usefulness of the profiling results. This paper proposes XSP - an across-stack profiling design that gives a holistic and hierarchical view of ML model execution. XSP leverages distributed tracing to aggregate and correlates profile data from different sources. XSP introduces a leveled and iterative measurement approach that accurately captures the latencies at all levels of the HW/SW stack in spite of the profiling overhead. We couple the profiling design with an automated analysis pipeline to systematically analyze 65 state-of-the-art ML models. We demonstrate that XSP provides insights which would be difficult to discern otherwise.

cs.LG↗

arXiv2020

Scalable near-infrared graphene plasmonic resonators exhibiting strong non-local and electron quantization effects

Graphene plasmonic resonators have been broadly studied in the terahertz and mid-infrared ranges because of their electrical tunability and large confinement factors which can enable dramatic enhancement of light-matter coupling. In this work, we demonstrate that the characteristic scaling laws of graphene plasmons change for smaller (< 40 nm) plasmonic wavelengths, expanding the operational frequencies of graphene plasmonic resonators into the near-infrared (NIR) and modifying their optical confinement properties. We utilize a novel bottom-up block copolymer lithography method that substantially improves upon top-down methods to create resonators as narrow as 12 nm over centimeter-scale areas. Measurements of these structures reveal that their plasmonic resonances are strongly influenced by non-local and quantum effects, which push their resonant frequency into the NIR (2.2 um), almost double the frequency of previous experimental works. The confinement factors of these resonators, meanwhile, reach 137 +/- 25, amongst the largest reported in literature for an optical cavity. While our findings indicate that the enhancement of some 'forbidden' transitions are an order of magnitude weaker than predicted, the combined NIR response and large confinement of these structures make them an attractive platform to explore ultra-strongly enhanced spontaneous emission.

physics.app-ph↗

arXiv2020

Ball k-means

This paper presents a novel accelerated exact k-means algorithm called the Ball k-means algorithm, which uses a ball to describe a cluster, focusing on reducing the point-centroid distance computation. The Ball k-means can accurately find the neighbor clusters for each cluster resulting distance computations only between a point and its neighbor clusters' centroids instead of all centroids. Moreover, each cluster can be divided into a stable area and an active area, and the later one can be further divided into annulus areas. The assigned cluster of the points in the stable area is not changed in the current iteration while the points in the annulus area will be adjusted within a few neighbor clusters in the current iteration. Also, there are no upper or lower bounds in the proposed Ball k-means. Furthermore, reducing centroid-centroid distance computation between iterations makes it efficient for large k clustering. The fast speed, no extra parameters and simple design of the Ball k-means make it an all-around replacement of the naive k-means algorithm.

cs.LG↗

arXiv2020

Low-frequency noise parameter extraction method for single layer graphene FETs

In this paper, a detailed parameter extraction methodology is proposed for low-frequency noise (LFN) in single layer (SL) graphene transistors (GFETs) based on a recently established compact LFN model. Drain current and LFN of two short channel back-gated GFETs (L=300, 100 nm) were measured at lower and higher drain voltages, for a wide range of gate voltages covering the region away from charge neutrality point (CNP) up to CNP at p-type operation region. Current-voltage (IV) and LFN data were also available from a long channel SL top solution-gated (SG) GFET (L=5 um), for both p- and n-type regions near and away CNP. At each of these regimes, the appropriate IV and LFN parameters can be accurately extracted. Regarding LFN, mobility fluctuation effect is dominant at CNP and from there the Hooge parameter aH can be extracted while the carrier number fluctuation contribution which is responsible for the well-known M-shape bias dependence of output noise divided by squared drain current, also observed in our data, makes possible the extraction of the NT parameter related to the number of traps. In the less possible case of a Lambda-shape trend, NT and aH can be extracted simultaneously from the region near CNP. Away from CNP, contact resistance can have a significant contribution to LFN and from there the relevant parameter SDR^2 is defined. The LFN parameters described above can be estimated from the low drain voltage region of operation where the effect of Velocity Saturation (VS) mechanism is negligible. VS effect results in the reduction of LFN at higher drain voltages and from there the IV parameter hOmega which represents the phonon energy and is related to VS effect can be derived both from drain current and LFN data.

physics.app-ph↗

arXiv2020

Distributed Generalized Nash Equilibrium Seeking for Energy Sharing Games

With the proliferation of distributed generators and energy storage systems, traditional passive consumers in power systems have been gradually evolving into the so-called "prosumers", i.e., proactive consumers, which can both produce and consume power. To encourage energy exchange among prosumers, energy sharing is increasingly adopted, which is usually formulated as a generalized Nash game (GNG). In this paper, a distributed approach is proposed to seek the Generalized Nash equilibrium (GNE) of the energy sharing game. To this end, we convert the GNG into an equivalent optimization problem. A Krasnosel'ski{ǐ}-Mann iteration type algorithm is thereby devised to solve the problem and consequently find the GNE in a distributed manner. The convergence of the proposed algorithm is proved rigorously based on the nonexpansive operator theory. The performance of the algorithm is validated by experiments with three prosumers, and the scalability is tested by simulations using 123 prosumers.

eess.SY↗

arXiv2020

PONE: A Novel Automatic Evaluation Metric for Open-Domain Generative Dialogue Systems

Open-domain generative dialogue systems have attracted considerable attention over the past few years. Currently, how to automatically evaluate them, is still a big challenge problem. As far as we know, there are three kinds of automatic methods to evaluate the open-domain generative dialogue systems: (1) Word-overlap-based metrics; (2) Embedding-based metrics; (3) Learning-based metrics. Due to the lack of systematic comparison, it is not clear which kind of metrics are more effective. In this paper, we will first measure systematically all kinds of automatic evaluation metrics over the same experimental setting to check which kind is best. Through extensive experiments, the learning-based metrics are demonstrated that they are the most effective evaluation metrics for open-domain generative dialogue systems. Moreover, we observe that nearly all learning-based metrics depend on the negative sampling mechanism, which obtains an extremely imbalanced and low-quality dataset to train a score model. In order to address this issue, we propose a novel and feasible learning-based metric that can significantly improve the correlation with human judgments by using augmented POsitive samples and valuable NEgative samples, called PONE. Extensive experiments demonstrate that our proposed evaluation method significantly outperforms the state-of-the-art learning-based evaluation methods, with an average correlation improvement of 13.18%. In addition, we have publicly released the codes of our proposed method and state-of-the-art baselines.

cs.CL↗

arXiv2020

COVID-19 infection and recovery in various countries: Modeling the dynamics and evaluating the non-pharmaceutical mitigation scenarios

The coronavirus disease 2019 (COVID-19) pandemic radically impacts our lives, while the transmission/infection and recovery dynamics of COVID-19 remain obscure. A time-dependent Susceptible, Exposed, Infectious, and Recovered (SEIR) model was proposed and applied to fit and then predict the time series of COVID-19 evolution observed in the last three months (till 3/22/2020) in various provinces and metropolises in China. The model results revealed the space dependent transmission/infection rate and the significant spatiotemporal variation in the recovery rate, likely due to the continuous improvement of screening techniques and public hospital systems, as well as full city lockdowns in China. The validated SEIR model was then applied to predict COVID-19 evolution in United States, Italy, Japan, and South Korea which have responded differently to monitoring and mitigating COVID-19 so far, although these predictions contain high uncertainty due to the intrinsic change of the maximum infected population and the infection/recovery rates within the different countries. In addition, a stochastic model based on the random walk particle tracking scheme, analogous to a mixing-limited bimolecular reaction model, was developed to evaluate non-pharmaceutical strategies to mitigate COVID-19 spread. Preliminary tests using the stochastic model showed that self-quarantine may not be as efficient as strict social distancing in slowing COVID-19 spread, if not all of the infected people can be promptly diagnosed and quarantined.

q-bio.PE↗

arXiv2020

Exponential Stability of Partial Primal-Dual Gradient Dynamics with Nonsmooth Objective Functions

In this paper, we investigate the continuous time partial primal-dual gradient dynamics (P-PDGD) for solving convex optimization problems with the form $ \min\limits_{x\in X,y\inΩ}\ f({x})+h(y),\ \textit{s.t.}\ A{x}+By=C $, where $ f({x}) $ is strongly convex and smooth, but $ h(y) $ is strongly convex and non-smooth. Affine equality and set constraints are included. We prove the exponential stability of P-PDGD, and bounds on decaying rates are provided. Moreover, it is also shown that the decaying rates can be regulated by setting the stepsize.

math.OC↗

arXiv2020

Crowd Counting via Hierarchical Scale Recalibration Network

The task of crowd counting is extremely challenging due to complicated difficulties, especially the huge variation in vision scale. Previous works tend to adopt a naive concatenation of multi-scale information to tackle it, while the scale shifts between the feature maps are ignored. In this paper, we propose a novel Hierarchical Scale Recalibration Network (HSRNet), which addresses the above issues by modeling rich contextual dependencies and recalibrating multiple scale-associated information. Specifically, a Scale Focus Module (SFM) first integrates global context into local features by modeling the semantic inter-dependencies along channel and spatial dimensions sequentially. In order to reallocate channel-wise feature responses, a Scale Recalibration Module (SRM) adopts a step-by-step fusion to generate final density maps. Furthermore, we propose a novel Scale Consistency loss to constrain that the scale-associated outputs are coherent with groundtruth of different scales. With the proposed modules, our approach can ignore various noises selectively and focus on appropriate crowd scales automatically. Extensive experiments on crowd counting datasets (ShanghaiTech, MALL, WorldEXPO'10, and UCSD) show that our HSRNet can deliver superior results over all state-of-the-art approaches. More remarkably, we extend experiments on an extra vehicle dataset, whose results indicate that the proposed model is generalized to other applications.

cs.CV↗

arXiv2020

Dynamic Graph Correlation Learning for Disease Diagnosis with Incomplete Labels

Disease diagnosis on chest X-ray images is a challenging multi-label classification task. Previous works generally classify the diseases independently on the input image without considering any correlation among diseases. However, such correlation actually exists, for example, Pleural Effusion is more likely to appear when Pneumothorax is present. In this work, we propose a Disease Diagnosis Graph Convolutional Network (DD-GCN) that presents a novel view of investigating the inter-dependency among different diseases by using a dynamic learnable adjacency matrix in graph structure to improve the diagnosis accuracy. To learn more natural and reliable correlation relationship, we feed each node with the image-level individual feature map corresponding to each type of disease. To our knowledge, our method is the first to build a graph over the feature maps with a dynamic adjacency matrix for correlation learning. To further deal with a practical issue of incomplete labels, DD-GCN also utilizes an adaptive loss and a curriculum learning strategy to train the model on incomplete labels. Experimental results on two popular chest X-ray (CXR) datasets show that our prediction accuracy outperforms state-of-the-arts, and the learned graph adjacency matrix establishes the correlation representations of different diseases, which is consistent with expert experience. In addition, we apply an ablation study to demonstrate the effectiveness of each component in DD-GCN.

cs.CV↗

arXiv2020

Differentiable Top-k Operator with Optimal Transport

The top-k operation, i.e., finding the k largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining. However, if the top-k operation is implemented in an algorithmic way, e.g., using bubble algorithm, the resulting model cannot be trained in an end-to-end way using prevalent gradient descent algorithms. This is because these implementations typically involve swapping indices, whose gradient cannot be computed. Moreover, the corresponding mapping from the input scores to the indicator vector of whether this element belongs to the top-k set is essentially discontinuous. To address the issue, we propose a smoothed approximation, namely the SOFT (Scalable Optimal transport-based diFferenTiable) top-k operator. Specifically, our SOFT top-k operator approximates the output of the top-k operation as the solution of an Entropic Optimal Transport (EOT) problem. The gradient of the SOFT operator can then be efficiently approximated based on the optimality conditions of EOT problem. We apply the proposed operator to the k-nearest neighbors and beam search algorithms, and demonstrate improved performance.

cs.LG↗

arXiv2020

Calorimetry with Deep Learning: Particle Simulation and Reconstruction for Collider Physics

Using detailed simulations of calorimeter showers as training data, we investigate the use of deep learning algorithms for the simulation and reconstruction of particles produced in high-energy physics collisions. We train neural networks on shower data at the calorimeter-cell level, and show significant improvements for simulation and reconstruction when using these networks compared to methods which rely on currently-used state-of-the-art algorithms. We define two models: an end-to-end reconstruction network which performs simultaneous particle identification and energy regression of particles when given calorimeter shower data, and a generative network which can provide reasonable modeling of calorimeter showers for different particle types at specified angles and energies. We investigate the optimization of our models with hyperparameter scans. Furthermore, we demonstrate the applicability of the reconstruction model to shower inputs from other detector geometries, specifically ATLAS-like and CMS-like geometries. These networks can serve as fast and computationally light methods for particle shower simulation and reconstruction for current and future experiments at particle colliders.

physics.ins-det↗

arXiv2020

COCO-GAN: Generation by Parts via Conditional Coordinating

Humans can only interact with part of the surrounding environment due to biological restrictions. Therefore, we learn to reason the spatial relationships across a series of observations to piece together the surrounding environment. Inspired by such behavior and the fact that machines also have computational constraints, we propose \underline{CO}nditional \underline{CO}ordinate GAN (COCO-GAN) of which the generator generates images by parts based on their spatial coordinates as the condition. On the other hand, the discriminator learns to justify realism across multiple assembled patches by global coherence, local appearance, and edge-crossing continuity. Despite the full images are never generated during training, we show that COCO-GAN can produce \textbf{state-of-the-art-quality} full images during inference. We further demonstrate a variety of novel applications enabled by teaching the network to be aware of coordinates. First, we perform extrapolation to the learned coordinate manifold and generate off-the-boundary patches. Combining with the originally generated full image, COCO-GAN can produce images that are larger than training samples, which we called "beyond-boundary generation". We then showcase panorama generation within a cylindrical coordinate system that inherently preserves horizontally cyclic topology. On the computation side, COCO-GAN has a built-in divide-and-conquer paradigm that reduces memory requisition during training and inference, provides high-parallelism, and can generate parts of images on-demand.

cs.LG↗

arXiv2019

On continuation criteria for the full compressible Navier-Stokes equations in Lorentz spaces

In this paper, we derive several new sufficient conditions of non-breakdown of strong solutions for for both the 3D heat-conducting compressible Navier-Stokes system and nonhomogeneous incompressible Navier-Stokes equations. First, it is shown that there exists a positive constant $\varepsilon$ such that the solution $(ρ,u,θ)$ to full compressible Navier-Stokes equations can be extended beyond $t=T$ provided that one of the following two conditions holds (1) $ρ\in L^{\infty}(0,T;L^{\infty}(\mathbb{R}^{3}))$, $u\in L^{p,\infty}(0,T;L^{q,\infty}(\mathbb{R}^{3}))$ and $$\| u\|_{L^{p,\infty}(0,T;L^{q,\infty}(\mathbb{R}^{3}))}\leq \varepsilon, ~~\text{with}~~ {2/p}+ {3/q}=1,\ \ q>3;$$ (2) $λ<3μ,$ $ρ\in L^{\infty}(0,T;L^{\infty}(\mathbb{R}^{3}))$, $θ\in L^{p,\infty}(0,T;L^{q,\infty}(\mathbb{R}^{3}))$ and $$\|θ\|_{L^{p,\infty}(0,T; L^{q,\infty}(\mathbb{R}^{3}))}\leq \varepsilon, ~~\text{with}~~ {2/p}+ {3/q}=2,\ \ q>3/2.$$ To the best of our knowledge, this is the first continuation theorem allowing the time direction to be in Lorentz spaces for the compressible fluid. Second, we establish some blow-up criteria in anisotropic Lebesgue spaces to the full Navier-Stokes system. Third, without the condition on $ρ$ in (0.1) and (0.3), the results also hold for the 3D nonhomogeneous incompressible Navier-Stokes equations. The appearance of vacuum in these systems could be allowed.

math.AP↗

arXiv2019

When to Talk: Chatbot Controls the Timing of Talking during Multi-turn Open-domain Dialogue Generation

Despite the multi-turn open-domain dialogue systems have attracted more and more attention and made great progress, the existing dialogue systems are still very boring. Nearly all the existing dialogue models only provide a response when the user's utterance is accepted. But during daily conversations, humans always decide whether to continue to utter an utterance based on the context. Intuitively, a dialogue model that can control the timing of talking autonomously based on the conversation context can chat with humans more naturally. In this paper, we explore the dialogue system that automatically controls the timing of talking during the conversation. Specifically, we adopt the decision module for the existing dialogue models. Furthermore, modeling conversation context effectively is very important for controlling the timing of talking. So we also adopt the graph neural networks to process the context with the natural graph structure. Extensive experiments on two benchmarks show that controlling the timing of talking can effectively improve the quality of dialogue generation, and the proposed methods significantly improve the accuracy of the timing of talking. In addition, we have publicly released the codes of our proposed model.

cs.CL↗

arXiv2019

Gravitational Wave Denoising of Binary Black Hole Mergers with Deep Learning

Gravitational wave detection requires an in-depth understanding of the physical properties of gravitational wave signals, and the noise from which they are extracted. Understanding the statistical properties of noise is a complex endeavor, particularly in realistic detection scenarios. In this article we demonstrate that deep learning can handle the non-Gaussian and non-stationary nature of gravitational wave data, and showcase its application to denoise the gravitational wave signals generated by the binary black hole mergers GW150914, GW170104, GW170608 and GW170814 from advanced LIGO noise. To exhibit the accuracy of this methodology, we compute the overlap between the time-series signals produced by our denoising algorithm, and the numerical relativity templates that are expected to describe these gravitational wave sources, finding overlaps ${\cal{O}}\gtrsim0.99$. We also show that our deep learning algorithm is capable of removing noise anomalies from numerical relativity signals that we inject in real advanced LIGO data. We discuss the implications of these results for the characterization of gravitational wave signals.

gr-qc↗

arXiv2019

Abstract Reasoning with Distracting Features

Abstraction reasoning is a long-standing challenge in artificial intelligence. Recent studies suggest that many of the deep architectures that have triumphed over other domains failed to work well in abstract reasoning. In this paper, we first illustrate that one of the main challenges in such a reasoning task is the presence of distracting features, which requires the learning algorithm to leverage counterevidence and to reject any of the false hypotheses in order to learn the true patterns. We later show that carefully designed learning trajectory over different categories of training data can effectively boost learning performance by mitigating the impacts of distracting features. Inspired by this fact, we propose feature robust abstract reasoning (FRAR) model, which consists of a reinforcement learning based teacher network to determine the sequence of training and a student network for predictions. Experimental results demonstrated strong improvements over baseline algorithms and we are able to beat the state-of-the-art models by 18.7% in the RAVEN dataset and 13.3% in the PGM dataset.

cs.AI↗

arXiv2019

Enabling real-time multi-messenger astrophysics discoveries with deep learning

Multi-messenger astrophysics is a fast-growing, interdisciplinary field that combines data, which vary in volume and speed of data processing, from many different instruments that probe the Universe using different cosmic messengers: electromagnetic waves, cosmic rays, gravitational waves and neutrinos. In this Expert Recommendation, we review the key challenges of real-time observations of gravitational wave sources and their electromagnetic and astroparticle counterparts, and make a number of recommendations to maximize their potential for scientific discovery. These recommendations refer to the design of scalable and computationally efficient machine learning algorithms; the cyber-infrastructure to numerically simulate astrophysical sources, and to process and interpret multi-messenger astrophysics data; the management of gravitational wave detections to trigger real-time alerts for electromagnetic and astroparticle follow-ups; a vision to harness future developments of machine learning and cyber-infrastructure resources to cope with the big-data requirements; and the need to build a community of experts to realize the goals of multi-messenger astrophysics.

gr-qc↗