Searcharxiv⌕ Search

arXiv subjects

Kun Wu

Publications and source records attributed to Kun Wu.

71 records · Page 4Linked to original sources
arXiv2023

A neural network-based scale-adaptive cloud-fraction scheme for GCMs

Cloud fraction significantly affects the short- and long-wave radiation. Its realistic representation in general circulation models (GCMs) still poses great challenges in modeling the atmosphere. Here, we present a neural network-based diagnostic scheme that uses the grid-mean temperature, pressure, liquid and ice water mixing ratios, and relative humidity to simulate the sub-grid cloud fraction. The scheme, trained using CloudSat data with explicit consideration of grid sizes, realistically simulates the observed cloud fraction with a correlation coefficient (r) > 0.9 for liquid-, mixed-, and ice-phase clouds. The scheme also captures the observed non-monotonic relationship between cloud fraction and relative humidity and is computationally efficient, and robust for GCMs with a variety of horizontal and vertical resolutions. For illustrative purposes, we conducted comparative analyses of the 2006-2019 climatological-mean cloud fractions among CloudSat, and simulations from the new scheme and the Xu-Randall scheme (optimized the same way as the new scheme). The network-based scheme improves not only the spatial distribution of the total cloud fraction but also the cloud vertical structure (r > 0.99). For example, the biases of too-many high-level clouds over the tropics and too-many low-level clouds over regions around 60°S and 60°N in the Xu-Randall scheme are significantly reduced. These improvements are also found to be insensitive to the spatio-temporal variability of large-scale meteorology conditions, implying that the scheme can be used in different climate regimes.

physics.ao-ph↗
arXiv2023

CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-based Autonomous Urban Driving

Vision-based autonomous urban driving in dense traffic is quite challenging due to the complicated urban environment and the dynamics of the driving behaviors. Widely-applied methods either heavily rely on hand-crafted rules or learn from limited human experience, which makes them hard to generalize to rare but critical scenarios. In this paper, we present a novel CAscade Deep REinforcement learning framework, CADRE, to achieve model-free vision-based autonomous urban driving. In CADRE, to derive representative latent features from raw observations, we first offline train a Co-attention Perception Module (CoPM) that leverages the co-attention mechanism to learn the inter-relationships between the visual and control information from a pre-collected driving dataset. Cascaded by the frozen CoPM, we then present an efficient distributed proximal policy optimization framework to online learn the driving policy under the guidance of particularly designed reward functions. We perform a comprehensive empirical study with the CARLA NoCrash benchmark as well as specific obstacle avoidance scenarios in autonomous urban driving tasks. The experimental results well justify the effectiveness of CADRE and its superiority over the state-of-the-art by a wide margin.

cs.CV↗
arXiv2022

Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsing

Data augmentation has attracted a lot of research attention in the deep learning era for its ability in alleviating data sparseness. The lack of labeled data for unseen evaluation databases is exactly the major challenge for cross-domain text-to-SQL parsing. Previous works either require human intervention to guarantee the quality of generated data, or fail to handle complex SQL queries. This paper presents a simple yet effective data augmentation framework. First, given a database, we automatically produce a large number of SQL queries based on an abstract syntax tree grammar. For better distribution matching, we require that at least 80% of SQL patterns in the training data are covered by generated queries. Second, we propose a hierarchical SQL-to-question generation model to obtain high-quality natural language questions, which is the major contribution of this work. Finally, we design a simple sampling strategy that can greatly improve training efficiency given large amounts of generated data. Experiments on three cross-domain datasets, i.e., WikiSQL and Spider in English, and DuSQL in Chinese, show that our proposed data augmentation framework can consistently improve performance over strong baselines, and the hierarchical generation component is the key for the improvement.

cs.CL↗
arXiv2022

Continual Few-Shot Learning with Adversarial Class Storage

Humans have a remarkable ability to quickly and effectively learn new concepts in a continuous manner without forgetting old knowledge. Though deep learning has made tremendous successes on various computer vision tasks, it faces challenges for achieving such human-level intelligence. In this paper, we define a new problem called continual few-shot learning, in which tasks arrive sequentially and each task is associated with a few training samples. We propose Continual Meta-Learner (CML) to solve this problem. CML integrates metric-based classification and a memory-based mechanism along with adversarial learning into a meta-learning framework, which leads to the desirable properties: 1) it can quickly and effectively learn to handle a new task; 2) it overcomes catastrophic forgetting; 3) it is model-agnostic. We conduct extensive experiments on two image datasets, MiniImageNet and CIFAR100. Experimental results show that CML delivers state-of-the-art performance in terms of classification accuracy on few-shot learning tasks without catastrophic forgetting.

cs.LG↗
arXiv2022

Lesion-Aware Contrastive Representation Learning for Histopathology Whole Slide Images Analysis

Local representation learning has been a key challenge to promote the performance of the histopathological whole slide images analysis. The previous representation learning methods followed the supervised learning paradigm. However, manual annotation for large-scale WSIs is time-consuming and labor-intensive. Hence, the self-supervised contrastive learning has recently attracted intensive attention. The present contrastive learning methods treat each sample as a single class, which suffers from class collision problems, especially in the domain of histopathology image analysis. In this paper, we proposed a novel contrastive representation learning framework named Lesion-Aware Contrastive Learning (LACL) for histopathology whole slide image analysis. We built a lesion queue based on the memory bank structure to store the representations of different classes of WSIs, which allowed the contrastive model to selectively define the negative pairs during the training. Moreover, We designed a queue refinement strategy to purify the representations stored in the lesion queue. The experimental results demonstrate that LACL achieves the best performance in histopathology image representation learning on different datasets, and outperforms state-of-the-art methods under different WSI classification benchmarks. The code is available at https://github.com/junl21/lacl.

cs.CV↗
arXiv2022

Faster and Better Grammar-based Text-to-SQL Parsing via Clause-level Parallel Decoding and Alignment Loss

Grammar-based parsers have achieved high performance in the cross-domain text-to-SQL parsing task, but suffer from low decoding efficiency due to the much larger number of actions for grammar selection than that of tokens in SQL queries. Meanwhile, how to better align SQL clauses and question segments has been a key challenge for parsing performance. Therefore, this paper proposes clause-level parallel decoding and alignment loss to enhance two high-performance grammar-based parsers, i.e., RATSQL and LGESQL. Experimental results of two parsers show that our method obtains consistent improvements both in accuracy and decoding speed.

cs.CL↗
arXiv2022

Theoretical study on the interfacial instability of a spherical droplet subject to vertical vibration

Interfacial instability would be aroused on a spherical liquid droplet when it is subject to external vertical vibration. In this paper, a linear analysis was conducted on this instability problem. The polar-angle dependent acceleration in the spherical coordinate is strongly coupled with the temporal and spatial component of the surface deformation displacement, which gives a recursion equation that implicitly expresses the dispersion relation between the growth rate and spherical mode numbers. The unstable regions (or unstable tongues) for the inviscid fluids considering latitudinal mode (longitudinal mode number m = 0) were derived and presented in the parameter plane. Compared with the solution of the spherical Faraday instability under radial vibration acceleration, the regions of harmonic unstable tongues for the mono-directional vibration case is much narrowed and the subharmonic unstable tongues almost become straight lines. The analysis shows that the latitudinal waves emerging on the spherical droplet surface ought to oscillate harmonically instead of subharmonically, which is opposite to the results for the case under radial vibration acceleration. A corresponding experiment of a liquid droplet lying on a vertically vibrating plate was conducted and the observations substantiate our theoretical predictions.

physics.flu-dyn↗
arXiv2021

Graph Neural Network Training with Data Tiering

Graph Neural Networks (GNNs) have shown success in learning from graph-structured data, with applications to fraud detection, recommendation, and knowledge graph reasoning. However, training GNN efficiently is challenging because: 1) GPU memory capacity is limited and can be insufficient for large datasets, and 2) the graph-based data structure causes irregular data access patterns. In this work, we provide a method to statistical analyze and identify more frequently accessed data ahead of GNN training. Our data tiering method not only utilizes the structure of input graph, but also an insight gained from actual GNN training process to achieve a higher prediction result. With our data tiering method, we additionally provide a new data placement and access strategy to further minimize the CPU-GPU communication overhead. We also take into account of multi-GPU GNN training as well and we demonstrate the effectiveness of our strategy in a multi-GPU system. The evaluation results show that our work reduces CPU-GPU traffic by 87-95% and improves the training speed of GNN over the existing solutions by 1.6-2.1x on graphs with hundreds of millions of nodes and billions of edges.

cs.LG↗
arXiv2021

Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture

Graph Convolutional Networks (GCNs) are increasingly adopted in large-scale graph-based recommender systems. Training GCN requires the minibatch generator traversing graphs and sampling the sparsely located neighboring nodes to obtain their features. Since real-world graphs often exceed the capacity of GPU memory, current GCN training systems keep the feature table in host memory and rely on the CPU to collect sparse features before sending them to the GPUs. This approach, however, puts tremendous pressure on host memory bandwidth and the CPU. This is because the CPU needs to (1) read sparse features from memory, (2) write features into memory as a dense format, and (3) transfer the features from memory to the GPUs. In this work, we propose a novel GPU-oriented data communication approach for GCN training, where GPU threads directly access sparse features in host memory through zero-copy accesses without much CPU help. By removing the CPU gathering stage, our method significantly reduces the consumption of the host resources and data access latency. We further present two important techniques to achieve high host memory access efficiency by the GPU: (1) automatic data access address alignment to maximize PCIe packet efficiency, and (2) asynchronous zero-copy access and kernel execution to fully overlap data transfer with training. We incorporate our method into PyTorch and evaluate its effectiveness using several graphs with sizes up to 111 million nodes and 1.6 billion edges. In a multi-GPU training setup, our method is 65-92% faster than the conventional data transfer method, and can even match the performance of all-in-GPU-memory training for some graphs that fit in GPU memory.

cs.LG↗
arXiv2021

TEMPI: An Interposed MPI Library with a Canonical Representation of CUDA-aware Datatypes

MPI derived datatypes are an abstraction that simplifies handling of non-contiguous data in MPI applications. These datatypes are recursively constructed at runtime from primitive Named Types defined in the MPI standard. More recently, the development and deployment of CUDA-aware MPI implementations has encouraged the transition of distributed high-performance MPI codes to use GPUs. Such implementations allow MPI functions to directly operate on GPU buffers, easing integration of GPU compute into MPI codes. This work first presents a novel datatype handling strategy for nested strided datatypes, which finds a middle ground between the specialized or generic handling in prior work. This work also shows that the performance characteristics of non-contiguous data handling can be modeled with empirical system measurements, and used to transparently improve MPI_Send/Recv latency. Finally, despite substantial attention to non-contiguous GPU data and CUDA-aware MPI implementations, good performance cannot be taken for granted. This work demonstrates its contributions through an MPI interposer library, TEMPI. TEMPI can be used with existing MPI deployments without system or application changes. Ultimately, the interposed-library model of this work demonstrates MPI_Pack speedup of up to 242000x and MPI_Send speedup of up to 59000x compared to the MPI implementation deployed on a leadership-class supercomputer. This yields speedup of more than 917x in a 3D halo exchange with 3072 processes.

cs.DC↗
arXiv2021

PyTorch-Direct: Enabling GPU Centric Data Access for Very Large Graph Neural Network Training with Irregular Accesses

With the increasing adoption of graph neural networks (GNNs) in the machine learning community, GPUs have become an essential tool to accelerate GNN training. However, training GNNs on very large graphs that do not fit in GPU memory is still a challenging task. Unlike conventional neural networks, mini-batching input samples in GNNs requires complicated tasks such as traversing neighboring nodes and gathering their feature values. While this process accounts for a significant portion of the training time, we find existing GNN implementations using popular deep neural network (DNN) libraries such as PyTorch are limited to a CPU-centric approach for the entire data preparation step. This "all-in-CPU" approach has negative impact on the overall GNN training performance as it over-utilizes CPU resources and hinders GPU acceleration of GNN training. To overcome such limitations, we introduce PyTorch-Direct, which enables a GPU-centric data accessing paradigm for GNN training. In PyTorch-Direct, GPUs are capable of efficiently accessing complicated data structures in host memory directly without CPU intervention. Our microbenchmark and end-to-end GNN training results show that PyTorch-Direct reduces data transfer time by 47.1% on average and speeds up GNN training by up to 1.6x. Furthermore, by reducing CPU utilization, PyTorch-Direct also saves system power by 12.4% to 17.5% during training. To minimize programmer effort, we introduce a new "unified tensor" type along with necessary changes to the PyTorch memory allocator, dispatch logic, and placement rules. As a result, users need to change at most two lines of their PyTorch GNN training code for each tensor object to take advantage of PyTorch-Direct.

cs.LG↗
arXiv2020

Knowledge Transfer in Multi-Task Deep Reinforcement Learning for Continuous Control

While Deep Reinforcement Learning (DRL) has emerged as a promising approach to many complex tasks, it remains challenging to train a single DRL agent that is capable of undertaking multiple different continuous control tasks. In this paper, we present a Knowledge Transfer based Multi-task Deep Reinforcement Learning framework (KTM-DRL) for continuous control, which enables a single DRL agent to achieve expert-level performance in multiple different tasks by learning from task-specific teachers. In KTM-DRL, the multi-task agent first leverages an offline knowledge transfer algorithm designed particularly for the actor-critic architecture to quickly learn a control policy from the experience of task-specific teachers, and then it employs an online learning algorithm to further improve itself by learning from new online transition samples under the guidance of those teachers. We perform a comprehensive empirical study with two commonly-used benchmarks in the MuJoCo continuous control task suite. The experimental results well justify the effectiveness of KTM-DRL and its knowledge transfer and online learning algorithms, as well as its superiority over the state-of-the-art by a large margin.

cs.LG↗
arXiv2018

A Machine Learning Approach To Prevent Malicious Calls Over Telephony Networks

Malicious calls, i.e., telephony spams and scams, have been a long-standing challenging issue that causes billions of dollars of annual financial loss worldwide. This work presents the first machine learning-based solution without relying on any particular assumptions on the underlying telephony network infrastructures. The main challenge of this decade-long problem is that it is unclear how to construct effective features without the access to the telephony networks' infrastructures. We solve this problem by combining several innovations. We first develop a TouchPal user interface on top of a mobile App to allow users tagging malicious calls. This allows us to maintain a large-scale call log database. We then conduct a measurement study over three months of call logs, including 9 billion records. We design 29 features based on the results, so that machine learning algorithms can be used to predict malicious calls. We extensively evaluate different state-of-the-art machine learning approaches using the proposed features, and the results show that the best approach can reduce up to 90% unblocked malicious calls while maintaining a precision over 99.99% on the benign call traffic. The results also show the models are efficient to implement without incurring a significant latency overhead. We also conduct ablation analysis, which reveals that using 10 out of the 29 features can reach a performance comparable to using all features.

cs.CR↗
arXiv2015

Bounds on Higgs And Top Quark Masses From The Other Degenerate Vacua Near The Planck Scale With Gravitational Contributions

Based on the weak coupling expansion of gravity, we calculate the gravitational contributions to yukawa coupling, scalar quartic coupling as well as gauge couplings with general Landau-DeWitt gauge-fixing choice and a gauge preserving (of SM gauge group) cut off regularization scheme. We find that the results depend on the Landau-DeWitt gauge-fixing parameter. Based on the two loop RGE of SM couplings with one loop full gravitational contributions in harmonic gauge, we study the constraints on the higgs and top quark mass from the requirement of existing the other degenerate vacua at the Planck-dominated region. Our numerical calculations show that nature will not develop the other degenerate vacua at the Planck-dominated region with current higgs and top quark masses. On the other hand, requiring the existence of the other degenerate vacua at the Planck-dominated region will constrain the higgs and top mass to lie at approximately 130 and 174 GeV, respectively.

hep-ph↗
arXiv2015

Charged Higgs Pair Production at the LHC as a Probe of the Top-Seesaw Assisted Technicolor Models

The top-seesaw assisted technicolor (TC) model, which was proposed recently to explain the 126 GeV Higgs mass discovered by the Large Hadron Colliders (LHC), predicts light and heavy charged Higgs bosons in addition to the neutral Higgses. In this paper we will study the pair productions of the charged Higgs, proceeding through gluon-gluon fusion and quark-anti-quark annihilation, at the LHC in the frame of the top-seesaw assisted TC model. We find that in a large part of parameter space the production cross sections of the light charged Higgs pair at the LHC can be quite large compared with the low standard model backgrounds, while it is impossible for the pair production of the heavy ones to be detected with the strong final mass suppression. Therefore, at the LHC future experiments, the light charged Higgs pair production may be served as a probe of this new TC model.

hep-ph↗