arXiv subjects

Duo Wang

Publications and source records attributed to Duo Wang.

66 records · Page 4Linked to original sources

arXiv2020

Probabilistic Dual Network Architecture Search on Graphs

We present the first differentiable Network Architecture Search (NAS) for Graph Neural Networks (GNNs). GNNs show promising performance on a wide range of tasks, but require a large amount of architecture engineering. First, graphs are inherently a non-Euclidean and sophisticated data structure, leading to poor adaptivity of GNN architectures across different datasets. Second, a typical graph block contains numerous different components, such as aggregation and attention, generating a large combinatorial search space. To counter these problems, we propose a Probabilistic Dual Network Architecture Search (PDNAS) framework for GNNs. PDNAS not only optimises the operations within a single graph block (micro-architecture), but also considers how these blocks should be connected to each other (macro-architecture). The dual architecture (micro- and marco-architectures) optimisation allows PDNAS to find deeper GNNs on diverse datasets with better performance compared to other graph NAS methods. Moreover, we use a fully gradient-based search approach to update architectural parameters, making it the first differentiable graph NAS method. PDNAS outperforms existing hand-designed GNNs and NAS results, for example, on the PPI dataset, PDNAS beats its best competitors by 1.67 and 0.17 in F1 scores.

cs.LG↗

arXiv2020

Are Registration Uncertainty and Error Monotonically Associated

In image-guided neurosurgery, current commercial systems usually provide only rigid registration, partly because it is harder to predict, validate and understand non-rigid registration error. For instance, when surgeons see a discrepancy in aligned image features, they may not be able to distinguish between registration error and actual tissue deformation caused by tumor resection. In this case, the spatial distribution of registration error could help them make more informed decisions, e.g., ignoring the registration where the estimated error is high. However, error estimates are difficult to acquire. Probabilistic image registration (PIR) methods provide measures of registration uncertainty, which could be a surrogate for assessing the registration error. It is intuitive and believed by many clinicians that high uncertainty indicates a large error. However, the monotonic association between uncertainty and error has not been examined in image registration literature. In this pilot study, we attempt to address this fundamental problem by looking at one PIR method, the Gaussian process (GP) registration. We systematically investigate the relation between GP uncertainty and error based on clinical data and show empirically that there is a weak-to-moderate positive monotonic correlation between point-wise GP registration uncertainty and non-rigid registration error.

cs.CV↗

arXiv2019

fastMRI: An Open Dataset and Benchmarks for Accelerated MRI

Accelerating Magnetic Resonance Imaging (MRI) by taking fewer measurements has the potential to reduce medical costs, minimize stress to patients and make MRI possible in applications where it is currently prohibitively slow or expensive. We introduce the fastMRI dataset, a large-scale collection of both raw MR measurements and clinical MR images, that can be used for training and evaluation of machine-learning approaches to MR image reconstruction. By introducing standardized evaluation criteria and a freely-accessible dataset, our goal is to help the community make rapid advances in the state of the art for MR image reconstruction. We also provide a self-contained introduction to MRI for machine learning researchers with no medical imaging background.

cs.CV↗

arXiv2019

A Hybrid Approach with Optimization and Metric-based Meta-Learner for Few-Shot Learning

Few-shot learning aims to learn classifiers for new classes with only a few training examples per class. Most existing few-shot learning approaches belong to either metric-based meta-learning or optimization-based meta-learning category, both of which have achieved successes in the simplified "$k$-shot $N$-way" image classification settings. Specifically, the optimization-based approaches train a meta-learner to predict the parameters of the task-specific classifiers. The task-specific classifiers are required to be homogeneous-structured to ease the parameter prediction, so the meta-learning approaches could only handle few-shot learning problems where the tasks share a uniform number of classes. The metric-based approaches learn one task-invariant metric for all the tasks. Even though the metric-learning approaches allow different numbers of classes, they require the tasks all coming from a similar domain such that there exists a uniform metric that could work across tasks. In this work, we propose a hybrid meta-learning model called Meta-Metric-Learner which combines the merits of both optimization- and metric-based approaches. Our meta-metric-learning approach consists of two components, a task-specific metric-based learner as a base model, and a meta-learner that learns and specifies the base model. Thus our model is able to handle flexible numbers of classes as well as generate more generalized metrics for classification across tasks. We test our approach in the standard "$k$-shot $N$-way" few-shot learning setting following previous works and a new realistic few-shot setting with flexible class numbers in both single-source form and multi-source forms. Experiments show that our approach can obtain superior performance in all settings.

cs.LG↗

arXiv2019

Mixed-Supervised Dual-Network for Medical Image Segmentation

Deep learning based medical image segmentation models usually require large datasets with high-quality dense segmentations to train, which are very time-consuming and expensive to prepare. One way to tackle this challenge is by using the mixed-supervised learning framework, in which only a part of data is densely annotated with segmentation label and the rest is weakly labeled with bounding boxes. The model is trained jointly in a multi-task learning setting. In this paper, we propose Mixed-Supervised Dual-Network (MSDN), a novel architecture which consists of two separate networks for the detection and segmentation tasks respectively, and a series of connection modules between the layers of the two networks. These connection modules are used to transfer useful information from the auxiliary detection task to help the segmentation task. We propose to use a recent technique called "Squeeze and Excitation" in the connection module to boost the transfer. We conduct experiments on two medical image segmentation datasets. The proposed MSDN model outperforms multiple baselines.

cs.CV↗

arXiv2019

On orbit performance of the GRACE Follow-On Laser Ranging Interferometer

The Laser Ranging Interferometer (LRI) instrument on the Gravity Recovery and Climate Experiment (GRACE) Follow-On mission has provided the first laser interferometric range measurements between remote spacecraft, separated by approximately 220 km. Autonomous controls that lock the laser frequency to a cavity reference and establish the 5 degree of freedom two-way laser link between remote spacecraft succeeded on the first attempt. Active beam pointing based on differential wavefront sensing compensates spacecraft attitude fluctuations. The LRI has operated continuously without breaks in phase tracking for more than 50 days, and has shown biased range measurements similar to the primary ranging instrument based on microwaves, but with much less noise at a level of $1\,{\rm nm}/\sqrt{\rm Hz}$ at Fourier frequencies above 100 mHz.

astro-ph.IM↗

arXiv2019

Towards Obtaining 2D and 3D and 1D PtPN with Pentagonal Pattern

We apply an alloying strategy to single-layer PtN$_2$ and PtP$_2$, aiming to obtain a single-layer Pt-P-N alloy with a relatively low formation energy with reference to its bulk structure. We perform structure search based on a cluster-expansion method and predict single-layer and bulk PtPN consisting of pentagonal networks. The formation energy of single-layer PtPN is significantly lower in comparison with that of single-layer PtP$_2$. The predicted bulk structure of PtPN adopts a structure that is similar to the pyrite structure. We also find that single-layer pentagonal PtPN, unlike PtN$_2$ and PtP$_2$, exhibits a sizable, direct PBE band gap of 0.84 eV. Furthermore, the band gap of single-layer pentagonal PtPN calculated with the hybrid density functional theory is 1.60 eV, which is within visible light spectrum and promising for optoelectronics applications. In addition to predicting PtPN in the 2D and 3D forms, we study the flexural rigidity and electronic structure of PtPN in the nanotube form. We find that single-layer PtPN has similar flexural rigidity to that of single-layer carbon and boron nitride nanosheets and that the band gaps of PtPN nanotubes depend on their radii. Our work shed light on obtaining an isolated 2D planar, pentagonal PtPN nanosheet from its 3D counterpart and on obtaining 1D nanotubes with tunable bandgaps.

cond-mat.mtrl-sci↗

arXiv2019

Unified Flow and Thermal Law-of-the-wall Complete Formulations for Type-A Turbulent Boundary Layers

In-depth analyses of existing direct numerical simulations (DNS) data from various sources supported a logical and important classification of generic turbulent boundary layers (TBL), namely Type-A, -B and -C TBL, based on distribution patterns of time-averaged wall-shear stress. Among these types, Type-A TBL and its related law, as represented by the DNS data of turbulence on a zero-pressure-gradient semi-infinite flat-plate, was investigated in terms of analytical formulations of velocity independent on Reynolds ( ) number. With reference to the analysis from von Karman in developing the conventional law-of-the-wall, the current study first physically distinguished the time-averaged local scale used by von Karman from the ensemble-averaged scale defined in the paper, and then derived the governing equations with the -independency under the ensemble-averaged scales. Based on indicator function (IDF) and TBL thickness, the sublayer partitions were rigorously defined. The analytical formulations for entire TBL, namely the complete law-of-the-wall, were established, including the formula in inner, buffer, semi-logarithmic (semi-log) and wake layer. The researches were featured by introducing the general damping and enhancing functions (GDF and GEF) and applying these functions to both linear and logarithmic coordinates. These law formulations were proved uniform and consistent in time-averaged local and ensemble-averaged scales, which were validated by the existing DNS and experiment data. Based on the similarity of relevant properly-scaled governing equations, the law formulations were logically reasoned being applicable to the temperature in Type-A thermal TBL. The findings advance the current understandings of the conventional TBL theory and its well-known foundations of law-of-the-wall.

physics.flu-dyn↗

arXiv2019

Unsupervised and interpretable scene discovery with Discrete-Attend-Infer-Repeat

In this work we present Discrete Attend Infer Repeat (Discrete-AIR), a Recurrent Auto-Encoder with structured latent distributions containing discrete categorical distributions, continuous attribute distributions, and factorised spatial attention. While inspired by the original AIR model andretaining AIR model's capability in identifying objects in an image, Discrete-AIR provides direct interpretability of the latent codes. We show that for Multi-MNIST and a multiple-objects version of dSprites dataset, the Discrete-AIR model needs just one categorical latent variable, one attribute variable (for Multi-MNIST only), together with spatial attention variables, for efficient inference. We perform analysis to show that the learnt categorical distributions effectively capture the categories of objects in the scene for Multi-MNIST and for Multi-Sprites.

cs.CV↗

arXiv2019

Dimension Engineering of Single-Layer PtN$_2$ with the Cairo Tessellation

Single-layer PtN$_2$ exhibits an intriguing structure consisting of a tessellation pattern called the Cairo tessellation of type 2 pentagons, which belong to one of the existing 15 types of convex pentagons discovered so far that can monohedrally tile a plane. Single-layer PtN$_2$ has also been predicted to show semiconducting behavior with direct band gaps. Full exploration of the structure-property relationship awaits the successful exfoliation or synthesis of this novel single-layer material, which depends on the structure of its bulk counterpart with the same stoichiometry to some extent. Bulk PtN$_2$ with the pyrite structure is commonly regarded as the most stable structure in the literature. But comparing the energies of single-layer PtN$_2$ and bulk PtN$_2$ leads to a dilemma that a single-layer material is more stable than its bulk counterpart. To solve this dilemma, we propose stacking single-layer PtN$_2$ sheets infinitely to form a new bulk structure of PtN$_2$. The resulting tetrahedral layered structure is energetically more stable than the pyrite structure and single-layer PtN$_2$. We also find that the predicted bulk structure is metallic, in contrast to the semiconducting pyrite structure. In addition to predicting the 3D structure, we explore the possibility of rolling single-layer PtN$_2$ sheets into nanotubes. The required energies are comparable to those needed to form carbon or boron nitride nanotubes from their single-layer sheets, implying the feasibility of obtaining PtN$_2$ nanotubes. We finally study the electronic structures of PtN$_2$ nanotubes and find that the band gaps of PtN$_2$ nanotubes are tunable by changing the number of unit cells $N$ of single-layer PtN$_2$ used to construct the nanotubes. Our work shows that dimension engineering of PtN$_2$ not only leads to a more stable 3D structure but also 1D materials with novel properties.

cond-mat.mtrl-sci↗

arXiv2018

Unified Flow and Thermal Law-of-the-wall for Type-A and B Turbulent Boundary Layers

Systematic study of the existing statistical data from direct numerical simulations (DNS) lead to a logical and important classification of generic turbulent boundary layers (TBL), namely Type A,B and C TBL, based on the distribution patterns of time averaged wall shear stress. Among these three types, Type A and B TBLs are investigated in terms of its universal statistical laws of velocity and temperature with independency on both Reynolds (Re) and Prandtl (Pr) numbers. With reference to the analysis by von Karman in developing the conventional law of the wall for Type A TBL, the current study first physically distinguishes the time averaged local velocity scale from the ensemble averaged velocity scale, and properly defines the TBL temperature scale based on wall heat flux, similar to the well known frictional velocity scale based on wall shear stress. The unified law of the wall formulations are then derived for the inner layer of Type A and B thermal TBLs using these scales. The formulations are constructed by introducing the general damping and enhancing functions in inner layer, and further, these functions are extended to logarithmic coordinate based on the multiscale characteristics of TBL velocity profiles. The validations of the law are conducted both by the DNS guided near wall integration of Type B TBL governing equations, which uncovered the Re and Pr independency mechanisms of the law. The research advances the current understandings of the conventional TBL theory and develop the complete analytical forms of law of the wall for Type A and B TBLs, including inner layer, transition layer, logarithmic layer and wake layer.

physics.flu-dyn↗

arXiv2016

X-CNN: Cross-modal Convolutional Neural Networks for Sparse Datasets

In this paper we propose cross-modal convolutional neural networks (X-CNNs), a novel biologically inspired type of CNN architectures, treating gradient descent-specialised CNNs as individual units of processing in a larger-scale network topology, while allowing for unconstrained information flow and/or weight sharing between analogous hidden layers of the network---thus generalising the already well-established concept of neural network ensembles (where information typically may flow only between the output layers of the individual networks). The constituent networks are individually designed to learn the output function on their own subset of the input data, after which cross-connections between them are introduced after each pooling operation to periodically allow for information exchange between them. This injection of knowledge into a model (by prior partition of the input data through domain knowledge or unsupervised methods) is expected to yield greatest returns in sparse data environments, which are typically less suitable for training CNNs. For evaluation purposes, we have compared a standard four-layer CNN as well as a sophisticated FitNet4 architecture against their cross-modal variants on the CIFAR-10 and CIFAR-100 datasets with differing percentages of the training data being removed, and find that at lower levels of data availability, the X-CNNs significantly outperform their baselines (typically providing a 2--6% benefit, depending on the dataset size and whether data augmentation is used), while still maintaining an edge on all of the full dataset tests.

stat.ML↗