arXiv subjects

Duo Wang

Publications and source records attributed to Duo Wang.

At least 55 records · Page 3Linked to original sources

arXiv2022

Multi-node Acceleration for Large-scale GCNs

Limited by the memory capacity and compute power, singe-node graph convolutional neural network (GCN) accelerators cannot complete the execution of GCNs within a reasonable amount of time, due to the explosive size of graphs nowadays. Thus, large-scale GCNs call for a multi-node acceleration system (MultiAccSys) like TPU-Pod for large-scale neural networks. In this work, we aim to scale up single-node GCN accelerators to accelerate GCNs on large-scale graphs. We first identify the communication pattern and challenges of multi-node acceleration for GCNs on large-scale graphs. We observe that (1) coarse-grained communication patterns exist in the execution of GCNs in MultiAccSys, which introduces massive amount of redundant network transmissions and off-chip memory accesses; (2) overall, the acceleration of GCNs in MultiAccSys is bandwidth-bound and latency-tolerant. Guided by these two observations, we then propose MultiGCN, the first MultiAccSys for large-scale GCNs that trades network latency for network bandwidth. Specifically, by leveraging the network latency tolerance, we first propose a topology-aware multicast mechanism with a one put per multicast message-passing model to reduce transmissions and alleviate network bandwidth requirements. Second, we introduce a scatter-based round execution mechanism which cooperates with the multicast mechanism and reduces redundant off-chip memory accesses. Compared to the baseline MultiAccSys, MultiGCN achieves 4~12x speedup using only 28%~68% energy, while reducing 32% transmissions and 73% off-chip memory accesses on average. It not only achieves 2.5~8x speedup over the state-of-the-art multi-GPU solution, but also scales to large-scale graphs as opposed to single-node GCN accelerators.

cs.AR↗

arXiv2022

Nonlinear variation of bedload thickness with fluid flow rate in laminar shearing flow

The movement of subaqueous sediment in laminar shearing flow is numerically investigated by the coupled lattice Boltzmann and discrete element methods. First, the numerical method is validated by comparing the phase diagram proposed by Ouriemi et al. ({\it J. Fluid Mech}., vol. 636, 2009, pp. 321-336). Second, a detailed study on sediment movement is performed for sediment with varying solid volume fractions, and a nonlinear relationship between the normalised thickness of the mobile layer and the normalised fluid flow rate is observed for a densely-packed sediment. Third, an independent investigation on the effective viscosity and friction coefficient of the sediment under different fluid flow rates is conducted in a shear cell; and substitution of these two critical parameters into a theoretical expression proposed by Aussillous et al. ({\it J. Fluid Mech}., vol. 736, 2013, pp. 594-615) provides consistent predictions of bedload thickness with the simulation results of sediment movement. Therefore, we conclude that the non-Newtonian behaviour of densely-packed sediment leads to the nonlinear relationship between the normalised thickness of the mobile layer and the normalised fluid flow rate.

physics.flu-dyn↗

arXiv2022

Alleviating Datapath Conflicts and Design Centralization in Graph Analytics Acceleration

Previous graph analytics accelerators have achieved great improvement on throughput by alleviating irregular off-chip memory accesses. However, on-chip side datapath conflicts and design centralization have become the critical issues hindering further throughput improvement. In this paper, a general solution, Multiple-stage Decentralized Propagation network (MDP-network), is proposed to address these issues, inspired by the key idea of trading latency for throughput. Besides, a novel High throughput Graph analytics accelerator, HiGraph, is proposed by deploying MDP-network to address each issue in practice. The experiment shows that compared with state-of-the-art accelerator, HiGraph achieves up to 2.2x speedup (1.5x on average) as well as better scalability.

cs.AR↗

arXiv2022

Model Architecture Adaption for Bayesian Neural Networks

Bayesian Neural Networks (BNNs) offer a mathematically grounded framework to quantify the uncertainty of model predictions but come with a prohibitive computation cost for both training and inference. In this work, we show a novel network architecture search (NAS) that optimizes BNNs for both accuracy and uncertainty while having a reduced inference latency. Different from canonical NAS that optimizes solely for in-distribution likelihood, the proposed scheme searches for the uncertainty performance using both in- and out-of-distribution data. Our method is able to search for the correct placement of Bayesian layer(s) in a network. In our experiments, the searched models show comparable uncertainty quantification ability and accuracy compared to the state-of-the-art (deep ensemble). In addition, the searched models use only a fraction of the runtime compared to many popular BNN baselines, reducing the inference runtime cost by $2.98 \times$ and $2.92 \times$ respectively on the CIFAR10 dataset when compared to MCDropout and deep ensemble.

cs.LG↗

arXiv2021

Fine-grained Multi-Modal Self-Supervised Learning

Multi-Modal Self-Supervised Learning from videos has been shown to improve model's performance on various downstream tasks. However, such Self-Supervised pre-training requires large batch sizes and a large amount of computation resources due to the noise present in the uncurated data. This is partly due to the fact that the prevalent training scheme is trained on coarse-grained setting, in which vectors representing the whole video clips or natural language sentences are used for computing similarity. Such scheme makes training noisy as part of the video clips can be totally not correlated with the other-modality input such as text description. In this paper, we propose a fine-grained multi-modal self-supervised training scheme that computes the similarity between embeddings at finer-scale (such as individual feature map embeddings and embeddings of phrases), and uses attention mechanisms to reduce noisy pairs' weighting in the loss function. We show that with the proposed pre-training scheme, we can train smaller models, with smaller batch-size and much less computational resources to achieve downstream tasks performances comparable to State-Of-The-Art, for tasks including action recognition and text-image retrievals.

cs.CV↗

arXiv2021

Multihyperuniform Long-Range Order in Medium-Entropy Alloys

We provide strong numerical evidence for a hidden multihyperuniform long-range order (MHLRO) in SiGeSn medium-entropy alloys (MEAs), in which the normalized infinite-wavelength composition fluctuations for all three atomic species are completely suppressed as in a perfect crystalline state. We show this MHLRO naturally leads to the emergence of short-range order (SRO) recently discovered in MEAs, which results in stable lower-energy states compared to alloy models with random or special quasi-random structures (SQSs) possessing no atomic SROs. The MHLRO MEAs approximately realize the Vegard's law, which offers a rule-of-mixture type predictions of the lattice constants and electronic band gap, and thus can be considered as an ideal mixing state. The MHLRO also directly gives rise to enhanced electronic band gaps and superior thermal transport properties at low temperatures compared to random structures and SQSs, which open up novel potential applications in optoelectronics and thermoelectrics. Our analysis of the SiGeSn system leads to the formulation of general organizing principles applicable in other medium- and high-entropy alloys (HEAs), and a highly efficient computational model for rendering realistic large-scale configurations of MEAs and HEAs.

cond-mat.mtrl-sci↗

arXiv2021

Unraveling complex magnetism in two-dimensional FeS$_2$

Atomically thin two dimensional magnets have given rise to emergent phenomena due to magnetic exchange and spin-orbit coupling showing a great promise for realizing ultrathin device structures. In this paper, we critically examine the magnetic properties of 2D FeS$_2$, which has been recently claimed to exhibit room temperature ferromagnetism in the (111) orientation. Our ab initio study based on collinear density functional theory has revealed the ground state as an antiferromagnetic one with an ordering temperature of around 100K along with a signature of spin-phonon coupling, which may trigger a ferromagnetic coupling via strain. Moreover, our calculations based on spin-spirals indicate the possibility of non-collinear magnetic structures, which is also supported by Monte Carlo simulations based on ab initio magnetic exchange parameters. This opens up an excellent possibility to manipulate magnetic structures by the application of directional strain.

cond-mat.mes-hall↗

arXiv2021

Pairwise Relations Discriminator for Unsupervised Raven's Progressive Matrices

The ability to hypothesise, develop abstract concepts based on concrete observations and apply these hypotheses to justify future actions has been paramount in human development. An existing line of research in outfitting intelligent machines with abstract reasoning capabilities revolves around the Raven's Progressive Matrices (RPM). There have been many breakthroughs in supervised approaches to solving RPM in recent years. However, this process requires external assistance, and thus it cannot be claimed that machines have achieved reasoning ability comparable to humans. Namely, humans can solve RPM problems without supervision or prior experience once the RPM rule that relations can only exist row/column-wise is properly introduced. In this paper, we introduce a pairwise relations discriminator (PRD), a technique to develop unsupervised models with sufficient reasoning abilities to tackle an RPM problem. PRD reframes the RPM problem into a relation comparison task, which we can solve without requiring the labelling of the RPM problem. We can identify the optimal candidate by adapting the application of PRD to the RPM problem. Our approach, the PRD, establishes a new state-of-the-art unsupervised learning benchmark with an accuracy of 55.9% on the I-RAVEN, presenting a significant improvement and a step forward in equipping machines with abstract reasoning.

cs.AI↗

arXiv2021

Top-Related Meta-Learning Method for Few-Shot Object Detection

Many meta-learning methods are proposed for few-shot detection. However, previous most methods have two main problems, poor detection APs, and strong bias because of imbalance and insufficient datasets. Previous works mainly alleviate these issues by additional datasets, multi-relation attention mechanisms and sub-modules. However, they require more cost. In this work, for meta-learning, we find that the main challenges focus on related or irrelevant semantic features between categories. Therefore, based on semantic features, we propose a Top-C classification loss (i.e., TCL-C) for classification task and a category-based grouping mechanism for category-based meta-features obtained by the meta-model. The TCL-C exploits the true-label prediction and the most likely C-1 false classification predictions to improve detection performance on few-shot classes. According to similar appearance (i.e., visual appearance, shape, and limbs etc.) and environment in which objects often appear, the category-based grouping mechanism splits categories into disjoint groups to make similar semantic features more compact between categories within a group and obtain more significant difference between groups, alleviating the strong bias problem and further improving detection APs. The whole training consists of the base model and the fine-tuning phases. According to grouping mechanism, we group the meta-features vectors obtained by meta-model, so that the distribution difference between groups is obvious, and the one within each group is less. Extensive experiments on Pascal VOC dataset demonstrate that ours which combines the TCL-C with category-based grouping significantly outperforms previous state-of-the-art methods for few-shot detection. Compared with previous competitive baseline, ours improves detection APs by almost 4% for few-shot detection.

cs.CV↗

arXiv2021

Room temperature two-dimensional antiferromagnetic Weyl semimetal CrO with giant spin-splitting and spin-momentum locked transport

Giant spin-splitting was recently predicted in collinear antiferromagnetic materials with a specific class of magnetic space group. In this work, we have predicted a two-dimensional (2D) antiferromagnetic Weyl semimetal (WS), CrO with large spin-split band structure, spin-momentum locked transport properties and high Néel temperature. It has two pairs of spin-polarized Weyl points at the Fermi level. By manipulating the position of the Weyl points with strain, four different antiferromagnetic spintronic states can be achieved: WSs with two spin-polarized transport channels (STCs), WSs with single STC, semiconductors with two STCs, and semiconductors with single STC. Based on these properties, a new avenue in spintronics with 2D collinear antiferromagnets is proposed.

cond-mat.mtrl-sci↗

arXiv2020

An artificial intelligence system for predicting the deterioration of COVID-19 patients in the emergency department

During the coronavirus disease 2019 (COVID-19) pandemic, rapid and accurate triage of patients at the emergency department is critical to inform decision-making. We propose a data-driven approach for automatic prediction of deterioration risk using a deep neural network that learns from chest X-ray images and a gradient boosting model that learns from routine clinical variables. Our AI prognosis system, trained using data from 3,661 patients, achieves an area under the receiver operating characteristic curve (AUC) of 0.786 (95% CI: 0.745-0.830) when predicting deterioration within 96 hours. The deep neural network extracts informative areas of chest X-ray images to assist clinicians in interpreting the predictions and performs comparably to two radiologists in a reader study. In order to verify performance in a real clinical setting, we silently deployed a preliminary version of the deep neural network at New York University Langone Health during the first wave of the pandemic, which produced accurate predictions in real-time. In summary, our findings demonstrate the potential of the proposed system for assisting front-line physicians in the triage of COVID-19 patients.

cs.LG↗

arXiv2020

Extrapolatable Relational Reasoning With Comparators in Low-Dimensional Manifolds

While modern deep neural architectures generalise well when test data is sampled from the same distribution as training data, they fail badly for cases when the test data distribution differs from the training distribution even along a few dimensions. This lack of out-of-distribution generalisation is increasingly manifested when the tasks become more abstract and complex, such as in relational reasoning. In this paper we propose a neuroscience-inspired inductive-biased module that can be readily amalgamated with current neural network architectures to improve out-of-distribution (o.o.d) generalisation performance on relational reasoning tasks. This module learns to project high-dimensional object representations to low-dimensional manifolds for more efficient and generalisable relational comparisons. We show that neural nets with this inductive bias achieve considerably better o.o.d generalisation performance for a range of relational reasoning tasks. We finally analyse the proposed inductive bias module to understand the importance of lower dimension projection, and propose an augmentation to the algorithmic alignment theory to better measure algorithmic alignment with generalisation.

cs.LG↗

arXiv2020

Learned Low Precision Graph Neural Networks

Deep Graph Neural Networks (GNNs) show promising performance on a range of graph tasks, yet at present are costly to run and lack many of the optimisations applied to DNNs. We show, for the first time, how to systematically quantise GNNs with minimal or no loss in performance using Network Architecture Search (NAS). We define the possible quantisation search space of GNNs. The proposed novel NAS mechanism, named Low Precision Graph NAS (LPGNAS), constrains both architecture and quantisation choices to be differentiable. LPGNAS learns the optimal architecture coupled with the best quantisation strategy for different components in the GNN automatically using back-propagation in a single search round. On eight different datasets, solving the task of classifying unseen nodes in a graph, LPGNAS generates quantised models with significant reductions in both model and buffer sizes but with similar accuracy to manually designed networks and other NAS results. In particular, on the Pubmed dataset, LPGNAS shows a better size-accuracy Pareto frontier compared to seven other manual and searched baselines, offering a 2.3 times reduction in model size but a 0.4% increase in accuracy when compared to the best NAS competitor. Finally, from our collected quantisation statistics on a wide range of datasets, we suggest a W4A8 (4-bit weights, 8-bit activations) quantisation strategy might be the bottleneck for naive GNN quantisations.

cs.LG↗

arXiv2020

High-throughput computational characterization of two-dimensional compositionally complex transition-metal chalcogenide alloys

Two-dimensional (2D) binary transition-metal chalcogenides (TMCs) like molybdenum disulfide exhibits excellent properties as materials for light adsorption devices. Alloying binary TMCs can form 2D compositionally complex TMC alloys (CCTMCAs) that possess remarkable properties from the constituent TMCs. We adopt a high-throughput workflow performing density functional theory (DFT) calculations based on the virtual crystal approximation (VCA) model (VCA-DFT). We test the workflow by predicting properties including in-plane lattice constants, band gaps, effective masses, spin-orbit coupling (SOC), and band alignments of the Mo-W-S-Se, Mo-W-S-Te, and Mo-W-Se-Te 2D CCTMCAs. We validate the VCA-DFT results by computing the same properties using unit cells and supercells of selected compositions. The VCA-DFT results of the abovementioned five properties are comparable to that of DFT calculations, with some inaccuracies in several properties of MoSTe and WSTe. Moreover, 2D CCTMCAs can form type II heterostructures as used in photovoltaics. Finally, we use Mo0.5W0.5SSe, Mo0.5W0.5STe, and Mo0.5W0.5SeTe 2D CCTMCAs to demonstrate the room-temperature entropy-stabilized alloys. They also exhibit high electrical conductivities at 300K, promising for light adsorption devices. Our work shows that the high-throughput workflow using VCA-DFT calculations provides a tradeoff between efficiency and accuracy, opening up opportunities in the computational design of other 2D CCTMCAs for various applications.

cond-mat.mtrl-sci↗

arXiv2020

Electrical and thermal transport properties of medium-entropy SiyGeySnx alloys

Electrical and thermal transport properties of disordered materials have long been of both theoretical interest and engineering importance. As a new class of materials with an intrinsic compositional disorder, high/medium-entropy alloys (HEAs/MEAs) are being immensely studied mainly for their excellent mechanical properties. By contrast, electrical and thermal transport properties of HEAs/MEAs are less well studied. Here we investigate these two properties of silicon (Si)-germanium (Ge)-tin (Sn) MEAs, where we keep the same content of Si and Ge while increasing the content of Sn from 0 to 1/3 to tune the configurational entropy and thus the degree of compositional disorder. We predict all SiyGeySnx MEAs to be semiconductors with a wide range of bandgaps from near-infrared (0.28 eV) to visible (1.11 eV) in the light spectrum. We find that the bandgaps and effective carrier masses decrease with increasing Sn content. As a result, increasing the compositional disorder in SiyGeySnx MEAs enhances their electrical conductivity. For the thermal transport properties of SiyGeySnx MEAs, our molecular dynamics simulations show an opposite trend in the thermal conductivity of these MEAs at room temperature, which decreases with increasing compositional disorder, owing to enhanced Anderson localization and strong phonon-phonon anharmonic interactions. The enhanced electrical conductivity and weakened thermal conductivity make SiyGeySnx MEAs with high Sn content promising functional materials for thermoelectric applications. Our work demonstrates that HEAs/MEAs not only represent a new class of structural alloys but also a novel category of functional alloys with unique electrical and thermal transport properties.

cond-mat.mtrl-sci↗

arXiv2020

Spin qubit based on the nitrogen-vacancy center analog in a diamond-like compound C3BN

The Nitrogen-vacancy (NV) center in diamond plays important roles in emerging quantum technologies. Currently available methods to fabricate the NV center often involve complex processes such as N implantation. By contrast, in a diamond-like compound C3BN, creating a boron (B) vacancy immediately leads to an NV center analog. We use the strongly constrained and appropriately normed (SCAN) semilocal density functional - this functional leads to nearly the same zero-phonon line (ZPL) energy as the experiment and as obtained from the more time-consuming hybrid density functional calculations - to explore the potential of this NV center analog as a novel spin qubit for applications in quantum information processing. We show that the NV center analog in C3BN possesses many similar properties to the NV center in diamond including a wide band gap, weak spin-orbit coupling, an energetically stable negatively charged state, a highly localized spin density, a paramagnetic triplet ground state, and strong hyperfine interactions, which are the properties that make the NV center in diamond stand out as a suitable quantum bit (qubit). We predict the NV center analog in C3BN to exhibit two ZPL energies that correspond to longer wavelengths close to the ideal telecommunication band for quantum communications. C3BN studied here represents only one example of A3XY (A: group IV element; X/Y: group III/V elements) compounds. We expect many other compounds of this family to have similar NV center analogs with a wide range of ZPL energies and functional properties, promising to be new hosts of qubits for quantum technology applications. Furthermore, A3XY compounds often contain group IV elements such as silicon and germanium, so they are compatible with the sophisticated semiconductor processing techniques. Our work opens up ample opportunities towards scalable qubit host materials and novel quantum devices.

cond-mat.mtrl-sci↗

arXiv2020

Abstract Diagrammatic Reasoning with Multiplex Graph Networks

Abstract reasoning, particularly in the visual domain, is a complex human ability, but it remains a challenging problem for artificial neural learning systems. In this work we propose MXGNet, a multilayer graph neural network for multi-panel diagrammatic reasoning tasks. MXGNet combines three powerful concepts, namely, object-level representation, graph neural networks and multiplex graphs, for solving visual reasoning tasks. MXGNet first extracts object-level representations for each element in all panels of the diagrams, and then forms a multi-layer multiplex graph capturing multiple relations between objects across different diagram panels. MXGNet summarises the multiple graphs extracted from the diagrams of the task, and uses this summarisation to pick the most probable answer from the given candidates. We have tested MXGNet on two types of diagrammatic reasoning tasks, namely Diagram Syllogisms and Raven Progressive Matrices (RPM). For an Euler Diagram Syllogism task MXGNet achieves state-of-the-art accuracy of 99.8%. For PGM and RAVEN, two comprehensive datasets for RPM reasoning, MXGNet outperforms the state-of-the-art models by a considerable margin.

cs.LG↗

arXiv2020

A Survey of Model Compression and Acceleration for Deep Neural Networks

Deep neural networks (DNNs) have recently achieved great success in many visual recognition tasks. However, existing deep neural network models are computationally expensive and memory intensive, hindering their deployment in devices with low memory resources or in applications with strict latency requirements. Therefore, a natural thought is to perform model compression and acceleration in deep networks without significantly decreasing the model performance. During the past five years, tremendous progress has been made in this area. In this paper, we review the recent techniques for compacting and accelerating DNN models. In general, these techniques are divided into four categories: parameter pruning and quantization, low-rank factorization, transferred/compact convolutional filters, and knowledge distillation. Methods of parameter pruning and quantization are described first, after that the other techniques are introduced. For each category, we also provide insightful analysis about the performance, related applications, advantages, and drawbacks. Then we go through some very recent successful methods, for example, dynamic capacity networks and stochastic depths networks. After that, we survey the evaluation matrices, the main datasets used for evaluating the model performance, and recent benchmark efforts. Finally, we conclude this paper, discuss remaining the challenges and possible directions for future work.

cs.LG↗