arXiv subjects

Xiaoyan Liu

Publications and source records attributed to Xiaoyan Liu.

33 records · Page 2Linked to original sources

arXiv2024

INSPIRIT: Optimizing Heterogeneous Task Scheduling through Adaptive Priority in Task-based Runtime Systems

As modern HPC computing platforms become increasingly heterogeneous, it is challenging for programmers to fully leverage the computation power of massive parallelism offered by such heterogeneity. Consequently, task-based runtime systems have been proposed as an intermediate layer to hide the complex heterogeneity from the application programmers. The core functionality of these systems is to realize efficient task-to-resource mapping in the form of Directed Acyclic Graph (DAG) scheduling. However, existing scheduling schemes face several drawbacks to determine task priorities due to the heavy reliance on domain knowledge or failure to efficiently exploit the interaction of application and hardware characteristics. In this paper, we propose INSPIRIT, an efficient and lightweight scheduling framework with adaptive priority designed for task-based runtime systems. INSPIRIT introduces two novel task attributes \textit{inspiring ability} and \textit{inspiring efficiency} for dictating scheduling, eliminating the need for application domain knowledge. In addition, INSPIRIT jointly considers runtime information such as ready tasks in worker queues to guide task scheduling. This approach exposes more performance opportunities in heterogeneous hardware at runtime while effectively reducing the overhead for adjusting task priorities. Our evaluation results demonstrate that INSPIRIT achieves superior performance compared to cutting edge scheduling schemes on both synthesized and real-world task DAGs.

cs.DC↗

arXiv2023

A general designing approach for polarization-independent photonic on-chip systems

Due to that the polarization states in optical fibers change randomly during transmission, polarization-independent (PID) devices are demanded to receive lights with arbitrary polarization states. Compared with their orthogonal polarization states, the optical profiles of various modes of the same polarization are similar, and their directions of the main electric field are same. Therefore, it's much easier to design PID devices using multi-modes of one polarization state instead of orthogonal polarizations. This paper firstly presents a scalable method to achieve PID devices and systems by transforming the light orthogonal polarization states into one polarization with different modes. Taking thermo-optical switches as an example, the PDL of the fabricated 2 x 2 switch cell was about 0.8 dB at 1300-1360 nm, and its extinction ratio (ER) was larger than 19 dB for both polarizations. A 4 x 4 switch was also demonstrated and its function of dual polarization switching was implemented, whose ER was larger than 11 dB for both polarizations. This method has potential in constructing PID on-chip systems for optical communications, optical interconnections, etc.

physics.optics↗

arXiv2023

Sentence Bag Graph Formulation for Biomedical Distant Supervision Relation Extraction

We introduce a novel graph-based framework for alleviating key challenges in distantly-supervised relation extraction and demonstrate its effectiveness in the challenging and important domain of biomedical data. Specifically, we propose a graph view of sentence bags referring to an entity pair, which enables message-passing based aggregation of information related to the entity pair over the sentence bag. The proposed framework alleviates the common problem of noisy labeling in distantly supervised relation extraction and also effectively incorporates inter-dependencies between sentences within a bag. Extensive experiments on two large-scale biomedical relation datasets and the widely utilized NYT dataset demonstrate that our proposed framework significantly outperforms the state-of-the-art methods for biomedical distant supervision relation extraction while also providing excellent performance for relation extraction in the general text mining domain.

cs.LG↗

arXiv2023

Full Point Encoding for Local Feature Aggregation in 3D Point Clouds

Point cloud processing methods exploit local point features and global context through aggregation which does not explicity model the internal correlations between local and global features. To address this problem, we propose full point encoding which is applicable to convolution and transformer architectures. Specifically, we propose Full Point Convolution (FPConv) and Full Point Transformer (FPTransformer) architectures. The key idea is to adaptively learn the weights from local and global geometric connections, where the connections are established through local and global correlation functions respectively. FPConv and FPTransformer simultaneously model the local and global geometric relationships as well as their internal correlations, demonstrating strong generalization ability and high performance. FPConv is incorporated in classical hierarchical network architectures to achieve local and global shape-aware learning. In FPTransformer, we introduce full point position encoding in self-attention, that hierarchically encodes each point position in the global and local receptive field. We also propose a shape aware downsampling block which takes into account the local shape and the global context. Experimental comparison to existing methods on benchmark datasets show the efficacy of FPConv and FPTransformer for semantic segmentation, object detection, classification, and normal estimation tasks. In particular, we achieve state-of-the-art semantic segmentation results of 76% mIoU on S3DIS 6-fold and 72.2% on S3DIS Area5.

cs.CV↗

arXiv2022

Enhanced Decentralized Federated Learning based on Consensus in Connected Vehicles

Advanced researches on connected vehicles have recently targeted to the integration of vehicle-to-everything (V2X) networks with Machine Learning (ML) tools and distributed decision making. Federated learning (FL) is emerging as a new paradigm to train machine learning (ML) models in distributed systems, including vehicles in V2X networks. Rather than sharing and uploading the training data to the server, the updating of model parameters (e.g., neural networks' weights and biases) is applied by large populations of interconnected vehicles, acting as local learners. Despite these benefits, the limitation of existing approaches is the centralized optimization which relies on a server for aggregation and fusion of local parameters, leading to the drawback of a single point of failure and scaling issues for increasing V2X network size. Meanwhile, in intelligent transport scenarios, data collected from onboard sensors are redundant, which degrades the performance of aggregation. To tackle these problems, we explore a novel idea of decentralized data processing and introduce a federated learning framework for in-network vehicles, C-DFL(Consensus based Decentralized Federated Learning), to tackle federated learning on connected vehicles and improve learning quality. Extensive simulations have been implemented to evaluate the performance of C-DFL, that demonstrates C-DFL outperforms the performance of conventional methods in all cases.

cs.LG↗

arXiv2022

Distantly-Supervised Long-Tailed Relation Extraction Using Constraint Graphs

Label noise and long-tailed distributions are two major challenges in distantly supervised relation extraction. Recent studies have shown great progress on denoising, but paid little attention to the problem of long-tailed relations. In this paper, we introduce a constraint graph to model the dependencies between relation labels. On top of that, we further propose a novel constraint graph-based relation extraction framework(CGRE) to handle the two challenges simultaneously. CGRE employs graph convolution networks to propagate information from data-rich relation nodes to data-poor relation nodes, and thus boosts the representation learning of long-tailed relations. To further improve the noise immunity, a constraint-aware attention module is designed in CGRE to integrate the constraint information. Extensive experimental results indicate that CGRE achieves significant improvements over the previous methods for both denoising and long-tailed relation extraction. The pre-processed datasets and source code are publicly available at https://github.com/tmliang/CGRE.

cs.CL↗

arXiv2022

A Discrete-event-based Simulator for Distributed Deep Learning

New intelligence applications are driving increasing interest in deploying deep neural networks (DNN) in a distributed way. To set up distributed deep learning involves alterations of a great number of the parameter configurations of network/edge devices and DNN models, which are crucial to achieve best performances. Simulations measure scalability of intelligence applications in the early stage, as well as to determine the effects of different configurations, thus highly desired. However, work on simulating the distributed intelligence environment is still in its infancy. The existing simulation frameworks, such as NS-3, etc., cannot extended in a straightforward way to support simulations of distributed learning. In this paper, we propose a novel discrete event simulator, sim4DistrDL, which includes a deep learning module and a network simulation module to facilitate simulation of DNN-based distributed applications. Specifically, we give the design and implementation of the proposed learning simulator and present an illustrative use case.

cs.LG↗

arXiv2021

Accelerating Sparse Approximate Matrix Multiplication on GPUs

Although the matrix multiplication plays a vital role in computational linear algebra, there are few efficient solutions for matrix multiplication of the near-sparse matrices. The Sparse Approximate Matrix Multiply (SpAMM) is one of the algorithms to fill the performance gap neglected by traditional optimizations for dense/sparse matrix multiplication. However, existing SpAMM algorithms fail to exploit the performance potential of GPUs for acceleration. In this paper, we present cuSpAMM, the first parallel SpAMM algorithm optimized for multiple GPUs. Several performance optimizations have been proposed, including algorithm re-design to adapt to the thread parallelism, blocking strategies for memory access optimization, and the acceleration with the tensor core. In addition, we scale cuSpAMM to run on multiple GPUs with an effective load balance scheme. We evaluate cuSpAMM on both synthesized and real-world datasets on multiple GPUs. The experiment results show that cuSpAMM achieves significant performance speedup compared to vendor optimized cuBLAS and cuSPARSE libraries.

cs.PF↗

arXiv2020

Differentially Private Synthetic Data: Applied Evaluations and Enhancements

Machine learning practitioners frequently seek to leverage the most informative available data, without violating the data owner's privacy, when building predictive models. Differentially private data synthesis protects personal details from exposure, and allows for the training of differentially private machine learning models on privately generated datasets. But how can we effectively assess the efficacy of differentially private synthetic data? In this paper, we survey four differentially private generative adversarial networks for data synthesis. We evaluate each of them at scale on five standard tabular datasets, and in two applied industry scenarios. We benchmark with novel metrics from recent literature and other standard machine learning tools. Our results suggest some synthesizers are more applicable for different privacy budgets, and we further demonstrate complicating domain-based tradeoffs in selecting an approach. We offer experimental learning on applied machine learning scenarios with private internal data to researchers and practioners alike. In addition, we propose QUAIL, an ensemble-based modeling approach to generating synthetic data. We examine QUAIL's tradeoffs, and note circumstances in which it outperforms baseline differentially private supervised learning models under the same budget constraint.

cs.LG↗

arXiv2020

The Deep Learning Compiler: A Comprehensive Survey

The difficulty of deploying various deep learning (DL) models on diverse DL hardware has boosted the research and development of DL compilers in the community. Several DL compilers have been proposed from both industry and academia such as Tensorflow XLA and TVM. Similarly, the DL compilers take the DL models described in different DL frameworks as input, and then generate optimized codes for diverse DL hardware as output. However, none of the existing survey has analyzed the unique design architecture of the DL compilers comprehensively. In this paper, we perform a comprehensive survey of existing DL compilers by dissecting the commonly adopted design in details, with emphasis on the DL oriented multi-level IRs, and frontend/backend optimizations. Specifically, we provide a comprehensive comparison among existing DL compilers from various aspects. In addition, we present detailed analysis on the design of multi-level IRs and illustrate the commonly adopted optimization techniques. Finally, several insights are highlighted as the potential research directions of DL compiler. This is the first survey paper focusing on the design architecture of DL compilers, which we hope can pave the road for future research towards DL compiler.

cs.DC↗

arXiv2019

Device and Circuit Interaction Analysis of Stochastic Behaviors in Cross-Point RRAM Arrays

Stochastic behaviors of resistive random access memory (RRAM) play an important role in the design of cross-point memory arrays. A Monte Carlo compact model of oxide RRAM is developed and calibrated with experiments on various device stack configurations. With Monte Carlo SPICE simulations, we show that an increase in array size and interconnect wire resistance will statistically deteriorate write functionality. Write failure probability (WFP) has an exponential dependency on device uniformity and supply voltage (VDD), and the array bias scheme is a key knob. Lowering array VDD leads to higher effective energy consumption (EEC) due to the increase in WFP when the variation statistics are included in the analysis. Random-access simulations indicate that data sparsity statistically benefits write functionality and energy consumption. Finally, we show that a pseudo-sub-array topology with uniformly distributed pre-forming cells in the pristine high resistance state is able to reduce both WFP and EEC, enabling higher net capacity for memory circuits due to improved variation tolerance.

cs.ET↗

arXiv2017

Imaging Ferroelectric Domains via Charge Gradient Microscopy Enhanced by Principal Component Analysis

Local domain structures of ferroelectrics have been studied extensively using various modes of scanning probes at the nanoscale, including piezoresponse force microscopy (PFM) and Kelvin probe force microscopy (KPFM), though none of these techniques measure the polarization directly, and the fast formation kinetics of domains and screening charges cannot be captured by these quasi-static measurements. In this study, we used charge gradient microscopy (CGM) to image ferroelectric domains of lithium niobate based on current measured during fast scanning, and applied principal component analysis (PCA) to enhance the signal-to-noise ratio of noisy raw data. We found that the CGM signal increases linearly with the scan speed while decreases with the temperature under power-law, consistent with proposed imaging mechanisms of scraping and refilling of surface charges within domains, and polarization change across domain wall. We then, based on CGM mappings, estimated the spontaneous polarization and the density of surface charges with order of magnitude agreement with literature data. The study demonstrates that PCA is a powerful method in imaging analysis of scanning probe microscopy (SPM), with which quantitative analysis of noisy raw data becomes possible.

cond-mat.mtrl-sci↗

arXiv2013

Pocket doping method on Improving the Performance of Graphene Nanoribbons Tunneling FET

A tunneling field effect transistor based on armchair graphene nanoribbons is studied using ballistic quantum transport simulation based on 3D real space nonequilibrium Green's function formalism. By introducing a pocket doping region near the source, the performance of GNRs TFET is improved in terms of larger on current and steeper subthreshold swings compared to conventional tunneling FET. It is also found that pocket region introduces two interesting features. By increasing the pocket length the ambipolar can be effectively suppressed. Furthermore, there is negative trans-conductance at negative gate voltage due to resonance.

cond-mat.mes-hall↗

arXiv2013

Improved Performance of Tunneling FET Based on Hetero Graphene Nanoribbons

A heterojunction tunneling field effect transistor based on armchair graphene nanoribbons is proposed and studied using ballistic quantum transport simulation based on 3D real space nonequilibrium Green's function formalism. By using low band gap nanoribbons as the source/drain material, the hetero- structure shows better performance including higher on current, lower off current, and improved steep subthreshold swings compared with homostructure. It is also found the device performance greatly depends on the source/drain dopping density. High doping density leads to fewer density states in the source and degrades the device performance.

cond-mat.mes-hall↗

arXiv2006

AC Conductance of Finite-length Carbon Nanotubes

We propose a nonequilibrium Green's function approach to calculate the ac conductance of various finite-length carbon nanotubes. The simulated ac conductance differs significantly from that of the infinite-length ones. At the low-frequency limit, the profiles of the quantized conductance are still observable in the finite-length carbon nanotubes, but many more peaks appear on the conductance curves. We also show that the conductance of finite-length carbon nanotubes will oscillate as a function of the ac frequency. The dependence of the oscillation on the lengths, helicities and defects of the carbon nanotubes are also investigated. The knowledge we gain from this research will help us make carbon-nanotube-based interconnects or other ac devices in the future

cond-mat.mes-hall↗