arXiv subjects

Shinjae Yoo

Publications and source records attributed to Shinjae Yoo.

114 records · Page 7Linked to original sources

arXiv2019

Performance Analysis of Deep Learning Workloads on Leading-edge Systems

This work examines the performance of leading-edge systems designed for machine learning computing, including the NVIDIA DGX-2, Amazon Web Services (AWS) P3, IBM Power System Accelerated Compute Server AC922, and a consumer-grade Exxact TensorEX TS4 GPU server. Representative deep learning workloads from the fields of computer vision and natural language processing are the focus of the analysis. Performance analysis is performed along with a number of important dimensions. Performance of the communication interconnects and large and high-throughput deep learning models are considered. Different potential use models for the systems as standalone and in the cloud also are examined. The effect of various optimization of the deep learning models and system configurations is included in the analysis.

cs.PF↗

arXiv2019

Re-examination of the Role of Latent Variables in Sequence Modeling

With latent variables, stochastic recurrent models have achieved state-of-the-art performance in modeling sound-wave sequence. However, opposite results are also observed in other domains, where standard recurrent networks often outperform stochastic models. To better understand this discrepancy, we re-examine the roles of latent variables in stochastic recurrent models for speech density estimation. Our analysis reveals that under the restriction of fully factorized output distribution in previous evaluations, the stochastic models were implicitly leveraging intra-step correlation but the standard recurrent baselines were prohibited to do so, resulting in an unfair comparison. To correct the unfairness, we remove such restriction in our re-examination, where all the models can explicitly leverage intra-step correlation with an auto-regressive structure. Over a diverse set of sequential data, including human speech, MIDI music, handwriting trajectory and frame-permuted speech, our results show that stochastic recurrent models fail to exhibit any practical advantage despite the claimed theoretical superiority. In contrast, standard recurrent models equipped with an auto-regressive output distribution consistently perform better, significantly advancing the state-of-the-art results on three speech datasets.

cs.LG↗

arXiv2019

Layered SGD: A Decentralized and Synchronous SGD Algorithm for Scalable Deep Neural Network Training

Stochastic Gradient Descent (SGD) is the most popular algorithm for training deep neural networks (DNNs). As larger networks and datasets cause longer training times, training on distributed systems is common and distributed SGD variants, mainly asynchronous and synchronous SGD, are widely used. Asynchronous SGD is communication efficient but suffers from accuracy degradation due to delayed parameter updating. Synchronous SGD becomes communication intensive when the number of nodes increases regardless of its advantage. To address these issues, we introduce Layered SGD (LSGD), a new decentralized synchronous SGD algorithm. LSGD partitions computing resources into subgroups that each contain a communication layer (communicator) and a computation layer (worker). Each subgroup has centralized communication for parameter updates while communication between subgroups is handled by communicators. As a result, communication time is overlapped with I/O latency of workers. The efficiency of the algorithm is tested by training a deep network on the ImageNet classification task.

cs.LG↗

arXiv2019

Detector Tomography on IBM 5-qubit Quantum Computers and Mitigation of Imperfect Measurement

We use quantum detector tomography to characterize the qubit readout in terms of measurement POVMs on IBM Quantum Computers IBM Q 5 Tenerife and IBM Q 5 Yorktown. Our results suggest that the characterized detector model deviates from the ideal projectors by a few percent. Further improvement on this characterization can be made by adopting two- or more-qubit detector models instead of independent single-qubit detectors for all the qubits in one device. An unexpected behavior was seen in the physical qubit labelled as qubit 3 of IBM Q 5 Tenerife, which can be a consequence of detector crosstalk or qubit operations influencing each other and requires further investigation. This peculiar behavior is consistent with characterization from the more sophisticated approach of the gate set tomography. We also discuss how the characterized detectors' POVM, despite deviation from the ideal projectors, can be used to estimate the ideal detection distribution.

quant-ph↗

arXiv2019

Classification of Local Chemical Environments from X-ray Absorption Spectra using Supervised Machine Learning

X-ray absorption spectroscopy is a premier element-specific technique for materials characterization. Specifically, the x-ray absorption near-edge structure (XANES) encodes important information about the local chemical environment of an absorbing atom, including coordination number, symmetry, and oxidation state. Interpreting XANES spectra is a key step towards understanding the structural and electronic properties of materials, and as such, extracting structural and electronic descriptors from XANES spectra is akin to solving a challenging inverse problem. Existing methods rely on empirical fingerprints, which are often qualitative or semiquantitative and not transferable. In this paper, we present a machine learning-based approach, which is capable of classifying the local coordination environments of the absorbing atom from simulated K-edge XANES spectra. The machine learning classifiers can learn important spectral features in a broad energy range without human bias and once trained, can make predictions on the fly. The robustness and fidelity of the machine learning method are demonstrated by an average 86% accuracy across the wide chemical space of oxides in eight 3d transition-metal families. We found that spectral features beyond the preedge region play an important role in the local structure classification problem especially for the late 3d transition-metal elements.

cond-mat.mtrl-sci↗

arXiv2017

Unveiling two types of local order in liquid water using machine learning

Machine learning methods are being explored in many areas of science, with the aim of finding solution to problems that evade traditional scientific approaches due to their complexity. In general, an order parameter capable of identifying two different phases of matter separated by a correspond- ing phase transition is constructed based on symmetry arguments. This parameter measures the degree of order as the phase transition proceeds. However, when the two distinct phases are highly disordered it is not trivial to identify broken symmetries with which to find an order parameter. This poses an excellent problem to be addressed using machine learning procedures. Room tem- perature liquid water is hypothesized to be a supercritical liquid, with fluctuations of two different molecular orders associated to two parent liquid phases, one with high density and another one with low density. The validity of this hypothesis is linked to the existence of an order parameter capable of identifying the two distinct liquid phases and their fluctuations. In this work we show how two different machine learning procedures are capable of recognizing local order in liquid water. We argue that when in order to learn relevant features from this complexity, an initial, physically motivated preparation of the available data is as important as the quality of the data set, and that machine learning can become a successful analysis tool only when coupled to high level physical information.

cond-mat.soft↗