arXiv subjects

Wenhao Sun

Publications and source records attributed to Wenhao Sun.

46 records · Page 3Linked to original sources

arXiv2022

SteppingNet: A Stepping Neural Network with Incremental Accuracy Enhancement

Deep neural networks (DNNs) have successfully been applied in many fields in the past decades. However, the increasing number of multiply-and-accumulate (MAC) operations in DNNs prevents their application in resource-constrained and resource-varying platforms, e.g., mobile phones and autonomous vehicles. In such platforms, neural networks need to provide acceptable results quickly and the accuracy of the results should be able to be enhanced dynamically according to the computational resources available in the computing system. To address these challenges, we propose a design framework called SteppingNet. SteppingNet constructs a series of subnets whose accuracy is incrementally enhanced as more MAC operations become available. Therefore, this design allows a trade-off between accuracy and latency. In addition, the larger subnets in SteppingNet are built upon smaller subnets, so that the results of the latter can directly be reused in the former without recomputation. This property allows SteppingNet to decide on-the-fly whether to enhance the inference accuracy by executing further MAC operations. Experimental results demonstrate that SteppingNet provides an effective incremental accuracy improvement and its inference accuracy consistently outperforms the state-of-the-art work under the same limit of computational resources.

cs.LG↗

arXiv2022

Class-based Quantization for Neural Networks

In deep neural networks (DNNs), there are a huge number of weights and multiply-and-accumulate (MAC) operations. Accordingly, it is challenging to apply DNNs on resource-constrained platforms, e.g., mobile phones. Quantization is a method to reduce the size and the computational complexity of DNNs. Existing quantization methods either require hardware overhead to achieve a non-uniform quantization or focus on model-wise and layer-wise uniform quantization, which are not as fine-grained as filter-wise quantization. In this paper, we propose a class-based quantization method to determine the minimum number of quantization bits for each filter or neuron in DNNs individually. In the proposed method, the importance score of each filter or neuron with respect to the number of classes in the dataset is first evaluated. The larger the score is, the more important the filter or neuron is and thus the larger the number of quantization bits should be. Afterwards, a search algorithm is adopted to exploit the different importance of filters and neurons to determine the number of quantization bits of each filter or neuron. Experimental results demonstrate that the proposed method can maintain the inference accuracy with low bit-width quantization. Given the same number of quantization bits, the proposed method can also achieve a better inference accuracy than the existing methods.

cs.LG↗

arXiv2022

Sense: Model Hardware Co-design for Accelerating Sparse CNN on Systolic Array

Sparsity is an intrinsic property of convolutional neural network(CNN) and worth exploiting for CNN accelerators, but extra processing comes with hardware overhead, causing many architectures suffering from only minor profit. Meanwhile, systolic array has been increasingly competitive on CNNs acceleration for its high spatiotemporal locality and low hardware overhead. However, the irregularity of sparsity induces imbalanced workload under the rigid systolic dataflow, causing performance degradation. Thus, this paper proposed a systolicarray-based architecture, called Sense, for sparse CNN acceleration by model-hardware co-design, achieving large performance improvement. To balance input feature map(IFM) and weight loads across Processing Element(PE) array, we applied channel clustering to gather IFMs with approximate sparsity for array computation, and co-designed a load-balancing weight pruning method to keep the sparsity ratio of each kernel at a certain value with little accuracy loss, improving PE utilization and overall performance. Additionally, Adaptive Dataflow Configuration is applied to determine the computing strategy based on the storage ratio of IFMs and weights, lowering 1.17x-1.8x DRAM access compared with Swallow and further reducing system energy consumption. The whole design is implemented on ZynqZCU102 with 200MHz and performs at 471-, 34-, 53- and 191-image/s for AlexNet, VGG-16, ResNet-50 and GoogleNet respectively. Compared against sparse systolic-array-based accelerators, Swallow, FESA and SPOTS, Sense achieves 1x-2.25x, 1.95x-2.5x and 1.17x-2.37x performance improvement on these CNNs respectively with reasonable overhead.

cs.AR↗

arXiv2021

Dataset of Solution-based Inorganic Materials Synthesis Recipes Extracted from the Scientific Literature

The development of a materials synthesis route is usually based on heuristics and experience. A possible new approach would be to apply data-driven approaches to learn the patterns of synthesis from past experience and use them to predict the syntheses of novel materials. However, this route is impeded by the lack of a large-scale database of synthesis formulations. In this work, we applied advanced machine learning and natural language processing techniques to construct a dataset of 35,675 solution-based synthesis "recipes" extracted from the scientific literature. Each recipe contains essential synthesis information including the precursors and target materials, their quantities, and the synthesis actions and corresponding attributes. Every recipe is also augmented with the reaction formula. Through this work, we are making freely available the first large dataset of solution-based inorganic materials synthesis recipes.

cond-mat.mtrl-sci↗

arXiv2021

Visualizing Temperature-Dependent Phase Stability in High Entropy Alloys

High Entropy Alloys (HEAs) contain near equimolar amounts of five or more elements and are a compelling space for materials design. Great emphasis is placed on identifying HEAs that form a homogeneous solid-solution, but the design of such HEAs is hindered by the difficulty of navigating stability relationships in high-component spaces. Traditional phase diagrams use barycentric coordinates to represent composition axes, which require D = (N - 1) spatial dimensions to represent an N-component system, meaning that HEA systems with N > 4 components cannot be readily visualized. Here, we propose forgoing barycentric composition axes in favor of two energy axes: a formation-energy axis and a 'reaction energy' axis. These Inverse Hull Webs offer an information-dense 2D representation that successfully capture complex phase stability relationships in N > 4 component systems. We use our new diagrams to visualize the transition of HEA solid-solutions from high-temperature stability to metastability upon quenching, and identify important thermodynamic features that are correlated with the persistence or decomposition of metastable HEAs.

cond-mat.mtrl-sci↗

arXiv2021

Observing and modeling the sequential pairwise reactions that drive solid-state ceramic synthesis

Solid-state synthesis from powder precursors is the primary processing route to advanced multicomponent ceramic materials. Designing ceramic synthesis routes is usually a laborious, trial-and-error process, as heterogeneous mixtures of powder precursors often evolve through a complicated series of reaction intermediates. Here, we show that phase evolution from multiple precursors can be modeled as a sequence of pairwise interfacial reactions, with thermodynamic driving forces that can be efficiently calculated using ab initio methods. Using the synthesis of the classic high-temperature superconductor YBa$_2$Cu$_3$O$_{6+x}$ (YBCO) as a representative system, we rationalize how replacing the common BaCO$_3$ precursor with BaO$_2$ redirects phase evolution through a kinetically-facile pathway. Our model is validated from in situ X-ray diffraction and in situ microscopy observations, which show rapid YBCO formation from BaO$_2$ in only 30 minutes. By combining thermodynamic modeling with in situ characterization, we introduce a new computable framework to interpret and ultimately design synthesis pathways to complex ceramic materials.

cond-mat.mtrl-sci↗

arXiv2020

Similarity of Precursors in Solid-state Synthesis as Text-Mined from Scientific Literature

Collecting and analyzing the vast amount of information available in the solid-state chemistry literature may accelerate our understanding of materials synthesis. However, one major problem is the difficulty of identifying which materials from a synthesis paragraph are precursors or are target materials. In this study, we developed a two-step Chemical Named Entity Recognition (CNER) model to identify precursors and targets, based on information from the context around material entities. Using the extracted data, we conducted a meta-analysis to study the similarities and differences between precursors in the context of solid-state synthesis. To quantify precursor similarity, we built a substitution model to calculate the viability of substituting one precursor with another while retaining the target. From a hierarchical clustering of the precursors, we demonstrate that "chemical similarity" of precursors can be extracted from text data. Quantifying the similarity of precursors helps provide a foundation for suggesting candidate reactants in a predictive synthesis model.

cond-mat.mtrl-sci↗

arXiv2018

Ternary Nitride Semiconductors in the Rocksalt Crystal Structure

Inorganic nitrides with wurtzite crystal structures are well-known semiconductors used in optoelectronic devices. In contrast, rocksalt-based nitrides are known for their metallic and refractory properties. Breaking this dichotomy, here we report on ternary nitride semiconductors with rocksalt crystal structures, remarkable optoelectronic properties, and the general chemical formula Mg$_{x}$TM$_{1-x}$N (TM=Ti, Zr, Hf, Nb). These compounds form over a broad metal composition range and our experiments show that Mg-rich compositions are nondegenerate semiconductors with visible-range optical absorption onsets (1.8-2.1 eV). Lattice parameters are compatible with growth on a variety of substrates, and epitaxially grown MgZrN$_{2}$ exhibits remarkable electron mobilities approaching 100 cm$^{2}$V$^{-1}$s$^{-1}$. Ab initio calculations reveal that these compounds have disorder-tunable optical properties, large dielectric constants and low carrier effective masses that are insensitive to disorder. Overall, these experimental and theoretical results highlight Mg$_{G-3}$TMN$_{G-2}$ rocksalts as a new class of semiconductor materials with promising properties for optoelectronic applications.

cond-mat.mtrl-sci↗

arXiv2018

A Map of the Inorganic Ternary Metal Nitrides

Exploratory synthesis in novel chemical spaces is the essence of solid-state chemistry. However, uncharted chemical spaces can be difficult to navigate, especially when materials synthesis is challenging. Nitrides represent one such space, where stringent synthesis constraints have limited the exploration of this important class of functional materials. Here, we employ a suite of computational materials discovery and informatics tools to construct a large stability map of the inorganic ternary metal nitrides. Our map clusters the ternary nitrides into chemical families with distinct stability and metastability, and highlights hundreds of promising new ternary nitride spaces for experimental investigation--from which we experimentally realized 7 new Zn- and Mg-based ternary nitrides. By extracting the mixed metallicity, ionicity, and covalency of solid-state bonding from the DFT-computed electron density, we reveal the complex interplay between chemistry, composition, and electronic structure in governing large-scale stability trends in ternary nitride materials.

cond-mat.mtrl-sci↗

arXiv2016

Synthesis of a mixed-valent tin nitride and considerations of its possible crystal structures

Recent advances in theoretical structure prediction methods and high-throughput computational techniques are revolutionizing experimental discovery of the thermodynamically stable inorganic materials. Metastable materials represent a new frontier for studies, since even simple binary non ground state compounds of common elements may be awaiting discovery. However, there are significant research challenges related to non-equilibrium thin film synthesis and crystal structure predictions, such as small strained crystals in the experimental samples and energy minimization based theoretical algorithms. Here we report on experimental synthesis and characterization, as well as theoretical first-principles calculations of a previously unreported mixed-valent binary tin nitride. Thin film experiments indicate that this novel material is N-deficient SnN with tin in the mixed II/IV valence state and a small low-symmetry unit cell. Theoretical calculations suggest that the most likely crystal structure has the space group 2 (SG2) related to the distorted delafossite (SG166), which is nearly 0.1 eV/atom above the ground state SnN polymorph. This observation is rationalized by the structural similarity of the SnN distorted delafossite to the chemically related Sn3N4 spinel compound, which provides a fresh scientific insight into the reasons for growth of polymorphs of the metastable material. In addition to reporting on the discovery of the simple binary SnN compound, this paper illustrates a possible way of combining a wide range of advanced characterization techniques with the first-principle property calculation methods, to elucidate the most likely crystal structure of the previously unreported metastable materials.

cond-mat.mtrl-sci↗