SearcharxivSearch

arXiv subjects

John L. Gustafson

Publications and source records attributed to John L. Gustafson.

5 recordsLinked to original sources

Closing the Gap Between Float and Posit Hardware Efficiency

The b-posit, or bounded posit, is a variation of the posit format designed for high performance computing (HPC) and AI applications. Unlike traditional floating-point formats (floats), posits use variable-length fields for exponent scaling and significand, providing better efficiency for the same bit width. However, this flexibility introduces high worst-case overhead in decode-encode logic, exceeding the cost of handling subnormals for floats. To address this, the b-posit restricts the regime field to a 6-bit limit, reducing variability in regime and fraction sizes. With an exponent size eS of 5 bits, the dynamic range is $2^{-192}$ to $2^{192}$ (about $10^{-58}$ to $10^{58}$) and the quire size is 800 bits, for any precision $n>12$. This constraint improves numerical properties and simplifies hardware implementation by allowing decode-encode operations with basic multiplexers. Our 32-bit b-posit decoder circuits achieve significant improvements: 79 percent less power consumption, 71 percent smaller area, and 60 percent reduced latency compared to standard posit decoders. The 32-bit b-posit encoder shows 68 percent lower power usage, 46 percent less area, and 44 percent shorter delay. The proposed b-posit hardware exhibits superior scalability with increasing bit widths, outperforming standard posit hardware at higher precisions, with even greater advantages at 64-bit. Notably, the b-posit decode-encode hardware matches or exceeds IEEE compliant 32-bit floating-point performance, offering faster and smaller area implementation, with slight increase in worst-case power due to higher speed. The b-posit hardware design provides the clean mathematical behavior and higher accuracy of posits versus IEEE floats without the power, area, or latency costs observed for the Posit Standard (2022). We believe the b-posit should influence future standard revisions.

cs.AR

Performance-Efficiency Trade-off of Low-Precision Numerical Formats in Deep Neural Networks

Deep neural networks (DNNs) have been demonstrated as effective prognostic models across various domains, e.g. natural language processing, computer vision, and genomics. However, modern-day DNNs demand high compute and memory storage for executing any reasonably complex task. To optimize the inference time and alleviate the power consumption of these networks, DNN accelerators with low-precision representations of data and DNN parameters are being actively studied. An interesting research question is in how low-precision networks can be ported to edge-devices with similar performance as high-precision networks. In this work, we employ the fixed-point, floating point, and posit numerical formats at $\leq$8-bit precision within a DNN accelerator, Deep Positron, with exact multiply-and-accumulate (EMAC) units for inference. A unified analysis quantifies the trade-offs between overall network efficiency and performance across five classification tasks. Our results indicate that posits are a natural fit for DNN inference, outperforming at $\leq$8-bit precision, and can be realized with competitive resource requirements relative to those of floating point.

cs.DC

Deep Positron: A Deep Neural Network Using the Posit Number System

The recent surge of interest in Deep Neural Networks (DNNs) has led to increasingly complex networks that tax computational and memory resources. Many DNNs presently use 16-bit or 32-bit floating point operations. Significant performance and power gains can be obtained when DNN accelerators support low-precision numerical formats. Despite considerable research, there is still a knowledge gap on how low-precision operations can be realized for both DNN training and inference. In this work, we propose a DNN architecture, Deep Positron, with posit numerical format operating successfully at $\leq$8 bits for inference. We propose a precision-adaptable FPGA soft core for exact multiply-and-accumulate for uniform comparison across three numerical formats, fixed, floating-point and posit. Preliminary results demonstrate that 8-bit posit has better accuracy than 8-bit fixed or floating-point for three different low-dimensional datasets. Moreover, the accuracy is comparable to 32-bit floating-point on a Xilinx Virtex-7 FPGA device. The trade-offs between DNN performance and hardware resources, i.e. latency, power, and resource utilization, show that posit outperforms in accuracy and latency at 8-bit and below.

cs.DC

Tensors Come of Age: Why the AI Revolution will help HPC

This article discusses how the automation of tensor algorithms, based on A Mathematics of Arrays and Psi Calculus, and a new way to represent numbers, Unum Arithmetic, enables mechanically provable, scalable, portable, and more numerically accurate software.

cs.AI

Asymptotic approximations for symmetric elliptic integrals

Symmetric elliptic integrals, which have been used as replacements for Legendre's integrals in recent integral tables and computer codes, are homogeneous functions of three or four variables. When some of the variables are much larger than the others, asymptotic approximations with error bounds are presented. In most cases they are derived from a uniform approximation to the integrand. As an application the symmetric elliptic integrals of the first, second, and third kinds are proved to be linearly independent with respect to coefficients that are rational functions.

math.CA