arXiv subjects

Shiqing Wang

Publications and source records attributed to Shiqing Wang.

3 recordsLinked to original sources

arXiv2025

First Demonstration of Second-order Training of Deep Neural Networks with In-memory Analog Matrix Computing

Second-order optimization methods, which leverage curvature information, offer faster and more stable convergence than first-order methods such as stochastic gradient descent (SGD) and Adam. However, their practical adoption is hindered by the prohibitively high cost of inverting the second-order information matrix, particularly in large-scale neural network training. Here, we present the first demonstration of a second-order optimizer powered by in-memory analog matrix computing (AMC) using resistive random-access memory (RRAM), which performs matrix inversion (INV) in a single step. We validate the optimizer by training a two-layer convolutional neural network (CNN) for handwritten letter classification, achieving 26% and 61% fewer training epochs than SGD with momentum and Adam, respectively. On a larger task using the same second-order method, our system delivers a 5.88x improvement in throughput and a 6.9x gain in energy efficiency compared to state-of-the-art digital processors. These results demonstrate the feasibility and effectiveness of AMC circuits for second-order neural network training, opening a new path toward energy-efficient AI acceleration.

cs.ET↗

arXiv2025

GRAMC: General-purpose and reconfigurable analog matrix computing architecture

In-memory analog matrix computing (AMC) with resistive random-access memory (RRAM) represents a highly promising solution that solves matrix problems in one step. However, the existing AMC circuits each have a specific connection topology to implement a single computing function, lack of the universality as a matrix processor. In this work, we design a reconfigurable AMC macro for general-purpose matrix computations, which is achieved by configuring proper connections between memory array and amplifier circuits. Based on this macro, we develop a hybrid system that incorporates an on-chip write-verify scheme and digital functional modules, to deliver a general-purpose AMC solver for various applications.

cs.AR↗

arXiv2024

The maximum storage capacity of open-loop written RRAM is around 4 bits

There have been a plethora of research on multi-level memory devices, where the resistive random-access memory (RRAM) is a prominent example. Although it is easy to write an RRAM device into multiple (even quasi-continuous) states, it suffers from the inherent variations that should limit the storage capacity, especially in the open-loop writing scenario. There have been many experimental results in this regard, however, it lacks a comprehensive analysis of the valid multi-bit storage capability, especially in theoretical terms. The absence of such an insight usually results in misleading conclusions that either exaggerate or underestimate the storage capacity of RRAM devices. Here, by the concept of information theory, we present a model for evaluating the storage capacity of open-loop written RRAM. Based on the experimental results in the literature and the test results of our own devices, we have carefully examined the effects of number of pre-defined levels, conductance variation, and conductance range, on the storage capacity. The analysis leads to a conclusion that the maximum capacity of RRAM devices is around 4 bits.

cs.ET↗