arXiv subjects

Olga Krestinskaya

Publications and source records attributed to Olga Krestinskaya.

At least 19 recordsLinked to original sources

arXiv2026

Joint Hardware-Workload Co-Optimization for In-Memory Computing Accelerators

Software-hardware co-design is essential for optimizing in-memory computing (IMC) hardware accelerators for neural networks. However, most existing optimization frameworks target a single workload, leading to highly specialized hardware designs that do not generalize well across models and applications. In contrast, practical deployment scenarios require a single IMC platform that can efficiently support multiple neural network workloads. This work presents a joint hardware-workload co-optimization framework based on an optimized evolutionary algorithm for designing generalized IMC accelerator architectures. By explicitly capturing cross-workload trade-offs rather than optimizing for a single model, the proposed approach significantly reduces the performance gap between workload-specific and generalized IMC designs. The framework is evaluated on both RRAM- and SRAM-based IMC architectures, demonstrating strong robustness and adaptability across diverse design scenarios. Compared to baseline methods, the optimized designs achieve energy-delay-area product (EDAP) reductions of up to 76.2% and 95.5% when optimizing across a small set (4 workloads) and a large set (9 workloads), respectively. The source code of the framework is available at https://github.com/OlgaKrestinskaya/JointHardwareWorkloadOptimizationIMC.

cs.AR↗

arXiv2025

Mixed-Precision Quantization for Language Models: Techniques and Prospects

The rapid scaling of language models (LMs) has resulted in unprecedented computational, memory, and energy requirements, making their training and deployment increasingly unsustainable. Quantization has emerged as an essential compression technique to reduce model size, alleviate memory bottlenecks, and accelerate inference. However, while uniform low-bit quantization (e.g., INT8, INT4) provides significant efficiency gains, it can degrade accuracy in sensitive components of transformer-based LMs. Mixed-precision quantization offers a promising alternative by selectively allocating precision across layers or within tensors to balance efficiency and accuracy. This survey provides a comprehensive overview of Mixed-Precision quantization frameworks for LMs (MXPLMs). We first review quantization fundamentals, including uniform and non-uniform quantizers, quantization granularity, and methods widely used in post-training quantization. We then categorize and compare recent MXPLM frameworks according to their bit allocation strategies and precision configurations across weights, activations, and key-value caches. A comparative analysis highlights differences in perplexity, zero-shot task performance, and deployment trade-offs. Furthermore, we contrast MXPLMs with earlier mixed-precision quantization methods for deep neural networks, identifying strategies that transfer and those that face challenges in the LM setting. Finally, we summarize open issues and future directions, including hardware-aware design, activation quantization, and scalable optimization methods for billion-parameter models. By consolidating recent advances, this work serves as a reference for understanding the current landscape and research prospects of mixed-precision quantization for large-scale language models.

cs.LG↗

arXiv2025

CIMNAS: A Joint Framework for Compute-In-Memory-Aware Neural Architecture Search

To maximize hardware efficiency and performance accuracy in Compute-In-Memory (CIM)-based neural network accelerators for Artificial Intelligence (AI) applications, co-optimizing both software and hardware design parameters is essential. Manual tuning is impractical due to the vast number of parameters and their complex interdependencies. To effectively automate the design and optimization of CIM-based neural network accelerators, hardware-aware neural architecture search (HW-NAS) techniques can be applied. This work introduces CIMNAS, a joint model-quantization-hardware optimization framework for CIM architectures. CIMNAS simultaneously searches across software parameters, quantization policies, and a broad range of hardware parameters, incorporating device-, circuit-, and architecture-level co-optimizations. CIMNAS experiments were conducted over a search space of 9.9x10^85 potential parameter combinations with the MobileNet model as a baseline and RRAM-based CIM architecture. Evaluated on the ImageNet dataset, CIMNAS achieved a reduction in energy-delay-area product (EDAP) ranging from 90.1x to 104.5x, an improvement in TOPS/W between 4.68x and 4.82x, and an enhancement in TOPS/mm^2 from 11.3x to 12.78x relative to various baselines, all while maintaining an accuracy of 73.81%. The adaptability and robustness of CIMNAS are demonstrated by extending the framework to support the SRAM-based ResNet50 architecture, achieving up to an 819.5x reduction in EDAP. Unlike other state-of-the-art methods, CIMNAS achieves EDAP-focused optimization without any accuracy loss, generating diverse software-hardware parameter combinations for high-performance CIM-based neural network designs. The source code of CIMNAS is available at https://github.com/OlgaKrestinskaya/CIMNAS.

cs.AI↗

arXiv2025

Towards Efficient IMC Accelerator Design Through Joint Hardware-Workload Co-optimization

Designing generalized in-memory computing (IMC) hardware that efficiently supports a variety of workloads requires extensive design space exploration, which is infeasible to perform manually. Optimizing hardware individually for each workload or solely for the largest workload often fails to yield the most efficient generalized solutions. To address this, we propose a joint hardware-workload optimization framework that identifies optimised IMC chip architecture parameters, enabling more efficient, workload-flexible hardware. We show that joint optimization achieves 36%, 36%, 20%, and 69% better energy-latency-area scores for VGG16, ResNet18, AlexNet, and MobileNetV3, respectively, compared to the separate architecture parameters search optimizing for a single largest workload. Additionally, we quantify the performance trade-offs and losses of the resulting generalized IMC hardware compared to workload-specific IMC designs.

cs.AR↗

arXiv2023

Towards Efficient In-memory Computing Hardware for Quantized Neural Networks: State-of-the-art, Open Challenges and Perspectives

The amount of data processed in the cloud, the development of Internet-of-Things (IoT) applications, and growing data privacy concerns force the transition from cloud-based to edge-based processing. Limited energy and computational resources on edge push the transition from traditional von Neumann architectures to In-memory Computing (IMC), especially for machine learning and neural network applications. Network compression techniques are applied to implement a neural network on limited hardware resources. Quantization is one of the most efficient network compression techniques allowing to reduce the memory footprint, latency, and energy consumption. This paper provides a comprehensive review of IMC-based Quantized Neural Networks (QNN) and links software-based quantization approaches to IMC hardware implementation. Moreover, open challenges, QNN design requirements, recommendations, and perspectives along with an IMC-based QNN hardware roadmap are provided.

cs.AR↗

arXiv2020

AM-DCGAN: Analog Memristive Hardware Accelerator for Deep Convolutional Generative Adversarial Networks

Generative Adversarial Network (GAN) is a well known computationally complex algorithm requiring signficiant computational resources in software implementations including large amount of data to be trained. This makes its implementation in edge devices with conventional microprocessor hardware a slow and difficult task. In this paper, we propose to accelerate the computationally intensive GAN using memristive neural networks in analog domain. We present a fully analog hardware design of Deep Convolutional GAN (DCGAN) based on CMOS-memristive convolutional and deconvolutional networks simulated using 180nm CMOS technology.

cs.ET↗

arXiv2019

Variation-aware Binarized Memristive Networks

The quantization of weights to binary states in Deep Neural Networks (DNNs) can replace resource-hungry multiply accumulate operations with simple accumulations. Such Binarized Neural Networks (BNNs) exhibit greatly reduced resource and power requirements. In addition, memristors have been shown as promising synaptic weight elements in DNNs. In this paper, we propose and simulate novel Binarized Memristive Convolutional Neural Network (BMCNN) architectures employing hybrid weight and parameter representations. We train the proposed architectures offline and then map the trained parameters to our binarized memristive devices for inference. To take into account the variations in memristive devices, and to study their effect on the performance, we introduce variations in $R_{ON}$ and $R_{OFF}$. Moreover, we introduce means to mitigate the adverse effect of memristive variations in our proposed networks. Finally, we benchmark our BMCNNs and variation-aware BMCNNs using the MNIST dataset.

cs.ET↗

arXiv2019

Instrumentation Amplifier design: Comparison of CMOS-memristive to CMOS design

An instrumentation amplifier (InAmp) is an electronic device used in many applications, where test and measuring accuracy is required. However, one of the drawbacks of an InAmp is limited operation gain range. In this paper, we investigate the possibility of replacing CMOS transistors in InAmp with memristive devices. The application of memristors in CMOS instrumentation amplifier design has lead to reduction of on-chip area and power consumption, comparing to the original design. The memristor based implementation of InAmp design has an improved gain. The advantages of memristor application are shown, and DC and operation gain range are discussed in this paper. Furthermore, variability analysis and performance variation with respect to the temperature variation is provided. In addition, the noise sensitivity analysis is performed. Moreover, varying values of resistance levels of memristors, the operation gain range, gain accuracy as well as a the noise reduction can be improved.

cs.ET↗

arXiv2019

Sense amplifier design using CMOS-memristor circuits

With the increase of the speed of computers, timing and power requirements are becoming crucial for memory devices. The main objective of the paper is to modify 180nm CMOS sense amplifier design by using memristive devices and improve the design in terms of on-chip area, power efficiency, resistance to temperatures and speed. To achieve this, NOT gates in the circuit were constructed using memristor and CMOS. The main aim of the paper is to check the effect of memristors on characteristics of sense amplifier. The design was tested on Conventional Current Sense Amplifier (CSA) circuit. Changes in power, area, sensing delay and offset are reported in the paper.

cs.ET↗

arXiv2019

Perceptron Linear Function Design with CMOS-Memristive Circuits

In the last decade, the interest to emulation of the functionality and structure of the human brain to solve the problems related to image processing and pattern recognition, especially using to Artificial Neural Network (ANN), has increased. Since the capability of ANN to compute at high-speed has been proven to be very useful for various computational problems. One of the simple ANN models is perceptron. Since the perceptron is the basic form of a neural network, the efficient implementation of analog activation functions is required. As various works introduce the design of sigmoid and tangent activation functions, the other activation functions remain an open research problem. This paper describes the design of the perception circuit with the linear activation function using operational amplifier and memristive crossbar. Additionally, the variation of performance with temperature, noise of the circuit is presented.

cs.ET↗

arXiv2019

Variability analysis of Memristor-based Sigmoid Function

Activation functions are widely used in neural networks to decide the activation value of the neural unit based upon linear combinations of the weighted inputs. The effective implementation of activation function is highly important, as they help to represent non-linear complex functional mappings between inputs and outputs of the neural network. One of the non-linear approaches is to use a sigmoid function. Therefore, there is a growing need in enhancing the performance of sigmoid circuits. In this paper, the main objective is to modify existing current mirror based sigmoid model by replacing CMOS transistors with memristor devices. This model was tested varying different circuit parameters, transistor size and temperature. The the area, power and noise in the modified CMOS-memristive sigmoid circuit are shown. The application of memristors in the sigmoid circuit results in higher component density in an on-chip area, allowing a reduction of power and area by 7$\%$. The proposed sigmoid circuit was simulated in SPICE using 180nm TSMC CMOS technology.

cs.ET↗

arXiv2019

Analog multiplier design with CMOS-memristor circuits

CMOS-transistors circuits have been used as a conventional approach for designing an analog multiplier in modern era of industrial electronics. However, previous studies have shown, that based on the working region of transistors, such as saturation or weak inversion regions, the circuit may face issues with output ranges and accuracy. One possible solution to that problem could be choosing CMOS-memristors as a basis for the circuit. Although memristor research is still a growing and promising field, one could argue that its implementation could bring many benefits such as increased circuit density and superior computation speeds, etc. Additionally, the era of Moore's Law of downscaling the size of transistors is to eventually come to an end. No one knows whether the end of a scaling paradigm is to happen within the next five or twenty years. Hence, the research on this particular subject is quite important. This paper proposes an analog multiplier design with CMOS and memristive components. Mainly, the aim of the paper is to compare the power consumption and overall characteristic of the multiplier such as the accuracy and the output range to that of the conventional multiplier. The designed circuit is expected to be suitable for low power applications, and it is built using 18um CMOS technology. The circuit simulations will be conducted using SPICE software. Finally, the effects of channel modulation and temperature on the multiplier performance will be discussed.

cs.ET↗

arXiv2019

Analysis of Multilayer Perceptron with Rectifier Linear Unit Activation Function

The implementation of analog neural network and online analog learning circuits based on memristive crossbar has been intensively explored in recent years. The implementation of various activation functions is important, especially for deep leaning neural networks. There are several implementations of sigmoid and tangent activation function, while the implementation of the neural networks with linear activation functions is an open problem. Therefore, this paper introduces a multilayer perceptron design with linear activation function. The temperature and noise analysis was performed. The perceptron showed a good performance and strong durability to temperature changes.

cs.ET↗

arXiv2019

CMOS-Memristive Analog Multiplier Design

This paper proposes four quadrant analog multiplier using CMOS-memristor circuit. Currently, there are plenty of analog multipliers using resistors and CMOS transistors. They can attain perfect multiplication but have several disadvantages such as lower processing speed, higher power consumption and larger chip areas. Memristor based circuits are introduced to resolve the mentioned drawbacks. In this paper current mode four quadrant multiplier based on squaring circuits is taken as a framework, and CMOS transistors are replaced with memristors. The circuit design is simulated with SPICE, and variability analysis and performance variation with temperature is performed. The proposed circuit allows faster processing with retained data while dissipating less power retaining the multiplication characteristics.

cs.ET↗

arXiv2018

Neuro-memristive Circuits for Edge Computing: A review

The volume, veracity, variability, and velocity of data produced from the ever-increasing network of sensors connected to Internet pose challenges for power management, scalability, and sustainability of cloud computing infrastructure. Increasing the data processing capability of edge computing devices at lower power requirements can reduce several overheads for cloud computing solutions. This paper provides the review of neuromorphic CMOS-memristive architectures that can be integrated into edge computing devices. We discuss why the neuromorphic architectures are useful for edge devices and show the advantages, drawbacks and open problems in the field of neuro-memristive circuits for edge computing.

cs.ET↗

arXiv2018

Wafer Quality Inspection using Memristive LSTM, ANN, DNN and HTM

The automated wafer inspection and quality control is a complex and time-consuming task, which can speed up using neuromorphic memristive architectures, as a separate inspection device or integrating directly into sensors. This paper presents the performance analysis and comparison of different neuromorphic architectures for patterned wafer quality inspection and classification. The application of non-volatile memristive devices in these architectures ensures low power consumption, small on-chip area scalability. We demonstrate that Long-Short Term Memory (LSTM) outperforms other architectures for the same number of training iterations, and has relatively low on-chip area and power consumption.

cs.ET↗

arXiv2018

Programmable Memristive Threshold Logic Gate Array

This paper proposes the implementation of programmable threshold logic gate (TLG) crossbar array based on modified TLG cells for high speed processing and computation. The proposed TLG array operation does not depend on input signal and time pulses, comparing to the existing architectures. The circuit is implemented using TSMC $180nm$ CMOS technology. The on-chip area and power dissipation of the simulated $3\times 4$ TLG array is $1463 μm^2$ and $425 μW$, respectively.

cs.ET↗

arXiv2018

Learning in Memristive Neural Network Architectures using Analog Backpropagation Circuits

The on-chip implementation of learning algorithms would speed-up the training of neural networks in crossbar arrays. The circuit level design and implementation of backpropagation algorithm using gradient descent operation for neural network architectures is an open problem. In this paper, we proposed the analog backpropagation learning circuits for various memristive learning architectures, such as Deep Neural Network (DNN), Binary Neural Network (BNN), Multiple Neural Network (MNN), Hierarchical Temporal Memory (HTM) and Long-Short Term Memory (LSTM). The circuit design and verification is done using TSMC 180nm CMOS process models, and TiO2 based memristor models. The application level validations of the system are done using XOR problem, MNIST character and Yale face image databases

cs.ET↗