Searcharxiv⌕ Search

arXiv subjects

HamidReza Imani

Publications and source records attributed to HamidReza Imani.

4 recordsLinked to original sources

Efficient Spintronic Stochastic In-Sensor Computing Modules

In-sensor processing (ISP) is used in various fields, including autonomous driving automotive systems, wearable healthcare devices, environmental monitoring, wireless sensor networks, smart cities, and industrial control systems. One of the main features of most of these applications is energy and hardware restriction. This restriction amplifies the importance of using high-performance and low-energy consumption computing systems and methods. This paper proposes efficient spintronic stochastic computing modules for in-sensor computing systems to address this challenge. The proposed design utilized the stochastic behavior of the magnetic tunnel junction in the subcritical switching regime to generate a probability-adjustable random stream. Probability-adjustable random stream generation is the most important step in any stochastic system because it determines the quality of the final output. The simulation results indicate that the proposed design offers almost the same peak signal-to-noise ratio and structural similarity while occupying at least 62\% lower area and consuming 60\% lower energy. These advantages made the proposed design a good choice for ISP applications with limited energy.

cs.ET↗

SEU Hardened Multi-Context Spintronic FPGA

Due to susceptibility to radiation-induced soft errors and resource constraints implementing large and complex designs on field programmable gate arrays (FPGAs) in harsh environments is challenging. Therefore, to mitigate these issues, this paper proposes an innovative approach through the introduction of a single event upset (SEU)-hardened spintronic FPGA architecture that incorporates multi-context capabilities. Leveraging the non-volatile properties of magnetic tunnel junction (MTJ), the proposed design can store two/four distinct configurations internally, eliminating the dependence on external non-volatile memory which significantly boosts the system performance. Additionally, through post-layout simulation, we demonstrate our design immunity to SEU, which enhances its reliability and operational stability in environments susceptible to radiation-induced soft errors. The efficiency of the proposed design was also evaluated using the ROSSETA benchmark, which serves as a representative model for real-world FPGA applications.

cs.ET↗

QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration

The deployment of mixture-of-experts (MoE) large language models (LLMs) presents significant challenges due to their high memory demands. These challenges become even more pronounced in multi-tenant environments, where shared resources must accommodate multiple models, limiting the effectiveness of conventional virtualization techniques. This paper addresses the problem of efficiently serving multiple fine-tuned MoE-LLMs on a single-GPU. We propose a serving system that employs \textit{similarity-based expert consolidation} to reduce the overall memory footprint by sharing similar experts across models. To ensure output quality, we introduce \textit{runtime partial reconfiguration}, dynamically replacing non-expert layers when processing requests from different models. As a result, our approach achieves a competitive output quality while maintaining throughput comparable to serving a single model while incurring a negligible increase in time-to-first-token (TTFT). Experiments on a server with a single NVIDIA A100 GPU (80GB) using Mixtral-8x7B models demonstrate an 85\% average reduction in turnaround time compared to NVIDIA's multi-instance GPU (MIG). Furthermore, experiments on Google's Switch Transformer Base-8 model with up to four variants demonstrate the scalability and resilience of our approach in maintaining output quality compared to other model merging baselines, highlighting its effectiveness.

cs.LG↗

Mixture of Experts with Mixture of Precisions for Tuning Quality of Service

The increasing demand for deploying large Mixture-of-Experts (MoE) models in resource-constrained environments necessitates efficient approaches to address their high memory and computational requirements challenges. Moreover, given that tasks come in different user-defined constraints and the available resources change over time in multi-tenant environments, it is necessary to design an approach which provides a flexible configuration space. This paper presents an adaptive serving approach for the efficient deployment of MoE models, capitalizing on partial quantization of the experts. By dynamically determining the number of quantized experts and their distribution across CPU and GPU, our approach explores the Pareto frontier and offers a fine-grained range of configurations for tuning throughput and model quality. Our evaluation on an NVIDIA A100 GPU using a Mixtral 8x7B MoE model for three language modelling benchmarks demonstrates that the throughput of token generation can be adjusted from 0.63 to 13.00 token per second. This enhancement comes with a marginal perplexity increase of 3.81 to 4.00, 13.59 to 14.17, and 7.24 to 7.40 for WikiText2, PTB, and C4 datasets respectively under maximum quantization. These results highlight the practical applicability of our approach in dynamic and accuracy-sensitive applications where both memory usage and output quality are important.

cs.DC↗