arXiv subjects

Michael Lunglmayr

Publications and source records attributed to Michael Lunglmayr.

At least 19 recordsLinked to original sources

arXiv2026

Linearized Bregman Iterations for Sparse Spiking Neural Networks

Spiking Neural Networks (SNNs) offer an energy efficient alternative to conventional Artificial Neural Networks (ANNs) but typically still require a large number of parameters. This work introduces Linearized Bregman Iterations (LBI) as an optimizer for training SNNs, enforcing sparsity through iterative minimization of the Bregman distance and proximal soft thresholding updates. To improve convergence and generalization, we employ the AdaBreg optimizer, a momentum and bias corrected Bregman variant of Adam. Experiments on three established neuromorphic benchmarks, i.e. the Spiking Heidelberg Digits (SHD), the Spiking Speech Commands (SSC), and the Permuted Sequential MNIST (PSMNIST) datasets, show that LBI based optimization reduces the number of active parameters by about 50% while maintaining accuracy comparable to models trained with the Adam optimizer, demonstrating the potential of convex sparsity inducing methods for efficient neuromorphic learning.

eess.SP↗

arXiv2025

Lightweight LIF-only SNN accelerator using differential time encoding

Spiking Neural Networks (SNNs) offer a promising solution to the problem of increasing computational and energy requirements for modern Machine Learning (ML) applications. Due to their unique data representation choice of using spikes and spike trains, they mostly rely on additions and thresholding operations to achieve results approaching state-of-the-art (SOTA) Artificial Neural Networks (ANNs). This advantage is hindered by the fact that their temporal characteristic does not map well to already existing accelerator hardware like GPUs. Therefore, this work will introduce a hardware accelerator architecture capable of computing feedforward LIF-only SNNs, as well as an accompanying encoding method to efficiently encode already existing data into spike trains. Together, this leads to a design capable of >99% accuracy on the MNIST dataset, with ~0.29ms inference times on a Xilinx Ultrascale+ FPGA, as well as ~0.17ms on a custom ASIC using the open-source predictive 7nm ASAP7 PDK. Furthermore, this work will showcase the advantages of the previously presented differential time encoding for spikes, as well as provide proof that merging spikes from different synapses given in differential time encoding can be done efficiently in hardware.

cs.NE↗

arXiv2025

Integrate-and-Fire from a Mathematical and Signal Processing Perspective

Integrate-and-Fire (IF) is an idealized model of the spike-triggering mechanism of a biological neuron. It is used to realize the bio-inspired event-based principle of information processing in neuromorphic computing. We show that IF is closely related to the concept of Send-on-Delta (SOD) as used in threshold-based sampling. It turns out that the IF model can be adjusted in a way that SOD can be understood as differential version of IF. As a result, we gain insight into the underlying metric structure based on the Alexiewicz norm with consequences for clarifying the underlying signal space including bounded integrable signals with superpositions of finitely many Dirac impulses, the identification of a maximum sparsity property, error bounds for signal reconstruction and a characterization in terms of sparse regularization.

eess.SP↗

arXiv2025

Spiking Neural Network Accelerator Architecture for Differential-Time Representation using Learned Encoding

Spiking Neural Networks (SNNs) have garnered attention over recent years due to their increased energy efficiency and advantages in terms of operational complexity compared to traditional Artificial Neural Networks (ANNs). Two important questions when implementing SNNs are how to best encode existing data into spike trains and how to efficiently process these spike trains in hardware. This paper addresses both of these problems by incorporating the encoding into the learning process, thus allowing the network to learn the spike encoding alongside the weights. Furthermore, this paper proposes a hardware architecture based on a recently introduced differential-time representation for spike trains allowing decoupling of spike time and processing time. Together these contributions lead to a feedforward SNN using only Leaky-Integrate and Fire (LIF) neurons that surpasses 99% accuracy on the MNIST dataset while still being implementable on medium-sized FPGAs with inference times of less than 295us.

cs.NE↗

arXiv2024

On the Sampling Sparsity of Neuromorphic Analog-to-Spike Conversion based on Leaky Integrate-and-Fire

In contrast to the traditional principle of periodic sensing neuromorphic engineering pursues a paradigm shift towards bio-inspired event-based sensing, where events are primarily triggered by a change in the perceived stimulus. We show in a rigorous mathematical way that information encoding by means of Threshold-Based Representation based on either Leaky Integrate-and-Fire (LIF) or Send-on-Delta (SOD) is linked to an analog-to-spike conversion that guarantees maximum sparsity while satisfying an approximation condition based on the Alexiewicz norm.

eess.SP↗

arXiv2024

On the Solvability of the {XOR} Problem by Spiking Neural Networks

The linearly inseparable XOR problem and the related problem of representing binary logical gates is revisited from the point of view of temporal encoding and its solvability by spiking neural networks with minimal configurations of leaky integrate-and-fire (LIF) neurons. We use this problem as an example to study the effect of different hyper parameters such as information encoding, the number of hidden units in a fully connected reservoir, the choice of the leaky parameter and the reset mechanism in terms of reset-to-zero and reset-by-subtraction based on different refractory times. The distributions of the weight matrices give insight into the difficulty, respectively the probability, to find a solution. This leads to the observation that zero refractory time together with graded spikes and an adapted reset mechanism, reset-to-mod, makes it possible to realize sparse solutions of a minimal configuration with only two neurons in the hidden layer to resolve all binary logic gate constellations with XOR as a special case.

cs.NE↗

arXiv2024

On Leaky-Integrate-and Fire as Spike-Train-Quantization Operator on Dirac-Superimposed Continuous-Time Signals

Leaky-integrate-and-fire (LIF) is studied as a non-linear operator that maps an integrable signal $f$ to a sequence $η_f$ of discrete events, the spikes. In the case without any Dirac pulses in the input, it makes no difference whether to set the neuron's potential to zero or to subtract the threshold $\vartheta$ immediately after a spike triggering event. However, in the case of superimpose Dirac pulses the situation is different which raises the question of a mathematical justification of each of the proposed reset variants. In the limit case of zero refractory time the standard reset scheme based on threshold subtraction results in a modulo-based reset scheme which allows to characterize LIF as a quantization operator based on a weighted Alexiewicz norm $\|.\|_{A, α}$ with leaky parameter $α$. We prove the quantization formula $\|η_f - f\|_{A, α} < \vartheta$ under the general condition of local integrability, almost everywhere boundedness and locally finitely many superimposed weighted Dirac pulses which provides a much larger signal space and more flexible sparse signal representation than manageable by classical signal processing.

cs.NE↗

arXiv2024

Spiking Neural Networks in the Alexiewicz Topology: A New Perspective on Analysis and Error Bounds

In order to ease the analysis of error propagation in neuromorphic computing and to get a better understanding of spiking neural networks (SNN), we address the problem of mathematical analysis of SNNs as endomorphisms that map spike trains to spike trains. A central question is the adequate structure for a space of spike trains and its implication for the design of error measurements of SNNs including time delay, threshold deviations, and the design of the reinitialization mode of the leaky-integrate-and-fire (LIF) neuron model. First we identify the underlying topology by analyzing the closure of all sub-threshold signals of a LIF model. For zero leakage this approach yields the Alexiewicz topology, which we adopt to LIF neurons with arbitrary positive leakage. As a result LIF can be understood as spike train quantization in the corresponding norm. This way we obtain various error bounds and inequalities such as a quasi isometry relation between incoming and outgoing spike trains. Another result is a Lipschitz-style global upper bound for the error propagation and a related resonance-type phenomenon.

cs.NE↗

arXiv2024

Quantization in Spiking Neural Networks

In spiking neural networks (SNN), at each node, an incoming sequence of weighted Dirac pulses is converted into an output sequence of weighted Dirac pulses by a leaky-integrate-and-fire (LIF) neuron model based on spike aggregation and thresholding. We show that this mapping can be understood as a quantization operator and state a corresponding formula for the quantization error by means of the Alexiewicz norm. This analysis has implications for rethinking re-initialization in the LIF model, leading to the proposal of 'reset-to-mod' as a modulo-based reset variant.

cs.NE↗

arXiv2023

SNN Architecture for Differential Time Encoding Using Decoupled Processing Time

Spiking neural networks (SNNs) have gained attention in recent years due to their ability to handle sparse and event-based data better than regular artificial neural networks (ANNs). Since the structure of SNNs is less suited for typically used accelerators such as GPUs than conventional ANNs, there is a demand for custom hardware accelerators for processing SNNs. In the past, the main focus was on platforms that resemble the structure of multiprocessor systems. In this work, we propose a lightweight neuron layer architecture that allows network structures to be directly mapped onto digital hardware. Our approach is based on differential time coding of spike sequences and the decoupling of processing time and spike timing that allows the SNN to be processed on different hardware platforms. We present synthesis and performance results showing that this architecture can be implemented for networks of more than 1000 neurons with high clock speeds on a State-of-the-Art FPGA. We furthermore show results on the robustness of our approach to quantization. These results demonstrate that high-accuracy inference can be performed with bit widths as low as 4.

eess.SP↗

arXiv2022

Efficient Non-sequential Division for FPGAs

The division operation is important for many areas of data processing. Especially considering today's demand for hardware accelerators for machine learning algorithms, there is a high demand for an efficient calculation of the division function, e.g. for averaging operations or the online calculation of activation functions. For such algorithms, which are often iterative in nature, one would like to have a non-sequential way of calculating the division operation. The work presents such an approach especially tailored to FPGAs as processing platforms. It is based on an efficient way of calculating the reciprocal operation, based on a low complexity approximation combined with a correction function. The described approach allows approximating the division operation (with errors that can be made arbitrarily low), within one clock cycle using only low hardware requirements. These hardware requirements are scaleable depending on the desired precision. We show results obtained by synthesis and hardware simulations demonstrating the low complexity and high clock speeds achievable with the described method compared to other methods described in the literature.

eess.SP↗

arXiv2021

A Fiber Measurement System with Approximate Deconvolution Based on the Analysis of Fault Clusters in Linearized Bregman Iterations

Automatic detection of faults in optical fibers is an active area of research that plays a significant role in the design of reliable and stable optical networks. A fiber measurement system that combines automated data acquisition and processing represents a disruptive impact in the management of optical fiber networks with fast and reliable event detection. It has been shown in the literature that the linearized Bregman iterations (LBI) algorithm and variations can be successfully used for processing and accurately identifying faults in a fiber profile. One of the factors that impact the performance of these algorithms is the degradation of spatial resolution, which is mainly caused by the appearance of fault clusters due to a reduced number of iterations. In this paper, a method is proposed based on an approximate deconvolution approach for increasing the spatial resolution, possible after a thorough analysis of fault clusters that appear in the algorithm's output. The effect of such approximate deconvolution is shown to extend beyond the improvement of spatial resolution, allowing for better performances to be reached at shorter processing times. An efficient hardware architecture that implements the approximate deconvolution, compatible with the hardware structure recently presented for the LBI algorithm, is also proposed and discussed.

eess.SP↗

arXiv2021

Efficient Majority Voting in Digital Hardware

In recent years, machine learning methods became increasingly important for a manifold number of applications. However, they often suffer from high computational requirements impairing their efficient use in real-time systems, even when employing dedicated hardware accelerators. Ensemble learning methods are especially suitable for hardware acceleration since they can be constructed from individual learners of low complexity and thus offer large parallelization potential. For classification, the outputs of these learners are typically combined by majority voting, which often represents the bottleneck of a hardware accelerator for ensemble inference. In this work, we present a novel architecture that allows obtaining a majority decision in a number of clock cycles that is logarithmic in the number of inputs. We show, that for the example application of handwritten digit recognition a random forest processing engine employing this majority decision architecture implemented on an FPGA allows the classification of more than seven million images per second.

eess.SP↗

arXiv2020

Fast approximate reciprocal approximations for iterative algorithms

The reciprocal function, 1/x, is important for many real-time algorithms. It is used in a large variety of algorithms from areas ranging from iterative estimation to machine learning. Many of these algorithms are iterative in nature and require the online computation of the reciprocal. Such an iterative structure often prevents effective use of pipelining for implementation of the reciprocal. For this reason, a reciprocal algorithm requiring only a low amount of clock cycles is desired. Many real-time algorithms, often being of approximate nature, can tolerate the use of only an approximate solution of the reciprocal. For this reason, we present a low complexity non-iterative approximation of the reciprocal function. This approximation can be calculated using only combinatorial logic. We present synthesis results showing that the proposed approach can be implemented with low area requirements at high clock frequencies. We analytically describe the error of the approximation and show that by optimizing a constant value used in the approximation, different variants with different error behaviors can be obtained. We furthermore present performance results of application examples that, when using our proposed method, show only negligible performance degradation compared to when using the exact reciprocal function, demonstrating the versatility of our proposed approach.

eess.SP↗

arXiv2019

FPGA-Embedded Linearized Bregman Iterations Algorithm for Trend Break Detection

Detection of level shifts in a noisy signal, or trend break detection, is a problem that appears in several research fields, from biophysics to optics and economics. Although many algorithms have been developed to deal with such problem, accurate and low-complexity trend break detection is still an active topic of research. The linearized Bregman Iterations have been recently presented as a low-complexity and computationally-efficient algorithm to tackle this problem, with a formidable structure that could benefit immensely from hardware implementation. In this work, a hardware architecture of the Linearized Bregman Iterations algorithm is presented and tested on a Field Programmable Gate Array (FPGA). The hardware is synthesized in different sized FPGAs and the percentage of used hardware as well as the maximum frequency enabled by the design indicate that an approximately 100 gain factor in processing time, with respect to the software implementation, can be achieved. This represents a tremendous advantage in using a dedicated unit for trend break detection applications.

eess.SP↗

arXiv2018

Linearized Bregman Iterations for Automatic Optical Fiber Fault Analysis

Supervision of the physical layer of optical networks is an extremely relevant subject. To detect fiber faults, single-ended solutions such as the Optical Time Domain Reflectometer (OTDR) allow for precise measurements of fault profiles. Combining the OTDR with a signal processing approach for high-dimensional sparse parameter estimation allows for automated and reliable results in reduced time. In this work, a measurement system composed of a Photon-Counting OTDR data acquisition unit and a processing unit based on a Linearized Bregman Iterations algorithm for automatic fault finding is proposed. An in-depth comparative study of the proposed algorithm's fault-finding prowess in the presence of noise is presented. Characteristics such as sensitivity, specificity, processing time, and complexity, are analysed in simulated environments. Real-life measurements that are conducted using the Photon-Counting OTDR subsystem for data acquisition and the Linearized Bregman-based processing unit for automated data analysis demonstrated accurate results. It is concluded that the proposed measurement system is particularly well suited to the task of fault finding. The natural characteristic of the algorithm fosters embedding the solution in digital hardware, allowing for reduced costs and processing time.

eess.SP↗

arXiv2018

High-Accuracy and Fault Tolerant Stochastic Inner Product Design

In this work, we present a novel inner product design for stochastic computing. Stochastic computing is an emerging computing technique, that encodes a number in the probability of observing a one in a random bit stream. This leads to reduced hardware costs and high error tolerance. The proposed inner product design is based on a two-line bipolar encoding format and applies sequential processing of the input in a central accumulation unit. Sequential processing significantly increases the computation accuracy, since it allows for preliminary cancelation of carry bits. Moreover, the central accumulation unit gives a much better scalability compared to conventional adder tree approaches. We show that the proposed inner product design outperforms state-of-the-art designs in terms of hardware costs for high accuracy requirements and fault tolerance.

cs.ET↗

arXiv2018

A stochastic computing architecture for iterative estimation

Stochastic computing (SC) is a promising candidate for fault tolerant computing in digital circuits. We present a novel stochastic computing estimation architecture allowing to solve a large group of estimation problems including least squares estimation as well as sparse estimation. This allows utilizing the high fault tolerance of stochastic computing for implementing estimation algorithms. The presented architecture is based on the recently proposed linearized-Bregman-based Sparse Kaczmarz algorithm. To realize this architecture, we develop a shrink function in stochastic computing and analytically describe its error probability. We compare the stochastic computing architecture to a fixed-point binary implementation and present bit-true simulation results as well as synthesis results demonstrating the feasibility of the proposed architecture for practical implementation.

eess.SP↗