arXiv subjects

Eva Memmel

Publications and source records attributed to Eva Memmel.

3 recordsLinked to original sources

arXiv2025

Automatic Structure Identification for Highly Nonlinear MIMO Volterra Tensor Networks

The Volterra Tensor Network lifts the curse of dimensionality for truncated, discrete times Volterra models, enabling scalable representation of highly nonlinear system. This scalability comes at the cost of introducing randomness through initialization, and leaves open the challenge of how to efficiently determine the hyperparameters model order and memory length. In this paper, we present a unified framework that simultaneously addresses both challenges: We derive two algorithms that incrementally increase the model order and memory length along. Further we proof that the updates are performed along conjugate directions by establishing a mathematical equivalence between our proposed algorithms and equality constrained least squares systems. We present several strategies how to use our proposed algorithms for initialization and hyperparameter selection. In numerical experiments, we demonstrate that our proposed algorithms are more accurate and efficient than the state-of-the-art Volterra Tensor Network and achieve competitive results to several state-of-the-art Volterra models.

math.OC↗

arXiv2023

Projecting basis functions with tensor networks for Gaussian process regression

This paper presents a method for approximate Gaussian process (GP) regression with tensor networks (TNs). A parametric approximation of a GP uses a linear combination of basis functions, where the accuracy of the approximation depends on the total number of basis functions $M$. We develop an approach that allows us to use an exponential amount of basis functions without the corresponding exponential computational complexity. The key idea to enable this is using low-rank TNs. We first find a suitable low-dimensional subspace from the data, described by a low-rank TN. In this low-dimensional subspace, we then infer the weights of our model by solving a Bayesian inference problem. Finally, we project the resulting weights back to the original space to make GP predictions. The benefit of our approach comes from the projection to a smaller subspace: It modifies the shape of the basis functions in a way that it sees fit based on the given data, and it allows for efficient computations in the smaller subspace. In an experiment with an 18-dimensional benchmark data set, we show the applicability of our method to an inverse dynamics problem.

stat.ML↗

arXiv2022

Position: Tensor Networks are a Valuable Asset for Green AI

For the first time, this position paper introduces a fundamental link between tensor networks (TNs) and Green AI, highlighting their synergistic potential to enhance both the inclusivity and sustainability of AI research. We argue that TNs are valuable for Green AI due to their strong mathematical backbone and inherent logarithmic compression potential. We undertake a comprehensive review of the ongoing discussions on Green AI, emphasizing the importance of sustainability and inclusivity in AI research to demonstrate the significance of establishing the link between Green AI and TNs. To support our position, we first provide a comprehensive overview of efficiency metrics proposed in Green AI literature and then evaluate examples of TNs in the fields of kernel machines and deep learning using the proposed efficiency metrics. This position paper aims to incentivize meaningful, constructive discussions by bridging fundamental principles of Green AI and TNs. We advocate for researchers to seriously evaluate the integration of TNs into their research projects, and in alignment with the link established in this paper, we support prior calls encouraging researchers to treat Green AI principles as a research priority.

cs.LG↗