Searcharxiv⌕ Search

arXiv subjects

Emanuele Zappala

Publications and source records attributed to Emanuele Zappala.

At least 37 records · Page 2Linked to original sources
arXiv2024

Deep Neural Networks as the Semi-classical Limit of Topological Quantum Neural Networks: The problem of generalisation

Deep Neural Networks miss a principled model of their operation. A novel framework for supervised learning based on Topological Quantum Field Theory that looks particularly well suited for implementation on quantum processors has been recently explored. We propose using this framework to understand the problem of generalisation in Deep Neural Networks. More specifically, in this approach, Deep Neural Networks are viewed as the semi-classical limit of Topological Quantum Neural Networks. A framework of this kind explains the overfitting behavior of Deep Neural Networks during the training step and the corresponding generalisation capabilities. We explore the paradigmatic case of the perceptron, which we implement as the semiclassical limit of Topological Quantum Neural Networks. We apply a novel algorithm we developed, showing that it obtains similar results to standard neural networks, but without the need for training (optimisation).

quant-ph↗
arXiv2024

CaLMFlow: Volterra Flow Matching using Causal Language Models

We introduce CaLMFlow (Causal Language Models for Flow Matching), a novel framework that casts flow matching as a Volterra integral equation (VIE), leveraging the power of large language models (LLMs) for continuous data generation. CaLMFlow enables the direct application of LLMs to learn complex flows by formulating flow matching as a sequence modeling task, bridging discrete language modeling and continuous generative modeling. Our method implements tokenization across space and time, thereby solving a VIE over these domains. This approach enables efficient handling of high-dimensional data and outperforms ODE solver-dependent methods like conditional flow matching (CFM). We demonstrate CaLMFlow's effectiveness on synthetic and real-world data, including single-cell perturbation response prediction, showcasing its ability to incorporate textual context and generalize to unseen conditions. Our results highlight LLM-driven flow matching as a promising paradigm in generative modeling, offering improved scalability, flexibility, and context-awareness.

cs.LG↗
arXiv2024

Neural Integral Equations

Nonlinear operators with long distance spatiotemporal dependencies are fundamental in modeling complex systems across sciences, yet learning these nonlocal operators remains challenging in machine learning. Integral equations (IEs), which model such nonlocal systems, have wide ranging applications in physics, chemistry, biology, and engineering. We introduce Neural Integral Equations (NIE), a method for learning unknown integral operators from data using an IE solver. To improve scalability and model capacity, we also present Attentional Neural Integral Equations (ANIE), which replaces the integral with self-attention. Both models are grounded in the theory of second kind integral equations, where the indeterminate appears both inside and outside the integral operator. We provide theoretical analysis showing how self-attention can approximate integral operators under mild regularity assumptions, further deepening previously reported connections between transformers and integration, and deriving corresponding approximation results for integral operators. Through numerical benchmarks on synthetic and real world data, including Lotka-Volterra, Navier-Stokes, and Burgers' equations, as well as brain dynamics and integral equations, we showcase the models' capabilities and their ability to derive interpretable dynamics embeddings. Our experiments demonstrate that ANIE outperforms existing methods, especially for longer time intervals and higher dimensional problems. Our work addresses a critical gap in machine learning for nonlocal operators and offers a powerful tool for studying unknown complex systems with long range dependencies.

cs.LG↗
arXiv2024

FIMP: Foundation Model-Informed Message Passing for Graph Neural Networks

Foundation models have achieved remarkable success across many domains, relying on pretraining over vast amounts of data. Graph-structured data often lacks the same scale as unstructured data, making the development of graph foundation models challenging. In this work, we propose Foundation-Informed Message Passing (FIMP), a Graph Neural Network (GNN) message-passing framework that leverages pretrained non-textual foundation models in graph-based tasks. We show that the self-attention layers of foundation models can effectively be repurposed on graphs to perform cross-node attention-based message-passing. Our model is evaluated on a real-world image network dataset and two biological applications (single-cell RNA sequencing data and fMRI brain activity recordings) in both finetuned and zero-shot settings. FIMP outperforms strong baselines, demonstrating that it can effectively leverage state-of-the-art foundation models in graph tasks.

cs.LG↗
arXiv2024

Perturbative Expansion of Yang-Baxter Operators

We study the deformations of a wide class of Yang-Baxter (YB) operators arising from Lie algebras. We relate the higher order deformations of YB operators to Lie algebra deformations. We show that the obstruction to integrating deformations of self-distributive (SD) objects lie in the corresponding Lie algebra third cohomology group, and interpret this result in terms of YB deformations. We show that there are YB operators that admit integrable deformations (i.e. that can be deformed infinitely many times), and that therefore give rise to a full perturbative series in the deformation parameter $\hbar$. We consider the second cohomology group of YB operators corresponding to certain types of Lie algebras, and show that this can be nontrivial even if the Lie algebra is rigid, providing examples of nontrivial YB deformations that do not arise from SD deformations.

math.QA↗
arXiv2023

Yang-Baxter Solutions from Categorical Augmented Racks

An augmented rack is a set with a self-distributive binary operation induced by a group action, and has been extensively used in knot theory. Solutions to the Yang-Baxter equation (YBE) have been also used for knots, since the discovery of the Jones polynomial. In this paper, an interpretation of augmented racks in tensor categories is given for coalgebras that are Hopf algebra modules, and associated solutions to the YBE are constructed. Explicit constructions are given using quantum heaps and the adjoint of Hopf algebras. Furthermore, an inductive construction of Yang-Baxter solutions is given by means of the categorical augmented racks, yielding infinite families of solutions. Constructions of braided monoidal categories are also provided using categorical augmented racks.

math.QA↗
arXiv2023

The exact evaluation of hexagonal spin-networks and topological quantum neural networks

The physical scalar product between spin-networks has been shown to be a fundamental tool in the theory of topological quantum neural networks (TQNN), which are quantum neural networks previously introduced by the authors in the context of quantum machine learning. However, the effective evaluation of the scalar product remains a bottleneck for the applicability of the theory. We introduce an algorithm for the evaluation of the physical scalar product defined by Noui and Perez between spin-network with hexagonal shape. By means of recoupling theory and the properties of the Haar integration we obtain an efficient algorithm, and provide several proofs regarding the main steps. We investigate the behavior of the TQNN evaluations on certain classes of spin-networks with the classical and quantum recoupling. All results can be independently reproduced through the "idea.deploy" framework~\href{https://github.com/lullimat/idea.deploy}{\nolinkurl{https://github.com/lullimat/idea.deploy}}

quant-ph↗
arXiv2023

Operator Learning Meets Numerical Analysis: Improving Neural Networks through Iterative Methods

Deep neural networks, despite their success in numerous applications, often function without established theoretical foundations. In this paper, we bridge this gap by drawing parallels between deep learning and classical numerical analysis. By framing neural networks as operators with fixed points representing desired solutions, we develop a theoretical framework grounded in iterative methods for operator equations. Under defined conditions, we present convergence proofs based on fixed point theory. We demonstrate that popular architectures, such as diffusion models and AlphaFold, inherently employ iterative operator learning. Empirical assessments highlight that performing iterations through network operators improves performance. We also introduce an iterative graph neural network, PIGN, that further demonstrates benefits of iterations. Our work aims to enhance the understanding of deep learning by merging insights from numerical analysis, potentially guiding the design of future networks with clearer theoretical underpinnings and improved performance.

cs.LG↗
arXiv2023

Continuous Spatiotemporal Transformers

Modeling spatiotemporal dynamical systems is a fundamental challenge in machine learning. Transformer models have been very successful in NLP and computer vision where they provide interpretable representations of data. However, a limitation of transformers in modeling continuous dynamical systems is that they are fundamentally discrete time and space models and thus have no guarantees regarding continuous sampling. To address this challenge, we present the Continuous Spatiotemporal Transformer (CST), a new transformer architecture that is designed for the modeling of continuous systems. This new framework guarantees a continuous and smooth output via optimization in Sobolev space. We benchmark CST against traditional transformers as well as other spatiotemporal dynamics modeling methods and achieve superior performance in a number of tasks on synthetic and real systems, including learning brain dynamics from calcium imaging data.

cs.LG↗
arXiv2023

On the representation theory of cyclic and dihedral quandles

Quandle representations are homomorphisms from a quandle to the group of invertible matrices on some vector space taken with the conjugation operation. We study certain families of quandle representations. More specifically, we introduce the notion of regular representation for quandles, investigating in detail the regular representations of dihedral quandles and \emph{completely classifying} them. Then, we study representations of cyclic quandles, giving some necessary conditions for irreducibility and providing a complete classification under some restrictions. Moreover, we provide various counterexamples to constructions that hold for group representations, and show to what extent such theory has the same properties of the representation theory of finite groups. In particular, we show that Maschke's theorem does not hold for quandle representations.

math.RT↗
arXiv2022

Neural Integro-Differential Equations

Modeling continuous dynamical systems from discretely sampled observations is a fundamental problem in data science. Often, such dynamics are the result of non-local processes that present an integral over time. As such, these systems are modeled with Integro-Differential Equations (IDEs); generalizations of differential equations that comprise both an integral and a differential component. For example, brain dynamics are not accurately modeled by differential equations since their behavior is non-Markovian, i.e. dynamics are in part dictated by history. Here, we introduce the Neural IDE (NIDE), a novel deep learning framework based on the theory of IDEs where integral operators are learned using neural networks. We test NIDE on several toy and brain activity datasets and demonstrate that NIDE outperforms other models. These tasks include time extrapolation as well as predicting dynamics from unseen initial conditions, which we test on whole-cortex activity recordings in freely behaving mice. Further, we show that NIDE can decompose dynamics into their Markovian and non-Markovian constituents via the learned integral operator, which we test on fMRI brain activity recordings of people on ketamine. Finally, the integrand of the integral operator provides a latent space that gives insight into the underlying dynamics, which we demonstrate on wide-field brain imaging recordings. Altogether, NIDE is a novel approach that enables modeling of complex non-local dynamics with neural networks.

cs.LG↗
arXiv2022

3-Lie Algebras, Ternary Nambu-Lie algebras and the Yang-Baxter equation

We construct ternary self-distributive (TSD) objects from compositions of binary Lie algebras, $3$-Lie algebras and, in particular, ternary Nambu-Lie algebras. We show that the structures obtained satisfy an invertibility property resembling that of racks. We prove that these structures give rise to Yang-Baxter operators in the tensor product of the base vector space and, upon defining suitable twisting isomorphisms, we obtain representations of the infinite (framed) braid group. We consider examples for low-dimensional Lie algebras, where the ternary bracket is defined by composition of the binary ones, along with simple $3$-Lie algebras. We show that the Yang-Baxter operators obtained are not gauge equivalent to the transposition operator, and we consider the problem of deforming the operators to obtain new solutions to the Yang-Baxter equation. We discuss the applications of this deformation procedure to the construction of (framed) link invariants.

math.GT↗
arXiv2022

Deformations of Yang-Baxter operators via $n$-Lie algebra cohomology

We introduce a cohomology theory of $n$-ary self-distributive objects in the tensor category of vector spaces that classifies their infinitesimal deformations. For $n$-ary self-distributive objects obtained from $n$-Lie algebras we show that ($n$-ary) Lie cohomology naturally injects in the self-distributive cohomology and we prove, under mild additional assumptions, that the map is an isomorphism of second cohomology groups. This shows that the self-distribuitve deformations are completely classified by the deformations of the Lie bracket. This theory has important applications in the study of Yang-Baxter operators as the self-distributive deformations determine nontrivial deformations of the Yang-Baxter operators derived from $n$-ary self-distributive structures. In particular, we show that there is a homomorphism from the second self-distributive cohomology to the second cohomology of the associated Yang-Baxter operator. Moreover, we prove that when the self-distributive structure is induced by a Lie algebra with trivial center, we get a monomorphism. We construct a deformation theory based on simultaneous deformations, where both the coalgebra and self-distributive structures are deformed simultaneously. We show that when the Lie algebra has nontrivial cohomology (e.g. for semi-simple Lie algebras) the simultaneous deformations might still be nontrivial, producing corresponding Yang-Baxter operator deformations. We provide examples and computations in low dimensions, and we completely characterize $2$-cocycles for the self-distributive objects obtained from all the nontrivial real Lie algebras of dimension $3$, i.e. the Bianchi I-IX, and all the nontrivial complex Lie algebras of dimension $3$.

math.GT↗
arXiv2022

On the dynamical emergence of $SU_q(2)$ from the regularization of $2+1D$ gravity with cosmological constant

The quantization of the reduced phase-space of the Einstein-Hilbert action for gravity in $2+1D$ has been shown to bring about the emergence, at the quantum level, of a topological quantum field theory endowed with an $SU_q(2)$ quantum group symmetry structure. We hereby tackle the same problem, but start from the kinematical $SU(2)$ (quantum) Hilbert space of the theory of $2+1D$ gravity with non-zero cosmological constant in the Palatini formalism, and subsequently impose the constraints. We hence show the dynamical emergence of the $SU_q(2)$ quantum group at the quantum level within the spin-foam framework. The regularized curvature constraint is responsible for the effective representations of $SU_q(2)$ that are recovered for any Wilson loop evaluated at the $SU(2)$ group element that encodes the discretization of the space-time curvature induced by the cosmological constant. The extension to the spin-network basis, and consequently to any transition amplitude between its generic states, enables us to derive in full generality the recoupling theory of $SU_q(2)$. We provide constructive examples for the scalar product of two loop states and spin-networks encoding trivalent vertices. We further comment on the diffeomorphism symmetry generated by the implementation of the curvature constraint, and finally derive explicitly the partition function amplitude of the Turaev-Viro model.

gr-qc↗
arXiv2022

Extensions of Augmented Racks and Surface Ribbon Cocycle Invariants

A rack is a set with a binary operation that is right-invertible and self-distributive, properties diagrammatically corresponding to Reidemeister moves II and III, respectively. A rack is said to be an {\it augmented rack} if the operation is written by a group action. Racks and their cohomology theories have been extensively used for knot and knotted surface invariants. Similarly to group cohomology, rack 2-cocycles relate to extensions, and a natural question that arises is to characterize the extensions of augmented racks that are themselves augmented racks. In this paper, we characterize such extensions in terms of what we call {\it fibrant and additive} cohomology of racks. Simultaneous extensions of racks and groups are considered, where the respective $2$-cocycles are related through a certain formula. Furthermore, we construct coloring and cocycle invariants for compact orientable surfaces with boundary in ribbon forms embedded in $3$-space.

math.GT↗
arXiv2022

Fundamental Heap for Framed Links and Ribbon Cocycle Invariants

A heap is a set with a certain ternary operation that is self-distributive (TSD) and exemplified by a group with the operation $(x,y,z)\mapsto xy^{-1}z$. We introduce and investigate framed link invariants using heaps. In analogy with the knot group, we define the fundamental heap of framed links using group presentations. The fundamental heap is determined for some classes of links such as certain families of torus and pretzel links. We show that for these families of links there exist epimorphisms from fundamental heaps to Vinberg and Coxeter groups, implying that corresponding groups are infinite. A relation to the Wirtinger presentation is also described. The cocycle invariant is defined using ternary self-distributive (TSD) cohomology, by means of a state sum that uses ternary heap $2$-cocycles as weights. This invariant corresponds to a rack cocycle invariant for the rack constructed by doubling of a heap, while colorings can be regarded as heap morphisms from the fundamental heap. For the construction of the invariant, first computational methods for the heap cohomology are developed. It is shown that the cohomology splits into two types, called degenerate and nondegenerate, and that the degenerate part is one dimensional. Subcomplexes are constructed based on group cosets, that allow computations of the nondegenerate part. Computations of the cocycle invariants are presented using the cocycles constructed, and conversely, it is proved that the invariant values can be used to derive algebraic properties of the cohomology.

math.GT↗
arXiv2021

Fundamental Heaps for Surface Ribbons and Cocycle Invariants

We introduce the notion of fundamental heap for compact orientable surfaces with boundary embedded in $3$-space, which is an isotopy invariant of the embedding. It is a group, endowed with a ternary heap operation, defined using diagrams of surfaces in a form of thickened trivalent graphs called surface ribbons. We prove that the fundamental heap has a free part whose rank is given by the number of connected components of the surface. We study the behavior of the invariant under boundary connected sum, as well as addition/deletion of twisted bands, and provide formulas relating the number of generators of the fundamental heap to the Euler characteristics. We describe in detail the effect of stabilization on the fundamental heap, and determine that for each given finitely presented group there exists a surface ribbon whose fundamental heap is isomorphic to it, up to extra free factors. A relation between the fundamental heap and the Wirtinger presentation is also described. Moreover, we introduce cocycle invariants for surface ribbons using the notion of mutually distributive cohomology and heap colorings. Explicit computations of fundamental heap and cocycle invariants are presented.

math.GT↗