arXiv subjects

Aurelio Sulser

Publications and source records attributed to Aurelio Sulser.

4 recordsLinked to original sources

arXiv2025

Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory

Despite their successes, deep learning models struggle with tasks requiring complex reasoning and function composition. We present a theoretical and empirical investigation into the limitations of Structured State Space Models (SSMs) and Transformers in such tasks. We prove that one-layer SSMs cannot efficiently perform function composition over large domains without impractically large state sizes, and even with Chain-of-Thought prompting, they require a number of steps that scale unfavorably with the complexity of the function composition. Also, the language of a finite-precision SSM is within the class of regular languages. Our experiments corroborate these theoretical findings. Evaluating models on tasks including various function composition settings, multi-digit multiplication, dynamic programming, and Einstein's puzzle, we find significant performance degradation even with advanced prompting techniques. Models often resort to shortcuts, leading to compounding errors. These findings highlight fundamental barriers within current deep learning architectures rooted in their computational capacities. We underscore the need for innovative solutions to transcend these constraints and achieve reliable multi-step reasoning and compositional task-solving, which is critical for advancing toward general artificial intelligence.

cs.LG↗

arXiv2024

GLAudio Listens to the Sound of the Graph

We propose GLAudio: Graph Learning on Audio representation of the node features and the connectivity structure. This novel architecture propagates the node features through the graph network according to the discrete wave equation and then employs a sequence learning architecture to learn the target node function from the audio wave signal. This leads to a new paradigm of learning on graph-structured data, in which information propagation and information processing are separated into two distinct steps. We theoretically characterize the expressivity of our model, introducing the notion of the receptive field of a vertex, and investigate our model's susceptibility to over-smoothing and over-squashing both theoretically as well as experimentally on various graph datasets.

cs.LG↗

arXiv2023

Dynablox: Real-time Detection of Diverse Dynamic Objects in Complex Environments

Real-time detection of moving objects is an essential capability for robots acting autonomously in dynamic environments. We thus propose Dynablox, a novel online mapping-based approach for robust moving object detection in complex environments. The central idea of our approach is to incrementally estimate high confidence free-space areas by modeling and accounting for sensing, state estimation, and mapping limitations during online robot operation. The spatio-temporally conservative free space estimate enables robust detection of moving objects without making any assumptions on the appearance of objects or environments. This allows deployment in complex scenes such as multi-storied buildings or staircases, and for diverse moving objects such as people carrying various items, doors swinging or even balls rolling around. We thoroughly evaluate our approach on real-world data sets, achieving 86% IoU at 17 FPS in typical robotic settings. The method outperforms a recent appearance-based classifier and approaches the performance of offline methods. We demonstrate its generality on a novel data set with rare moving objects in complex environments. We make our efficient implementation and the novel data set available as open-source.

cs.RO↗

arXiv2022

Ramsey numbers for multiple copies of sparse graphs

For a graph $H$ and an integer $n$, we let $nH$ denote the disjoint union of $n$ copies of $H$. In 1975, Burr, Erdős, and Spencer initiated the study of Ramsey numbers for $nH$, one of few instances for which Ramsey numbers are now known precisely. They showed that there is a constant $c = c(H)$ such that $r(nH) = (2|H| - α(H))n + c$, provided $n$ is sufficiently large. Subsequently, Burr gave an implicit way of computing $c$ and noted that this long term behaviour occurs when $n$ is triply exponential in $|H|$. Very recently, Bucić and Sudakov revived the problem and established an essentially tight bound on $n$ by showing $r(nH)$ follows this behaviour already when the number of copies is just a single exponential. We provide significantly stronger bounds on $n$ in case $H$ is a sparse graph, most notably of bounded maximum degree. These are relatable to the current state of the art bounds on $r(H)$ and (in a way) tight. Our methods rely on a beautiful classic proof of Graham, Rödl, and Ruciński, with the emphasis on developing an efficient absorbing method for bounded degree graphs.

math.CO↗