arXiv subjects

Marco Lippi

Publications and source records attributed to Marco Lippi.

At least 19 recordsLinked to original sources

arXiv2026

Large Language Models, Encoder Architectures and Hybrid Approaches for Patent Classification

Automated patent classification is essential for organizing technological knowledge and constructing indicators of technological change, specialization, and leadership. To identify the relative strengths and weaknesses of popular state- of-the-art approaches to this problem, we perform a controlled comparison of patent-specific encoders and open-weight local LLMs for hierarchical multi-label Cooperative Patent Classification (CPC). We find that the best-performing encoder (task-adapted BERT-for-Patents) outperforms the best-performing LLM (fine-tuned Qwen3.5-9B), while requiring one to two orders of magnitude less energy for inference. The two model families show complementary capabilities, which we leverage through a hybrid pipeline that routes patents with the highest encoder uncertainty to the LLM, yielding significant gains on this subset. We also find that across models, classification errors are especially pronounced in CPC categories that are cross-cutting or semantically broad - such as Section Y - and that they have substantial consequences for technology mapping and country and assignee rankings. The analysis covers predictive and hierarchical performance, computational cost and energy consumption, external validation on EPO patents, and the propagation of classification errors into downstream technological indicators, with implications for automated patent classification procedures used by patent offices, technology analysts, and scientometric researchers.

cs.CE↗

arXiv2025

Testing Transformer Learnability on the Arithmetic Sequence of Rooted Trees

We study whether a transformer network can learn the deterministic sequence of trees generated by the iterated prime factorization of the natural numbers. Each integer is mapped into a rooted planar tree and the resulting sequence $\mathbb{N}\mathscr{T}$ defines an arithmetic text with measurable statistical structure. A transformer network (the GPT-2 architecture) is trained from scratch on the first $10^{11}$ elements and evaluated on Next-Token and masked-word prediction tasks, with a Hidden Markov Model as baseline and a scaling analysis over context window, dataset size, vocabulary size and model size. The model reaches a word accuracy of about $0.4$, well above the baseline, and its performance remains stable on test blocks located at $10^{13}$--$10^{15}$, far beyond the training interval. Moreover, the likelihood assigned by the model separates the arithmetic text from two controls: synthetic sequences reproducing its word frequencies exactly but carrying no sequential organization, with a separation that widens as the evaluated context grows; and sequences containing more than three consecutive square-free integers, a configuration that arithmetic forbids. These results indicate that the transformer captures regularities of the arithmetic text that go beyond its frequency profile.

cs.AI↗

arXiv2025

Towards Reliable Retrieval in RAG Systems for Large Legal Datasets

Retrieval-Augmented Generation (RAG) is a promising approach to mitigate hallucinations in Large Language Models (LLMs) for legal applications, but its reliability is critically dependent on the accuracy of the retrieval step. This is particularly challenging in the legal domain, where large databases of structurally similar documents often cause retrieval systems to fail. In this paper, we address this challenge by first identifying and quantifying a critical failure mode we term Document-Level Retrieval Mismatch (DRM), where the retriever selects information from entirely incorrect source documents. To mitigate DRM, we investigate a simple and computationally efficient technique which we refer to as Summary-Augmented Chunking (SAC). This method enhances each text chunk with a document-level synthetic summary, thereby injecting crucial global context that would otherwise be lost during a standard chunking process. Our experiments on a diverse set of legal information retrieval tasks show that SAC greatly reduces DRM and, consequently, also improves text-level retrieval precision and recall. Interestingly, we find that a generic summarization strategy outperforms an approach that incorporates legal expert domain knowledge to target specific legal elements. Our work provides evidence that this practical, scalable, and easily integrable technique enhances the reliability of RAG systems when applied to large-scale legal document datasets.

cs.CL↗

arXiv2025

LTLZinc: a Benchmarking Framework for Continual Learning and Neuro-Symbolic Temporal Reasoning

Neuro-symbolic artificial intelligence aims to combine neural architectures with symbolic approaches that can represent knowledge in a human-interpretable formalism. Continual learning concerns with agents that expand their knowledge over time, improving their skills while avoiding to forget previously learned concepts. Most of the existing approaches for neuro-symbolic artificial intelligence are applied to static scenarios only, and the challenging setting where reasoning along the temporal dimension is necessary has been seldom explored. In this work we introduce LTLZinc, a benchmarking framework that can be used to generate datasets covering a variety of different problems, against which neuro-symbolic and continual learning methods can be evaluated along the temporal and constraint-driven dimensions. Our framework generates expressive temporal reasoning and continual learning tasks from a linear temporal logic specification over MiniZinc constraints, and arbitrary image classification datasets. Fine-grained annotations allow multiple neural and neuro-symbolic training settings on the same generated datasets. Experiments on six neuro-symbolic sequence classification and four class-continual learning tasks generated by LTLZinc, demonstrate the challenging nature of temporal learning and reasoning, and highlight limitations of current state-of-the-art methods. We release the LTLZinc generator and ten ready-to-use tasks to the neuro-symbolic and continual learning communities, in the hope of fostering research towards unified temporal learning and reasoning frameworks.

cs.AI↗

arXiv2025

A Neuro-Symbolic Framework for Sequence Classification with Relational and Temporal Knowledge

One of the goals of neuro-symbolic artificial intelligence is to exploit background knowledge to improve the performance of learning tasks. However, most of the existing frameworks focus on the simplified scenario where knowledge does not change over time and does not cover the temporal dimension. In this work we consider the much more challenging problem of knowledge-driven sequence classification where different portions of knowledge must be employed at different timesteps, and temporal relations are available. Our experimental evaluation compares multi-stage neuro-symbolic and neural-only architectures, and it is conducted on a newly-introduced benchmarking framework. Results demonstrate the challenging nature of this novel setting, and also highlight under-explored shortcomings of neuro-symbolic methods, representing a precious reference for future research.

cs.AI↗

arXiv2024

The KANDY Benchmark: Incremental Neuro-Symbolic Learning and Reasoning with Kandinsky Patterns

Artificial intelligence is continuously seeking novel challenges and benchmarks to effectively measure performance and to advance the state-of-the-art. In this paper we introduce KANDY, a benchmarking framework that can be used to generate a variety of learning and reasoning tasks inspired by Kandinsky patterns. By creating curricula of binary classification tasks with increasing complexity and with sparse supervisions, KANDY can be used to implement benchmarks for continual and semi-supervised learning, with a specific focus on symbol compositionality. Classification rules are also provided in the ground truth to enable analysis of interpretable solutions. Together with the benchmark generation pipeline, we release two curricula, an easier and a harder one, that we propose as new challenges for the research community. With a thorough experimental evaluation, we show how both state-of-the-art neural models and purely symbolic approaches struggle with solving most of the tasks, thus calling for the application of advanced neuro-symbolic methods trained over time.

cs.AI↗

arXiv2023

No More Trade-Offs. GPT and Fully Informative Privacy Policies

The paper reports the results of an experiment aimed at testing to what extent ChatGPT 3.5 and 4 is able to answer questions regarding privacy policies designed in the new format that we propose. In a world of human-only interpreters, there was a trade-off between comprehensiveness and comprehensibility of privacy policies, leading to the actual policies not containing enough information for users to learn anything meaningful. Having shown that GPT performs relatively well with the new format, we provide experimental evidence supporting our policy suggestion, namely that the law should require fully comprehensive privacy policies, even if this means they become less concise.

cs.CY↗

arXiv2023

Multi-Task Attentive Residual Networks for Argument Mining

We explore the use of residual networks and neural attention for multiple argument mining tasks. We propose a residual architecture that exploits attention, multi-task learning, and makes use of ensemble, without any assumption on document or argument structure. We present an extensive experimental evaluation on five different corpora of user-generated comments, scientific publications, and persuasive essays. Our results show that our approach is a strong competitor against state-of-the-art architectures with a higher computational footprint or corpus-specific design, representing an interesting compromise between generality, performance accuracy and reduced model size.

cs.CL↗

arXiv2023

Band-Pass Filtering with High-Dimensional Time Series

The paper deals with the construction of a synthetic indicator of economic growth, obtained by projecting a quarterly measure of aggregate economic activity, namely gross domestic product (GDP), into the space spanned by a finite number of smooth principal components, representative of the medium-to-long-run component of economic growth of a high-dimensional time series, available at the monthly frequency. The smooth principal components result from applying a cross-sectional filter distilling the low-pass component of growth in real time. The outcome of the projection is a monthly nowcast of the medium-to-long-run component of GDP growth. After discussing the theoretical properties of the indicator, we deal with the assessment of its reliability and predictive validity with reference to a panel of macroeconomic U.S. time series.

econ.EM↗

arXiv2022

High-dimensional dynamic factor models: a selective survey and lines of future research

High-Dimensional Dynamic Factor Models are presented in detail: The main assumptions and their motivation, main results, illustrations by means of elementary examples. In particular, the role of singular ARMA models in the theory and applications of High-Dimensional Dynamic Factor Models is discussed.The emphasis of the paper is on model classes and their structure theory, rather than on estimation in the narrow sense. Our aim is not a comprehensive survey. Rather we try to point out promising lines of research and applications that have not yet been sufficiently developed.

eess.SY↗

arXiv2021

Attention in Natural Language Processing

Attention is an increasingly popular mechanism used in a wide range of neural architectures. The mechanism itself has been realized in a variety of formats. However, because of the fast-paced advances in this domain, a systematic overview of attention is still missing. In this article, we define a unified model for attention architectures in natural language processing, with a focus on those designed to work with vector representations of the textual data. We propose a taxonomy of attention models according to four dimensions: the representation of the input, the compatibility function, the distribution function, and the multiplicity of the input and/or output. We present the examples of how prior information can be exploited in attention models and discuss ongoing research efforts and open challenges in the area, providing the first extensive categorization of the vast body of literature in this exciting domain.

cs.CL↗

arXiv2021

Individual and Collective Autonomous Development

The increasing complexity and unpredictability of many ICT scenarios let us envision that future systems will have to dynamically learn how to act and adapt to face evolving situations with little or no a priori knowledge, both at the level of individual components and at the collective level. In other words, such systems should become able to autonomously develop models of themselves and of their environment. Autonomous development includes: learning models of own capabilities; learning how to act purposefully towards the achievement of specific goals; and learning how to act collectively, i.e., accounting for the presence of others. In this paper, we introduce the vision of autonomous development in ICT systems, by framing its key concepts and by illustrating suitable application domains. Then, we overview the many research areas that are contributing or can potentially contribute to the realization of the vision, and identify some key research challenges.

cs.AI↗

arXiv2021

Tree-Constrained Graph Neural Networks For Argument Mining

We propose a novel architecture for Graph Neural Networks that is inspired by the idea behind Tree Kernels of measuring similarity between trees by taking into account their common substructures, named fragments. By imposing a series of regularization constraints to the learning problem, we exploit a pooling mechanism that incorporates such notion of fragments within the node soft assignment function that produces the embeddings. We present an extensive experimental evaluation on a collection of sentence classification tasks conducted on several argument mining corpora, showing that the proposed approach performs well with respect to state-of-the-art techniques.

cs.CL↗

arXiv2021

Combining Transformers with Natural Language Explanations

Many NLP applications require models to be interpretable. However, many successful neural architectures, including transformers, still lack effective interpretation methods. A possible solution could rely on building explanations from domain knowledge, which is often available as plain, natural language text. We thus propose an extension to transformer models that makes use of external memories to store natural language explanations and use them to explain classification outputs. We conduct an experimental evaluation on two domains, legal text analysis and argument mining, to show that our approach can produce relevant explanations while retaining or even improving classification performance.

cs.CL↗

arXiv2020

Memory networks for consumer protection:unfairness exposed

Recent work has demonstrated how data-driven AI methods can leverage consumer protection by supporting the automated analysis of legal documents. However, a shortcoming of data-driven approaches is poor explainability. We posit that in this domain useful explanations of classifier outcomes can be provided by resorting to legal rationales. We thus consider several configurations of memory-augmented neural networks where rationales are given a special role in the modeling of context knowledge. Our results show that rationales not only contribute to improve the classification accuracy, but are also able to offer meaningful, natural language explanations of otherwise opaque classifier outcomes.

cs.CY↗

arXiv2020

Large-Dimensional Dynamic Factor Models: Estimation of Impulse-Response Functions with $I(1)$ Cointegrated Factors

We study a large-dimensional Dynamic Factor Model where: (i)~the vector of factors $\mathbf F_t$ is $I(1)$ and driven by a number of shocks that is smaller than the dimension of $\mathbf F_t$; and, (ii)~the idiosyncratic components are either $I(1)$ or $I(0)$. Under~(i), the factors $\mathbf F_t$ are cointegrated and can be modeled as a Vector Error Correction Model (VECM). Under (i) and (ii), we provide consistent estimators, as both the cross-sectional size $n$ and the time dimension $T$ go to infinity, for the factors, the loadings, the shocks, the coefficients of the VECM and therefore the Impulse-Response Functions (IRF) of the observed variables to the shocks.~Furthermore: possible deterministic linear trends are fully accounted for, and the case of an unrestricted VAR in the levels $\mathbf F_t$, instead of a VECM, is also studied. The finite-sample properties the proposed estimators are explored by means of a MonteCarlo exercise. Finally, we revisit two distinct and widely studied empirical applications. By correctly modeling the long-run dynamics of the factors, our results partly overturn those obtained by recent literature. Specifically, we find that: (i) oil price shocks have just a temporary effect on US real activity; and, (ii) in response to a positive news shock, the economy first experiences a significant boom, and then a milder recession.

stat.ME↗

arXiv2020

Parallelizing Machine Learning as a Service for the End-User

As ML applications are becoming ever more pervasive, fully-trained systems are made increasingly available to a wide public, allowing end-users to submit queries with their own data, and to efficiently retrieve results. With increasingly sophisticated such services, a new challenge is how to scale up to evergrowing user bases. In this paper, we present a distributed architecture that could be exploited to parallelize a typical ML system pipeline. We propose a case study consisting of a text mining service and discuss how the method can be generalized to many similar applications. We demonstrate the significance of the computational gain boosted by the distributed architecture by way of an extensive experimental evaluation.

cs.DC↗

arXiv2020

Neural-Symbolic Argumentation Mining: an Argument in Favor of Deep Learning and Reasoning

Deep learning is bringing remarkable contributions to the field of argumentation mining, but the existing approaches still need to fill the gap toward performing advanced reasoning tasks. In this position paper, we posit that neural-symbolic and statistical relational learning could play a crucial role in the integration of symbolic and sub-symbolic methods to achieve this goal.

cs.AI↗