arXiv subjects

Luis Antunes

Publications and source records attributed to Luis Antunes.

9 recordsLinked to original sources

arXiv2024

Machines of Meaning

One goal of Artificial Intelligence is to learn meaningful representations for natural language expressions, but what this entails is not always clear. A variety of new linguistic behaviours present themselves embodied as computers, enhanced humans, and collectives with various kinds of integration and communication. But to measure and understand the behaviours generated by such systems, we must clarify the language we use to talk about them. Computational models are often confused with the phenomena they try to model and shallow metaphors are used as justifications for (or to hype) the success of computational techniques on many tasks related to natural language; thus implying their progress toward human-level machine intelligence without ever clarifying what that means. This paper discusses the challenges in the specification of "machines of meaning", machines capable of acquiring meaningful semantics from natural language in order to achieve their goals. We characterize "meaning" in a computational setting, while highlighting the need for detachment from anthropocentrism in the study of the behaviour of machines of meaning. The pressing need to analyse AI risks and ethics requires a proper measurement of its capabilities which cannot be productively studied and explained while using ambiguous language. We propose a view of "meaning" to facilitate the discourse around approaches such as neural language models and help broaden the research perspectives for technology that facilitates dialogues between humans and machines.

cs.AI↗

arXiv2021

TensorX: Extensible API for Neural Network Model Design and Deployment

TensorX is a Python library for prototyping, design, and deployment of complex neural network models in TensorFlow. A special emphasis is put on ease of use, performance, and API consistency. It aims to make available high-level components like neural network layers that are, in effect, stateful functions, easy to compose and reuse. Its architecture allows for the expression of patterns commonly found when building neural network models either on research or industrial settings. Incorporating ideas from several other deep learning libraries, it makes it easy to use components commonly found in state-of-the-art models. The library design mixes functional dataflow computation graphs with object-oriented neural network building blocks. TensorX combines the dynamic nature of Python with the high-performance GPU-enabled operations of TensorFlow. This library has minimal core dependencies (TensorFlow and NumPy) and is distributed under Apache License 2.0 licence, encouraging its use in both an academic and commercial settings. Full documentation, source code, and binaries can be found in https://tensorx.org/.

cs.LG↗

arXiv2019

Generational political dynamics of retirement pensions systems: An agent based model

The increasing difficulties in financing the welfare state and in particular public retirement pensions have been one of the outcomes both of the decrease of fertility and birth rates combined with the increase of life expectancy. The dynamics of retirement pensions are usually studied in Economics using overlapping generation models. These models are based on simplifying assumptions like the use of a representative agent to ease the problem of tractability. Alternatively, we propose to use agent-based modelling (ABM), relaxing the need for those assumptions and enabling the use of interacting and heterogeneous agents assigning special importance to the study of inter-generational relations. We treat pension dynamics both in economics and political perspectives. The model we build, following the ODD protocol, will try to understand the dynamics of choice of public versus private retirement pensions resulting from the conflicting preferences of different agents but also from the cooperation between them. The aggregation of these individual preferences is done by voting. We combine a microsimulation approach following the evolution of synthetic populations along time, with the ABM approach studying the interactions between the different agent types. Our objective is to depict the conditions for the survival of the public pensions system emerging from the relation between egoistic and altruistic individual and collective behaviours.

econ.GN↗

arXiv2019

A look at the way we look at Complex Networks Robustness and Resilience

This article offers a brief overview of the current research topics concerning strategies to mitigate the adverse effects of perturbations in complex networks. It addresses the issue of an unclear use of Robustness and Resilience terminology and proposes a common interdisciplinary framework for comparing strategies across different fields. It concludes with a high-level discussion of existing challenges and suggestions for future research.

physics.soc-ph↗

arXiv2018

Neural Random Projections for Language Modelling

Neural network-based language models deal with data sparsity problems by mapping the large discrete space of words into a smaller continuous space of real-valued vectors. By learning distributed vector representations for words, each training sample informs the neural network model about a combinatorial number of other patterns. In this paper, we exploit the sparsity in natural language even further by encoding each unique input word using a fixed sparse random representation. These sparse codes are then projected onto a smaller embedding space which allows for the encoding of word occurrences from a possibly unknown vocabulary, along with the creation of more compact language models using a reduced number of parameters. We investigate the properties of our encoding mechanism empirically, by evaluating its performance on the widely used Penn Treebank corpus. We show that guaranteeing approximately equidistant (nearly orthogonal) vector representations for unique discrete inputs is enough to provide the neural network model with enough information to learn --and make use-- of distributed representations for these inputs.

cs.CL↗

arXiv2015

Modelling Structured Societies: a Multi-relational Approach to Context Permeability

The structure of social relations is fundamental for the construction of plausible simulation scenarios. It shapes the way actors interact and create their identity within overlapping social contexts. Each actor interacts in multiple contexts within different types of social relations that constitute their social space. In this article, we present an approach to model structured agent societies with multiple coexisting social networks. We study the notion of context permeability, using a game in which agents try to achieve global consensus. We design and analyse two different models of permeability. In the first model, agents interact concurrently in multiple social networks. In the second, we introduce a context switching mechanism which adds a dynamic temporal component to agent interaction in the model. Agents switch between the different networks spending more or less time in each one. We compare these models and analyse the influence of different social networks regarding the speed of convergence to consensus. We conduct a series of experiments that show the impact of different configurations for coexisting social networks. This approach unveils both the limitations of the current modelling approaches and possible research directions for complex social space simulations.

cs.MA↗

arXiv2014

Continuous On-line Evolution of Agent Behaviours with Cartesian Genetic Programming

Evolutionary Computation has been successfully used to synthesise controllers for embodied agents and multi-agent systems in general. Notwithstanding this, continuous on-line adaptation by the means of evolutionary algorithms is still under-explored, especially outside the evolutionary robotics domain. In this paper, we present an on-line evolutionary programming algorithm that searches in the agent design space for the appropriate behavioural policies to cope with the underlying environment. We discuss the current problems of continuous agent adaptation, present our on-line evolution testbed for evolutionary simulation.

cs.NE↗

arXiv2010

Entropy Measures vs. Algorithmic Information

Algorithmic entropy and Shannon entropy are two conceptually different information measures, as the former is based on size of programs and the later in probability distributions. However, it is known that, for any recursive probability distribution, the expected value of algorithmic entropy equals its Shannon entropy, up to a constant that depends only on the distribution. We study if a similar relationship holds for Rényi and Tsallis entropies of order $α$, showing that it only holds for Rényi and Tsallis entropies of order 1 (i.e., for Shannon entropy). Regarding a time bounded analogue relationship, we show that, for distributions such that the cumulative probability distribution is computable in time $t(n)$, the expected value of time-bounded algorithmic entropy (where the alloted time is $nt(n)\log (nt(n))$) is in the same range as the unbounded version. So, for these distributions, Shannon entropy captures the notion of computationally accessible information. We prove that, for universal time-bounded distribution $\m^t(x)$, Tsallis and Rényi entropies converge if and only if $α$ is greater than 1.

cs.IT↗

arXiv2008

Depth as Randomness Deficiency

Depth of an object concerns a tradeoff between computation time and excess of program length over the shortest program length required to obtain the object. It gives an unconditional lower bound on the computation time from a given program in absence of auxiliary information. Variants known as logical depth and computational depth are expressed in Kolmogorov complexity theory. We derive quantitative relation between logical depth and computational depth and unify the different depth notions by relating them to A. Kolmogorov and L. Levin's fruitful notion of randomness deficiency. Subsequently, we revisit the computational depth of infinite strings, introducing the notion of super deep sequences and relate it with other approaches.

cs.CC↗