arXiv subjects

José Lopes

Publications and source records attributed to José Lopes.

14 recordsLinked to original sources

arXiv2023

Exploring Multi-Modal Representations for Ambiguity Detection & Coreference Resolution in the SIMMC 2.0 Challenge

Anaphoric expressions, such as pronouns and referential descriptions, are situated with respect to the linguistic context of prior turns, as well as, the immediate visual environment. However, a speaker's referential descriptions do not always uniquely identify the referent, leading to ambiguities in need of resolution through subsequent clarificational exchanges. Thus, effective Ambiguity Detection and Coreference Resolution are key to task success in Conversational AI. In this paper, we present models for these two tasks as part of the SIMMC 2.0 Challenge (Kottur et al. 2021). Specifically, we use TOD-BERT and LXMERT based models, compare them to a number of baselines and provide ablation experiments. Our results show that (1) language models are able to exploit correlations in the data to detect ambiguity; and (2) unimodal coreference resolution models can avoid the need for a vision component, through the use of smart object representations.

cs.CL↗

arXiv2022

Going for GOAL: A Resource for Grounded Football Commentaries

Recent video+language datasets cover domains where the interaction is highly structured, such as instructional videos, or where the interaction is scripted, such as TV shows. Both of these properties can lead to spurious cues to be exploited by models rather than learning to ground language. In this paper, we present GrOunded footbAlL commentaries (GOAL), a novel dataset of football (or `soccer') highlights videos with transcribed live commentaries in English. As the course of a game is unpredictable, so are commentaries, which makes them a unique resource to investigate dynamic language grounding. We also provide state-of-the-art baselines for the following tasks: frame reordering, moment retrieval, live commentary retrieval and play-by-play live commentary generation. Results show that SOTA models perform reasonably well in most tasks. We discuss the implications of these results and suggest new tasks for which GOAL can be used. Our codebase is available at: https://gitlab.com/grounded-sport-convai/goal-baselines.

cs.CV↗

arXiv2021

On asymmetric dark matter constraints from the asteroseismology of a subgiant star

The asteroseismic modelling of solar-like stars has proved to be valuable in constraining dark matter. In this work we study for the first time the influence of asymmetric dark matter (ADM) in the evolution of a subgiant star (KIC 8228742) by direct comparison with observational data. Both spectroscopic and seismic data are analysed with a new approach to the stellar calibration method, in which DM properties can also be considered as free inputs. In another phase of this study, a calibrated standard stellar model (without DM) is used as the benchmark for DM models. We find that the latter models consistently outperform the former for $10^{-40} \leq σ_\mathrm{SD} < 10^{-38}$ cm$^2$, hinting that the presence of ADM in stars of this type does not go against observations. Moreover, we show that stellar seismology allows us to suggest exclusion limits that complement the constraints set by direct detection experiments. Different seismic observables are proposed to study DM properties and $ΔΠ_\ell$ is found to be the most reliable, having the potential to build future DM exclusion diagrams. This new methodology can be a powerful tool in the analysis of the data coming from the next generation of asteroseismic missions.

astro-ph.SR↗

arXiv2021

Dark matter capture and annihilation in stars: Impact on the red giant branch tip

Context: While stars have often been used as laboratories to study dark matter (DM), red giant branch (RGB) stars and all the rich phenomenology they encompass have frequently been overlooked by such endeavors. Aims: We study the capture, evaporation, and annihilation of weakly interacting massive particle (WIMP) DM in low-mass RGB stars ($M=0.8-1.4~\mathrm{M_{\odot}}$). Methods: We used a modified stellar evolution code to study the effects of DM self-annihilation on the structure and evolution of low-mass RGB stars. Results: We find that the number of DM particles that accumulate inside low-mass RGB stars is not only constant during this phase of evolution, but also mostly independent of the stellar mass and to some extent stellar metallicity. Moreover, we find that the energy injected into the stellar core due to DM annihilation can promote the conditions necessary for helium burning and thus trigger an early end of the RGB phase. The premature end of the RGB, which is most pronounced for DM particles with $m_χ\simeq 100~\mathrm{GeV}$, is thus achieved at a lower helium core mass, which results in a lower luminosity at the tip of the red giant branch (TRGB). Although in the current WIMP paradigm, these effects are only relevant if the number of DM particles inside the star is extremely large, we find that for light WIMPs ($m_χ\simeq 4~\mathrm{GeV}$), relevant deviations from the standard TRGB luminosity ($\sim 8\%$) can be achieved with conditions that can be realistic in the inner parsec of the Milky Way.

astro-ph.SR↗

arXiv2021

Domain Adaptation in Dialogue Systems using Transfer and Meta-Learning

Current generative-based dialogue systems are data-hungry and fail to adapt to new unseen domains when only a small amount of target data is available. Additionally, in real-world applications, most domains are underrepresented, so there is a need to create a system capable of generalizing to these domains using minimal data. In this paper, we propose a method that adapts to unseen domains by combining both transfer and meta-learning (DATML). DATML improves the previous state-of-the-art dialogue model, DiKTNet, by introducing a different learning technique: meta-learning. We use Reptile, a first-order optimization-based meta-learning algorithm as our improved training method. We evaluated our model on the MultiWOZ dataset and outperformed DiKTNet in both BLEU and Entity F1 scores when the same amount of data is available.

cs.CL↗

arXiv2021

Towards a more rigorous treatment of uncertainties on the velocity distribution of dark matter particles for capture in stars

Dark matter (DM) capture in stars offers a rich phenomenology that makes it possible to probe a wide variety of particle DM scenarios in diverse astrophysical environments. In spite of decades of improvements to refine predictions of capture-related observables and better quantify astrophysical and particle-physics uncertainties, the actual impact of the Galactic phase-space distribution function of DM has been overlooked. In this work, we tackle this problem by making use of self-consistent equilibrium phase-space models based on the Eddington inversion formalism and an extension of this method to a DM halo with some degree of anisotropy in velocity space. We demonstrate that incorrectly accounting for the variation of the DM velocity distribution with position in the Galaxy leads to a systematic error between a factor two and two orders of magnitude, depending in particular on the target star, the DM candidate mass and the type of interaction involved. Moreover, we show that underlying phase-space properties, such as the anisotropy of the velocity tensor, actually play an important part -- previously disregarded -- and can have a sizable impact on predictions of capture rates and subsequent observables. We argue that Eddington-like methods, which self-consistently account for kinematic constraints on the components of the Galaxy, actually provide a reliable next-to-minimal approach to narrow down uncertainties from phase-space modeling on predictions of observables related to DM capture in stars.

astro-ph.CO↗

arXiv2020

The Lab vs The Crowd: An Investigation into Data Quality for Neural Dialogue Models

Challenges around collecting and processing quality data have hampered progress in data-driven dialogue models. Previous approaches are moving away from costly, resource-intensive lab settings, where collection is slow but where the data is deemed of high quality. The advent of crowd-sourcing platforms, such as Amazon Mechanical Turk, has provided researchers with an alternative cost-effective and rapid way to collect data. However, the collection of fluid, natural spoken or textual interaction can be challenging, particularly between two crowd-sourced workers. In this study, we compare the performance of dialogue models for the same interaction task but collected in two different settings: in the lab vs. crowd-sourced. We find that fewer lab dialogues are needed to reach similar accuracy, less than half the amount of lab data as crowd-sourced data. We discuss the advantages and disadvantages of each data collection method.

cs.CL↗

arXiv2020

Natural Language Interaction to Facilitate Mental Models of Remote Robots

Increasingly complex and autonomous robots are being deployed in real-world environments with far-reaching consequences. High-stakes scenarios, such as emergency response or offshore energy platform and nuclear inspections, require robot operators to have clear mental models of what the robots can and can't do. However, operators are often not the original designers of the robots and thus, they do not necessarily have such clear mental models, especially if they are novice users. This lack of mental model clarity can slow adoption and can negatively impact human-machine teaming. We propose that interaction with a conversational assistant, who acts as a mediator, can help the user with understanding the functionality of remote robots and increase transparency through natural language explanations, as well as facilitate the evaluation of operators' mental models.

cs.HC↗

arXiv2020

CRWIZ: A Framework for Crowdsourcing Real-Time Wizard-of-Oz Dialogues

Large corpora of task-based and open-domain conversational dialogues are hugely valuable in the field of data-driven dialogue systems. Crowdsourcing platforms, such as Amazon Mechanical Turk, have been an effective method for collecting such large amounts of data. However, difficulties arise when task-based dialogues require expert domain knowledge or rapid access to domain-relevant information, such as databases for tourism. This will become even more prevalent as dialogue systems become increasingly ambitious, expanding into tasks with high levels of complexity that require collaboration and forward planning, such as in our domain of emergency response. In this paper, we propose CRWIZ: a framework for collecting real-time Wizard of Oz dialogues through crowdsourcing for collaborative, complex tasks. This framework uses semi-guided dialogue to avoid interactions that breach procedures and processes only known to experts, while enabling the capture of a wide variety of interactions. The framework is available at https://github.com/JChiyah/crwiz

cs.HC↗

arXiv2019

Asymmetric Dark Matter Imprint on Low-mass Main-sequence Stars in the Milky Way Nuclear Star Cluster

In this work, we study the impact of asymmetric dark matter (ADM) on low-mass main-sequence stars in the Milky Way's nuclear star cluster, where the dark matter (DM) density is expected to be orders of magnitude above what is found near the Sun (${ρ}_{\mathrm{DM}}\gtrsim {10}^{3}\ \mathrm{GeV}\ {\mathrm{cm}}^{-3}$). Using a modified stellar evolution code and considering a DM particle ($m_χ = 4 \text{ GeV}$) with a spin-dependent interaction cross section close to the limits allowed by direct detection, we found that the interactions of ADM with baryons in the star's core can have two separate effects on the evolution of these stars: a decrease in the hydrogen burning rate, extending the duration of the main-sequence of stars with $M ~ 1M_{\odot}$ by a few Gyr; the suppression of the onset of convection in the core of stars with $M \lesssim 1.5M_{\odot}$ and consequent quench of supply for the nuclear reactions. If we consider $ρ_{\text{DM}} > 10^3 \ \text{GeV cm}^{-3}$ (corresponding to the inner 5 pc of the Milky Way), stars lighter than the Sun will have a main-sequence life span comparable to the current age of the universe. Stars heavier than two solar masses are not sensitive to the DM particles considered here.

astro-ph.SR↗

arXiv2019

Challenges in Collaborative HRI for Remote Robot Teams

Collaboration between human supervisors and remote teams of robots is highly challenging, particularly in high-stakes, distant, hazardous locations, such as off-shore energy platforms. In order for these teams of robots to truly be beneficial, they need to be trusted to operate autonomously, performing tasks such as inspection and emergency response, thus reducing the number of personnel placed in harm's way. As remote robots are generally trusted less than robots in close-proximity, we present a solution to instil trust in the operator through a `mediator robot' that can exhibit social skills, alongside sophisticated visualisation techniques. In this position paper, we present general challenges and then take a closer look at one challenge in particular, discussing an initial study, which investigates the relationship between the level of control the supervisor hands over to the mediator robot and how this affects their trust. We show that the supervisor is more likely to have higher trust overall if their initial experience involves handing over control of the emergency situation to the robotic assistant. We discuss this result, here, as well as other challenges and interaction techniques for human-robot collaboration.

cs.RO↗

arXiv2018

The Spot the Difference corpus: a multi-modal corpus of spontaneous task oriented spoken interactions

This paper describes the Spot the Difference Corpus which contains 54 interactions between pairs of subjects interacting to find differences in two very similar scenes. The setup used, the participants' metadata and details about collection are described. We are releasing this corpus of task-oriented spontaneous dialogues. This release includes rich transcriptions, annotations, audio and video. We believe that this dataset constitutes a valuable resource to study several dimensions of human communication that go from turn-taking to the study of referring expressions. In our preliminary analyses we have looked at task success (how many differences were found out of the total number of differences) and how it evolves over time. In addition we have looked at scene complexity provided by the RGB components' entropy and how it could relate to speech overlaps, interruptions and the expression of uncertainty. We found there is a tendency that more complex scenes have more competitive interruptions.

cs.CL↗

arXiv2017

Assessing User Expertise in Spoken Dialog System Interactions

Identifying the level of expertise of its users is important for a system since it can lead to a better interaction through adaptation techniques. Furthermore, this information can be used in offline processes of root cause analysis. However, not much effort has been put into automatically identifying the level of expertise of an user, especially in dialog-based interactions. In this paper we present an approach based on a specific set of task related features. Based on the distribution of the features among the two classes - Novice and Expert - we used Random Forests as a classification approach. Furthermore, we used a Support Vector Machine classifier, in order to perform a result comparison. By applying these approaches on data from a real system, Let's Go, we obtained preliminary results that we consider positive, given the difficulty of the task and the lack of competing approaches for comparison.

cs.CL↗

arXiv2016

New Limits on Thermally annihilating Dark Matter from Neutrino Telescopes

We used a consistent and robust solar model to obtain upper limits placed by neutrino telescopes, such as Ice- Cube and Super-Kamiokande, on the Dark Matter-nucleon scattering cross-section, for a general model of Dark Matter with a velocity dependent (p-wave) thermally averaged cross-section. In this picture, the Boltzmann equation for the Dark Matter abundance is numerically solved satisfying the Dark Matter density measured from the Cosmic Microwave Background (CMB). We show that for lower cross-sections and higher masses, the Dark Matter annihilation rate drops sharply, resulting in upper bounds on the scattering cross-section one order of magnitude above those derived from a velocity independent (s-wave) annihilation cross-section. Our results show that upper limits on the scattering cross-section obtained from Dark Matter annihilating in the Sun are sensible to the uncertainty in current standard solar models, fluctuating a maximum of 20 % depending on the annihilation channel.

astro-ph.CO↗