arXiv subjects

Thomas Müller

Publications and source records attributed to Thomas Müller.

At least 55 records · Page 3Linked to original sources

arXiv2021

TAPAS at SemEval-2021 Task 9: Reasoning over tables with intermediate pre-training

We present the TAPAS contribution to the Shared Task on Statement Verification and Evidence Finding with Tables (SemEval 2021 Task 9, Wang et al. (2021)). SEM TAB FACT Task A is a classification task of recognizing if a statement is entailed, neutral or refuted by the content of a given table. We adopt the binary TAPAS model of Eisenschlos et al. (2020) to this task. We learn two binary classification models: A first model to predict if a statement is neutral or non-neutral and a second one to predict if it is entailed or refuted. As the shared task training set contains only entailed or refuted examples, we generate artificial neutral examples to train the first model. Both models are pre-trained using a MASKLM objective, intermediate counter-factual and synthetic data (Eisenschlos et al., 2020) and TABFACT (Chen et al., 2020), a large table entailment dataset. We find that the artificial neutral examples are somewhat effective at training the first model, achieving 68.03 test F1 versus the 60.47 of a majority baseline. For the second stage, we find that the pre-training on the intermediate data and TABFACT improves the results over MASKLM pre-training (68.03 vs 57.01).

cs.CL↗

arXiv2020

Collective defense of honeybee colonies: experimental results and theoretical modeling

Social insect colonies routinely face large vertebrate predators, against which they need to mount a collective defense. To do so, honeybees use an alarm pheromone that recruits nearby bees into mass stinging of the perceived threat. This alarm pheromone is carried directly on the stinger, hence its concentration builds up during the course of the attack. Here, we investigate how individual bees react to different alarm pheromone concentrations, and how this evolved response-pattern leads to better coordination at the group level. We first present an individual dose-response curve to the alarm pheromone, obtained experimentally. Second, we apply Projective Simulation to model each bee as an artificial learning agent that relies on the pheromone concentration to decide whether to sting or not. If the emergent collective performance benefits the colony, the individual reactions that led to it are enhanced via reinforcement learning, thus emulating natural selection. Predators are modeled in a realistic way so that the effect of factors such as their resistance, their killing rate or their frequency of attacks can be studied. We are able to reproduce the experimentally measured response-pattern of real bees, and to identify the main selection pressures that shaped it. Finally, we apply the model to a case study: by tuning the parameters to represent the environmental conditions of European or African bees, we can predict the difference in aggressiveness observed between these two subspecies.

q-bio.PE↗

arXiv2020

Understanding tables with intermediate pre-training

Table entailment, the binary classification task of finding if a sentence is supported or refuted by the content of a table, requires parsing language and table structure as well as numerical and discrete reasoning. While there is extensive work on textual entailment, table entailment is less well studied. We adapt TAPAS (Herzig et al., 2020), a table-based BERT model, to recognize entailment. Motivated by the benefits of data augmentation, we create a balanced dataset of millions of automatically created training examples which are learned in an intermediate step prior to fine-tuning. This new data is not only useful for table entailment, but also for SQA (Iyyer et al., 2017), a sequential table QA task. To be able to use long examples as input of BERT models, we evaluate table pruning techniques as a pre-processing step to drastically improve the training and prediction efficiency at a moderate drop in accuracy. The different methods set the new state-of-the-art on the TabFact (Chen et al., 2020) and SQA datasets.

cs.CL↗

arXiv2020

IDEAS: Immersive Dome Experiences for Accelerating Science

Astrophysics lies at the crossroads of big datasets (such as the Large Synoptic Survey Telescope and Gaia), open source software to visualize and interpret high dimensional datasets (such as Glue, WorldWide Telescope, and OpenSpace), and uniquely skilled software engineers who bridge data science and research fields. At the same time, more than 4,000 planetariums across the globe immerse millions of visitors in scientific data. We have identified the potential for critical synergy across data, software, hardware, locations, and content that -- if prioritized over the next decade -- will drive discovery in astronomical research. Planetariums can and should be used for the advancement of scientific research. Current facilities such as the Hayden Planetarium in New York City, Adler Planetarium in Chicago, Morrison Planetarium in San Francisco, the Iziko Planetarium and Digital Dome Research Consortium in Cape Town, and Visualization Center C in Norrkoping are already developing software which ingests catalogs of astronomical and multi-disciplinary data critical for exploration research primarily for the purpose of creating scientific storylines for the general public. We propose a transformative model whereby scientists become the audience and explorers in planetariums, utilizing software for their own investigative purposes. In this manner, research benefits from the authentic and unique experience of data immersion contained in an environment bathed in context and equipped for collaboration. Consequently, in this white paper we argue that over the next decade the research astronomy community should partner with planetariums to create visualization-based research opportunities for the field. Realizing this vision will require new investments in software and human capital.

astro-ph.IM↗

arXiv2020

Neural Control Variates

We propose neural control variates (NCV) for unbiased variance reduction in parametric Monte Carlo integration. So far, the core challenge of applying the method of control variates has been finding a good approximation of the integrand that is cheap to integrate. We show that a set of neural networks can face that challenge: a normalizing flow that approximates the shape of the integrand and another neural network that infers the solution of the integral equation. We also propose to leverage a neural importance sampler to estimate the difference between the original integrand and the learned control variate. To optimize the resulting parametric estimator, we derive a theoretically optimal, variance-minimizing loss function, and propose an alternative, composite loss for stable online training in practice. When applied to light transport simulation, neural control variates are capable of matching the state-of-the-art performance of other unbiased approaches, while providing means to develop more performant, practical solutions. Specifically, we show that the learned light-field approximation is of sufficient quality for high-order bounces, allowing us to omit the error correction and thereby dramatically reduce the noise at the cost of negligible visible bias.

cs.LG↗

arXiv2020

On the suitability of hBN as an insulator for 2D material-based ultrascaled CMOS devices

Complementary metal oxide semiconductor (CMOS) logic circuits at the ultimate scaling limit place the utmost demands on the properties of all materials involved. The requirements for semiconductors are well explored and could possibly be satisfied by a number of layered two-dimensional (2D) materials, like for example transition-metal dichalcogenides or black phosphorus. The requirements for the gate insulator are arguably even more challenging and difficult to meet. In particular the combination of insulator to semiconductor which forms the central element of the metal oxide semiconductor field effect transistor (MOSFET) has to be of superior quality in order to build competitive devices. At the moment, hexagonal boron nitride (hBN) is the most common two-dimensional insulator and widely considered to be the most promising gate insulator in nanoscaled 2D material-based transistors. Here, we critically assess the material parameters of hBN and conclude that while its properties render hBN an ideal candidate for many applications in 2D nanoelectronics, hBN is most likely not suitable as a gate insulator for ultrascaled CMOS devices.

physics.app-ph↗

arXiv2020

TAPAS: Weakly Supervised Table Parsing via Pre-training

Answering natural language questions over tables is usually seen as a semantic parsing task. To alleviate the collection cost of full logical forms, one popular approach focuses on weak supervision consisting of denotations instead of logical forms. However, training semantic parsers from weak supervision poses difficulties, and in addition, the generated logical forms are only used as an intermediate step prior to retrieving the denotation. In this paper, we present TAPAS, an approach to question answering over tables without generating logical forms. TAPAS trains from weak supervision, and predicts the denotation by selecting table cells and optionally applying a corresponding aggregation operator to such selection. TAPAS extends BERT's architecture to encode tables as input, initializes from an effective joint pre-training of text segments and tables crawled from Wikipedia, and is trained end-to-end. We experiment with three different semantic parsing datasets, and find that TAPAS outperforms or rivals semantic parsing models by improving state-of-the-art accuracy on SQA from 55.1 to 67.2 and performing on par with the state-of-the-art on WIKISQL and WIKITQ, but with a simpler model architecture. We additionally find that transfer learning, which is trivial in our setting, from WIKISQL to WIKITQ, yields 48.7 accuracy, 4.2 points above the state-of-the-art.

cs.IR↗

arXiv2020

Development of swarm behavior in artificial learning agents that adapt to different foraging environments

Collective behavior, and swarm formation in particular, has been studied from several perspectives within a large variety of fields, ranging from biology to physics. In this work, we apply Projective Simulation to model each individual as an artificial learning agent that interacts with its neighbors and surroundings in order to make decisions and learn from them. Within a reinforcement learning framework, we discuss one-dimensional learning scenarios where agents need to get to food resources to be rewarded. We observe how different types of collective motion emerge depending on the distance the agents need to travel to reach the resources. For instance, strongly aligned swarms emerge when the food source is placed far away from the region where agents are situated initially. In addition, we study the properties of the individual trajectories that occur within the different types of emergent collective dynamics. Agents trained to find distant resources exhibit individual trajectories with Lévy-like characteristics as a consequence of the collective motion, whereas agents trained to reach nearby resources present Brownian-like trajectories.

q-bio.PE↗

arXiv2020

Small Bodies: Near and Far Database for thermal infrared observations of small bodies in the Solar System

In this paper we present the "Small Bodies: Near and Far" Infrared Database, an easy-to-use tool intended to facilitate the modeling of thermal emission of small Solar System bodies. Our database collects thermal emission measurements of small Solar Systems targets that are otherwise available in scattered sources and gives a complete description of the data, with all information necessary to perform direct scientific analyses and without the need to access additional, external resources. This public database contains representative data of asteroid observations of large surveys (e.g. AKARI, IRAS and WISE) as well as a collection of small body observations of infrared space telescopes (e.g. the Herschel Space Observatory) and provides a web interface to access this data (https://ird.konkoly.hu). We also provide an example for the direct application of the database and show how it can be used to estimate the thermal inertia of specific populations, e.g. asteroids within a given size range. We show how different scalings of thermal inertia with heliocentric distance (i.e. temperature) may affect our interpretation of the data and discuss why the widely-used radiative conductivity exponent ($α$=-3/4) might not be adequate in general, as hinted by previous studies.

astro-ph.IM↗

arXiv2019

How a minimal learning agent can infer the existence of unobserved variables in a complex environment

According to a mainstream position in contemporary cognitive science and philosophy, the use of abstract compositional concepts is both a necessary and a sufficient condition for the presence of genuine thought. In this article, we show how the ability to develop and utilise abstract conceptual structures can be achieved by a particular kind of learning agents. More specifically, we provide and motivate a concrete operational definition of what it means for these agents to be in possession of abstract concepts, before presenting an explicit example of a minimal architecture that supports this capability. We then proceed to demonstrate how the existence of abstract conceptual structures can be operationally useful in the process of employing previously acquired knowledge in the face of new experiences, thereby vindicating the natural conjecture that the cognitive functions of abstraction and generalisation are closely related. Keywords: concept formation, projective simulation, reinforcement learning, transparent artificial intelligence, theory formation, explainable artificial intelligence (XAI)

cs.LG↗

arXiv2019

Neural Importance Sampling

We propose to use deep neural networks for generating samples in Monte Carlo integration. Our work is based on non-linear independent components estimation (NICE), which we extend in numerous ways to improve performance and enable its application to integration problems. First, we introduce piecewise-polynomial coupling transforms that greatly increase the modeling power of individual coupling layers. Second, we propose to preprocess the inputs of neural networks using one-blob encoding, which stimulates localization of computation and improves inference. Third, we derive a gradient-descent-based optimization for the KL and the $χ^2$ divergence for the specific application of Monte Carlo integration with unnormalized stochastic estimates of the target distribution. Our approach enables fast and accurate inference and efficient sample generation independently of the dimensionality of the integration domain. We show its benefits on generating natural images and in two applications to light-transport simulation: first, we demonstrate learning of joint path-sampling densities in the primary sample space and importance sampling of multi-dimensional path prefixes thereof. Second, we use our technique to extract conditional directional densities driven by the product of incident illumination and the BSDF in the rendering equation, and we leverage the densities for path guiding. In all applications, our approach yields on-par or higher performance than competing techniques at equal sample count.

cs.LG↗

arXiv2019

Answering Conversational Questions on Structured Data without Logical Forms

We present a novel approach to answering sequential questions based on structured objects such as knowledge bases or tables without using a logical form as an intermediate representation. We encode tables as graphs using a graph neural network model based on the Transformer architecture. The answers are then selected from the encoded graph using a pointer network. This model is appropriate for processing conversations around structured data, where the attention mechanism that selects the answers to a question can also be used to resolve conversational references. We demonstrate the validity of this approach with competitive results on the Sequential Question Answering (SQA) task (Iyyer et al., 2017).

cs.CL↗

arXiv2019

Finite non-cyclic $p$-groups whose number of subgroups is minimal

Recent results of Qu and Tuarnauceanu describe explicitly the finite p-groups which are not elementary abelian and have the property that the number of their subgroups is maximal among p-groups of a given order. We complement these results from the bottom level up by determining completely the non-cyclic finite p-groups whose number of subgroups among p-groups of a given order is minimal.

math.GR↗

arXiv2019

Trans-Neptunian objects and Centaurs at thermal wavelengths

The thermal emission of transneptunian objects (TNO) and Centaurs has been observed at mid- and far-infrared wavelengths - with the biggest contributions coming from the Spitzer and Herschel space observatories-, and the brightest ones also at sub-millimeter and millimeter wavelengths. These measurements allowed to determine the sizes and albedos for almost 180 objects, and densities for about 25 multiple systems. The derived very low thermal inertias show evidence for a decrease at large heliocentric distances and for high-albedo objects, which indicates porous and low-conductivity surfaces. The radio emissivity was found to be low ($ε_r$=0.70$\pm$0.13) with possible spectral variations in a few cases. The general increase of density with object size points to different formation locations or times. The mean albedos increase from about 5-6% (Centaurs, Scattered-Disk Objects) to 15% for the Detached objects, with distinct cumulative albedo distributions for hot and cold classicals. The color-albedo separation in our sample is evidence for a compositional discontinuity in the young Solar System. The median albedo of the sample (excluding dwarf planets and the Haumea family) is 0.08, the albedo of Haumea family members is close to 0.5, best explained by the presence of water ice. The existing thermal measurements remain a treasure trove at times where the far-infrared regime is observationally not accessible.

astro-ph.EP↗

arXiv2018

First Data Release of the COSMOS Lyman-Alpha Mapping And Tomography Observations: 3D Lyman-$α$ Forest Tomography at 2.05 < z < 2.55

Faint star-forming galaxies at $z\sim 2-3$ can be used as alternative background sources to probe the Lyman-$α$ forest in addition to quasars, yielding high sightline densities that enable 3D tomographic reconstruction of the foreground absorption field. Here, we present the first data release from the COSMOS Lyman-Alpha Mapping And Mapping Observations (CLAMATO) Survey, which was conducted with the LRIS spectrograph on the Keck-I telescope. Over an observational footprint of 0.157$\mathrm{deg}^2$ within the COSMOS field, we used 240 galaxies and quasars at $2.17<z<3.00$, with a mean comoving transverse separation of $2.37\,h^{-1}\,\mathrm{Mpc^3}$, as background sources probing the foreground Lyman-$α$ forest absorption at $2.05<z<2.55$. The Lyman-$α$ forest data was then used to create a Wiener-filtered tomographic reconstruction over a comoving volume of $3.15\,\times 10^5\,h^{-3}\,\mathrm{Mpc^3}$ with an effective smoothing scale of $2.5\,h^{-1}\,\mathrm{Mpc}$. In addition to traditional figures, this map is also presented as a virtual-reality YouTube360 video visualization and manipulable interactive figure. We see large overdensities and underdensities that visually agree with the distribution of coeval galaxies from spectroscopic redshift surveys in the same field, including overdensities associated with several recently-discovered galaxy protoclusters in the volume. This data release includes the redshift catalog, reduced spectra, extracted Lyman-$α$ forest pixel data, and tomographic map of the absorption.

astro-ph.CO↗

arXiv2018

Asteroid Ryugu Before the Hayabusa2 Encounter

Asteroid (162173) Ryugu is the target object of Hayabusa2, an asteroid exploration and sample return mission led by Japan Aerospace Exploration Agency (JAXA). Ground-based observations indicate that Ryugu is a C-type near-Earth asteroid with a diameter of less than 1 km, but the knowledge of its detailed properties is still very limited. This paper summarizes our best understanding of the physical and dynamical properties of Ryugu based on remote sensing and theoretical modeling. This information is used to construct a design reference model of the asteroid that is used for formulation of mission operations plans in advance of asteroid arrival. Particular attention is given to the surface properties of Ryugu that are relevant to sample acquisition. This reference model helps readers to appropriately interpret the data that will be directly obtained by Hayabusa2 and promotes scientific studies not only for Ryugu itself and other small bodies but also for the Solar System evolution that small bodies shed light on.

astro-ph.EP↗

arXiv2018

Exploring the Dust Content of Galactic Winds With Herschel. II. Nearby Dwarf Galaxies

We present the results from an analysis of deep Herschel Space Observatory observations of six nearby dwarf galaxies known to host galactic-scale winds. The superior far-infrared sensitivity and angular resolution of Herschel have allowed detection of cold circumgalactic dust features beyond the stellar components of the host galaxies traced by Spitzer 4.5 $μ$m images. Comparisons of these cold dust features with ancillary data reveal an imperfect spatial correlation with the ionized gas and warm dust wind components. We find that typically $\sim$10-20\% of the total dust mass in these galaxies resides outside of their stellar disks, but this fraction reaches $\sim$60\% in the case of NGC 1569. This galaxy also has the largest metallicity (O/H) deficit in our sample for its stellar mass. Overall, the small number of objects in our sample precludes drawing strong conclusions on the origin of the circumgalactic dust. We detect no statistically significant trends with star formation properties of the host galaxies, as might be expected if the dust were lifted above the disk by energy inputs from on-going star formation activity. Although a case for dust entrained in a galactic wind is seen in NGC 1569, in all cases, we cannot rule out the possibility that some of the circumgalactic dust might be associated instead with gas accreted or removed from the disk by recent galaxy interaction events, or that it is part of the outer gas-rich portion of the disk that lies below the sensitivity limit of the Spitzer 4.5 $μ$m data.

astro-ph.GA↗

arXiv2017

Modelling collective motion based on the principle of agency

Collective motion is an intriguing phenomenon, especially considering that it arises from a set of simple rules governing local interactions between individuals. In theoretical models, these rules are normally \emph{assumed} to take a particular form, possibly constrained by heuristic arguments. We propose a new class of models, which describe the individuals as \emph{agents}, capable of deciding for themselves how to act and learning from their experiences. The local interaction rules do not need to be postulated in this model, since they \emph{emerge} from the learning process. We apply this ansatz to a concrete scenario involving marching locusts, in order to model the phenomenon of density-dependent alignment. We show that our learning agent-based model can account for a Fokker-Planck equation that describes the collective motion and, most notably, that the agents can learn the appropriate local interactions, requiring no strong previous assumptions on their form. These results suggest that learning agent-based models are a powerful tool for studying a broader class of problems involving collective motion and animal agency in general.

q-bio.PE↗