arXiv subjects

Romain Mueller

Publications and source records attributed to Romain Mueller.

18 recordsLinked to original sources

arXiv2025

MObI: Multimodal Object Inpainting Using Diffusion Models

Safety-critical applications, such as autonomous driving, require extensive multimodal data for rigorous testing. Methods based on synthetic data are gaining prominence due to the cost and complexity of gathering real-world data but require a high degree of realism and controllability in order to be useful. This paper introduces MObI, a novel framework for Multimodal Object Inpainting that leverages a diffusion model to create realistic and controllable object inpaintings across perceptual modalities, demonstrated for both camera and lidar simultaneously. Using a single reference RGB image, MObI enables objects to be seamlessly inserted into existing multimodal scenes at a 3D location specified by a bounding box, while maintaining semantic consistency and multimodal coherence. Unlike traditional inpainting methods that rely solely on edit masks, our 3D bounding box conditioning gives objects accurate spatial positioning and realistic scaling. As a result, our approach can be used to insert novel objects flexibly into multimodal scenes, providing significant advantages for testing perception models.

cs.CV↗

arXiv2023

Lift-Attend-Splat: Bird's-eye-view camera-lidar fusion using transformers

Combining complementary sensor modalities is crucial to providing robust perception for safety-critical robotics applications such as autonomous driving (AD). Recent state-of-the-art camera-lidar fusion methods for AD rely on monocular depth estimation which is a notoriously difficult task compared to using depth information from the lidar directly. Here, we find that this approach does not leverage depth as expected and show that naively improving depth estimation does not lead to improvements in object detection performance. Strikingly, we also find that removing depth estimation altogether does not degrade object detection performance substantially, suggesting that relying on monocular depth could be an unnecessary architectural bottleneck during camera-lidar fusion. In this work, we introduce a novel fusion method that bypasses monocular depth estimation altogether and instead selects and fuses camera and lidar features in a bird's-eye-view grid using a simple attention mechanism. We show that our model can modulate its use of camera features based on the availability of lidar features and that it yields better 3D object detection on the nuScenes dataset than baselines relying on monocular depth estimation.

cs.CV↗

arXiv2023

Attacking Motion Planners Using Adversarial Perception Errors

Autonomous driving (AD) systems are often built and tested in a modular fashion, where the performance of different modules is measured using task-specific metrics. These metrics should be chosen so as to capture the downstream impact of each module and the performance of the system as a whole. For example, high perception quality should enable prediction and planning to be performed safely. Even though this is true in general, we show here that it is possible to construct planner inputs that score very highly on various perception quality metrics but still lead to planning failures. In an analogy to adversarial attacks on image classifiers, we call such inputs \textbf{adversarial perception errors} and show they can be systematically constructed using a simple boundary-attack algorithm. We demonstrate the effectiveness of this algorithm by finding attacks for two different black-box planners in several urban and highway driving scenarios using the CARLA simulator. Finally, we analyse the properties of these attacks and show that they are isolated in the input space of the planner, and discuss their implications for AD system deployment and testing.

cs.RO↗

arXiv2022

An Active Learning Reliability Method for Systems with Partially Defined Performance Functions

In engineering design, one often wishes to calculate the probability that the performance of a system is satisfactory under uncertainty. State of the art algorithms exist to solve this problem using active learning with Gaussian process models. However, these algorithms cannot be applied to problems which often occur in the autonomous vehicle domain where the performance of a system may be undefined under certain circumstances. To solve this problem, we introduce a hierarchical model for the system performance, where undefined performance is classified before the performance is regressed. This enables active learning Gaussian process methods to be applied to problems where the performance of the system is sometimes undefined, and we demonstrate the effectiveness of our approach by testing our methodology on synthetic numerical examples for the autonomous driving domain.

cs.LG↗

arXiv2022

Query-based Hard-Image Retrieval for Object Detection at Test Time

There is a longstanding interest in capturing the error behaviour of object detectors by finding images where their performance is likely to be unsatisfactory. In real-world applications such as autonomous driving, it is also crucial to characterise potential failures beyond simple requirements of detection performance. For example, a missed detection of a pedestrian close to an ego vehicle will generally require closer inspection than a missed detection of a car in the distance. The problem of predicting such potential failures at test time has largely been overlooked in the literature and conventional approaches based on detection uncertainty fall short in that they are agnostic to such fine-grained characterisation of errors. In this work, we propose to reformulate the problem of finding "hard" images as a query-based hard image retrieval task, where queries are specific definitions of "hardness", and offer a simple and intuitive method that can solve this task for a large family of queries. Our method is entirely post-hoc, does not require ground-truth annotations, is independent of the choice of a detector, and relies on an efficient Monte Carlo estimation that uses a simple stochastic model in place of the ground-truth. We show experimentally that it can be applied successfully to a wide variety of queries for which it can reliably identify hard images for a given detector without any labelled data. We provide results on ranking and classification tasks using the widely used RetinaNet, Faster-RCNN, Mask-RCNN, and Cascade Mask-RCNN object detectors. The code for this project is available at https://github.com/fiveai/hardest.

cs.CV↗

arXiv2022

Bridging microscopic cell dynamics to nematohydrodynamics of cell monolayers

It is increasingly being realized that liquid-crystalline features can play an important role in the properties and dynamics of cell monolayers. Here, we present a cell-based model of cell layers, based on the phase-field formulation, that connects mechanical properties at the single cell level to large-scale nematic and hydrodynamic properties of the tissue. In particular, we present a minimal formulation that reproduces the well-known bend and splay hydrodynamic instabilities of the continuum nemato-hydrodynamic formulation of active matter, together with an analytical description of the instability threshold in terms of activity and elasticity of the cells. Furthermore, we provide a quantitative characterisation and comparison of flows and topological defects for extensile and contractile stress generation mechanisms, and demonstrate activity-induced heterogeneity and spontaneous formation of gaps within a confluent monolayer. Together, these results contribute to bridging the gap between micro-scale cell dynamics and tissue-scale collective cellular organisation.

cond-mat.soft↗

arXiv2022

Attacking deep networks with surrogate-based adversarial black-box methods is easy

A recent line of work on black-box adversarial attacks has revived the use of transfer from surrogate models by integrating it into query-based search. However, we find that existing approaches of this type underperform their potential, and can be overly complicated besides. Here, we provide a short and simple algorithm which achieves state-of-the-art results through a search which uses the surrogate network's class-score gradients, with no need for other priors or heuristics. The guiding assumption of the algorithm is that the studied networks are in a fundamental sense learning similar functions, and that a transfer attack from one to the other should thus be fairly "easy". This assumption is validated by the extremely low query counts and failure rates achieved: e.g. an untargeted attack on a VGG-16 ImageNet network using a ResNet-152 as the surrogate yields a median query count of 6 at a success rate of 99.9%. Code is available at https://github.com/fiveai/GFCS.

cs.LG↗

arXiv2022

Parameter-free Online Test-time Adaptation

Training state-of-the-art vision models has become prohibitively expensive for researchers and practitioners. For the sake of accessibility and resource reuse, it is important to focus on adapting these models to a variety of downstream scenarios. An interesting and practical paradigm is online test-time adaptation, according to which training data is inaccessible, no labelled data from the test distribution is available, and adaptation can only happen at test time and on a handful of samples. In this paper, we investigate how test-time adaptation methods fare for a number of pre-trained models on a variety of real-world scenarios, significantly extending the way they have been originally evaluated. We show that they perform well only in narrowly-defined experimental setups and sometimes fail catastrophically when their hyperparameters are not selected for the same scenario in which they are being tested. Motivated by the inherent uncertainty around the conditions that will ultimately be encountered at test time, we propose a particularly "conservative" approach, which addresses the problem with a Laplacian Adjusted Maximum-likelihood Estimation (LAME) objective. By adapting the model's output (not its parameters), and solving our objective with an efficient concave-convex procedure, our approach exhibits a much higher average accuracy across scenarios than existing methods, while being notably faster and have a much lower memory footprint. The code is available at https://github.com/fiveai/LAME.

cs.CV↗

arXiv2022

Fluctuation-induced dynamics of nematic topological defects

Topological defects are increasingly being identified in various biological systems, where their characteristic flow fields and stress patterns are associated with continuous active stress generation by biological entities. Here, using numerical simulations of continuum fluctuating nematohydrodynamics we show that even in the absence of any specific form of active stresses associated with self-propulsion, mesoscopic fluctuations in either orientational alignment or hydrodynamics can independently result in flow patterns around topological defects that resemble the ones observed in active systems. Our simulations further show the possibility of extensile- and contractile-like motion of fluctuation-induced positive half-integer topological defects. Remarkably, isotropic stress fields also reproduce the experimentally measured stress patterns around topological defects in epithelia. Our findings further reveal that extensile- or contractile-like flow and stress patterns around fluctuation-induced defects are governed by passive elastic stresses and flow-aligning behavior of the nematics.

cond-mat.soft↗

arXiv2021

Half-integer and full-integer topological defects in polar active matter: Emergence, crossover, and coexistence

The presence and significance of active topological defects is increasingly realised in diverse biological and biomimetic systems. We introduce a continuum model of polar active matter, based on conservation laws and symmetry arguments, that recapitulates both polar and apolar (nematic) features of topological defects in active turbulence. Using numerical simulations of the continuum model, we demonstrate the emergence of both half- and full-integer topological defects in polar active matter. Interestingly, we find that crossover from active turbulence with half- to full-integer defects can emerge with the coexistence region characterized by both defect types. These results put forward a minimal, generic framework for studying topological defect patterns in active matter which is capable of explaining the emergence of half-integer defects in polar systems such as bacteria and cell monolayers, as well as predicting the emergence of coexisting defect states in active matter.

cond-mat.soft↗

arXiv2021

Phase field models of active matter

We present an overview of phase field modeling of active matter systems as a tool for capturing various aspects of complex and active interfaces. We first describe how interfaces between different phases are characterized in phase field models and provide simple fundamental governing equations that describe their evolution. For a simple model, we then show how physical properties of the interface, such as surface tension and interface thickness, can be recovered from these equations. We then explain how the phase field formulation can be coupled to various active matter realizations and discuss three particular examples of continuum biphasic active matter: active nematic-isotropic interfaces, active matter in viscoelastic environments, and active shells in fluid background. Finally, we describe how multiple phase fields can be used to model active cellular monolayers and present a general framework that can be applied to the study of tissue behaviour and collective migration.

cond-mat.soft↗

arXiv2020

Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks

Deep neural networks have improved image classification dramatically over the past decade, but have done so by focusing on performance measures that treat all classes other than the ground truth as equally wrong. This has led to a situation in which mistakes are less likely to be made than before, but are equally likely to be absurd or catastrophic when they do occur. Past works have recognised and tried to address this issue of mistake severity, often by using graph distances in class hierarchies, but this has largely been neglected since the advent of the current deep learning era in computer vision. In this paper, we aim to renew interest in this problem by reviewing past approaches and proposing two simple modifications of the cross-entropy loss which outperform the prior art under several metrics on two large datasets with complex class hierarchies: tieredImageNet and iNaturalist'19.

cs.CV↗

arXiv2020

Active Inter-cellular Forces in Collective Cell Motility

The collective behaviour of confluent cell sheets is strongly influenced both by polar forces, arising through cytoskeletal propulsion and by active inter-cellular forces, which are mediated by interactions across cell-cell junctions. We use a phase-field model to explore the interplay between these two contributions and compare the dynamics of a cell sheet when the polarity of the cells aligns to (i) their main axis of elongation, (ii) their velocity, and (iii) when the polarity direction executes a persistent random walk.In all three cases, we observe a sharp transition from a jammed state (where cell rearrangements are strongly suppressed) to a liquid state (where the cells can move freely relative to each other) when either the polar or the inter-cellular forces are increased. In addition, for case (ii) only, we observe an additional dynamical state, flocking (solid or liquid), where the majority of the cells move in the same direction. The flocking state is seen for strong polar forces, but is destroyed as the strength of the inter-cellular activity is increased.

cond-mat.soft↗

arXiv2019

Active Matter Invasion

Biological active materials such as bacterial biofilms and eukaryotic cells thrive in confined micro-spaces. Here, we show through numerical simulations that confinement can serve as a mechanical guidance to achieve distinct modes of collective invasion when combined with growth dynamics and the intrinsic activity of biological materials. We assess the dynamics of the growing interface and classify these collective modes of invasion based on the activity of the constituent particles of the growing matter. While at small and moderate activities the active material grows as a coherent unit, we find that blobs of active material collectively detach from the cohort above a well-defined activity threshold. We further characterise the mechanical mechanisms underlying the crossovers between different modes of invasion and quantify their impact on the overall invasion speed.

cond-mat.soft↗

arXiv2019

Emergence of active nematic behaviour in monolayers of isotropic cells

There is now growing evidence of the emergence and biological functionality of liquid crystal features, including nematic order and topological defects, in cellular tissues. However, how such features that intrinsically rely on particle elongation, emerge in monolayers of cells with isotropic shapes is an outstanding question. In this article we present a minimal model of cellular monolayers based on cell deformation and force transmission at the cell-cell interface that explains the formation of topological defects and captures the flow-field and stress patterns around them. By including mechanical properties at the individual cell level, we further show that the instability that drives the formation of topological defects and leads to active turbulence, emerges from a feedback between shape deformation and active driving. The model allows us to suggest new explanations for experimental observations in tissue mechanics, and to propose designs for future experiments.

cond-mat.soft↗

arXiv2016

On the computation of finite bottom-quark mass effects in Higgs boson production

We present analytic results for the partonic cross-sections contributing to the top-bottom interference in Higgs production via gluon fusion at hadron colliders at NLO accuracy in QCD. We develop a method of expansion in small bottom-mass for master integrals and combine it with the usual infinite top-mass effective theory. Our method of expansion admits a simple algorithmic description and can be easily generalized to any small parameter. These results for the integrated cross-sections will be needed in the computation of the renormalization counter-terms entering the computation of finite bottom-quark mass effects at NNLO.

hep-ph↗

arXiv2014

NNLO QCD corrections to $pp \to γ^* γ^*$ in the large $N_F$ limit

We compute the NNLO QCD corrections for the hadroproduction of a pair of off-shell photons in the limit of a large number of quark flavors. We perform a reduction of the two-loop amplitude to master integrals and calculate the latter analytically as a Laurent series in the dimensional regulator using modern integration methods. Real radiation corrections are evaluated numerically with a direct subtraction of infrared limits which we cast in a simple factorized form. The results presented here constitute a gauge invariant part of the full NNLO corrections but are not necessarily dominant. We view this calculation as a step towards a complete computation. Our partial corrections to the total cross-section are about $1\%-3\%$ and vary with the virtuality of the two off-shell photons.

hep-ph↗

arXiv2012

The fully differential hadronic production of a Higgs boson via bottom quark fusion at NNLO

The fully differential computation of the hadronic production cross section of a Higgs boson via bottom quarks is presented at NNLO in QCD. Several differential distributions with their corresponding scale uncertainties are presented for the 8 TeV LHC. This is the first application of the method of non-linear mappings for NNLO differential calculations at hadron colliders.

hep-ph↗