arXiv subjects

Tobias Fischer

Publications and source records attributed to Tobias Fischer.

At least 109 records · Page 6Linked to original sources

arXiv2022

Spiking Neural Networks for Visual Place Recognition via Weighted Neuronal Assignments

Spiking neural networks (SNNs) offer both compelling potential advantages, including energy efficiency and low latencies and challenges including the non-differentiable nature of event spikes. Much of the initial research in this area has converted deep neural networks to equivalent SNNs, but this conversion approach potentially negates some of the advantages of SNN-based approaches developed from scratch. One promising area for high-performance SNNs is template matching and image recognition. This research introduces the first high-performance SNN for the Visual Place Recognition (VPR) task: given a query image, the SNN has to find the closest match out of a list of reference images. At the core of this new system is a novel assignment scheme that implements a form of ambiguity-informed salience, by up-weighting single-place-encoding neurons and down-weighting "ambiguous" neurons that respond to multiple different reference places. In a range of experiments on the challenging Nordland, Oxford RobotCar, SPEDTest, Synthia, and St Lucia datasets, we show that our SNN achieves comparable VPR performance to state-of-the-art and classical techniques, and degrades gracefully in performance with an increasing number of reference places. Our results provide a significant milestone towards SNNs that can provide robust, energy-efficient, and low latency robot localization.

cs.CV↗

arXiv2021

A RoboStack Tutorial: Using the Robot Operating System Alongside the Conda and Jupyter Data Science Ecosystems

We argue that it is beneficial to tightly couple the widely-used Robot Operating System with Conda, a cross-platform, language-agnostic package manager, and Jupyter, a web-based interactive computational environment affording scientific computing. We provide new ROS packages for Conda, enabling the installation of ROS alongside data-science and machine-learning packages with ease. Multiple ROS versions (currently ROS1 Melodic and Noetic, as well as ROS2 Foxy and Galactic) can run simultaneously on one machine, with pre-compiled binaries available for Linux, Windows and OSX, and the ARM architecture (e.g. the Raspberry Pi and the new Apple Silicon). To deal with the large size of the ROS ecosystem, we significantly improved the speed of the Conda solver and build system by rewriting the crucial parts in C++. We further contribute a collection of JupyterLab extensions for ROS, including plugins for live plotting, debugging and robot control, as well as tight integration with Zethus, an RViz like visualization tool. Taken together, RoboStack combines the best of the data-science and robotics worlds to help researchers and developers to build custom solutions for their academic and industrial projects.

cs.RO↗

arXiv2021

Unsupervised Complementary-aware Multi-process Fusion for Visual Place Recognition

A recent approach to the Visual Place Recognition (VPR) problem has been to fuse the place recognition estimates of multiple complementary VPR techniques simultaneously. However, selecting the optimal set of techniques to use in a specific deployment environment a-priori is a difficult and unresolved challenge. Further, to the best of our knowledge, no method exists which can select a set of techniques on a frame-by-frame basis in response to image-to-image variations. In this work, we propose an unsupervised algorithm that finds the most robust set of VPR techniques to use in the current deployment environment, on a frame-by-frame basis. The selection of techniques is determined by an analysis of the similarity scores between the current query image and the collection of database images and does not require ground-truth information. We demonstrate our approach on a wide variety of datasets and VPR techniques and show that the proposed dynamic multi-process fusion (Dyn-MPF) has superior VPR performance compared to a variety of challenging competitive methods, some of which are given an unfair advantage through access to the ground-truth information.

cs.CV↗

arXiv2021

Observable signatures of enhanced axion emission from protoneutron stars

We perform general relativistic one-dimensional supernova (SN) simulations to identify observable signatures of enhanced axion emission from the pion-induced reaction $π^- + p \rightarrow n + a$ inside a newly born protoneutron star (PNS).We focus on the early evolution after the onset of the supernova explosion to predict the temporal and spectral features of the neutrino and axion emission during the first 10 s. Pions are included as explicit new degrees of freedom in hot and dense matter. Their thermal population and their role in axion production are both determined consistently to include effects due to their interactions with nucleons. For a wide range of ambient conditions encountered inside a PNS, we find that the pion-induced axion production dominates over nucleon-nucleon bremsstrahlung processes. By consistently including the role of pions on the dense matter equation of state and on the energy loss, our simulations predict robust discernible features of neutrino and axion emission from a galactic supernova that can be observed in terrestrial detectors. For axion couplings that are compatible with current bounds, we find a significant suppression with time of the neutrino luminosity during the first 10 s. This suggests that current bounds derived from the neutrino signal from SN 1987A can be improved and that future galactic supernovae may provide significantly more stringent constraints.

hep-ph↗

arXiv2021

Where is your place, Visual Place Recognition?

Visual Place Recognition (VPR) is often characterized as being able to recognize the same place despite significant changes in appearance and viewpoint. VPR is a key component of Spatial Artificial Intelligence, enabling robotic platforms and intelligent augmentation platforms such as augmented reality devices to perceive and understand the physical world. In this paper, we observe that there are three "drivers" that impose requirements on spatially intelligent agents and thus VPR systems: 1) the particular agent including its sensors and computational resources, 2) the operating environment of this agent, and 3) the specific task that the artificial agent carries out. In this paper, we characterize and survey key works in the VPR area considering those drivers, including their place representation and place matching choices. We also provide a new definition of VPR based on the visual overlap -- akin to spatial view cells in the brain -- that enables us to find similarities and differences to other research areas in the robotics and computer vision fields. We identify numerous open challenges and suggest areas that require more in-depth attention in future works.

cs.RO↗

arXiv2021

Ambiguities in Direction-of-Arrival Estimation with Linear Arrays

In this paper, we present a novel approach to compute ambiguities in thinned uniform linear arrays, i.e., sparse non-uniform linear arrays, via a mixed-integer program. Ambiguities arise when there exists a set of distinct directions-of-arrival, for which the corresponding steering matrix is rank-deficient and are associated with nonunique parameter estimation. Our approach uses Young tableaux for which a submatrix of the steering matrix has a vanishing determinant, which can be expressed through vanishing sums of unit roots. Each of these vanishing sums then corresponds to an ambiguous set of directions-of-arrival. We derive a method to enumerate such ambiguous sets using a mixed-integer program and present results on several examples.

eess.SP↗

arXiv2021

Core-collapse supernova simulations and the formation of neutron stars, hybrid stars, and black holes

We investigate observable signatures of a first-order quantum chromodynamics (QCD) phase transition in the context of core collapse supernovae. To this end, we conduct axially symmetric numerical relativity simulations with multi-energy neutrino transport, using a hadron-quark hybrid equation of state (EOS). We consider four non-rotating progenitor models, whose masses range from $9.6$ to $70$ M$_\odot$. We find that the two less massive progenitor stars (9.6 and 11.2 M$_\odot$) show a successful explosion, which is driven by the neutrino heating. They do not undergo the QCD phase transition and leave behind a neutron star (NS). As for the more massive progenitor stars (50 and 70 M$_\odot$), the proto-neutron star (PNS) core enters the phase transition region and experiences the second collapse. Because of a sudden stiffening of the EOS entering to the pure quark matter regime, a strong shock wave is formed and blows off the PNS envelope in the 50 M$_\odot$ model. Consequently, the remnant becomes a quark core surrounded by hadronic matters, leading to the formation of the hybrid star. However for the 70 M$_\odot$ model, the shock wave cannot overcome the continuous mass accretion and it readily becomes a black hole. We find that the neutrino and gravitational wave (GW) signals from supernova explosions driven by the hadron-quark phase transition are detectable for the present generation of neutrino and GW detectors. Furthermore, the analysis of the GW detector response reveals unique kHz signatures, which will allow us to distinguish this class of supernova explosions from failed and neutrino-driven explosions.

astro-ph.HE↗

arXiv2021

QCD phase transition drives supernova explosion of a very massive star

The nature of core-collapse supernova (SN) explosions is yet incompletely understood. The present article revisits the scenario in which the release of latent heat due to a first-order phase transition, from normal nuclear matter to the quark-gluon plasma, liberates the necessary energy to explain observed SN explosions. Here, the role of the metallicity of the stellar progenitor is investigated, comparing a solar metallicity and a low-metallicity case, both having a zero-age main sequence (ZAMS) mass of 75 $M_\odot$. It is found that low-metallicity models belong exclusively to the failed SN branch, featuring the formation of black holes without explosions. It excludes this class of massive star explosions as possible site for the nucleosynthesis of heavy elements at extremely low metallicity, usually associated with the early universe.

astro-ph.HE↗

arXiv2021

Bio-inspired robot perception coupled with robot-modeled human perception

My overarching research goal is to provide robots with perceptional abilities that allow interactions with humans in a human-like manner. To develop these perceptional abilities, I believe that it is useful to study the principles of the human visual system. I use these principles to develop new computer vision algorithms and validate their effectiveness in intelligent robotic systems. I am enthusiastic about this approach as it offers the dual benefit of uncovering principles inherent in the human visual system, as well as applying these principles to its artificial counterpart. Fig. 1 contains a depiction of my research.

cs.RO↗

arXiv2021

Fast and Robust Bio-inspired Teach and Repeat Navigation

Fully autonomous mobile robots have a multitude of potential applications, but guaranteeing robust navigation performance remains an open research problem. For many tasks such as repeated infrastructure inspection, item delivery, or inventory transport, a route repeating capability can be sufficient and offers potential practical advantages over a full navigation stack. Previous teach and repeat research has achieved high performance in difficult conditions predominantly by using sophisticated, expensive sensors, and has often had high computational requirements. Biological systems, such as small animals and insects like seeing ants, offer a proof of concept that robust and generalisable navigation can be achieved with extremely limited visual systems and computing power. In this work we create a novel asynchronous formulation for teach and repeat navigation that fully utilises odometry information, paired with a correction signal driven by much more computationally lightweight visual processing than is typically required. This correction signal is also decoupled from the robot's motor control, allowing its rate to be modulated by the available computing capacity. We evaluate this approach with extensive experimentation on two different robotic platforms, the Consequential Robotics Miro and the Clearpath Jackal robots, across navigation trials totalling more than 6000 metres in a range of challenging indoor and outdoor environments. Our approach continues to succeed when multiple state-of-the-art systems fail due to low resolution images, unreliable odometry, or lighting change, while requiring significantly less compute. We also - for the first time - demonstrate versatile cross-platform teach and repeat without changing parameters, in which we learn to navigate a route with one robot and repeat that route using a completely different robot.

cs.RO↗

arXiv2021

Probabilistic Appearance-Invariant Topometric Localization with New Place Awareness

Probabilistic state-estimation approaches offer a principled foundation for designing localization systems, because they naturally integrate sequences of imperfect motion and exteroceptive sensor data. Recently, probabilistic localization systems utilizing appearance-invariant visual place recognition (VPR) methods as the primary exteroceptive sensor have demonstrated state-of-the-art performance in the presence of substantial appearance change. However, existing systems 1) do not fully utilize odometry data within the motion models, and 2) are unable to handle route deviations, due to the assumption that query traverses exactly repeat the mapping traverse. To address these shortcomings, we present a new probabilistic topometric localization system which incorporates full 3-dof odometry into the motion model and furthermore, adds an "off-map" state within the state-estimation framework, allowing query traverses which feature significant route detours from the reference map to be successfully localized. We perform extensive evaluation on multiple query traverses from the Oxford RobotCar dataset exhibiting both significant appearance change and deviations from routes previously traversed. In particular, we evaluate performance on two practically relevant localization tasks: loop closure detection and global localization. Our approach achieves major performance improvements over both existing and improved state-of-the-art systems.

cs.RO↗

arXiv2021

Monocular Quasi-Dense 3D Object Tracking

A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform. The object association leverages quasi-dense similarity learning to identify objects in various poses and viewpoints with appearance cues only. After initial 2D association, we further utilize 3D bounding boxes depth-ordering heuristics for robust instance association and motion-based 3D trajectory prediction for re-identification of occluded vehicles. In the end, an LSTM-based object velocity learning module aggregates the long-term trajectory information for more accurate motion extrapolation. Experiments on our proposed simulation data and real-world benchmarks, including KITTI, nuScenes, and Waymo datasets, show that our tracking framework offers robust object association and tracking on urban-driving scenarios. On the Waymo Open benchmark, we establish the first camera-only baseline in the 3D tracking and 3D detection challenges. Our quasi-dense 3D tracking pipeline achieves impressive improvements on the nuScenes 3D tracking benchmark with near five times tracking accuracy of the best vision-only submission among all published methods. Our code, data and trained models are available at https://github.com/SysCV/qd-3dt.

cs.CV↗

arXiv2021

Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition

Visual Place Recognition is a challenging task for robotics and autonomous systems, which must deal with the twin problems of appearance and viewpoint change in an always changing world. This paper introduces Patch-NetVLAD, which provides a novel formulation for combining the advantages of both local and global descriptor methods by deriving patch-level features from NetVLAD residuals. Unlike the fixed spatial neighborhood regime of existing local keypoint features, our method enables aggregation and matching of deep-learned local features defined over the feature-space grid. We further introduce a multi-scale fusion of patch features that have complementary scales (i.e. patch sizes) via an integral feature space and show that the fused features are highly invariant to both condition (season, structure, and illumination) and viewpoint (translation and rotation) changes. Patch-NetVLAD outperforms both global and local feature descriptor-based methods with comparable compute, achieving state-of-the-art visual place recognition results on a range of challenging real-world datasets, including winning the Facebook Mapillary Visual Place Recognition Challenge at ECCV2020. It is also adaptable to user requirements, with a speed-optimised version operating over an order of magnitude faster than the state-of-the-art. By combining superior performance with improved computational efficiency in a configurable framework, Patch-NetVLAD is well suited to enhance both stand-alone place recognition capabilities and the overall performance of SLAM systems.

cs.CV↗

arXiv2021

Muonization of supernova matter

The present article investigates the impact of muons on core-collapse supernovae, with particular focus on the early muon neutrino emission. While the presence of muons is well understood in the context of neutron stars, until the recent study by Bollig et al. [Phys. Rev. Lett. 119, 242702 (2017)] the role of muons in core-collapse supernovae had been neglected--electrons and neutrinos were the only leptons considered. In their study, Bollig et al. disentangled the muon and tau neutrinos and antineutrinos and included a variety of muonic weak reactions, all of which the present paper follows closely. Only then does it becomes possible to quantify the appearance of muons shortly before stellar core bounce and how the post-bounce prompt neutrino emission is modified

astro-ph.HE↗

arXiv2021

Intelligent Reference Curation for Visual Place Recognition via Bayesian Selective Fusion

A key challenge in visual place recognition (VPR) is recognizing places despite drastic visual appearance changes due to factors such as time of day, season, weather or lighting conditions. Numerous approaches based on deep-learnt image descriptors, sequence matching, domain translation, and probabilistic localization have had success in addressing this challenge, but most rely on the availability of carefully curated representative reference images of the possible places. In this paper, we propose a novel approach, dubbed Bayesian Selective Fusion, for actively selecting and fusing informative reference images to determine the best place match for a given query image. The selective element of our approach avoids the counterproductive fusion of every reference image and enables the dynamic selection of informative reference images in environments with changing visual conditions (such as indoors with flickering lights, outdoors during sunshowers or over the day-night cycle). The probabilistic element of our approach provides a means of fusing multiple reference images that accounts for their varying uncertainty via a novel training-free likelihood function for VPR. On difficult query images from two benchmark datasets, we demonstrate that our approach matches and exceeds the performance of several alternative fusion approaches along with state-of-the-art techniques that are provided with prior (unfair) knowledge of the best reference images. Our approach is well suited for long-term robot autonomy where dynamic visual environments are commonplace since it is training-free, descriptor-agnostic, and complements existing techniques such as sequence matching.

cs.CV↗

arXiv2020

Heavy axion-like particles and core-collapse supernovae: constraints and impact on the explosion mechanism

Heavy axion-like particles (ALPs), with masses $m_a \gtrsim 100$ keV, coupled with photons, would be copiously produced in a supernova (SN) core via Primakoff process and photon coalescence. Using a state-of-the-art SN model, we revisit the energy-loss SN 1987A bounds on axion-photon coupling. Moreover, we point out that heavy ALPs with masses $m_a \gtrsim 100$ MeV and axion-photon coupling $g_{aγ} \gtrsim 4 \times 10^{-9}$ GeV$^{-1}$ would decay into photons behind the shock-wave producing a possible enhancement in the energy deposition that would boost the SN shock revival.

hep-ph↗

arXiv2020

Constraining the onset density of the hadron-quark phase transition with gravitational-wave observations

We study the possible occurrence of the hadron-quark phase transition (PT) during the merging of neutron star binaries by hydrodynamical simulations employing a set of temperature dependent hybrid equations of state (EoSs). Following previous work we describe an unambiguous and measurable signature of deconfined quark matter in the gravitational-wave (GW) signal of neutron star binary mergers including equal-mass and unequal-mass systems of different total binary mass. The softening of the EoS by the PT at higher densities, i.e. after merging, leads to a characteristic increase of the dominant postmerger GW frequency f_peak relative to the tidal deformability Lambda inferred during the premerger inspiral phase. Hence, measuring such an increase of the postmerger frequency provides evidence for the presence of a strong PT. If the postmerger frequency and the tidal deformability are compatible with results from purely baryonic EoS models yielding very tight relations between f_peak and Lambda, a strong PT can be excluded up to a certain density. We find tight correlations of f_peak and Lambda with the maximum density during the early postmerger remnant evolution. These GW observables thus inform about the density regime which is probed by the remnant and its GW emission. Exploiting such relations we devise a directly applicable, concrete procedure to constrain the onset density of the QCD PT from future GW measurements. We point out two interesting scenarios: if no indications for a PT are inferred from a GW detection, our procedure yields a lower limit on the onset density of the hadron quark PT. On the contrary, if a merger event reveals evidence for the occurrence of deconfined quark matter, the inferred GW parameters set an upper limit on the PT onset density. (abridged)

astro-ph.HE↗

arXiv2020

Equation of state constraints from the threshold binary mass for prompt collapse of neutron star mergers

Using hydrodynamical simulations for a large set of high-density matter equations of state (EoSs) we systematically determine the threshold mass M_thres for prompt black-hole formation in equal-mass and asymmetric neutron star (NS) mergers. We devise the so far most direct, general and accurate method to determine the unknown maximum mass of nonrotating NSs from merger observations revealing M_thres. Considering hybrid EoSs with hadron-quark phase transition, we identify a new, observable signature of quark matter in NS mergers. Furthermore, our findings have direct applications in gravitational wave searches, kilonova interpretations and multi-messenger constraints on NS properties.

astro-ph.HE↗