arXiv subjects

Tobias Fischer

Publications and source records attributed to Tobias Fischer.

At least 91 records · Page 5Linked to original sources

arXiv2023

Image Labels Are All You Need for Coarse Seagrass Segmentation

Seagrass meadows serve as critical carbon sinks, but estimating the amount of carbon they store requires knowledge of the seagrass species present. Underwater and surface vehicles equipped with machine learning algorithms can help to accurately estimate the composition and extent of seagrass meadows at scale. However, previous approaches for seagrass detection and classification have required supervision from patch-level labels. In this paper, we reframe seagrass classification as a weakly supervised coarse segmentation problem where image-level labels are used during training (25 times fewer labels compared to patch-level labeling) and patch-level outputs are obtained at inference time. To this end, we introduce SeaFeats, an architecture that uses unsupervised contrastive pre-training and feature similarity, and SeaCLIP, a model that showcases the effectiveness of large language models as a supervisory signal in domain-specific applications. We demonstrate that an ensemble of SeaFeats and SeaCLIP leads to highly robust performance. Our method outperforms previous approaches that require patch-level labels on the multi-species 'DeepSeagrass' dataset by 6.8% (absolute) for the class-weighted F1 score, and by 12.1% (absolute) for the seagrass presence/absence F1 score on the 'Global Wetlands' dataset. We also present two case studies for real-world deployment: outlier detection on the Global Wetlands dataset, and application of our method on imagery collected by the FloatyBoat autonomous surface vehicle.

cs.CV↗

arXiv2023

R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras

Dense 3D reconstruction and ego-motion estimation are key challenges in autonomous driving and robotics. Compared to the complex, multi-modal systems deployed today, multi-camera systems provide a simpler, low-cost alternative. However, camera-based 3D reconstruction of complex dynamic scenes has proven extremely difficult, as existing solutions often produce incomplete or incoherent results. We propose R3D3, a multi-camera system for dense 3D reconstruction and ego-motion estimation. Our approach iterates between geometric estimation that exploits spatial-temporal information from multiple cameras, and monocular depth refinement. We integrate multi-camera feature correlation and dense bundle adjustment operators that yield robust geometric depth and pose estimates. To improve reconstruction where geometric depth is unreliable, e.g. for moving objects or low-textured regions, we introduce learnable scene priors via a depth refinement network. We show that this design enables a dense, consistent 3D reconstruction of challenging, dynamic outdoor environments. Consequently, we achieve state-of-the-art dense depth prediction on the DDAD and NuScenes benchmarks.

cs.CV↗

arXiv2023

SAFE: Sensitivity-Aware Features for Out-of-Distribution Object Detection

We address the problem of out-of-distribution (OOD) detection for the task of object detection. We show that residual convolutional layers with batch normalisation produce Sensitivity-Aware FEatures (SAFE) that are consistently powerful for distinguishing in-distribution from out-of-distribution detections. We extract SAFE vectors for every detected object, and train a multilayer perceptron on the surrogate task of distinguishing adversarially perturbed from clean in-distribution examples. This circumvents the need for realistic OOD training data, computationally expensive generative models, or retraining of the base object detector. SAFE outperforms the state-of-the-art OOD object detectors on multiple benchmarks by large margins, e.g. reducing the FPR95 by an absolute 30.6% from 48.3% to 17.7% on the OpenImages dataset.

cs.CV↗

arXiv2023

Visual Place Recognition: A Tutorial

Localization is an essential capability for mobile robots. A rapidly growing field of research in this area is Visual Place Recognition (VPR), which is the ability to recognize previously seen places in the world based solely on images. This present work is the first tutorial paper on visual place recognition. It unifies the terminology of VPR and complements prior research in two important directions: 1) It provides a systematic introduction for newcomers to the field, covering topics such as the formulation of the VPR problem, a general-purpose algorithmic pipeline, an evaluation methodology for VPR approaches, and the major challenges for VPR and how they may be addressed. 2) As a contribution for researchers acquainted with the VPR problem, it examines the intricacies of different VPR problem types regarding input, data processing, and output. The tutorial also discusses the subtleties behind the evaluation of VPR algorithms, e.g., the evaluation of a VPR system that has to find all matching database images per query, as opposed to just a single match. Practical code examples in Python illustrate to prospective practitioners and researchers how VPR is implemented and evaluated.

cs.CV↗

arXiv2023

Trajectory Tracking via Multiscale Continuous Attractor Networks

Animals and insects showcase remarkably robust and adept navigational abilities, up to literally circumnavigating the globe. Primary progress in robotics inspired by these natural systems has occurred in two areas: highly theoretical computational neuroscience models, and handcrafted systems like RatSLAM and NeuroSLAM. In this research, we present work bridging the gap between the two, in the form of Multiscale Continuous Attractor Networks (MCAN), that combine the multiscale parallel spatial neural networks of the previous theoretical models with the real-world robustness of the robot-targeted systems, to enable trajectory tracking over large velocity ranges. To overcome the limitations of the reliance of previous systems on hand-tuned parameters, we present a genetic algorithm-based approach for automated tuning of these networks, substantially improving their usability. To provide challenging navigational scale ranges, we open source a flexible city-scale navigation simulator that adapts to any street network, enabling high throughput experimentation. In extensive experiments using the city-scale navigation environment and Kitti, we show that the system is capable of stable dead reckoning over a wide range of velocities and environmental scales, where a single-scale approach fails.

cs.RO↗

arXiv2023

Ensembles of Compact, Region-specific & Regularized Spiking Neural Networks for Scalable Place Recognition

Spiking neural networks have significant potential utility in robotics due to their high energy efficiency on specialized hardware, but proof-of-concept implementations have not yet typically achieved competitive performance or capability with conventional approaches. In this paper, we tackle one of the key practical challenges of scalability by introducing a novel modular ensemble network approach, where compact, localized spiking networks each learn and are solely responsible for recognizing places in a local region of the environment only. This modular approach creates a highly scalable system. However, it comes with a high-performance cost where a lack of global regularization at deployment time leads to hyperactive neurons that erroneously respond to places outside their learned region. Our second contribution introduces a regularization approach that detects and removes these problematic hyperactive neurons during the initial environmental learning phase. We evaluate this new scalable modular system on benchmark localization datasets Nordland and Oxford RobotCar, with comparisons to standard techniques NetVLAD, DenseVLAD, and SAD, and a previous spiking neural network system. Our system substantially outperforms the previous SNN system on its small dataset, but also maintains performance on 27 times larger benchmark datasets where the operation of the previous system is computationally infeasible, and performs competitively with the conventional localization systems.

cs.CV↗

arXiv2023

OVTrack: Open-Vocabulary Multiple Object Tracking

The ability to recognize, localize and track dynamic objects in a scene is fundamental to many real-world applications, such as self-driving and robotic systems. Yet, traditional multiple object tracking (MOT) benchmarks rely only on a few object categories that hardly represent the multitude of possible objects that are encountered in the real world. This leaves contemporary MOT methods limited to a small set of pre-defined object categories. In this paper, we address this limitation by tackling a novel task, open-vocabulary MOT, that aims to evaluate tracking beyond pre-defined training categories. We further develop OVTrack, an open-vocabulary tracker that is capable of tracking arbitrary object classes. Its design is based on two key ingredients: First, leveraging vision-language models for both classification and association via knowledge distillation; second, a data hallucination strategy for robust appearance feature learning from denoising diffusion probabilistic models. The result is an extremely data-efficient open-vocabulary tracker that sets a new state-of-the-art on the large-scale, large-vocabulary TAO benchmark, while being trained solely on static images. Project page: https://www.vis.xyz/pub/ovtrack/

cs.CV↗

arXiv2023

Evolution of collisional neutrino flavor instabilities in spherically symmetric supernova models

We implement a multi-group and discrete-ordinate neutrino transport model in spherical symmetry which allows to simulate collective neutrino oscillations by including realistic collisional rates in a self-consistent way. We utilize this innovative model, based on strategic parameter rescaling, to study a recently proposed collisional flavor instability caused by the asymmetry of emission and absorption rates between $ν_e$ and $\barν_e$ for four different static backgrounds taken from different stages in a core-collapse supernova simulation. Our results confirm that collisional instabilities generally exist around the neutrinosphere during the SN accretion and post-accretion phase, as suggested by [arXiv:2104.11369]. However, the growth and transport of flavor instabilities can only be fully captured by models with global simulations as done in this work. With minimal ingredient to trigger collisional instabilities, we find that the flavor oscillations and transport mainly affect (anti)neutrinos of heavy lepton flavors around their decoupling sphere, which then leave imprints on their energy spectra in the free-streaming regime. For electron (anti)neutrinos, their properties remain nearly intact. We also explore various effects due to the decoherence from neutrino-nucleon scattering, artificially enhanced decoherence from emission and absorption, neutrino vacuum mixing, and inhomogeneous matter profile, and discuss the implication of our work.

astro-ph.HE↗

arXiv2022

CC-3DT: Panoramic 3D Object Tracking via Cross-Camera Fusion

To track the 3D locations and trajectories of the other traffic participants at any given time, modern autonomous vehicles are equipped with multiple cameras that cover the vehicle's full surroundings. Yet, camera-based 3D object tracking methods prioritize optimizing the single-camera setup and resort to post-hoc fusion in a multi-camera setup. In this paper, we propose a method for panoramic 3D object tracking, called CC-3DT, that associates and models object trajectories both temporally and across views, and improves the overall tracking consistency. In particular, our method fuses 3D detections from multiple cameras before association, reducing identity switches significantly and improving motion modeling. Our experiments on large-scale driving datasets show that fusion before association leads to a large margin of improvement over post-hoc fusion. We set a new state-of-the-art with 12.6% improvement in average multi-object tracking accuracy (AMOTA) among all camera-based methods on the competitive NuScenes 3D tracking benchmark, outperforming previously published methods by 6.5% in AMOTA with the same 3D detector.

cs.CV↗

arXiv2022

Early deconfinement of asymptotically conformal color-superconducting quark matter in neutron stars

We present a relativistic density functional approach to color superconducting quark matter that mimics quark confinement by a fast growth of the quasiparticle selfenergy in the confining region. The approach is shown to be equivalent to a chiral model of quark matter with medium dependent couplings. While the (pseudo)scalar sector of the model is fitted to the vacuum phenomenology of quantum chromodynamics, the strength of interaction in the vector and diquark channels is varied in order to provide the best agreement with the observational constraints on the mass-radius relation and tidal deformability of neutron stars modelled with our approach. In order to recover the conformal behavior of quark matter at asymptotically high densities we introduce a medium dependence of the vector and diquark couplings motivated by the nonperturbative gluon exchange. Our analysis signals that the onset of deconfinement to color superconducting quark matter is likely to occur in neutron stars with masses below 1.0 $M_\odot$.

nucl-th↗

arXiv2022

How Many Events do You Need? Event-based Visual Place Recognition Using Sparse But Varying Pixels

Event cameras continue to attract interest due to desirable characteristics such as high dynamic range, low latency, virtually no motion blur, and high energy efficiency. One of the potential applications that would benefit from these characteristics lies in visual place recognition for robot localization, i.e. matching a query observation to the corresponding reference place in the database. In this letter, we explore the distinctiveness of event streams from a small subset of pixels (in the tens or hundreds). We demonstrate that the absolute difference in the number of events at those pixel locations accumulated into event frames can be sufficient for the place recognition task, when pixels that display large variations in the reference set are used. Using such sparse (over image coordinates) but varying (variance over the number of events per pixel location) pixels enables frequent and computationally cheap updates of the location estimates. Furthermore, when event frames contain a constant number of events, our method takes full advantage of the event-driven nature of the sensory stream and displays promising robustness to changes in velocity. We evaluate our proposed approach on the Brisbane-Event-VPR dataset in an outdoor driving scenario, as well as the newly contributed indoor QCR-Event-VPR dataset that was captured with a DAVIS346 camera mounted on a mobile robotic platform. Our results show that our approach achieves competitive performance when compared to several baseline methods on those datasets, and is particularly well suited for compute- and energy-constrained platforms such as interplanetary rovers.

cs.CV↗

arXiv2022

Boosting Performance of a Baseline Visual Place Recognition Technique by Predicting the Maximally Complementary Technique

One recent promising approach to the Visual Place Recognition (VPR) problem has been to fuse the place recognition estimates of multiple complementary VPR techniques using methods such as SRAL and multi-process fusion. These approaches come with a substantial practical limitation: they require all potential VPR methods to be brute-force run before they are selectively fused. The obvious solution to this limitation is to predict the viable subset of methods ahead of time, but this is challenging because it requires a predictive signal within the imagery itself that is indicative of high performance methods. Here we propose an alternative approach that instead starts with a known single base VPR technique, and learns to predict the most complementary additional VPR technique to fuse with it, that results in the largest improvement in performance. The key innovation here is to use a dimensionally reduced difference vector between the query image and the top-retrieved reference image using this baseline technique as the predictive signal of the most complementary additional technique, both during training and inference. We demonstrate that our approach can train a single network to select performant, complementary technique pairs across datasets which span multiple modes of transportation (train, car, walking) as well as to generalise to unseen datasets, outperforming multiple baseline strategies for manually selecting the best technique pairs based on the same training data.

cs.CV↗

arXiv2022

Hyperdimensional Feature Fusion for Out-Of-Distribution Detection

We introduce powerful ideas from Hyperdimensional Computing into the challenging field of Out-of-Distribution (OOD) detection. In contrast to most existing work that performs OOD detection based on only a single layer of a neural network, we use similarity-preserving semi-orthogonal projection matrices to project the feature maps from multiple layers into a common vector space. By repeatedly applying the bundling operation $\oplus$, we create expressive class-specific descriptor vectors for all in-distribution classes. At test time, a simple and efficient cosine similarity calculation between descriptor vectors consistently identifies OOD samples with better performance than the current state-of-the-art. We show that the hyperdimensional fusion of multiple network layers is critical to achieve best general performance.

cs.CV↗

arXiv2022

Early quark deconfinement in compact star astrophysics and heavy-ion collisions

Based on a recently developed relativistic density functional approach to color-superconducting quark matter and a novel quark-hadron transition construction which phenomenologically accounts for the effects of inhomogeneous pasta phases and quark-hadron continuity, we construct a class of hybrid equations of state applicable at the regimes typical for compact star astrophysics and heavy ion collisions. We outline that early quark deconfinement is a notable consequence of strong diquark pairing providing a good agreement with the observational data and driving the trajectories of the matter evolution during the supernovae explosions toward the regimes typical for the compact star mergers and heavy-ion collisions.

nucl-th↗

arXiv2022

Universal relations for rapidly rotating cold and hot hybrid star

Several global parameters of compact stars are related via empirical relations, which are (nearly) independent of the underlying equation of state (EoS) of dense matter and, therefore, are said to be universal. We investigate the universality of relations that express the maximum mass and the radius of non-rotating and maximally rapidly rotating configurations, as well as their moment of inertia, in terms of the compactness of the star. For this, we first utilize a collection of cold (zero-temperature) and hot (isentropic) nucleonic EoS and confirm that the universal relations are holding for our collection of EoS. We then go on, to add to our collection and test for the same universality models of EoS that admit a strong first-order phase transition from nucleonic to deconfined quark matter. Also in this case we find that the universal relations hold, in particular for hot, isentropic hybrid stars. By fitting the universal relations to our computed data, we determine the coefficients entering these relations and the accuracy to which they hold.

astro-ph.HE↗

arXiv2022

Point Label Aware Superpixels for Multi-species Segmentation of Underwater Imagery

Monitoring coral reefs using underwater vehicles increases the range of marine surveys and availability of historical ecological data by collecting significant quantities of images. Analysis of this imagery can be automated using a model trained to perform semantic segmentation, however it is too costly and time-consuming to densely label images for training supervised models. In this letter, we leverage photo-quadrat imagery labeled by ecologists with sparse point labels. We propose a point label aware method for propagating labels within superpixel regions to obtain augmented ground truth for training a semantic segmentation model. Our point label aware superpixel method utilizes the sparse point labels, and clusters pixels using learned features to accurately generate single-species segments in cluttered, complex coral images. Our method outperforms prior methods on the UCSD Mosaics dataset by 3.62% for pixel accuracy and 8.35% for mean IoU for the label propagation task, while reducing computation time reported by previous approaches by 76%. We train a DeepLabv3+ architecture and outperform state-of-the-art for semantic segmentation by 2.91% for pixel accuracy and 9.65% for mean IoU on the UCSD Mosaics dataset and by 4.19% for pixel accuracy and 14.32% mean IoU for the Eilat dataset.

cs.CV↗

arXiv2022

Effects of a strong phase transition on supernova explosions, compact stars and their mergers

We outline a theoretical approach supporting strong phase transitions from normal nuclear matter to the deconfined quark-gluon plasma, in the equation of state (EOS) for compact star matter. Implications of this hypothesis are discussed for astrophysical applications. Special emphasis is devoted to potentially detectable signatures, which can be directly related with the occurrence of a sufficiently strong phase transition. Therefore, simulations of core-collapse supernovae and binary compact star mergers are considered, including the subsequent emission of gravitational waves and, in the case of supernova, in addition the neutrinos play the role of messengers.

nucl-th↗

arXiv2022

3D template-based $Fermi$-LAT constraints on the diffuse supernova axion-like particle background

Axion-like particles (ALPs) may be abundantly produced in core-collapse (CC) supernovae (SNe), hence the cumulative signal from all past SN events can create a diffuse flux peaked at energies of about 25~MeV. We improve upon the modeling of the ALPs flux by including a set of CC SN models with different progenitor masses, as well as the effects of failed CC SNe -- which yield the formation of black holes instead of explosions. Relying on the coupling strength of ALPs to photons and the related Primakoff process, the diffuse SN ALP flux is converted into gamma rays while traversing the magnetic field of the Milky Way. The spatial morphology of this signal is expected to follow the shape of the Galactic magnetic field lines. We make use of this via a template-based analysis that utilizes 12 years of $Fermi$-LAT data in the energy range from 50 MeV to 500 GeV. In our benchmark case of the realization of astrophysical and cosmological parameters, we find an upper limit of $g_{aγ} \lesssim 3.76\times10^{-11}\;\mathrm{GeV}^{-1}$ at 95$\%$ confidence level for $m_a \ll 10^{-11}$ eV, while we find that systematic deviations from this benchmark scenario induce an uncertainty as large as about a factor of two. Our result slightly improves the CAST bound, while still being a factor of six (baseline scenario) weaker than the SN1987A gamma-ray burst limit.

astro-ph.HE↗