arXiv subjects

Nuno Gracias

Publications and source records attributed to Nuno Gracias.

At least 19 recordsLinked to original sources

arXiv2026

Multi-Session Multimodal Underwater Mapping with Acoustic and Optical Imaging

Accurate seafloor mapping is essential for marine science, archaeology, and environmental monitoring. However, integrating data from different sensors, such as side-scan sonar and optical cameras, collected across separate survey sessions, remains challenging due to positioning drift and sensor offsets. This paper presents a multi-session, multimodal underwater mapping framework based on factor graph optimization. The method jointly optimizes vehicle trajectories, 3D landmark positions, sensor extrinsics, and per-session global alignment transformations. By combining rigid inter-session corrections with local trajectory deformations, it compensates for both inter-session offsets and intra-session distortions from accumulated navigation errors. The proposed methodology was validated on real-world datasets collected along the Catalan coast. Results show measurable improvements in map consistency over both unoptimized and rigid-alignment baselines across all metrics, including Pixel Accuracy and mean Intersection over Union. The method achieves a 3.4% improvement in pixel accuracy over the unoptimized baseline, corresponding to improved semantic labelling across approximately 14700 $\text{m}^2$ of mapped area. Qualitative results further show consistent co-registration between sonar and optical maps, even in the presence of significant trajectory distortions and inter-session misalignments. These findings demonstrate the potential of the proposed framework to generate coherent multimodal seafloor maps from heterogeneous underwater surveys.

cs.RO↗

arXiv2026

Multi-Robot Scanner for Automated Full-Body Dermoscopic Imaging

This paper outlines the specifications and design approach used to construct a full body imaging scanner capable of capturing skin lesions at a dermatoscopic level using cameras mounted on the end-effectors of four UR10 manipulators. The system possesses a view-planning algorithm capable of appropriately selecting the best camera position to acquire images of moles, a high-level controller to allow the manipulators to work simultaneously and a collision-detector that halts the manipulators when they make contact with an object or a person. We evaluate the system through real-patient full-body scans, comparing acquired images against contact dermoscopy and an existing total-body photography system (Vectra) across clinically relevant lesion features, and quantify true optical resolving power using a USAF 1951 resolution target, yielding a smallest resolvable feature size of 22.1 microns for our scanner compared to 8.8 microns for contact dermoscopy. Results show the scanner consistently outperforms Vectra across most clinically relevant features and achieves comparable performance to contact dermoscopy for the majority of features assessed. By acquiring dermatoscopic-quality images automatically and without contact, and without requiring a separate manual dermoscopic examination, the scanner closes part of the gap between total-body photography and handheld dermoscopy, suggesting potential for future integration into screening workflows.

cs.RO↗

arXiv2026

Advancing Accessible Underwater Robotics: The Mini-Girona I-AUV at RAMI 2025

The Mini-Girona Intervention Autonomous Underwater Vehicle (I-AUV) represents an advancement in accessible underwater robotics, designed to bridge the gap between costly, specialized research AUVs and basic Remotely Operated Vehicles (ROVs). Developed with a focus on affordability and usability, the Mini-Girona, priced at approximately $50,000, integrates advanced components such as a 5-DOF manipulator arm, stereo vision, and AI-driven processing for autonomous navigation and intervention tasks. This paper presents the design and development of the Mini-Girona, detailing its performance during the RAMI 2025 student competition. Despite challenges such as thermal management issues and restricted team access, the Mini-Girona achieved second place overall, excelling in vision-based perception and intervention tasks. This work highlights the platform's potential as a tool for underwater robotics research and education, fostering innovation in real-world underwater applications.

cs.RO↗

arXiv2026

Weakly Supervised Seafloor Segmentation for Seagrass Habitat Mapping in Side-Scan Sonar Imagery

Seagrass meadows are crucial blue-carbon habitats, and mapping their extent is a prerequisite for coastal management and carbon inventory. Optical satellite sensors cover large areas but cannot reach deep or turbid water, whereas side-scan sonar (SSS) images the seabed at high resolution and at any depth. Interpreting SSS, however, still relies on dense manual annotation, which is slow and costly. We address this by adapting a weakly supervised semantic segmentation framework to SSS benthic habitat mapping, so that pixel-level maps are learned from image-level labels alone. The framework couples a ViT-based encoder-decoder with a classification branch, extracts class activation maps, and refines them into pseudo-labels with a dense conditional random field that we tune for the noise and weak boundaries of acoustic imagery. It follows an iterative self-training scheme, together with a sampling strategy to cope with the strong class imbalance of the data. We also study the effect of different loss functions on segmentation quality, finding Lovász-Softmax loss the most effective. On a held-out transect, the refined pseudo-labels reached an mIoU of 89.3\% against the ground truth, and the segmentation branch, trained without any pixel-level labels, reached 87.6\%. Self-supervised pretraining on unlabelled SSS added a further 3\% in mean intersection-over-union. Field trials further demonstrate the generalizability of the trained model. These results show that accurate and label-efficient benthic habitat mapping from side-scan sonar is feasible at the scale needed for coast-wide seagrass monitoring.

cs.CV↗

arXiv2026

BenthicDINO: Physics-Informed Self-Distillation for View-Invariant Side-Scan Sonar Representations

Automated perception in side-scan sonar (SSS) imagery is severely hindered by physical acoustic artifacts, resulting in representations that inextricably mix intrinsic seabed reflectivity with transient viewing geometries. Existing self-supervised learning (SSL) frameworks rely on augmentations designed for natural images, failing to account for acoustic degradation and explicitly enforce view-invariance. To address this gap, we introduce a physics-informed self-distillation framework built upon the DINOv3 architecture utilizing a ConvNeXt-v2-Tiny backbone to maximize data efficiency. The proposed methodology enforces view-invariance through two primary mechanisms: physically motivated augmentations that simulate speckle noise, range-dependent attenuation, and radiometric miscalibration; and a Hilbert-Schmidt Independence Criterion (HSIC) penalty that explicitly decouples learned dense patch features from physical viewing parameters. Furthermore, we propose a dense, hierarchical feature fusion strategy across all four network stages to preserve fine-grained sediment details alongside deep semantic abstractions. Extensive evaluation demonstrates that the framework natively groups complex benthic topographies into stable, noise-free semantic clusters without relying on manual annotations. During supervised downstream tasks on the S3Seg dataset, the fused representations exhibited exceptional data efficiency, achieving 96% of its absolute peak performance using only 10% of the available annotated data, ultimately reaching a mean Intersection over Union (mIoU) of 71.4% and an overall accuracy of 86.5%.

cs.CV↗

arXiv2026

Geometry-Driven Opti-Acoustic Co-Registration and View-Invariant Reflectivity Mapping for Side-Scan Sonar

Side-Scan Sonar (SSS) is a primary modality for large-scale underwater mapping, yet automated perception and cross-modal alignment are severely bottlenecked by acoustic complexities such as speckle noise, shadows, and extreme viewpoint dependencies. Traditional handcrafted descriptors and modern deep learning matchers fail to bridge the physical domain gap between optical and acoustic imagery without 3D geometric constraints. To overcome these limitations, we propose a novel geometry-driven framework for pixel-level opti-acoustic co-registration and view-invariant reflectivity mapping. Our method utilizes Structure-from-Motion (SfM) to reconstruct a dense 3D seafloor mesh, acting as a geometric anchor between the visual and acoustic domains. We introduce a First Bottom Return (FBR) extraction algorithm to dynamically correct non-linear altitude drift caused by uncalibrated SfM reconstruction. Furthermore, we apply an inverse Lambertian model and a dual-Gaussian weighting function to isolate the intrinsic seabed reflectivity, effectively neutralizing slant-range propagation loss and geometric view-dependence. By deterministically associating these isolated acoustic properties with optical pixels, our pipeline generates highly accurate, strictly co-registered multi-modal datasets. This automated, physics-guided approach eliminates the need for manual annotation and paves the way for advanced self-supervised learning in benthic habitat mapping.

cs.CV↗

arXiv2026

SC-Match: Scale-Space Matching with Context Consistency for Side-Scan Sonar Mapping

Reliable estimation of spatial correspondences between overlapping side-scan sonar (SSS) measurements is essential for mapping, but acoustic appearance variations, weak seabed texture, repetitive patterns, and shadows make such correspondences sparse, unstable, and context-dependent. Scarce point-level annotations further limit sonar-specific training or fine-tuning of deep matching models. To this end, we propose SC-Match, a training-free scale-space matching framework with context-consistent correspondence refinement that adapts pretrained feature extraction and matching components to SSS observations without sonar-specific retraining. The framework improves correspondence reliability through scale-space representation and context-consistent refinement. For feature representation, a frozen extractor is applied to multiple observation scales, detector responses are calibrated according to the structure--texture tendency of the input SSS image, and score-aware cross-scale fusion is used to retain compact feature candidates. For correspondence refinement, adjacent local matching cases are used as neighboring contexts to verify stable fixed--moving relations and preserve non-conflicting complementary matches for alignment. Experiments on datasets acquired in different environments using different sonar platforms show that SC-Match provides more accurate correspondences and more consistent geometric alignment than representative pretrained methods, while maintaining stable behavior under unseen cross-platform acquisition conditions.

physics.ao-ph↗

arXiv2026

BenthiCat: An opti-acoustic dataset for advancing benthic classification and habitat mapping

Benthic habitat mapping is fundamental for understanding marine ecosystems, guiding conservation efforts, and supporting sustainable resource management. Yet, the scarcity of large, annotated datasets limits the development and benchmarking of machine learning models in this domain. This paper introduces a thorough multi-modal dataset, comprising about a million side-scan sonar (SSS) tiles collected along the coast of Catalonia (Spain), complemented by bathymetric maps and a set of co-registered optical images from targeted surveys using an autonomous underwater vehicle (AUV). Approximately 36000 of the SSS tiles have been manually annotated with segmentation masks to enable supervised fine-tuning of classification models. All the raw sensor data, together with mosaics, are also released to support further exploration and algorithm development. To address challenges in multi-sensor data fusion for AUVs, we spatially associate optical images with corresponding SSS tiles, facilitating self-supervised, cross-modal representation learning. Accompanying open-source preprocessing and annotation tools are provided to enhance accessibility and encourage research. This resource aims to establish a standardized benchmark for underwater habitat mapping, promoting advancements in autonomous seafloor classification and multi-sensor integration.

cs.CV↗

arXiv2026

Geometric Correction of Side-Scan Sonar Images with Image-Consistent Attitude Refinement

Side-scan sonar (SSS) images are susceptible to motion-induced geometric distortion, which degrades their reliability for seabed interpretation and downstream tasks. Existing correction methods either exploit image-domain consistency without adequately preserving global geometric referencing, or rely on navigation-based geocoding whose effectiveness is limited when recorded attitude and motion fail to capture ping-scale perturbations. To address this issue, we propose a geometric correction method for SSS images with image-consistent attitude refinement. The core idea is to refine the yaw-pitch sequence used in geocoding by explicitly linking stripe-wise distortion patterns in dual-sided waterfall images to geometric deformation modes. Specifically, a navigation-derived macro-scale attitude baseline is fused with image-inferred microscopic perturbations, where port-starboard symmetry is used to separate pitch-related common-mode responses from yaw-related differential-mode responses. The refined attitude is then incorporated into a physically geocoding framework with track-aligned gridding and normalized-convolution-based hole completion to generate the corrected image. Experiments on real SSS datasets from different sonar platforms and environments show that the proposed method reduces inter-ping misalignment, local stretching, and structural discontinuity, and improves local geometric consistency under both degraded-attitude and cross-dataset evaluation settings, demonstrating its effectiveness for geometrically consistent SSS correction.

physics.ao-ph↗

arXiv2026

Physically Motivated Knowledge Distillation for Blind Geometric Correction of Side-Scan Sonar Imagery

Side-scan sonar (SSS) imagery is susceptible to geometric distortions caused by platform motion instability, which degrade geometric consistency and limit downstream analyses such as mosaicking and perception. Conventional correction methods typically rely on navigation and attitude measurements, which are often unreliable in real ocean conditions. This unreliability necessitates blind geometric correction from a single distorted image, a highly ill-posed problem. To address this issue, we propose a physically motivated knowledge distillation framework for blind geometric correction of SSS imagery. Specifically, a teacher network is trained using paired distorted and geocoded reference images to learn distortion-related geometric differences, and this knowledge is transferred to a student network that performs correction using only a single distorted image during blind inference. To ensure physically plausible deformation estimation, we design a parametric decoder that represents distortions as row-wise affine transformations consistent with the SSS line-scanning imaging mechanism. To compensate for the absence of reference information during blind inference, a hallucination context module is introduced to approximate the teachers geometric reasoning from distorted features under a multi-level distillation scheme. In addition, a differentiable forward warping strategy is adopted to handle the non-bijective deformation characteristics of SSS imagery in an end-to-end manner. Extensive experiments on multiple datasets show that the proposed method outperforms state-of-the-art baselines and generalizes well across different platforms and acquisition conditions.

physics.ao-ph↗

arXiv2026

eStonefish-Scenes: A Sim-to-Real Validated and Robot-Centric Event-based Optical Flow Dataset for Underwater Vehicles

Event-based cameras (EBCs) are poised to transform underwater robotics, yet the absence of labelled event-based datasets for underwater environments severely limits progress in tasks such as visual odometry and obstacle avoidance. Real-world event-based optical flow datasets are scarce, resource-intensive to collect, and lack diversity, while no prior benchmarks target underwater applications. To bridge this gap, we introduce eStonefish-Scenes, a synthetic event-based optical flow dataset generated using the Stonefish simulator, together with an open data generation pipeline for creating customizable underwater environments featuring realistic coral reefs and biologically inspired schools of fish with reactive navigation behaviours. We also present eWiz, a comprehensive library for event-based data processing, encompassing data loading, augmentation, visualization, encoding, training utilities, loss functions, and evaluation metrics. To validate sim-to-real transferability, we collected real-world data using a DAVIS346 hybrid event-and-frame camera mounted on a BlueROV2 in an indoor testing pool. Ground-truth optical flow was derived via homography-based frame-to-poster registration, and per-pixel uncertainty was estimated through Monte Carlo perturbation of keypoint correspondences. This uncertainty was incorporated into the evaluation metrics, enabling reliability-aware performance assessment. A ConvGRU-based optical flow network, trained exclusively on synthetic eStonefish-Scenes data, was evaluated on the real-world sequences without fine-tuning, achieving an uncertainty-weighted average endpoint error of 0.79 pixels. These results demonstrate that the proposed synthetic dataset effectively supports sim-to-real transfer for underwater event-based optical flow estimation, substantially reducing the need for costly real-world data collection.

cs.CV↗

arXiv2025

PhysDNet: Physics-Guided Decomposition Network of Side-Scan Sonar Imagery

Side-scan sonar (SSS) imagery is widely used for seafloor mapping and underwater remote sensing, yet the measured intensity is strongly influenced by seabed reflectivity, terrain elevation, and acoustic path loss. This entanglement makes the imagery highly view-dependent and reduces the robustness of downstream analysis. In this letter, we present PhysDNet, a physics-guided multi-branch network that decouples SSS images into three interpretable fields: seabed reflectivity, terrain elevation, and propagation loss. By embedding the Lambertian reflection model, PhysDNet reconstructs sonar intensity from these components, enabling self-supervised training without ground-truth annotations. Experiments show that the decomposed representations preserve stable geological structures, capture physically consistent illumination and attenuation, and produce reliable shadow maps. These findings demonstrate that physics-guided decomposition provides a stable and interpretable domain for SSS analysis, improving both physical consistency and downstream tasks such as registration and shadow interpretation.

physics.ao-ph↗

arXiv2025

A Geometrically Consistent Matching Framework for Side-Scan Sonar Mapping

Robust matching of side-scan sonar imagery remains a fundamental challenge in seafloor mapping due to view-dependent backscatter, shadows, and geometric distortion. This paper proposes a novel matching framework that combines physical decoupling and geometric consistency to enhance correspondence accuracy and consistency across viewpoints. A multi-branch network, derived from the Lambertian reflection model, decomposes raw sonar images into seabed reflectivity, terrain elevation, and acoustic path loss. The reflectivity map, serving as a stable matching domain, is used in conjunction with a training-free matching pipeline combining SuperPoint and MINIMA-LightGlue. Geometry-aware outlier rejection leverages both terrain elevation and its physically derived shadow map to further remove mismatches in acoustically occluded and topographically inconsistent regions, thereby improving registration accuracy. Quantitative and visual evaluations against traditional, CNN-, and Transformer-based state-of-the-art methods demonstrate that our method achieves lower matching error, higher geometric consistency, and greater robustness to viewpoint variations. The proposed approach provides a data-efficient, physically interpretable solution for high-precision side-scan sonar image matching in complex seafloor environments.

physics.ins-det↗

arXiv2025

Coarse-to-Fine Non-Rigid Registration for Side-Scan Sonar Mosaicking

Side-scan sonar mosaicking plays a crucial role in large-scale seabed mapping but is challenged by complex non-linear, spatially varying distortions due to diverse sonar acquisition conditions. Existing rigid or affine registration methods fail to model such complex deformations, whereas traditional non-rigid techniques tend to overfit and lack robustness in sparse-texture sonar data. To address these challenges, we propose a coarse-to-fine hierarchical non-rigid registration framework tailored for large-scale side-scan sonar images. Our method begins with a global Thin Plate Spline initialization from sparse correspondences, followed by superpixel-guided segmentation that partitions the image into structurally consistent patches preserving terrain integrity. Each patch is then refined by a pretrained SynthMorph network in an unsupervised manner, enabling dense and flexible alignment without task-specific training. Finally, a fusion strategy integrates both global and local deformations into a smooth, unified deformation field. Extensive quantitative and visual evaluations demonstrate that our approach significantly outperforms state-of-the-art rigid, classical non-rigid, and learning-based methods in accuracy, structural consistency, and deformation smoothness on the challenging sonar dataset.

physics.geo-ph↗

arXiv2025

Stonefish: Supporting Machine Learning Research in Marine Robotics

Simulations are highly valuable in marine robotics, offering a cost-effective and controlled environment for testing in the challenging conditions of underwater and surface operations. Given the high costs and logistical difficulties of real-world trials, simulators capable of capturing the operational conditions of subsea environments have become key in developing and refining algorithms for remotely-operated and autonomous underwater vehicles. This paper highlights recent enhancements to the Stonefish simulator, an advanced open-source platform supporting development and testing of marine robotics solutions. Key updates include a suite of additional sensors, such as an event-based camera, a thermal camera, and an optical flow camera, as well as, visual light communication, support for tethered operations, improved thruster modelling, more flexible hydrodynamics, and enhanced sonar accuracy. These developments and an automated annotation tool significantly bolster Stonefish's role in marine robotics research, especially in the field of machine learning, where training data with a known ground truth is hard or impossible to collect.

cs.RO↗

arXiv2025

Real-time Seafloor Segmentation and Mapping

Posidonia oceanica meadows are a species of seagrass highly dependent on rocks for their survival and conservation. In recent years, there has been a concerning global decline in this species, emphasizing the critical need for efficient monitoring and assessment tools. While deep learning-based semantic segmentation and visual automated monitoring systems have shown promise in a variety of applications, their performance in underwater environments remains challenging due to complex water conditions and limited datasets. This paper introduces a framework that combines machine learning and computer vision techniques to enable an autonomous underwater vehicle (AUV) to inspect the boundaries of Posidonia oceanica meadows autonomously. The framework incorporates an image segmentation module using an existing Mask R-CNN model and a strategy for Posidonia oceanica meadow boundary tracking. Furthermore, a new class dedicated to rocks is introduced to enhance the existing model, aiming to contribute to a comprehensive monitoring approach and provide a deeper understanding of the intricate interactions between the meadow and its surrounding environment. The image segmentation model is validated using real underwater images, while the overall inspection framework is evaluated in a realistic simulation environment, replicating actual monitoring scenarios with real underwater images. The results demonstrate that the proposed framework enables the AUV to autonomously accomplish the main tasks of underwater inspection and segmentation of rocks. Consequently, this work holds significant potential for the conservation and protection of marine environments, providing valuable insights into the status of Posidonia oceanica meadows and supporting targeted preservation efforts

cs.CV↗

arXiv2024

eCARLA-scenes: A synthetically generated dataset for event-based optical flow prediction

The joint use of event-based vision and Spiking Neural Networks (SNNs) is expected to have a large impact in robotics in the near future, in tasks such as, visual odometry and obstacle avoidance. While researchers have used real-world event datasets for optical flow prediction (mostly captured with Unmanned Aerial Vehicles (UAVs)), these datasets are limited in diversity, scalability, and are challenging to collect. Thus, synthetic datasets offer a scalable alternative by bridging the gap between reality and simulation. In this work, we address the lack of datasets by introducing eWiz, a comprehensive library for processing event-based data. It includes tools for data loading, augmentation, visualization, encoding, and generation of training data, along with loss functions and performance metrics. We further present a synthetic event-based datasets and data generation pipelines for optical flow prediction tasks. Built on top of eWiz, eCARLA-scenes makes use of the CARLA simulator to simulate self-driving car scenarios. The ultimate goal of this dataset is the depiction of diverse environments while laying a foundation for advancing event-based camera applications in autonomous field vehicle navigation, paving the way for using SNNs on neuromorphic hardware such as the Intel Loihi.

cs.CV↗

arXiv2023

A Convolutional Vision Transformer for Semantic Segmentation of Side-Scan Sonar Data

Distinguishing among different marine benthic habitat characteristics is of key importance in a wide set of seabed operations ranging from installations of oil rigs to laying networks of cables and monitoring the impact of humans on marine ecosystems. The Side-Scan Sonar (SSS) is a widely used imaging sensor in this regard. It produces high-resolution seafloor maps by logging the intensities of sound waves reflected back from the seafloor. In this work, we leverage these acoustic intensity maps to produce pixel-wise categorization of different seafloor types. We propose a novel architecture adapted from the Vision Transformer (ViT) in an encoder-decoder framework. Further, in doing so, the applicability of ViTs is evaluated on smaller datasets. To overcome the lack of CNN-like inductive biases, thereby making ViTs more conducive to applications in low data regimes, we propose a novel feature extraction module to replace the Multi-layer Perceptron (MLP) block within transformer layers and a novel module to extract multiscale patch embeddings. A lightweight decoder is also proposed to complement this design in order to further boost multiscale feature extraction. With the modified architecture, we achieve state-of-the-art results and also meet real-time computational requirements. We make our code available at ~\url{https://github.com/hayatrajani/s3seg-vit

cs.CV↗