arXiv subjects

Hayat Rajani

Publications and source records attributed to Hayat Rajani.

15 recordsLinked to original sources

arXiv2026

Weakly Supervised Seafloor Segmentation for Seagrass Habitat Mapping in Side-Scan Sonar Imagery

Seagrass meadows are crucial blue-carbon habitats, and mapping their extent is a prerequisite for coastal management and carbon inventory. Optical satellite sensors cover large areas but cannot reach deep or turbid water, whereas side-scan sonar (SSS) images the seabed at high resolution and at any depth. Interpreting SSS, however, still relies on dense manual annotation, which is slow and costly. We address this by adapting a weakly supervised semantic segmentation framework to SSS benthic habitat mapping, so that pixel-level maps are learned from image-level labels alone. The framework couples a ViT-based encoder-decoder with a classification branch, extracts class activation maps, and refines them into pseudo-labels with a dense conditional random field that we tune for the noise and weak boundaries of acoustic imagery. It follows an iterative self-training scheme, together with a sampling strategy to cope with the strong class imbalance of the data. We also study the effect of different loss functions on segmentation quality, finding Lov\'asz-Softmax loss the most effective. On a held-out transect, the refined pseudo-labels reached an mIoU of 89.3\% against the ground truth, and the segmentation branch, trained without any pixel-level labels, reached 87.6\%. Self-supervised pretraining on unlabelled SSS added a further 3\% in mean intersection-over-union. Field trials further demonstrate the generalizability of the trained model. These results show that accurate and label-efficient benthic habitat mapping from side-scan sonar is feasible at the scale needed for coast-wide seagrass monitoring.

cs.CV↗

arXiv2026

BenthicDINO: Physics-Informed Self-Distillation for View-Invariant Side-Scan Sonar Representations

Automated perception in side-scan sonar (SSS) imagery is severely hindered by physical acoustic artifacts, resulting in representations that inextricably mix intrinsic seabed reflectivity with transient viewing geometries. Existing self-supervised learning (SSL) frameworks rely on augmentations designed for natural images, failing to account for acoustic degradation and explicitly enforce view-invariance. To address this gap, we introduce a physics-informed self-distillation framework built upon the DINOv3 architecture utilizing a ConvNeXt-v2-Tiny backbone to maximize data efficiency. The proposed methodology enforces view-invariance through two primary mechanisms: physically motivated augmentations that simulate speckle noise, range-dependent attenuation, and radiometric miscalibration; and a Hilbert-Schmidt Independence Criterion (HSIC) penalty that explicitly decouples learned dense patch features from physical viewing parameters. Furthermore, we propose a dense, hierarchical feature fusion strategy across all four network stages to preserve fine-grained sediment details alongside deep semantic abstractions. Extensive evaluation demonstrates that the framework natively groups complex benthic topographies into stable, noise-free semantic clusters without relying on manual annotations. During supervised downstream tasks on the S3Seg dataset, the fused representations exhibited exceptional data efficiency, achieving 96% of its absolute peak performance using only 10% of the available annotated data, ultimately reaching a mean Intersection over Union (mIoU) of 71.4% and an overall accuracy of 86.5%.

cs.CV↗

arXiv2026

SC-Match: Scale-Space Matching with Context Consistency for Side-Scan Sonar Mapping

Reliable estimation of spatial correspondences between overlapping side-scan sonar (SSS) measurements is essential for mapping, but acoustic appearance variations, weak seabed texture, repetitive patterns, and shadows make such correspondences sparse, unstable, and context-dependent. Scarce point-level annotations further limit sonar-specific training or fine-tuning of deep matching models. To this end, we propose SC-Match, a training-free scale-space matching framework with context-consistent correspondence refinement that adapts pretrained feature extraction and matching components to SSS observations without sonar-specific retraining. The framework improves correspondence reliability through scale-space representation and context-consistent refinement. For feature representation, a frozen extractor is applied to multiple observation scales, detector responses are calibrated according to the structure--texture tendency of the input SSS image, and score-aware cross-scale fusion is used to retain compact feature candidates. For correspondence refinement, adjacent local matching cases are used as neighboring contexts to verify stable fixed--moving relations and preserve non-conflicting complementary matches for alignment. Experiments on datasets acquired in different environments using different sonar platforms show that SC-Match provides more accurate correspondences and more consistent geometric alignment than representative pretrained methods, while maintaining stable behavior under unseen cross-platform acquisition conditions.

physics.ao-ph↗

arXiv2026

Geometric Correction of Side-Scan Sonar Images with Image-Consistent Attitude Refinement

Side-scan sonar (SSS) images are susceptible to motion-induced geometric distortion, which degrades their reliability for seabed interpretation and downstream tasks. Existing correction methods either exploit image-domain consistency without adequately preserving global geometric referencing, or rely on navigation-based geocoding whose effectiveness is limited when recorded attitude and motion fail to capture ping-scale perturbations. To address this issue, we propose a geometric correction method for SSS images with image-consistent attitude refinement. The core idea is to refine the yaw-pitch sequence used in geocoding by explicitly linking stripe-wise distortion patterns in dual-sided waterfall images to geometric deformation modes. Specifically, a navigation-derived macro-scale attitude baseline is fused with image-inferred microscopic perturbations, where port-starboard symmetry is used to separate pitch-related common-mode responses from yaw-related differential-mode responses. The refined attitude is then incorporated into a physically geocoding framework with track-aligned gridding and normalized-convolution-based hole completion to generate the corrected image. Experiments on real SSS datasets from different sonar platforms and environments show that the proposed method reduces inter-ping misalignment, local stretching, and structural discontinuity, and improves local geometric consistency under both degraded-attitude and cross-dataset evaluation settings, demonstrating its effectiveness for geometrically consistent SSS correction.

physics.ao-ph↗

arXiv2026

Physically Motivated Knowledge Distillation for Blind Geometric Correction of Side-Scan Sonar Imagery

Side-scan sonar (SSS) imagery is susceptible to geometric distortions caused by platform motion instability, which degrade geometric consistency and limit downstream analyses such as mosaicking and perception. Conventional correction methods typically rely on navigation and attitude measurements, which are often unreliable in real ocean conditions. This unreliability necessitates blind geometric correction from a single distorted image, a highly ill-posed problem. To address this issue, we propose a physically motivated knowledge distillation framework for blind geometric correction of SSS imagery. Specifically, a teacher network is trained using paired distorted and geocoded reference images to learn distortion-related geometric differences, and this knowledge is transferred to a student network that performs correction using only a single distorted image during blind inference. To ensure physically plausible deformation estimation, we design a parametric decoder that represents distortions as row-wise affine transformations consistent with the SSS line-scanning imaging mechanism. To compensate for the absence of reference information during blind inference, a hallucination context module is introduced to approximate the teachers geometric reasoning from distorted features under a multi-level distillation scheme. In addition, a differentiable forward warping strategy is adopted to handle the non-bijective deformation characteristics of SSS imagery in an end-to-end manner. Extensive experiments on multiple datasets show that the proposed method outperforms state-of-the-art baselines and generalizes well across different platforms and acquisition conditions.

physics.ao-ph↗

arXiv2025

PhysDNet: Physics-Guided Decomposition Network of Side-Scan Sonar Imagery

Side-scan sonar (SSS) imagery is widely used for seafloor mapping and underwater remote sensing, yet the measured intensity is strongly influenced by seabed reflectivity, terrain elevation, and acoustic path loss. This entanglement makes the imagery highly view-dependent and reduces the robustness of downstream analysis. In this letter, we present PhysDNet, a physics-guided multi-branch network that decouples SSS images into three interpretable fields: seabed reflectivity, terrain elevation, and propagation loss. By embedding the Lambertian reflection model, PhysDNet reconstructs sonar intensity from these components, enabling self-supervised training without ground-truth annotations. Experiments show that the decomposed representations preserve stable geological structures, capture physically consistent illumination and attenuation, and produce reliable shadow maps. These findings demonstrate that physics-guided decomposition provides a stable and interpretable domain for SSS analysis, improving both physical consistency and downstream tasks such as registration and shadow interpretation.

physics.ao-ph↗

arXiv2025

A Geometrically Consistent Matching Framework for Side-Scan Sonar Mapping

Robust matching of side-scan sonar imagery remains a fundamental challenge in seafloor mapping due to view-dependent backscatter, shadows, and geometric distortion. This paper proposes a novel matching framework that combines physical decoupling and geometric consistency to enhance correspondence accuracy and consistency across viewpoints. A multi-branch network, derived from the Lambertian reflection model, decomposes raw sonar images into seabed reflectivity, terrain elevation, and acoustic path loss. The reflectivity map, serving as a stable matching domain, is used in conjunction with a training-free matching pipeline combining SuperPoint and MINIMA-LightGlue. Geometry-aware outlier rejection leverages both terrain elevation and its physically derived shadow map to further remove mismatches in acoustically occluded and topographically inconsistent regions, thereby improving registration accuracy. Quantitative and visual evaluations against traditional, CNN-, and Transformer-based state-of-the-art methods demonstrate that our method achieves lower matching error, higher geometric consistency, and greater robustness to viewpoint variations. The proposed approach provides a data-efficient, physically interpretable solution for high-precision side-scan sonar image matching in complex seafloor environments.

physics.ins-det↗

arXiv2025

Coarse-to-Fine Non-Rigid Registration for Side-Scan Sonar Mosaicking

Side-scan sonar mosaicking plays a crucial role in large-scale seabed mapping but is challenged by complex non-linear, spatially varying distortions due to diverse sonar acquisition conditions. Existing rigid or affine registration methods fail to model such complex deformations, whereas traditional non-rigid techniques tend to overfit and lack robustness in sparse-texture sonar data. To address these challenges, we propose a coarse-to-fine hierarchical non-rigid registration framework tailored for large-scale side-scan sonar images. Our method begins with a global Thin Plate Spline initialization from sparse correspondences, followed by superpixel-guided segmentation that partitions the image into structurally consistent patches preserving terrain integrity. Each patch is then refined by a pretrained SynthMorph network in an unsupervised manner, enabling dense and flexible alignment without task-specific training. Finally, a fusion strategy integrates both global and local deformations into a smooth, unified deformation field. Extensive quantitative and visual evaluations demonstrate that our approach significantly outperforms state-of-the-art rigid, classical non-rigid, and learning-based methods in accuracy, structural consistency, and deformation smoothness on the challenging sonar dataset.

physics.geo-ph↗

arXiv2025

BenthiCat: An opti-acoustic dataset for advancing benthic classification and habitat mapping

Benthic habitat mapping is fundamental for understanding marine ecosystems, guiding conservation efforts, and supporting sustainable resource management. Yet, the scarcity of large, annotated datasets limits the development and benchmarking of machine learning models in this domain. This paper introduces a thorough multi-modal dataset, comprising about a million side-scan sonar (SSS) tiles collected along the coast of Catalonia (Spain), complemented by bathymetric maps and a set of co-registered optical images from targeted surveys using an autonomous underwater vehicle (AUV). Approximately 36000 of the SSS tiles have been manually annotated with segmentation masks to enable supervised fine-tuning of classification models. All the raw sensor data, together with mosaics, are also released to support further exploration and algorithm development. To address challenges in multi-sensor data fusion for AUVs, we spatially associate optical images with corresponding SSS tiles, facilitating self-supervised, cross-modal representation learning. Accompanying open-source preprocessing and annotation tools are provided to enhance accessibility and encourage research. This resource aims to establish a standardized benchmark for underwater habitat mapping, promoting advancements in autonomous seafloor classification and multi-sensor integration.

cs.CV↗

arXiv2025

Stonefish: Supporting Machine Learning Research in Marine Robotics

Simulations are highly valuable in marine robotics, offering a cost-effective and controlled environment for testing in the challenging conditions of underwater and surface operations. Given the high costs and logistical difficulties of real-world trials, simulators capable of capturing the operational conditions of subsea environments have become key in developing and refining algorithms for remotely-operated and autonomous underwater vehicles. This paper highlights recent enhancements to the Stonefish simulator, an advanced open-source platform supporting development and testing of marine robotics solutions. Key updates include a suite of additional sensors, such as an event-based camera, a thermal camera, and an optical flow camera, as well as, visual light communication, support for tethered operations, improved thruster modelling, more flexible hydrodynamics, and enhanced sonar accuracy. These developments and an automated annotation tool significantly bolster Stonefish's role in marine robotics research, especially in the field of machine learning, where training data with a known ground truth is hard or impossible to collect.

cs.RO↗

arXiv2025

eStonefish-Scenes: A Sim-to-Real Validated and Robot-Centric Event-based Optical Flow Dataset for Underwater Vehicles

Event-based cameras (EBCs) are poised to transform underwater robotics, yet the absence of labelled event-based datasets for underwater environments severely limits progress in tasks such as visual odometry and obstacle avoidance. Real-world event-based optical flow datasets are scarce, resource-intensive to collect, and lack diversity, while no prior benchmarks target underwater applications. To bridge this gap, we introduce eStonefish-Scenes, a synthetic event-based optical flow dataset generated using the Stonefish simulator, together with an open data generation pipeline for creating customizable underwater environments featuring realistic coral reefs and biologically inspired schools of fish with reactive navigation behaviours. We also present eWiz, a comprehensive library for event-based data processing, encompassing data loading, augmentation, visualization, encoding, training utilities, loss functions, and evaluation metrics. To validate sim-to-real transferability, we collected real-world data using a DAVIS346 hybrid event-and-frame camera mounted on a BlueROV2 in an indoor testing pool. Ground-truth optical flow was derived via homography-based frame-to-poster registration, and per-pixel uncertainty was estimated through Monte Carlo perturbation of keypoint correspondences. This uncertainty was incorporated into the evaluation metrics, enabling reliability-aware performance assessment. A ConvGRU-based optical flow network, trained exclusively on synthetic eStonefish-Scenes data, was evaluated on the real-world sequences without fine-tuning, achieving an uncertainty-weighted average endpoint error of 0.79 pixels. These results demonstrate that the proposed synthetic dataset effectively supports sim-to-real transfer for underwater event-based optical flow estimation, substantially reducing the need for costly real-world data collection.

cs.CV↗

arXiv2025

Hyperspectral Imaging for Identifying Foreign Objects on Pork Belly

Ensuring food safety and quality is critical in the food processing industry, where the detection of contaminants remains a persistent challenge. This study presents an automated solution for detecting foreign objects on pork belly meat using hyperspectral imaging (HSI). A hyperspectral camera was used to capture data across various bands in the near-infrared (NIR) spectrum (900-1700 nm), enabling accurate identification of contaminants that are often undetectable through traditional visual inspection methods. The proposed solution combines pre-processing techniques with a segmentation approach based on a lightweight Vision Transformer (ViT) to distinguish contaminants from meat, fat, and conveyor belt materials. The adopted strategy demonstrates high detection accuracy and training efficiency, while also addressing key industrial challenges such as inherent noise, temperature variations, and spectral similarity between contaminants and pork belly. Experimental results validate the effectiveness of hyperspectral imaging in enhancing food safety, highlighting its potential for broad real-time applications in automated quality control processes.

cs.CV↗

arXiv2025

The iToBoS dataset: skin region images extracted from 3D total body photographs for lesion detection

Artificial intelligence has significantly advanced skin cancer diagnosis by enabling rapid and accurate detection of malignant lesions. In this domain, most publicly available image datasets consist of single, isolated skin lesions positioned at the center of the image. While these lesion-centric datasets have been fundamental for developing diagnostic algorithms, they lack the context of the surrounding skin, which is critical for improving lesion detection. The iToBoS dataset was created to address this challenge. It includes 16,954 images of skin regions from 100 participants, captured using 3D total body photography. Each image roughly corresponds to a $7 \times 9$ cm section of skin with all suspicious lesions annotated using bounding boxes. Additionally, the dataset provides metadata such as anatomical location, age group, and sun damage score for each image. This dataset aims to facilitate training and benchmarking of algorithms, with the goal of enabling early detection of skin cancer and deployment of this technology in non-clinical environments.

eess.IV↗

arXiv2024

eCARLA-scenes: A synthetically generated dataset for event-based optical flow prediction

The joint use of event-based vision and Spiking Neural Networks (SNNs) is expected to have a large impact in robotics in the near future, in tasks such as, visual odometry and obstacle avoidance. While researchers have used real-world event datasets for optical flow prediction (mostly captured with Unmanned Aerial Vehicles (UAVs)), these datasets are limited in diversity, scalability, and are challenging to collect. Thus, synthetic datasets offer a scalable alternative by bridging the gap between reality and simulation. In this work, we address the lack of datasets by introducing eWiz, a comprehensive library for processing event-based data. It includes tools for data loading, augmentation, visualization, encoding, and generation of training data, along with loss functions and performance metrics. We further present a synthetic event-based datasets and data generation pipelines for optical flow prediction tasks. Built on top of eWiz, eCARLA-scenes makes use of the CARLA simulator to simulate self-driving car scenarios. The ultimate goal of this dataset is the depiction of diverse environments while laying a foundation for advancing event-based camera applications in autonomous field vehicle navigation, paving the way for using SNNs on neuromorphic hardware such as the Intel Loihi.

cs.CV↗

arXiv2023

A Convolutional Vision Transformer for Semantic Segmentation of Side-Scan Sonar Data

Distinguishing among different marine benthic habitat characteristics is of key importance in a wide set of seabed operations ranging from installations of oil rigs to laying networks of cables and monitoring the impact of humans on marine ecosystems. The Side-Scan Sonar (SSS) is a widely used imaging sensor in this regard. It produces high-resolution seafloor maps by logging the intensities of sound waves reflected back from the seafloor. In this work, we leverage these acoustic intensity maps to produce pixel-wise categorization of different seafloor types. We propose a novel architecture adapted from the Vision Transformer (ViT) in an encoder-decoder framework. Further, in doing so, the applicability of ViTs is evaluated on smaller datasets. To overcome the lack of CNN-like inductive biases, thereby making ViTs more conducive to applications in low data regimes, we propose a novel feature extraction module to replace the Multi-layer Perceptron (MLP) block within transformer layers and a novel module to extract multiscale patch embeddings. A lightweight decoder is also proposed to complement this design in order to further boost multiscale feature extraction. With the modified architecture, we achieve state-of-the-art results and also meet real-time computational requirements. We make our code available at ~\url{https://github.com/hayatrajani/s3seg-vit

cs.CV↗