arXiv subjects

Lakmal Seneviratne

Publications and source records attributed to Lakmal Seneviratne.

29 records · Page 2Linked to original sources

arXiv2023

Vision-Based Autonomous Navigation for Unmanned Surface Vessel in Extreme Marine Conditions

Visual perception is an important component for autonomous navigation of unmanned surface vessels (USV), particularly for the tasks related to autonomous inspection and tracking. These tasks involve vision-based navigation techniques to identify the target for navigation. Reduced visibility under extreme weather conditions in marine environments makes it difficult for vision-based approaches to work properly. To overcome these issues, this paper presents an autonomous vision-based navigation framework for tracking target objects in extreme marine conditions. The proposed framework consists of an integrated perception pipeline that uses a generative adversarial network (GAN) to remove noise and highlight the object features before passing them to the object detector (i.e., YOLOv5). The detected visual features are then used by the USV to track the target. The proposed framework has been thoroughly tested in simulation under extremely reduced visibility due to sandstorms and fog. The results are compared with state-of-the-art de-hazing methods across the benchmarked MBZIRC simulation dataset, on which the proposed scheme has outperformed the existing methods across various metrics.

cs.CV↗

arXiv2023

High Speed Neuromorphic Vision-Based Inspection of Countersinks in Automated Manufacturing Processes

Countersink inspection is crucial in various automated assembly lines, especially in the aerospace and automotive sectors. Advancements in machine vision introduced automated robotic inspection of countersinks using laser scanners and monocular cameras. Nevertheless, the aforementioned sensing pipelines require the robot to pause on each hole for inspection due to high latency and measurement uncertainties with motion, leading to prolonged execution times of the inspection task. The neuromorphic vision sensor, on the other hand, has the potential to expedite the countersink inspection process, but the unorthodox output of the neuromorphic technology prohibits utilizing traditional image processing techniques. Therefore, novel event-based perception algorithms need to be introduced. We propose a countersink detection approach on the basis of event-based motion compensation and the mean-shift clustering principle. In addition, our framework presents a robust event-based circle detection algorithm to precisely estimate the depth of the countersink specimens. The proposed approach expedites the inspection process by a factor of 10$\times$ compared to conventional countersink inspection methods. The work in this paper was validated for over 50 trials on three countersink workpiece variants. The experimental results show that our method provides a precision of 0.025 mm for countersink depth inspection despite the low resolution of commercially available neuromorphic cameras.

cs.RO↗

arXiv2022

Static Hovering Realization for Multirotor Aerial Vehicles with Tiltable Propellers

This paper presents a theoretical study on the ability of multi-rotor aerial vehicles (MRAVs) with tiltable propellers to achieve and sustain static hovering at different orientations. To analyze the ability of MRAVs with tiltable propellers to achieve static hovering, a novel linear map between the platform's control inputs and applied forces and moments is introduced. The relation between the introduced map and the platform's ability to hover at different orientations is developed. Correspondingly, the conditions for MRAVs with tiltable propellers to realize and sustain static hovering are detailed. A numerical metric is then introduced, which reflects the ability of MRAVs to sustain static hovering at different orientations. A subclass of MRAVs with tiltable propellers is defined as the Critically Statically Hoverable platforms (CSH), where CSH platforms are MRAVs that cannot sustain static hovering with fixed propellers, but can achieve static hovering with tilting propellers. Finally, extensive simulations are conducted to test and validate the above findings, and to demonstrate the effect of the proposed numerical metric on the platform's dynamics.

cs.RO↗

arXiv2022

A Neuromorphic Vision-Based Measurement for Robust Relative Localization in Future Space Exploration Missions

Space exploration has witnessed revolutionary changes upon landing of the Perseverance Rover on the Martian surface and demonstrating the first flight beyond Earth by the Mars helicopter, Ingenuity. During their mission on Mars, Perseverance Rover and Ingenuity collaboratively explore the Martian surface, where Ingenuity scouts terrain information for rover's safe traversability. Hence, determining the relative poses between both the platforms is of paramount importance for the success of this mission. Driven by this necessity, this work proposes a robust relative localization system based on a fusion of neuromorphic vision-based measurements (NVBMs) and inertial measurements. The emergence of neuromorphic vision triggered a paradigm shift in the computer vision community, due to its unique working principle delineated with asynchronous events triggered by variations of light intensities occurring in the scene. This implies that observations cannot be acquired in static scenes due to illumination invariance. To circumvent this limitation, high frequency active landmarks are inserted in the scene to guarantee consistent event firing. These landmarks are adopted as salient features to facilitate relative localization. A novel event-based landmark identification algorithm using Gaussian Mixture Models (GMM) is developed for matching the landmarks correspondences formulating our NVBMs. The NVBMs are fused with inertial measurements in proposed state estimators, landmark tracking Kalman filter (LTKF) and translation decoupled Kalman filter (TDKF) for landmark tracking and relative localization, respectively. The proposed system was tested in a variety of experiments and has outperformed state-of-the-art approaches in accuracy and range.

cs.CV↗

arXiv2022

Neuromorphic Camera Denoising using Graph Neural Network-driven Transformers

Neuromorphic vision is a bio-inspired technology that has triggered a paradigm shift in the computer-vision community and is serving as a key-enabler for a multitude of applications. This technology has offered significant advantages including reduced power consumption, reduced processing needs, and communication speed-ups. However, neuromorphic cameras suffer from significant amounts of measurement noise. This noise deteriorates the performance of neuromorphic event-based perception and navigation algorithms. In this paper, we propose a novel noise filtration algorithm to eliminate events which do not represent real log-intensity variations in the observed scene. We employ a Graph Neural Network (GNN)-driven transformer algorithm, called GNN-Transformer, to classify every active event pixel in the raw stream into real-log intensity variation or noise. Within the GNN, a message-passing framework, called EventConv, is carried out to reflect the spatiotemporal correlation among the events, while preserving their asynchronous nature. We also introduce the Known-object Ground-Truth Labeling (KoGTL) approach for generating approximate ground truth labels of event streams under various illumination conditions. KoGTL is used to generate labeled datasets, from experiments recorded in chalenging lighting conditions. These datasets are used to train and extensively test our proposed algorithm. When tested on unseen datasets, the proposed algorithm outperforms existing methods by 8.8% in terms of filtration accuracy. Additional tests are also conducted on publicly available datasets to demonstrate the generalization capabilities of the proposed algorithm in the presence of illumination variations and different motion dynamics. Compared to existing solutions, qualitative results verified the superior capability of the proposed algorithm to eliminate noise while preserving meaningful scene events.

cs.CV↗

arXiv2022

Noise Tolerant Identification and Tuning Approach Using Deep Neural Networks For Visual Servoing Applications

Vision based control of Unmanned Aerial Vehicles (UAVs) has been adopted by a wide range of applications due to the availability of low-cost on-board sensors and computers. Tuning such systems to work properly requires extensive domain specific experience, which limits the growth of emerging applications. Moreover, obtaining performance limits of UAV based visual servoing is difficult due to the complexity of the models used. In this paper, we propose a novel noise tolerant approach for real-time identification and tuning of visual servoing systems, based on deep neural networks (DNN) classification of system response generated by the modified relay feedback test (MRFT). The proposed method, called DNN with noise protected MRFT (DNN-NP-MRFT), can be used with a multitude of vision sensors and estimation algorithms despite the high levels of sensor's noise. Response of DNN-NP-MRFT to noise perturbations is investigated and its effect on identification and tuning performance is analyzed. The proposed DNN-NP-MRFT is able to detect performance changes due to the use of high latency vision sensors, or due to the integration of inertial measurement unit (IMU) measurements in the UAV states estimation. Experimental identification closely matches simulation results, which can be used to explain system behaviour and predict the closed loop performance limits for a given hardware and software setup. We also demonstrate the ability of DNN-NP-MRFT tuned UAVs to reject external disturbances like wind, or human push and pull. Finally, we discuss the advantages of the proposed DNN-NP-MRFT visual servoing design approach compared with other approaches in literature.

cs.RO↗

arXiv2021

Real-Time Grasping Strategies Using Event Camera

Robotic vision plays a key role for perceiving the environment in grasping applications. However, the conventional framed-based robotic vision, suffering from motion blur and low sampling rate, may not meet the automation needs of evolving industrial requirements. This paper, for the first time, proposes an event-based robotic grasping framework for multiple known and unknown objects in a cluttered scene. Compared with standard frame-based vision, neuromorphic vision has advantages of microsecond-level sampling rate and no motion blur. Building on that, the model-based and model-free approaches are developed for known and unknown objects' grasping respectively. For the model-based approach, event-based multi-view approach is used to localize the objects in the scene, and then point cloud processing allows for the clustering and registering of objects. Differently, the proposed model-free approach utilizes the developed event-based object segmentation, visual servoing and grasp planning to localize, align to, and grasp the targeting object. The proposed approaches are experimentally validated with objects of different sizes, using a UR10 robot with an eye-in-hand neuromorphic camera and a Barrett hand gripper. Moreover, the robustness of the two proposed event-based grasping approaches are validated in a low-light environment. This low-light operating ability shows a great advantage over the grasping using the standard frame-based vision. Furthermore, the developed model-free approach demonstrates the advantage of dealing with unknown object without prior knowledge compared to the proposed model-based approach.

cs.RO↗

arXiv2020

Neuromorphic Event-Based Slip Detection and suppression in Robotic Grasping and Manipulation

Slip detection is essential for robots to make robust grasping and fine manipulation. In this paper, a novel dynamic vision-based finger system for slip detection and suppression is proposed. We also present a baseline and feature based approach to detect object slips under illumination and vibration uncertainty. A threshold method is devised to autonomously sample noise in real-time to improve slip detection. Moreover, a fuzzy based suppression strategy using incipient slip feedback is proposed for regulating the grip force. A comprehensive experimental study of our proposed approaches under uncertainty and system for high-performance precision manipulation are presented. We also propose a slip metric to evaluate such performance quantitatively. Results indicate that the system can effectively detect incipient slip events at a sampling rate of 2kHz ($Δt = 500μs$) and suppress them before a gross slip occurs. The event-based approach holds promises to high precision manipulation task requirement in industrial manufacturing and household services.

cs.RO↗

arXiv2020

Neuromorphic Eye-in-Hand Visual Servoing

Robotic vision plays a major role in factory automation to service robot applications. However, the traditional use of frame-based camera sets a limitation on continuous visual feedback due to their low sampling rate and redundant data in real-time image processing, especially in the case of high-speed tasks. Event cameras give human-like vision capabilities such as observing the dynamic changes asynchronously at a high temporal resolution ($1μs$) with low latency and wide dynamic range. In this paper, we present a visual servoing method using an event camera and a switching control strategy to explore, reach and grasp to achieve a manipulation task. We devise three surface layers of active events to directly process stream of events from relative motion. A purely event based approach is adopted to extract corner features, localize them robustly using heat maps and generate virtual features for tracking and alignment. Based on the visual feedback, the motion of the robot is controlled to make the temporal upcoming event features converge to the desired event in spatio-temporal space. The controller switches its strategy based on the sequence of operation to establish a stable grasp. The event based visual servoing (EVBS) method is validated experimentally using a commercial robot manipulator in an eye-in-hand configuration. Experiments prove the effectiveness of the EBVS method to track and grasp objects of different shapes without the need for re-tuning.

cs.RO↗

arXiv2019

Risk-Aware Reasoning for Autonomous Vehicles

A significant barrier to deploying autonomous vehicles (AVs) on a massive scale is safety assurance. Several technical challenges arise due to the uncertain environment in which AVs operate such as road and weather conditions, errors in perception and sensory data, and also model inaccuracy. In this paper, we propose a system architecture for risk-aware AVs capable of reasoning about uncertainty and deliberately bounding the risk of collision below a given threshold. We discuss key challenges in the area, highlight recent research developments, and propose future research directions in three subsystems. First, a perception subsystem that detects objects within a scene while quantifying the uncertainty that arises from different sensing and communication modalities. Second, an intention recognition subsystem that predicts the driving-style and the intention of agent vehicles (and pedestrians). Third, a planning subsystem that takes into account the uncertainty, from perception and intention recognition subsystems, and propagates all the way to control policies that explicitly bound the risk of collision. We believe that such a white-box approach is crucial for future adoption of AVs on a large scale.

cs.AI↗

arXiv2017

Discrete Cosserat Approach for Multi-Section Soft Robots Dynamics

In spite of recent progress, soft robotics still suffers from a lack of unified modeling framework. Nowadays, the most adopted model for the design and control of soft robots is the piece-wise constant curvature model, with its consolidated benefits and drawbacks. In this work, an alternative model for multisection soft robots dynamics is presented based on a discrete Cosserat approach, which, not only takes into account shear and torsional deformations, essentials to cope with out-of-plane external loads, but also inherits the geometrical and mechanical properties of the continuous Cosserat model, making it the natural soft robotics counterpart of the traditional rigid robotics dynamics model. The soundness of the model is demonstrated through extensive simulation and experimental results for both plane and out-of-plane motions.

cs.RO↗