arXiv subjects

Yahya Zweiri

Publications and source records attributed to Yahya Zweiri.

33 records · Page 2Linked to original sources

arXiv2022

Static Hovering Realization for Multirotor Aerial Vehicles with Tiltable Propellers

This paper presents a theoretical study on the ability of multi-rotor aerial vehicles (MRAVs) with tiltable propellers to achieve and sustain static hovering at different orientations. To analyze the ability of MRAVs with tiltable propellers to achieve static hovering, a novel linear map between the platform's control inputs and applied forces and moments is introduced. The relation between the introduced map and the platform's ability to hover at different orientations is developed. Correspondingly, the conditions for MRAVs with tiltable propellers to realize and sustain static hovering are detailed. A numerical metric is then introduced, which reflects the ability of MRAVs to sustain static hovering at different orientations. A subclass of MRAVs with tiltable propellers is defined as the Critically Statically Hoverable platforms (CSH), where CSH platforms are MRAVs that cannot sustain static hovering with fixed propellers, but can achieve static hovering with tilting propellers. Finally, extensive simulations are conducted to test and validate the above findings, and to demonstrate the effect of the proposed numerical metric on the platform's dynamics.

cs.RO↗

arXiv2022

A Neuromorphic Vision-Based Measurement for Robust Relative Localization in Future Space Exploration Missions

Space exploration has witnessed revolutionary changes upon landing of the Perseverance Rover on the Martian surface and demonstrating the first flight beyond Earth by the Mars helicopter, Ingenuity. During their mission on Mars, Perseverance Rover and Ingenuity collaboratively explore the Martian surface, where Ingenuity scouts terrain information for rover's safe traversability. Hence, determining the relative poses between both the platforms is of paramount importance for the success of this mission. Driven by this necessity, this work proposes a robust relative localization system based on a fusion of neuromorphic vision-based measurements (NVBMs) and inertial measurements. The emergence of neuromorphic vision triggered a paradigm shift in the computer vision community, due to its unique working principle delineated with asynchronous events triggered by variations of light intensities occurring in the scene. This implies that observations cannot be acquired in static scenes due to illumination invariance. To circumvent this limitation, high frequency active landmarks are inserted in the scene to guarantee consistent event firing. These landmarks are adopted as salient features to facilitate relative localization. A novel event-based landmark identification algorithm using Gaussian Mixture Models (GMM) is developed for matching the landmarks correspondences formulating our NVBMs. The NVBMs are fused with inertial measurements in proposed state estimators, landmark tracking Kalman filter (LTKF) and translation decoupled Kalman filter (TDKF) for landmark tracking and relative localization, respectively. The proposed system was tested in a variety of experiments and has outperformed state-of-the-art approaches in accuracy and range.

cs.CV↗

arXiv2022

The Role of Time Delay in Sim2real Transfer of Reinforcement Learning for Cyber-Physical Systems

This paper analyzes the simulation to reality gap in reinforcement learning (RL) cyber-physical systems with fractional delays (i.e. delays that are non-integer multiple of the sampling period). The consideration of fractional delay has important implications on the nature of the cyber-physical system considered. Systems with delays are non-Markovian, and the system state vector needs to be extended to make the system Markovian. We show that this is not possible when the delay is in the output, and the problem would always be non-Markovian. Based on this analysis, a sampling scheme is proposed that results in efficient RL training and agents that perform well in realistic multirotor unmanned aerial vehicle simulations. We demonstrate that the resultant agents do not produce excessive oscillations, which is not the case with RL agents that do not consider time delay in the model.

eess.SY↗

arXiv2022

Analysis of the Effect of Time Delay for Unmanned Aerial Vehicles with Applications to Vision Based Navigation

In this paper, we analyze the effect of time delay dynamics on controller design for Unmanned Aerial Vehicles (UAVs) with vision based navigation. Time delay is an inevitable phenomenon in cyber-physical systems, and has important implications on controller design and trajectory generation for UAVs. The impact of time delay on UAV dynamics increases with the use of the slower vision based navigation stack. We show that the existing models in the literature, which exclude time delay, are unsuitable for controller tuning since a trivial solution for minimizing an error cost functional always exists. The trivial solution that we identify suggests use of infinite controller gains to achieve optimal performance, which contradicts practical findings. We avoid such shortcomings by introducing a novel nonlinear time delay model for UAVs, and then obtain a set of linear decoupled models corresponding to each of the UAV control loops. The cost functional of the linearized time delay model of angular and altitude dynamics is analyzed, and in contrast to the delay-free models, we show the existence of finite optimal controller parameters. Due to the use of time delay models, we experimentally show that the proposed model accurately represents system stability limits. Due to time delay consideration, we achieved a tracking results of RMSE 5.01 cm when tracking a lemniscate trajectory with a peak velocity of 2.09 m/s using visual odometry (VO) based UAV navigation, which is on par with the state-of-the-art.

cs.RO↗

arXiv2022

Design of Dynamics Invariant LSTM for Touch Based Human-UAV Interaction Detection

The field of Unmanned Aerial Vehicles (UAVs) has reached a high level of maturity in the last few years. Hence, bringing such platforms from closed labs, to day-to-day interactions with humans is important for commercialization of UAVs. One particular human-UAV scenario of interest for this paper is the payload handover scheme, where a UAV hands over a payload to a human upon their request. In this scope, this paper presents a novel real-time human-UAV interaction detection approach, where Long short-term memory (LSTM) based neural network is developed to detect state profiles resulting from human interaction dynamics. A novel data pre-processing technique is presented; this technique leverages estimated process parameters of training and testing UAVs to build dynamics invariant testing data. The proposed detection algorithm is lightweight and thus can be deployed in real-time using off the shelf UAV platforms; in addition, it depends solely on inertial and position measurements present on any classical UAV platform. The proposed approach is demonstrated on a payload handover task between multirotor UAVs and humans. Training and testing data were collected using real-time experiments. The detection approach has achieved an accuracy of 96\%, giving no false positives even in the presence of external wind disturbances, and when deployed and tested on two different UAVs.

cs.RO↗

arXiv2022

Neuromorphic Camera Denoising using Graph Neural Network-driven Transformers

Neuromorphic vision is a bio-inspired technology that has triggered a paradigm shift in the computer-vision community and is serving as a key-enabler for a multitude of applications. This technology has offered significant advantages including reduced power consumption, reduced processing needs, and communication speed-ups. However, neuromorphic cameras suffer from significant amounts of measurement noise. This noise deteriorates the performance of neuromorphic event-based perception and navigation algorithms. In this paper, we propose a novel noise filtration algorithm to eliminate events which do not represent real log-intensity variations in the observed scene. We employ a Graph Neural Network (GNN)-driven transformer algorithm, called GNN-Transformer, to classify every active event pixel in the raw stream into real-log intensity variation or noise. Within the GNN, a message-passing framework, called EventConv, is carried out to reflect the spatiotemporal correlation among the events, while preserving their asynchronous nature. We also introduce the Known-object Ground-Truth Labeling (KoGTL) approach for generating approximate ground truth labels of event streams under various illumination conditions. KoGTL is used to generate labeled datasets, from experiments recorded in chalenging lighting conditions. These datasets are used to train and extensively test our proposed algorithm. When tested on unseen datasets, the proposed algorithm outperforms existing methods by 8.8% in terms of filtration accuracy. Additional tests are also conducted on publicly available datasets to demonstrate the generalization capabilities of the proposed algorithm in the presence of illumination variations and different motion dynamics. Compared to existing solutions, qualitative results verified the superior capability of the proposed algorithm to eliminate noise while preserving meaningful scene events.

cs.CV↗

arXiv2022

Noise Tolerant Identification and Tuning Approach Using Deep Neural Networks For Visual Servoing Applications

Vision based control of Unmanned Aerial Vehicles (UAVs) has been adopted by a wide range of applications due to the availability of low-cost on-board sensors and computers. Tuning such systems to work properly requires extensive domain specific experience, which limits the growth of emerging applications. Moreover, obtaining performance limits of UAV based visual servoing is difficult due to the complexity of the models used. In this paper, we propose a novel noise tolerant approach for real-time identification and tuning of visual servoing systems, based on deep neural networks (DNN) classification of system response generated by the modified relay feedback test (MRFT). The proposed method, called DNN with noise protected MRFT (DNN-NP-MRFT), can be used with a multitude of vision sensors and estimation algorithms despite the high levels of sensor's noise. Response of DNN-NP-MRFT to noise perturbations is investigated and its effect on identification and tuning performance is analyzed. The proposed DNN-NP-MRFT is able to detect performance changes due to the use of high latency vision sensors, or due to the integration of inertial measurement unit (IMU) measurements in the UAV states estimation. Experimental identification closely matches simulation results, which can be used to explain system behaviour and predict the closed loop performance limits for a given hardware and software setup. We also demonstrate the ability of DNN-NP-MRFT tuned UAVs to reject external disturbances like wind, or human push and pull. Finally, we discuss the advantages of the proposed DNN-NP-MRFT visual servoing design approach compared with other approaches in literature.

cs.RO↗

arXiv2022

Neuromorphic Vision Based Control for the Precise Positioning of Robotic Drilling Systems

The manufacturing industry is currently witnessing a paradigm shift with the unprecedented adoption of industrial robots, and machine vision is a key perception technology that enables these robots to perform precise operations in unstructured environments. However, the sensitivity of conventional vision sensors to lighting conditions and high-speed motion sets a limitation on the reliability and work-rate of production lines. Neuromorphic vision is a recent technology with the potential to address the challenges of conventional vision with its high temporal resolution, low latency, and wide dynamic range. In this paper and for the first time, we propose a novel neuromorphic vision based controller for faster and more reliable machining operations, and present a complete robotic system capable of performing drilling tasks with sub-millimeter accuracy. Our proposed system localizes the target workpiece in 3D using two perception stages that we developed specifically for the asynchronous output of neuromorphic cameras. The first stage performs multi-view reconstruction for an initial estimate of the workpiece's pose, and the second stage refines this estimate for a local region of the workpiece using circular hole detection. The robot then precisely positions the drilling end-effector and drills the target holes on the workpiece using a combined position-based and image-based visual servoing approach. The proposed solution is validated experimentally for drilling nutplate holes on workpieces placed arbitrarily in an unstructured environment with uncontrolled lighting. Experimental results prove the effectiveness of our solution with an average positional errors of less than 0.1 mm, and demonstrate that the use of neuromorphic vision overcomes the lighting and speed limitations of conventional cameras.

cs.RO↗

arXiv2021

Systematic Online Tuning of Multirotor UAVs for Accurate Trajectory Tracking Under Wind Disturbances and In-Flight Dynamics Changes

The demand for accurate and fast trajectory tracking for multirotor Unmanned Aerial Vehicles (UAVs) have grown recently due to advances in UAV avionics technology and application domains. In many applications, the multirotor UAV is required to accurately perform aggressive maneuvers in challenging scenarios like the presence of external wind disturbances or in-flight payload changes. In this paper, we propose a systematic controller tuning approach based on identification results obtained by a recently developed Deep Neural Networks with the Modified Relay Feedback Test (DNN-MRFT) algorithm. We formulate a linear equivalent representation suitable for DNN-MRFT using feedback linearization. This representation enables the analytical investigation of different controller structures and tuning settings, and captures the non-linearity trends of the system. With this approach, the trade-off between performance and robustness in design was made possible which is convenient for the design of controllers of UAVs operating in uncertain environments. We demonstrate that our approach is adaptive and robust through a set of experiments, where accurate trajectory tracking is maintained despite significant changes to the UAV aerodynamic characteristics and the application of wind disturbance. Due to the model-based system design, it was possible to obtain low discrepancy between simulation and experimental results which is beneficial for potential use of the proposed approach for real-time model-based planning and fault detection tasks. We obtained RMSE of $3.59 \; cm$ when tracking aggressive trajectories in the presence of strong wind, which is on par with state-of-the-art.

cs.RO↗

arXiv2021

Multirotors from Takeoff to Real-Time Full Identification Using the Modified Relay Feedback Test and Deep Neural Networks

Low cost real-time identification of multirotor unmanned aerial vehicle (UAV) dynamics is an active area of research supported by the surge in demand and emerging application domains. Such real-time identification capabilities shorten development time and cost, making UAVs' technology more accessible, and enable a wide variety of advanced applications. In this paper, we present a novel comprehensive approach, called DNN-MRFT, for real-time identification and tuning of multirotor UAVs using the Modified Relay Feedback Test (MRFT) and Deep Neural Networks (DNN). The main contribution is the development of a generalized framework for the application of DNN-MRFT to higher-order systems. One of the notable advantages of DNN-MRFT is the exact estimation of identified process gain, which mitigates the inaccuracies introduced due to the use of the describing function method in approximating the response of Lure's systems. A secondary contribution is a generalized controller based on DNN-MRFT that takes-off a UAV with unknown dynamics and identifies the inner loops dynamics in-flight. Using the developed framework, DNN-MRFT is sequentially applied to the outer translational loops of the UAV utilizing in-flight results obtained for the inner attitude loops. DNN-MRFT takes on average 15 seconds to get the full knowledge of multirotor UAV dynamics and without any further tuning or calibration the UAV would be able to pass through a vertical window, and accurately follow trajectories achieving state-of-the-art performance. Such demonstrated accuracy, speed, and robustness of identification pushes the limits of state-of-the-art in real-time identification of UAVs.

eess.SY↗

arXiv2021

Real-Time Grasping Strategies Using Event Camera

Robotic vision plays a key role for perceiving the environment in grasping applications. However, the conventional framed-based robotic vision, suffering from motion blur and low sampling rate, may not meet the automation needs of evolving industrial requirements. This paper, for the first time, proposes an event-based robotic grasping framework for multiple known and unknown objects in a cluttered scene. Compared with standard frame-based vision, neuromorphic vision has advantages of microsecond-level sampling rate and no motion blur. Building on that, the model-based and model-free approaches are developed for known and unknown objects' grasping respectively. For the model-based approach, event-based multi-view approach is used to localize the objects in the scene, and then point cloud processing allows for the clustering and registering of objects. Differently, the proposed model-free approach utilizes the developed event-based object segmentation, visual servoing and grasp planning to localize, align to, and grasp the targeting object. The proposed approaches are experimentally validated with objects of different sizes, using a UR10 robot with an eye-in-hand neuromorphic camera and a Barrett hand gripper. Moreover, the robustness of the two proposed event-based grasping approaches are validated in a low-light environment. This low-light operating ability shows a great advantage over the grasping using the standard frame-based vision. Furthermore, the developed model-free approach demonstrates the advantage of dealing with unknown object without prior knowledge compared to the proposed model-based approach.

cs.RO↗

arXiv2021

Dynamic Based Estimator for UAVs with Real-time Identification Using DNN and the Modified Relay Feedback Test

Control performance of Unmanned Aerial Vehicles (UAVs) is directly affected by their ability to estimate their states accurately. With the increasing popularity of autonomous UAV solutions in real world applications, it is imperative to develop robust adaptive estimators that can ameliorate sensor noises in low-cost UAVs. Utilizing the knowledge of UAV dynamics in estimation can provide significant advantages, but remains challenging due to the complex and expensive pre-flight experiments required to obtain UAV dynamic parameters. In this paper, we propose two decoupled dynamic model based Extended Kalman Filters for UAVs, that provide high rate estimates for position, and velocity of rotational and translational states, as well as filtered inertial acceleration. The dynamic model parameters are estimated online using the Deep Neural Network and Modified Relay Feedback Test (DNN-MRFT) framework, without requiring any prior knowledge of the UAV physical parameters. The designed filters with real-time identified process model parameters are tested experimentally and showed two advantages. Firstly, smooth and lag-free estimates of the UAV rotational speed and inertial acceleration are obtained, and used to improve the closed loop system performance, reducing the controller action by over 6 %. Secondly, the proposed approach enabled the UAV to track aggressive trajectories with low rate position measurements, a task usually infeasible under those conditions. The experimental data shows that we achieved estimation performance matching other methods that requires full knowledge of the UAV parameters.

cs.RO↗

arXiv2020

Real-time System Identification Using Deep Learning for Linear Processes with Application to Unmanned Aerial Vehicles

This paper proposes a novel parametric identification approach for linear systems using Deep Learning (DL) and the Modified Relay Feedback Test (MRFT). The proposed methodology utilizes MRFT to reveal distinguishing frequencies about an unknown process; which are then passed to a trained DL model to identify the underlying process parameters. The presented approach guarantees stability and performance in the identification and control phases respectively, and requires few seconds of observation data to infer the dynamic system parameters. Quadrotor Unmanned Aerial Vehicle (UAV) attitude and altitude dynamics were used in simulation and experimentation to verify the presented methodology. Results show the effectiveness and real-time capabilities of the proposed approach, which outperforms the conventional Prediction Error Method in terms of accuracy, robustness to biases, computational efficiency and data requirements.

eess.SY↗

arXiv2020

Neuromorphic Event-Based Slip Detection and suppression in Robotic Grasping and Manipulation

Slip detection is essential for robots to make robust grasping and fine manipulation. In this paper, a novel dynamic vision-based finger system for slip detection and suppression is proposed. We also present a baseline and feature based approach to detect object slips under illumination and vibration uncertainty. A threshold method is devised to autonomously sample noise in real-time to improve slip detection. Moreover, a fuzzy based suppression strategy using incipient slip feedback is proposed for regulating the grip force. A comprehensive experimental study of our proposed approaches under uncertainty and system for high-performance precision manipulation are presented. We also propose a slip metric to evaluate such performance quantitatively. Results indicate that the system can effectively detect incipient slip events at a sampling rate of 2kHz ($Δt = 500μs$) and suppress them before a gross slip occurs. The event-based approach holds promises to high precision manipulation task requirement in industrial manufacturing and household services.

cs.RO↗

arXiv2020

Neuromorphic Eye-in-Hand Visual Servoing

Robotic vision plays a major role in factory automation to service robot applications. However, the traditional use of frame-based camera sets a limitation on continuous visual feedback due to their low sampling rate and redundant data in real-time image processing, especially in the case of high-speed tasks. Event cameras give human-like vision capabilities such as observing the dynamic changes asynchronously at a high temporal resolution ($1μs$) with low latency and wide dynamic range. In this paper, we present a visual servoing method using an event camera and a switching control strategy to explore, reach and grasp to achieve a manipulation task. We devise three surface layers of active events to directly process stream of events from relative motion. A purely event based approach is adopted to extract corner features, localize them robustly using heat maps and generate virtual features for tracking and alignment. Based on the visual feedback, the motion of the robot is controlled to make the temporal upcoming event features converge to the desired event in spatio-temporal space. The controller switches its strategy based on the sequence of operation to establish a stable grasp. The event based visual servoing (EVBS) method is validated experimentally using a commercial robot manipulator in an eye-in-hand configuration. Experiments prove the effectiveness of the EBVS method to track and grasp objects of different shapes without the need for re-tuning.

cs.RO↗