arXiv subjects

Yang Tang

Publications and source records attributed to Yang Tang.

At least 91 records · Page 5Linked to original sources

arXiv2022

Towards Generalization on Real Domain for Single Image Dehazing via Meta-Learning

Learning-based image dehazing methods are essential to assist autonomous systems in enhancing reliability. Due to the domain gap between synthetic and real domains, the internal information learned from synthesized images is usually sub-optimal in real domains, leading to severe performance drop of dehaizing models. Driven by the ability on exploring internal information from a few unseen-domain samples, meta-learning is commonly adopted to address this issue via test-time training, which is hyperparameter-sensitive and time-consuming. In contrast, we present a domain generalization framework based on meta-learning to dig out representative and discriminative internal properties of real hazy domains without test-time training. To obtain representative domain-specific information, we attach two entities termed adaptation network and distance-aware aggregator to our dehazing network. The adaptation network assists in distilling domain-relevant information from a few hazy samples and caching it into a collection of features. The distance-aware aggregator strives to summarize the generated features and filter out misleading information for more representative internal properties. To enhance the discrimination of distilled internal information, we present a novel loss function called domain-relevant contrastive regularization, which encourages the internal features generated from the same domain more similar and that from diverse domains more distinct. The generated representative and discriminative features are regarded as some external variables of our dehazing network to regress a particular and powerful function for a given domain. The extensive experiments on real hazy datasets, such as RTTS and URHI, validate that our proposed method has superior generalization ability than the state-of-the-art competitors.

cs.CV↗

arXiv2022

Visual Semantic Segmentation Based on Few/Zero-Shot Learning: An Overview

Visual semantic segmentation aims at separating a visual sample into diverse blocks with specific semantic attributes and identifying the category for each block, and it plays a crucial role in environmental perception. Conventional learning-based visual semantic segmentation approaches count heavily on large-scale training data with dense annotations and consistently fail to estimate accurate semantic labels for unseen categories. This obstruction spurs a craze for studying visual semantic segmentation with the assistance of few/zero-shot learning. The emergence and rapid progress of few/zero-shot visual semantic segmentation make it possible to learn unseen-category from a few labeled or zero-labeled samples, which advances the extension to practical applications. Therefore, this paper focuses on the recently published few/zero-shot visual semantic segmentation methods varying from 2D to 3D space and explores the commonalities and discrepancies of technical settlements under different segmentation circumstances. Specifically, the preliminaries on few/zero-shot visual semantic segmentation, including the problem definitions, typical datasets, and technical remedies, are briefly reviewed and discussed. Moreover, three typical instantiations are involved to uncover the interactions of few/zero-shot learning with visual semantic segmentation, including image semantic segmentation, video object segmentation, and 3D segmentation. Finally, the future challenges of few/zero-shot visual semantic segmentation are discussed.

cs.CV↗

arXiv2022

Stability of Multi-Dimensional Switched Systems with an Application to Open Multi-Agent Systems

A multi-dimensional switched system or multi-mode multi-dimensional ($M^3D$) system extends the classic switched system by allowing different subsystem dimensions. The stability problem of the $M^3D$ system, whose state transitions at switching instants can be discontinuous due to the dimension-varying feature, is studied. The discontinuous state transition is formulated by an affine map that captures both the dimension variations and the state impulses, with no extra constraint imposed. In the presence of unstable subsystems, the general criteria featuring a series of Lyapunov-like conditions for the practical and asymptotic stability properties of the $M^3D$ system are provided under the proposed slow/fast transition-dependent average dwell time framework. Then, by considering linear subsystems, we propose a class of parametric multiple Lyapunov functions to verify the obtained Lyapunov-like stability conditions and explicitly reveal a connection between the practical/asymptotic stability property and the non-vanishing/vanishing property of the impulsive effects in the state transition process. Further, the obtained stability results for the $M^3D$ system are applied to the consensus problem of the open multi-agent system (MAS), whose network topology can be switching and size-varying due to the migrations of agents. It shows that through a proper transformation, the seeking of the (practical) consensus performance of the open MAS with disconnected digraphs boils down to that of the (practical) stability property of an $M^3D$ system with unstable subsystems.

eess.SY↗

arXiv2022

Rethinking Individual Global Max in Cooperative Multi-Agent Reinforcement Learning

In cooperative multi-agent reinforcement learning, centralized training and decentralized execution (CTDE) has achieved remarkable success. Individual Global Max (IGM) decomposition, which is an important element of CTDE, measures the consistency between local and joint policies. The majority of IGM-based research focuses on how to establish this consistent relationship, but little attention has been paid to examining IGM's potential flaws. In this work, we reveal that the IGM condition is a lossy decomposition, and the error of lossy decomposition will accumulated in hypernetwork-based methods. To address the above issue, we propose to adopt an imitation learning strategy to separate the lossy decomposition from Bellman iterations, thereby avoiding error accumulation. The proposed strategy is theoretically proved and empirically verified on the StarCraft Multi-Agent Challenge benchmark problem with zero sight view. The results also confirm that the proposed method outperforms state-of-the-art IGM-based approaches.

cs.MA↗

arXiv2022

MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer

Self-supervised monocular depth estimation is an attractive solution that does not require hard-to-source depth labels for training. Convolutional neural networks (CNNs) have recently achieved great success in this task. However, their limited receptive field constrains existing network architectures to reason only locally, dampening the effectiveness of the self-supervised paradigm. In the light of the recent successes achieved by Vision Transformers (ViTs), we propose MonoViT, a brand-new framework combining the global reasoning enabled by ViT models with the flexibility of self-supervised monocular depth estimation. By combining plain convolutions with Transformer blocks, our model can reason locally and globally, yielding depth prediction at a higher level of detail and accuracy, allowing MonoViT to achieve state-of-the-art performance on the established KITTI dataset. Moreover, MonoViT proves its superior generalization capacities on other datasets such as Make3D and DrivingStereo.

cs.CV↗

arXiv2022

Unsupervised Monocular Depth Estimation in Highly Complex Environments

With the development of computational intelligence algorithms, unsupervised monocular depth and pose estimation framework, which is driven by warped photometric consistency, has shown great performance in the daytime scenario. While in some challenging environments, like night and rainy night, the essential photometric consistency hypothesis is untenable because of the complex lighting and reflection, so that the above unsupervised framework cannot be directly applied to these complex scenarios. In this paper, we investigate the problem of unsupervised monocular depth estimation in highly complex scenarios and address this challenging problem by adopting an image transfer-based domain adaptation framework. We adapt the depth model trained on day-time scenarios to be applicable to night-time scenarios, and constraints on both feature space and output space promote the framework to learn the key features for depth decoding. Meanwhile, we further tackle the effects of unstable image transfer quality on domain adaptation, and an image adaptation approach is proposed to evaluate the quality of transferred images and re-weight the corresponding losses, so as to improve the performance of the adapted depth model. Extensive experiments show the effectiveness of the proposed unsupervised framework in estimating the dense depth map from highly complex images.

cs.CV↗

arXiv2022

Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey

Autonomous systems possess the features of inferring their own state, understanding their surroundings, and performing autonomous navigation. With the applications of learning systems, like deep learning and reinforcement learning, the visual-based self-state estimation, environment perception and navigation capabilities of autonomous systems have been efficiently addressed, and many new learning-based algorithms have surfaced with respect to autonomous visual perception and navigation. In this review, we focus on the applications of learning-based monocular approaches in ego-motion perception, environment perception and navigation in autonomous systems, which is different from previous reviews that discussed traditional methods. First, we delineate the shortcomings of existing classical visual simultaneous localization and mapping (vSLAM) solutions, which demonstrate the necessity to integrate deep learning techniques. Second, we review the visual-based environmental perception and understanding methods based on deep learning, including deep learning-based monocular depth estimation, monocular ego-motion prediction, image enhancement, object detection, semantic segmentation, and their combinations with traditional vSLAM frameworks. Then, we focus on the visual navigation based on learning systems, mainly including reinforcement learning and deep reinforcement learning. Finally, we examine several challenges and promising directions discussed and concluded in related research of learning systems in the era of computer science and robotics.

cs.CV↗

arXiv2022

Sat2lod2: A Software For Automated Lod-2 Modeling From Satellite-Derived Orthophoto And Digital Surface Model

Deriving LoD2 models from orthophoto and digital surface models (DSM) reconstructed from satellite images is a challenging task. Existing solutions are mostly system approaches that require complicated step-wise processes, including not only heuristic geometric operations, but also high-level steps such as machine learning-based semantic segmentation and building detection. Here in this paper, we describe an open-source tool, called SAT2LOD2, built based on a minorly modified version of our recently published work. SAT2LoD2 is a fully open-source and GUI (Graphics User Interface) based software, coded in Python, which takes an orthophoto and DSM as inputs, and outputs individual building models, and it can additionally take road network shapefiles, and customized classification maps to further improve the reconstruction results. We further improve the robustness of the method by 1) intergrading building segmentation based on HRNetV2 into our software; and 2) having implemented a decision strategy to identify complex buildings and directly generate mesh to avoid erroneous LoD2 reconstruction from a system point of view. The software can process a moderate level of data (around 5000*5000 size of orthophoto and DSM) using a PC with a graphics card supporting CUDA. Furthermore, the GUI is self-contained and stores the intermediate processing results facilitating researchers to learn the process easily and reuse intermediate files as needed. The updated codes and software are available under this GitHub page: https://github.com/GDAOSU/LOD2BuildingModel.

cs.CV↗

arXiv2022

Learn to Adapt for Monocular Depth Estimation

Monocular depth estimation is one of the fundamental tasks in environmental perception and has achieved tremendous progress in virtue of deep learning. However, the performance of trained models tends to degrade or deteriorate when employed on other new datasets due to the gap between different datasets. Though some methods utilize domain adaptation technologies to jointly train different domains and narrow the gap between them, the trained models cannot generalize to new domains that are not involved in training. To boost the transferability of depth estimation models, we propose an adversarial depth estimation task and train the model in the pipeline of meta-learning. Our proposed adversarial task mitigates the issue of meta-overfitting, since the network is trained in an adversarial manner and aims to extract domain invariant representations. In addition, we propose a constraint to impose upon cross-task depth consistency to compel the depth estimation to be identical in different adversarial tasks, which improves the performance of our method and smoothens the training process. Experiments demonstrate that our method adapts well to new datasets after few training steps during the test procedure.

cs.CV↗

arXiv2022

Aggressive Quadrotor Flight Using Curiosity-Driven Reinforcement Learning

The ability to perform aggressive movements, which are called aggressive flights, is important for quadrotors during navigation. However, aggressive quadrotor flights are still a great challenge to practical applications. The existing solutions to aggressive flights heavily rely on a predefined trajectory, which is a time-consuming preprocessing step. To avoid such path planning, we propose a curiosity-driven reinforcement learning method for aggressive flight missions and a similarity-based curiosity module is introduced to speed up the training procedure. A branch structure exploration (BSE) strategy is also applied to guarantee the robustness of the policy and to ensure the policy trained in simulations can be performed in real-world experiments directly. The experimental results in simulations demonstrate that our reinforcement learning algorithm performs well in aggressive flight tasks, speeds up the convergence process and improves the robustness of the policy. Besides, our algorithm shows a satisfactory simulated to real transferability and performs well in real-world experiments.

cs.RO↗

arXiv2022

Event-Triggered Optimal Attitude Consensus of Multiple Rigid Body Networks with Unknown Dynamics

In this paper, an event-triggered Reinforcement Learning (RL) method is proposed for the optimal attitude consensus of multiple rigid body networks with unknown dynamics. Firstly, the consensus error is constructed through the attitude dynamics. According to the Bellman optimality principle, the implicit form of the optimal controller and the corresponding Hamilton-Jacobi-Bellman (HJB) equations are obtained. Because of the augmented system, the optimal controller can be obtained directly without relying on the system dynamics. Secondly, the self-triggered mechanism is applied to reduce the computing and communication burden when updating the controller. In order to address the problem that the HJB equations are difficult to solve analytically, a RL method which only requires measurement data at the event-triggered instants is proposed. For each agent, only one neural network is designed to approximate the optimal value function. Each neural network is updated only at the event triggered instants. Meanwhile, the Uniformly Ultimately Bounded (UUB) of the closed-loop system is obtained, and the Zeno behavior is also avoided. Finally, the simulation results on a multiple rigid body network demonstrate the validity of the proposed method.

math.OC↗

arXiv2021

Masked GANs for Unsupervised Depth and Pose Prediction with Scale Consistency

Previous work has shown that adversarial learning can be used for unsupervised monocular depth and visual odometry (VO) estimation, in which the adversarial loss and the geometric image reconstruction loss are utilized as the mainly supervisory signals to train the whole unsupervised framework. However, the performance of the adversarial framework and image reconstruction is usually limited by occlusions and the visual field changes between frames. This paper proposes a masked generative adversarial network (GAN) for unsupervised monocular depth and ego-motion estimation.The MaskNet and Boolean mask scheme are designed in this framework to eliminate the effects of occlusions and impacts of visual field changes on the reconstruction loss and adversarial loss, respectively. Furthermore, we also consider the scale consistency of our pose network by utilizing a new scale-consistency loss, and therefore, our pose network is capable of providing the full camera trajectory over a long monocular sequence. Extensive experiments on the KITTI dataset show that each component proposed in this paper contributes to the performance, and both our depth and trajectory predictions achieve competitive performance on the KITTI and Make3D datasets.

cs.CV↗

arXiv2021

Deep Direct Visual Odometry

Traditional monocular direct visual odometry (DVO) is one of the most famous methods to estimate the ego-motion of robots and map environments from images simultaneously. However, DVO heavily relies on high-quality images and accurate initial pose estimation during tracking. With the outstanding performance of deep learning, previous works have shown that deep neural networks can effectively learn 6-DoF (Degree of Freedom) poses between frames from monocular image sequences in the unsupervised manner. However, these unsupervised deep learning-based frameworks cannot accurately generate the full trajectory of a long monocular video because of the scale-inconsistency between each pose. To address this problem, we use several geometric constraints to improve the scale-consistency of the pose network, including improving the previous loss function and proposing a novel scale-to-trajectory constraint for unsupervised training. We call the pose network trained by the proposed novel constraint as TrajNet. In addition, a new DVO architecture, called deep direct sparse odometry (DDSO), is proposed to overcome the drawbacks of the previous direct sparse odometry (DSO) framework by embedding TrajNet. Extensive experiments on the KITTI dataset show that the proposed constraints can effectively improve the scale-consistency of TrajNet when compared with previous unsupervised monocular methods, and integration with TrajNet makes the initialization and tracking of DSO more robust and accurate.

cs.CV↗

arXiv2021

Optimal Stationary State Estimation Over Multiple Markovian Packet Drop Channels

In this paper, we investigate the state estimation problem over multiple Markovian packet drop channels. In this problem setup, a remote estimator receives measurement data transmitted from multiple sensors over individual channels. By the method of Markovian jump linear systems, an optimal stationary estimator that minimizes the error variance in the steady state is obtained, based on the mean-square (MS) stabilizing solution to the coupled algebraic Riccati equations. An explicit necessary and sufficient condition is derived for the existence of the MS stabilizing solution, which coincides with that of the standard Kalman filter. More importantly, we provide a sufficient condition under which the MS detectability with multiple Markovian packet drop channels can be decoupled, and propose a locally optimal stationary estimator but computationally more tractable. Analytic sufficient and necessary MS detectability conditions are presented for the decoupled subsystems subsequently. Finally, numerical simulations are conducted to illustrate the results on the MS stabilizing solution, the MS detectability, and the performance of the optimal and locally optimal stationary estimators.

eess.SY↗

arXiv2020

Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency

Semantic segmentation and depth completion are two challenging tasks in scene understanding, and they are widely used in robotics and autonomous driving. Although several works are proposed to jointly train these two tasks using some small modifications, like changing the last layer, the result of one task is not utilized to improve the performance of the other one despite that there are some similarities between these two tasks. In this paper, we propose multi-task generative adversarial networks (Multi-task GANs), which are not only competent in semantic segmentation and depth completion, but also improve the accuracy of depth completion through generated semantic images. In addition, we improve the details of generated semantic images based on CycleGAN by introducing multi-scale spatial pooling blocks and the structural similarity reconstruction loss. Furthermore, considering the inner consistency between semantic and geometric structures, we develop a semantic-guided smoothness loss to improve depth completion results. Extensive experiments on Cityscapes dataset and KITTI depth completion benchmark show that the Multi-task GANs are capable of achieving competitive performance for both semantic segmentation and depth completion tasks.

cs.CV↗

arXiv2020

Monocular Depth Estimation Based On Deep Learning: An Overview

Depth information is important for autonomous systems to perceive environments and estimate their own state. Traditional depth estimation methods, like structure from motion and stereo vision matching, are built on feature correspondences of multiple viewpoints. Meanwhile, the predicted depth maps are sparse. Inferring depth information from a single image (monocular depth estimation) is an ill-posed problem. With the rapid development of deep neural networks, monocular depth estimation based on deep learning has been widely studied recently and achieved promising performance in accuracy. Meanwhile, dense depth maps are estimated from single images by deep neural networks in an end-to-end manner. In order to improve the accuracy of depth estimation, different kinds of network frameworks, loss functions and training strategies are proposed subsequently. Therefore, we survey the current monocular depth estimation methods based on deep learning in this review. Initially, we conclude several widely used datasets and evaluation indicators in deep learning-based depth estimation. Furthermore, we review some representative existing methods according to different training manners: supervised, unsupervised and semi-supervised. Finally, we discuss the challenges and provide some ideas for future researches in monocular depth estimation.

cs.CV↗

arXiv2020

When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey

With widespread applications of artificial intelligence (AI), the capabilities of the perception, understanding, decision-making and control for autonomous systems have improved significantly in the past years. When autonomous systems consider the performance of accuracy and transferability, several AI methods, like adversarial learning, reinforcement learning (RL) and meta-learning, show their powerful performance. Here, we review the learning-based approaches in autonomous systems from the perspectives of accuracy and transferability. Accuracy means that a well-trained model shows good results during the testing phase, in which the testing set shares a same task or a data distribution with the training set. Transferability means that when a well-trained model is transferred to other testing domains, the accuracy is still good. Firstly, we introduce some basic concepts of transfer learning and then present some preliminaries of adversarial learning, RL and meta-learning. Secondly, we focus on reviewing the accuracy or transferability or both of them to show the advantages of adversarial learning, like generative adversarial networks (GANs), in typical computer vision tasks in autonomous systems, including image style transfer, image superresolution, image deblurring/dehazing/rain removal, semantic segmentation, depth estimation, pedestrian detection and person re-identification (re-ID). Then, we further review the performance of RL and meta-learning from the aspects of accuracy or transferability or both of them in autonomous systems, involving pedestrian tracking, robot navigation and robotic manipulation. Finally, we discuss several challenges and future topics for using adversarial learning, RL and meta-learning in autonomous systems.

cs.LG↗

arXiv2019

Leader Selection in Multi-Agent Networks with Switching Topologies via Submodular Optimization

In leader-follower multi-agent networks with switching topologies, choosing a subset of agents as leaders is a critical step to achieve desired performances. In this paper, we concentrate on the problem of selecting a minimum-size set of leaders that ensure the tracking of a reference signal in a highorder linear multi-agent network with a set of given topology dependent dwell time (TDDT). First, we derive a sufficient condition that guarantees the states of all agents converging to an expected state trajectory. Second, by exploiting submodular optimization method, we formulate the problem of identifying a minimal leader set which satisfies the proposed sufficient condition. Third, we present an algorithm with the provable optimality bound to solve the formulated problem. Finally, several numerical examples are provided to verify the effectiveness of the designed selection scheme.

cs.MA↗