Searcharxiv⌕ Search

arXiv subjects

Shiyu Zhao

Publications and source records attributed to Shiyu Zhao.

69 records · Page 4Linked to original sources
arXiv2022

Global Matching with Overlapping Attention for Optical Flow Estimation

Optical flow estimation is a fundamental task in computer vision. Recent direct-regression methods using deep neural networks achieve remarkable performance improvement. However, they do not explicitly capture long-term motion correspondences and thus cannot handle large motions effectively. In this paper, inspired by the traditional matching-optimization methods where matching is introduced to handle large displacements before energy-based optimizations, we introduce a simple but effective global matching step before the direct regression and develop a learning-based matching-optimization framework, namely GMFlowNet. In GMFlowNet, global matching is efficiently calculated by applying argmax on 4D cost volumes. Additionally, to improve the matching quality, we propose patch-based overlapping attention to extract large context features. Extensive experiments demonstrate that GMFlowNet outperforms RAFT, the most popular optimization-only method, by a large margin and achieves state-of-the-art performance on standard benchmarks. Thanks to the matching and overlapping attention, GMFlowNet obtains major improvements on the predictions for textureless regions and large motions. Our code is made publicly available at https://github.com/xiaofeng94/GMFlowNet

cs.CV↗
arXiv2021

Deep Animation Video Interpolation in the Wild

In the animation industry, cartoon videos are usually produced at low frame rate since hand drawing of such frames is costly and time-consuming. Therefore, it is desirable to develop computational models that can automatically interpolate the in-between animation frames. However, existing video interpolation methods fail to produce satisfying results on animation data. Compared to natural videos, animation videos possess two unique characteristics that make frame interpolation difficult: 1) cartoons comprise lines and smooth color pieces. The smooth areas lack textures and make it difficult to estimate accurate motions on animation videos. 2) cartoons express stories via exaggeration. Some of the motions are non-linear and extremely large. In this work, we formally define and study the animation video interpolation problem for the first time. To address the aforementioned challenges, we propose an effective framework, AnimeInterp, with two dedicated modules in a coarse-to-fine manner. Specifically, 1) Segment-Guided Matching resolves the "lack of textures" challenge by exploiting global matching among color pieces that are piece-wise coherent. 2) Recurrent Flow Refinement resolves the "non-linear and extremely large motion" challenge by recurrent predictions using a transformer-like architecture. To facilitate comprehensive training and evaluations, we build a large-scale animation triplet dataset, ATD-12K, which comprises 12,000 triplets with rich annotations. Extensive experiments demonstrate that our approach outperforms existing state-of-the-art interpolation methods for animation videos. Notably, AnimeInterp shows favorable perceptual quality and robustness for animation scenarios in the wild. The proposed dataset and code are available at https://github.com/lisiyao21/AnimeInterp/.

cs.CV↗
arXiv2018

Infinitesimal Weak Rigidity, Formation Control of Three Agents, and Extension to 3-dimensional Space

In this paper, we introduce new concepts of weak rigidity matrix and infinitesimal weak rigidity for planar frameworks. The weak rigidity matrix is used to directly check if a framework is infinitesimally weakly rigid while previous work can check a weak rigidity of a framework indirectly. An infinitesimal weak rigidity framework can be uniquely determined up to a translation and a rotation (and a scaling also when the framework does not include any edge) by its inter-neighbor distances and angles. We apply the new concepts to a three-agent formation control problem with a gradient control law, and prove instability of the control system at any incorrect equilibrium point and convergence to a desired target formation. Also, we propose a modified Henneberg construction, which is a technique to generate minimally rigid (or weakly rigid) graphs. Finally, we extend the concept of the weak rigidity in R^2 to the concept in R^3.

eess.SY↗
arXiv2018

Bearing Rigidity Theory and its Applications for Control and Estimation of Network Systems: Life Beyond Distance Rigidity

The problem of distributed control and estimation for multi-agent systems with limited sensing capabilities is a practical challenge motivated by incomplete and imperfect sensing. This article addresses an important case where each agent in a network can only sense the relative bearings to their nearest neighbors. The study of this topic is motivated mainly by the rapid development of bearing-only sensors such as optical cameras or sensor arrays. This article provides a tutorial review on this topic focusing on the problems of formation control and network localization. A key component of this review is a presentation of the recently developed bearing rigidity theory, which defines a necessary architectural feature of multi-agent systems aiming to solve these two problems. This article presents a high-level summary of recently developed algorithms solving these problems, various simulation examples, and discussions pointing to the relevant literature and important remaining challenges in this area.

eess.SY↗
arXiv2017

Laman Graphs are Generically Bearing Rigid in Arbitrary Dimensions

This paper addresses the problem of constructing bearing rigid networks in arbitrary dimensions. We first show that the bearing rigidity of a network is a generic property that is critically determined by the underlying graph of the network. A new notion termed generic bearing rigidity is defined for graphs. If the underlying graph of a network is generically bearing rigid, then the network is bearing rigid for almost all configurations; otherwise, the network is not bearing rigid for any configuration. As a result, the key to construct bearing rigid networks is to construct generically bearing rigid graphs. The main contribution of this paper is to prove that Laman graphs, which can be generated by the Henneberg construction, are generically bearing rigid in arbitrary dimensions. As a consequence, if the underlying graph of a network is Laman, the network is bearing rigid for almost all configurations in arbitrary dimensions.

eess.SY↗
arXiv2016

Localizability and Distributed Protocols for Bearing-Based Network Localization in Arbitrary Dimensions

This paper addresses the problem of bearing-based network localization, which aims to localize all the nodes in a static network given the locations of a subset of nodes termed anchors and inter-node bearings measured in a common reference frame. The contributions of the paper are twofold. Firstly, we propose necessary and sufficient conditions for network localizability with both algebraic and rigidity theoretic interpretations. The analysis of the localizability heavily relies on the recently developed bearing rigidity theory and a special matrix termed the bearing Laplacian. Secondly, we propose a linear distributed protocol for bearing-based network localization. The protocol can globally localize a network if and only if the network is localizable. The sensitivity of the protocol to constant measurement errors is also analyzed. One novelty of this work is that the localizability analysis and localization protocol are applicable to networks in arbitrary dimensional spaces.

math.OC↗
arXiv2015

Translational and Scaling Formation Maneuver Control via a Bearing-Based Approach

This paper studies distributed maneuver control of multi-agent formations in arbitrary dimensions. The objective is to control the translation and scale of the formation while maintaining the desired formation pattern. Unlike conventional approaches where the target formation is defined by relative positions or distances, we propose a novel bearing-based approach where the target formation is defined by inter-neighbor bearings. Since the bearings are invariant to the translation and scale of the formation, the bearing-based approach provides a simple solution to the problem of translational and scaling formation maneuver control. Linear formation control laws for double-integrator dynamics are proposed and the global formation stability is analyzed. This paper also studies bearing-based formation control in the presence of practical problems including input disturbances, acceleration saturation, and collision avoidance. The theoretical results are illustrated with numerical simulations.

eess.SY↗
arXiv2015

Bearing-Based Formation Stabilization with Directed Interaction Topologies

This paper studies the problem of stabilizing target formations specified by inter-neighbor bearings with relative position measurements. While the undirected case has been studied in the existing works, this paper focuses on the case where the interaction topology is directed. It is shown that a linear distributed control law, which was proposed previously for undirected cases, can still be applied to the directed case. The formation stability in the directed case, however, relies on a new notion termed bearing persistence, which describes whether or not the directed underlying graph is persistent with the bearing rigidity of a formation. If a target formation is not bearing persistent, undesired equilibriums will appear and global formation stability cannot be guaranteed. The notion of bearing persistence is defined by the bearing Laplacian matrix and illustrated by simulation examples.

eess.SY↗
arXiv2015

Bearing-Based Formation Maneuvering

This paper studies the problem of multi-agent formation maneuver control where both of the centroid and scale of a formation are required to track given velocity references while maintaining the formation shape. Unlike the conventional approaches where the target formation is defined by inter-neighbor relative positions or distances, we propose a bearing-based approach where the target formation is defined by inter-neighbor bearings. Due to the invariance of the bearings, the bearing-based approach provides a natural solution to formation scale control. We assume the dynamics of each agent as a single integrator and propose a globally stable proportional-integral formation maneuver control law. It is shown that at least two leaders are required to collaborate in order to control the centroid and scale of the formation whereas the followers are not required to have access to any global information, such as the velocities of the leaders.

eess.SY↗
arXiv2015

Bearing Rigidity and Almost Global Bearing-Only Formation Stabilization

A fundamental problem that the bearing rigidity theory studies is to determine when a framework can be uniquely determined up to a translation and a scaling factor by its inter-neighbor bearings. While many previous works focused on the bearing rigidity of two-dimensional frameworks, a first contribution of this paper is to extend these results to arbitrary dimensions. It is shown that a framework in an arbitrary dimension can be uniquely determined up to a translation and a scaling factor by the bearings if and only if the framework is infinitesimally bearing rigid. In this paper, the proposed bearing rigidity theory is further applied to the bearing-only formation stabilization problem where the target formation is defined by inter-neighbor bearings and the feedback control uses only bearing measurements. Nonlinear distributed bearing-only formation control laws are proposed for the cases with and without a global orientation. It is proved that the control laws can almost globally stabilize infinitesimally bearing rigid formations. Numerical simulations are provided to support the analysis.

eess.SY↗
arXiv2015

Bearing-Based Distributed Control and Estimation of Multi-Agent Systems

This paper studies the distributed control and estimation of multi-agent systems based on bearing information. In particular, we consider two problems: (i) the distributed control of bearing-constrained formations using relative position measurements and (ii) the distributed localization of sensor networks using bearing measurements. Both of the two problems are considered in arbitrary dimensional spaces. The analyses of the two problems rely on the recently developed bearing rigidity theory. We show that the two problems have the same mathematical formulation and can be solved by identical protocols. The proposed controller and estimator can globally solve the two problems without ambiguity. The results are supported with illustrative simulations.

eess.SY↗
arXiv2013

Finite-time Stabilization of Circular Formations using Bearing-only Measurements

This paper studies decentralized formation control of multiple vehicles when each vehicle can only measure the local bearings of their neighbors by using bearing-only sensors. Since the inter-vehicle distance cannot be measured, the target formation involves no distance constraints. More specifically, the target formation considered in this paper is an angle-constrained circular formation, where each vehicle has exactly two neighbors and the angle at each vehicle subtended by its two neighbors is pre-specified. To stabilize the target formation, we propose a discontinuous control law that only requires the sign information of the angle errors. Due to the discontinuity of the proposed control law, the stability of the closed-loop system is analyzed by employing a locally Lipschitz Lyapunov function and nonsmooth analysis tools. We prove that the target formation is locally finite-time stable with collision avoidance guaranteed. The evolution of the vehicle positions in the plane is also characterized.

eess.SY↗
arXiv2012

Distributed Control of Angle-constrained Circular Formations using Bearing-only Measurements

This paper studies distributed formation control of multiple agents in the plane using bearing-only measurements. It is assumed that each agent only measures the local bearings of their neighbor agents. The target formation considered in this paper is a circular formation, where each agent has exactly two neighbors. In the target formation, the angle subtended at each agent by their two neighbors is specified. We propose a distributed control law that stabilizes angle-constrained target formations merely using local bearing measurements. The stability of the target formation is analyzed based on Lyapunov approaches. We present a unified proof to show that our control law not only can ensure local exponential stability but also can give local finite-time stability. The exponential or finite-time stability can be easily switched by tuning a parameter in the control law.

eess.SY↗
arXiv2012

Optimal Sensor Placement for Target Localization and Tracking in 2D and 3D

This paper analytically characterizes optimal sensor placements for target localization and tracking in 2D and 3D. Three types of sensors are considered: bearing-only, range-only, and received-signal-strength. The optimal placement problems of the three sensor types are formulated as an identical parameter optimization problem and consequently analyzed in a unified framework. Recently developed frame theory is applied to the optimality analysis. We prove necessary and sufficient conditions for optimal placements in 2D and 3D. A number of important analytical properties of optimal placements are further explored. In order to verify the analytical analysis, we present a gradient control law that can numerically construct generic optimal placements.

math.OC↗