arXiv subjects

Peng Zhou

Publications and source records attributed to Peng Zhou.

At least 145 records · Page 8Linked to original sources

arXiv2021

LaSeSOM: A Latent and Semantic Representation Framework for Soft Object Manipulation

Soft object manipulation has recently gained popularity within the robotics community due to its potential applications in many economically important areas. Although great progress has been recently achieved in these types of tasks, most state-of-the-art methods are case-specific; They can only be used to perform a single deformation task (e.g. bending), as their shape representation algorithms typically rely on "hard-coded" features. In this paper, we present LaSeSOM, a new feedback latent representation framework for semantic soft object manipulation. Our new method introduces internal latent representation layers between low-level geometric feature extraction and high-level semantic shape analysis; This allows the identification of each compressed semantic function and the formation of a valid shape classifier from different feature extraction levels. The proposed latent framework makes soft object representation more generic (independent from the object's geometry and its mechanical properties) and scalable (it can work with 1D/2D/3D tasks). Its high-level semantic layer enables to perform (quasi) shape planning tasks with soft objects, a valuable and underexplored capability in many soft manipulation tasks. To validate this new methodology, we report a detailed experimental study with robotic manipulators.

cs.RO↗

arXiv2021

CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis

The style-based GAN (StyleGAN) architecture achieved state-of-the-art results for generating high-quality images, but it lacks explicit and precise control over camera poses. The recently proposed NeRF-based GANs made great progress towards 3D-aware generators, but they are unable to generate high-quality images yet. This paper presents CIPS-3D, a style-based, 3D-aware generator that is composed of a shallow NeRF network and a deep implicit neural representation (INR) network. The generator synthesizes each pixel value independently without any spatial convolution or upsampling operation. In addition, we diagnose the problem of mirror symmetry that implies a suboptimal solution and solve it by introducing an auxiliary discriminator. Trained on raw, single-view images, CIPS-3D sets new records for 3D-aware image synthesis with an impressive FID of 6.97 for images at the $256\times256$ resolution on FFHQ. We also demonstrate several interesting directions for CIPS-3D such as transfer learning and 3D-aware face stylization. The synthesis results are best viewed as videos, so we recommend the readers to check our github project at https://github.com/PeterouZh/CIPS-3D

cs.CV↗

arXiv2021

Keypoint-Based Bimanual Shaping of Deformable Linear Objects under Environmental Constraints using Hierarchical Action Planning

This paper addresses the problem of contact-based manipulation of deformable linear objects (DLOs) towards desired shapes with a dual-arm robotic system. To alleviate the burden of high-dimensional continuous state-action spaces, we model the DLO as a kinematic multibody system via our proposed keypoint detection network. This new perception network is trained on a synthetic labeled image dataset and transferred to real manipulation scenarios without conducting any manual annotations. Our goal-conditioned policy can efficiently learn to rearrange the configuration of the DLO based on the detected keypoints. The proposed hierarchical action framework tackles the manipulation problem in a coarse-to-fine manner (with high-level task planning and low-level motion control) by leveraging on two action primitives. The identification of deformation properties is avoided since the algorithm replans its motion after each bimanual execution. The conducted experimental results reveal that our method achieves high performance in state representation of the DLO, and is robust to uncertain environmental constraints.

cs.RO↗

arXiv2021

Learning Cloth Folding Tasks with Refined Flow Based Spatio-Temporal Graphs

Cloth folding is a widespread domestic task that is seemingly performed by humans but which is highly challenging for autonomous robots to execute due to the highly deformable nature of textiles; It is hard to engineer and learn manipulation pipelines to efficiently execute it. In this paper, we propose a new solution for robotic cloth folding (using a standard folding board) via learning from demonstrations. Our demonstration video encoding is based on a high-level abstraction, namely, a refined optical flow-based spatiotemporal graph, as opposed to a low-level encoding such as image pixels. By constructing a new spatiotemporal graph with an advanced visual corresponding descriptor, the policy learning can focus on key points and relations with a 3D spatial configuration, which allows to quickly generalize across different environments. To further boost the policy searching, we combine optical flow and static motion saliency maps to discriminate the dominant motions for better handling the system dynamics in real-time, which aligns with the attentional motion mechanism that dominates the human imitation process. To validate the proposed approach, we analyze the manual folding procedure and developed a custom-made end-effector to efficiently interact with the folding board. Multiple experiments on a real robotic platform were conducted to validate the effectiveness and robustness of the proposed method.

cs.RO↗

arXiv2021

A Novel Approach to Model the Kinematics of Human Fingers Based on an Elliptic Multi-Joint Configuration

In this paper, we present a novel kinematic model of the human phalanges based on the elliptical motion of their joints. The presence of the soft elastic tissues and the general anatomical structure of the hand joints highly affect the relative movement of the bones. Commonly used assumption of circular trajectories simplifies the designing process but leads to divergence with the actual hand behavior. The advantages of the proposed model are demonstrated through the comparison with the conventional revolute joint model. Conducted simulations and experiments validate designed forward and inverse kinematic algorithms. Obtained results show a high performance of the model in mimicking the human fingertip motion trajectory.

cs.RO↗

arXiv2021

Contour Moments Based Manipulation of Composite Rigid-Deformable Objects with Finite Time Model Estimation and Shape/Position Control

The robotic manipulation of composite rigid-deformable objects (i.e. those with mixed non-homogeneous stiffness properties) is a challenging problem with clear practical applications that, despite the recent progress in the field, it has not been sufficiently studied in the literature. To deal with this issue, in this paper we propose a new visual servoing method that has the capability to manipulate this broad class of objects (which varies from soft to rigid) with the same adaptive strategy. To quantify the object's infinite-dimensional configuration, our new approach computes a compact feedback vector of 2D contour moments features. A sliding mode control scheme is then designed to simultaneously ensure the finite-time convergence of both the feedback shape error and the model estimation error. The stability of the proposed framework (including the boundedness of all the signals) is rigorously proved with Lyapunov theory. Detailed simulations and experiments are presented to validate the effectiveness of the proposed approach. To the best of the author's knowledge, this is the first time that contour moments along with finite-time control have been used to solve this difficult manipulation problem.

cs.RO↗

arXiv2021

Zero loci of Bernstein-Sato ideals -- II

We have recently proved a precise relation between Bernstein-Sato ideals of collections of polynomials and monodromy of generalized nearby cycles. In this article we extend this result to other ideals of Bernstein-Sato type.

math.AG↗

arXiv2021

Towards Latent Space Based Manipulation of Elastic Rods using Autoencoder Models and Robust Centerline Extractions

The automatic shape control of deformable objects is a challenging (and currently hot) manipulation problem due to their high-dimensional geometric features and complex physical properties. In this study, a new methodology to manipulate elastic rods automatically into 2D desired shapes is presented. An efficient vision-based controller that uses a deep autoencoder network is designed to compute a compact representation of the object's infinite-dimensional shape. An online algorithm that approximates the sensorimotor mapping between the robot's configuration and the object's shape features is used to deal with the latter's (typically unknown) mechanical properties. The proposed approach computes the rod's centerline from raw visual data in real-time by introducing an adaptive algorithm on the basis of a self-organizing network. Its effectiveness is thoroughly validated with simulations and experiments.

cs.RO↗

arXiv2021

Searching towards Class-Aware Generators for Conditional Generative Adversarial Networks

Conditional Generative Adversarial Networks (cGAN) were designed to generate images based on the provided conditions, \eg, class-level distributions. However, existing methods have used the same generating architecture for all classes. This paper presents a novel idea that adopts NAS to find a distinct architecture for each class. The search space contains regular and class-modulated convolutions, where the latter is designed to introduce class-specific information while avoiding the reduction of training data for each class generator. The search algorithm follows a weight-sharing pipeline with mixed-architecture optimization so that the search cost does not grow with the number of classes. To learn the sampling policy, a Markov decision process is embedded into the search algorithm and a moving average is applied for better stability. We evaluate our approach on CIFAR10 and CIFAR100. Besides achieving better image generation quality in terms of FID scores, we discover several insights that are helpful in designing cGAN models. Code is available at https://github.com/PeterouZh/NAS_cGAN.

cs.CV↗

arXiv2021

Omni-GAN: On the Secrets of cGANs and Beyond

The conditional generative adversarial network (cGAN) is a powerful tool of generating high-quality images, but existing approaches mostly suffer unsatisfying performance or the risk of mode collapse. This paper presents Omni-GAN, a variant of cGAN that reveals the devil in designing a proper discriminator for training the model. The key is to ensure that the discriminator receives strong supervision to perceive the concepts and moderate regularization to avoid collapse. Omni-GAN is easily implemented and freely integrated with off-the-shelf encoding methods (e.g., implicit neural representation, INR). Experiments validate the superior performance of Omni-GAN and Omni-INR-GAN in a wide range of image generation and restoration tasks. In particular, Omni-INR-GAN sets new records on the ImageNet dataset with impressive Inception scores of 262.85 and 343.22 for the image sizes of 128 and 256, respectively, surpassing the previous records by 100+ points. Moreover, leveraging the generator prior, Omni-INR-GAN can extrapolate low-resolution images to arbitrary resolution, even up to x60+ higher resolution. Code is available.

cs.CV↗

arXiv2021

A Generalized Tunneling Current Formula for Metal/Insulator Heterojunctions under Large Bias and Finite Temperature

The Fowler-Nordheim tunneling current formula has been widely used in the design of devices based on metal/insulator (metal/semiconductor) heterojunctions with triangle potential barriers, such as the flash memory. Here we adopt the model that was used to derive the Landauer formula at finite temperature, the nearly-free electron approximation to describe the electronic states in semi-infinite metal electrode and the Wentzel-Kramers-Brillouin (WKB) approximation to describe the transmission coefficient, and derive a tunneling current formula for metal/insulator heterojunctions under large bias and electric field. In contrast to the Fowler-Nordheim formula which is the limit at zero temperature, our formula is generalized to the finite temperature (with the thermal excitation of electrons in metal electrode considered) and the potential barriers beyond triangle ones, which may be used for the design of more complicated heterojunction devices based on the carrier tunneling.

cond-mat.mes-hall↗

arXiv2021

On the Coupling Effects between Elastic and Electromagnetic Fields from the Perspective of Conservation of Energy

Coupling effects among different physical fields substantially reflect the conversion of energies from one form into another. For simple physical processes, their governing or constitutive equations all satisfy the law of conservation of energy. Then, analysis is extended to coupling effects. First, it is found for the linear direct and converse piezoelectric and piezomagnetic effects, their constitutive equations guarantee that the total energy is conserved during the process of energy conversion between the elastic and electromagnetic fields; however, energies are converted via work terms, $(β_{ijk} E_i )_{,k} v_j$ and $(γ_{ijk} H_i)_{,k} v_j$, rather than via energy terms, $β_{ijk} E_i e_{jk}$ and $γ_{ijk} H_i e_{jk}$. Second, for the generalized Villari effects, the electromagnetic energy can be treated as an extra contribution to the generalized elastic energy. Third, for electrostriction and magnetostriction, it is argued both effects are induced by the Maxwell stress; moreover, their energy is purely electromagnetic and thus both have no converse effects. During these processes, energy can be converted in three different ways, i.e., via non-potential forces, via cross-dependence of energy terms and directly via the electromagnetic interactions of ions and electrons. In the end, general coupling processes which involve elastic, electromagnetic fields and diffusion are also analyzed. The advantage of using this energy formulation is that it facilitates discussions of the conversion of energies and provides better physical insights into the mechanisms of these coupling effects.

physics.gen-ph↗

arXiv2021

Deep Video Inpainting Detection

This paper studies video inpainting detection, which localizes an inpainted region in a video both spatially and temporally. In particular, we introduce VIDNet, Video Inpainting Detection Network, which contains a two-stream encoder-decoder architecture with attention module. To reveal artifacts encoded in compression, VIDNet additionally takes in Error Level Analysis frames to augment RGB frames, producing multimodal features at different levels with an encoder. Exploring spatial and temporal relationships, these features are further decoded by a Convolutional LSTM to predict masks of inpainted regions. In addition, when detecting whether a pixel is inpainted or not, we present a quad-directional local attention module that borrows information from its surrounding pixels from four directions. Extensive experiments are conducted to validate our approach. We demonstrate, among other things, that VIDNet not only outperforms by clear margins alternative inpainting detection methods but also generalizes well on novel videos that are unseen during training.

cs.CV↗

arXiv2020

Zero loci of Bernstein-Sato ideals

We prove a conjecture of the first author relating the Bernstein-Sato ideal of a finite collection of multivariate polynomials with cohomology support loci of rank one complex local systems. This generalizes a classical theorem of Malgrange and Kashiwara relating the b-function of a multivariate polynomial with the monodromy eigenvalues on the Milnor fibers cohomology.

math.AG↗

arXiv2020

Path Planning with Automatic Seam Extraction over Point Cloud Models for Robotic Arc Welding

This paper presents a point cloud based robotic system for arc welding. Using hand gesture controls, the system scans partial point cloud views of workpiece and reconstructs them into a complete 3D model by a linear iterative closest point algorithm. Then, a bilateral filter is extended to denoise the workpiece model and preserve important geometrical information. To extract the welding seam from the model, a novel intensity-based algorithm is proposed that detects edge points and generates a smooth 6-DOF welding path. The methods are tested on multiple workpieces with different joint types and poses. Experimental results prove the robustness and efficiency of this robotic system on automatic path planning for welding applications.

cs.RO↗

arXiv2020

Variation of GIT and Variation of Lagrangian Skeletons II: Quasi-Symmetric Case

Consider $(\mathbb{C}^*)^k$ acting on $\mathbb{C}^N$ satisfying certain 'quasi-symmetric' condition which produces a class of toric Calabi-Yau GIT quotient stacks. Using subcategories of $Coh([\mathbb{C}^N / (\mathbb{C}^*)^k])$ generated by line bundles whose weights are inside certain zonotope called the 'magic window', Halpern-Leistner and Sam give a combinatorial construction of equivalences between derived categories of coherent sheaves for various GIT quotients. We apply the coherent-constructible correspondence for toric varieties to the magic windows and obtain a non-characteristic deformation of Lagrangian skeletons in $\mathbb{R}^{N-k}$ parameterized by $\mathbb{R}^k$, exhibiting derived equivalences between A-models of the various phases. Moreover, by translating the magic window zonotope in $\mathbb{R}^k$, we obtain a universal skeleton over $\mathbb{R}^k \times \mathbb{R}^k \setminus \mathcal{D}$ for some fattening of hyperplane arrangements $\mathcal{D}$, and we show that the the universal skeleton induces a local system of categories over $\mathbb{R}^k \times \mathbb{R}^k \setminus \mathcal{D}$. We also connect our results to the perverse schober structure identified by Špenko and Van den Bergh.

math.SG↗

arXiv2020

Variation of GIT and Variation of Lagrangian Skeletons I: Flip and Flop

Coherent-Constructible Correspondence for toric variety assigns to each $n$-dimensional toric variety $X_Σ$ a Lagrangian skeleton $Λ_Σ\subset T^*T^n$, such that the derived category of coherent sheaves $Coh(X_Σ)$ is equivalent to the (wrapped) constructible sheaves $Sh^w(T^n, Λ_Σ)$. In this paper, we extend this correspondence, so that flip and flop between toric varieties corresponds to variation of Lagrangian skeletons. The main idea is to translate window subcategory in variation of GIT to a window skeleton.

math.SG↗

arXiv2020

M2KD: Multi-model and Multi-level Knowledge Distillation for Incremental Learning

Incremental learning targets at achieving good performance on new categories without forgetting old ones. Knowledge distillation has been shown critical in preserving the performance on old classes. Conventional methods, however, sequentially distill knowledge only from the last model, leading to performance degradation on the old classes in later incremental learning steps. In this paper, we propose a multi-model and multi-level knowledge distillation strategy. Instead of sequentially distilling knowledge only from the last model, we directly leverage all previous model snapshots. In addition, we incorporate an auxiliary distillation to further preserve knowledge encoded at the intermediate feature levels. To make the model more memory efficient, we adapt mask based pruning to reconstruct all previous models with a small memory footprint. Experiments on standard incremental learning benchmarks show that our method preserves the knowledge on old classes better and improves the overall performance over standard distillation techniques.

cs.CV↗