arXiv subjects

Siheng Chen

Publications and source records attributed to Siheng Chen.

At least 199 records · Page 11Linked to original sources

arXiv2020

Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction

We propose novel dynamic multiscale graph neural networks (DMGNN) to predict 3D skeleton-based human motions. The core idea of DMGNN is to use a multiscale graph to comprehensively model the internal relations of a human body for motion feature learning. This multiscale graph is adaptive during training and dynamic across network layers. Based on this graph, we propose a multiscale graph computational unit (MGCU) to extract features at individual scales and fuse features across scales. The entire model is action-category-agnostic and follows an encoder-decoder framework. The encoder consists of a sequence of MGCUs to learn motion features. The decoder uses a proposed graph-based gate recurrent unit to generate future poses. Extensive experiments show that the proposed DMGNN outperforms state-of-the-art methods in both short and long-term predictions on the datasets of Human 3.6M and CMU Mocap. We further investigate the learned multiscale graphs for the interpretability. The codes could be downloaded from https://github.com/limaosen0/DMGNN.

cs.CV↗

arXiv2020

MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps

The ability to reliably perceive the environmental states, particularly the existence of objects and their motion behavior, is crucial for autonomous driving. In this work, we propose an efficient deep model, called MotionNet, to jointly perform perception and motion prediction from 3D point clouds. MotionNet takes a sequence of LiDAR sweeps as input and outputs a bird's eye view (BEV) map, which encodes the object category and motion information in each grid cell. The backbone of MotionNet is a novel spatio-temporal pyramid network, which extracts deep spatial and temporal features in a hierarchical fashion. To enforce the smoothness of predictions over both space and time, the training of MotionNet is further regularized with novel spatial and temporal consistency losses. Extensive experiments show that the proposed method overall outperforms the state-of-the-arts, including the latest scene-flow- and 3D-object-detection-based methods. This indicates the potential value of the proposed method serving as a backup to the bounding-box-based system, and providing complementary information to the motion planner in autonomous driving. Code is available at https://github.com/pxiangwu/MotionNet.

cs.CV↗

arXiv2020

Collaborative Motion Prediction via Neural Motion Message Passing

Motion prediction is essential and challenging for autonomous vehicles and social robots. One challenge of motion prediction is to model the interaction among traffic actors, which could cooperate with each other to avoid collisions or form groups. To address this challenge, we propose neural motion message passing (NMMP) to explicitly model the interaction and learn representations for directed interactions between actors. Based on the proposed NMMP, we design the motion prediction systems for two settings: the pedestrian setting and the joint pedestrian and vehicle setting. Both systems share a common pattern: we use an individual branch to model the behavior of a single actor and an interactive branch to model the interaction between actors, while with different wrappers to handle the varied input formats and characteristics. The experimental results show that both systems outperform the previous state-of-the-art methods on several existing benchmarks. Besides, we provide interpretability for interaction learning.

cs.CV↗

arXiv2020

Deterministic and stochastic control of kirigami topology

Kirigami, the creative art of paper cutting, is a promising paradigm for mechanical metamaterials. However, to make kirigami-inspired structures a reality requires controlling the topology of kirigami to achieve connectivity and rigidity. We address this question by deriving the maximum number of cuts (minimum number of links) that still allow us to preserve global rigidity and connectivity of the kirigami. A deterministic hierarchical construction method yields an efficient topological way to control both the number of connected pieces and the total degrees of freedom. A statistical approach to the control of rigidity and connectivity in kirigami with random cuts complements the deterministic pathway, and shows that both the number of connected pieces and the degrees of freedom show percolation transitions as a function of the density of cuts (links). Together this provides a general framework for the control of rigidity and connectivity in planar kirigami.

cond-mat.soft↗

arXiv2020

3D Point Cloud Processing and Learning for Autonomous Driving

We present a review of 3D point cloud processing and learning for autonomous driving. As one of the most important sensors in autonomous vehicles, light detection and ranging (LiDAR) sensors collect 3D point clouds that precisely record the external surfaces of objects and scenes. The tools for 3D point cloud processing and learning are critical to the map creation, localization, and perception modules in an autonomous vehicle. While much attention has been paid to data collected from cameras, such as images and videos, an increasing number of researchers have recognized the importance and significance of LiDAR in autonomous driving and have proposed processing and learning algorithms to exploit 3D point clouds. We review the recent progress in this research area and summarize what has been tried and what is needed for practical and safe autonomous vehicles. We also offer perspectives on open issues that are needed to be solved in the future.

cs.CV↗

arXiv2020

Damage-sensitive and domain-invariant feature extraction for vehicle-vibration-based bridge health monitoring

We introduce a physics-guided signal processing approach to extract a damage-sensitive and domain-invariant (DS & DI) feature from acceleration response data of a vehicle traveling over a bridge to assess bridge health. Motivated by indirect sensing methods' benefits, such as low-cost and low-maintenance, vehicle-vibration-based bridge health monitoring has been studied to efficiently monitor bridges in real-time. Yet applying this approach is challenging because 1) physics-based features extracted manually are generally not damage-sensitive, and 2) features from machine learning techniques are often not applicable to different bridges. Thus, we formulate a vehicle bridge interaction system model and find a physics-guided DS & DI feature, which can be extracted using the synchrosqueezed wavelet transform representing non-stationary signals as intrinsic-mode-type components. We validate the effectiveness of the proposed feature with simulated experiments. Compared to conventional time- and frequency-domain features, our feature provides the best damage quantification and localization results across different bridges in five of six experiments.

cs.CE↗

arXiv2020

Efficient and Stable Graph Scattering Transforms via Pruning

Graph convolutional networks (GCNs) have well-documented performance in various graph learning tasks, but their analysis is still at its infancy. Graph scattering transforms (GSTs) offer training-free deep GCN models that extract features from graph data, and are amenable to generalization and stability analyses. The price paid by GSTs is exponential complexity in space and time that increases with the number of layers. This discourages deployment of GSTs when a deep architecture is needed. The present work addresses the complexity limitation of GSTs by introducing an efficient so-termed pruned (p)GST approach. The resultant pruning algorithm is guided by a graph-spectrum-inspired criterion, and retains informative scattering features on-the-fly while bypassing the exponential complexity associated with GSTs. Stability of the novel pGSTs is also established when the input graph data or the network structure are perturbed. Furthermore, the sensitivity of pGST to random and localized signal perturbations is investigated analytically and experimentally. Numerical tests showcase that pGST performs comparably to the baseline GST at considerable computational savings. Furthermore, pGST achieves comparable performance to state-of-the-art GCNs in graph and 3D point cloud classification tasks. Upon analyzing the pGST pruning patterns, it is shown that graph data in different domains call for different network architectures, and that the pruning algorithm may be employed to guide the design choices for contemporary GCNs.

eess.SP↗

arXiv2019

Deep Unsupervised Learning of 3D Point Clouds via Graph Topology Inference and Filtering

We propose a deep autoencoder with graph topology inference and filtering to achieve compact representations of unorganized 3D point clouds in an unsupervised manner. Many previous works discretize 3D points to voxels and then use lattice-based methods to process and learn 3D spatial information; however, this leads to inevitable discretization errors. In this work, we handle raw 3D points without such compromise. The proposed networks follow the autoencoder framework with a focus on designing the decoder. The encoder adopts similar architectures as in PointNet. The decoder involves three novel modules. The folding module folds a canonical 2D lattice to the underlying surface of a 3D point cloud, achieving coarse reconstruction; the graph-topology-inference module learns a graph topology to represent pairwise relationships between 3D points, pushing the latent code to preserve both coordinates and pairwise relationships of points in 3D point clouds; and the graph-filtering module couples the above two modules, refining the coarse reconstruction through a learnt graph topology to obtain the final reconstruction. The proposed decoder leverages a learnable graph topology to push the codeword to preserve representative features and further improve the unsupervised-learning performance. We further provide theoretical analyses of the proposed architecture. In the experiments, we validate the proposed networks in three tasks, including 3D point cloud reconstruction, visualization, and transfer classification. The experimental results show that (1) the proposed networks outperform the state-of-the-art methods in various tasks; (2) a graph topology can be inferred as auxiliary information without specific supervision on graph topology inference; and (3) graph filtering refines the reconstruction, leading to better performances.

cs.CV↗

arXiv2019

Symbiotic Graph Neural Networks for 3D Skeleton-based Human Action Recognition and Motion Prediction

3D skeleton-based action recognition and motion prediction are two essential problems of human activity understanding. In many previous works: 1) they studied two tasks separately, neglecting internal correlations; 2) they did not capture sufficient relations inside the body. To address these issues, we propose a symbiotic model to handle two tasks jointly; and we propose two scales of graphs to explicitly capture relations among body-joints and body-parts. Together, we propose symbiotic graph neural networks, which contain a backbone, an action-recognition head, and a motion-prediction head. Two heads are trained jointly and enhance each other. For the backbone, we propose multi-branch multi-scale graph convolution networks to extract spatial and temporal features. The multi-scale graph convolution networks are based on joint-scale and part-scale graphs. The joint-scale graphs contain actional graphs, capturing action-based relations, and structural graphs, capturing physical constraints. The part-scale graphs integrate body-joints to form specific parts, representing high-level relations. Moreover, dual bone-based graphs and networks are proposed to learn complementary features. We conduct extensive experiments for skeleton-based action recognition and motion prediction with four datasets, NTU-RGB+D, Kinetics, Human3.6M, and CMU Mocap. Experiments show that our symbiotic graph neural networks achieve better performances on both tasks compared to the state-of-the-art methods.

cs.CV↗

arXiv2019

Node Attribute Generation on Graphs

Graph structured data provide two-fold information: graph structures and node attributes. Numerous graph-based algorithms rely on both information to achieve success in supervised tasks, such as node classification and link prediction. However, node attributes could be missing or incomplete, which significantly deteriorates the performance. The task of node attribute generation aims to generate attributes for those nodes whose attributes are completely unobserved. This task benefits many real-world problems like profiling, node classification and graph data augmentation. To tackle this task, we propose a deep adversarial learning based method to generate node attributes; called node attribute neural generator (NANG). NANG learns a unifying latent representation which is shared by both node attributes and graph structures and can be translated to different modalities. We thus use this latent representation as a bridge to convert information from one modality to another. We further introduce practical applications to quantify the performance of node attribute generation. Extensive experiments are conducted on four real-world datasets and the empirical results show that node attributes generated by the proposed method are high-qualitative and beneficial to other applications. The datasets and codes are available online.

stat.ML↗

arXiv2019

Large-scale 3D point cloud representations via graph inception networks with applications to autonomous driving

We present a novel graph-neural-network-based system to effectively represent large-scale 3D point clouds with the applications to autonomous driving. Many previous works studied the representations of 3D point clouds based on two approaches, voxelization, which causes discretization errors and learning, which is hard to capture huge variations in large-scale scenarios. In this work, we combine voxelization and learning: we discretize the 3D space into voxels and propose novel graph inception networks to represent 3D points in each voxel. This combination makes the system avoid discretization errors and work for large-scale scenarios. The entire system for large-scale 3D point clouds acts like the blocked discrete cosine transform for 2D images; we thus call it the point cloud neural transform (PCT). We further apply the proposed PCT to represent real-time LiDAR sweeps produced by self-driving cars and the PCT with graph inception networks significantly outperforms its competitors.

eess.SP↗

arXiv2019

Handwritten Chinese Font Generation with Collaborative Stroke Refinement

Automatic character generation is an appealing solution for new typeface design, especially for Chinese typefaces including over 3700 most commonly-used characters. This task has two main pain points: (i) handwritten characters are usually associated with thin strokes of few information and complex structure which are error prone during deformation; (ii) thousands of characters with various shapes are needed to synthesize based on a few manually designed characters. To solve those issues, we propose a novel convolutional-neural-network-based model with three main techniques: collaborative stroke refinement, using collaborative training strategy to recover the missing or broken strokes; online zoom-augmentation, taking the advantage of the content-reuse phenomenon to reduce the size of training set; and adaptive pre-deformation, standardizing and aligning the characters. The proposed model needs only 750 paired training samples; no pre-trained network, extra dataset resource or labels is needed. Experimental results show that the proposed method significantly outperforms the state-of-the-art methods under the practical restriction on handwritten font synthesis.

cs.CV↗

arXiv2019

Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition

Action recognition with skeleton data has recently attracted much attention in computer vision. Previous studies are mostly based on fixed skeleton graphs, only capturing local physical dependencies among joints, which may miss implicit joint correlations. To capture richer dependencies, we introduce an encoder-decoder structure, called A-link inference module, to capture action-specific latent dependencies, i.e. actional links, directly from actions. We also extend the existing skeleton graphs to represent higher-order dependencies, i.e. structural links. Combing the two types of links into a generalized skeleton graph, we further propose the actional-structural graph convolution network (AS-GCN), which stacks actional-structural graph convolution and temporal convolution as a basic building block, to learn both spatial and temporal features for action recognition. A future pose prediction head is added in parallel to the recognition head to help capture more detailed action patterns through self-supervision. We validate AS-GCN in action recognition using two skeleton data sets, NTU-RGB+D and Kinetics. The proposed AS-GCN achieves consistently large improvement compared to the state-of-the-art methods. As a side product, AS-GCN also shows promising results for future pose prediction.

cs.CV↗

arXiv2019

3D Point Cloud Denoising via Deep Neural Network based Local Surface Estimation

We present a neural-network-based architecture for 3D point cloud denoising called neural projection denoising (NPD). In our previous work, we proposed a two-stage denoising algorithm, which first estimates reference planes and follows by projecting noisy points to estimated reference planes. Since the estimated reference planes are inevitably noisy, multi-projection is applied to stabilize the denoising performance. NPD algorithm uses a neural network to estimate reference planes for points in noisy point clouds. With more accurate estimations of reference planes, we are able to achieve better denoising performances with only one-time projection. To the best of our knowledge, NPD is the first work to denoise 3D point clouds with deep learning techniques. To conduct the experiments, we sample 40000 point clouds from the 3D data in ShapeNet to train a network and sample 350 point clouds from the 3D data in ModelNet10 to test. Experimental results show that our algorithm can estimate normal vectors of points in noisy point clouds. Comparing to five competitive methods, the proposed algorithm achieves better denoising performance and produces much smaller variances.

cs.CV↗

arXiv2018

Weighted Multi-projection: 3D Point Cloud Denoising with Estimated Tangent Planes

As a collection of 3D points sampled from surfaces of objects, a 3D point cloud is widely used in robotics, autonomous driving and augmented reality. Due to the physical limitations of 3D sensing devices, 3D point clouds are usually noisy, which influences subsequent computations, such as surface reconstruction, recognition and many others. To denoise a 3D point cloud, we present a novel algorithm, called weighted multi-projection. Compared to many previous works on denoising, instead of directly smoothing the coordinates of 3D points, we use a two-fold smoothing: We first estimate a local tangent plane at each 3D point and then reconstruct each 3D point by weighted averaging of its projections on multiple tangent planes. We also provide the theoretical analysis for the surface normal estimation and achieve a tighter bound than in a previous work. We validate the empirical performance on the dataset of ShapeNetCore and show that weighted multi-projection outperforms its competitors in all nine classes.

cs.CG↗

arXiv2018

The eigenstructures of real (skew) circulant matrices with some applications

The circulant matrices and skew-circulant matrices are two special classes of Toeplitz matrices and play vital roles in the computation of Toeplitz matrices. In this paper, we focus on real circulant and skewcirculant matrices. We first investigate their real Schur forms, which are closely related to the family of discrete cosine transform (DCT) and discrete sine transform (DST). Using those real Schur forms, we then develop some fast algorithms for computing real circulant, skew-circulant and Toeplitz matrix-real vector multiplications. Also, we develop a DCT-DST version of circulant and skew-circulant splitting (CSCS) iteration for real positive definite Toeplitz systems. Compared with the fast Fourier transform (FFT) version of CSCS iteration, the DCTDST version is more efficient and saves a half storage. Numerical experiments are presented to illustrate the effectiveness of our method.

math.NA↗

arXiv2018

Multiresolution Representations for Piecewise-Smooth Signals on Graphs

What is a mathematically rigorous way to describe the taxi-pickup distribution in Manhattan, or the profile information in online social networks? A deep understanding of representing those data not only provides insights to the data properties, but also benefits to many subsequent processing procedures, such as denoising, sampling, recovery and localization. In this paper, we model those complex and irregular data as piecewise-smooth graph signals and propose a graph dictionary to effectively represent those graph signals. We first propose the graph multiresolution analysis, which provides a principle to design good representations. We then propose a coarse-to-fine approach, which iteratively partitions a graph into two subgraphs until we reach individual nodes. This approach efficiently implements the graph multiresolution analysis and the induced graph dictionary promotes sparse representations piecewise-smooth graph signals. Finally, we validate the proposed graph dictionary on two tasks: approximation and localization. The empirical results show that the proposed graph dictionary outperforms eight other representation methods on six datasets, including traffic networks, social networks and point cloud meshes.

eess.SP↗

arXiv2017

Fast Path Localization on Graphs via Multiscale Viterbi Decoding

We consider a problem of localizing a path-signal that evolves over time on a graph. A path-signal can be viewed as the trajectory of a moving agent on a graph in several consecutive time points. Combining dynamic programming and graph partitioning, we propose a path-localization algorithm with significantly reduced computational complexity. We analyze the localization error for the proposed approach both in the Hamming distance and the destination's distance between the path estimate and the true path using numerical bounds. Unlike usual theoretical bounds that only apply to restricted graph models, the obtained numerical bounds apply to all graphs and all non-overlapping graph-partitioning schemes. In random geometric graphs, we are able to derive a closed-form expression for the localization error bound, and a tradeoff between localization error and the computational complexity. Finally, we compare the proposed technique with the maximum likelihood estimate under the path constraint in terms of computational complexity and localization error, and show significant speedup (100 times) with comparable localization error (4 times) on a graph from real data. Variants of the proposed technique can be applied to tracking, road congestion monitoring, and brain signal processing.

eess.SP↗