arXiv subjects

Liang Pan

Publications and source records attributed to Liang Pan.

At least 109 records · Page 6Linked to original sources

arXiv2022

Robust Partial-to-Partial Point Cloud Registration in a Full Range

Point cloud registration for 3D objects is a challenging task due to sparse and noisy measurements, incomplete observations and large transformations. In this work, we propose \textbf{G}raph \textbf{M}atching \textbf{C}onsensus \textbf{Net}work (\textbf{GMCNet}), which estimates pose-invariant correspondences for full-range Partial-to-Partial point cloud Registration (PPR) in the object-level registration scenario. To encode robust point descriptors, \textbf{1)} we first comprehensively investigate transformation-robustness and noise-resilience of various geometric features. \textbf{2)} Then, we employ a novel {T}ransformation-robust {P}oint {T}ransformer (\textbf{TPT}) module to adaptively aggregate local features regarding the structural relations, which takes advantage from both handcrafted rotation-invariant ({\textit{RI}}) features and noise-resilient spatial coordinates. \textbf{3)} Based on a synergy of hierarchical graph networks and graphical modeling, we propose the {H}ierarchical {G}raphical {M}odeling (\textbf{HGM}) architecture to encode robust descriptors consisting of i) a unary term learned from {\textit{RI}} features; and ii) multiple smoothness terms encoded from neighboring point relations at different scales through our TPT modules. Moreover, we construct a challenging PPR dataset (\textbf{MVP-RG}) based on the recent MVP dataset that features high-quality scans. Extensive experiments show that GMCNet outperforms previous state-of-the-art methods for PPR. Notably, GMCNet encodes point descriptors for each point cloud individually without using cross-contextual information, or ground truth correspondences for training. Our code and datasets are available at: https://github.com/paul007pl/GMCNet.

cs.CV↗

arXiv2022

PTTR: Relational 3D Point Cloud Object Tracking with Transformer

In a point cloud sequence, 3D object tracking aims to predict the location and orientation of an object in the current search point cloud given a template point cloud. Motivated by the success of transformers, we propose Point Tracking TRansformer (PTTR), which efficiently predicts high-quality 3D tracking results in a coarse-to-fine manner with the help of transformer operations. PTTR consists of three novel designs. 1) Instead of random sampling, we design Relation-Aware Sampling to preserve relevant points to given templates during subsampling. 2) Furthermore, we propose a Point Relation Transformer (PRT) consisting of a self-attention and a cross-attention module. The global self-attention operation captures long-range dependencies to enhance encoded point features for the search area and the template, respectively. Subsequently, we generate the coarse tracking results by matching the two sets of point features via cross-attention. 3) Based on the coarse tracking results, we employ a novel Prediction Refinement Module to obtain the final refined prediction. In addition, we create a large-scale point cloud single object tracking benchmark based on the Waymo Open Dataset. Extensive experiments show that PTTR achieves superior point cloud tracking in both accuracy and efficiency.

cs.CV↗

arXiv2022

TCTrack: Temporal Contexts for Aerial Tracking

Temporal contexts among consecutive frames are far from being fully utilized in existing visual trackers. In this work, we present TCTrack, a comprehensive framework to fully exploit temporal contexts for aerial tracking. The temporal contexts are incorporated at \textbf{two levels}: the extraction of \textbf{features} and the refinement of \textbf{similarity maps}. Specifically, for feature extraction, an online temporally adaptive convolution is proposed to enhance the spatial features using temporal information, which is achieved by dynamically calibrating the convolution weights according to the previous frames. For similarity map refinement, we propose an adaptive temporal transformer, which first effectively encodes temporal knowledge in a memory-efficient way, before the temporal knowledge is decoded for accurate adjustment of the similarity map. TCTrack is effective and efficient: evaluation on four aerial tracking benchmarks shows its impressive performance; real-world UAV tests show its high speed of over 27 FPS on NVIDIA Jetson AGX Xavier.

cs.CV↗

arXiv2022

Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capture

Due to the visual ambiguity, purely kinematic formulations on monocular human motion capture are often physically incorrect, biomechanically implausible, and can not reconstruct accurate interactions. In this work, we focus on exploiting the high-precision and non-differentiable physics simulator to incorporate dynamical constraints in motion capture. Our key-idea is to use real physical supervisions to train a target pose distribution prior for sampling-based motion control to capture physically plausible human motion. To obtain accurate reference motion with terrain interactions for the sampling, we first introduce an interaction constraint based on SDF (Signed Distance Field) to enforce appropriate ground contact modeling. We then design a novel two-branch decoder to avoid stochastic error from pseudo ground-truth and train a distribution prior with the non-differentiable physics simulator. Finally, we regress the sampling distribution from the current state of the physical character with the trained prior and sample satisfied target poses to track the estimated reference motion. Qualitative and quantitative results show that we can obtain physically plausible human motion with complex terrain interactions, human shape variations, and diverse behaviors. More information can be found at~\url{https://www.yangangwang.com/papers/HBZ-NM-2022-03.html}

cs.CV↗

arXiv2022

Versatile Multi-Modal Pre-Training for Human-Centric Perception

Human-centric perception plays a vital role in vision and graphics. But their data annotations are prohibitively expensive. Therefore, it is desirable to have a versatile pre-train model that serves as a foundation for data-efficient downstream tasks transfer. To this end, we propose the Human-Centric Multi-Modal Contrastive Learning framework HCMoCo that leverages the multi-modal nature of human data (e.g. RGB, depth, 2D keypoints) for effective representation learning. The objective comes with two main challenges: dense pre-train for multi-modality data, efficient usage of sparse human priors. To tackle the challenges, we design the novel Dense Intra-sample Contrastive Learning and Sparse Structure-aware Contrastive Learning targets by hierarchically learning a modal-invariant latent space featured with continuous and ordinal feature distribution and structure-aware semantic consistency. HCMoCo provides pre-train for different modalities by combining heterogeneous datasets, which allows efficient usage of existing task-specific human data. Extensive experiments on four downstream tasks of different modalities demonstrate the effectiveness of HCMoCo, especially under data-efficient settings (7.16% and 12% improvement on DensePose Estimation and Human Parsing). Moreover, we demonstrate the versatility of HCMoCo by exploring cross-modality supervision and missing-modality inference, validating its strong ability in cross-modal association and reasoning.

cs.CV↗

arXiv2022

A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion

3D point cloud is an important 3D representation for capturing real world 3D objects. However, real-scanned 3D point clouds are often incomplete, and it is important to recover complete point clouds for downstream applications. Most existing point cloud completion methods use Chamfer Distance (CD) loss for training. The CD loss estimates correspondences between two point clouds by searching nearest neighbors, which does not capture the overall point density distribution on the generated shape, and therefore likely leads to non-uniform point cloud generation. To tackle this problem, we propose a novel Point Diffusion-Refinement (PDR) paradigm for point cloud completion. PDR consists of a Conditional Generation Network (CGNet) and a ReFinement Network (RFNet). The CGNet uses a conditional generative model called the denoising diffusion probabilistic model (DDPM) to generate a coarse completion conditioned on the partial observation. DDPM establishes a one-to-one pointwise mapping between the generated point cloud and the uniform ground truth, and then optimizes the mean squared error loss to realize uniform generation. The RFNet refines the coarse output of the CGNet and further improves quality of the completed point cloud. Furthermore, we develop a novel dual-path architecture for both networks. The architecture can (1) effectively and efficiently extract multi-level features from partially observed point clouds to guide completion, and (2) accurately manipulate spatial locations of 3D points to obtain smooth surfaces and sharp details. Extensive experimental results on various benchmark datasets show that our PDR paradigm outperforms previous state-of-the-art methods for point cloud completion. Remarkably, with the help of the RFNet, we can accelerate the iterative generation process of the DDPM by up to 50 times without much performance drop.

cs.CV↗

arXiv2022

TAda! Temporally-Adaptive Convolutions for Video Understanding

Spatial convolutions are widely used in numerous deep video models. It fundamentally assumes spatio-temporal invariance, i.e., using shared weights for every location in different frames. This work presents Temporally-Adaptive Convolutions (TAdaConv) for video understanding, which shows that adaptive weight calibration along the temporal dimension is an efficient way to facilitate modelling complex temporal dynamics in videos. Specifically, TAdaConv empowers the spatial convolutions with temporal modelling abilities by calibrating the convolution weights for each frame according to its local and global temporal context. Compared to previous temporal modelling operations, TAdaConv is more efficient as it operates over the convolution kernels instead of the features, whose dimension is an order of magnitude smaller than the spatial resolutions. Further, the kernel calibration brings an increased model capacity. We construct TAda2D and TAdaConvNeXt networks by replacing the 2D convolutions in ResNet and ConvNeXt with TAdaConv, which leads to at least on par or better performance compared to state-of-the-art approaches on multiple video action recognition and localization benchmarks. We also demonstrate that as a readily plug-in operation with negligible computation overhead, TAdaConv can effectively improve many existing video models with a convincing margin.

cs.CV↗

arXiv2022

Three-dimensional discontinuous Galerkin based high-order gas-kinetic scheme and GPU implementation

In this paper, the discontinuous Galerkin based high-order gas-kinetic schemes (DG-HGKS) are developed for the three-dimensional Euler and Navier-Stokes equations. Different from the traditional discontinuous Galerkin (DG) methods with Riemann solvers, the current method adopts a kinetic evolution process, which is provided by the integral solution of Bhatnagar-Gross-Krook (BGK) model. In the weak formulation of DG method, a time-dependent evolution function is provided, and both inviscid and viscous fluxes can be calculated uniformly. The temporal accuracy is achieved by the two-stage fourth-order discretization, and the second-order gas-kinetic solver is adopted for the fluxes over the cell interface and the fluxes inside a cell. Numerical examples, including accuracy tests and Taylor-Green vortex problem, are presented to validate the efficiency and accuracy of DG-HGKS. Both optimal convergence and super-convergence are achieved by the current scheme. The comparison between DG-HGKS and high-order gas-kinetic scheme with weighted essential non-oscillatory reconstruction (WENO-HGKS) is also given, and the numerical performances are comparable with the approximate number of degree of freedom. To accelerate the computation, the DG-HGKS is implemented with the graphics processing unit (GPU) using compute unified device architecture (CUDA). The obtained results are also compared with those calculated by the central processing units (CPU) code in terms of the computational efficiency. The speedup of GPU code suggests the potential of high-order gas-kinetic schemes for the large scale computation.

math.NA↗

arXiv2021

Multi-View Partial (MVP) Point Cloud Challenge 2021 on Completion and Registration: Methods and Results

As real-scanned point clouds are mostly partial due to occlusions and viewpoints, reconstructing complete 3D shapes based on incomplete observations becomes a fundamental problem for computer vision. With a single incomplete point cloud, it becomes the partial point cloud completion problem. Given multiple different observations, 3D reconstruction can be addressed by performing partial-to-partial point cloud registration. Recently, a large-scale Multi-View Partial (MVP) point cloud dataset has been released, which consists of over 100,000 high-quality virtual-scanned partial point clouds. Based on the MVP dataset, this paper reports methods and results in the Multi-View Partial Point Cloud Challenge 2021 on Completion and Registration. In total, 128 participants registered for the competition, and 31 teams made valid submissions. The top-ranked solutions will be analyzed, and then we will discuss future research directions.

cs.CV↗

arXiv2021

Non-equilibrium time-relaxation kinetic model for compressible turbulence modeling

For the first time, the non-equilibrium time-relaxation kinetic model (NTRKM) is proposed for compressible turbulence modeling on unresolved grids. Within the non-equilibrium time-relaxation framework, NTRKM is extended in the form of modified Bhatnagar-Gross-Krook model. Based on the first-order Chapman-Enskog expansion, NTRKM connects with the six-variable macroscopic governing equations. The first five governing equations correspond to the conservative laws in mass, momentum and total energy, while the sixth equation governs the evolution of unresolved turbulence kinetic energy Kutke. The unknowns in NTRKM, including turbulent relaxation time and source term, are determined by essential gradient-type assumption and standard dynamic modeling approach. Current generalized kinetic model on unresolved grids consequently offers a profound mesoscopic understanding for one-equation subgrid-scale turbulence kinetic energy Ksgs model in compressible large eddy simulation. To solve NTRKM accurately and robustly, a non-equilibrium gas-kinetic scheme is developed, which succeeds the well-established gas-kinetic scheme for simulating Navier-Stokes equations. Three-dimensional decaying compressible isotropic turbulence and temporal compressible plane mixing layer on unresolved grids are simulated to evaluate the generalized kinetic model and non-equilibrium gas-kinetic scheme. The performance of key turbulent quantities up to second-order statistics confirms that NTRKM is comparable with the widely-used eddy-viscosity Smagorinsky model (SM) and dynamic Smagorinsky model (DSM). Specifically, compared with the DNS solution in temporal compressible plane mixing layer, the performance of NTRKM is much closer with DSM and better than SM. This study provides a workable approach for compressible turbulence modeling on unresolved grids.

physics.flu-dyn↗

arXiv2021

Garment4D: Garment Reconstruction from Point Cloud Sequences

Learning to reconstruct 3D garments is important for dressing 3D human bodies of different shapes in different poses. Previous works typically rely on 2D images as input, which however suffer from the scale and pose ambiguities. To circumvent the problems caused by 2D images, we propose a principled framework, Garment4D, that uses 3D point cloud sequences of dressed humans for garment reconstruction. Garment4D has three dedicated steps: sequential garments registration, canonical garment estimation, and posed garment reconstruction. The main challenges are two-fold: 1) effective 3D feature learning for fine details, and 2) capture of garment dynamics caused by the interaction between garments and the human body, especially for loose garments like skirts. To unravel these problems, we introduce a novel Proposal-Guided Hierarchical Feature Network and Iterative Graph Convolution Network, which integrate both high-level semantic features and low-level geometric features for fine details reconstruction. Furthermore, we propose a Temporal Transformer for smooth garment motions capture. Unlike non-parametric methods, the reconstructed garment meshes by our method are separable from the human body and have strong interpretability, which is desirable for downstream tasks. As the first attempt at this task, high-quality reconstruction results are qualitatively and quantitatively illustrated through extensive experiments. Codes are available at https://github.com/hongfz16/Garment4D.

cs.CV↗

arXiv2021

Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion

Chamfer Distance (CD) and Earth Mover's Distance (EMD) are two broadly adopted metrics for measuring the similarity between two point sets. However, CD is usually insensitive to mismatched local density, and EMD is usually dominated by global distribution while overlooks the fidelity of detailed structures. Besides, their unbounded value range induces a heavy influence from the outliers. These defects prevent them from providing a consistent evaluation. To tackle these problems, we propose a new similarity measure named Density-aware Chamfer Distance (DCD). It is derived from CD and benefits from several desirable properties: 1) it can detect disparity of density distributions and is thus a more intensive measure of similarity compared to CD; 2) it is stricter with detailed structures and significantly more computationally efficient than EMD; 3) the bounded value range encourages a more stable and reasonable evaluation over the whole test set. We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other. We can also use DCD as the training loss, which outperforms the same model trained with CD loss on all three metrics. In addition, we propose a novel point discriminator module that estimates the priority for another guided down-sampling step, and it achieves noticeable improvements under DCD together with competitive results for both CD and EMD. We hope our work could pave the way for a more comprehensive and practical point cloud similarity evaluation. Our code will be available at: https://github.com/wutong16/Density_aware_Chamfer_Distance .

cs.CV↗

arXiv2021

High-order gas-kinetic scheme for radiation hydrodynamics in equilibrium-diffusion limit

In this paper, a high-order gas-kinetic scheme is developed for the equation of radiation hydrodynamics in equilibrium-diffusion limit which describes the interaction between matter and radiation. To recover RHE, the Bhatnagar-Gross-Krook (BGK) model with modified equilibrium state is considered. In the equilibrium-diffusion limit, the time scales of radiation diffusion and hydrodynamic part are different, and it will make the time step very small for the fully explicit scheme. An implicit-explicit (IMEX) scheme is applied, in which the hydrodynamic part is treated explicitly and the radiation diffusion is treated implicitly. For the hydrodynamics part, a time dependent gas distribution function can be constructed by the integral solution of modified BGK equation, and the time dependent numerical fluxes can be obtained by taking moments of gas distribution function. For the radiation diffusion term, the nonlinear generalized minimal residual (GMRES) method is used. To achieve the temporal accuracy, a two-stage method is developed, which is an extension of two-stage method for hyperbolic conservation law. For the spatial accuracy, the multidimensional weighted essential non-oscillation (WENO) scheme is used for the spatial reconstruction. A variety of numerical tests are provided for the performance of current scheme, including the order of accuracy and robustness.

math.NA↗

arXiv2021

CSG-Stump: A Learning Friendly CSG-Like Representation for Interpretable Shape Parsing

Generating an interpretable and compact representation of 3D shapes from point clouds is an important and challenging problem. This paper presents CSG-Stump Net, an unsupervised end-to-end network for learning shapes from point clouds and discovering the underlying constituent modeling primitives and operations as well. At the core is a three-level structure called {\em CSG-Stump}, consisting of a complement layer at the bottom, an intersection layer in the middle, and a union layer at the top. CSG-Stump is proven to be equivalent to CSG in terms of representation, therefore inheriting the interpretable, compact and editable nature of CSG while freeing from CSG's complex tree structures. Particularly, the CSG-Stump has a simple and regular structure, allowing neural networks to give outputs of a constant dimensionality, which makes itself deep-learning friendly. Due to these characteristics of CSG-Stump, CSG-Stump Net achieves superior results compared to previous CSG-based methods and generates much more appealing shapes, as confirmed by extensive experiments. Project page: https://kimren227.github.io/projects/CSGStump/

cs.CV↗

arXiv2021

High-order gas-kinetic scheme in general curvilinear coordinate for iLES of compressible wall-bounded turbulent flows

In this paper, a high-order gas-kinetic scheme in general curvilinear coordinate (HGKS-cur) is developed for the numerical simulation of compressible turbulence. Based on the coordinate transformation, the Bhatnagar-Gross-Krook (BGK) equation is transformed from physical space to computational space. To deal with the general mesh given by discretized points, the geometrical metrics need to be constructed by the dimension-by-dimension Lagrangian interpolation. The multidimensional weighted essentially non-oscillatory (WENO) reconstruction is adopted in the computational domain for spatial accuracy, where the reconstructed variables are the cell averaged Jacobian and the Jacobian-weighted conservative variables. The two-stage fourth-order method, which was developed for spatial-temporal coupled flow solvers, is used for temporal discretization. The numerical examples for inviscid and laminar flows validate the accuracy and geometrical conservation law of HGKS-cur. As a direct application, HGKS-cur is implemented for the implicit large eddy simulation (iLES) in compressible wall-bounded turbulent flows, including the compressible turbulent channel flow and compressible turbulent flow over periodic hills. The iLES results with HGKS-cur are in good agreement with the refereed spectral methods and high-order finite volume methods. The performance of HGKS-cur demonstrates its capability as a powerful tool for the numerical simulation of compressible wall-bounded turbulent flows and massively separated flows.

physics.flu-dyn↗

arXiv2021

Unsupervised 3D Shape Completion through GAN Inversion

Most 3D shape completion approaches rely heavily on partial-complete shape pairs and learn in a fully supervised manner. Despite their impressive performances on in-domain data, when generalizing to partial shapes in other forms or real-world partial scans, they often obtain unsatisfactory results due to domain gaps. In contrast to previous fully supervised approaches, in this paper we present ShapeInversion, which introduces Generative Adversarial Network (GAN) inversion to shape completion for the first time. ShapeInversion uses a GAN pre-trained on complete shapes by searching for a latent code that gives a complete shape that best reconstructs the given partial input. In this way, ShapeInversion no longer needs paired training data, and is capable of incorporating the rich prior captured in a well-trained generative model. On the ShapeNet benchmark, the proposed ShapeInversion outperforms the SOTA unsupervised method, and is comparable with supervised methods that are learned using paired data. It also demonstrates remarkable generalization ability, giving robust results for real-world scans and partial inputs of various forms and incompleteness levels. Importantly, ShapeInversion naturally enables a series of additional abilities thanks to the involvement of a pre-trained GAN, such as producing multiple valid complete shapes for an ambiguous partial input, as well as shape manipulation and interpolation.

cs.CV↗

arXiv2021

Variational Relational Point Completion Network

Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic partial-to-complete mapping, but overlook structural relations in man-made objects. To tackle these challenges, this paper proposes a variational framework, Variational Relational point Completion network (VRCNet) with two appealing properties: 1) Probabilistic Modeling. In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds. One path consumes complete point clouds for reconstruction by learning a point VAE. The other path generates complete shapes for partial point clouds, whose embedded distribution is guided by distribution obtained from the reconstruction path during training. 2) Relational Enhancement. Specifically, we carefully design point self-attention kernel and point selective kernel module to exploit relational point features, which refines local shape details conditioned on the coarse completion. In addition, we contribute a multi-view partial point cloud dataset (MVP dataset) containing over 100,000 high-quality scans, which renders partial 3D shapes from 26 uniformly distributed camera poses for each 3D CAD model. Extensive experiments demonstrate that VRCNet outperforms state-of-theart methods on all standard point cloud completion benchmarks. Notably, VRCNet shows great generalizability and robustness on real-world point cloud scans.

cs.CV↗

arXiv2020

Three dimensional high-order gas-kinetic scheme for supersonic isotropic turbulence II: coarse-grained analysis of compressible $K_{sgs}$ budget

The direct numerical simulation (DNS) of compressible isotropic turbulence up to the supersonic regime $Ma_{t} = 1.2$ has been investigated by high-order gas-kinetic scheme (HGKS) [{\it{Computers}} \& {\it{Fluids, 192, 2019}}]. In this study, the coarse-grained analysis of subgrid-scale (SGS) turbulent kinetic energy $K_{sgs}$ budget is fully analyzed for constructing one-equation SGS model in the compressible large eddy simulation (LES). The DNS on a much higher turbulent Mach number up to $Ma_{t} = 2.0$ has been obtained by HGKS, which confirms the super robustness of HGKS. Then, the exact compressible SGS turbulent kinetic energy $K_{sgs}$ transport equation is derived with density weighted filtering process. Based on the compressible $K_{sgs}$ transport equation, the coarse-grained processes are implemented on three sets of unresolved grids with the Box filter. The coarse-grained analysis of compressible $K_{sgs}$ budgets shows that all unresolved source terms are dominant terms in current system. Especially, the magnitude of SGS pressure-dilation term is in the order of SGS solenoidal dissipation term within the initial acoustic time scale. Therefore, it can be concluded that the SGS pressure-dilation term cannot be neglected as the previous work. The delicate coarse-grained analysis of SGS diffusion terms in compressible $K_{sgs}$ equation confirms that both the fluctuation velocity triple correlation term and the pressure-velocity correlation term are dominant terms. Current coarse-grained analysis gives an indication of the order of magnitude of all SGS terms in compressible $K_{sgs}$ budget, which provides a solid basis for compressible LES modeling in high Mach number turbulent flow.

physics.flu-dyn↗