arXiv subjects

Cheng Lin

Publications and source records attributed to Cheng Lin.

69 records · Page 4Linked to original sources

arXiv2022

SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views

We introduce SparseNeuS, a novel neural rendering based method for the task of surface reconstruction from multi-view images. This task becomes more difficult when only sparse images are provided as input, a scenario where existing neural reconstruction approaches usually produce incomplete or distorted results. Moreover, their inability of generalizing to unseen new scenes impedes their application in practice. Contrarily, SparseNeuS can generalize to new scenes and work well with sparse images (as few as 2 or 3). SparseNeuS adopts signed distance function (SDF) as the surface representation, and learns generalizable priors from image features by introducing geometry encoding volumes for generic surface prediction. Moreover, several strategies are introduced to effectively leverage sparse views for high-quality reconstruction, including 1) a multi-level geometry reasoning framework to recover the surfaces in a coarse-to-fine manner; 2) a multi-scale color blending scheme for more reliable color prediction; 3) a consistency-aware fine-tuning scheme to control the inconsistent regions caused by occlusion and noise. Extensive experiments demonstrate that our approach not only outperforms the state-of-the-art methods, but also exhibits good efficiency, generalizability, and flexibility.

cs.CV↗

arXiv2022

3PSDF: Three-Pole Signed Distance Function for Learning Surfaces with Arbitrary Topologies

Recent advances in learning 3D shapes using neural implicit functions have achieved impressive results by breaking the previous barrier of resolution and diversity for varying topologies. However, most of such approaches are limited to closed surfaces as they require the space to be divided into inside and outside. More recent works based on unsigned distance function have been proposed to handle complex geometry containing both the open and closed surfaces. Nonetheless, as their direct outputs are point clouds, robustly obtaining high-quality meshing results from discrete points remains an open question. We present a novel learnable implicit representation, called the three-pole signed distance function (3PSDF), that can represent non-watertight 3D shapes with arbitrary topologies while supporting easy field-to-mesh conversion using the classic Marching Cubes algorithm. The key to our method is the introduction of a new sign, the NULL sign, in addition to the conventional in and out labels. The existence of the null sign could stop the formation of a closed isosurface derived from the bisector of the in/out regions. Further, we propose a dedicated learning framework to effectively learn 3PSDF without worrying about the vanishing gradient due to the null labels. Experimental results show that our approach outperforms the previous state-of-the-art methods in a wide range of benchmarks both quantitatively and qualitatively.

cs.CV↗

arXiv2022

Coverage Axis: Inner Point Selection for 3D Shape Skeletonization

In this paper, we present a simple yet effective formulation called Coverage Axis for 3D shape skeletonization. Inspired by the set cover problem, our key idea is to cover all the surface points using as few inside medial balls as possible. This formulation inherently induces a compact and expressive approximation of the Medial Axis Transform (MAT) of a given shape. Different from previous methods that rely on local approximation error, our method allows a global consideration of the overall shape structure, leading to an efficient high-level abstraction and superior robustness to noise. Another appealing aspect of our method is its capability to handle more generalized input such as point clouds and poor-quality meshes. Extensive comparisons and evaluations demonstrate the remarkable effectiveness of our method for generating compact and expressive skeletal representation to approximate the MAT.

cs.GR↗

arXiv2021

Adaptive Surface Normal Constraint for Depth Estimation

We present a novel method for single image depth estimation using surface normal constraints. Existing depth estimation methods either suffer from the lack of geometric constraints, or are limited to the difficulty of reliably capturing geometric context, which leads to a bottleneck of depth estimation quality. We therefore introduce a simple yet effective method, named Adaptive Surface Normal (ASN) constraint, to effectively correlate the depth estimation with geometric consistency. Our key idea is to adaptively determine the reliable local geometry from a set of randomly sampled candidates to derive surface normal constraint, for which we measure the consistency of the geometric contextual features. As a result, our method can faithfully reconstruct the 3D geometry and is robust to local shape variations, such as boundaries, sharp corners and noises. We conduct extensive evaluations and comparisons using public datasets. The experimental results demonstrate our method outperforms the state-of-the-art methods and has superior efficiency and robustness.

cs.CV↗

arXiv2021

Point2Skeleton: Learning Skeletal Representations from Point Clouds

We introduce Point2Skeleton, an unsupervised method to learn skeletal representations from point clouds. Existing skeletonization methods are limited to tubular shapes and the stringent requirement of watertight input, while our method aims to produce more generalized skeletal representations for complex structures and handle point clouds. Our key idea is to use the insights of the medial axis transform (MAT) to capture the intrinsic geometric and topological natures of the original input points. We first predict a set of skeletal points by learning a geometric transformation, and then analyze the connectivity of the skeletal points to form skeletal mesh structures. Extensive evaluations and comparisons show our method has superior performance and robustness. The learned skeletal representation will benefit several unsupervised tasks for point clouds, such as surface reconstruction and segmentation.

cs.CV↗

arXiv2020

Learnable Motion Coherence for Correspondence Pruning

Motion coherence is an important clue for distinguishing true correspondences from false ones. Modeling motion coherence on sparse putative correspondences is challenging due to their sparsity and uneven distributions. Existing works on motion coherence are sensitive to parameter settings and have difficulty in dealing with complex motion patterns. In this paper, we introduce a network called Laplacian Motion Coherence Network (LMCNet) to learn motion coherence property for correspondence pruning. We propose a novel formulation of fitting coherent motions with a smooth function on a graph of correspondences and show that this formulation allows a closed-form solution by graph Laplacian. This closed-form solution enables us to design a differentiable layer in a learning framework to capture global motion coherence from putative correspondences. The global motion coherence is further combined with local coherence extracted by another local layer to robustly detect inlier correspondences. Experiments demonstrate that LMCNet has superior performances to the state of the art in relative camera pose estimation and correspondences pruning of dynamic scenes.

cs.CV↗

arXiv2020

Adaptive Compact Attention For Few-shot Video-to-video Translation

This paper proposes an adaptive compact attention model for few-shot video-to-video translation. Existing works in this domain only use features from pixel-wise attention without considering the correlations among multiple reference images, which leads to heavy computation but limited performance. Therefore, we introduce a novel adaptive compact attention mechanism to efficiently extract contextual features jointly from multiple reference images, of which encoded view-dependent and motion-dependent information can significantly benefit the synthesis of realistic videos. Our core idea is to extract compact basis sets from all the reference images as higher-level representations. To further improve the reliability, in the inference phase, we also propose a novel method based on the Delaunay Triangulation algorithm to automatically select the resourceful references according to the input label. We extensively evaluate our method on a large-scale talking-head video dataset and a human dancing dataset; the experimental results show the superior performance of our method for producing photorealistic and temporally consistent videos, and considerable improvements over the state-of-the-art method.

cs.CV↗

arXiv2020

SEG-MAT: 3D Shape Segmentation Using Medial Axis Transform

Segmenting arbitrary 3D objects into constituent parts that are structurally meaningful is a fundamental problem encountered in a wide range of computer graphics applications. Existing methods for 3D shape segmentation suffer from complex geometry processing and heavy computation caused by using low-level features and fragmented segmentation results due to the lack of global consideration. We present an efficient method, called SEG-MAT, based on the medial axis transform (MAT) of the input shape. Specifically, with the rich geometrical and structural information encoded in the MAT, we are able to develop a simple and principled approach to effectively identify the various types of junctions between different parts of a 3D shape. Extensive evaluations and comparisons show that our method outperforms the state-of-the-art methods in terms of segmentation quality and is also one order of magnitude faster.

cs.GR↗

arXiv2020

Modeling 3D Shapes by Reinforcement Learning

We explore how to enable machines to model 3D shapes like human modelers using deep reinforcement learning (RL). In 3D modeling software like Maya, a modeler usually creates a mesh model in two steps: (1) approximating the shape using a set of primitives; (2) editing the meshes of the primitives to create detailed geometry. Inspired by such artist-based modeling, we propose a two-step neural framework based on RL to learn 3D modeling policies. By taking actions and collecting rewards in an interactive environment, the agents first learn to parse a target shape into primitives and then to edit the geometry. To effectively train the modeling agents, we introduce a novel training algorithm that combines heuristic policy, imitation learning and reinforcement learning. Our experiments show that the agents can learn good policies to produce regular and structure-aware mesh models, which demonstrates the feasibility and effectiveness of the proposed RL framework.

cs.CV↗

arXiv2020

Low-Energy Interference Structure with Attosecond Temporal Resolution

Accessing precisely to the phase variation of electronic wave-packet (EWP) provides unprecedented spatiotemporal information of microworld. A radial interference pattern at near-zero energy has been widely observed in experiments of strong-field photoionization. However, the underlying physical picture of this interference pattern is still under debate. Here we report an experimental and theoretical investigation of this low-energy interference structure (LEIS) in mid-infrared laser fields. We clarify that the LEIS arises due to the soft-recollision mechanism, which was previously found to play a pivotal role in producing the pronounced low-energy structure. Specifically, the LEIS is induced by the interference between direct and soft-recollision EWPs launched within a 1/18 laser-cycle time scale in our experiments. Moreover, the observation of LEIS is independent of laser wavelength and specific atomic targets. Our result opens a promising new avenue for retrieving the structure and dynamics of EWPs in atoms and molecules with attosecond time resolution.

physics.atom-ph↗

arXiv2019

Floorplan-Jigsaw: Jointly Estimating Scene Layout and Aligning Partial Scans

We present a novel approach to align partial 3D reconstructions which may not have substantial overlap. Using floorplan priors, our method jointly predicts a room layout and estimates the transformations from a set of partial 3D data. Unlike the existing methods relying on feature descriptors to establish correspondences, we exploit the 3D "box" structure of a typical room layout that meets the Manhattan World property. We first estimate a local layout for each partial scan separately and then combine these local layouts to form a globally aligned layout with loop closure. Without the requirement of feature matching, the proposed method enables some novel applications ranging from large or featureless scene reconstruction and modeling from sparse input. We validate our method quantitatively and qualitatively on real and synthetic scenes of various sizes and complexities. The evaluations and comparisons show superior effectiveness and accuracy of our method.

cs.CV↗

arXiv2017

Spatiotemporal steering of photoelectron emission in multiphoton above-threshold ionization

We experimentally demonstrate spatiotemporal steering of photoelectron emission in multiphoton above-threshold single ionization of atoms exposed to a phase-controlled orthogonally polarized two-color (OTC) laser pulse. Spatial and energy resolved photoelectron angular distributions are measured as a function of the laser phase, allowing us to look into the fine structures and emission dynamics. The slow and fast photoelectrons, distinguished by the energy larger or smaller than 2Up with Up being the ponderomotive energy of a free electron in the laser field, have distinct spatiotemporal dependences of the laser waveform. The phase-of-phase of the slow electron oscillates as functions of both the energy and emission direction, however, the fast electron present rather flat phase structure which merely depends on its emission direction. Three-dimensional generalized quantum trajectory Monte Carlo simulations are performed to explore the sub-cycle dynamics of the electron emission in the phase-controlled OTC pulse.

physics.optics↗

arXiv2016

Momentum mapping of continuum electron wave packet interference

We analyze the two-dimensional photoelectrons momentum distribution of Ar atom ionized by midinfrared laser pulses and mainly concentrate on the energy range below 2Up. By using a generalized quantum trajectory Monte Carlo (GQTMC) simulation and comparing with the numerical solution of time-dependent Schrodinger equation (TDSE), we show that in the deep tunneling regime, the rescattered electron trajectories plays unimportant role and the interplay between the intracycle and inter-cycle results in a ring-like interference pattern. The ring-like interference pattern will mask the holographic interference structure in the low longitudinal momentum region. When the nonadiabatic tunneling contributes significantly to ionization, i.e., the Keldysh parameter 1, the contribution of the rescattered electron trajectories become large, thus holographic interference pattern can be clearly observed. Our results help paving the way for gaining physical insight into ultrafast electron dynamic process with attosecond temporal resolution.

physics.atom-ph↗

arXiv2016

Unraveling nonadiabatic ionization and Coulomb potential effects in strong-field photoelectron holography

Strong field photoelectron holography has been proposed as a means for interrogating the spatial and temporal information of electrons and ions in a dynamic system. After ionization, part of the electron wave packet may directly go to the detector (the reference wave), while another part may be driven back to the ion where it scatters off (the signal wave). The interference hologram of the two waves may be used to retrieve the target information. However, unlike conventional optical holography, the propagations of electron wave packets are affected by the Coulomb potential as well as by the laser field. In addition, electrons are emitted over the whole laser pulse duration, thus multiple interferences may occur. In this work, we used a generalized quantum-trajectory Monte Carlo method to investigate the effect of Coulomb potential and the nonadiabatic subcycle ionization on the photoelectron hologram. We showed that photoelectron hologram can be well described only when the nonadiabatic effect in ionization is accounted for, and Coulomb potential can be neglected only in the tunnel ionization regime. Our results help establishing photoelectron holography for probing spatial and dynamic properties of atoms and molecules.

physics.atom-ph↗

arXiv2016

Attosecond Interference Induced by Coulomb-Field-Driven Transverse Backward-Scattering Electron Wave-Packets

A novel and universal interference structure is found in the photoelectron momentum distribution of atoms in intense infrared laser field. Theoretical analysis shows that this structure can be attributed to a new form of Coulomb-field-driven backward-scattering of photoelectrons in the direction perpendicular to the laser field, in contrast to the conventional rescattering along the laser polarization direction. This transverse backward-scattering process is closely related to a family of photoelectrons initially ionized within a time interval of less than 200 attosecond around the crest of the laser electric field. Those electrons, acquiring near-zero return energy in the laser field, will be pulled back solely by the ionic Coulomb field and backscattered in the transverse direction. Moreover, this rescattering process mainly occurs at the first or the second return times, giving rise to different phases of the photoelectrons. The interference between these photoelectrons leads to unique curved interference fringes which are observable for most current intense field experiments, opening a new way to record the electron dynamics in atoms and molecules on a time scale much shorter than an optical cycle.

physics.atom-ph↗