arXiv subjects

Manyi Li

Publications and source records attributed to Manyi Li.

26 records · Page 2Linked to original sources

arXiv2023

Laplacian2Mesh: Laplacian-Based Mesh Understanding

Geometric deep learning has sparked a rising interest in computer graphics to perform shape understanding tasks, such as shape classification and semantic segmentation. When the input is a polygonal surface, one has to suffer from the irregular mesh structure. Motivated by the geometric spectral theory, we introduce Laplacian2Mesh, a novel and flexible convolutional neural network (CNN) framework for coping with irregular triangle meshes (vertices may have any valence). By mapping the input mesh surface to the multi-dimensional Laplacian-Beltrami space, Laplacian2Mesh enables one to perform shape analysis tasks directly using the mature CNNs, without the need to deal with the irregular connectivity of the mesh structure. We further define a mesh pooling operation such that the receptive field of the network can be expanded while retaining the original vertex set as well as the connections between them. Besides, we introduce a channel-wise self-attention block to learn the individual importance of feature ingredients. Laplacian2Mesh not only decouples the geometry from the irregular connectivity of the mesh structure but also better captures the global features that are central to shape classification and segmentation. Extensive tests on various datasets demonstrate the effectiveness and efficiency of Laplacian2Mesh, particularly in terms of the capability of being vulnerable to noise to fulfill various learning tasks.

cs.CV↗

arXiv2022

RIM-Net: Recursive Implicit Fields for Unsupervised Learning of Hierarchical Shape Structures

We introduce RIM-Net, a neural network which learns recursive implicit fields for unsupervised inference of hierarchical shape structures. Our network recursively decomposes an input 3D shape into two parts, resulting in a binary tree hierarchy. Each level of the tree corresponds to an assembly of shape parts, represented as implicit functions, to reconstruct the input shape. At each node of the tree, simultaneous feature decoding and shape decomposition are carried out by their respective feature and part decoders, with weight sharing across the same hierarchy level. As an implicit field decoder, the part decoder is designed to decompose a sub-shape, via a two-way branched reconstruction, where each branch predicts a set of parameters defining a Gaussian to serve as a local point distribution for shape reconstruction. With reconstruction losses accounted for at each hierarchy level and a decomposition loss at each node, our network training does not require any ground-truth segmentations, let alone hierarchies. Through extensive experiments and comparisons to state-of-the-art alternatives, we demonstrate the quality, consistency, and interpretability of hierarchical structural inference by RIM-Net.

cs.CV↗

arXiv2021

CAPRI-Net: Learning Compact CAD Shapes with Adaptive Primitive Assembly

We introduce CAPRI-Net, a neural network for learning compact and interpretable implicit representations of 3D computer-aided design (CAD) models, in the form of adaptive primitive assemblies. Our network takes an input 3D shape that can be provided as a point cloud or voxel grids, and reconstructs it by a compact assembly of quadric surface primitives via constructive solid geometry (CSG) operations. The network is self-supervised with a reconstruction loss, leading to faithful 3D reconstructions with sharp edges and plausible CSG trees, without any ground-truth shape assemblies. While the parametric nature of CAD models does make them more predictable locally, at the shape level, there is a great deal of structural and topological variations, which present a significant generalizability challenge to state-of-the-art neural models for 3D shapes. Our network addresses this challenge by adaptive training with respect to each test shape, with which we fine-tune the network that was pre-trained on a model collection. We evaluate our learning framework on both ShapeNet and ABC, the largest and most diverse CAD dataset to date, in terms of reconstruction quality, shape edges, compactness, and interpretability, to demonstrate superiority over current alternatives suitable for neural CAD reconstruction.

cs.CV↗

arXiv2021

LayoutGMN: Neural Graph Matching for Structural Layout Similarity

We present a deep neural network to predict structural similarity between 2D layouts by leveraging Graph Matching Networks (GMN). Our network, coined LayoutGMN, learns the layout metric via neural graph matching, using an attention-based GMN designed under a triplet network setting. To train our network, we utilize weak labels obtained by pixel-wise Intersection-over-Union (IoUs) to define the triplet loss. Importantly, LayoutGMN is built with a structural bias which can effectively compensate for the lack of structure awareness in IoUs. We demonstrate this on two prominent forms of layouts, viz., floorplans and UI designs, via retrieval experiments on large-scale datasets. In particular, retrieval results by our network better match human judgement of structural layout similarity compared to both IoUs and other baselines including a state-of-the-art method based on graph neural networks and image convolution. In addition, LayoutGMN is the first deep model to offer both metric learning of structural layout similarity and structural matching between layout elements.

cs.CV↗

arXiv2020

D$^2$IM-Net: Learning Detail Disentangled Implicit Fields from Single Images

We present the first single-view 3D reconstruction network aimed at recovering geometric details from an input image which encompass both topological shape structures and surface features. Our key idea is to train the network to learn a detail disentangled reconstruction consisting of two functions, one implicit field representing the coarse 3D shape and the other capturing the details. Given an input image, our network, coined D$^2$IM-Net, encodes it into global and local features which are respectively fed into two decoders. The base decoder uses the global features to reconstruct a coarse implicit field, while the detail decoder reconstructs, from the local features, two displacement maps, defined over the front and back sides of the captured object. The final 3D reconstruction is a fusion between the base shape and the displacement maps, with three losses enforcing the recovery of coarse shape, overall structure, and surface details via a novel Laplacian term.

cs.CV↗

arXiv2020

Femtosecond imbalanced time-stretch spectroscopy for ultrafast gas detection

Dual-comb spectroscopy is a promising method for precise optical spectrum analysis with fast data acquisition speed. However, its implementation and applications are often hindered by the complexity of optical comb systems. Here, as a compact and robust system, femtosecond imbalanced time-stretch spectroscopy (FITSS) with simple optical layout is proposed and demonstrated. The time-stretch interferometry from one femtosecond laser builds a mapping from the optical frequency domain to the radio frequency regime. In experiment, the absorption line of a hydrogen cyanide cell is encoded in the probing arm of a Mach-Zehnder interferometer (MZI). The down-converted radio frequency comb is transformed from a periodically chirped waveform, which is the interferogram of the MZI with different dispersion values on two arms. By turning the optical filter, the spectrum over a wide range is analyzed.

physics.optics↗

arXiv2019

GRAINS: Generative Recursive Autoencoders for INdoor Scenes

We present a generative neural network which enables us to generate plausible 3D indoor scenes in large quantities and varieties, easily and highly efficiently. Our key observation is that indoor scene structures are inherently hierarchical. Hence, our network is not convolutional; it is a recursive neural network or RvNN. Using a dataset of annotated scene hierarchies, we train a variational recursive autoencoder, or RvNN-VAE, which performs scene object grouping during its encoding phase and scene generation during decoding. Specifically, a set of encoders are recursively applied to group 3D objects based on support, surround, and co-occurrence relations in a scene, encoding information about object spatial properties, semantics, and their relative positioning with respect to other objects in the hierarchy. By training a variational autoencoder (VAE), the resulting fixed-length codes roughly follow a Gaussian distribution. A novel 3D scene can be generated hierarchically by the decoder from a randomly sampled code from the learned distribution. We coin our method GRAINS, for Generative Recursive Autoencoders for INdoor Scenes. We demonstrate the capability of GRAINS to generate plausible and diverse 3D indoor scenes and compare with existing methods for 3D scene synthesis. We show applications of GRAINS including 3D scene modeling from 2D layouts, scene editing, and semantic scene segmentation via PointNet whose performance is boosted by the large quantity and variety of 3D scenes generated by our method.

cs.GR↗

arXiv2017

Adaptive inversion algorithm for 1.5 um visibility lidar incorporating in situ Angstrom wavelength exponent

As one of the most popular applications of lidar systems, the atmospheric visibility is defined to be inversely proportional to the atmospheric extinction coefficient at 0.55 um. Since the laser at 1.5 um shows the highest maximum permissible exposure in the wavelength ranging from 0.3 um to 10 um, the eye-safe 1.5 um lidar can be deployed in urban areas. In such a case, the measured extinction coefficient at 1.5 um should be converted to that at 0.55 um for visibility retrieval. Although several models have been established since 1962, the accurate wavelength conversion remains a challenge. An adaptive inversion algorithm for 1.5 um visibility lidar is proposed and demonstrated by using the in situ Angstrom wavelength exponent, which is derived from either a sun photometer or an aerosol spectrometer. The impact of the particle size distribution of atmospheric aerosols and the Rayleigh backscattering of atmospheric molecules are taken into account. In comparison, the Angstrom wavelength exponent derived from the sun photometer is 7.7% higher than that derived from the aerosol spectrometer. Then, using 1.5 um visibility lidar, the visibility with a temporal resolution of 5 min is detected over 48 hours in Hefei (31.83 N, 117.25 E). The average visibility error between the new method and a visibility sensor (Vaisala, PWD52) is 5.2% with the R-square value of 0.96, while the relative error between another reference visibility lidar at 532 nm and the visibility sensor is 6.7% with the R-square value of 0.91. All results agree with each other well, demonstrating the accuracy and stability of the algorithm.

physics.ao-ph↗