arXiv subjects

Bailin Deng

Publications and source records attributed to Bailin Deng.

At least 19 recordsLinked to original sources

arXiv2026

Domain-Varying 2D Green' s Functions for Cage-based Deformation

In this work, we propose a novel theoretical view of cage-based deformation based on domain-varying Green' s functions and treat this domain as a new control space for the deformation effects. Harmonic Coordinates (HC) and Green Coordinates (GC) are classic methods in cage-based deformation and serve as the theoretical foundation for shape editing in a range of practical deformation tools. Our method revisits these two classical approaches. Specifically, we propose a framework based on Green' s functions across diverse domains (independent of the cage-enclosed domain) to unify these two techniques. To our knowledge, this represents the first such attempt in nearly two decades. Based on this perspective, we propose a novel cage-based deformation technique that introduces a new control space and utilizes domain-varying Green' s functions to yield varying deformation effects. Our method also establishes a continuous transition of effects from HC to GC as the Green' s function domain $\Theta$ expands from the cage region $\Omega$ to the entire $\mathbb{R}^2$. We call our method Domain-Varying Green Coordinates (DVGC). When $\Theta$ is a disk or a rectangle, the Green' s function possesses analytic or semi-analytic expressions, respectively, enabling the DVGC to be computed without finite element discretization. Furthermore, when $\Theta$ is a disk, the DVGC admit a closed-form expression for 2D simplicial cages, thereby eliminating the need for numerical integration. Experiments demonstrate that our method provides a novel control space ranging from more consistent with the cage to more shape-preserving, generating diverse deformation effects by varying the Green' s function domains.

cs.GR↗

arXiv2026

Double-Freeform Lens Design for Angular-Spatial Control of Light Fields

Precise simultaneous control of both angular and spatial light-field distributions remains a longstanding challenge in optical design, often requiring complex multi-element configurations. In this work, we propose a compact single-lens solution that achieves unified angular-spatial modulation through the co-optimization of double freeform surfaces. The problem is formulated as an extended caustic design that enforces prescribed irradiance patterns on two distinct receptive planes, where the dual-plane constraint implicitly defines the directional characteristics of the light field while preserving spatial accuracy. This framework eliminates the need for auxiliary optical components while delivering performance comparable to that of conventional multi-lens systems. Comprehensive numerical simulations verify the method's effectiveness, demonstrating accurate and stable control of both angular and spatial light-field properties. The proposed approach establishes a practical foundation for compact, high-performance optical systems and provides a promising route toward integrated angular-spatial light-field engineering.

physics.optics↗

arXiv2025

Anisotropic Green Coordinates

We live in a world filled with anisotropy, a ubiquitous characteristic of both natural and engineered systems. In this study, we concentrate on space deformation and introduce Anisotropic Green Coordinates (AGC), which provide versatile effects for cage-based and variational deformations in both two and three dimensions. The AGC are derived from the anisotropic Laplace equation $\nabla\cdot(\mathbf{A}\nabla u)=0$, where $\mathbf{A}$ is a symmetric positive definite (SPD) matrix. Based on this equation, we establish the boundary integral formulation, which is subsequently discretized to derive the deformation coordinates defined on the vertices and normals of oriented simplicial cages. Our method satisfies basic properties such as linear reproduction and translation invariance, and possesses closed-form expressions for both 2D and 3D scenarios. We also give an intuitive geometric interpretation of the approach, demonstrating that our method can generate a quasi-conformal mapping. We demonstrate both theoretically and empirically that the deformation effect is more pronounced when the normal of the cage face aligns with the eigenvector corresponding to the larger eigenvalue of $\mathbf{A}$. This indicates that anisotropy amplifies the deformation sensitivity along this direction, enabling more targeted cage design and matrix selection. Furthermore, we derive the gradients and Hessians of the deformation coordinates and employ the local-global optimization framework to facilitate variational shape deformation, enabling flexible shape manipulation while achieving as-rigid-as-possible (ARAP) shape deformation. Experimental results demonstrate that AGC offer versatile and diverse deformation options, providing artists with enhanced flexibility and introducing a novel perspective on spatial deformation.

cs.GR↗

arXiv2025

Learning Conjugate Direction Fields for Planar Quadrilateral Mesh Generation

Planar quadrilateral (PQ) mesh generation is a key process in computer-aided design, particularly for architectural applications where the goal is to discretize a freeform surface using planar quad faces. The conjugate direction field (CDF) defined on the freeform surface plays a significant role in generating a PQ mesh, as it largely determines the PQ mesh layout. Conventionally, a CDF is obtained by solving a complex non-linear optimization problem that incorporates user preferences, i.e., aligning the CDF with user-specified strokes on the surface. This often requires a large number of iterations that are computationally expensive, preventing the interactive CDF design process for a desirable PQ mesh. To address this challenge, we propose a data-driven approach based on neural networks for controlled CDF generation. Our approach can effectively learn and fuse features from the freeform surface and the user strokes, and efficiently generate quality CDF respecting user guidance. To enable training and testing, we also present a dataset composed of 50000+ freeform surfaces with ground-truth CDFs, as well as a set of metrics for quantitative evaluation. The effectiveness and efficiency of our work are demonstrated by extensive experiments using testing data, architectural surfaces, and general 3D shapes.

cs.GR↗

arXiv2025

Computational Caustic Design for Surface Light Source

Designing freeform surfaces to control light based on real-world illumination patterns is challenging, as existing caustic lens designs often assume oversimplified point or parallel light sources. We propose representing surface light sources using an optimized set of point sources, whose parameters are fitted to the real light source's illumination using a novel differentiable rendering framework. Our physically-based rendering approach simulates light transmission using flux, without requiring prior knowledge of the light source's intensity distribution. To efficiently explore the light source parameter space during optimization, we apply a contraction mapping that converts the constrained problem into an unconstrained one. Using the optimized light source model, we then design the freeform lens shape considering flux consistency and normal integrability. Simulations and physical experiments show our method more accurately represents real surface light sources compared to point-source approximations, yielding caustic lenses that produce images closely matching the target light distributions.

cs.GR↗

arXiv2025

ROSGS: Relightable Outdoor Scenes With Gaussian Splatting

Image data captured outdoors often exhibit unbounded scenes and unconstrained, varying lighting conditions, making it challenging to decompose them into geometry, reflectance, and illumination. Recent works have focused on achieving this decomposition using Neural Radiance Fields (NeRF) or the 3D Gaussian Splatting (3DGS) representation but remain hindered by two key limitations: the high computational overhead associated with neural networks of NeRF and the use of low-frequency lighting representations, which often result in inefficient rendering and suboptimal relighting accuracy. We propose ROSGS, a two-stage pipeline designed to efficiently reconstruct relightable outdoor scenes using the Gaussian Splatting representation. By leveraging monocular normal priors, ROSGS first reconstructs the scene's geometry with the compact 2D Gaussian Splatting (2DGS) representation, providing an efficient and accurate geometric foundation. Building upon this reconstructed geometry, ROSGS then decomposes the scene's texture and lighting through a hybrid lighting model. This model effectively represents typical outdoor lighting by employing a spherical Gaussian function to capture the directional, high-frequency components of sunlight, while learning a radiance transfer function via Spherical Harmonic coefficients to model the remaining low-frequency skylight comprehensively. Both quantitative metrics and qualitative comparisons demonstrate that ROSGS achieves state-of-the-art performance in relighting outdoor scenes and highlight its ability to deliver superior relighting accuracy and rendering efficiency.

cs.CV↗

arXiv2025

Geometric Prior-Guided Neural Implicit Surface Reconstruction in the Wild

Neural implicit surface reconstruction using volume rendering techniques has recently achieved significant advancements in creating high-fidelity surfaces from multiple 2D images. However, current methods primarily target scenes with consistent illumination and struggle to accurately reconstruct 3D geometry in uncontrolled environments with transient occlusions or varying appearances. While some neural radiance field (NeRF)-based variants can better manage photometric variations and transient objects in complex scenes, they are designed for novel view synthesis rather than precise surface reconstruction due to limited surface constraints. To overcome this limitation, we introduce a novel approach that applies multiple geometric constraints to the implicit surface optimization process, enabling more accurate reconstructions from unconstrained image collections. First, we utilize sparse 3D points from structure-from-motion (SfM) to refine the signed distance function estimation for the reconstructed surface, with a displacement compensation to accommodate noise in the sparse points. Additionally, we employ robust normal priors derived from a normal predictor, enhanced by edge prior filtering and multi-view consistency constraints, to improve alignment with the actual surface geometry. Extensive testing on the Heritage-Recon benchmark and other datasets has shown that the proposed method can accurately reconstruct surfaces from in-the-wild images, yielding geometries with superior accuracy and granularity compared to existing techniques. Our approach enables high-quality 3D reconstruction of various landmarks, making it applicable to diverse scenarios such as digital preservation of cultural heritage sites.

cs.CV↗

arXiv2025

Shape from Semantics: 3D Shape Generation from Multi-View Semantics

Existing 3D reconstruction methods utilize guidances such as 2D images, 3D point clouds, shape contours and single semantics to recover the 3D surface, which limits the creative exploration of 3D modeling. In this paper, we propose a novel 3D modeling task called ``Shape from Semantics'', which aims to create 3D models whose geometry and appearance are consistent with the given text semantics when viewed from different views. The reconstructed 3D models incorporate more than one semantic elements and are easy for observers to distinguish. We adopt generative models as priors and disentangle the connection between geometry and appearance to solve this challenging problem. Specifically, we propose Local Geometry-Aware Distillation (LGAD), a strategy that employs multi-view normal-depth diffusion priors to complete partial geometries, ensuring realistic shape generation. We also integrate view-adaptive guidance scales to enable smooth semantic transitions across views. For appearance modeling, we adopt physically based rendering to generate high-quality material properties, which are subsequently baked into fabricable meshes. Extensive experimental results demonstrate that our method can generate meshes with well-structured, intricately detailed geometries, coherent textures, and smooth transitions, resulting in visually appealing 3D shape designs. Project page: https://shapefromsemantics.github.io

cs.CV↗

arXiv2025

Piecewise Ruled Approximation for Freeform Mesh Surfaces

A ruled surface is a shape swept out by moving a line in 3D space. Due to their simple geometric forms, ruled surfaces have applications in various domains such as architecture and engineering. In the past, various approaches have been proposed to approximate a target shape using developable surfaces, which are special ruled surfaces with zero Gaussian curvature. However, methods for shape approximation using general ruled surfaces remain limited and often require the target shape to be either represented as parametric surfaces or have non-positive Gaussian curvature. In this paper, we propose a method to compute a piecewise ruled surface that approximates an arbitrary freeform mesh surface. We first use a group-sparsity formulation to optimize the given mesh shape into an approximately piecewise ruled form, in conjunction with a tangent vector field that indicates the ruling directions. Afterward, we utilize the optimization result to extract seams that separate smooth families of rulings, and use the seams to construct the initial rulings. Finally, we further optimize the positions and orientations of the rulings to improve the alignment with the input target shape. We apply our method to a variety of freeform shapes with different topologies and complexity, demonstrating its effectiveness in approximating arbitrary shapes.

cs.GR↗

arXiv2025

Scalable and High-Quality Neural Implicit Representation for 3D Reconstruction

Various SDF-based neural implicit surface reconstruction methods have been proposed recently, and have demonstrated remarkable modeling capabilities. However, due to the global nature and limited representation ability of a single network, existing methods still suffer from many drawbacks, such as limited accuracy and scale of the reconstruction. In this paper, we propose a versatile, scalable and high-quality neural implicit representation to address these issues. We integrate a divide-and-conquer approach into the neural SDF-based reconstruction. Specifically, we model the object or scene as a fusion of multiple independent local neural SDFs with overlapping regions. The construction of our representation involves three key steps: (1) constructing the distribution and overlap relationship of the local radiance fields based on object structure or data distribution, (2) relative pose registration for adjacent local SDFs, and (3) SDF blending. Thanks to the independent representation of each local region, our approach can not only achieve high-fidelity surface reconstruction, but also enable scalable scene reconstruction. Extensive experimental results demonstrate the effectiveness and practicality of our proposed method.

cs.CV↗

arXiv2024

Neural Shadow Art

Shadow art is a captivating form of sculptural expression where the projection of a sculpture in a specific direction reveals a desired shape with high precision. In this work, we introduce Neural Shadow Art, which leverages implicit occupancy function representation to significantly expand the possibilities of shadow art. This representation enables the design of high-quality, 3D-printable geometric models with arbitrary topologies at any resolution, surpassing previous voxel- and mesh-based methods. Our method provides a more flexible framework, enabling projections to match input binary images under various light directions and screen orientations, without requiring light sources to be perpendicular to the screens. Furthermore, we allow rigid transformations of the projected geometries relative to the input binary images and simultaneously optimize light directions and screen orientations to ensure that the projections closely resemble the target images, especially when dealing with inputs of complex topologies. In addition, our model promotes surface smoothness and reduces material usage. This is particularly advantageous for efficient industrial production and enhanced artistic effect by generating compelling shadow art that avoids trivial, intersecting cylindrical structures. In summary, we propose a more flexible representation for shadow art, significantly improving projection accuracy while simultaneously meeting industrial requirements and delivering awe-inspiring artistic effects.

cs.CV↗

arXiv2024

End-to-end Surface Optimization for Light Control

Designing a freeform surface to reflect or refract light to achieve a target distribution is a challenging inverse problem. In this paper, we propose an end-to-end optimization strategy for an optical surface mesh. Our formulation leverages a novel differentiable rendering model, and is directly driven by the difference between the resulting light distribution and the target distribution. We also enforce geometric constraints related to fabrication requirements, to facilitate CNC milling and polishing of the designed surface. To address the issue of local minima, we formulate a face-based optimal transport problem between the current mesh and the target distribution, which makes effective large changes to the surface shape. The combination of our optimal transport update and rendering-guided optimization produces an optical surface design with a resulting image closely resembling the target, while the geometric constraints in our optimization help to ensure consistency between the rendering model and the final physical results. The effectiveness of our algorithm is demonstrated on a variety of target images using both simulated rendering and physical prototypes.

cs.GR↗

arXiv2024

SPARE: Symmetrized Point-to-Plane Distance for Robust Non-Rigid 3D Registration

Existing optimization-based methods for non-rigid registration typically minimize an alignment error metric based on the point-to-point or point-to-plane distance between corresponding point pairs on the source surface and target surface. However, these metrics can result in slow convergence or a loss of detail. In this paper, we propose SPARE, a novel formulation that utilizes a symmetrized point-to-plane distance for robust non-rigid registration. The symmetrized point-to-plane distance relies on both the positions and normals of the corresponding points, resulting in a more accurate approximation of the underlying geometry and can achieve higher accuracy than existing methods. To solve this optimization problem efficiently, we introduce an as-rigid-as-possible regulation term to estimate the deformed normals and propose an alternating minimization solver using a majorization-minimization strategy. Moreover, for effective initialization of the solver, we incorporate a deformation graph-based coarse alignment that improves registration quality and efficiency. Extensive experiments show that the proposed method greatly improves the accuracy of non-rigid registration problems and maintains relatively high solution efficiency. The code is publicly available at https://github.com/yaoyx689/spare.

cs.CV↗

arXiv2024

Training-free Editioning of Text-to-Image Models

Inspired by the software industry's practice of offering different editions or versions of a product tailored to specific user groups or use cases, we propose a novel task, namely, training-free editioning, for text-to-image models. Specifically, we aim to create variations of a base text-to-image model without retraining, enabling the model to cater to the diverse needs of different user groups or to offer distinct features and functionalities. To achieve this, we propose that different editions of a given text-to-image model can be formulated as concept subspaces in the latent space of its text encoder (e.g., CLIP). In such a concept subspace, all points satisfy a specific user need (e.g., generating images of a cat lying on the grass/ground/falling leaves). Technically, we apply Principal Component Analysis (PCA) to obtain the desired concept subspaces from representative text embedding that correspond to a specific user need or requirement. Projecting the text embedding of a given prompt into these low-dimensional subspaces enables efficient model editioning without retraining. Intuitively, our proposed editioning paradigm enables a service provider to customize the base model into its "cat edition" (or other editions) that restricts image generation to cats, regardless of the user's prompt (e.g., dogs, people, etc.). This introduces a new dimension for product differentiation, targeted functionality, and pricing strategies, unlocking novel business models for text-to-image generators. Extensive experimental results demonstrate the validity of our approach and its potential to enable a wide range of customized text-to-image model editions across various domains and applications.

cs.CV↗

arXiv2024

Oblique-MERF: Revisiting and Improving MERF for Oblique Photography

Neural implicit fields have established a new paradigm for scene representation, with subsequent work achieving high-quality real-time rendering. However, reconstructing 3D scenes from oblique aerial photography presents unique challenges, such as varying spatial scale distributions and a constrained range of tilt angles, often resulting in high memory consumption and reduced rendering quality at extrapolated viewpoints. In this paper, we enhance MERF to accommodate these data characteristics by introducing an innovative adaptive occupancy plane optimized during the volume rendering process and a smoothness regularization term for view-dependent color to address these issues. Our approach, termed Oblique-MERF, surpasses state-of-the-art real-time methods by approximately 0.7 dB, reduces VRAM usage by about 40%, and achieves higher rendering frame rates with more realistic rendering outcomes across most viewpoints.

cs.CV↗

arXiv2024

Learning with Unreliability: Fast Few-shot Voxel Radiance Fields with Relative Geometric Consistency

We propose a voxel-based optimization framework, ReVoRF, for few-shot radiance fields that strategically address the unreliability in pseudo novel view synthesis. Our method pivots on the insight that relative depth relationships within neighboring regions are more reliable than the absolute color values in disoccluded areas. Consequently, we devise a bilateral geometric consistency loss that carefully navigates the trade-off between color fidelity and geometric accuracy in the context of depth consistency for uncertain regions. Moreover, we present a reliability-guided learning strategy to discern and utilize the variable quality across synthesized views, complemented by a reliability-aware voxel smoothing algorithm that smoothens the transition between reliable and unreliable data patches. Our approach allows for a more nuanced use of all available data, promoting enhanced learning from regions previously considered unsuitable for high-quality reconstruction. Extensive experiments across diverse datasets reveal that our approach attains significant gains in efficiency and accuracy, delivering rendering speeds of 3 FPS, 7 mins to train a $360^\circ$ scene, and a 5\% improvement in PSNR over existing few-shot methods. Code is available at https://github.com/HKCLynn/ReVoRF.

cs.CV↗

arXiv2024

Winding Clearness for Differentiable Point Cloud Optimization

We propose to explore the properties of raw point clouds through the \emph{winding clearness}, a concept we first introduce for measuring the clarity of the interior/exterior relationships represented by the winding number field of the point cloud. In geometric modeling, the winding number is a powerful tool for distinguishing the interior and exterior of a given surface $\partial \Omega$, and it has been previously used for point normal orientation and surface reconstruction. In this work, we introduce a novel approach to evaluate and optimize the quality of point clouds based on the winding clearness. We observe that point clouds with less noise generally exhibit better winding clearness. Accordingly, we propose an objective function that quantifies the error in winding clearness, solely utilizing the coordinates of the point clouds. Moreover, we demonstrate that the winding clearness error is differentiable and can serve as a loss function in point cloud processing. We present this observation from two aspects: 1) We update the coordinates of the points by back-propagating the loss function for individual point clouds, resulting in an overall improvement without involving a neural network. 2) We incorporate winding clearness as a geometric constraint in the diffusion-based 3D generative model and update the network parameters to generate point clouds with less noise. Experimental results demonstrate the effectiveness of optimizing the winding clearness in enhancing the point cloud quality. Notably, our method exhibits superior performance in handling noisy point clouds with thin structures, highlighting the benefits of the global perspective enabled by the winding number.

cs.GR↗

arXiv2023

Efficient Multi-View Inverse Rendering Using a Hybrid Differentiable Rendering Method

Recovering the shape and appearance of real-world objects from natural 2D images is a long-standing and challenging inverse rendering problem. In this paper, we introduce a novel hybrid differentiable rendering method to efficiently reconstruct the 3D geometry and reflectance of a scene from multi-view images captured by conventional hand-held cameras. Our method follows an analysis-by-synthesis approach and consists of two phases. In the initialization phase, we use traditional SfM and MVS methods to reconstruct a virtual scene roughly matching the real scene. Then in the optimization phase, we adopt a hybrid approach to refine the geometry and reflectance, where the geometry is first optimized using an approximate differentiable rendering method, and the reflectance is optimized afterward using a physically-based differentiable rendering method. Our hybrid approach combines the efficiency of approximate methods with the high-quality results of physically-based methods. Extensive experiments on synthetic and real data demonstrate that our method can produce reconstructions with similar or higher quality than state-of-the-art methods while being more efficient.

cs.CV↗