Neural Implicit Mapping via Nested Neighborhoods

Abstract

We introduce a novel approach for rendering static and dynamic 3D neural signed distance functions (SDF) in real-time. We rely on nested neighborhoods of zero-level sets of neural SDFs, and mappings between them. This framework supports animations and achieves real-time performance without the use of spatial data-structures. It consists of three uncoupled algorithms representing the rendering steps. The multiscale sphere tracing focuses on minimizing iteration time by using coarse approximations on earlier iterations. The neural normal mapping transfers details from a fine neural SDF to a surface nested on a neighborhood of its zero-level set. It is smooth and it does not depend on surface parametrizations. As a result, it can be used to fetch smooth normals for discrete surfaces such as meshes and to skip later iterations when sphere tracing level sets. Finally, we propose an algorithm for analytic normal calculation for MLPs and describe ways to obtain sequences of neural SDFs to use with the algorithms.