arXiv subjects

Thomas W. Mitchel

Publications and source records attributed to Thomas W. Mitchel.

13 recordsLinked to original sources

arXiv2026

DeepShapeMatchingKit: Accelerated Functional Map Solver and Shape Matching Pipelines Revisited

Deep functional maps, leveraging learned feature extractors and spectral correspondence solvers, are fundamental to non-rigid 3D shape matching. Based on an analysis of open-source implementations, we find that standard functional map implementations solve k independent linear systems serially, which is a computational bottleneck at higher spectral resolution. We thus propose a vectorized reformulation that solves all systems in a single kernel call, achieving up to a 33x speedup while preserving the exact solution. Furthermore, we identify and document a previously unnoticed implementation divergence in the spatial gradient features of the mainstay DiffusionNet: two variants that parameterize distinct families of tangent-plane transformations, and present experiments analyzing their respective behaviors across diverse benchmarks. We additionally revisit overlap prediction evaluation for partial-to-partial matching and show that balanced accuracy provides a useful complementary metric under varying overlap ratios. To share these advancements with the wider community, we present an open-source codebase, DeepShapeMatchingKit, that incorporates these improvements and standardizes training, evaluation, and data pipelines for common deep shape matching methods. The codebase is available at: https://github.com/xieyizheng/DeepShapeMatchingKit

cs.CV↗

arXiv2026

True Self-Supervised Novel View Synthesis is Transferable

In this paper, we identify that the key criterion for determining whether a model is truly capable of novel view synthesis (NVS) is transferability: Whether any pose representation extracted from one video sequence can be used to re-render the same camera trajectory in another. We analyze prior work on self-supervised NVS and find that their predicted poses do not transfer: The same set of poses lead to different camera trajectories in different 3D scenes. Here, we present XFactor, the first geometry-free self-supervised model capable of true NVS. XFactor combines pair-wise pose estimation with a simple augmentation scheme of the inputs and outputs that jointly enables disentangling camera pose from scene content and facilitates geometric reasoning. Remarkably, we show that XFactor achieves transferability with unconstrained latent pose variables, without any 3D inductive biases or concepts from multi-view geometry -- such as an explicit parameterization of poses as elements of SE(3). We introduce a new metric to quantify transferability, and through large-scale experiments, we demonstrate that XFactor significantly outperforms prior pose-free NVS transformers, and show that latent poses are highly correlated with real-world poses through probing experiments.

cs.CV↗

arXiv2026

Scaling View Synthesis Transformers

Geometry-free view synthesis transformers have recently achieved state-of-the-art performance in Novel View Synthesis (NVS), outperforming traditional approaches that rely on explicit geometry modeling. Yet the factors governing their scaling with compute remain unclear. We present a systematic study of scaling laws for view synthesis transformers and derive design principles for training compute-optimal NVS models. Contrary to prior findings, we show that encoder-decoder architectures can be compute-optimal; we trace earlier negative results to suboptimal architectural choices and comparisons across unequal training compute budgets. Across several compute levels, we demonstrate that our encoder-decoder architecture, which we call the Scalable View Synthesis Model (SVSM), scales as effectively as decoder-only models, achieves a superior performance-compute Pareto frontier, and surpasses the previous state-of-the-art on real-world NVS benchmarks with substantially reduced training compute.

cs.CV↗

arXiv2025

Continuous body 3-D reconstruction of limbless animals

Limbless animals such as snakes, limbless lizards, worms, eels, and lampreys move their slender, long bodies in three dimensions to traverse diverse environments. Accurately quantifying their continuous body's 3-D shape and motion is important for understanding body-environment interactions in complex terrain, but this is difficult to achieve (especially for local orientation and rotation). Here, we describe an interpolation method to quantify continuous body 3-D position and orientation. We simplify the body as an elastic rod and apply a backbone optimization method to interpolate continuous body shape between end constraints imposed by tracked markers. Despite over-simplifying the biomechanics, our method achieves a higher interpolation accuracy (~50% error) in both 3-D position and orientation compared with the widely-used cubic B-spline interpolation method. Beyond snakes traversing large obstacles as demonstrated, our method applies to other long, slender, limbless animals and continuum robots. We provide codes and demo files for easy application of our method.

physics.bio-ph↗

arXiv2025

Lateral oscillation and body compliance help snakes and snake robots stably traverse large, smooth obstacles

Snakes can move through almost any terrain. Similarly, snake robots hold the promise as a versatile platform to traverse complex environments like earthquake rubble. Unlike snake locomotion on flat surfaces which is inherently stable, when snakes traverse complex terrain by deforming their body out of plane, it becomes challenging to maintain stability. Here, we review our recent progress in understanding how snakes and snake robots traverse large, smooth obstacles that lack anchor points for gripping or bracing. First, we discovered that the generalist variable kingsnake combines lateral oscillation and cantilevering. Regardless of step height and surface friction, the overall gait is preserved. Next, to quantify static stability of the snake, we developed a method to interpolate continuous body in three dimensions (both position and orientation) between discrete tracked markers. By analyzing the base of support using the interpolated continuous body 3-D kinematics, we discovered that the snake maintained perfect stability during traversal, even on the most challenging low friction, high step. Finally, we applied this gait to a snake robot and systematically tested its performance traversing large steps with variable heights to further understand stability principles. The robot rapidly and stably traversed steps nearly as high as a third of its body length. As step height increased, the robot rolled more frequently to the extent of flipping over, reducing traversal probability. The absence of such failure in the snake with a compliant body inspired us to add body compliance to the robot. With better surface contact, the compliant body robot suffered less roll instability and traversed high steps at higher probability, without sacrificing traversal speed. Our robot traversed large step-like obstacles more rapidly than most previous snake robots, approaching that of the animal.

physics.bio-ph↗

arXiv2025

Snakes partition their body to traverse large steps stably

Many snakes live in deserts, forests, and river valleys and traverse challenging 3-D terrain like rocks, felled trees, and rubble, with obstacles as large as themselves and variable surface properties. By contrast, apart from branch cantilevering, burrowing, swimming, and gliding, laboratory studies of snake locomotion focused on that on simple flat surfaces. Here, to begin to understand snake locomotion in complex 3-D terrain, we study how the variable kingsnake, a terrestrial generalist, traversed a large step of variable surface friction and step height (up to 30% snout-vent length). The snake traversed by partitioning its body into three sections with distinct functions. Body sections below and above the step oscillated laterally on horizontal surfaces for propulsion, while the body section in between cantilevered in a vertical plane to bridge the large height increase. As the animal progressed, these three sections traveled down its body, conforming overall body shape to the step. In addition, the snake adjusted the partitioned gait in response to increase in step height and decrease in surface friction, at the cost of reduced speed. As surface friction decreased, body movement below and above the step changed from a continuous lateral undulation with little slip to an intermittent oscillatory movement with much slip, and initial head lift-off became closer to the step. Given these adjustments, body partitioning allowed the snake to be always stable, even when initially cantilevering but before reaching the surface above. Such a partitioned gait may be generally useful for diverse, complex 3-D terrain.

physics.bio-ph↗

arXiv2024

Neural Isometries: Taming Transformations for Equivariant ML

Real-world geometry and 3D vision tasks are replete with challenging symmetries that defy tractable analytical expression. In this paper, we introduce Neural Isometries, an autoencoder framework which learns to map the observation space to a general-purpose latent space wherein encodings are related by isometries whenever their corresponding observations are geometrically related in world space. Specifically, we regularize the latent space such that maps between encodings preserve a learned inner product and commute with a learned functional operator, in the same manner as rigid-body transformations commute with the Laplacian. This approach forms an effective backbone for self-supervised representation learning, and we demonstrate that a simple off-the-shelf equivariant network operating in the pre-trained latent space can achieve results on par with meticulously-engineered, handcrafted networks designed to handle complex, nonlinear symmetries. Furthermore, isometric maps capture information about the respective transformations in world space, and we show that this allows us to regress camera poses directly from the coefficients of the maps between encodings of adjacent views of a scene.

cs.CV↗

arXiv2024

Single Mesh Diffusion Models with Field Latents for Texture Generation

We introduce a framework for intrinsic latent diffusion models operating directly on the surfaces of 3D shapes, with the goal of synthesizing high-quality textures. Our approach is underpinned by two contributions: field latents, a latent representation encoding textures as discrete vector fields on the mesh vertices, and field latent diffusion models, which learn to denoise a diffusion process in the learned latent space on the surface. We consider a single-textured-mesh paradigm, where our models are trained to generate variations of a given texture on a mesh. We show the synthesized textures are of superior fidelity compared those from existing single-textured-mesh generative models. Our models can also be adapted for user-controlled editing tasks such as inpainting and label-guided generation. The efficacy of our approach is due in part to the equivariance of our proposed framework under isometries, allowing our models to seamlessly reproduce details across locally similar regions and opening the door to a notion of generative texture transfer.

cs.CV↗

arXiv2022

Möbius Convolutions for Spherical CNNs

Möbius transformations play an important role in both geometry and spherical image processing - they are the group of conformal automorphisms of 2D surfaces and the spherical equivalent of homographies. Here we present a novel, Möbius-equivariant spherical convolution operator which we call Möbius convolution, and with it, develop the foundations for Möbius-equivariant spherical CNNs. Our approach is based on a simple observation: to achieve equivariance, we only need to consider the lower-dimensional subgroup which transforms the positions of points as seen in the frames of their neighbors. To efficiently compute Möbius convolutions at scale we derive an approximation of the action of the transformations on spherical filters, allowing us to compute our convolutions in the spectral domain with the fast Spherical Harmonic Transform. The resulting framework is both flexible and descriptive, and we demonstrate its utility by achieving promising results in both shape classification and image segmentation tasks.

cs.CV↗

arXiv2021

Field Convolutions for Surface CNNs

We present a novel surface convolution operator acting on vector fields that is based on a simple observation: instead of combining neighboring features with respect to a single coordinate parameterization defined at a given point, we have every neighbor describe the position of the point within its own coordinate frame. This formulation combines intrinsic spatial convolution with parallel transport in a scattering operation while placing no constraints on the filters themselves, providing a definition of convolution that commutes with the action of isometries, has increased descriptive potential, and is robust to noise and other nuisance factors. The result is a rich notion of convolution which we call field convolution, well-suited for CNNs on surfaces. Field convolutions are flexible, straight-forward to incorporate into surface learning frameworks, and their highly discriminating nature has cascading effects throughout the learning pipeline. Using simple networks constructed from residual field convolution blocks, we achieve state-of-the-art results on standard benchmarks in fundamental geometry processing tasks, such as shape classification, segmentation, correspondence, and sparse matching.

cs.CV↗

arXiv2020

Efficient Spatially Adaptive Convolution and Correlation

Fast methods for convolution and correlation underlie a variety of applications in computer vision and graphics, including efficient filtering, analysis, and simulation. However, standard convolution and correlation are inherently limited to fixed filters: spatial adaptation is impossible without sacrificing efficient computation. In early work, Freeman and Adelson have shown how steerable filters can address this limitation, providing a way for rotating the filter as it is passed over the signal. In this work, we provide a general, representation-theoretic, framework that allows for spatially varying linear transformations to be applied to the filter. This framework allows for efficient implementation of extended convolution and correlation for transformation groups such as rotation (in 2D and 3D) and scale, and provides a new interpretation for previous methods including steerable filters and the generalized Hough transform. We present applications to pattern matching, image feature description, vector field visualization, and adaptive image filtering.

cs.CV↗

arXiv2019

Quotienting Impertinent Camera Kinematics for 3D Video Stabilization

With the recent advent of methods that allow for real-time computation, dense 3D flows have become a viable basis for fast camera motion estimation. Most importantly, dense flows are more robust than the sparse feature matching techniques used by existing 3D stabilization methods, able to better handle large camera displacements and occlusions similar to those often found in consumer videos. Here we introduce a framework for 3D video stabilization that relies on dense scene flow alone. The foundation of this approach is a novel camera motion model that allows for real-world camera poses to be recovered directly from 3D motion fields. Moreover, this model can be extended to describe certain types of non-rigid artifacts that are commonly found in videos, such as those resulting from zooms. This framework gives rise to several robust regimes that produce high-quality stabilization of the kind achieved by prior full 3D methods while avoiding the fragility typically present in feature-based approaches. As an added benefit, our framework is fast: the simplicity of our motion model and efficient flow calculations combine to enable stabilization at a high frame rate.

cs.CV↗

arXiv2018

Signal Alignment for Humanoid Skeletons via the Globally Optimal Reparameterization Algorithm

The general ability to analyze and classify the 3D kinematics of the human form is an essential step in the development of socially adept humanoid robots. A variety of different types of signals can be used by machines to represent and characterize actions such as RGB videos, infrared maps, and optical flow. In particular, skeleton sequences provide a natural 3D kinematic description of human motions and can be acquired in real time using RGB+D cameras. Moreover, skeleton sequences are generalizable to characterize the motions of both humans and humanoid robots. The Globally Optimal Reparameterization Algorithm (GORA) is a novel, recently proposed algorithm for signal alignment in which signals are reparameterized to a globally optimal universal standard timescale (UST). Here, we introduce a variant of GORA for humanoid action recognition with skeleton sequences, which we call GORA-S. We briefly review the algorithm's mathematical foundations and contextualize them in the problem of action recognition with skeleton sequences. Subsequently, we introduce GORA-S and discuss parameters and numerical techniques for its effective implementation. We then compare its performance with that of the DTW and FastDTW algorithms, in terms of computational efficiency and accuracy in matching skeletons. Our results show that GORA-S attains a complexity that is significantly less than that of any tested DTW method. In addition, it displays a favorable balance between speed and accuracy that remains invariant under changes in skeleton sampling frequency, lending it a degree of versatility that could make it well-suited for a variety of action recognition tasks.

cs.CV↗