arXiv subjects

Jernej Barbic

Publications and source records attributed to Jernej Barbic.

4 recordsLinked to original sources

arXiv2025

Seeing the Wind from a Falling Leaf

A longstanding goal in computer vision is to model motions from videos, while the representations behind motions, i.e. the invisible physical interactions that cause objects to deform and move, remain largely unexplored. In this paper, we study how to recover the invisible forces from visual observations, e.g., estimating the wind field by observing a leaf falling to the ground. Our key innovation is an end-to-end differentiable inverse graphics framework, which jointly models object geometry, physical properties, and interactions directly from videos. Through backpropagation, our approach enables the recovery of force representations from object motions. We validate our method on both synthetic and real-world scenarios, and the results demonstrate its ability to infer plausible force fields from videos. Furthermore, we show the potential applications of our approach, including physics-based video generation and editing. We hope our approach sheds light on understanding and modeling the physical process behind pixels, bridging the gap between vision and physics. Please check more video results in our \href{https://chaoren2357.github.io/seeingthewind/}{project page}.

cs.CV↗

arXiv2024

Tuning Nonlinear Elastic Materials under Small and Large Deformations

In computer graphics and engineering, nonlinear elastic material properties of 3D volumetric solids are typically adjusted by selecting a material family, such as St. Venant Kirchhoff, Linear Corotational, (Stable) Neo-Hookean, Ogden, etc., and then selecting the values of the specific parameters for that family, such as the Lame parameters, Ogden exponents, or whatever the parameterization of a particular family may be. However, the relationships between those parameter values, and visually intuitive material properties such as object's "stiffness", volume preservation, or the "amount of nonlinearity", are less clear and can be tedious to tune. For an arbitrary isotropic hyperelastic energy density function psi that is not parameterized in terms of the Lame parameters, it is not even clear what the Lame parameters and Young's modulus and Poisson's ratio are. Starting from psi, we first give a concise definition of Lame parameters, and therefore Young's modulus and Poisson's ratio. Second, we give a method to adjust the object's three salient properties, namely two small-deformation properties (overall "stiffness", and amount of volume preservation, prescribed by object's Young's modulus and Poisson's ratio), and one large-deformation property (material nonlinearity). We do this in a manner whereby each of these three properties is decoupled from the other two properties, and can therefore be set independently. This permits a new ability, namely "normalization" of materials: starting from two distinct materials, we can "normalize" them so that they have the same small deformation properties, or the same large-deformation nonlinearity behavior, or both. Furthermore, our analysis produced a useful theoretical result, namely it establishes that Linear Corotational materials (arguably the most widely used materials in computer graphics) are the simplest possible nonlinear materials.

cs.GR↗

arXiv2024

Volume Rendering of Human Hand Anatomy

We study the design of transfer functions for volumetric rendering of magnetic resonance imaging (MRI) datasets of human hands. Human hands are anatomically complex, containing various organs within a limited space, which presents challenges for volumetric rendering. We focus on hand musculoskeletal organs because they are volumetrically the largest inside the hand, and most important for the hand's main function, namely manipulation of objects. While volumetric rendering is a mature field, the choice of the transfer function for the different organs is arguably just as important as the choice of the specific volume rendering algorithm; we demonstrate that it significantly influences the clarity and interpretability of the resulting images. We assume that the hand MRI scans have already been segmented into the different organs (bones, muscles, tendons, ligaments, subcutaneous fat, etc.). Our method uses the hand MRI volume data, and the geometry of its inner organs and their known segmentation, to produce high-quality volume rendering images of the hand, and permits fine control over the appearance of each tissue. We contribute two families of transfer functions to emphasize different hand tissues of interest, while preserving the visual context of the hand. We also discuss and reduce artifacts present in standard volume ray-casting of human hands. We evaluate our volumetric rendering on five challenging hand motion sequences. Our experimental results demonstrate that our method improves hand anatomy visualization, compared to standard surface and volume rendering techniques.

cs.GR↗

arXiv2020

Modeling of Personalized Anatomy using Plastic Strains

We give a method for modeling solid objects undergoing large spatially varying and/or anisotropic strains, and use it to reconstruct human anatomy from medical images. Our novel shape deformation method uses plastic strains and the Finite Element Method to successfully model shapes undergoing large and/or anisotropic strains, specified by sparse point constraints on the boundary of the object. We extensively compare our method to standard second-order shape deformation methods, variational methods and surface-based methods and demonstrate that our method avoids the spikiness, wiggliness and other artefacts of previous methods. We demonstrate how to perform such shape deformation both for attached and un-attached ("free flying") objects, using a novel method to solve linear systems with singular matrices with a known nullspace. While our method is applicable to general large-strain shape deformation modeling, we use it to create personalized 3D triangle and volumetric meshes of human organs, based on MRI or CT scans. Given a medically accurate anatomy template of a generic individual, we optimize the geometry of the organ to match the MRI or CT scan of a specific individual. Our examples include human hand muscles, a liver, a hip bone, and a gluteus medius muscle ("hip abductor").

cs.GR↗