arXiv subjects

Jiayin Lu

Publications and source records attributed to Jiayin Lu.

12 recordsLinked to original sources

arXiv2026

JARVIS: A Just-in-Time Augmented Reality VLM-Powered Instruction System for Cross-Reality Task Guidance

Many everyday tasks rely on external tutorials such as manuals and videos, requiring users to constantly switch between reading instructions and performing actions, which disrupts workflow and increases cognitive load. Augmented reality (AR) enables in-situ guidance, while recent advances in large language models (LLMs) and vision-language models (VLMs) make it possible to automatically generate such guidance. However, existing AI-powered AR tutorial systems primarily focus on physical procedural tasks and provide limited support for hybrid physical and virtual workspaces. To address this gap, we conduct a formative study of cross-reality tasks and identify key requirements for state awareness and cross-reality coordination. We present JARVIS, a VLM-driven AR instruction system that generates contextual, step-by-step guidance from a single prompt, with real-time state verification and adaptive visual feedback. To inform the system design, we conducted a formative study to understand guidance needs across cross-reality tasks, which we categorize into four types, real-to-real (R2R), real-to-virtual (R2V), virtual-to-real (V2R), and virtual-to-virtual (V2V). A within-subjects study (N=14) across four domains shows JARVIS improves usability, workload, success rate, and visualization effectiveness over baselines.

cs.HC↗

arXiv2026

VoroLight: Learning Voronoi Surface Meshes via Sphere Intersection

Voronoi diagrams naturally produce convex, watertight, and topologically consistent cells, making them an appealing representation for 3D shape reconstruction. However, standard differentiable Voronoi approaches typically optimize generator positions in stable configurations, which can lead to locally uneven surface geometry. We present VoroLight, a differentiable framework that promotes controlled Voronoi degeneracy for smooth surface reconstruction. Instead of optimizing generator positions alone, VoroLight associates each Voronoi surface vertex with a trainable sphere and introduces a sphere--intersection loss that encourages higher-order equidistance among face-incident generators. This formulation improves surface regularity while preserving intrinsic Voronoi properties such as watertightness and convexity. Because losses are defined directly on surface vertices, VoroLight supports multimodal shape supervision from implicit fields, point clouds, meshes, and multi--view images. By introducing additional interior generators optimized under a centroidal Voronoi tessellation objective, the framework naturally extends to volumetric Voronoi meshes with consistent surface--interior topology. Across diverse input modalities, VoroLight achieves competitive reconstruction fidelity while producing smoother and more geometrically regular Voronoi surfaces. Project page: https://jiayinlu19960224.github.io/vorolight/

cs.CG↗

arXiv2026

SeeClear: Reliable Transparent Object Depth Estimation via Generative Opacification

Monocular depth estimation remains challenging for transparent objects, where refraction and transmission are difficult to model and break the appearance assumptions used by depth networks. As a result, state-of-the-art estimators often produce unstable or incorrect depth predictions for transparent materials. We propose SeeClear, a novel framework that converts transparent objects into generative opaque images, enabling stable monocular depth estimation for transparent objects. Given an input image, we first localize transparent regions and transform their refractive appearance into geometrically consistent opaque shapes using a diffusion-based generative opacification module. The processed image is then fed into an off-the-shelf monocular depth estimator without retraining or architectural changes. To train the opacification model, we construct SeeClear-396k, a synthetic dataset containing 396k paired transparent-opaque renderings. Experiments on both synthetic and real-world datasets show that SeeClear significantly improves depth estimation for transparent objects. Project page: https://heyumeng.com/SeeClear-web/

cs.CV↗

arXiv2025

Recent Advances in Metallic Glasses

This paper reviews recent advances in the field of metallic glasses, focusing on the development of novel experimental techniques and in silico models. We discuss progress in experimental characterization, additive manufacturing, multiscale modeling approaches, and the growing role of machine learning in understanding and designing these complex materials. On the experimental side, we highlight measurements of thermophysical properties of supercooled liquids via fast chip calorimetry and enhancements in mechanical properties through rejuvenation treatments. This work underscores the crucial role of short-range order and medium-range order in controlling metallic glass mechanical properties. Recent progress in structural probes allows in situ observations of deformation mechanisms, positioning the field well to further advance our understanding of mechanical properties. Additive manufacturing of metallic glasses is discussed as one encouraging new manufacturing route for metallic glasses. We examine laser powder-bed fusion process physics and the central trade-off between amorphicity and densification, including heat affected zone devitrification and defects formation, together with emerging mitigation strategies and applications. On the theoretical and simulation side, we review advances in nanoscale, mesoscale, and continuum modeling of metallic glasses that have led to promising approaches by which multiscale schemes can incorporate data sourced from atomic-scale simulations. These efforts have helped to elucidate the connection between the glass structure and mechanical and rheological responses. We also cover the development of machine learning interatomic potentials for metallic glasses, along with machine learning driven prediction of glass forming ability and inverse design methods. Finally, challenges and directions for future research are presented and discussed.

cond-mat.mtrl-sci↗

arXiv2025

SPARK: Sim-ready Part-level Articulated Reconstruction with VLM Knowledge

Articulated 3D objects are critical for embodied AI, robotics, and interactive scene understanding, yet creating simulation-ready assets remains labor-intensive and requires expert modeling of part hierarchies and motion structures. We introduce SPARK, a framework for reconstructing physically consistent, kinematic part-level articulated objects from a single RGB image. Given an input image, we first leverage VLMs to extract coarse URDF parameters and generate part-level reference images. We then integrate the part-image guidance and the inferred structure graph into a generative diffusion transformer to synthesize consistent part and complete shapes of articulated objects. To further refine the URDF parameters, we incorporate differentiable forward kinematics and differentiable rendering to optimize joint types, axes, and origins under VLM-generated open-state supervision. Extensive experiments show that SPARK produces high-quality, simulation-ready articulated assets across diverse categories, enabling downstream applications such as robotic manipulation and interaction modeling. Project page: https://heyumeng.com/SPARK/index.html.

cs.CV↗

arXiv2025

Birth of a Painting: Differentiable Brushstroke Reconstruction

Painting embodies a unique form of visual storytelling, where the creation process is as significant as the final artwork. Although recent advances in generative models have enabled visually compelling painting synthesis, most existing methods focus solely on final image generation or patch-based process simulation, lacking explicit stroke structure and failing to produce smooth, realistic shading. In this work, we present a differentiable stroke reconstruction framework that unifies painting, stylized texturing, and smudging to faithfully reproduce the human painting-smudging loop. Given an input image, our framework first optimizes single- and dual-color Bezier strokes through a parallel differentiable paint renderer, followed by a style generation module that synthesizes geometry-conditioned textures across diverse painting styles. We further introduce a differentiable smudge operator to enable natural color blending and shading. Coupled with a coarse-to-fine optimization strategy, our method jointly optimizes stroke geometry, color, and texture under geometric and semantic guidance. Extensive experiments on oil, watercolor, ink, and digital paintings demonstrate that our approach produces realistic and expressive stroke reconstructions, smooth tonal transitions, and richly stylized appearances, offering a unified model for expressive digital painting creation. See our project page for more demos: https://yingjiang96.github.io/DiffPaintWebsite/.

cs.CV↗

arXiv2025

Numerical methods and improvements for simulating quasi-static elastoplastic materials

Hypo-elastoplasticity is a framework suitable for modeling the mechanics of many hard materials that have small elastic deformation and large plastic deformation. In most laboratory tests for these materials the Cauchy stress is in quasi-static equilibrium. Rycroft et al. discovered a mathematical correspondence between this physical system and the incompressible Navier-Stokes equations, and developed a projection method similar to Chorin's projection method (1968) for incompressible Newtonian fluids. Here, we improve the original projection method to simulate quasi-static hypo-elastoplasticity, by making three improvements. First, drawing inspiration from the second-order projection method for incompressible Newtonian fluids, we formulate a second-order in time numerical scheme for quasi-static hypo-elastoplasticity. Second, we implement a finite element method for solving the elliptic equations in the projection step, which provides both numerical benefits and flexibility. Third, we develop an adaptive global time-stepping scheme, which can compute accurate solutions in fewer timesteps. Our numerical tests use an example physical model of a bulk metallic glass based on the shear transformation zone theory, but the numerical methods can be applied to any elastoplastic material.

physics.comp-ph↗

arXiv2024

TriMe++: Multi-threaded triangular meshing in two dimensions

We present TriMe++, a multi-threaded software library designed for generating two-dimensional meshes for intricate geometric shapes using the Delaunay triangulation. Multi-threaded parallel computing is implemented throughout the meshing procedure, making it suitable for fast generation of large-scale meshes. Three iterative meshing algorithms are implemented: the DistMesh algorithm, the centroidal Voronoi diagram meshing, and a hybrid of the two. We compare the performance of the three meshing methods in TriMe++, and show that the hybrid method retains the advantages of the other two. The software library achieves significant parallel speedup when generating large-scale meshes containing between $10^4$ to $10^7$ points. TriMe++ can handle complicated geometries and generates adaptive meshes of high quality.

cs.CG↗

arXiv2024

Characterizing structural features of two-dimensional particle systems through Voronoi topology

This paper introduces a new approach toward characterizing local structural features of two-dimensional particle systems. The approach can accurately identify and characterize defects in high-temperature crystals, distinguish a wide range of nominally disordered systems, and robustly describe complex structures such as grain boundaries. This paper also introduces two-dimensional functionality into the open-source software program VoroTop which automates this analysis. This software package is built on a recently-introduced multithreaded version of VORO++, enabling the analysis of systems with billions of particles on high-performance computer architectures.

cond-mat.mtrl-sci↗

arXiv2024

Stochastic evolution elasto-plastic modeling of a metallic glass

This paper develops a general data-driven approach to stochastic elastoplastic modelling that leverages atomistic simulation data directly rather than by fitting parameters. The approach is developed in the context of metallic glasses, which present inherent complexities due to their disordered structure. By harvesting statistics from simulated metallic glass shear response histories, the material state is mapped onto a two-dimensional state space consisting of the shear stress and the inelastic contribution to the potential energy. The resulting elastoplastic model is intrinsically stochastic and represented as a non-deterministic dynamical map. The state space statistics provide insights into the deformation physics of metallic glasses, revealing that two state variables are sufficient to describe the main features of the elastoplastic response. In this two-dimensional state space, the gradually quenched metallic glass rejuvenates during the initial quasi-elastic shearing, ultimately reaching a steady state that fluctuates about a fixed point in the state space as rejuvenation and aging balance.

cond-mat.stat-mech↗

arXiv2023

An extension to VORO++ for multithreaded computation of Voronoi cells

VORO++ is a software library written in C++ for computing the Voronoi tessellation, a technique in computational geometry that is widely used for analyzing systems of particles. VORO++ was released in 2009 and is based on computing the Voronoi cell for each particle individually. Here, we take advantage of modern computer hardware, and extend the original serial version to allow for multithreaded computation of Voronoi cells via the OpenMP application programming interface. We test the performance of the code, and demonstrate that we can achieve parallel efficiencies greater than 95% in many cases. The multithreaded extension follows standard OpenMP programming paradigms, allowing it to be incorporated into other programs. We provide an example of this using the VoroTop software library, performing a multithreaded Voronoi cell topology analysis of up to 102.4 million particles.

physics.comp-ph↗

arXiv2022

Voronoi cell analysis: The shapes of particle systems

Many physical systems can be studied as collections of particles embedded in space, evolving through deterministic evolution equations. Natural questions arise concerning how to characterize these arrangements - are they ordered or disordered? If they are ordered, how are they ordered and what kinds of defects do they possess? Originally introduced to study problems in pure mathematics, Voronoi tessellations have become a powerful and versatile tool for analyzing countless problems in pure and applied physics. In this paper we explain the basics of Voronoi tessellations and the shapes they produce, and describe how they can be used to study many physical systems.

physics.comp-ph↗