arXiv subjects

Jiale Shi

Publications and source records attributed to Jiale Shi.

8 recordsLinked to original sources

arXiv2026

Motion Reinforces Appearance: RGB-Skeleton Gated Residual Fusion for Micro-Gesture Online Recognition

Micro-gesture analysis attracts increasing attention for inferring spontaneous emotion from subtle body movements. Micro-gesture online recognition, which localizes and classifies each gesture instance in untrimmed videos, is a core task in the 4th EI-MiGA-IJCAI Challenge. Compared with typical temporal action detection, MGR emphasizes the localization and classification of actions, requiring the model to output the start time, end time, and category of each micro-gesture. Moreover, since micro-gestures are highly spontaneous, relying solely on a single modality makes it difficult to capture the complete and accurate multi-modal cues. In this work, we propose DyFADet+, which extends DyFADet into a dual-stream RGB-skeleton framework. In our model, both modalities are projected into shared multi-scale temporal embeddings and fused through a gated residual module, which adaptively injects skeleton motion into the RGB representation rather than using naive concatenation. Finally, these fused features are decoded by a Dynamic TAD head for online classification and boundary regression. On the SMG dataset, our method achieves an F1 score of 40.88, ranking 2nd in the Micro-gesture Online Recognition track.

cs.CV↗

arXiv2026

GaussianZoom: Progressive Zoom-in Generative 3D Gaussian Splatting with Geometric and Semantic Guidance

We introduce GaussianZoom, a generative zoom-in 3D reconstruction system with an iterative progressive framework that combines geometry-consistent scene modeling and multi-scale semantic reasoning to enable high-fidelity extreme zoom-in rendering from low-resolution inputs. To achieve this, we develop a novel multi-view consistent super-resolution module with depth-based feature warping and VLM-driven detail synthesis, ensuring accurate multi-view correspondence while enriching fine-scale appearance beyond the observed resolution. To support zooming across large magnification ranges, we further introduce a new expandable continuous Level-of-Detail hierarchy that dynamically modulates Gaussian visibility for smooth, alias-free cross-scale rendering. Experiments on Mip-NeRF360 and Tanks\&Temples demonstrate that GaussianZoom achieves superior perceptual quality, multi-view consistency, and robustness under extreme magnification, establishing a strong baseline for generative zoom-in 3D scene reconstruction.

cs.CV↗

arXiv2025

Free360: Layered Gaussian Splatting for Unbounded 360-Degree View Synthesis from Extremely Sparse and Unposed Views

Neural rendering has demonstrated remarkable success in high-quality 3D neural reconstruction and novel view synthesis with dense input views and accurate poses. However, applying it to extremely sparse, unposed views in unbounded 360{\deg} scenes remains a challenging problem. In this paper, we propose a novel neural rendering framework to accomplish the unposed and extremely sparse-view 3D reconstruction in unbounded 360{\deg} scenes. To resolve the spatial ambiguity inherent in unbounded scenes with sparse input views, we propose a layered Gaussian-based representation to effectively model the scene with distinct spatial layers. By employing a dense stereo reconstruction model to recover coarse geometry, we introduce a layer-specific bootstrap optimization to refine the noise and fill occluded regions in the reconstruction. Furthermore, we propose an iterative fusion of reconstruction and generation alongside an uncertainty-aware training approach to facilitate mutual conditioning and enhancement between these two processes. Comprehensive experiments show that our approach outperforms existing state-of-the-art methods in terms of rendering quality and surface reconstruction accuracy. Project page: https://zju3dv.github.io/free360/

cs.CV↗

arXiv2024

GURecon: Learning Detailed 3D Geometric Uncertainties for Neural Surface Reconstruction

Neural surface representation has demonstrated remarkable success in the areas of novel view synthesis and 3D reconstruction. However, assessing the geometric quality of 3D reconstructions in the absence of ground truth mesh remains a significant challenge, due to its rendering-based optimization process and entangled learning of appearance and geometry with photometric losses. In this paper, we present a novel framework, i.e, GURecon, which establishes a geometric uncertainty field for the neural surface based on geometric consistency. Different from existing methods that rely on rendering-based measurement, GURecon models a continuous 3D uncertainty field for the reconstructed surface, and is learned by an online distillation approach without introducing real geometric information for supervision. Moreover, in order to mitigate the interference of illumination on geometric consistency, a decoupled field is learned and exploited to finetune the uncertainty field. Experiments on various datasets demonstrate the superiority of GURecon in modeling 3D geometric uncertainty, as well as its plug-and-play extension to various neural surface representations and improvement on downstream tasks such as incremental reconstruction. The code and supplementary material are available on the project website: https://zju3dv.github.io/GURecon/.

cs.CV↗

arXiv2023

14 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathon

Large-language models (LLMs) such as GPT-4 caught the interest of many scientists. Recent studies suggested that these models could be useful in chemistry and materials science. To explore these possibilities, we organized a hackathon. This article chronicles the projects built as part of this hackathon. Participants employed LLMs for various applications, including predicting properties of molecules and materials, designing novel interfaces for tools, extracting knowledge from unstructured data, and developing new educational applications. The diverse topics and the fact that working prototypes could be generated in less than two days highlight that LLMs will profoundly impact the future of our fields. The rich collection of ideas and projects also indicates that the applications of LLMs are not limited to materials science and chemistry but offer potential benefits to a wide range of scientific disciplines.

cond-mat.mtrl-sci↗

arXiv2023

Transfer Learning Facilitates the Prediction of Polymer-Surface Adhesion Strength

Machine learning (ML) accelerates the exploration of material properties and their links to the structure of the underlying molecules. In previous work [J. Shi, M. J. Quevillon, P. H. A. Valença, and J. K. Whitmer, \textit{ACS Appl. Mater. Interfaces.}, 2022, 14, 32, 37161--37169], ML models were applied to predict the adhesive free energy of polymer--surface interactions with high accuracy from the knowledge of the sequence data, demonstrating successes in inverse-design of polymer sequence for known surface compositions. While the method was shown to be successful in designing polymers for a known surface, extensive datasets were needed for each specific surface in order to train the surrogate models. Ideally, one should be able to infer information about similar surfaces without having to regenerate a full complement of adhesion data for each new case. In the current work, we demonstrate a transfer learning (TL) technique using a deep neural network to improve the accuracy of ML models trained on small datasets by pre-training on a larger database from a related system and fine-tuning the weights of all layers with a small amount of additional data. The shared knowledge from the pre-trained model facilitates the prediction accuracy significantly on small datasets. We also explore the limits of database size on accuracy and the optimal tuning of network architecture and parameters for our learning tasks. While applied to a relatively simple coarse-grained (CG) polymer model, the general lessons of this study apply to detailed modeling studies and the broader problems of inverse materials design.

cond-mat.soft↗

arXiv2021

Free Energy Landscape and Isomerization Rates of Au$_4$ Clusters at Finite Temperature

In metallic nanoparticles, the cluster geometric structures control the particle's electronic band structure, polarizability, and catalytic properties. Analyzing the structural properties is a complex problem; the structure of an assembled cluster changes from moment to moment due to thermal fluctuations. Conventional structural analyses based on spectroscopy or diffraction cannot determine the instantaneous structure exactly and can merely provide an averaged structure. Molecular simulations offer an opportunity to examine the assembly and evolution of metallic clusters, as the preferred assemblies and conformations can easily be visualized and explored. Here, we utilize the adaptive biasing force algorithm applied to first principles molecular dynamics to demonstrate exploration of a relatively simple system which permits comprehensive study of the small metal cluster $\ce{Au4}$ in both neutral and charged configurations. Our simulation work offers a quantitative understanding of these clusters' dynamic structure, which is significant for single-site catalytic reactions on metal clusters and provides a starting point for a detailed quantitative understanding of more complex pure metal and alloy clusters' dynamic properties.

cond-mat.mtrl-sci↗

arXiv2021

Predicting Adhesive Free Energies of Polymer--Surface Interactions with Machine Learning

Polymer-surface interactions are crucial to many biological processes and industrial applications. Here we propose a machine-learning method to connect a model polymer's sequence with its adhesion to decorated surfaces. We simulate the adhesive free energies of $20,000$ unique coarse-grained 1D sequential polymers interacting with functionalized surfaces and build support vector regression (SVR) models that demonstrate inexpensive and reliable prediction of the adhesive free energy as a function of the sequence. Our work highlights the promising integration of coarse-grained simulation with data-driven machine learning methods for the design of new functional polymers and represents an important step toward linking polymer compositions with polymer-surface interactions.

cond-mat.soft↗