arXiv subjects

Yuhui Yin

Publications and source records attributed to Yuhui Yin.

28 records · Page 2Linked to original sources

arXiv2025

WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generation

Recent rapid advancements in text-to-video (T2V) generation, such as SoRA and Kling, have shown great potential for building world simulators. However, current T2V models struggle to grasp abstract physical principles and generate videos that adhere to physical laws. This challenge arises primarily from a lack of clear guidance on physical information due to a significant gap between abstract physical principles and generation models. To this end, we introduce the World Simulator Assistant (WISA), an effective framework for decomposing and incorporating physical principles into T2V models. Specifically, WISA decomposes physical principles into textual physical descriptions, qualitative physical categories, and quantitative physical properties. To effectively embed these physical attributes into the generation process, WISA incorporates several key designs, including Mixture-of-Physical-Experts Attention (MoPA) and a Physical Classifier, enhancing the model's physics awareness. Furthermore, most existing datasets feature videos where physical phenomena are either weakly represented or entangled with multiple co-occurring processes, limiting their suitability as dedicated resources for learning explicit physical principles. We propose a novel video dataset, WISA-32K, collected based on qualitative physical categories. It consists of 32,000 videos, representing 17 physical laws across three domains of physics: dynamics, thermodynamics, and optics. Experimental results demonstrate that WISA can effectively enhance the compatibility of T2V models with real-world physical laws, achieving a considerable improvement on the VideoPhy benchmark. The visual exhibitions of WISA and WISA-32K are available in the https://360cvgroup.github.io/WISA/.

cs.CV↗

arXiv2025

U-StyDiT: Ultra-high Quality Artistic Style Transfer Using Diffusion Transformers

Ultra-high quality artistic style transfer refers to repainting an ultra-high quality content image using the style information learned from the style image. Existing artistic style transfer methods can be categorized into style reconstruction-based and content-style disentanglement-based style transfer approaches. Although these methods can generate some artistic stylized images, they still exhibit obvious artifacts and disharmonious patterns, which hinder their ability to produce ultra-high quality artistic stylized images. To address these issues, we propose a novel artistic image style transfer method, U-StyDiT, which is built on transformer-based diffusion (DiT) and learns content-style disentanglement, generating ultra-high quality artistic stylized images. Specifically, we first design a Multi-view Style Modulator (MSM) to learn style information from a style image from local and global perspectives, conditioning U-StyDiT to generate stylized images with the learned style information. Then, we introduce a StyDiT Block to learn content and style conditions simultaneously from a style image. Additionally, we propose an ultra-high quality artistic image dataset, Aes4M, comprising 10 categories, each containing 400,000 style images. This dataset effectively solves the problem that the existing style transfer methods cannot produce high-quality artistic stylized images due to the size of the dataset and the quality of the images in the dataset. Finally, the extensive qualitative and quantitative experiments validate that our U-StyDiT can create higher quality stylized images compared to state-of-the-art artistic style transfer methods. To our knowledge, our proposed method is the first to address the generation of ultra-high quality stylized images using transformer-based diffusion.

cs.CV↗

arXiv2025

Efficient neural topology optimization via active learning for enhancing turbulent mass transfer in fluid channels

The design of fluid channel structures of reactors or separators of chemical processes is key to enhancing the mass transfer processes inside the devices. However, the systematic design of channel topological structures is difficult for complex turbulent flows. Here, we address this challenge by developing a machine learning framework to efficiently perform topology optimization of channel structures for turbulent mass transfer. We represent a topological structure using a neural network (referred to as `neural topology', which is optimized by employing pre-trained neural operators combined with a fine-tuning strategy with active data augmentation. The optimization is performed with two objectives: maximization of mass transfer efficiency and minimization of energy consumption, for the possible considerations of compromise between the two in real-world designs. The developed neural operator with active learning is data efficient in network training and demonstrates superior computational efficiency compared with traditional methods in obtaining optimal structures across a large design space. The optimization results are validated through experiments, proving that the optimized channel improves concentration uniformity by 37% compared with the original channel. We also demonstrate the variation of the optimal structures with changes in inlet velocity conditions, providing a reference for designing turbulent mass-transfer devices under different operating conditions.

physics.flu-dyn↗

arXiv2024

HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generation

The task of layout-to-image generation involves synthesizing images based on the captions of objects and their spatial positions. Existing methods still struggle in complex layout generation, where common bad cases include object missing, inconsistent lighting, conflicting view angles, etc. To effectively address these issues, we propose a \textbf{Hi}erarchical \textbf{Co}ntrollable (HiCo) diffusion model for layout-to-image generation, featuring object seperable conditioning branch structure. Our key insight is to achieve spatial disentanglement through hierarchical modeling of layouts. We use a multi branch structure to represent hierarchy and aggregate them in fusion module. To evaluate the performance of multi-objective controllable layout generation in natural scenes, we introduce the HiCo-7K benchmark, derived from the GRIT-20M dataset and manually cleaned. https://github.com/360CVGroup/HiCo_T2I.

cs.CV↗

arXiv2024

Qihoo-T2X: An Efficient Proxy-Tokenized Diffusion Transformer for Text-to-Any-Task

The global self-attention mechanism in diffusion transformers involves redundant computation due to the sparse and redundant nature of visual information, and the attention map of tokens within a spatial window shows significant similarity. To address this redundancy, we propose the Proxy-Tokenized Diffusion Transformer (PT-DiT), which employs sparse representative token attention (where the number of representative tokens is much smaller than the total number of tokens) to model global visual information efficiently. Specifically, within each transformer block, we compute an averaging token from each spatial-temporal window to serve as a proxy token for that region. The global semantics are captured through the self-attention of these proxy tokens and then injected into all latent tokens via cross-attention. Simultaneously, we introduce window and shift window attention to address the limitations in detail modeling caused by the sparse attention mechanism. Building on the well-designed PT-DiT, we further develop the Qihoo-T2X family, which includes a variety of models for T2I, T2V, and T2MV tasks. Experimental results show that PT-DiT achieves competitive performance while reducing the computational complexity in both image and video generation tasks (e.g., a 49% reduction compared to DiT and a 34% reduction compared to PixArt-$α$). The visual exhibition and source code of Qihoo-T2X is available at https://360cvgroup.github.io/Qihoo-T2X/.

cs.CV↗

arXiv2024

Periodic domain inversion in single crystal barium titanate-on-insulator thin film

We report experimentally achieving first-ever electric field periodic poling of single crystal barium titanate (BTO, or BaTiO3) thin film on insulator. Owing to the outstanding optical nonlinearities of BTO, this result is a key step towards achieving quasi-phase-matching in BTO. We first grow the BTO thin film on a dysprosium scandate substrate using pulsed laser deposition with a thin layer of strontium ruthenate later serving as the bottom electrode for poling. We present characterization of the BTO thin film using x-ray diffraction and piezo-response force microscopy to clearly demonstrate single crystal, single domain growth of the film which enables the desired periodic poling. To investigate the poling quality, we apply both non-destructive piezo force response microscopy and destructive etching-assisted scanning electron microscopy and we show that high quality, uniform and intransient poling with 50 % duty cycle and periods ranging from 2 μm to 10 μm is achieved. The successful realization of periodic poling in BTO thin film unlocks the potential for highly efficient nonlinear processes under quasi-phase-matching that seemed far-fetched with prior polycrystalline BTO thin films which predominantly relied on efficiency-limited random or non-phase matching conditions and is a key step towards integration of BTO photonic devices.

physics.optics↗

arXiv2023

PF-DMD: Physics-fusion dynamic mode decomposition for accurate and robust forecasting of dynamical systems with imperfect data and physics

The DMD (Dynamic Mode Decomposition) method has attracted widespread attention as a representative modal-decomposition method and can build a predictive model. However, the DMD may give predicted results that deviate from physical reality in some scenarios, such as dealing with translation problems or noisy data. Therefore, this paper proposes a physics-fusion dynamic mode decomposition (PFDMD) method to address this issue. The proposed PFDMD method first obtains a data-driven model using DMD, then calculates the residual of the physical equations, and finally corrects the predicted results using Kalman filtering and gain coefficients. In this way, the PFDMD method can integrate the physics-informed equations with the data-driven model generated by DMD. Numerical experiments are conducted using the PFDMD, including the Allen-Cahn, advection-diffusion, and Burgers' equations. The results demonstrate that the proposed PFDMD method can significantly reduce the reconstruction and prediction errors by incorporating physics-informed equations, making it usable for translation and shock problems where the standard DMD method has failed.

physics.comp-ph↗

arXiv2023

StarGraph: Knowledge Representation Learning based on Incomplete Two-hop Subgraph

Conventional representation learning algorithms for knowledge graphs (KG) map each entity to a unique embedding vector, ignoring the rich information contained in the neighborhood. We propose a method named StarGraph, which gives a novel way to utilize the neighborhood information for large-scale knowledge graphs to obtain entity representations. An incomplete two-hop neighborhood subgraph for each target node is at first generated, then processed by a modified self-attention network to obtain the entity representation, which is used to replace the entity embedding in conventional methods. We achieved SOTA performance on ogbl-wikikg2 and got competitive results on fb15k-237. The experimental results proves that StarGraph is efficient in parameters, and the improvement made on ogbl-wikikg2 demonstrates its great effectiveness of representation learning on large-scale knowledge graphs. The code is now available at \url{https://github.com/hzli-ucas/StarGraph}.

cs.CL↗

arXiv2022

An iterative data-driven turbulence modeling framework based on Reynolds stress representation

Data-driven turbulence modeling studies have reached such a stage that the fundamental framework is basically settled, but several essential issues remain that strongly affect the performance, including accuracy, smoothness, and generalization capacity. Two problems are studied in the current research: (1) the processing of the Reynolds stress tensor and (2) the coupling method between the machine learning turbulence model and flow solver. The first determines the form of predicting targets and the resulting physical completeness and interpretability. The second determines the training process and intrinsic relevance between the mean flow features and Reynolds stress. For the Reynolds stress processing issue, we perform the theoretical derivation to extend the relevant tensor arguments of Reynolds stress in addition to the strain rate and rotation rate. Then, the tensor representation theorem is employed to give the complete irreducible invariants and integrity basis. In addition, an adaptive regularization term is employed to enhance the representation performance. For the coupling issue, an iterative coupling data-driven turbulence modeling framework with consistent convergence is proposed. The training data preparation, predicting target selection, and computation platform are illustrated. The framework is then applied to a canonical separated flow for verification. The mean flow results obtained by coupling computation of the trained machine learning model and flow solver have high consistency with the direct numerical simulation true values, which proves the validity of the current approach.

physics.flu-dyn↗

arXiv2020

Feature selection and processing of turbulence modeling based on an artificial neural network

Data-driven turbulence modeling has been considered an effective method for improving the prediction accuracy of Reynolds-averaged Navier-Stokes equations. Related studies aimed to solve the discrepancy of traditional turbulence modeling by acquiring specific patterns from high-fidelity data through machine learning methods, such as artificial neural networks. The present study focuses on the unsmoothness and prediction error problems from the aspect of feature selection and processing. The selection criteria for the input features are summarized, and an effective input set is constructed. The effect of the computation grid on the smoothness is studied. A modified feature decomposition method for the spatial orientation feature of the Reynolds stress is proposed. The improved machine learning framework is then applied to the periodic hill database with notably varying geometries. The results of the modified method show significant enhancement in the prediction accuracy and smoothness, including the shape and size of separation areas and the friction and pressure distributions on the wall, which confirms the validity of the approach.

physics.flu-dyn↗