Searcharxiv⌕ Search

arXiv subjects

Zhao Dong

Publications and source records attributed to Zhao Dong.

At least 37 records · Page 2Linked to original sources
arXiv2024

TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion

We present TextureDreamer, a novel image-guided texture synthesis method to transfer relightable textures from a small number of input images (3 to 5) to target 3D shapes across arbitrary categories. Texture creation is a pivotal challenge in vision and graphics. Industrial companies hire experienced artists to manually craft textures for 3D assets. Classical methods require densely sampled views and accurately aligned geometry, while learning-based methods are confined to category-specific shapes within the dataset. In contrast, TextureDreamer can transfer highly detailed, intricate textures from real-world environments to arbitrary objects with only a few casually captured images, potentially significantly democratizing texture creation. Our core idea, personalized geometry-aware score distillation (PGSD), draws inspiration from recent advancements in diffuse models, including personalized modeling for texture information extraction, variational score distillation for detailed appearance synthesis, and explicit geometry guidance with ControlNet. Our integration and several essential modifications substantially improve the texture quality. Experiments on real images spanning different categories show that TextureDreamer can successfully transfer highly realistic, semantic meaningful texture to arbitrary objects, surpassing the visual quality of previous state-of-the-art.

cs.CV↗
arXiv2023

Efficient Graphics Representation with Differentiable Indirection

We introduce differentiable indirection -- a novel learned primitive that employs differentiable multi-scale lookup tables as an effective substitute for traditional compute and data operations across the graphics pipeline. We demonstrate its flexibility on a number of graphics tasks, i.e., geometric and image representation, texture mapping, shading, and radiance field representation. In all cases, differentiable indirection seamlessly integrates into existing architectures, trains rapidly, and yields both versatile and efficient results.

cs.GR↗
arXiv2023

Spatiotemporally Consistent HDR Indoor Lighting Estimation

We propose a physically-motivated deep learning framework to solve a general version of the challenging indoor lighting estimation problem. Given a single LDR image with a depth map, our method predicts spatially consistent lighting at any given image position. Particularly, when the input is an LDR video sequence, our framework not only progressively refines the lighting prediction as it sees more regions, but also preserves temporal consistency by keeping the refinement smooth. Our framework reconstructs a spherical Gaussian lighting volume (SGLV) through a tailored 3D encoder-decoder, which enables spatially consistent lighting prediction through volume ray tracing, a hybrid blending network for detailed environment maps, an in-network Monte-Carlo rendering layer to enhance photorealism for virtual object insertion, and recurrent neural networks (RNN) to achieve temporally consistent lighting prediction with a video sequence as the input. For training, we significantly enhance the OpenRooms public dataset of photorealistic synthetic indoor scenes with around 360K HDR environment maps of much higher resolution and 38K video sequences, rendered with GPU-based path tracing. Experiments show that our framework achieves lighting prediction with higher quality compared to state-of-the-art single-image or video-based methods, leading to photorealistic AR applications such as object insertion.

cs.CV↗
arXiv2023

Multimodal Brain Disease Classification with Functional Interaction Learning from Single fMRI Volume

In neuroimaging analysis, fMRI can well assess the function changes for brain diseases with no obvious structural lesions. To date, most deep-learning-based fMRI studies have employed functional connectivity (FC) as the basic feature for disease classification. However, FC is calculated on time series of predefined regions of interest and neglects detailed information contained in each voxel. Another drawback of using FC is the limited sample size for the training of deep models. The low representation ability of FC leads to poor performance in clinical practice, especially when dealing with multimodal medical data involving multiple types of visual signals and textual records for brain diseases. To overcome this bottleneck problem in the fMRI feature modality, we propose BrainFormer, an end-to-end functional interaction learning method for brain disease classification with single fMRI volume. Unlike traditional deep learning methods that construct convolution and transformers on FC, BrainFormer learns the functional interaction from fMRI signals, by modeling the local cues within each voxel with 3D convolutions and capturing the global correlations among distant regions with specially designed global attention mechanisms from shallow layers to deep layers. Meanwhile, BrainFormer can deal with multimodal medical data including fMRI volume, structural MRI, FC features and phenotypic data to achieve more comprehensive brain disease diagnosis. We evaluate BrainFormer on five independent multi-site datasets on autism, Alzheimer's disease, depression, attention deficit hyperactivity disorder and headache disorders. The results demonstrate its effectiveness and generalizability for multiple brain diseases diagnosis with multimodal features. BrainFormer may promote precision of neuroimaging-based diagnosis in clinical practice and motivate future studies on fMRI analysis.

eess.IV↗
arXiv2023

Global well-posedness and regularity of 3D Burgers equation with multiplicative noise

In this paper, we develop low regularity theory for 3D Burgers equation perturbed by a linear multiplicative stochastic force. This method is new and essentially different from the deterministic partial differential equations(PDEs). Our results and method can be widely applied to other stochastic hydrodynamic equations and the deterministic PDEs. As a further study, we establish a random version of maximum principle for random 3D Burgers equations, which will be an important tool for the study of 3D stochastic Burgers equations. As we know establishing moment estimates for highly nonlinear stochastic hydrodynamic equations is difficult. But moment estimates are very important for us to study the probabilistic properties and long-time behavior for the stochastic systems. Here, the random maximum principle helps us to achieve some important moment estimates for 3D stochastic Burgers equations and lays a solid foundation for the further study of 3D stochastic Burgers equations.

math.PR↗
arXiv2023

Neural Shadow Mapping

We present a neural extension of basic shadow mapping for fast, high quality hard and soft shadows. We compare favorably to fast pre-filtering shadow mapping, all while producing visual results on par with ray traced hard and soft shadows. We show that combining memory bandwidth-aware architecture specialization and careful temporal-window training leads to a fast, compact and easy-to-train neural shadowing method. Our technique is memory bandwidth conscious, eliminates the need for post-process temporal anti-aliasing or denoising, and supports scenes with dynamic view, emitters and geometry while remaining robust to unseen objects.

cs.GR↗
arXiv2022

Sharp non-uniqueness of solutions to stochastic Navier-Stokes equations

In this paper we establish a sharp non-uniqueness result for stochastic $d$-dimensional ($d\geq2$) incompressible Navier-Stokes equations. First, for every divergence free initial condition in $L^2$ we show existence of infinite many global in time probabilistically strong and analytically weak solutions in the class $L^α\big(Ω,L^p_tL^\infty\big)$ for any $1\leq p<2,α\geq1$. Second, we prove the above result is sharp in the sense that pathwise uniqueness holds in the class of $L^p_tL^q$ for some $p\in[2,\infty],q\in(2,\infty]$ such that $\frac2{p}+\frac{d}{q}\leq1$, which is a stochastic version of Ladyzhenskaya-Prodi-Serrin criteria. Moreover, for stochastic $d$-dimensional incompressible Euler equation, existence of infinitely many global in time probabilistically strong and analytically weak solutions is obtained. Compared to the stopping time argument used in \cite{HZZ19, HZZ21a}, we developed a new stochastic version of the convex integration. More precisely, we introduce expectation during convex integration scheme and construct directly solutions on the whole time interval $[0,\infty)$.

math.PR↗
arXiv2022

Physics-Based Inverse Rendering using Combined Implicit and Explicit Geometries

Mathematically representing the shape of an object is a key ingredient for solving inverse rendering problems. Explicit representations like meshes are efficient to render in a differentiable fashion but have difficulties handling topology changes. Implicit representations like signed-distance functions, on the other hand, offer better support of topology changes but are much more difficult to use for physics-based differentiable rendering. We introduce a new physics-based inverse rendering pipeline that uses both implicit and explicit representations. Our technique enjoys the benefit of both representations by supporting both topology changes and differentiable rendering of complex effects such as environmental illumination, soft shadows, and interreflection. We demonstrate the effectiveness of our technique using several synthetic and real examples.

cs.GR↗
arXiv2022

Random periodic solutions of nonautonomous stochastic feedback systems with multiplicative noise

We investigate the dynamical behavior of pull-back trajectories for nonautonomous stochastic feedback systems with multiplicative noise. We proved that there exists a random periodic solution of this system and all pull-back trajectories converge to this random periodic solution as time goes to infinitely almost surely. Our results can be applied to nonautonomous stochastic Goodwin negative feedback system, nonautonomous stochastic Othmer-Tyson positive feedback system and nonautonomous stochastic competitive systems etc.

math.DS↗
arXiv2022

The concentration of zero-noise limits of invariant measures for stochastic dynamical systems

In this paper, we study concentration phenomena of zero-noise limits of invariant measures for stochastic differential equations defined on $\mathbb{R}^d$ with locally Lipschitz continuous coefficients and more than one ergodic state. Under some dissipative conditions, by using Lyapunov-like functions and large deviations methods, we estimate the invariant measures in neighborhoods of stable sets, neighborhoods of unstable sets and their complement, respectively. Our result illustrates that invariant measures concentrate on the intersection of stable sets where a cost functional $W(K_i)$ is minimized and the Birkhoff center of the corresponding deterministic systems as noise tends down to zero. Furthermore, we prove the large deviations principle of invariant measures. At the end of this paper, we provide some explicit examples and their numerical simulations.

math.PR↗
arXiv2021

Large time behavior of strong solutions for stochastic Burgers equation

We consider the large time behavior of strong solutions to a kind of stochastic Burgers equation, where the position x is perturbed by a Brownian noise. It is well known that both the rarefaction wave and viscous shock wave are time-asymptotically stable for deterministic Burgers equation since the pioneer work of A. Ilin and O. Oleinik [21] in 1964. However, the stability of these wave patterns under stochastic perturbation is not known until now. In this paper, we give a definite answer to the stability problem of the rarefaction and viscous shock waves for the 1-d stochastic Burgers equation. That is, the rarefaction wave is still stable under white noise perturbation and the viscous shock is not stable yet. Moreover, a time-convergence rate toward the rarefaction wave is obtained. To get the desired decay rate, an important inequality (denoted by Area Inequality) is derived. This inequality plays essential role in the proof, and may have applications in the related problems for both the stochastic and deterministic PDEs.

math.PR↗
arXiv2021

Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Rendering

Reconstructing the shape and appearance of real-world objects using measured 2D images has been a long-standing problem in computer vision. In this paper, we introduce a new analysis-by-synthesis technique capable of producing high-quality reconstructions through robust coarse-to-fine optimization and physics-based differentiable rendering. Unlike most previous methods that handle geometry and reflectance largely separately, our method unifies the optimization of both by leveraging image gradients with respect to both object reflectance and geometry. To obtain physically accurate gradient estimates, we develop a new GPU-based Monte Carlo differentiable renderer leveraging recent advances in differentiable rendering theory to offer unbiased gradients while enjoying better performance than existing tools like PyTorch3D and redner. To further improve robustness, we utilize several shape and material priors as well as a coarse-to-fine optimization strategy to reconstruct geometry. We demonstrate that our technique can produce reconstructions with higher quality than previous methods such as COLMAP and Kinect Fusion.

cs.CV↗
arXiv2021

Random periodic solutions of non-autonomous stochastic differential equations

In this paper, we study the existence of random periodic solutions for nonlinear stochastic differential equations with additive white noise. We extend the input-to-state characteristic operator of the system to the non-autonomous stochastic differential equation via the pull-back of the discretised stochastic differential equation. We then use the completeness of the measurable function space which we construct skillfully and the Banach fixed point theorem to prove the existence of a fixed point of the gain operator. And we prove that the image for the input-to-state characteristic operator at this fixed point is a random periodic solution for the forward stochastic flow generated by the non-autonomous stochastic differential equation. Finally, we present some examples.

math.DS↗
arXiv2021

Deep Deterministic Policy Gradient for Relay Selection and Power Allocation in Cooperative Communication Network

Perfect channel state information (CSI) is usually required when considering relay selection and power allocation in cooperative communication. However, it is difficult to get an accurate CSI in practical situations. In this letter, we study the outage probability minimizing problem based on optimizing relay selection and transmission power. We propose a prioritized experience replay aided deep deterministic policy gradient learning framework, which can find an optimal solution by dealing with continuous action space, without any prior knowledge of CSI. Simulation results reveal that our approach outperforms reinforcement learning based methods in existing literatures, and improves the communication success rate by about 4%.

cs.IT↗
arXiv2020

Ergodicity for stochastic conservation laws with multiplicative noise

We proved that there exists a unique invariant measure for solutions of stochastic conservation laws with Dirichlet boundary condition driven by multiplicative noise. Moreover, a polynomial mixing property is established. This is done in the setting of kinetic solutions taking values in an L^1-weighted space.

math.PR↗