arXiv subjects

Min Zhao

Publications and source records attributed to Min Zhao.

At least 37 records · Page 2Linked to original sources

arXiv2025

FlexWorld: Progressively Expanding 3D Scenes for Flexiable-View Synthesis

Generating flexible-view 3D scenes, including 360° rotation and zooming, from single images is challenging due to a lack of 3D data. To this end, we introduce FlexWorld, a novel framework consisting of two key components: (1) a strong video-to-video (V2V) diffusion model to generate high-quality novel view images from incomplete input rendered from a coarse scene, and (2) a progressive expansion process to construct a complete 3D scene. In particular, leveraging an advanced pre-trained video model and accurate depth-estimated training pairs, our V2V model can generate novel views under large camera pose variations. Building upon it, FlexWorld progressively generates new 3D content and integrates it into the global scene through geometry-aware scene fusion. Extensive experiments demonstrate the effectiveness of FlexWorld in generating high-quality novel view videos and flexible-view 3D scenes from single images, achieving superior visual quality under multiple popular metrics and datasets compared to existing state-of-the-art methods. Qualitatively, we highlight that FlexWorld can generate high-fidelity scenes with flexible views like 360° rotations and zooming. Project page: https://ml-gsai.github.io/FlexWorld.

cs.CV↗

arXiv2025

FedDA-TSformer: Federated Domain Adaptation with Vision TimeSformer for Left Ventricle Segmentation on Gated Myocardial Perfusion SPECT Image

Background and Purpose: Functional assessment of the left ventricle using gated myocardial perfusion (MPS) single-photon emission computed tomography relies on the precise extraction of the left ventricular contours while simultaneously ensuring the security of patient data. Methods: In this paper, we introduce the integration of Federated Domain Adaptation with TimeSformer, named 'FedDA-TSformer' for left ventricle segmentation using MPS. FedDA-TSformer captures spatial and temporal features in gated MPS images, leveraging spatial attention, temporal attention, and federated learning for improved domain adaptation while ensuring patient data security. In detail, we employed Divide-Space-Time-Attention mechanism to extract spatio-temporal correlations from the multi-centered MPS datasets, ensuring that predictions are spatio-temporally consistent. To achieve domain adaptation, we align the model output on MPS from three different centers using local maximum mean discrepancy (LMMD) loss. This approach effectively addresses the dual requirements of federated learning and domain adaptation, enhancing the model's performance during training with multi-site datasets while ensuring the protection of data from different hospitals. Results: Our FedDA-TSformer was trained and evaluated using MPS datasets collected from three hospitals, comprising a total of 150 subjects. Each subject's cardiac cycle was divided into eight gates. The model achieved Dice Similarity Coefficients (DSC) of 0.842 and 0.907 for left ventricular (LV) endocardium and epicardium segmentation, respectively. Conclusion: Our proposed FedDA-TSformer model addresses the challenge of multi-center generalization, ensures patient data privacy protection, and demonstrates effectiveness in left ventricular (LV) segmentation.

eess.IV↗

arXiv2024

Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Model

Diffusion models have obtained substantial progress in image-to-video generation. However, in this paper, we find that these models tend to generate videos with less motion than expected. We attribute this to the issue called conditional image leakage, where the image-to-video diffusion models (I2V-DMs) tend to over-rely on the conditional image at large time steps. We further address this challenge from both inference and training aspects. First, we propose to start the generation process from an earlier time step to avoid the unreliable large-time steps of I2V-DMs, as well as an initial noise distribution with optimal analytic expressions (Analytic-Init) by minimizing the KL divergence between it and the actual marginal distribution to bridge the training-inference gap. Second, we design a time-dependent noise distribution (TimeNoise) for the conditional image during training, applying higher noise levels at larger time steps to disrupt it and reduce the model's dependency on it. We validate these general strategies on various I2V-DMs on our collected open-domain image benchmark and the UCF101 dataset. Extensive results show that our methods outperform baselines by producing higher motion scores with lower errors while maintaining image alignment and temporal consistency, thereby yielding superior overall performance and enabling more accurate motion control. The project page: \url{https://cond-image-leak.github.io/}.

cs.CV↗

arXiv2024

Unrolling Plug-and-Play Network for Hyperspectral Unmixing

Deep learning based unmixing methods have received great attention in recent years and achieve remarkable performance. These methods employ a data-driven approach to extract structure features from hyperspectral image, however, they tend to be less physical interpretable. Conventional unmixing methods are with much more interpretability, whereas they require manually designing regularization and choosing penalty parameters. To overcome these limitations, we propose a novel unmixing method by unrolling the plug-and-play unmixing algorithm to conduct the deep architecture. Our method integrates both inner and outer priors. The carefully designed unfolding deep architecture is used to learn the spectral and spatial information from the hyperspectral image, which we refer to as inner priors. Additionally, our approach incorporates deep denoisers that have been pretrained on a large volume of image data to leverage the outer priors. Secondly, we design a dynamic convolution to model the multiscale information. Different scales are fused using an attention module. Experimental results of both synthetic and real datasets demonstrate that our method outperforms compared methods.

eess.IV↗

arXiv2024

A new approach to inverse Sturm-Liouville problems based on point interaction

In the present paper, motivated by point interaction, we propose a new and explicit approach to inverse Sturm-Liouville eigenvalue problems under Dirichlet boundary. More precisely, when a given Sturm-Liouville eigenvalue problem with the unknown integrable potential interacts with $δ$-function potentials, we obtain a family of perturbation problems, called point interaction models in quantum mechanics. Then, only depending on the first eigenvalues of these perturbed problems, we define and study the first eigenvalue function, by which the desired potential can be expressed explicitly and uniquely. As by-products, using the analytic function theoretic tools, we also generalize several fundamental theorems of classical Sturm-Liouville problems to measure differential equations.

math.SP↗

arXiv2024

PoseCrafter: One-Shot Personalized Video Synthesis Following Flexible Pose Control

In this paper, we introduce PoseCrafter, a one-shot method for personalized video generation following the control of flexible poses. Built upon Stable Diffusion and ControlNet, we carefully design an inference process to produce high-quality videos without the corresponding ground-truth frames. First, we select an appropriate reference frame from the training video and invert it to initialize all latent variables for generation. Then, we insert the corresponding training pose into the target pose sequences to enhance faithfulness through a trained temporal attention module. Furthermore, to alleviate the face and hand degradation resulting from discrepancies between poses of training videos and inference poses, we implement simple latent editing through an affine transformation matrix involving facial and hand landmarks. Extensive experiments on several datasets demonstrate that PoseCrafter achieves superior results to baselines pre-trained on a vast collection of videos under 8 commonly used metrics. Besides, PoseCrafter can follow poses from different individuals or artificial edits and simultaneously retain the human identity in an open-domain training video. Our project page is available at https://ml-gsai.github.io/PoseCrafter-demo/.

cs.CV↗

arXiv2024

Traveling waves with continuous profile for hyperbolic Keller-Segel equation

This work describes a hyperbolic model for cell-cell repulsion with population dynamics. We consider the pressure produced by a population of cells to describe their motion. We assume that cells try to avoid crowded areas and prefer locally empty spaces far away from the carrying capacity. Here, our main goal is to prove the existence of traveling waves with continuous profiles. This article complements our previous results about sharp traveling waves. We conclude the paper with numerical simulations of the PDE problem, illustrating such a result.

math.AP↗

arXiv2024

Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models

We introduce Vidu, a high-performance text-to-video generator that is capable of producing 1080p videos up to 16 seconds in a single generation. Vidu is a diffusion model with U-ViT as its backbone, which unlocks the scalability and the capability for handling long videos. Vidu exhibits strong coherence and dynamism, and is capable of generating both realistic and imaginative videos, as well as understanding some professional photography techniques, on par with Sora -- the most powerful reported text-to-video generator. Finally, we perform initial experiments on other controllable video generation, including canny-to-video generation, video prediction and subject-driven generation, which demonstrate promising results.

cs.CV↗

arXiv2024

Testing The Weak Cosmic Censorship Conjecture in Short Haired Black Holes

The Weak Cosmic Censorship Conjecture is a hypothesis regarding the properties of event horizons and singularities during the formation of black holes, stating that singularities are always encompassed by the event horizons(TEH) of black holes, thus preventing naked singularities from affecting the causal structure of spacetime. In this paper, we explore the Weak Cosmic Censorship Conjecture in the context of rotating hairy black holes, aiming to understand the impact of hairiness on the conjecture for Kerr black holes. We investigate whether TEH of rotating hairy black holes can be disrupted by incoming test particles and scalar fields. When test particles and scalar fields incident on the rotating hairy black hole are found. In extreme cases, when considering a second order approximation, if the parameter $κ$ (The parameter here is a function related to hair strength $κ(Q_m^{2k})$.)falls within the range of $0<κ<\sqrt{1/3}$, TEH of a hairy black hole can be disrupted. Conversely, in the range of $\sqrt{1/3}<κ<1$, TEH of a hairy black hole cannot be disrupted. When considering the second order approximation in near extreme cases, the parameter $κ$, within the range of $0<κ<1$, can lead to the disruption of TEH in this spacetime. When an incident scalar field is present, in near extreme conditions, TEH of a rotating short hair black hole cannot be disrupted. Therefore, the value of the parameter $κ$ reveals the connection between rotating short hair black holes and the weak cosmic censorship conjecture, indicating that the presence of short hair significantly affects TEH of black holes. This will aid in further understanding the nature of rotating short hair black holes.

gr-qc↗

arXiv2024

Quasinormal modes of dark matter core-black hole spacetime

In the galactic core, when the scale of dark matter is small, the distribution of dark matter is that of a constant density dark matter core.Considering the case of a supermassive black hole coupled to a constant density dark matter core, we study the quasinormal modes of the black hole in the constant density dark matter core black hole system and calculate the quasinormal modes frequency of the black hole using the third order WKB approximation and the prony method.In addition, we study the effect of the constant density dark matter core parameter $r_0$ on the quasinormal modes of black holes in the vicinity of black holes.As the angular quantum number increases, the ringdown process becomes closer and closer to the case of the ringdown process of a schwarzschild black hole.The presence of a constant density dark matter core affects the quasinormal modes of the black hole, with relative deviations on the order of $10^{-15}-10^{-13}$ with respect to the detector.These features suggest that with future improvements in detector accuracy, we can use them for the detection of gravitational waves in the spacetime of constant density dark matter core-black hole systems, which in turn opens up the possibility of understanding the behavior of dark matter in the vicinity of black holes.

gr-qc↗

arXiv2024

On the $p$-fractional Schrödinger-Kirchhoff equations with electromagnetic fields and the Hardy-Littlewood-Sobolev nonlinearity

In this article, we deal with the following $p$-fractional Schrödinger-Kirchhoff equations with electromagnetic fields and the Hardy-Littlewood-Sobolev nonlinearity: $$ M\left([u]_{s,A}^{p}\right)(-Δ)_{p, A}^{s} u+V(x)|u|^{p-2} u=λ\left(\int_{\mathbb{R}^{N}} \frac{|u|^{p_{μ, s}^{*}}}{|x-y|^μ} \mathrm{d}y\right)|u|^{p_{μ, s}^{*}-2} u+k|u|^{q-2}u,\ x \in \mathbb{R}^{N},$$ where $0<s<1<p$, $ps < N$, $p<q<2p^{*}_{s,μ}$, $0<μ<N$, $λ$ and $k$ are some positive parameters, $p^{*}_{s,μ}=\frac{pN-p\fracμ{2}}{N-ps}$ is the critical exponent with respect to the Hardy-Littlewood-Sobolev inequality, and functions $V$, $M$ satisfy the suitable conditions. By proving the compactness results with the help of the fractional version of concentration compactness principle, we establish the existence of nontrivial solutions to this problem.

math.AP↗

arXiv2023

ControlVideo: Conditional Control for One-shot Text-driven Video Editing and Beyond

This paper presents \emph{ControlVideo} for text-driven video editing -- generating a video that aligns with a given text while preserving the structure of the source video. Building on a pre-trained text-to-image diffusion model, ControlVideo enhances the fidelity and temporal consistency by incorporating additional conditions (such as edge maps), and fine-tuning the key-frame and temporal attention on the source video-text pair via an in-depth exploration of the design space. Extensive experimental results demonstrate that ControlVideo outperforms various competitive baselines by delivering videos that exhibit high fidelity w.r.t. the source content, and temporal consistency, all while aligning with the text. By incorporating Low-rank adaptation layers into the model before training, ControlVideo is further empowered to generate videos that align seamlessly with reference images. More importantly, ControlVideo can be readily extended to the more challenging task of long video editing (e.g., with hundreds of frames), where maintaining long-range temporal consistency is crucial. To achieve this, we propose to construct a fused ControlVideo by applying basic ControlVideo to overlapping short video segments and key frame videos and then merging them by pre-defined weight functions. Empirical results validate its capability to create videos across 140 frames, which is approximately 5.83 to 17.5 times more than what previous works achieved. The code is available at \href{https://github.com/thu-ml/controlvideo}{https://github.com/thu-ml/controlvideo} and the visualization results are available at \href{https://drive.google.com/file/d/1wEgc2io3UwmoC5vTPbkccFvTkwVqsZlK/view?usp=drive_link}{HERE}.

cs.CV↗

arXiv2023

AE-RED: A Hyperspectral Unmixing Framework Powered by Deep Autoencoder and Regularization by Denoising

Spectral unmixing has been extensively studied with a variety of methods and used in many applications. Recently, data-driven techniques with deep learning methods have obtained great attention to spectral unmixing for its superior learning ability to automatically learn the structure information. In particular, autoencoder based architectures are elaborately designed to solve blind unmixing and model complex nonlinear mixtures. Nevertheless, these methods perform unmixing task as blackboxes and lack of interpretability. On the other hand, conventional unmixing methods carefully design the regularizer to add explicit information, in which algorithms such as plug-and-play (PnP) strategies utilize off-the-shelf denoisers to plug powerful priors. In this paper, we propose a generic unmixing framework to integrate the autoencoder network with regularization by denoising (RED), named AE-RED. More specially, we decompose the unmixing optimized problem into two subproblems. The first one is solved using deep autoencoders to implicitly regularize the estimates and model the mixture mechanism. The second one leverages the denoiser to bring in the explicit information. In this way, both the characteristics of the deep autoencoder based unmixing methods and priors provided by denoisers are merged into our well-designed framework to enhance the unmixing performance. Experiment results on both synthetic and real data sets show the superiority of our proposed framework compared with state-of-the-art unmixing approaches.

cs.CV↗

arXiv2023

Guided Deep Generative Model-based Spatial Regularization for Multiband Imaging Inverse Problems

When adopting a model-based formulation, solving inverse problems encountered in multiband imaging requires to define spatial and spectral regularizations. In most of the works of the literature, spectral information is extracted from the observations directly to derive data-driven spectral priors. Conversely, the choice of the spatial regularization often boils down to the use of conventional penalizations (e.g., total variation) promoting expected features of the reconstructed image (e.g., piecewise constant). In this work, we propose a generic framework able to capitalize on an auxiliary acquisition of high spatial resolution to derive tailored data-driven spatial regularizations. This approach leverages on the ability of deep learning to extract high level features. More precisely, the regularization is conceived as a deep generative network able to encode spatial semantic features contained in this auxiliary image of high spatial resolution. To illustrate the versatility of this approach, it is instantiated to conduct two particular tasks, namely multiband image fusion and multiband image inpainting. Experimental results obtained on these two tasks demonstrate the benefit of this class of informed regularizations when compared to more conventional ones.

eess.IV↗

arXiv2023

Persistence of solutions in a nonlocal predator-prey system with a shifting habitat

In this paper, we mainly study the propagation properties of a nonlocal dispersal predator-prey system in a shifting environment. It is known that Choi et al. [J. Differ. Equ. 302 (2021), pp. 807-853] studied the persistence or extinction of the prey and the predator separately in various moving frames. In particular, they achieved a complete picture in the local diffusion case. However, the question of the persistence of the prey and the predator in some intermediate moving frames in the nonlocal diffusion case is left open in Choi et al.'s paper. By using some prior estimates, the Arzela-Ascoli theorem and a diagonal extraction process, we can extend and improve the main results of Choi et al. to achieve a complete picture in the nonlocal diffusion case.

math.AP↗

arXiv2023

Traveling Wave in a Ratio-dependent Holling-Tanner System with Nonlocal Diffusion and Strong Allee Effect

In this paper, a ratio-dependent Holling-Tanner system with nonlocal diffusion is taken into account, where the prey is subject to a strong Allee effect. To be special, by applying Schauder's fixed point theorem and iterative technique, we provide a general theory on the existence of traveling waves for such system. Then appropriate upper and lower solutions and a novel sequence, similar to squeeze method, are constructed to demonstrate the existence of traveling waves for c>c*. Moreover, the existence of traveling wave for c=c* is also established by spreading speed theory and comparison principle. Finally, the nonexistence of traveling waves for c<c* is investigated, and the minimal wave speed then is determined.

math.AP↗

arXiv2023

Traveling Waves of Modified Leslie-Gower Predator-prey Systems

The spreading phenomena in modified Leslie-Gower reaction-diffusion predator-prey systems are the topic of this paper. We mainly study the existence of two different types of traveling waves. Be specific, with the aid of the upper and lower solutions method, we establish the existence of traveling wave connecting the prey-present state and the coexistence state or the prey-present state and the prey-free state by constructing different and appropriate Lyapunov functions. Moreover, for traveling wave connecting the prey-present state and the prey-free state, we gain more monotonicity information on wave profile based on the asymptotic behavior at negative infinite. Finally, our results are applied to modified Leslie-Gower system with Holling II type or Lotka-Volterra type, and then a novel Lyapunov function is constructed for the latter, which further enhances our results. Meanwhile, some numerical simulations are carried to support our results.

math.AP↗

arXiv2023

A sufficient condition on successful invasion by the predator

In this paper, we provide a sufficient condition on successful invasion by the predator. Specially, we obtain the persistence of traveling wave solutions of predator-prey system, in which the predator can survive without the predation of the prey. This proof heavily depends on comparison principle of scalar monostable equation, the rescaling method and phase-plane analysis.

math.AP↗