arXiv subjects

Yanze Wu

Publications and source records attributed to Yanze Wu.

33 records · Page 2Linked to original sources

arXiv2023

T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models

The incredible generative ability of large-scale text-to-image (T2I) models has demonstrated strong power of learning complex structures and meaningful semantics. However, relying solely on text prompts cannot fully take advantage of the knowledge learned by the model, especially when flexible and accurate controlling (e.g., color and structure) is needed. In this paper, we aim to ``dig out" the capabilities that T2I models have implicitly learned, and then explicitly use them to control the generation more granularly. Specifically, we propose to learn simple and lightweight T2I-Adapters to align internal knowledge in T2I models with external control signals, while freezing the original large T2I models. In this way, we can train various adapters according to different conditions, achieving rich control and editing effects in the color and structure of the generation results. Further, the proposed T2I-Adapters have attractive properties of practical value, such as composability and generalization ability. Extensive experiments demonstrate that our T2I-Adapter has promising generation quality and a wide range of applications.

cs.CV↗

arXiv2023

AnimeSR: Learning Real-World Super-Resolution Models for Animation Videos

This paper studies the problem of real-world video super-resolution (VSR) for animation videos, and reveals three key improvements for practical animation VSR. First, recent real-world super-resolution approaches typically rely on degradation simulation using basic operators without any learning capability, such as blur, noise, and compression. In this work, we propose to learn such basic operators from real low-quality animation videos, and incorporate the learned ones into the degradation generation pipeline. Such neural-network-based basic operators could help to better capture the distribution of real degradations. Second, a large-scale high-quality animation video dataset, AVC, is built to facilitate comprehensive training and evaluations for animation VSR. Third, we further investigate an efficient multi-scale network structure. It takes advantage of the efficiency of unidirectional recurrent networks and the effectiveness of sliding-window-based methods. Thanks to the above delicate designs, our method, AnimeSR, is capable of restoring real-world low-quality animation videos effectively and efficiently, achieving superior performance to previous state-of-the-art methods. Codes and models are available at https://github.com/TencentARC/AnimeSR.

cs.CV↗

arXiv2022

Non-adiabatic Dynamics in a Continuous Circularly Polarized Laser Field with Floquet Phase-space Surface Hopping

Non-adiabatic chemical reactions involving continuous circularly polarized light (cw CPL) have not attracted as much attention as dynamics in unpolarized/linearly polarized light. However, including circularly (in contrast to linearly) polarized light allows one to effectively introduce a complex-valued time-dependent Hamiltonian, which offers a new path for control or exploration through the introduction of Berry forces. Here, we investigate several inexpensive semiclassical approaches for modeling such nonadiabatic dynamics in the presence of a time-dependent complex-valued Hamiltonian, beginning with a straightforward instantaneous adiabatic fewest-switches surface hopping (IA-FSSH) approach (where the electronic states depend on position and time), continuing to a standard Floquet fewest switches surface hopping (F-FSSH) approach (where the electronic states depend on position and frequency), and ending with an exotic Floquet phase-space surface hopping (F-PSSH) approach (where the electronic states depend on position, frequency, and momentum). Using a set of model systems with time-dependent complex-valued Hamiltonians, we show that the Floquet phase-space adiabats are the optimal choice of basis as far as accounting for Berry phase effects and delivering accuracy. Thus, the F-PSSH algorithm sets the stage for modeling nonadiabatic dynamics under strong externally pumped circular polarization in the future.

physics.chem-ph↗

arXiv2022

A Quantum-Classical Liouville Formalism in a Preconditioned Basis and Its Connection with Phase-Space Surface Hopping

We revisit a recent proposal to model nonadiabatic problems with a complex-valued Hamiltonian through a phase-space surface hopping (PSSH) algorithm employing a pseudo-diabatic basis. Here, we show that such a pseudo-diabatic PSSH (PD-PSSH) ansatz is consistent with a quantum-classical Liouville equation (QCLE) that can be derived following a preconditioning process, and we demonstrate that a proper PD-PSSH algorithm is able to capture some geometric magnetic effects (whereas the standard FSSH approach cannot). We also find that a preconditioned QCLE can outperform the standard QCLE in certain cases, highlighting the fact that there is no unique QCLE. Lastly, we also point out that one can construct a mean-field Ehrenfest algorithm using a phase-space representation similar to what is done for PSSH. These findings would appear extremely helpful as far understanding and simulating nonadiabatic dynamics with complex-valued Hamiltonians and/or spin degeneracy.

quant-ph↗

arXiv2022

The Effect of Duschinskii Rotations on Spin-Dependent Electron Transfer Dynamics

We investigate spin-dependent electron transfer in the presence of a Duschinskii rotation. In particular, we propagate dynamics for a two-level model system for which spin-orbit coupling introduces an interstate coupling of the form $e^{iWx}$, which is both position(x)-dependent and complex-valued. We demonstrate that two-level systems coupled to Brownian oscillators with Duschinskii rotations (and thus entangled normal modes) can produce marked increases in transient spin polarization relative to two-level systems coupled to simple shifted harmonic oscillators. These conclusions should have significant relevance for modeling the effect of nuclear motion on chiral induced spin selectivity.

quant-ph↗

arXiv2022

Modeling Spin-Dependent Nonadiabatic Dynamics with Electronic Degeneracy: A Phase-Space Surface-Hopping Method

Nuclear Berry curvature effects emerge from electronic spin degeneracy and canlead to non-trivial spin-dependent (nonadiabatic) nuclear dynamics. However, such effects are completely neglected in all current mixed quantum-classical methods such as fewest switches surface-hopping. In this work, we present a phase-space surface-hopping (PSSH) approach to simulate singlet-triplet intersystem crossing dynamics. We show that with a simple pseudo-diabatic ansatz, a PSSH algorithm can capture the relevant Berry curvature effects and make predictions in agreement with exact quantum dynamics for a simple singlet-triplet model Hamiltonian. Thus, this approach represents an important step towards simulating photochemical and spin processes concomitantly, as relevant to intersystem crossing and spin-lattice relaxation dynamics.

physics.chem-ph↗

arXiv2022

Towards Vivid and Diverse Image Colorization with Generative Color Prior

Colorization has attracted increasing interest in recent years. Classic reference-based methods usually rely on external color images for plausible results. A large image database or online search engine is inevitably required for retrieving such exemplars. Recent deep-learning-based methods could automatically colorize images at a low cost. However, unsatisfactory artifacts and incoherent colors are always accompanied. In this work, we propose GCP-Colorization that leverages the rich and diverse color priors encapsulated in a pretrained Generative Adversarial Networks (GAN) for automatic colorization. Specifically, we first "retrieve" matched features (similar to exemplars) via a GAN encoder and then incorporate these features into the colorization process with feature modulations. Thanks to the powerful generative color prior (GCP) and delicate designs, our GCP-Colorization could produce vivid colors with a single forward pass. Moreover, it is highly convenient to obtain diverse results by modifying GAN latent codes. GCP-Colorization also inherits the merit of interpretable controls of GANs and could attain controllable and smooth transitions by walking through GAN latent space. Extensive experiments and user studies demonstrate that GCP-Colorization achieves superior performance than previous works. Codes are available at https://github.com/ToTheBeginning/GCP-Colorization.

cs.CV↗

arXiv2022

MM-RealSR: Metric Learning based Interactive Modulation for Real-World Super-Resolution

Interactive image restoration aims to restore images by adjusting several controlling coefficients, which determine the restoration strength. Existing methods are restricted in learning the controllable functions under the supervision of known degradation types and levels. They usually suffer from a severe performance drop when the real degradation is different from their assumptions. Such a limitation is due to the complexity of real-world degradations, which can not provide explicit supervision to the interactive modulation during training. However, how to realize the interactive modulation in real-world super-resolution has not yet been studied. In this work, we present a Metric Learning based Interactive Modulation for Real-World Super-Resolution (MM-RealSR). Specifically, we propose an unsupervised degradation estimation strategy to estimate the degradation level in real-world scenarios. Instead of using known degradation levels as explicit supervision to the interactive mechanism, we propose a metric learning strategy to map the unquantifiable degradation levels in real-world scenarios to a metric space, which is trained in an unsupervised manner. Moreover, we introduce an anchor point strategy in the metric learning process to normalize the distribution of metric space. Extensive experiments demonstrate that the proposed MM-RealSR achieves excellent modulation and restoration performance in real-world super-resolution. Codes are available at https://github.com/TencentARC/MM-RealSR.

cs.CV↗

arXiv2022

A Phase-Space Semiclassical Approach for Modeling Nonadiabatic Nuclear Dynamics with Electronic Spin

Chemical relaxation phenomena, including photochemistry and electron transfer processes, form a vigorous area of research in which nonadiabatic dynamics plays a fundamental role. Here, we show that for nonadiabatic dynamics with two electronic states and a complex-valued Hamiltonian that does not obey time-reversal symmetry, the optimal semiclassical approach is to run surface hopping dynamics on a set of phase-space adiabatic surfaces. In order to generate such phase-adiabats, one must isolate a proper set of diabats and apply a phase gauge transformation, before eventually diagonalizing the total Hamiltonian (which is now parameterized by both R and P). The resulting algorithm is valid in both the adiabatic and nonadiabatic limits, incorporates all Berry curvature effects, and allows for the study of semiclassical nonadiabatic dynamics in the presence of spin-orbit coupling and/or external magnetic fields.

physics.chem-ph↗

arXiv2022

Electron Transfer and Spin-Orbit Coupling: How Strong are Berry Force Effects In and Out of Equilibrium in the Presence of Nuclear Friction?

We investigate a spin-boson inspired model of electron transfer, where the diabatic coupling is given by a position-dependent phase, exp(iWx). We consider both equilibrium and nonequilibrium initial conditions. We show that, for this model, all equilibrium results are completely invariant to the sign of W(to infinite order). However, the nonequilibrium results do depend on the sign of W, suggesting that photo-induced electron transfer dynamics are meaningfully affected by Berry forces even in the presence of nuclear friction; furthermore, whenever there is spin-orbit coupling, electronic spin polarization can emerge (at least for some time).

physics.chem-ph↗

arXiv2022

Incorporating Berry Force Effects into The Fewest Switches Surface Hopping Algorithm: Intersystem Crossing and The Case of Electronic Degeneracy

We present a preliminary surface-hopping approach for modeling intersystem crossing (ISC) dynamics between four electronic states: one singlet and one (triply degenerate) triplet. In order to incorporate all Berry force effects, the algorithm requires that, when moving along an adiabatic surface associated with the triplet manifold, \mycomment{one must also keep track of a quasi-diabatic index (akin to a "$m_s$" quantum number) for each trajectory. For a simple model problem, we find that a great deal of new physics can be captured by our algorithm, setting the stage for larger, more realistic (or perhaps even {\em ab initio}) simulations in the future.

physics.chem-ph↗

arXiv2020

Question Guided Modular Routing Networks for Visual Question Answering

This paper studies the task of Visual Question Answering (VQA), which is topical in Multimedia community recently. Particularly, we explore two critical research problems existed in VQA: (1) efficiently fusing the visual and textual modalities; (2) enabling the visual reasoning ability of VQA models in answering complex questions. To address these challenging problems, a novel Question Guided Modular Routing Networks (QGMRN) has been proposed in this paper. Particularly, The QGMRN is composed of visual, textual and routing network. The visual and textual network serve as the backbones for the generic feature extractors of visual and textual modalities. QGMRN can fuse the visual and textual modalities at multiple semantic levels. Typically, the visual reasoning is facilitated by the routing network in a discrete and stochastic way by using Gumbel-Softmax trick for module selection. When the input reaches a certain modular layer, routing network newly proposed in this paper, dynamically selects a portion of modules from that layer to process the input depending on the question features generated by the textual network. It can also learn to reason by routing between the generic modules without additional supervision information or expert knowledge. Benefiting from the dynamic routing mechanism, QGMRN can outperform the previous classical VQA methods by a large margin and achieve the competitive results against the state-of-the-art methods. Furthermore, attention mechanism is integrated into our QGMRN model and thus can further boost the model performance. Empirically, extensive experiments on the CLEVR and CLEVR-Humans datasets validate the effectiveness of our proposed model, and the state-of-the-art performance has been achieved.

cs.CV↗

arXiv2020

Near Total Electronic Spin Separation as Caused by Nuclear Dynamics: Perturbing a Real-Valued Conical Intersection with Complex-Valued Spin-Orbit Coupling

We investigate the nuclear dynamics near a real-valued conical intersection that is perturbed by a complex-valued spin-orbit coupling. For a model Hamiltonian with two outgoing channels, we find that even a small spin-orbit coupling can dramatically affect the pathway selection on account of Berry force, leading to extremely large spin selectivity (sometime as large as 100%). Thus, this Letter opens the door for organic chemists to start designing spintronic devices that use nuclear motion and conical intersections (combined with standard spin-orbit coupling) in order to achieve spin selection. Vice versa, for physical chemists, this Letter also emphasizes that future semiclassical simulations of intersystem crossing (which have heretofore ignored Berry force) should be corrected to account for the spin polarization that inevitably arises when dynamics pass near conical intersections.

physics.chem-ph↗

arXiv2020

Chemical Reaction Rates for Systems with Spin-Orbit Coupling and an Odd Number of Electrons: Does Berry's Phase Lead to Meaningful Spin-Dependent Nuclear Dynamics for a Two State Crossing?

Within the context of very simple avoided crossing, we investigate the investigate the effect of a complex diabatic coupling in determining spin-dependent rate constants and scattering states. We find that, if the molecular geometry is not linear and the Berry force is not zero, one can find significant spin polarization of the products. This study emphasizes that, when analyzing nonadiabatic reactions with spin orbit coupling (and a complex Hamiltonian), one must consider how Berry force affects nuclear motion -- at least in the context of gas phase reactions. Work is currently ongoing as far as extrapolating these conclusions to the condensed phase where interesting spin selection has been observed in recent years.

physics.chem-ph↗

arXiv2019

Swimming to Stability: Structural and Dynamical Control via Active Doping

External fields can decidedly alter the free energy landscape of soft materials and can be exploited as a powerful tool for the assembly of targeted nanostructures and colloidal materials. Here, we use computer simulations to demonstrate that nonequilibrium internal fields or forces -- forces that are generated by driven components within a system -- in the form of active particles can precisely modulate the dynamical free energy landscape of a model soft material, a colloidal gel. Embedding a small fraction of active particles within a gel can provide a unique pathway for the dynamically frustrated network to circumvent the kinetic barriers associated with reaching a lower free energy state through thermal fluctuations alone. Moreover, by carefully tuning the active particle properties (the propulsive swim force and persistence length) in comparison to those of the gel, the active particles may induce depletion-like forces between the constituent particles of the gel despite there being no geometric size asymmetry between the particles. These resulting forces can rapidly push the system toward disparate regions of phase space. Intriguingly, the state of the material can be altered by tuning macroscopic transport properties such as the solvent viscosity. Our findings highlight the potential wide-ranging structural and kinetic control facilitated by varying the dynamical properties of a remarkably small fraction of driven particles embedded in a host material.

cond-mat.soft↗