Searcharxiv⌕ Search

arXiv subjects

Mengqi Huang

Publications and source records attributed to Mengqi Huang.

At least 37 records · Page 2Linked to original sources
arXiv2025

CustomContrast: A Multilevel Contrastive Perspective For Subject-Driven Text-to-Image Customization

Subject-driven text-to-image (T2I) customization has drawn significant interest in academia and industry. This task enables pre-trained models to generate novel images based on unique subjects. Existing studies adopt a self-reconstructive perspective, focusing on capturing all details of a single image, which will misconstrue the specific image's irrelevant attributes (e.g., view, pose, and background) as the subject intrinsic attributes. This misconstruction leads to both overfitting or underfitting of irrelevant and intrinsic attributes of the subject, i.e., these attributes are over-represented or under-represented simultaneously, causing a trade-off between similarity and controllability. In this study, we argue an ideal subject representation can be achieved by a cross-differential perspective, i.e., decoupling subject intrinsic attributes from irrelevant attributes via contrastive learning, which allows the model to focus more on intrinsic attributes through intra-consistency (features of the same subject are spatially closer) and inter-distinctiveness (features of different subjects have distinguished differences). Specifically, we propose CustomContrast, a novel framework, which includes a Multilevel Contrastive Learning (MCL) paradigm and a Multimodal Feature Injection (MFI) Encoder. The MCL paradigm is used to extract intrinsic features of subjects from high-level semantics to low-level appearance through crossmodal semantic contrastive learning and multiscale appearance contrastive learning. To facilitate contrastive learning, we introduce the MFI encoder to capture cross-modal representations. Extensive experiments show the effectiveness of CustomContrast in subject similarity and text controllability.

cs.CV↗
arXiv2024

VMix: Improving Text-to-Image Diffusion Model with Cross-Attention Mixing Control

While diffusion models show extraordinary talents in text-to-image generation, they may still fail to generate highly aesthetic images. More specifically, there is still a gap between the generated images and the real-world aesthetic images in finer-grained dimensions including color, lighting, composition, etc. In this paper, we propose Cross-Attention Value Mixing Control (VMix) Adapter, a plug-and-play aesthetics adapter, to upgrade the quality of generated images while maintaining generality across visual concepts by (1) disentangling the input text prompt into the content description and aesthetic description by the initialization of aesthetic embedding, and (2) integrating aesthetic conditions into the denoising process through value-mixed cross-attention, with the network connected by zero-initialized linear layers. Our key insight is to enhance the aesthetic presentation of existing diffusion models by designing a superior condition control method, all while preserving the image-text alignment. Through our meticulous design, VMix is flexible enough to be applied to community models for better visual performance without retraining. To validate the effectiveness of our method, we conducted extensive experiments, showing that VMix outperforms other state-of-the-art methods and is compatible with other community modules (e.g., LoRA, ControlNet, and IPAdapter) for image generation. The project page is https://vmix-diffusion.github.io/VMix/.

cs.CV↗
arXiv2024

Observation of stacking engineered magnetic phase transitions within moiré supercells of twisted van der Waals magnets

Twist engineering of magnetic van der Waals (vdW) moiré superlattices provides an attractive way to achieve precise nanoscale control over the spin degree of freedom on two-dimensional flatland. Despite the very recent demonstrations of moiré magnetism featuring exotic phases with noncollinear spin order in twisted vdW magnet chromium triiodide CrI3, the local magnetic interactions, spin dynamics, and magnetic phase transitions within and across individual moiré supercells remain elusive. Taking advantage of a scanning single-spin magnetometry platform, here we report observation of two distinct magnetic phase transitions with separate critical temperatures within a moiré supercell of small-angle twisted double trilayer CrI3. By measuring temperature dependent spin fluctuations at the coexisting ferromagnetic and antiferromagnetic regions in twisted CrI3, we explicitly show that the Curie temperature of the ferromagnetic state is higher than the Néel temperature of the antiferromagnetic one by ~10 K. Our mean-field calculations attribute such a spatial and thermodynamic phase separation to the stacking order modulated interlayer exchange coupling at the twisted interface of the moiré superlattices. The presented results highlight twist engineering as a promising tuning knob to realize on-demand control of not only the nanoscale spin order of moiré quantum matter but also its dynamic magnetic responses, which may find relevant applications in developing transformative vdW electronic and magnetic devices.

cond-mat.mes-hall↗
arXiv2024

Sensing Spin Wave Excitations by Spin Defects in Few-Layer Thick Hexagonal Boron Nitride

Optically active spin defects in wide band-gap semiconductors serve as a local sensor of multiple degrees of freedom in a variety of "hard" and "soft" condensed matter systems. Taking advantage of the recent progress on quantum sensing using van der Waals (vdW) quantum materials, here we report direct measurements of spin waves excited in magnetic insulator Y3Fe5O12 (YIG) by boron vacancy $V_B^-$ spin defects contained in few-layer thick hexagonal boron nitride nanoflakes. We show that the ferromagnetic resonance and parametric spin excitations can be effectively detected by $V_B^-$ spin defects under various experimental conditions through optically detected magnetic resonance measurements. The off-resonant dipole interaction between YIG magnons and $V_B^-$ spin defects is mediated by multi-magnon scattering processes, which may find relevant applications in a range of emerging quantum sensing, computing, and metrology technologies. Our results also highlight the opportunities offered by quantum spin defects in layered two-dimensional vdW materials for investigating local spin dynamic behaviors in magnetic solid-state matters.

cond-mat.mtrl-sci↗
arXiv2024

RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customization

Text-to-image customization, which aims to synthesize text-driven images for the given subjects, has recently revolutionized content creation. Existing works follow the pseudo-word paradigm, i.e., represent the given subjects as pseudo-words and then compose them with the given text. However, the inherent entangled influence scope of pseudo-words with the given text results in a dual-optimum paradox, i.e., the similarity of the given subjects and the controllability of the given text could not be optimal simultaneously. We present RealCustom that, for the first time, disentangles similarity from controllability by precisely limiting subject influence to relevant parts only, achieved by gradually narrowing real text word from its general connotation to the specific subject and using its cross-attention to distinguish relevance. Specifically, RealCustom introduces a novel "train-inference" decoupled framework: (1) during training, RealCustom learns general alignment between visual conditions to original textual conditions by a novel adaptive scoring module to adaptively modulate influence quantity; (2) during inference, a novel adaptive mask guidance strategy is proposed to iteratively update the influence scope and influence quantity of the given subjects to gradually narrow the generation of the real text word. Comprehensive experiments demonstrate the superior real-time customization ability of RealCustom in the open domain, achieving both unprecedented similarity of the given subjects and controllability of the given text for the first time. The project page is https://corleone-huang.github.io/realcustom/.

cs.CV↗
arXiv2024

Gradual Residuals Alignment: A Dual-Stream Framework for GAN Inversion and Image Attribute Editing

GAN-based image attribute editing firstly leverages GAN Inversion to project real images into the latent space of GAN and then manipulates corresponding latent codes. Recent inversion methods mainly utilize additional high-bit features to improve image details preservation, as low-bit codes cannot faithfully reconstruct source images, leading to the loss of details. However, during editing, existing works fail to accurately complement the lost details and suffer from poor editability. The main reason is they inject all the lost details indiscriminately at one time, which inherently induces the position and quantity of details to overfit source images, resulting in inconsistent content and artifacts in edited images. This work argues that details should be gradually injected into both the reconstruction and editing process in a multi-stage coarse-to-fine manner for better detail preservation and high editability. Therefore, a novel dual-stream framework is proposed to accurately complement details at each stage. The Reconstruction Stream is employed to embed coarse-to-fine lost details into residual features and then adaptively add them to the GAN generator. In the Editing Stream, residual features are accurately aligned by our Selective Attention mechanism and then injected into the editing process in a multi-stage manner. Extensive experiments have shown the superiority of our framework in both reconstruction accuracy and editing quality compared with existing methods.

cs.CV↗
arXiv2023

Local control of a single nitrogen-vacancy center by nanoscale engineered magnetic domain wall motions

Effective control and readout of qubits form the technical foundation of next-generation, transformative quantum information sciences and technologies. The nitrogen-vacancy (NV) center, an intrinsic three-level spin system, is naturally relevant in this context due to its excellent quantum coherence, high fidelity of operations, and remarkable functionality over a broad range of experimental conditions. It is an active contender for the development and implementation of cutting-edge quantum technologies. Here, we report magnetic domain wall motion driven local control and measurements of NV spin properties. By engineering the local magnetic field environment of an NV center via nanoscale reconfigurable domain wall motions, we show that NV photoluminescence, spin level energies, and coherence time can be reliably controlled and correlated to the magneto-transport response of a magnetic device. Our results highlight the electrically tunable dipole interaction between NV centers and nanoscale magnetic structures, providing an attractive platform to realize interactive information transfer between spin qubits and non-volatile magnetic memory in hybrid quantum spintronic systems.

cond-mat.mes-hall↗
arXiv2023

Dimensionality crossover to 2D vestigial nematicity from 3D zigzag antiferromagnetism in an XY-type honeycomb van der Waals magnet

Fluctuations and disorder effects are substantially enhanced in reduced dimensionalities. While they are mostly considered as the foe for long-range orders, fluctuations and disorders can also stimulate the emergence of novel phases of matter, for example, vestigial orders. Taking 2D magnetism as a platform, existing efforts have been focused on maintaining 2D long-range magnetic orders by suppressing fluctuations, whereas the other side, exploiting fluctuations for realizing new 2D magnetic phases, remains as an uncharted territory. Here, using a combination of NV spin relaxometry, optical spectroscopy, and Monte Carlo simulations, we report, in an XY-type honeycomb magnet NiPS3, the phase transition from the zigzag AFM order in 3D bulk to a new Z3 vestigial Potts-nematicity in 2D few layers. Spin fluctuations are shown to significantly enhance over the GHz-THz range as the layer number of NiPS3 reduces, using the NV spin relaxometry and the optical Raman quasi-elastic scattering. As a result, the Raman signatures of the zigzag AFM for bulk NiPS3, a zone-folded phonon at ~30cm-1 from the broken translational symmetry (PBTS) and a degeneracy lift of two phonons at ~180cm-1 for the broken 3-fold rotational symmetry (PBRS), evolve into the disappearance of PBTS and the survival of PBRS in few-layer NiPS3, with a critical thickness of ~10nm. The optical linear dichroism microscopy images all three nematic domain states in a single few-layer NiPS3 flake. The large-scale Monte Carlo simulations for bilayer NiPS3 model confirms the absence of long-range zigzag AFM order but the formation of the Z3 vestigial Potts-nematic phase, corroborating with the experimental finding. Our results demonstrate the positivity of strong fluctuations in creating new phases of matter after destroying more conventional ones, and offer an unprecedented pathway for developing novel 2D phases.

cond-mat.mtrl-sci↗
arXiv2023

Layer-dependent magnetism and spin fluctuations in atomically thin van der Waals magnet CrPS4

van der Waals (vdW) magnets, an emerging family of two-dimensional (2D) materials, have received tremendous attention due to their rich fundamental physics and significant potential for cutting-edge technological applications. In contrast to the conventional bulk counterparts, vdW magnets exhibit significant tunability of local material properties, such as stacking engineered interlayer coupling and layer-number dependent magnetic and electronic interactions, which promise to deliver previously unavailable merits to develop multifunctional microelectronic devices. As a further ingredient of this emerging topic, here we report nanoscale quantum sensing and imaging of atomically thin vdW magnet chromium thiophosphate CrPS4, revealing its characteristic layer-dependent 2D static magnetism and dynamic spin fluctuations. We also show a large tunneling magnetoresistance in CrPS4-based spin filter vdW heterostructures. The excellent material stability, robust strategy against environmental degradation, in combination with tailored magnetic properties highlight the potential of CrPS4 in developing state-of-the-art 2D spintronic devices for next-generation information technologies.

cond-mat.mes-hall↗
arXiv2023

Quantum metric nonlinear Hall effect in a topological antiferromagnetic heterostructure

Quantum geometry - the geometry of electron Bloch wavefunctions - is central to modern condensed matter physics. Due to the quantum nature, quantum geometry has two parts, the real part quantum metric and the imaginary part Berry curvature. The studies of Berry curvature have led to countless breakthroughs, ranging from the quantum Hall effect in 2DEGs to the anomalous Hall effect (AHE) in ferromagnets. However, in contrast to Berry curvature, the quantum metric has rarely been explored. Here, we report a new nonlinear Hall effect induced by quantum metric by interfacing even-layered MnBi2Te4 (a PT-symmetric antiferromagnet (AFM)) with black phosphorus. This novel nonlinear Hall effect switches direction upon reversing the AFM spins and exhibits distinct scaling that suggests a non-dissipative nature. Like the AHE brought Berry curvature under the spotlight, our results open the door to discovering quantum metric responses. Moreover, we demonstrate that the AFM can harvest wireless electromagnetic energy via the new nonlinear Hall effect, therefore enabling intriguing applications that bridges nonlinear electronics with AFM spintronics.

cond-mat.mes-hall↗
arXiv2023

Revealing intrinsic domains and fluctuations of moiré magnetism by a wide-field quantum microscope

Moiré magnetism featured by stacking engineered atomic registry and lattice interactions has recently emerged as an appealing quantum state of matter at the forefront condensed matter physics research. Nanoscale imaging of moiré magnets is highly desirable and serves as a prerequisite to investigate a broad range of intriguing physics underlying the interplay between topology, electronic correlations, and unconventional nanomagnetism. Here we report spin defect-based wide-field imaging of magnetic domains and spin fluctuations in twisted double trilayer (tDT) chromium triiodide CrI3. We explicitly show that intrinsic moiré domains of opposite magnetizations appear over arrays of moiré supercells in low-twist-angle tDT CrI3. In contrast, spin fluctuations measured in tDT CrI3 manifest little spatial variations on the same mesoscopic length scale due to the dominant driving force of intralayer exchange interaction. Our results enrich the current understanding of exotic magnetic phases sustained by moiré magnetism and highlight the opportunities provided by quantum spin sensors in probing microscopic spin related phenomena on two-dimensional flatland.

cond-mat.mes-hall↗
arXiv2023

DreamIdentity: Improved Editability for Efficient Face-identity Preserved Image Generation

While large-scale pre-trained text-to-image models can synthesize diverse and high-quality human-centric images, an intractable problem is how to preserve the face identity for conditioned face images. Existing methods either require time-consuming optimization for each face-identity or learning an efficient encoder at the cost of harming the editability of models. In this work, we present an optimization-free method for each face identity, meanwhile keeping the editability for text-to-image models. Specifically, we propose a novel face-identity encoder to learn an accurate representation of human faces, which applies multi-scale face features followed by a multi-embedding projector to directly generate the pseudo words in the text embedding space. Besides, we propose self-augmented editability learning to enhance the editability of models, which is achieved by constructing paired generated face and edited face images using celebrity names, aiming at transferring mature ability of off-the-shelf text-to-image models in celebrity faces to unseen faces. Extensive experiments show that our methods can generate identity-preserved images under different scenes at a much faster speed.

cs.CV↗
arXiv2023

Not All Image Regions Matter: Masked Vector Quantization for Autoregressive Image Generation

Existing autoregressive models follow the two-stage generation paradigm that first learns a codebook in the latent space for image reconstruction and then completes the image generation autoregressively based on the learned codebook. However, existing codebook learning simply models all local region information of images without distinguishing their different perceptual importance, which brings redundancy in the learned codebook that not only limits the next stage's autoregressive model's ability to model important structure but also results in high training cost and slow generation speed. In this study, we borrow the idea of importance perception from classical image coding theory and propose a novel two-stage framework, which consists of Masked Quantization VAE (MQ-VAE) and Stackformer, to relieve the model from modeling redundancy. Specifically, MQ-VAE incorporates an adaptive mask module for masking redundant region features before quantization and an adaptive de-mask module for recovering the original grid image feature map to faithfully reconstruct the original images after quantization. Then, Stackformer learns to predict the combination of the next code and its position in the feature map. Comprehensive experiments on various image generation validate our effectiveness and efficiency. Code will be released at https://github.com/CrossmodalGroup/MaskedVectorQuantization.

cs.CV↗
arXiv2023

Towards Accurate Image Coding: Improved Autoregressive Image Generation with Dynamic Vector Quantization

Existing vector quantization (VQ) based autoregressive models follow a two-stage generation paradigm that first learns a codebook to encode images as discrete codes, and then completes generation based on the learned codebook. However, they encode fixed-size image regions into fixed-length codes and ignore their naturally different information densities, which results in insufficiency in important regions and redundancy in unimportant ones, and finally degrades the generation quality and speed. Moreover, the fixed-length coding leads to an unnatural raster-scan autoregressive generation. To address the problem, we propose a novel two-stage framework: (1) Dynamic-Quantization VAE (DQ-VAE) which encodes image regions into variable-length codes based on their information densities for an accurate and compact code representation. (2) DQ-Transformer which thereby generates images autoregressively from coarse-grained (smooth regions with fewer codes) to fine-grained (details regions with more codes) by modeling the position and content of codes in each granularity alternately, through a novel stacked-transformer architecture and shared-content, non-shared position input layers designs. Comprehensive experiments on various generation tasks validate our superiorities in both effectiveness and efficiency. Code will be released at https://github.com/CrossmodalGroup/DynamicVectorQuantization.

cs.CV↗
arXiv2023

Nanoscale magnetic domains in polycrystalline Mn3Sn films imaged by a scanning single-spin magnetometer

Noncollinear antiferromagnets with novel magnetic orders, vanishingly small net magnetization and exotic spin related properties hold enormous promise for developing next-generation, transformative spintronic applications. A major ongoing research focus of this community is to explore, control, and harness unconventional magnetic phases of this emergent material system to deliver state-of-the-art functionalities for modern microelectronics. Here we report direct imaging of magnetic domains of polycrystalline Mn3Sn films, a prototypical noncollinear antiferromagnet, using nitrogen-vacancy-based single-spin scanning microscopy. Nanoscale evolution of local stray field patterns of Mn3Sn samples are systematically investigated in response to external driving forces, revealing the characteristic "heterogeneous" magnetic switching behaviors in polycrystalline textured Mn3Sn films. Our results contribute to a comprehensive understanding of inhomogeneous magnetic orders of noncollinear antiferromagnets, highlighting the potential of nitrogen-vacancy centers to study microscopic spin properties of a broad range of emergent condensed matter systems.

cond-mat.mes-hall↗
arXiv2022

DSE-GAN: Dynamic Semantic Evolution Generative Adversarial Network for Text-to-Image Generation

Text-to-image generation aims at generating realistic images which are semantically consistent with the given text. Previous works mainly adopt the multi-stage architecture by stacking generator-discriminator pairs to engage multiple adversarial training, where the text semantics used to provide generation guidance remain static across all stages. This work argues that text features at each stage should be adaptively re-composed conditioned on the status of the historical stage (i.e., historical stage's text and image features) to provide diversified and accurate semantic guidance during the coarse-to-fine generation process. We thereby propose a novel Dynamical Semantic Evolution GAN (DSE-GAN) to re-compose each stage's text features under a novel single adversarial multi-stage architecture. Specifically, we design (1) Dynamic Semantic Evolution (DSE) module, which first aggregates historical image features to summarize the generative feedback, and then dynamically selects words required to be re-composed at each stage as well as re-composed them by dynamically enhancing or suppressing different granularity subspace's semantics. (2) Single Adversarial Multi-stage Architecture (SAMA), which extends the previous structure by eliminating complicated multiple adversarial training requirements and therefore allows more stages of text-image interactions, and finally facilitates the DSE module. We conduct comprehensive experiments and show that DSE-GAN achieves 7.48\% and 37.8\% relative FID improvement on two widely used benchmarks, i.e., CUB-200 and MSCOCO, respectively.

cs.CV↗
arXiv2022

Electric-Field-Induced Coherent Control of Nitrogen Vacancy Centers

Enabling scalable and energy-efficient control of spin defects in solid-state media is desirable for realizing transformative quantum information technologies. Exploiting voltage-controlled magnetic anisotropy, we report coherent manipulation of nitrogen-vacancy (NV) centers by the spatially confined magnetic stray fields produced by a proximate resonant magnetic tunnel junction (MTJ). Remarkably, the coherent coupling between NV centers and the MTJ can be systematically controlled by a DC bias voltage, allowing for appreciable electrical tunability in the presented hybrid system. In comparison with current state-of-the-art techniques, the demonstrated NV-based quantum operational platform exhibits significant advantages in scalability, device compatibility, and energy-efficiency, further expanding the role of NV centers in a broad range of quantum computing, sensing, and communications applications.

cond-mat.mes-hall↗
arXiv2022

Quantum sensing and imaging of spin-orbit-torque-driven spin dynamics in noncollinear antiferromagnet Mn3Sn

Novel noncollinear antiferromagnets with spontaneous time-reversal symmetry breaking, nontrivial band topology, and unconventional transport properties have received immense research interest over the past decade due to their rich physics and enormous promise in technological applications. One of the central focuses in this emerging field is exploring the relationship between the microscopic magnetic structure and exotic material properties. Here, the nanoscale imaging of both spin-orbit-torque-induced deterministic magnetic switching and chiral spin rotation in noncollinear antiferromagnet Mn3Sn films using nitrogen-vacancy (NV) centers is reported. Direct evidence of the off-resonance dipole-dipole coupling between the spin dynamics in Mn3Sn and proximate NV centers is also demonstrated with NV relaxometry measurements. These results demonstrate the unique capabilities of NV centers in accessing the local information of the magnetic order and dynamics in these emergent quantum materials and suggest new opportunities for investigating the interplay between topology and magnetism in a broad range of topological magnets.

cond-mat.mes-hall↗