arXiv subjects

Bowen Ma

Publications and source records attributed to Bowen Ma.

36 records · Page 2Linked to original sources

arXiv2024

Chiral phonons induced from spin dynamics via magnetoelastic anisotropy

We propose a mechanism to obtain chiral phonon-like excitations from the bond-dependent magnetoelastic couplings in the absence of out-of-plane magnetization and magnetic fields. By mapping the hybrid excitation to its phononic analog, we reveal the impact of the lattice symmetry on the origin of the chirality. In the example of a triangular lattice ferromagnet, we recognize that the system is equivalent to the class D of topological phonons, and show the tunable chirality and topology by an in-plane magnetic field. As a possible experimental probe, we evaluate the phonon magnetization and planar thermal Hall conductivity. Our study gives a new perspective on tunable topological and chiral excitations beyond the Raman spin-lattice coupling, suggesting possible applications of spintronics and phononics in various anisotropic magnets and/or Kitaev materials.

cond-mat.mes-hall↗

arXiv2024

Towards a Simultaneous and Granular Identity-Expression Control in Personalized Face Generation

In human-centric content generation, the pre-trained text-to-image models struggle to produce user-wanted portrait images, which retain the identity of individuals while exhibiting diverse expressions. This paper introduces our efforts towards personalized face generation. To this end, we propose a novel multi-modal face generation framework, capable of simultaneous identity-expression control and more fine-grained expression synthesis. Our expression control is so sophisticated that it can be specialized by the fine-grained emotional vocabulary. We devise a novel diffusion model that can undertake the task of simultaneously face swapping and reenactment. Due to the entanglement of identity and expression, it's nontrivial to separately and precisely control them in one framework, thus has not been explored yet. To overcome this, we propose several innovative designs in the conditional diffusion model, including balancing identity and expression encoder, improved midpoint sampling, and explicitly background conditioning. Extensive experiments have demonstrated the controllability and scalability of the proposed framework, in comparison with state-of-the-art text-to-image, face swapping, and face reenactment methods.

cs.CV↗

arXiv2024

AO-DETR: Anti-Overlapping DETR for X-Ray Prohibited Items Detection

Prohibited item detection in X-ray images is one of the most essential and highly effective methods widely employed in various security inspection scenarios. Considering the significant overlapping phenomenon in X-ray prohibited item images, we propose an Anti-Overlapping DETR (AO-DETR) based on one of the state-of-the-art general object detectors, DINO. Specifically, to address the feature coupling issue caused by overlapping phenomena, we introduce the Category-Specific One-to-One Assignment (CSA) strategy to constrain category-specific object queries in predicting prohibited items of fixed categories, which can enhance their ability to extract features specific to prohibited items of a particular category from the overlapping foreground-background features. To address the edge blurring problem caused by overlapping phenomena, we propose the Look Forward Densely (LFD) scheme, which improves the localization accuracy of reference boxes in mid-to-high-level decoder layers and enhances the ability to locate blurry edges of the final layer. Similar to DINO, our AO-DETR provides two different versions with distinct backbones, tailored to meet diverse application requirements. Extensive experiments on the PIXray and OPIXray datasets demonstrate that the proposed method surpasses the state-of-the-art object detectors, indicating its potential applications in the field of prohibited item detection. The source code will be released at https://github.com/Limingyuan001/AO-DETR-test.

cs.CV↗

arXiv2024

Topological transverse spin transport in a canted antiferromagnet/heavy metal heterostructure

We theoretically study the conditions under which a spin Nernst effect - a transverse spin current induced by an applied temperature gradient - can occur in a canted antiferromagnetic insulator, such as ${\rm LaFeO_3}$ and other materials of the same family. The spin Nernst effect may provide a microscopic mechanism for an experimentally observed anomalous thermovoltage in ${\rm LaFeO_3}$/Pt heterostructures, where spin is transferred across the insulator/metal interface when a temperature gradient is applied to ${\rm LaFeO_3}$ parallel to the interface [W. Lin ${\it et \; al}$, Nat. Phys. ${\bf 18}$, 800 (2022)]. We find that ${\rm LaFeO_3}$ exhibits a topological spin Nernst effect when inversion symmetry is broken on the axes parallel to both the applied temperature gradient and the direction of spin transport, which can result in a spin injection across the insulator/metal interface. Our work provides a general derivation of a symmetry-breaking-induced spin Nernst effect, which may open a path to engineering a finite spin Nernst effect in systems where it would otherwise not arise.

cond-mat.mes-hall↗

arXiv2023

Time-inconsistent Linear Quadratic Optimal Control Problem for Forward-Backward Stochastic Differential Equations

We study the time-inconsistent linear quadratic optimal control problem for forward-backward stochastic differential equations with potentially indefinite cost weighting matrices for both the state and the control variables. Our research makes two contributions. Firstly, we introduce a novel type of Riccati equation system with parameters and constraint conditions, known as the generalized equilibrium Riccati equation. This equation system offers a comprehensive solution for the closed-loop equilibrium strategy of the problem at hand. Secondly, we establish the well-posedness of the generalized equilibrium Riccati equation for the one-dimensional case, provided certain conditions are met.

math.OC↗

arXiv2023

Upper branch thermal Hall effect in quantum paramagnets

Inspired by the persistent thermal Hall effects at finite temperatures in various quantum paramagnets, we explore the origin of the thermal Hall effects from the perspective of the upper branch parts by invoking the dispersive and twisted crystal field excitations. It is shown that the upper branches of the local energy levels could hybridize and form the dispersive bands. The observation is that, upon the time-reversal symmetry breaking by the magnetic fields, these upper branch bands could acquire a Berry curvature distribution and contribute to the thermal Hall effect in the paramagnetic regime. As a proof of principle, we consider the setting on the kagomé lattice with one ground state singlet and an excited doublet, and show this is indeed possible. We expect this effect to be universal and has no strong connection with the underlying lattice. Although the thermal Hall signal can be contributed from other sources such as phonons and their scattering in the actual materials, we discuss the application to the Mott systems with the large local Hilbert spaces.

cond-mat.str-el↗

arXiv2023

Double exchange, itinerant ferromagnetism and topological Hall effect in moiré heterobilayer

Motivated by the recent experiments and the wide tunability on the MoTe$_2$/WSe$_2$ moiré heterobilayer, we consider a physical model to explore the underlying physics for the interplay between the itinerant carriers and the local magnetic moments. In the regime where the MoTe$_2$ is tuned to a triangular lattice Mott insulator and the WSe$_2$ layer is doped with the itinerant holes, we invoke the itinerant ferromagnetism from the double exchange mechanism for the itinerant holes on the WSe$_2$ layer and the local moments on the MoTe$_2$ layer. Together with the antiferromagnetic exchange on the MoTe$_2$ layer, the itinerant ferromagnetism generates the scalar spin chirality distribution in the system. We further point out the presence of spin-assisted hopping in addition to the Kondo coupling between the local spin and the itinerant holes, and demonstrate the topological Hall effect for the itinerant electrons in the presence of the non-collinear spin configurations. This work may improve our understanding of the correlated moiré systems and inspire further experimental efforts.

cond-mat.str-el↗

arXiv2023

FlowFace++: Explicit Semantic Flow-supervised End-to-End Face Swapping

This work proposes a novel face-swapping framework FlowFace++, utilizing explicit semantic flow supervision and end-to-end architecture to facilitate shape-aware face-swapping. Specifically, our work pretrains a facial shape discriminator to supervise the face swapping network. The discriminator is shape-aware and relies on a semantic flow-guided operation to explicitly calculate the shape discrepancies between the target and source faces, thus optimizing the face swapping network to generate highly realistic results. The face swapping network is a stack of a pre-trained face-masked autoencoder (MAE), a cross-attention fusion module, and a convolutional decoder. The MAE provides a fine-grained facial image representation space, which is unified for the target and source faces and thus facilitates final realistic results. The cross-attention fusion module carries out the source-to-target face swapping in a fine-grained latent space while preserving other attributes of the target image (e.g. expression, head pose, hair, background, illumination, etc). Lastly, the convolutional decoder further synthesizes the swapping results according to the face-swapping latent embedding from the cross-attention fusion module. Extensive quantitative and qualitative experiments on in-the-wild faces demonstrate that our FlowFace++ outperforms the state-of-the-art significantly, particularly while the source face is obstructed by uneven lighting or angle offset.

cs.CV↗

arXiv2023

Multi-modal Facial Affective Analysis based on Masked Autoencoder

Human affective behavior analysis focuses on analyzing human expressions or other behaviors to enhance the understanding of human psychology. The CVPR 2023 Competition on Affective Behavior Analysis in-the-wild (ABAW) is dedicated to providing high-quality and large-scale Aff-wild2 for the recognition of commonly used emotion representations, such as Action Units (AU), basic expression categories(EXPR), and Valence-Arousal (VA). The competition is committed to making significant strides in improving the accuracy and practicality of affective analysis research in real-world scenarios. In this paper, we introduce our submission to the CVPR 2023: ABAW5. Our approach involves several key components. First, we utilize the visual information from a Masked Autoencoder(MAE) model that has been pre-trained on a large-scale face image dataset in a self-supervised manner. Next, we finetune the MAE encoder on the image frames from the Aff-wild2 for AU, EXPR and VA tasks, which can be regarded as a static and uni-modal training. Additionally, we leverage the multi-modal and temporal information from the videos and implement a transformer-based framework to fuse the multi-modal features. Our approach achieves impressive results in the ABAW5 competition, with an average F1 score of 55.49\% and 41.21\% in the AU and EXPR tracks, respectively, and an average CCC of 0.6372 in the VA track. Our approach ranks first in the EXPR and AU tracks, and second in the VA track. Extensive quantitative experiments and ablation studies demonstrate the effectiveness of our proposed method.

cs.CV↗

arXiv2022

FlowFace: Semantic Flow-guided Shape-aware Face Swapping

In this work, we propose a semantic flow-guided two-stage framework for shape-aware face swapping, namely FlowFace. Unlike most previous methods that focus on transferring the source inner facial features but neglect facial contours, our FlowFace can transfer both of them to a target face, thus leading to more realistic face swapping. Concretely, our FlowFace consists of a face reshaping network and a face swapping network. The face reshaping network addresses the shape outline differences between the source and target faces. It first estimates a semantic flow (i.e., face shape differences) between the source and the target face, and then explicitly warps the target face shape with the estimated semantic flow. After reshaping, the face swapping network generates inner facial features that exhibit the identity of the source face. We employ a pre-trained face masked autoencoder (MAE) to extract facial features from both the source face and the target face. In contrast to previous methods that use identity embedding to preserve identity information, the features extracted by our encoder can better capture facial appearances and identity information. Then, we develop a cross-attention fusion module to adaptively fuse inner facial features from the source face with the target facial attributes, thus leading to better identity preservation. Extensive quantitative and qualitative experiments on in-the-wild faces demonstrate that our FlowFace outperforms the state-of-the-art significantly.

cs.CV↗

arXiv2022

Facial Action Unit Detection and Intensity Estimation from Self-supervised Representation

As a fine-grained and local expression behavior measurement, facial action unit (FAU) analysis (e.g., detection and intensity estimation) has been documented for its time-consuming, labor-intensive, and error-prone annotation. Thus a long-standing challenge of FAU analysis arises from the data scarcity of manual annotations, limiting the generalization ability of trained models to a large extent. Amounts of previous works have made efforts to alleviate this issue via semi/weakly supervised methods and extra auxiliary information. However, these methods still require domain knowledge and have not yet avoided the high dependency on data annotation. This paper introduces a robust facial representation model MAE-Face for AU analysis. Using masked autoencoding as the self-supervised pre-training approach, MAE-Face first learns a high-capacity model from a feasible collection of face images without additional data annotations. Then after being fine-tuned on AU datasets, MAE-Face exhibits convincing performance for both AU detection and AU intensity estimation, achieving a new state-of-the-art on nearly all the evaluation results. Further investigation shows that MAE-Face achieves decent performance even when fine-tuned on only 1\% of the AU training set, strongly proving its robustness and generalization performance.

cs.CV↗

arXiv2022

Transformer-based Multimodal Information Fusion for Facial Expression Analysis

Human affective behavior analysis has received much attention in human-computer interaction (HCI). In this paper, we introduce our submission to the CVPR 2022 Competition on Affective Behavior Analysis in-the-wild (ABAW). To fully exploit affective knowledge from multiple views, we utilize the multimodal features of spoken words, speech prosody, and facial expression, which are extracted from the video clips in the Aff-Wild2 dataset. Based on these features, we propose a unified transformer-based multimodal framework for Action Unit detection and also expression recognition. Specifically, the static vision feature is first encoded from the current frame image. At the same time, we clip its adjacent frames by a sliding window and extract three kinds of multimodal features from the sequence of images, audio, and text. Then, we introduce a transformer-based fusion module that integrates the static vision features and the dynamic multimodal features. The cross-attention module in the fusion module makes the output integrated features focus on the crucial parts that facilitate the downstream detection tasks. We also leverage some data balancing techniques, data augmentation techniques, and postprocessing methods to further improve the model performance. In the official test of ABAW3 Competition, our model ranks first in the EXPR and AU tracks. The extensive quantitative evaluations, as well as ablation studies on the Aff-Wild2 dataset, prove the effectiveness of our proposed method.

cs.CV↗

arXiv2021

Enhancing Identification of Structure Function of Academic Articles Using Contextual Information

With the enrichment of literature resources, researchers are facing the growing problem of information explosion and knowledge overload. To help scholars retrieve literature and acquire knowledge successfully, clarifying the semantic structure of the content in academic literature has become the essential research question. In the research on identifying the structure function of chapters in academic articles, only a few studies used the deep learning model and explored the optimization for feature input. This limits the application, optimization potential of deep learning models for the research task. This paper took articles of the ACL conference as the corpus. We employ the traditional machine learning models and deep learning models to construct the classifiers based on various feature input. Experimental results show that (1) Compared with the chapter content, the chapter title is more conducive to identifying the structure function of academic articles. (2) Relative position is a valuable feature for building traditional models. (3) Inspired by (2), this paper further introduces contextual information into the deep learning models and achieved significant results. Meanwhile, our models show good migration ability in the open test containing 200 sampled non-training samples. We also annotated the ACL main conference papers in recent five years based on the best practice performing models and performed a time series analysis of the overall corpus. This work explores and summarizes the practical features and models for this task through multiple comparative experiments and provides a reference for related text classification tasks. Finally, we indicate the limitations and shortcomings of the current model and the direction of further optimization.

cs.CL↗

arXiv2021

Comb-based photonic neural population for parallel and nonlinear processing

It is believed that neural information representation and processing relies on the neural population instead of a single neuron. In neuromorphic photonics, photonic neurons in the form of nonlinear responses have been extensively studied in single devices and temporal nodes. However, to construct a photonic neural population (PNP), the process of scaling up and massive interconnections remain challenging considering the physical complexity and response latency. Here, we propose a comb-based PNP interconnected by carrier coupling with superior scalability. Two unique properties of neural population are theoretically and experimentally demonstrated in the comb-based PNP, including nonlinear response curves and population activities coding. A classification task of three input patterns with dual radio-frequency (RF) tones is successfully implemented in a real-time manner, which manifests the comb-based PNP can make effective use of the ultra-broad bandwidth of photonics for parallel and nonlinear processing.

cs.ET↗

arXiv2021

Dzyaloshinskii-Moriya Induced Topological Magnon-Phonon Hybridization in 2D Antiferromagnetic Insulators with Tunable Chern Numbers

We theoretically study magnon-phonon hybrid excitations (magnon-polarons) in two-dimensional antiferromagnets on a honeycomb lattice. With an in-plane Dzyaloshinskii-Moriya interaction (DMI) allowed from mirror symmetry breaking from phonons, we find non-trivial Berry curvature around the anti-crossing rings among magnon and both optical and acoustic phonon bands, which gives rise to finite Chern numbers. We show that the Chern numbers of the magnon-polaron bands can be manipulated by changing the magnetic field direction or strength. We evaluate the thermal Hall conductivity reflecting the non-trivial Berry curvatures of magnon-polarons and propose a valley Hall effect resulting from spin-induced chiral phonons as a possible experimental signature. Our study complements prior work on magnon-phonon hybridized systems without optical phonons and suggests possible applications in spin caloritronics with topological magnons and chiral phonons.

cond-mat.mes-hall↗

arXiv2021

Intrinsic Magnon Nernst Effects in Pyrochlore Iridate Thin Films

We theoretically study the magnon spin thermal transport using a strong coupling approach in pyrochlore iridate trilayer thin films grown along the [111] direction. As a result of the Dzyaloshinskii-Moriya interaction (DMI), the spin configuration of the ground state is an all-in/all-out ordering on neighboring tetrahedra of the pyrochlore lattice. In such a state, the system has an inversion symmetry and a Nernst-type thermal spin current response is well defined. We calculate the temperature dependence of the magnon Nernst response with respect to the magnon band topology controlled by the spin-orbit coupling parameters and observe topologically protected chiral edge modes over a range of parameters. Our study complements prior work on the magnon thermal Hall effect in thin-film pyrochlore iridates and suggests that the [111] grown thin-film pyrochlore iridates are a promising candidate for thermal spin transport and spin caloritronic devices.

cond-mat.mtrl-sci↗

arXiv2019

Longitudinal Spin Seebeck effect in Pyrochlore Iridates with Bulk and Interfacial Dzyaloshinskii-Moriya interaction

The longitudinal spin-Seebeck effect (SSE) in magnetic insulator$|$non-magnetic metal heterostructures has been theoretically studied primarily with the assumption of an isotropic interfacial exchange coupling. Here, we present a general theory of the SSE in the case of an antisymmetric Dzyaloshinskii-Moriya interaction (DMI) at the interface, in addition to the usual Heisenberg form. We numerically evaluate the dependence of the spin current on the temperature and bulk DMI using a pyrochlore iridate as a model insulator with all-in all-out (AIAO) ground state configuration. We also compare the results of different crystalline surfaces arising from different crystalline orientations and conclude that the relative angles between the interfacial moments and Dzyaloshinskii-Moriya vectors play a significant role in the spin transfer. Our work extends the theory of the SSE by including the anisotropic nature of the interfacial Dzyaloshinskii-Moriya exchange interaction in magnetic insulator$|$non-magnetic metal heterostructures and can suggest possible materials to optimize the interfacial spin transfer in spintronic devices.

cond-mat.mes-hall↗

arXiv2018

Analog-to-digital conversion revolutionized by deep learning

As the bridge between the analog world and digital computers, analog-to-digital converters are generally used in modern information systems such as radar, surveillance, and communications. For the configuration of analog-to-digital converters in future high-frequency broadband systems, we introduce a revolutionary architecture that adopts deep learning technology to overcome tradeoffs between bandwidth, sampling rate, and accuracy. A photonic front-end provides broadband capability for direct sampling and speed multiplication. Trained deep neural networks learn the patterns of system defects, maintaining high accuracy of quantized data in a succinct and adaptive manner. Based on numerical and experimental demonstrations, we show that the proposed architecture outperforms state-of-the-art analog-to-digital converters, confirming the potential of our approach in future analog-to-digital converter design and performance enhancement of future information systems.

eess.SP↗