arXiv subjects

Yue Shi

Publications and source records attributed to Yue Shi.

At least 55 records · Page 3Linked to original sources

arXiv2024

SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis

Achieving high synchronization in the synthesis of realistic, speech-driven talking head videos presents a significant challenge. Traditional Generative Adversarial Networks (GAN) struggle to maintain consistent facial identity, while Neural Radiance Fields (NeRF) methods, although they can address this issue, often produce mismatched lip movements, inadequate facial expressions, and unstable head poses. A lifelike talking head requires synchronized coordination of subject identity, lip movements, facial expressions, and head poses. The absence of these synchronizations is a fundamental flaw, leading to unrealistic and artificial outcomes. To address the critical issue of synchronization, identified as the "devil" in creating realistic talking heads, we introduce SyncTalk. This NeRF-based method effectively maintains subject identity, enhancing synchronization and realism in talking head synthesis. SyncTalk employs a Face-Sync Controller to align lip movements with speech and innovatively uses a 3D facial blendshape model to capture accurate facial expressions. Our Head-Sync Stabilizer optimizes head poses, achieving more natural head movements. The Portrait-Sync Generator restores hair details and blends the generated head with the torso for a seamless visual experience. Extensive experiments and user studies demonstrate that SyncTalk outperforms state-of-the-art methods in synchronization and realism. We recommend watching the supplementary video: https://ziqiaopeng.github.io/synctalk

cs.CV↗

arXiv2024

Deep Equilibrium Diffusion Restoration with Parallel Sampling

Diffusion model-based image restoration (IR) aims to use diffusion models to recover high-quality (HQ) images from degraded images, achieving promising performance. Due to the inherent property of diffusion models, most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs. Moreover, such long sampling chains hinder understanding the relationship between inputs and restoration results since it is hard to compute the gradients in the whole chains. In this work, we aim to rethink the diffusion model-based IR models through a different perspective, i.e., a deep equilibrium (DEQ) fixed point system, called DeqIR. Specifically, we derive an analytical solution by modeling the entire sampling chain in these IR models as a joint multivariate fixed point system. Based on the analytical solution, we can conduct parallel sampling and restore HQ images without training. Furthermore, we compute fast gradients via DEQ inversion and found that initialization optimization can boost image quality and control the generation direction. Extensive experiments on benchmarks demonstrate the effectiveness of our method on typical IR tasks and real-world settings.

cs.CV↗

arXiv2024

A program for 3D nuclear static and time-dependent density-functional theory with full Skyrme energy density functional: HIT3D

This work presents a computer program that performs symmetry-unrestricted 3D nuclear time-dependent density function theory (DFT) calculations. The program features the augmented Lagrangian constraint in the static calculation. This allows for the calculation of the potential energy surface. In addition, the code includes the full energy density functionals derived from the Skyrme interaction, meaning that the time-even and time-odd tensor parts are included for the time-dependent calculations. The results of the hit3d code are carefully compared with the Sky3D and Ev8 programs. The testing cases include unconstrained DFT calculations for doubly magic nuclei, the constrained DFT + BCS calculations for medium-heavy nucleus 110Mo, and the dynamic applications for harmonic vibration and nuclear reactions.

nucl-th↗

arXiv2024

Absence of Weyl nodes in EuCd$_2$As$_2$ revealed by the carrier density dependence of the anomalous Hall effect

The antiferromagnetic layered compound EuCd$_2$As$_2$ is widely considered as a leading candidate of ideal Weyl semimetal, featuring a single pair of Weyl nodes in its field-induced ferromagnetic (FM) state. Nevertheless, this view has recently been challenged by an optical spectroscopy study, which suggests that it is a magnetic semiconductor. In this study, we have successfully synthesized highly insulating EuCd$_2$As$_2$ crystals with carrier density reaching as low as $2\times 10^{15}$ $\text{cm}^{-3}$. The magneto-transport measurements revealed a progressive decrease of the anomalous Hall conductivity (AHC) by several orders of magnitude as the carrier density decreases. This behavior contradicts with what is expected from the intrinsic AHC generated by the Weyl points, which is independent of carrier density as the Fermi level approaches the charge neutrality point. In contrast, the scaling relationship between AHC and longitudinal conductivity aligns with the characteristics of variable range hopping insulators. Our results suggest that EuCd$_2$As$_2$ is a magnetic semiconductor rather than a topological Weyl semimetal.

cond-mat.mtrl-sci↗

arXiv2024

Quantum Oscillations Measurement of the Heavy Electron Mass near the van Hove Singularity in a Kagome Metal

Kagome metals with the Fermi energy tuned near the van Hove singularities (vHss) have shown to host exotic phases including unconventional superconductivity and a chiral flux phase arising from a charge density wave. However, most quantum oscillations studies of the electronic structure of kagome metals focus on compounds which electronically or magnetically order, obscuring the unperturbed vHs. Here we present quantum oscillation measurements of YV$_6$Sn$_6$ which contains a pristine kagome lattice free from long range order. We discovered quantum oscillations corresponding to a large orbit ($\approx$70% of the Brillouin Zone area) with the heaviest mass ever observed in vanadium based kagome metals ($\approx3.3 m_e$), consistent with a Fermi pocket whose Fermi level is near the vHs. Comparing with first principles calculations suggests that the effective mass of this pocket is highly sensitive to the position of Fermi level. Our study establishes the enhanced density of states associated with a vHs in a kagome metal, allowing further insight into a potential driving mechanism for the unconventional electronic orderings in this class of materials.

cond-mat.str-el↗

arXiv2023

Spectral Evidence for Local-Moment Ferromagnetism in van der Waals Metals Fe$_3$GaTe$_2$ and Fe$_3$GeTe$_2$

Magnetism in two-dimensional (2D) materials has attracted considerable attention recently for both fundamental understanding of magnetism and their tunability towards device applications. The isostructural Fe$_3$GeTe$_2$ and Fe$_3$GaTe$_2$ are two members of the Fe-based van der Waals (vdW) ferromagnet family, but exhibit very different Curie temperatures (T$_C$) of 210 K and 360 K, respectively. Here, by using angle-resolved photoemission spectroscopy and density functional theory, we systematically compare the electronic structures of the two compounds. Qualitative similarities in the Fermi surface can be found between the two compounds, with expanded hole pockets in Fe$_3$GaTe$_2$ suggesting additional hole carriers compared to Fe$_3$GeTe$_2$. Interestingly, we observe no band shift in Fe$_3$GaTe$_2$ across its T$_C$ of 360 K, compared to a small shift in Fe$_3$GeTe$_2$ across its T$_C$ of 210 K. The weak temperature-dependent evolution strongly deviates from the expectations of an itinerant Stoner mechanism. Our results suggest that itinerant electrons have minimal contributions to the enhancement of T$_C$ in Fe$_3$GaTe$_2$ compared to Fe$_3$GeTe$_2$, and that the nature of ferromagnetism in these Fe-based vdW ferromagnets must be understood with considerations of the electron correlations.

cond-mat.str-el↗

arXiv2023

Non-Fermi liquid behavior in a correlated flatband pyrochlore lattice

Electronic correlation effects are manifested in quantum materials when either the onsite Coulomb repulsion is large or the electron kinetic energy is small. The former is the dominant effect in the cuprate superconductors or heavy fermion systems while the latter in twisted bilayer graphene or geometrically frustrated metals. However, the simultaneous cooperation of both effects in the same quantum material--the design principle to produce a correlated topological flat bands pinned at the Fermi level--remains rare. Here, using angle-resolved photoemission spectroscopy, we report the observation of a flat band at the Fermi level in a 3$d$ pyrochlore metal CuV$_2$S$_4$. From a combination of first-principles calculations and slave-spin calculations, we understand the origin of this band to be a destructive quantum-interference effect associated with the V pyrochlore sublattice and further renormalization to the Fermi level by electron interactions in the partially filled V $t_{2g}$ orbitals. As a result, we find transport behavior that indicates a deviation from Fermi-liquid behavior as well as a large Sommerfeld coefficient. Our work demonstrates the pathway into correlated topology by constructing and pinning correlated flat bands near the Fermi level out of a pure $d$-electron system by the combined cooperation of local Coulomb interactions and geometric frustration in a pyrochlore lattice system.

cond-mat.str-el↗

arXiv2023

Isoscalar Giant Monopole Resonance in Mo-100: TDHF calculations and ground state deformation

The isoscalar giant monopole resonance in molybdenum-100 has been measured in recent alpha scattering experiments and its strength function extracted. By performing time-dependent Hartree-Fock calculations with Skyrme effective interactions, we are able to reproduce the gross features of the observed strength. The details of the structure are found to be dependent on the ground state shape. We use a measure of fit to determine which ground state shape correlates with the observed strength and find evidence for triaxiality in this nucleus.

nucl-th↗

arXiv2023

The bionic neural network for external simulation of human locomotor system

Muscle forces and joint kinematics estimated with musculoskeletal (MSK) modeling techniques offer useful metrics describing movement quality. Model-based computational MSK models can interpret the dynamic interaction between the neural drive to muscles, muscle dynamics, body and joint kinematics, and kinetics. Still, such a set of solutions suffers from high computational time and muscle recruitment problems, especially in complex modeling. In recent years, data-driven methods have emerged as a promising alternative due to the benefits of flexibility and adaptability. However, a large amount of labeled training data is not easy to be acquired. This paper proposes a physics-informed deep learning method based on MSK modeling to predict joint motion and muscle forces. The MSK model is embedded into the neural network as an ordinary differential equation (ODE) loss function with physiological parameters of muscle activation dynamics and muscle contraction dynamics to be identified. These parameters are automatically estimated during the training process which guides the prediction of muscle forces combined with the MSK forward dynamics model. Experimental validations on two groups of data, including one benchmark dataset and one self-collected dataset from six healthy subjects, are performed. The results demonstrate that the proposed deep learning method can effectively identify subject-specific MSK physiological parameters and the trained physics-informed forward-dynamics surrogate yields accurate motion and muscle forces predictions.

cs.LG↗

arXiv2023

The online learning architecture with edge computing for high-level control for assisting patients

The prevalence of mobility impairments due to conditions such as spinal cord injuries, strokes, and degenerative diseases is on the rise globally. Lower-limb exoskeletons have been increasingly recognized as a viable solution for enhancing mobility and rehabilitation for individuals with such impairments. However, existing exoskeleton control systems often suffer from limitations such as latency, lack of adaptability, and computational inefficiency. To address these challenges, this paper introduces a novel online adversarial learning architecture integrated with edge computing for high-level lower-limb exoskeleton control. In the proposed architecture, sensor data from the user is processed in real-time through edge computing nodes, which then interact with an online adversarial learning model. This model adapts to the user's specific needs and controls the exoskeleton with minimal latency. Experimental evaluations demonstrate significant improvements in control accuracy and adaptability, as well as enhanced quality-of-service (QoS) metrics. These findings indicate that the integration of online adversarial learning with edge computing offers a robust and efficient approach for the next generation of lower-limb exoskeleton control systems.

cs.LG↗

arXiv2023

SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces

Speech-driven 3D face animation technique, extending its applications to various multimedia fields. Previous research has generated promising realistic lip movements and facial expressions from audio signals. However, traditional regression models solely driven by data face several essential problems, such as difficulties in accessing precise labels and domain gaps between different modalities, leading to unsatisfactory results lacking precision and coherence. To enhance the visual accuracy of generated lip movement while reducing the dependence on labeled data, we propose a novel framework SelfTalk, by involving self-supervision in a cross-modals network system to learn 3D talking faces. The framework constructs a network system consisting of three modules: facial animator, speech recognizer, and lip-reading interpreter. The core of SelfTalk is a commutative training diagram that facilitates compatible features exchange among audio, text, and lip shape, enabling our models to learn the intricate connection between these factors. The proposed framework leverages the knowledge learned from the lip-reading interpreter to generate more plausible lip shapes. Extensive experiments and user studies demonstrate that our proposed approach achieves state-of-the-art performance both qualitatively and quantitatively. We recommend watching the supplementary video.

cs.CV↗

arXiv2023

FocalDreamer: Text-driven 3D Editing via Focal-fusion Assembly

While text-3D editing has made significant strides in leveraging score distillation sampling, emerging approaches still fall short in delivering separable, precise and consistent outcomes that are vital to content creation. In response, we introduce FocalDreamer, a framework that merges base shape with editable parts according to text prompts for fine-grained editing within desired regions. Specifically, equipped with geometry union and dual-path rendering, FocalDreamer assembles independent 3D parts into a complete object, tailored for convenient instance reuse and part-wise control. We propose geometric focal loss and style consistency regularization, which encourage focal fusion and congruent overall appearance. Furthermore, FocalDreamer generates high-fidelity geometry and PBR textures which are compatible with widely-used graphics engines. Extensive experiments have highlighted the superior editing capabilities of FocalDreamer in both quantitative and qualitative evaluations.

cs.CV↗

arXiv2023

Qubits on programmable geometries with a trapped-ion quantum processor

Geometry and dimensionality have played crucial roles in our understanding of the fundamental laws of nature, with examples ranging from curved space-time in general relativity to modern theories of quantum gravity. In quantum many-body systems, the entanglement structure can change if the constituents are connected differently, leading to altered bounds for correlation growth and difficulties for classical computers to simulate large systems. While a universal quantum computer can perform digital simulations, an analog-digital hybrid quantum processor offers advantages such as parallelism. Here, we engineer a class of high-dimensional Ising interactions using a linear one-dimensional (1D) ion chain with up to 8 qubits through stroboscopic sequences of commuting Hamiltonians. %with a thorough understanding of the error sources and deviation from the target Hamiltonian. In addition, we extend this method to non-commuting circuits and demonstrate the quantum XY and Heisenberg models using Floquet periodic drives with tunable symmetries. The realization of higher dimensional spin models offers new opportunities ranging from studying topological phases of matter or quantum spin glasses to future fault-tolerant quantum computation.

quant-ph↗

arXiv2023

A Physics-Informed Low-Shot Learning For sEMG-Based Estimation of Muscle Force and Joint Kinematics

Muscle force and joint kinematics estimation from surface electromyography (sEMG) are essential for real-time biomechanical analysis of the dynamic interplay among neural muscle stimulation, muscle dynamics, and kinetics. Recent advances in deep neural networks (DNNs) have shown the potential to improve biomechanical analysis in a fully automated and reproducible manner. However, the small sample nature and physical interpretability of biomechanical analysis limit the applications of DNNs. This paper presents a novel physics-informed low-shot learning method for sEMG-based estimation of muscle force and joint kinematics. This method seamlessly integrates Lagrange's equation of motion and inverse dynamic muscle model into the generative adversarial network (GAN) framework for structured feature decoding and extrapolated estimation from the small sample data. Specifically, Lagrange's equation of motion is introduced into the generative model to restrain the structured decoding of the high-level features following the laws of physics. And a physics-informed policy gradient is designed to improve the adversarial learning efficiency by rewarding the consistent physical representation of the extrapolated estimations and the physical references. Experimental validations are conducted on two scenarios (i.e. the walking trials and wrist motion trials). Results indicate that the estimations of the muscle forces and joint kinematics are unbiased compared to the physics-based inverse dynamics, which outperforms the selected benchmark methods, including physics-informed convolution neural network (PI-CNN), vallina generative adversarial network (GAN), and multi-layer extreme learning machine (ML-ELM).

eess.SP↗

arXiv2023

Reversible Non-Volatile Electronic Switching in a Near Room Temperature van der Waals Ferromagnet

The ability to reversibly toggle between two distinct states in a non-volatile method is important for information storage applications. Such devices have been realized for phase-change materials, which utilizes local heating methods to toggle between a crystalline and an amorphous state with distinct electrical properties. To expand such kind of switching between two topologically distinct phases requires non-volatile switching between two crystalline phases with distinct symmetries. Here we report the observation of reversible and non-volatile switching between two stable and closely-related crystal structures with remarkably distinct electronic structures in the near room temperature van der Waals ferromagnet Fe$_{5-δ}$GeTe$_2$. From a combination of characterization techniques we show that the switching is enabled by the ordering and disordering of an Fe site vacancy that results in distinct crystalline symmetries of the two phases that can be controlled by a thermal annealing and quenching method. Furthermore, from symmetry analysis as well as first principle calculations, we provide understanding of the key distinction in the observed electronic structures of the two phases: topological nodal lines compatible with the preserved global inversion symmetry in the site-disordered phase, and flat bands resulting from quantum destructive interference on a bipartite crystaline lattice formed by the presence of the site order as well as the lifting of the topological degeneracy due to the broken inversion symmetry in the site-ordered phase. Our work not only reveals a rich variety of quantum phases emergent in the metallic van der Waals ferromagnets due to the presence of site ordering, but also demonstrates the potential of these highly tunable two-dimensional magnets for memory and spintronics applications.

cond-mat.str-el↗

arXiv2023

A Fast Fourier Convolutional Deep Neural Network For Accurate and Explainable Discrimination Of Wheat Yellow Rust And Nitrogen Deficiency From Sentinel-2 Time-Series Data

Accurate and timely detection of plant stress is essential for yield protection, allowing better-targeted intervention strategies. Recent advances in remote sensing and deep learning have shown great potential for rapid non-invasive detection of plant stress in a fully automated and reproducible manner. However, the existing models always face several challenges: 1) computational inefficiency and the misclassifications between the different stresses with similar symptoms; and 2) the poor interpretability of the host-stress interaction. In this work, we propose a novel fast Fourier Convolutional Neural Network (FFDNN) for accurate and explainable detection of two plant stresses with similar symptoms (i.e. Wheat Yellow Rust And Nitrogen Deficiency). Specifically, unlike the existing CNN models, the main components of the proposed model include: 1) a fast Fourier convolutional block, a newly fast Fourier transformation kernel as the basic perception unit, to substitute the traditional convolutional kernel to capture both local and global responses to plant stress in various time-scale and improve computing efficiency with reduced learning parameters in Fourier domain; 2) Capsule Feature Encoder to encapsulate the extracted features into a series of vector features to represent part-to-whole relationship with the hierarchical structure of the host-stress interactions of the specific stress. In addition, in order to alleviate over-fitting, a photochemical vegetation indices-based filter is placed as pre-processing operator to remove the non-photochemical noises from the input Sentinel-2 time series.

cs.CV↗

arXiv2023

Strain-Switchable Field-Induced Superconductivity

Field-induced superconductivity is a rare phenomenon where an applied magnetic field enhances or induces superconductivity. This fascinating effect arises from a complex interplay between magnetism and superconductivity, and it offers the tantalizing technological possibility of an infinite magnetoresistance superconducting spin valve. Here, we demonstrate field-induced superconductivity at a record-high temperature of T=9K in two samples of the ferromagnetic superconductor Eu(Fe$_{0.88}$Co$_{0.12}$)$_{2}$As$_{2}$. We combine tunable uniaxial stress and applied magnetic field to shift the temperature range of the zero-resistance state between 4K and 10K. We use x-ray diffraction and spectroscopy measurements under stress and field to demonstrate that stress tuning of the nematic order and field tuning of the ferromagnetism act as independent tuning knobs of the superconductivity. Finally, DFT calculations and analysis of the Eu dipole field reveal the electromagnetic mechanism of the field-induced superconductivity.

cond-mat.supr-con↗

arXiv2023

Fermi Surface Evolution and Anomalous Hall Effect in an Ideal Type-II Weyl Semimetal

Weyl semimetals (WSMs) are three-dimensional topological materials that exhibit fascinating properties due to the presence of Weyl nodes in their band structure. However, existing WSMs discovered so far often possess multiple pairs of Weyl nodes, posing a challenge in disentangling the contributions to transport phenomena from different energy bands. To overcome this challenge, we have identified field-induced ferromagnetic MnBi$_{2-x}$Sb$_{x}$Te$_{4}$ as an ideal type-II WSM with a single pair of Weyl nodes. By employing a combination of quantum oscillations and high-field Hall measurements, we have resolved the evolution of Fermi-surface sections as the Fermi level is tuned across the charge neutrality point, precisely matching the band structure of an ideal type-II WSM. Furthermore, the anomalous Hall conductivity exhibits a heartbeat-like behavior as the Fermi level is tuned across the Weyl nodes, a unique feature previously predicted for a type-II WSM. Our findings establish MnBi$_{2-x}$Sb$_{x}$Te$_{4}$ as an ideal platform for further investigation into Weyl physics.

cond-mat.mes-hall↗