arXiv subjects

Jinlong Yang

Publications and source records attributed to Jinlong Yang.

At least 73 records · Page 4Linked to original sources

arXiv2023

Learning Disentangled Avatars with Hybrid 3D Representations

Tremendous efforts have been made to learn animatable and photorealistic human avatars. Towards this end, both explicit and implicit 3D representations are heavily studied for a holistic modeling and capture of the whole human (e.g., body, clothing, face and hair), but neither representation is an optimal choice in terms of representation efficacy since different parts of the human avatar have different modeling desiderata. For example, meshes are generally not suitable for modeling clothing and hair. Motivated by this, we present Disentangled Avatars~(DELTA), which models humans with hybrid explicit-implicit 3D representations. DELTA takes a monocular RGB video as input, and produces a human avatar with separate body and clothing/hair layers. Specifically, we demonstrate two important applications for DELTA. For the first one, we consider the disentanglement of the human body and clothing and in the second, we disentangle the face and hair. To do so, DELTA represents the body or face with an explicit mesh-based parametric 3D model and the clothing or hair with an implicit neural radiance field. To make this possible, we design an end-to-end differentiable renderer that integrates meshes into volumetric rendering, enabling DELTA to learn directly from monocular videos without any 3D supervision. Finally, we show that how these two applications can be easily combined to model full-body avatars, such that the hair, face, body and clothing can be fully disentangled yet jointly rendered. Such a disentanglement enables hair and clothing transfer to arbitrary body shapes. We empirically validate the effectiveness of DELTA's disentanglement by demonstrating its promising performance on disentangled reconstruction, virtual clothing try-on and hairstyle transfer. To facilitate future research, we also release an open-sourced pipeline for the study of hybrid human avatar modeling.

cs.CV↗

arXiv2023

Two-dimensional bimetal-embedded expanded phthalocyanine monolayers: a class of multifunctional materials with fascinating properties

The expanded phthalocyanine (EPc) single-layer sheets with double transition metals (labeled as TM2EPc, TM = Sc-Zn) are predicted to be a new class of two-dimensional (2D) metal-organic materials with a series of favorable functional properties by means of systematic first-principle calculations and molecular dynamics simulations. The strong coordination between metal and EPc substrate accounts for the excellent structural stability. Chemical bonding analysis has demonstrated the absence of TM-TM bonding. Each metal center is isolated, but connected to the organic framework by four 2c-2e TM-N σ-bonds to form an extended 2D network. Unexpectedly, it is found that the V2EPc is an antiferromagnetic metal with Dirac cone, while Cr2EPc exhibits ferromagnetic Dirac half-metallicity, which is not common in 2D materials. Excitingly, the ferromagnetic Cr2EPc and antiferromagnetic Mn2- and Fe2-EPc have high magnetic transition temperatures of 223, 217, and 325 K, respectively, which are crucial for the practical applications of spintronics. Cr2EPc can maintain the Dirac half-metallicity under -6 % ~ 2 % biaxial strains, and Fe2EPc can transform from semiconductor to half-metal by applying -6 % ~ -10 % compressive strains. Additionally, the TM2EPc monolayers exhibit a full response to visible light and some materials have strong absorption in the ultraviolet and infrared regions in addition to visible light, showing extraordinary solar light-harvesting ability. Notably, the designed type-II heterojunctions Fe2EPc/SnC, Co2EPc/GeS, and Ni2EPc/2H-WSe2 have high power conversion efficiency (PCE > 15%), especially the PCE of Ni2EPc/2H-WSe2 reaches 25.19%, which has great potential in solar cell applications. All these desired properties render 2D TM2EPc monolayers promising candidates for future applications in nanoelectronics, spintronics,optoelectronics, and photovoltaic devices.

cond-mat.mtrl-sci↗

arXiv2023

BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion

We show, for the first time, that neural networks trained only on synthetic data achieve state-of-the-art accuracy on the problem of 3D human pose and shape (HPS) estimation from real images. Previous synthetic datasets have been small, unrealistic, or lacked realistic clothing. Achieving sufficient realism is non-trivial and we show how to do this for full bodies in motion. Specifically, our BEDLAM dataset contains monocular RGB videos with ground-truth 3D bodies in SMPL-X format. It includes a diversity of body shapes, motions, skin tones, hair, and clothing. The clothing is realistically simulated on the moving bodies using commercial clothing physics simulation. We render varying numbers of people in realistic scenes with varied lighting and camera motions. We then train various HPS regressors using BEDLAM and achieve state-of-the-art accuracy on real-image benchmarks despite training with synthetic data. We use BEDLAM to gain insights into what model design choices are important for accuracy. With good synthetic training data, we find that a basic method like HMR approaches the accuracy of the current SOTA method (CLIFF). BEDLAM is useful for a variety of tasks and all images, ground truth bodies, 3D clothing, support code, and more are available for research purposes. Additionally, we provide detailed information about our synthetic data generation pipeline, enabling others to generate their own datasets. See the project page: https://bedlam.is.tue.mpg.de/.

cs.CV↗

arXiv2023

AG3D: Learning to Generate 3D Avatars from 2D Image Collections

While progress in 2D generative models of human appearance has been rapid, many applications require 3D avatars that can be animated and rendered. Unfortunately, most existing methods for learning generative models of 3D humans with diverse shape and appearance require 3D training data, which is limited and expensive to acquire. The key to progress is hence to learn generative models of 3D avatars from abundant unstructured 2D image collections. However, learning realistic and complete 3D appearance and geometry in this under-constrained setting remains challenging, especially in the presence of loose clothing such as dresses. In this paper, we propose a new adversarial generative model of realistic 3D people from 2D images. Our method captures shape and deformation of the body and loose clothing by adopting a holistic 3D generator and integrating an efficient and flexible articulation module. To improve realism, we train our model using multiple discriminators while also integrating geometric cues in the form of predicted 2D normal maps. We experimentally find that our method outperforms previous 3D- and articulation-aware methods in terms of geometry and appearance. We validate the effectiveness of our model and the importance of each component via systematic ablation studies.

cs.CV↗

arXiv2023

ECON: Explicit Clothed humans Optimized via Normal integration

The combination of deep learning, artist-curated scans, and Implicit Functions (IF), is enabling the creation of detailed, clothed, 3D humans from images. However, existing methods are far from perfect. IF-based methods recover free-form geometry, but produce disembodied limbs or degenerate shapes for novel poses or clothes. To increase robustness for these cases, existing work uses an explicit parametric body model to constrain surface reconstruction, but this limits the recovery of free-form surfaces such as loose clothing that deviates from the body. What we want is a method that combines the best properties of implicit representation and explicit body regularization. To this end, we make two key observations: (1) current networks are better at inferring detailed 2D maps than full-3D surfaces, and (2) a parametric model can be seen as a "canvas" for stitching together detailed surface patches. Based on these, our method, ECON, has three main steps: (1) It infers detailed 2D normal maps for the front and back side of a clothed person. (2) From these, it recovers 2.5D front and back surfaces, called d-BiNI, that are equally detailed, yet incomplete, and registers these w.r.t. each other with the help of a SMPL-X body mesh recovered from the image. (3) It "inpaints" the missing geometry between d-BiNI surfaces. If the face and hands are noisy, they can optionally be replaced with the ones of SMPL-X. As a result, ECON infers high-fidelity 3D humans even in loose clothes and challenging poses. This goes beyond previous methods, according to the quantitative evaluation on the CAPE and Renderpeople datasets. Perceptual studies also show that ECON's perceived realism is better by a large margin. Code and models are available for research purposes at econ.is.tue.mpg.de

cs.CV↗

arXiv2023

Quantum Neural Network Inspired Hardware Adaptable Ansatz for Efficient Quantum Simulation of Chemical Systems

The variational quantum eigensolver is a promising way to solve the Schrödinger equation on a noisy intermediate-scale quantum (NISQ) computer, while its success relies on a well-designed wavefunction ansatz. Compared to physically motivated ansatzes, hardware heuristic ansatzes usually lead to a shallower circuit, but it may still be too deep for an NISQ device. Inspired by the quantum neural network, we propose a new hardware heuristic ansatz where the circuit depth can be significantly reduced by introducing ancilla qubits, which makes a practical simulation of a chemical reaction with more than 20 atoms feasible on a currently available quantum computer. More importantly, the expressibility of this new ansatz can be improved by increasing either the depth or the width of the circuit, which makes it adaptable to different hardware environments. These results open a new avenue to develop practical applications of quantum computation in the NISQ era.

quant-ph↗

arXiv2023

Quantum circuit matrix product state ansatz for large-scale simulations of molecules

As in the density matrix renormalization group (DMRG) method, approximating many-body wave function of electrons using a matrix product state (MPS) is a promising way to solve electronic structure problems. The expressibility of an MPS is determined by the size of the matrices or in other words the bond dimension, which unfortunately should be very large in many cases. In this study, we propose to calculate the ground state energies of molecular systems by variationally optimizing quantum circuit MPS (QCMPS) with a relatively small number of qubits. It is demonstrated that with carefully chosen circuit structure and orbital localization scheme, QCMPS can reach a similar accuracy as that achieved in DMRG with an exponentially large bond dimension. QCMPS simulation of a linear molecule with 50 orbitals can reach the chemical accuracy using only 6 qubits at a moderate circuit depth. These results suggest that QCMPS is a promising wave function ansatz in the variational quantum eigensolver algorithm for molecular systems.

quant-ph↗

arXiv2022

Capturing and Animation of Body and Clothing from Monocular Video

While recent work has shown progress on extracting clothed 3D human avatars from a single image, video, or a set of 3D scans, several limitations remain. Most methods use a holistic representation to jointly model the body and clothing, which means that the clothing and body cannot be separated for applications like virtual try-on. Other methods separately model the body and clothing, but they require training from a large set of 3D clothed human meshes obtained from 3D/4D scanners or physics simulations. Our insight is that the body and clothing have different modeling requirements. While the body is well represented by a mesh-based parametric 3D model, implicit representations and neural radiance fields are better suited to capturing the large variety in shape and appearance present in clothing. Building on this insight, we propose SCARF (Segmented Clothed Avatar Radiance Field), a hybrid model combining a mesh-based body with a neural radiance field. Integrating the mesh into the volumetric rendering in combination with a differentiable rasterizer enables us to optimize SCARF directly from monocular videos, without any 3D supervision. The hybrid modeling enables SCARF to (i) animate the clothed body avatar by changing body poses (including hand articulation and facial expressions), (ii) synthesize novel views of the avatar, and (iii) transfer clothing between avatars in virtual try-on applications. We demonstrate that SCARF reconstructs clothing with higher visual quality than existing methods, that the clothing deforms with changing body pose and body shape, and that clothing can be successfully transferred between avatars of different subjects. The code and models are available at https://github.com/YadiraF/SCARF.

cs.CV↗

arXiv2022

A standing Leidenfrost drop with Sufi-whirling

The mobility of Leidenfrost drop has been exploited for the manipulation of drop motions. In the classical model, the Leidenfrost drop was levitated by a vapor cushion, in the absence of touch to the surface. Here we report a standing Leidenfrost state on a heated hydrophobic surface where drop stands on the surface with partial adhesion and further self-rotates like Sufi-whirling. To elucidate this new phenomenon, we imaged the evolution of the partial adhesion, the inner circulation, and the ellipsoidal rotation of the drop. The stable partial adhesion is accompanied by thermal and mechanical equilibrium, and further drives the development of the drop rotation.

physics.flu-dyn↗

arXiv2022

High-Order Coupled Fully-Connected Tensor Network Decomposition for Hyperspectral Image Super-Resolution

Hyperspectral image super-resolution addresses the problem of fusing a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI) to produce a high-resolution hyperspectral image (HR-HSI). Tensor analysis has been proven to be an efficient method for hyperspectral image processing. However, the existing tensor-based methods of hyperspectral image super-resolution like the tensor train and tensor ring decomposition only establish an operation between adjacent two factors and are highly sensitive to the permutation of tensor modes, leading to an inadequate and inflexible representation. In this paper, we propose a novel method for hyperspectral image super-resolution by utilizing the specific properties of high-order tensors in fully-connected tensor network decomposition. The proposed method first tensorizes the target HR-HSI into a high-order tensor that has multiscale spatial structures. Then, a coupled fully-connected tensor network decomposition model is proposed to fuse the corresponding high-order tensors of LR-HSI and HR-MSI. Moreover, a weighted-graph regularization is imposed on the spectral core tensors to preserve spectral information. In the proposed model, the superiorities of the fully-connected tensor network decomposition lie in the outstanding capability for characterizing adequately the intrinsic correlations between any two modes of tensors and the essential invariance for transposition. Experimental results on three data sets show the effectiveness of the proposed approach as compared to other hyperspectral image super-resolution methods.

math.NA↗

arXiv2022

Neural Point-based Shape Modeling of Humans in Challenging Clothing

Parametric 3D body models like SMPL only represent minimally-clothed people and are hard to extend to clothing because they have a fixed mesh topology and resolution. To address these limitations, recent work uses implicit surfaces or point clouds to model clothed bodies. While not limited by topology, such methods still struggle to model clothing that deviates significantly from the body, such as skirts and dresses. This is because they rely on the body to canonicalize the clothed surface by reposing it to a reference shape. Unfortunately, this process is poorly defined when clothing is far from the body. Additionally, they use linear blend skinning to pose the body and the skinning weights are tied to the underlying body parts. In contrast, we model the clothing deformation in a local coordinate space without canonicalization. We also relax the skinning weights to let multiple body parts influence the surface. Specifically, we extend point-based methods with a coarse stage, that replaces canonicalization with a learned pose-independent "coarse shape" that can capture the rough surface geometry of clothing like skirts. We then refine this using a network that infers the linear blend skinning weights and pose dependent displacements from the coarse representation. The approach works well for garments that both conform to, and deviate from, the body. We demonstrate the usefulness of our approach by learning person-specific avatars from examples and then show how they can be animated in new poses and motions. We also show that the method can learn directly from raw scans with missing data, greatly simplifying the process of creating realistic avatars. Code is available for research purposes at {\small\url{https://qianlim.github.io/SkiRT}}.

cs.CV↗

arXiv2022

Divide-and-conquer variational quantum algorithms for large-scale electronic structure simulations

Exploring the potential application of quantum computers in material design and drug discovery has attracted a lot of interest in the age of quantum computing. However, the quantum resource requirement for solving practical electronic structure problems are far beyond the capacity of near-term quantum devices. In this work, we integrate the divide-and-conquer (DC) approaches into the variational quantum eigensolver (VQE) for large-scale quantum computational chemistry simulations. Two popular divide-and-conquer schemes, including many-body expansion~(MBE) fragmentation theory and density matrix embedding theory~(DMET), are employed to divide complicated problems into many small parts that are easy to implement on near-term quantum computers. Pilot applications of these methods to systems consisting of tens of atoms are performed with adaptive VQE algorithms. This work should encourage further studies of using the philosophy of DC to solve electronic structure problems on quantum computers.

quant-ph↗

arXiv2022

Q$^2$Chemistry: A quantum computation platform for quantum chemistry

Quantum computer provides new opportunities for quantum chemistry. In this article, we present a versatile, extensible, and efficient software package, named Q$^2$Chemistry, for developing quantum algorithms and quantum inspired classical algorithms in the field of quantum chemistry. In Q$^2$Chemistry, wave function and Hamiltonian can be conveniently mapped into the qubit space, then quantum circuits can be generated according to a specific quantum algorithm already implemented in the package or newly developed by the users. The generated circuits can be dispatched to either a physical quantum computer, if available, or to the internal virtual quantum computer realized by simulating quantum circuit on classical supercomputers. As demonstrated by our benchmark simulations with up to 72 qubit, Q$^2$Chemistry achieves excellent performance in simulating medium scale quantum circuits. Application of Q$^2$Chemistry to simulate molecules and periodic systems are given with performance analysis.

quant-ph↗

arXiv2022

Two-Dimensional Semiconducting Metal Organic Frameworks with Auxetic Effect, Room Temperature Ferrimagnetism, Chiral Ferroelectricity, Bipolar Spin Polarization and Topological Nodal Lines/Points

Two-dimensional (2D) semiconductors integrated with two or more functions are the cornerstone for constructing multifunctional nanodevices, but remain largely limited. Here, by tuning the spin state of organic linkers and the symmetry/topology of crystal lattice, we predict a class of unprecedented multifunctional semiconductors in 2D Cr(II) five-membered heterocyclic metal organic frameworks that simultaneously possess auxetic effect, room temperature ferrimagnetism, chiral ferroe-lectricity, electrically reversible spin polarization and topological nodal lines/points. Taking 2D Cr(TDZ)$_2$ (TDZ=1.2.5-thiadiazole) as an exemplification, the auxetic effect is produced by the anti-tetra-chiral lattice structure. The high temperature ferrimagnetism originates from the strong d-p direct magnetic exchange interaction between Cr cations and TDZ doublet radical anions. Meanwhile, the clockwise-counterclockwise alignment of TDZ' dipoles results in unique 2D chiral ferroelectricity with atomic-scale vortex-antivortex states. 2D Cr(TDZ)$_2$ is an intrinsic bipolar magnetic semiconductor where half-metallic conduction with switchable spin-polarization direction can be induced by applying a gate voltage. Besides, the symmetry of the little group C$_4$ of lattice structure endows 2D Cr(TDZ)$_2$ with topological nodal lines and a quadratic nodal point in the Brillouin zone near the Fermi level.

cond-mat.mtrl-sci↗

arXiv2022

Ferromagnetic Negative Charge-Transfer Insulator: from Theoretical Proposal to Material Realization

Here we propose another type of ferromagnetic semiconductors: ferromagnetic negative charge-transfer insulator (FNCTI). In FNCTI, the negative charge-transfer states strongly enhance the ferromagnetic (FM) exchange interactions and the orbital hybridization gap permits the magnetic molecular orbitals as the underlying magnetic units rather than local atomic orbitals. Thus the FM exchange interactions are rather strong and decay slowly due to the large spearding of magnetic molecular orbitals. This is distinct from the superexchange mechanism where FM exchange interactions are quite weak as summarized in the well-known Goodenough-Kanamori-Anderson semi-empirical rules.Through first-principle calculations with the hybrid functional, PbO-type CrAs monolayer is mapped out to be a FNCTI, which possesses a band gap $\sim$ 0.35 eV, FM nearest-/next-nearest-neighbor exchange coupling strength $\sim$ 57/40 meV, and a high $T_c$ $\sim$ 1500 K respectively. It is believed that the existence of FNCTI validates the long-pending hypothesis by D. I. Khomskii and G. A. Sawatzky in 1997 [Solid State Commun. 102, 87 (1997)].

cond-mat.str-el↗

arXiv2022

KSSOLV 2.0: An efficient MATLAB toolbox for solving the Kohn-Sham equations with plane-wave basis set

KSSOLV (Kohn-Sham Solver) is a MATLAB toolbox for performing Kohn-Sham density functional theory (DFT) calculations with a plane-wave basis set. KSSOLV 2.0 preserves the design features of the original KSSOLV software to allow users and developers to easily set up a problem and perform ground-state calculations as well as to prototype and test new algorithms. Furthermore, it includes new functionalities such as new iterative diagonalization algorithms, k-point sampling for electron band structures, geometry optimization and advanced algorithms for performing DFT calculations with local, semi-local, and hybrid exchange-correlation functionals. It can be used to study the electronic structures of both molecules and solids. We describe these new capabilities in this work through a few use cases. We also demonstrate the numerical accuracy and computational efficiency of KSSOLV on a variety of examples.

physics.comp-ph↗

arXiv2022

Exploring accurate potential energy surfaces via integrating variational quantum eigensovler with machine learning

The potential energy surface (PES) is crucial for interpreting a variety of chemical reaction processes. However, predicting accurate PESs with high-level electronic structure methods is a challenging task due to the high computational cost. As an appealing application of quantum computing, we show in this work that variational quantum algorithms can be integrated with machine learning (ML) techniques as a promising scheme for exploring accurate PESs. Different from using a ML model to represent the potential energy, we encode the molecular geometry information into a deep neural network (DNN) for representing parameters of the variational quantum eigensolver (VQE), leaving the PES to the wave function ansatz. Once the DNN model is trained, the variational optimization procedure that hinders the application of the VQE to complex systems is avoided and thus the evaluation of PESs is significantly accelerated. Numerical results demonstrate that a simple DNN model is able to reproduce accurate PESs for small molecules.

physics.chem-ph↗

arXiv2022

gDNA: Towards Generative Detailed Neural Avatars

To make 3D human avatars widely available, we must be able to generate a variety of 3D virtual humans with varied identities and shapes in arbitrary poses. This task is challenging due to the diversity of clothed body shapes, their complex articulations, and the resulting rich, yet stochastic geometric detail in clothing. Hence, current methods to represent 3D people do not provide a full generative model of people in clothing. In this paper, we propose a novel method that learns to generate detailed 3D shapes of people in a variety of garments with corresponding skinning weights. Specifically, we devise a multi-subject forward skinning module that is learned from only a few posed, un-rigged scans per subject. To capture the stochastic nature of high-frequency details in garments, we leverage an adversarial loss formulation that encourages the model to capture the underlying statistics. We provide empirical evidence that this leads to realistic generation of local details such as wrinkles. We show that our model is able to generate natural human avatars wearing diverse and detailed clothing. Furthermore, we show that our method can be used on the task of fitting human models to raw scans, outperforming the previous state-of-the-art.

cs.CV↗