arXiv subjects

Teng Xu

Publications and source records attributed to Teng Xu.

24 records · Page 2Linked to original sources

arXiv2022

Artemis: Articulated Neural Pets with Appearance and Motion synthesis

We, humans, are entering into a virtual era and indeed want to bring animals to the virtual world as well for companion. Yet, computer-generated (CGI) furry animals are limited by tedious off-line rendering, let alone interactive motion control. In this paper, we present ARTEMIS, a novel neural modeling and rendering pipeline for generating ARTiculated neural pets with appEarance and Motion synthesIS. Our ARTEMIS enables interactive motion control, real-time animation, and photo-realistic rendering of furry animals. The core of our ARTEMIS is a neural-generated (NGI) animal engine, which adopts an efficient octree-based representation for animal animation and fur rendering. The animation then becomes equivalent to voxel-level deformation based on explicit skeletal warping. We further use a fast octree indexing and efficient volumetric rendering scheme to generate appearance and density features maps. Finally, we propose a novel shading network to generate high-fidelity details of appearance and opacity under novel poses from appearance and density feature maps. For the motion control module in ARTEMIS, we combine state-of-the-art animal motion capture approach with recent neural character control scheme. We introduce an effective optimization scheme to reconstruct the skeletal motion of real animals captured by a multi-view RGB and Vicon camera array. We feed all the captured motion into a neural character control scheme to generate abstract control signals with motion styles. We further integrate ARTEMIS into existing engines that support VR headsets, providing an unprecedented immersive experience where a user can intimately interact with a variety of virtual animals with vivid movements and photo-realistic appearance. We make available our ARTEMIS model and dynamic furry animal dataset at https://haiminluo.github.io/publication/artemis/.

cs.GR↗

arXiv2021

Neural Relighting and Expression Transfer On Video Portraits

Photo-realistic video portrait reenactment benefits virtual production and numerous VR/AR experiences. The task remains challenging as the reenacted expression should match the source while the lighting should be adjustable to new environments. We present a neural relighting and expression transfer technique to transfer the facial expressions from a source performer to a portrait video of a target performer while enabling dynamic relighting. Our approach employs 4D reflectance field learning, model-based facial performance capture and target-aware neural rendering. Specifically, given a short sequence of the target performer's OLAT, we apply a rendering-to-video translation network to first synthesize the OLAT result of new sequences with unseen expressions. We then design a semantic-aware facial normalization scheme along with a multi-frame multi-task learning strategy to encode the content, segmentation, and motion flows for reliably inferring the reflectance field. This allows us to simultaneously control facial expression and apply virtual relighting. Extensive experiments demonstrate that our technique can robustly handle challenging expressions and lighting environments and produce results at a cinematographic quality.

cs.CV↗

arXiv2021

Amazon SageMaker Model Parallelism: A General and Flexible Framework for Large Model Training

With deep learning models rapidly growing in size, systems-level solutions for large-model training are required. We present Amazon SageMaker model parallelism, a software library that integrates with PyTorch, and enables easy training of large models using model parallelism and other memory-saving features. In contrast to existing solutions, the implementation of the SageMaker library is much more generic and flexible, in that it can automatically partition and run pipeline parallelism over arbitrary model architectures with minimal code change, and also offers a general and extensible framework for tensor parallelism, which supports a wider range of use cases, and is modular enough to be easily applied to new training scripts. The library also preserves the native PyTorch user experience to a much larger degree, supporting module re-use and dynamic graphs, while giving the user full control over the details of the training step. We evaluate performance over GPT-3, RoBERTa, BERT, and neural collaborative filtering, and demonstrate competitive performance over existing solutions.

cs.LG↗

arXiv2021

A Dialogue-based Information Extraction System for Medical Insurance Assessment

In the Chinese medical insurance industry, the assessor's role is essential and requires significant efforts to converse with the claimant. This is a highly professional job that involves many parts, such as identifying personal information, collecting related evidence, and making a final insurance report. Due to the coronavirus (COVID-19) pandemic, the previous offline insurance assessment has to be conducted online. However, for the junior assessor often lacking practical experience, it is not easy to quickly handle such a complex online procedure, yet this is important as the insurance company needs to decide how much compensation the claimant should receive based on the assessor's feedback. In order to promote assessors' work efficiency and speed up the overall procedure, in this paper, we propose a dialogue-based information extraction system that integrates advanced NLP technologies for medical insurance assessment. With the assistance of our system, the average time cost of the procedure is reduced from 55 minutes to 35 minutes, and the total human resources cost is saved 30% compared with the previous offline procedure. Until now, the system has already served thousands of online claim cases.

cs.CL↗

arXiv2020

Thermal generation, manipulation and detection of skyrmions

Recent years have witnessed significant progresses in realizing skyrmions in chiral magnets1-4 and asymmetric magnetic multilayers5-13, as well as their electrical manipulation2,7,8,10. Equally important, thermal generation, manipulation and detection of skyrmions can be exploited for prototypical new architecture with integrated computation14 and energy harvesting15. It has yet to verify if skyrmions can be purely generated by heating16,17, and if their resultant direction of motion driven by temperature gradients follows the diffusion or, oppositely, the magnonic spin torque17-21. Here, we address these important issues in microstructured devices made of multilayers: (Ta_CoFeB_MgO)15, (Pt_CoFeB_MgO_Ta)15 and (Pt_Co_Ta)15 integrated with on-chip heaters, by using a full-field soft X-ray microscopy. The thermal generation of densely packed skyrmions is attributed to the low energy barrier at the device edge, together with the thermally induced morphological transition from stripe domains to skyrmions. The unidirectional diffusion of skyrmions from the hot region towards the cold region is experimentally observed. It can be theoretically explained by the combined contribution from repulsive forces between skyrmions, and thermal spin-orbit torques in competing with magnonic spin torques17,18,20,21 and entropic forces22. These thermally generated skyrmions can be further electrically detected by measuring the accompanied anomalous Nernst voltages23. The on-chip thermoelectric generation, manipulation and detection of skyrmions could open another exciting avenue for enabling skyrmionics, and promote interdisciplinary studies among spin caloritronics15, magnonics24 and skyrmionics3,4,12.

cond-mat.mes-hall↗

arXiv2019

Compensated magnetic insulators for extremely fast spin-orbitronics

The fast spin dynamics provide many opportunities for the future communication and memory technologies. One of the most promising examples is the domain wall (DW) racetrack memory. To achieve fast device performances, the high-speed motion of DWs is naturally demanded that leaves antiferromagnets (AFMs) and compensated ferrimagnets (FIMs) as the promising materials. While controlling and probing the dynamics of DWs in AFMs remains challenging, the fast motion of DWs with velocities around 1500 m/s has been demonstrated in metallic FIMs. The velocity of DWs in metallic FIMs is, however, suppressed by the magnetic damping of conduction electrons, which motivates us to explore how fast DWs can move in insulating FIMs where the conduction electron is absent. In this work, through synthesizing compensated FIM insulator Gd3Fe5O12 thin films with a perpendicular magnetic anisotropy, we demonstrate that the spin-orbit torque (SOT) induced motion of DWs along the Gd3Fe5O12/Pt racetrack can approach 6000 m/s. Our results show that the exceptionally fast motion of DWs can be achieved in compensated FIM insulators owing to small damping inherent to magnetic insulators, which could potentially facilitate the emerging ultrahigh-speed spintronic logics and racetrack memories by introducing insulating compensated FIMs as a new material platform.

cond-mat.mtrl-sci↗