arXiv subjects

Tao Hu

Publications and source records attributed to Tao Hu.

At least 127 records · Page 7Linked to original sources

arXiv2023

Adaptive Model Prediction Control-Based Multi-Terrain Trajectory Tracking Framework for Mobile Spherical Robots

Owing to uncertainties in both kinematics and dynamics, the current trajectory tracking framework for mobile robots like spherical robots cannot function effectively on multiple terrains, especially uneven and unknown ones. Since this is a prerequisite for robots to execute tasks in the wild, we enhance our previous hierarchical trajectory tracking framework to handle this issue. First, a modified adaptive RBF neural network (RBFNN) is proposed to represent all uncertainties in kinodynamics. Then the Lyapunov function is utilized to design its adaptive law, and a variable step-size algorithm is employed in the weights update procedure to accelerate convergence and improve stability. Hence, a new adaptive model prediction control-based instruction planner (VAN-MPC) is proposed. Without modifying the bottom controllers, we finally develop the multi-terrain trajectory tracking framework by employing the new instruction planner VAN-MPC. The practical experiments demonstrate its effectiveness and robustness.

cs.RO↗

arXiv2023

Self-supervised Learning by View Synthesis

We present view-synthesis autoencoders (VSA) in this paper, which is a self-supervised learning framework designed for vision transformers. Different from traditional 2D pretraining methods, VSA can be pre-trained with multi-view data. In each iteration, the input to VSA is one view (or multiple views) of a 3D object and the output is a synthesized image in another target pose. The decoder of VSA has several cross-attention blocks, which use the source view as value, source pose as key, and target pose as query. They achieve cross-attention to synthesize the target view. This simple approach realizes large-angle view synthesis and learns spatial invariant representation, where the latter is decent initialization for transformers on downstream tasks, such as 3D classification on ModelNet40, ShapeNet Core55, and ScanObjectNN. VSA outperforms existing methods significantly for linear probing and is competitive for fine-tuning. The code will be made publicly available.

cs.CV↗

arXiv2023

Point2Pix: Photo-Realistic Point Cloud Rendering via Neural Radiance Fields

Synthesizing photo-realistic images from a point cloud is challenging because of the sparsity of point cloud representation. Recent Neural Radiance Fields and extensions are proposed to synthesize realistic images from 2D input. In this paper, we present Point2Pix as a novel point renderer to link the 3D sparse point clouds with 2D dense image pixels. Taking advantage of the point cloud 3D prior and NeRF rendering pipeline, our method can synthesize high-quality images from colored point clouds, generally for novel indoor scenes. To improve the efficiency of ray sampling, we propose point-guided sampling, which focuses on valid samples. Also, we present Point Encoding to build Multi-scale Radiance Fields that provide discriminative 3D point features. Finally, we propose Fusion Encoding to efficiently synthesize high-quality images. Extensive experiments on the ScanNet and ArkitScenes datasets demonstrate the effectiveness and generalization.

cs.CV↗

arXiv2023

TriVol: Point Cloud Rendering via Triple Volumes

Existing learning-based methods for point cloud rendering adopt various 3D representations and feature querying mechanisms to alleviate the sparsity problem of point clouds. However, artifacts still appear in rendered images, due to the challenges in extracting continuous and discriminative 3D features from point clouds. In this paper, we present a dense while lightweight 3D representation, named TriVol, that can be combined with NeRF to render photo-realistic images from point clouds. Our TriVol consists of triple slim volumes, each of which is encoded from the point cloud. TriVol has two advantages. First, it fuses respective fields at different scales and thus extracts local and non-local features for discriminative representation. Second, since the volume size is greatly reduced, our 3D decoder can be efficiently inferred, allowing us to increase the resolution of the 3D space to render more point details. Extensive experiments on different benchmarks with varying kinds of scenes/objects demonstrate our framework's effectiveness compared with current approaches. Moreover, our framework has excellent generalization ability to render a category of scenes/objects without fine-tuning.

cs.CV↗

arXiv2023

NeRFLiX: High-Quality Neural View Synthesis by Learning a Degradation-Driven Inter-viewpoint MiXer

Neural radiance fields (NeRF) show great success in novel view synthesis. However, in real-world scenes, recovering high-quality details from the source images is still challenging for the existing NeRF-based approaches, due to the potential imperfect calibration information and scene representation inaccuracy. Even with high-quality training frames, the synthetic novel views produced by NeRF models still suffer from notable rendering artifacts, such as noise, blur, etc. Towards to improve the synthesis quality of NeRF-based approaches, we propose NeRFLiX, a general NeRF-agnostic restorer paradigm by learning a degradation-driven inter-viewpoint mixer. Specially, we design a NeRF-style degradation modeling approach and construct large-scale training data, enabling the possibility of effectively removing NeRF-native rendering artifacts for existing deep neural networks. Moreover, beyond the degradation removal, we propose an inter-viewpoint aggregation framework that is able to fuse highly related high-quality training images, pushing the performance of cutting-edge NeRF models to entirely new levels and producing highly photo-realistic synthetic views.

cs.CV↗

arXiv2023

Electron-irradiation-facilitated production of chemically homogenized nanotwins in nanolaminated carbides

Twin boundaries have been exploited to stabilize ultrafine grains and improve the mechanical properties of nanomaterials. The production of twin boundaries and nanotwins is however prohibitively challenging in carbide ceramics. Using scanning transmission electron microscopes as a unique platform for atomic-scale structure engineering, we demonstrate that twin platelets could be produced in carbides by engineering antisite defects. Antisite defects at metal sites in various layered ternary carbides are collectively and controllably generated and the metal elements are homogenized by electron irradiation, which transforms the twin-like lamellae into nanotwin platelets. Accompanying the chemical homogenization, α-Ti3AlC2 transforms to unconventional \b{eta}-Ti3AlC2. The chemical homogeneity and the width of the twin platelets can be tuned by the dose and energy of bombarding electrons. Chemically homogenized nanotwins can boost the hardness by ~45%. Our results provide a new way to produce ultrathin (<5 nm) nanotwin platelets in scientifically and technologically important carbide materials and showcase the feasibility of defect engineering by an angstrom-sized electron probe.

cond-mat.mtrl-sci↗

arXiv2022

HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars

We propose a novel neural rendering pipeline, Hybrid Volumetric-Textural Rendering (HVTR), which synthesizes virtual human avatars from arbitrary poses efficiently and at high quality. First, we learn to encode articulated human motions on a dense UV manifold of the human body surface. To handle complicated motions (e.g., self-occlusions), we then leverage the encoded information on the UV manifold to construct a 3D volumetric representation based on a dynamic pose-conditioned neural radiance field. While this allows us to represent 3D geometry with changing topology, volumetric rendering is computationally heavy. Hence we employ only a rough volumetric representation using a pose-conditioned downsampled neural radiance field (PD-NeRF), which we can render efficiently at low resolutions. In addition, we learn 2D textural features that are fused with rendered volumetric features in image space. The key advantage of our approach is that we can then convert the fused features into a high-resolution, high-quality avatar by a fast GAN-based textural renderer. We demonstrate that hybrid rendering enables HVTR to handle complicated motions, render high-quality avatars under user-controlled poses/shapes and even loose clothing, and most importantly, be efficient at inference time. Our experimental results also demonstrate state-of-the-art quantitative results.

cs.CV↗

arXiv2022

EfficientNeRF: Efficient Neural Radiance Fields

Neural Radiance Fields (NeRF) has been wildly applied to various tasks for its high-quality representation of 3D scenes. It takes long per-scene training time and per-image testing time. In this paper, we present EfficientNeRF as an efficient NeRF-based method to represent 3D scene and synthesize novel-view images. Although several ways exist to accelerate the training or testing process, it is still difficult to much reduce time for both phases simultaneously. We analyze the density and weight distribution of the sampled points then propose valid and pivotal sampling at the coarse and fine stage, respectively, to significantly improve sampling efficiency. In addition, we design a novel data structure to cache the whole scene during testing to accelerate the rendering speed. Overall, our method can reduce over 88\% of training time, reach rendering speed of over 200 FPS, while still achieving competitive accuracy. Experiments prove that our method promotes the practicality of NeRF in the real world and enables many applications.

cs.CV↗

arXiv2022

Direction and Trajectory Tracking Control for Nonholonomic Spherical Robot by Combining Sliding Mode Controller and Model Prediction Controller

Spherical robot is a nonlinear, nonholonomic and unstable system which increases the difficulty of the direction and trajectory tracking problem. In this study, we propose a new direction controller HTSMC, an instruction planning controller MPC, and a trajectory tracking framework MHH. The HTSMC is designed by integrating a fast terminal algorithm, a hierarchical method, the motion features of a spherical robot, and its dynamics. In addition, the new direction controller has an excellent control effect with a quick response speed and strong stability. MPC can obtain optimal commands that are then transmitted to the velocity and direction controller. Since the two torque controllers in MHH are all Lyapunov-based sliding mode controllers, the MHH framework may achieve optimal control performance while assuring stability. Finally, the two controllers eliminate the requirement for MPC's stability and dynamic constraints. Finally, hardware experiments demonstrate the efficacy of the HTSMC, MPC, and MHH.

cs.RO↗

arXiv2022

Human-Object Interaction Detection via Disentangled Transformer

Human-Object Interaction Detection tackles the problem of joint localization and classification of human object interactions. Existing HOI transformers either adopt a single decoder for triplet prediction, or utilize two parallel decoders to detect individual objects and interactions separately, and compose triplets by a matching process. In contrast, we decouple the triplet prediction into human-object pair detection and interaction classification. Our main motivation is that detecting the human-object instances and classifying interactions accurately needs to learn representations that focus on different regions. To this end, we present Disentangled Transformer, where both encoder and decoder are disentangled to facilitate learning of two sub-tasks. To associate the predictions of disentangled decoders, we first generate a unified representation for HOI triplets with a base decoder, and then utilize it as input feature of each disentangled decoder. Extensive experiments show that our method outperforms prior work on two public HOI benchmarks by a sizeable margin. Code will be available.

cs.CV↗

arXiv2022

MixFormer: Mixing Features across Windows and Dimensions

While local-window self-attention performs notably in vision tasks, it suffers from limited receptive field and weak modeling capability issues. This is mainly because it performs self-attention within non-overlapped windows and shares weights on the channel dimension. We propose MixFormer to find a solution. First, we combine local-window self-attention with depth-wise convolution in a parallel design, modeling cross-window connections to enlarge the receptive fields. Second, we propose bi-directional interactions across branches to provide complementary clues in the channel and spatial dimensions. These two designs are integrated to achieve efficient feature mixing among windows and dimensions. Our MixFormer provides competitive results on image classification with EfficientNet and shows better results than RegNet and Swin Transformer. Performance in downstream tasks outperforms its alternatives by significant margins with less computational costs in 5 dense prediction tasks on MS COCO, ADE20k, and LVIS. Code is available at \url{https://github.com/PaddlePaddle/PaddleClas}.

cs.CV↗

arXiv2022

Exploring the impact of under-reported cases on the COVID-19 spatiotemporal distribution using healthcare worker infection data

A timely understanding of the spatiotemporal pattern and development trend of COVID-19 is critical for timely prevention and control. However, the under-reporting of cases is widespread in fields associated with public health. It is also possible to draw biased inferences and formulate inappropriate prevention and control policies if the phenomenon of under-reporting is not taken into account. Therefore, in this paper, a novel framework was proposed to explore the impact of under-reporting on COVID-19 spatiotemporal distributions, and empirical analysis was carried out using infection data of healthcare workers in Wuhan and Hubei (excluding Wuhan). The results show that (1) the lognormal distribution was the most suitable to describe the evolution of epidemic with time; (2) the estimated peak infection time of the reported cases lagged the peak infection time of the healthcare worker cases, and the estimated infection time interval of the reported cases was smaller than that of the healthcare worker cases. (3) The impact of under-reporting cases on the early stages of the pandemic was greater than that on its later stages, and the impact on the early onset area was greater than that on the late onset area. (4) Although the number of reported cases was lower than the actual number of cases, a high spatial correlation existed between the cumulatively reported cases and healthcare worker cases. The proposed framework of this study is highly extensible, and relevant researchers can use data sources from other counties to carry out similar research.

q-bio.PE↗

arXiv2021

EgoRenderer: Rendering Human Avatars from Egocentric Camera Images

We present EgoRenderer, a system for rendering full-body neural avatars of a person captured by a wearable, egocentric fisheye camera that is mounted on a cap or a VR headset. Our system renders photorealistic novel views of the actor and her motion from arbitrary virtual camera locations. Rendering full-body avatars from such egocentric images come with unique challenges due to the top-down view and large distortions. We tackle these challenges by decomposing the rendering process into several steps, including texture synthesis, pose construction, and neural image translation. For texture synthesis, we propose Ego-DPNet, a neural network that infers dense correspondences between the input fisheye images and an underlying parametric body model, and to extract textures from egocentric inputs. In addition, to encode dynamic appearances, our approach also learns an implicit texture stack that captures detailed appearance variation across poses and viewpoints. For correct pose generation, we first estimate body pose from the egocentric view using a parametric model. We then synthesize an external free-viewpoint pose image by projecting the parametric model to the user-specified target viewpoint. We next combine the target pose image and the textures into a combined feature image, which is transformed into the output color image using a neural image translation network. Experimental evaluations show that EgoRenderer is capable of generating realistic free-viewpoint avatars of a person wearing an egocentric camera. Comparisons to several baselines demonstrate the advantages of our approach.

cs.CV↗

arXiv2021

Probing magnetic anisotropy in Kagome antiferromagnetic Mn$_3$Ge with torque magnetometry

We investigate the magnetic symmetry of the topological antiferromagnetic material Mn$_3$Ge by using torque measurements. Below the Neel temperature, detailed angle-dependent torque measurements were performed on Mn$_3$Ge single crystals in directions parallel and perpendicular to the Kagome basal plane. The out-of plane torque data exhibit $\pm\sinθ$ and $\sin2θ$ behaviors, of which the former results from the spontaneous ferromagnetism within the basal plane and the latter from the in- and out-of-plane susceptibility anisotropy. The reversible component of the in-plane torque exhibits $\sin6φ$ behavior, revealing the six-fold symmetry of the in-plane magnetic free energy. Moreover, we find that the free energy minima are pinned to the direction of spontaneous ferromagnetism, which correspond to the maxima of the irreversible component of the in-plane torque. We provide an effective spin model to describe the in-plane magnetic anisotropy. Our results demonstrate that the ground state of Mn$_3$Ge is described by the coexistence of a strong six-fold antichiral order and a weak ferromagnetic order induced by second-order spin anisotropy.

cond-mat.str-el↗

arXiv2021

CRD-CGAN: Category-Consistent and Relativistic Constraints for Diverse Text-to-Image Generation

Generating photo-realistic images from a text description is a challenging problem in computer vision. Previous works have shown promising performance to generate synthetic images conditional on text by Generative Adversarial Networks (GANs). In this paper, we focus on the category-consistent and relativistic diverse constraints to optimize the diversity of synthetic images. Based on those constraints, a category-consistent and relativistic diverse conditional GAN (CRD-CGAN) is proposed to synthesize $K$ photo-realistic images simultaneously. We use the attention loss and diversity loss to improve the sensitivity of the GAN to word attention and noises. Then, we employ the relativistic conditional loss to estimate the probability of relatively real or fake for synthetic images, which can improve the performance of basic conditional loss. Finally, we introduce a category-consistent loss to alleviate the over-category issues between K synthetic images. We evaluate our approach using the Birds-200-2011, Oxford-102 flower and MSCOCO 2014 datasets, and the extensive experiments demonstrate superiority of the proposed method in comparison with state-of-the-art methods in terms of photorealistic and diversity of the generated synthetic images.

cs.CV↗

arXiv2021

Two-dimensional charge density wave TaX$_2$ (X=S, Se, Te) from first principles

Transition metal dichalcogenides are rich in their structural phases, e.g. 1T-TaS2 and 1T-TaSe2 form charge density wave (CDW) under low temperature with interesting and exotic properties. Here, we present a systematic study of different structures in two-dimensional TaX2 (X=S, Se, Te) using density functional theory calculations with consideration of van der Waals interaction. All the normal phases present metal characteristics with various ground state and magnetic properties. The lattice reconstruction of CDW drastically affects the electronic and structural characteristics of 1T-TaS2 and 1T-TaSe2, leading to a transition from metal to insulator and an emergence of magnetic moment within periodic atomic clusters called the Star of David. The evaluated Heisenberg couplings indicate the weak ferromagnetic coupling between the clusters in monolayer. Furthermore, in bilayer commensurate CDW cases, we find intriguing phenomenon of the varying magnetic properties with different stacking orders. The magnetic moment in each layer disappears when two layers are coupled, but may sustain in certain stackings of interlayer antiferromagnetic configurations.

cond-mat.mtrl-sci↗

arXiv2021

ODT FLOW: A Scalable Platform for Extracting, Analyzing, and Sharing Multi-source Multi-scale Human Mobility

In response to the soaring needs of human mobility data, especially during disaster events such as the COVID-19 pandemic, and the associated big data challenges, we develop a scalable online platform for extracting, analyzing, and sharing multi-source multi-scale human mobility flows. Within the platform, an origin-destination-time (ODT) data model is proposed to work with scalable query engines to handle heterogenous mobility data in large volumes with extensive spatial coverage, which allows for efficient extraction, query, and aggregation of billion-level origin-destination (OD) flows in parallel at the server-side. An interactive spatial web portal, ODT Flow Explorer, is developed to allow users to explore multi-source mobility datasets with user-defined spatiotemporal scales. To promote reproducibility and replicability, we further develop ODT Flow REST APIs that provide researchers with the flexibility to access the data programmatically via workflows, codes, and programs. Demonstrations are provided to illustrate the potential of the APIs integrating with scientific workflows and with the Jupyter Notebook environment. We believe the platform coupled with the derived multi-scale mobility data can assist human mobility monitoring and analysis during disaster events such as the ongoing COVID-19 pandemic and benefit both scientific communities and the general public in understanding human mobility dynamics.

cs.DC↗

arXiv2021

Spatiotemporal Characteristics and Factor Analysis of SARS-CoV-2 Infections among Healthcare Workers in Wuhan, China

Studying the spatiotemporal distribution of SARS-CoV-2 infections among healthcare workers (HCWs) can aid in protecting them from exposure. Existing studies related to HCW infections have emphasized infection rates and protective measures. However, the spatiotemporal patterns and related external environmental factors of HCW infections remain unclear. To fill this gap, an open-source dataset of HCW diagnoses was provided, and the spatiotemporal distributions of SARS-CoV-2 infections among HCWs in Wuhan, China were explored. A geographical detector technique was then used to investigate the impacts of hospital level, type, distance from the infection source, and other external indicators of HCW infections. The results showed that the number of daily HCW infections over time in Wuhan followed a log-normal distribution, with and its mean observed on January 23, 2020 and a standard deviation of 10.8 days. The implementation of high-impact measures, such as the lockdown of the city, may have increased the probability of HCW infections in the short term, especially for HCWs in the outer ring of Wuhan. The infection of HCWs Wuhan exhibited clear spatial heterogeneity. The number of HCW infections was higher in the central city and lower in the outer city. Moreover, HCW infections displayed significant spatial autocorrelation and dependence. Factors analyses revealed that hospital level and type had an even greater impact on HCW infections; third-class and general hospitals closer to infection sources were correlated with especially high risks of infection. These findings can aid national epidemic prevention and control departments to understand the spatiotemporal distributions of viral transmission resulting in HCW infections, as well as external influencing factors, which can facilitate the protection of HCWs in China.

physics.soc-ph↗