arXiv subjects

Yang Fu

Publications and source records attributed to Yang Fu.

At least 55 records · Page 3Linked to original sources

arXiv2023

3D Reconstruction with Generalizable Neural Fields using Scene Priors

High-fidelity 3D scene reconstruction has been substantially advanced by recent progress in neural fields. However, most existing methods train a separate network from scratch for each individual scene. This is not scalable, inefficient, and unable to yield good results given limited views. While learning-based multi-view stereo methods alleviate this issue to some extent, their multi-view setting makes it less flexible to scale up and to broad applications. Instead, we introduce training generalizable Neural Fields incorporating scene Priors (NFPs). The NFP network maps any single-view RGB-D image into signed distance and radiance values. A complete scene can be reconstructed by merging individual frames in the volumetric space WITHOUT a fusion module, which provides better flexibility. The scene priors can be trained on large-scale datasets, allowing for fast adaptation to the reconstruction of a new scene with fewer views. NFP not only demonstrates SOTA scene reconstruction performance and efficiency, but it also supports single-image novel-view synthesis, which is underexplored in neural fields. More qualitative results are available at: https://oasisyang.github.io/neural-prior

cs.CV↗

arXiv2023

FastPoseGait: A Toolbox and Benchmark for Efficient Pose-based Gait Recognition

We present FastPoseGait, an open-source toolbox for pose-based gait recognition based on PyTorch. Our toolbox supports a set of cutting-edge pose-based gait recognition algorithms and a variety of related benchmarks. Unlike other pose-based projects that focus on a single algorithm, FastPoseGait integrates several state-of-the-art (SOTA) algorithms under a unified framework, incorporating both the latest advancements and best practices to ease the comparison of effectiveness and efficiency. In addition, to promote future research on pose-based gait recognition, we provide numerous pre-trained models and detailed benchmark results, which offer valuable insights and serve as a reference for further investigations. By leveraging the highly modular structure and diverse methods offered by FastPoseGait, researchers can quickly delve into pose-based gait recognition and promote development in the field. In this paper, we outline various features of this toolbox, aiming that our toolbox and benchmarks can further foster collaboration, facilitate reproducibility, and encourage the development of innovative algorithms for pose-based gait recognition. FastPoseGait is available at https://github.com//BNU-IVC/FastPoseGait and is actively maintained. We will continue updating this report as we add new features.

cs.CV↗

arXiv2023

GPGait: Generalized Pose-based Gait Recognition

Recent works on pose-based gait recognition have demonstrated the potential of using such simple information to achieve results comparable to silhouette-based methods. However, the generalization ability of pose-based methods on different datasets is undesirably inferior to that of silhouette-based ones, which has received little attention but hinders the application of these methods in real-world scenarios. To improve the generalization ability of pose-based methods across datasets, we propose a \textbf{G}eneralized \textbf{P}ose-based \textbf{Gait} recognition (\textbf{GPGait}) framework. First, a Human-Oriented Transformation (HOT) and a series of Human-Oriented Descriptors (HOD) are proposed to obtain a unified pose representation with discriminative multi-features. Then, given the slight variations in the unified representation after HOT and HOD, it becomes crucial for the network to extract local-global relationships between the keypoints. To this end, a Part-Aware Graph Convolutional Network (PAGCN) is proposed to enable efficient graph partition and local-global spatial feature extraction. Experiments on four public gait recognition datasets, CASIA-B, OUMVLP-Pose, Gait3D and GREW, show that our model demonstrates better and more stable cross-domain capabilities compared to existing skeleton-based methods, achieving comparable recognition results to silhouette-based ones. Code is available at https://github.com/BNU-IVC/FastPoseGait.

cs.CV↗

arXiv2023

MonoNeRF: Learning Generalizable NeRFs from Monocular Videos without Camera Pose

We propose a generalizable neural radiance fields - MonoNeRF, that can be trained on large-scale monocular videos of moving in static scenes without any ground-truth annotations of depth and camera poses. MonoNeRF follows an Autoencoder-based architecture, where the encoder estimates the monocular depth and the camera pose, and the decoder constructs a Multiplane NeRF representation based on the depth encoder feature, and renders the input frames with the estimated camera. The learning is supervised by the reconstruction error. Once the model is learned, it can be applied to multiple applications including depth estimation, camera pose estimation, and single-image novel view synthesis. More qualitative results are available at: https://oasisyang.github.io/mononerf .

cs.CV↗

arXiv2023

Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wild

While 6D object pose estimation has wide applications across computer vision and robotics, it remains far from being solved due to the lack of annotations. The problem becomes even more challenging when moving to category-level 6D pose, which requires generalization to unseen instances. Current approaches are restricted by leveraging annotations from simulation or collected from humans. In this paper, we overcome this barrier by introducing a self-supervised learning approach trained directly on large-scale real-world object videos for category-level 6D pose estimation in the wild. Our framework reconstructs the canonical 3D shape of an object category and learns dense correspondences between input images and the canonical shape via surface embedding. For training, we propose novel geometrical cycle-consistency losses which construct cycles across 2D-3D spaces, across different instances and different time steps. The learned correspondence can be applied for 6D pose estimation and other downstream tasks such as keypoint transfer. Surprisingly, our method, without any human annotations or simulators, can achieve on-par or even better performance than previous supervised or semi-supervised methods on in-the-wild images. Our project page is: https://kywind.github.io/self-pose .

cs.CV↗

arXiv2022

Pressure-induced superconductivity in PdTeI with quasi-one-dimensional PdTe chains

The quasi-one-dimensional material PdTeI exhibits unusual electronic transport properties at ambient pressure. Here, we systematically investigate both the structural and electronic responses of PdTeI to external pressure, through a combination of electrical transport, synchrotron x-ray diffraction (XRD), and Raman spectroscopy measurements. The charge density wave (CDW) order in PdTeI is fragile and the transition temperature TCDW decreases rapidly with the application of external pressure. The resistivity hump is indiscernible when the pressure is increased to 1 GPa. Upon further compression, zero resistance is established above 20 GPa, suggesting the occurrence of superconductivity. Combined XRD and Raman data evidence that the emergence of superconductivity is accompanied by a pressure-induced amorphization of PdTeI.

cond-mat.supr-con↗

arXiv2022

DexMV: Imitation Learning for Dexterous Manipulation from Human Videos

While significant progress has been made on understanding hand-object interactions in computer vision, it is still very challenging for robots to perform complex dexterous manipulation. In this paper, we propose a new platform and pipeline DexMV (Dexterous Manipulation from Videos) for imitation learning. We design a platform with: (i) a simulation system for complex dexterous manipulation tasks with a multi-finger robot hand and (ii) a computer vision system to record large-scale demonstrations of a human hand conducting the same tasks. In our novel pipeline, we extract 3D hand and object poses from videos, and propose a novel demonstration translation method to convert human motion to robot demonstrations. We then apply and benchmark multiple imitation learning algorithms with the demonstrations. We show that the demonstrations can indeed improve robot learning by a large margin and solve the complex tasks which reinforcement learning alone cannot solve. More details can be found in the project page: https://yzqin.github.io/dexmv

cs.LG↗

arXiv2022

Category-Level 6D Object Pose Estimation in the Wild: A Semi-Supervised Learning Approach and A New Dataset

6D object pose estimation is one of the fundamental problems in computer vision and robotics research. While a lot of recent efforts have been made on generalizing pose estimation to novel object instances within the same category, namely category-level 6D pose estimation, it is still restricted in constrained environments given the limited number of annotated data. In this paper, we collect Wild6D, a new unlabeled RGBD object video dataset with diverse instances and backgrounds. We utilize this data to generalize category-level 6D object pose estimation in the wild with semi-supervised learning. We propose a new model, called Rendering for Pose estimation network RePoNet, that is jointly trained using the free ground-truths with the synthetic data, and a silhouette matching objective function on the real-world data. Without using any 3D annotations on real data, our method outperforms state-of-the-art methods on the previous dataset and our Wild6D test set (with manual annotations for evaluation) by a large margin. Project page with Wild6D data: https://oasisyang.github.io/semi-pose .

cs.CV↗

arXiv2022

Lattice QCD calculation of the two-photon exchange contribution to the muonic-hydrogen Lamb shift

We develop a method for lattice QCD calculation of the two-photon exchange (TPE) contribution to the muonic-hydrogen Lamb shift. To demonstrate the feasibility of this method, we also present an exploratory study with a gauge ensemble at $m_π= 142$ MeV. By adopting the infinite-volume reconstruction (IVR) method along with an optimized subtraction scheme, we obtain a preliminary result of the TPE contribution which agrees well with previous calculation using other methods and one magnitude smaller compare to the large $\sim300~μ$eV discrepancy for the proton radius puzzle.

hep-lat↗

arXiv2022

Evidence of Noncollinear Spin Texture in Magnetic Moiré Superlattices

Moiré magnetism, parallel with moiré electronics that has led to novel correlated and topological electronic states, emerges as a new venue to design and control exotic magnetic phases in twisted magnetic two-dimensional(2D) crystals. Here, we report direct evidence of noncollinear spin texture in 2D twisted double bilayer (tDB) magnet chromium triiodide (CrI$_3$). Using magneto-optical spectroscopy in tDB CrI$_3$, we revealed the presence of a net magnetization, unexpected from the composing antiferromagnetic bilayers with compensated magnetizations, and the emergence of noncollinear spins, originated from the moiré exchange coupling-induced spin frustrations. Exploring the twist angle dependence, we demonstrated that both features are present in tDB CrI$_3$ with twist angles from 0.5$^o$ to 5$^o$, but are most prominent in the 1.1$^o$ tDB CrI$_3$. Focusing on the temperature dependence of the 1.1$^o$ tDB CrI$_3$, we resolved the dramatic suppression in the net magnetization onset temperature and the significant softening of noncollinear spins, as a result of the moiré induced frustration. Our results demonstrate the power of moiré superlattices in introducing novel magnetic phenomena that are absent in natural 2D magnets.

cond-mat.mtrl-sci↗

arXiv2022

Lattice QCD calculation of the two-photon exchange contribution to the muonic-hydrogen Lamb shift

We develop a method for lattice QCD calculation of the two-photon exchange contribution to the muonic-hydrogen Lamb shift. To demonstrate its feasibility, we present the first lattice calculation with a gauge ensemble at $m_π= 142$ MeV. By adopting the infinite-volume reconstruction method along with an optimized subtraction scheme, we obtain $ΔE_{\text{TPE}} = -28.9(4.9)~μ\text{eV} + 93.72~μ\text{eV}/\text{fm}^2 \cdot\langle r_p^2 \rangle$, or $ΔE_{\text{TPE}} = 37.4(4.9)~μ$eV, which is consistent with the previous theoretical results in a range of 20-50 $μ$eV.

hep-lat↗

arXiv2021

Quantum transport evidence of topological band structures of kagome superconductor CsV3Sb5

We report the transport properties of kagome superconductor CsV$_{3}$Sb$_{5}$ single crystals at magnetic field up to 32 T. The Shubnikov de Haas (SdH) oscillations emerge at low temperature and four frequencies of $F_α=$ 27 T, $F_β=$ 73 T, $F_ε=$ 727 T, and $F_η=$ 786 T with relatively small cyclotron masses are observed. For $F_β$ and $F_ε$, the Berry phases are close to $π$, providing a clear evidence of nontrivial topological band structures of CsV$_{3}$Sb$_{5}$. Furthermore, the consistence between theoretical calculations and experimental results implies that these frequencies can be assigned to the Fermi surfaces locating near the boundary of Brillouin zone and confirms that the structure with an inverse Star of David distortion could be the most stable structure at charge density wave state. These results will shed light on the nature of correlated topological physics in kagome material CsV$_{3}$Sb$_{5}$.

cond-mat.supr-con↗

arXiv2021

Structural Monoclinicity and Its Coupling to Layered Magnetism in Few-Layer $\mathrm{CrI_{3}}$

Using polarization-resolved Raman spectroscopy, we investigate layer number, temperature, and magnetic field dependence of Raman spectra in one- to four-layer $\mathrm{CrI_{3}}$. Layer-number-dependent Raman spectra show that in the paramagnetic phase a doubly degenerated $E_{g}$ mode of monolayer $\mathrm{CrI_{3}}$ splits into one $A_{g}$ and one $B_{g}$ mode in N-layer (N > 1) $\mathrm{CrI_{3}}$ due to the monoclinic stacking. Their energy separation increases in thicker samples until an eventual saturation. Temperature-dependent measurements further show that the split modes tend to merge upon cooling but remain separated until 10 K, indicating a failed attempt of the monoclinic-to-rhombohedral structural phase transition that is present in the bulk crystal. Magnetic-field-dependent measurements reveal an additional monoclinic distortion across the magnetic-field-induced layered antiferromagnetism-to-ferromagnetism phase transition. We propose a structural change that consists of both a lateral sliding toward the rhombohedral stacking and a decrease in the interlayer distance to explain our experimental observations.

cond-mat.mes-hall↗

arXiv2021

Twist engineering of the two-dimensional magnetism in double bilayer chromium triiodide homostructures

Twist engineering, or the alignment of two-dimensional (2D) crystalline layers with desired orientations, has led to tremendous success in modulating the charge degree of freedom in hetero- and homo-structures, in particular, in achieving novel correlated and topological electronic phases in moiré electronic crystals. However, although pioneering theoretical efforts have predicted nontrivial magnetism and magnons out of twisting 2D magnets, experimental realization of twist engineering spin degree of freedom remains elusive. Here, we leverage the archetypal 2D Ising magnet chromium triiodide (CrI3) to fabricate twisted double bilayer homostructures with tunable twist angles and demonstrate the successful twist engineering of 2D magnetism in them. Using linear and circular polarization-resolved Raman spectroscopy, we identify magneto-Raman signatures of a new magnetic ground state that is sharply distinct from those in natural bilayer (2L) and four-layer (4L) CrI3. With careful magnetic field and twist angle dependence, we reveal that, for a very small twist angle (~ 0.5 degree), this emergent magnetism can be well-approximated by a weighted linear superposition of those of 2L and 4L CI3 whereas, for a relatively large twist angle (~ 5 degree), it mostly resembles that of isolated 2L CrI3. Remarkably, at an intermediate twist angle (~ 1.1 degree), its magnetism cannot be simply inferred from the 2L and 4L cases, because it lacks sharp spin-flip transitions that are present in 2L and 4L CrI3 and features a dramatic Raman circular dichroism that is absent in natural 2L and 4L ones. Our results demonstrate the possibility of designing and controlling the spin degree of freedom in 2D magnets using twist engineering.

cond-mat.mtrl-sci↗

arXiv2021

Learning to Track Instances without Video Annotations

Tracking segmentation masks of multiple instances has been intensively studied, but still faces two fundamental challenges: 1) the requirement of large-scale, frame-wise annotation, and 2) the complexity of two-stage approaches. To resolve these challenges, we introduce a novel semi-supervised framework by learning instance tracking networks with only a labeled image dataset and unlabeled video sequences. With an instance contrastive objective, we learn an embedding to discriminate each instance from the others. We show that even when only trained with images, the learned feature representation is robust to instance appearance variations, and is thus able to track objects steadily across frames. We further enhance the tracking capability of the embedding by learning correspondence from unlabeled videos in a self-supervised manner. In addition, we integrate this module into single-stage instance segmentation and pose estimation frameworks, which significantly reduce the computational complexity of tracking compared to two-stage networks. We conduct experiments on the YouTube-VIS and PoseTrack datasets. Without any video annotation efforts, our proposed method can achieve comparable or even better performance than most fully-supervised methods.

cs.CV↗

arXiv2021

Superconductivity and normal-state properties of kagome metal RbV3Sb5 single crystals

We report the discovery of superconductivity and detailed normal-state physical properties of RbV3Sb5 single crystals with V kagome lattice. RbV3Sb5 single crystals show a superconducting transition at Tc ~ 0.92 K. Meanwhile, resistivity, magnetization and heat capacity measurements indicate that it exhibits anomalies of properties at T* ~ 102 - 103 K, possibly related to the formation of charge ordering state. When T is lower than T*, the Hall coefficient RH undergoes a drastic change and sign reversal from negative to positive, which can be partially explained by the enhanced mobility of hole-type carriers. In addition, the results of quantum oscillations show that there are some very small Fermi surfaces with low effective mass, consistent with the existence of multiple highly dispersive Dirac band near the Fermi energy level.

cond-mat.supr-con↗

arXiv2020

Clique: Spatiotemporal Object Re-identification at the City Scale

Object re-identification (ReID) is a key application of city-scale cameras. While classic ReID tasks are often considered as image retrieval, we treat them as spatiotemporal queries for locations and times in which the target object appeared. Spatiotemporal reID is challenged by the accuracy limitation in computer vision algorithms and the colossal videos from city cameras. We present Clique, a practical ReID engine that builds upon two new techniques: (1) Clique assesses target occurrences by clustering fuzzy object features extracted by ReID algorithms, with each cluster representing the general impression of a distinct object to be matched against the input; (2) to search in videos, Clique samples cameras to maximize the spatiotemporal coverage and incrementally adds cameras for processing on demand. Through evaluation on 25 hours of videos from 25 cameras, Clique reached a high accuracy of 0.87 (recall at 5) across 70 queries and runs at 830x of video realtime in achieving high accuracy.

cs.DB↗

arXiv2020

CompFeat: Comprehensive Feature Aggregation for Video Instance Segmentation

Video instance segmentation is a complex task in which we need to detect, segment, and track each object for any given video. Previous approaches only utilize single-frame features for the detection, segmentation, and tracking of objects and they suffer in the video scenario due to several distinct challenges such as motion blur and drastic appearance change. To eliminate ambiguities introduced by only using single-frame features, we propose a novel comprehensive feature aggregation approach (CompFeat) to refine features at both frame-level and object-level with temporal and spatial context information. The aggregation process is carefully designed with a new attention mechanism which significantly increases the discriminative power of the learned features. We further improve the tracking capability of our model through a siamese design by incorporating both feature similarities and spatial similarities. Experiments conducted on the YouTube-VIS dataset validate the effectiveness of proposed CompFeat. Our code will be available at https://github.com/SHI-Labs/CompFeat-for-Video-Instance-Segmentation.

cs.CV↗