arXiv subjects

Shuangpeng Sun

Publications and source records attributed to Shuangpeng Sun.

6 recordsLinked to original sources

arXiv2024

Audio Matters Too! Enhancing Markerless Motion Capture with Audio Signals for String Performance Capture

In this paper, we touch on the problem of markerless multi-modal human motion capture especially for string performance capture which involves inherently subtle hand-string contacts and intricate movements. To fulfill this goal, we first collect a dataset, named String Performance Dataset (SPD), featuring cello and violin performances. The dataset includes videos captured from up to 23 different views, audio signals, and detailed 3D motion annotations of the body, hands, instrument, and bow. Moreover, to acquire the detailed motion annotations, we propose an audio-guided multi-modal motion capture framework that explicitly incorporates hand-string contacts detected from the audio signals for solving detailed hand poses. This framework serves as a baseline for string performance capture in a completely markerless manner without imposing any external devices on performers, eliminating the potential of introducing distortion in such delicate movements. We argue that the movements of performers, particularly the sound-producing gestures, contain subtle information often elusive to visual methods but can be inferred and retrieved from audio cues. Consequently, we refine the vision-based motion capture results through our innovative audio-guided approach, simultaneously clarifying the contact relationship between the performer and the instrument, as deduced from the audio. We validate the proposed framework and conduct ablation studies to demonstrate its efficacy. Our results outperform current state-of-the-art vision-based algorithms, underscoring the feasibility of augmenting visual motion capture with audio modality. To the best of our knowledge, SPD is the first dataset for musical instrument performance, covering fine-grained hand motion details in a multi-modal, large-scale collection.

cs.MM↗

arXiv2024

MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors

Foot contact is an important cue for human motion capture, understanding, and generation. Existing datasets tend to annotate dense foot contact using visual matching with thresholding or incorporating pressure signals. However, these approaches either suffer from low accuracy or are only designed for small-range and slow motion. There is still a lack of a vision-pressure multimodal dataset with large-range and fast human motion, as well as accurate and dense foot-contact annotation. To fill this gap, we propose a Multimodal MoCap Dataset with Vision and Pressure sensors, named MMVP. MMVP provides accurate and dense plantar pressure signals synchronized with RGBD observations, which is especially useful for both plausible shape estimation, robust pose fitting without foot drifting, and accurate global translation tracking. To validate the dataset, we propose an RGBD-P SMPL fitting method and also a monocular-video-based baseline framework, VP-MoCap, for human motion capture. Experiments demonstrate that our RGBD-P SMPL Fitting results significantly outperform pure visual motion capture. Moreover, VP-MoCap outperforms SOTA methods in foot-contact and global translation estimation accuracy. We believe the configuration of the dataset and the baseline frameworks will stimulate the research in this direction and also provide a good reference for MoCap applications in various domains. Project page: https://metaverse-ai-lab-thu.github.io/MMVP-Dataset/.

cs.CV↗

arXiv2019

HIKER: a halo-finding method based on kernel-shift algorithm

We introduce a new halo/subhalo finder, HIKER (a Halo fInder based on KERnel-shift algorithm), which takes advantage of a machine learning method -- the mean-shift algorithm combined with the Plummer kernel function, to effectively locate density peaks corresponding to halos/subhalos in density field. Based on these density peaks, dark matter halos are identified as spherical overdensity structures, and subhalos are bound substructures with boundaries at their tidal radius. By testing HIKER code with mock halos, we show that HIKER performs excellently in recovering input halo properties. Especially, HIKER has higher accuracy in locating halo/subhalo centres than most halo finders. With cosmological simulations, we further show that HIKER reproduces the abundance of dark matter halos and subhalos quite accurately, and the HIKER halo/subhalo mass functions and $V_{max}$ functions are in good agreement with two widely used halo finders, SUBFIND and AHF.

astro-ph.CO↗

arXiv2018

The Large-scale Effect of Environment on Galactic Conformity

We use a volume-limited galaxy sample from the SDSS Data Release 7 to explore the dependence of galactic conformity on the large-scale environment, measured on $\sim$ 4 Mpc scales. We find that the star formation activity of neighbour galaxies depends more strongly on the environment than on the activity of their primary galaxies. In under-dense regions most neighbour galaxies tend to be active, while in over-dense regions neighbour galaxies are mostly passive, regardless of the activity of their primary galaxies. At a given stellar mass, passive primary galaxies reside in higher density regions than active primary galaxies, leading to the apparently strong conformity signal. The dependence of the activity of neighbour galaxies on environment can be explained by the corresponding dependence of the fraction of satellite galaxies. Similar results are found for galaxies in a semi-analytical model, suggesting that no new physics is required to explain the observed large-scale conformity.

astro-ph.GA↗

arXiv2017

Massive Quiescent Galaxies at z>3 in The Millennium Simulation Populated by A Semi-analytic Galaxy Formation Model

We take advantage of the statistical power of the large-volume dark-matter-only Millennium simulation, combined with a sophisticated semi-analytic galaxy formation model, to explore whether the recently reported $z=3.7$ quiescent galaxy ZF-COSMOS-20115 (ZF; Glazebrook et al. 2017) can be accommodated in current galaxy formation models. In our model, a population of quiescent galaxies (QGs) with stellar masses and star formation rates comparable to those of ZF naturally emerges at redshifts $z<4$. There are two and five ZF analogues at the redshift $3.86$ and $3.58$ in the Millennium simulation volume, respectively. We demonstrate that, while the $z>3.5$ massive QGs are rare (about 2\% of the galaxies with the similar stellar masses), the existing AGN feedback model implemented in the semi-analytic galaxy formation model can successfully explain the formation of the high-redshift QGs as it does on their lower redshift counterparts.

astro-ph.GA↗

arXiv2017

A Universe of Ultra-Diffuse Galaxies: Theoretical Predictions from $Λ$CDM Simulations

A particular population of galaxies have drawn much interest recently, which are as faint as typical dwarf galaxies but have the sizes as large as $L^*$ galaxies, the so called "ultra-diffuse galaxie" (UDGs). The lack of tidal features of UDGs in dense environments suggests that their host halos are perhaps as massive as that of the Milky Way. On the other hand, galaxy formation efficiency should be much higher in the halos of such masses. Here we use the model galaxy catalog generated by populating two large simulations: the Millennium-II cosmological simulation and Phoenix simulations of 9 big clusters with the semi-analytic galaxy formation model. This model reproduces remarkably well the observed properties of UDGs in the nearby clusters, including the abundance, profile, color, and morphology, etc. We search for UDG candidates using the public data and find 2 UDG candidates in our Local Group and 23 in our Local Volume, in excellent agreement with the model predictions. We demonstrate that UDGs are genuine dwarf galaxies, formed in the halos of $\sim 10^{10}M_{\odot}$. It is the combination of the late formation time and high-spins of the host halos that results in the spatially extended feature of this particular population. The lack of tidal disruption features of UDGs in clusters can also be explained by their late infall-time.

astro-ph.GA↗