arXiv subjects

Xiaogang Peng

Publications and source records attributed to Xiaogang Peng.

15 recordsLinked to original sources

arXiv2026

Surface Keypoint Representation for Multi-Object and Articulated Human-Object Interaction Generation

Daily activities require humans to coordinate whole-body motion with the motion of surrounding objects. Despite recent progress in human-object interaction (HOI) generation, most existing methods assume interactions with a single rigid object and do not extend well to scenarios involving a variable number of objects or articulated objects with diverse joint mechanisms. We propose surface keypoint trajectories as an object motion representation: for each rigid component, whether a standalone object or one part of an articulated assembly, we track a small set of non-collinear surface points over time. This representation handles multi-object coordination and diverse articulation mechanisms directly from point dynamics without requiring explicit joint-type specification. To model when and where each body region contacts each object, we introduce a spatio-temporal contact distance field that extends distance-based contact modeling to whole-body, multi-object, and articulated settings. We factorize HOI generation into three stages: generating object motions from text or waypoints, predicting the contact distance field, and synthesizing whole-body motion with contact-guided optimization. Experiments on ParaHome, HIMO, ARCTIC, and OMOMO demonstrate better or comparable performance to existing methods across single-object, multi-object, and articulated interaction settings.

cs.CV↗

arXiv2025

Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression

Since 2023, Vector Quantization (VQ)-based discrete generation methods have rapidly dominated human motion generation, primarily surpassing diffusion-based continuous generation methods in standard performance metrics. However, VQ-based methods have inherent limitations. Representing continuous motion data as limited discrete tokens leads to inevitable information loss, reduces the diversity of generated motions, and restricts their ability to function effectively as motion priors or generation guidance. In contrast, the continuous space generation nature of diffusion-based methods makes them well-suited to address these limitations and with even potential for model scalability. In this work, we systematically investigate why current VQ-based methods perform well and explore the limitations of existing diffusion-based methods from the perspective of motion data representation and distribution. Drawing on these insights, we preserve the inherent strengths of a diffusion-based human motion generation model and gradually optimize it with inspiration from VQ-based approaches. Our approach introduces a human motion diffusion model enabled to perform masked autoregression, optimized with a reformed data representation and distribution. Additionally, we propose a more robust evaluation method to assess different approaches. Extensive experiments on various datasets demonstrate our method outperforms previous methods and achieves state-of-the-art performances.

cs.CV↗

arXiv2025

HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Models

We address the problem of generating realistic 3D human-object interactions (HOIs) driven by textual prompts. To this end, we take a modular design and decompose the complex task into simpler sub-tasks. We first develop a dual-branch diffusion model (HOI-DM) to generate both human and object motions conditioned on the input text, and encourage coherent motions by a cross-attention communication module between the human and object motion generation branches. We also develop an affordance prediction diffusion model (APDM) to predict the contacting area between the human and object during the interactions driven by the textual prompt. The APDM is independent of the results by the HOI-DM and thus can correct potential errors by the latter. Moreover, it stochastically generates the contacting points to diversify the generated motions. Finally, we incorporate the estimated contacting points into the classifier-guidance to achieve accurate and close contact between humans and objects. To train and evaluate our approach, we annotate BEHAVE dataset with text descriptions. Experimental results on BEHAVE and OMOMO demonstrate that our approach produces realistic HOIs with various interactions and different types of objects.

cs.CV↗

arXiv2025

Absolute Coordinates Make Motion Generation Easy

State-of-the-art text-to-motion generation models rely on the kinematic-aware, local-relative motion representation popularized by HumanML3D, which encodes motion relative to the pelvis and to the previous frame with built-in redundancy. While this design simplifies training for earlier generation models, it introduces critical limitations for diffusion models and hinders applicability to downstream tasks. In this work, we revisit the motion representation and propose a radically simplified and long-abandoned alternative for text-to-motion generation: absolute joint coordinates in global space. Through systematic analysis of design choices, we show that this formulation achieves significantly higher motion fidelity, improved text alignment, and strong scalability, even with a simple Transformer backbone and no auxiliary kinematic-aware losses. Moreover, our formulation naturally supports downstream tasks such as text-driven motion control and temporal/spatial editing without additional task-specific reengineering and costly classifier guidance generation from control signals. Finally, we demonstrate promising generalization to directly generate SMPL-H mesh vertices in motion from text, laying a strong foundation for future research and motion-related applications.

cs.CV↗

arXiv2024

Ultra-bright and energy-efficient quantum-dot LEDs by idealizing charge injection

Lighting and display, relying on electric and optical down-conversion emission with sluggish power efficiency, account for >15% global electricity consumption1,2. In 2014, quantum-dot (QD) LEDs (QLEDs) with near-optimal external quantum efficiency emerged3 and promised a pathway to avoid the vast down-conversion energy loss4,5. Despite a decade of progress4-22, fabrication of energy-efficient QLEDs with application-relevant brightness remains elusive. Here, the main roadblock is identified as the oxidative species adsorbed in the nanocrystalline electron-injection layer of QLEDs, which is then addressed by a simple reductive treatment to simultaneously boosts electron conductivity and hole blockage of the electron-injection layer. The resulting sub-bandgap-driven QLEDs with optimal efficiency achieve ultra-high brightness across the entire visible spectrum at least 2.6-fold higher than existing benchmarks. The brightness fully satisfies the demands of various forms of lighting and display, which surges to a remarkable level sufficient for QD laser diodes with a moderate bias (~9 V). Optimized electron injection further enables new types of QD-blend LEDs for diffuse white-light sources surpassing the 2035 R&D targets set by the U.S. Department of Energy. Our findings open a door for understanding and optimizing carrier transport in nanocrystalline semiconductors shared by various types of solution-processed optoelectronic devices.

physics.app-ph↗

arXiv2024

Learning Weakly Supervised Audio-Visual Violence Detection in Hyperbolic Space

In recent years, the task of weakly supervised audio-visual violence detection has gained considerable attention. The goal of this task is to identify violent segments within multimodal data based on video-level labels. Despite advances in this field, traditional Euclidean neural networks, which have been used in prior research, encounter difficulties in capturing highly discriminative representations due to limitations of the feature space. To overcome this, we propose HyperVD, a novel framework that learns snippet embeddings in hyperbolic space to improve model discrimination. Our framework comprises a detour fusion module for multimodal fusion, effectively alleviating modality inconsistency between audio and visual signals. Additionally, we contribute two branches of fully hyperbolic graph convolutional networks that excavate feature similarities and temporal relationships among snippets in hyperbolic space. By learning snippet representations in this space, the framework effectively learns semantic discrepancies between violent and normal events. Extensive experiments on the XD-Violence benchmark demonstrate that our method outperforms state-of-the-art methods by a sizable margin.

cs.CV↗

arXiv2023

The MI-Motion Dataset and Benchmark for 3D Multi-Person Motion Prediction

3D multi-person motion prediction is a challenging task that involves modeling individual behaviors and interactions between people. Despite the emergence of approaches for this task, comparing them is difficult due to the lack of standardized training settings and benchmark datasets. In this paper, we introduce the Multi-Person Interaction Motion (MI-Motion) Dataset, which includes skeleton sequences of multiple individuals collected by motion capture systems and refined and synthesized using a game engine. The dataset contains 167k frames of interacting people's skeleton poses and is categorized into 5 different activity scenes. To facilitate research in multi-person motion prediction, we also provide benchmarks to evaluate the performance of prediction methods in three settings: short-term, long-term, and ultra-long-term prediction. Additionally, we introduce a novel baseline approach that leverages graph and temporal convolutional networks, which has demonstrated competitive results in multi-person motion prediction. We believe that the proposed MI-Motion benchmark dataset and baseline will facilitate future research in this area, ultimately leading to better understanding and modeling of multi-person interactions.

cs.CV↗

arXiv2023

Trajectory-Aware Body Interaction Transformer for Multi-Person Pose Forecasting

Multi-person pose forecasting remains a challenging problem, especially in modeling fine-grained human body interaction in complex crowd scenarios. Existing methods typically represent the whole pose sequence as a temporal series, yet overlook interactive influences among people based on skeletal body parts. In this paper, we propose a novel Trajectory-Aware Body Interaction Transformer (TBIFormer) for multi-person pose forecasting via effectively modeling body part interactions. Specifically, we construct a Temporal Body Partition Module that transforms all the pose sequences into a Multi-Person Body-Part sequence to retain spatial and temporal information based on body semantics. Then, we devise a Social Body Interaction Self-Attention (SBI-MSA) module, utilizing the transformed sequence to learn body part dynamics for inter- and intra-individual interactions. Furthermore, different from prior Euclidean distance-based spatial encodings, we present a novel and efficient Trajectory-Aware Relative Position Encoding for SBI-MSA to offer discriminative spatial information and additional interactive clues. On both short- and long-term horizons, we empirically evaluate our framework on CMU-Mocap, MuPoTS-3D as well as synthesized datasets (6 ~ 10 persons), and demonstrate that our method greatly outperforms the state-of-the-art methods. Code will be made publicly available upon acceptance.

cs.CV↗

arXiv2023

SoMoFormer: Social-Aware Motion Transformer for Multi-Person Motion Prediction

Multi-person motion prediction remains a challenging problem, especially in the joint representation learning of individual motion and social interactions. Most prior methods only involve learning local pose dynamics for individual motion (without global body trajectory) and also struggle to capture complex interaction dependencies for social interactions. In this paper, we propose a novel Social-Aware Motion Transformer (SoMoFormer) to effectively model individual motion and social interactions in a joint manner. Specifically, SoMoFormer extracts motion features from sub-sequences in displacement trajectory space to effectively learn both local and global pose dynamics for each individual. In addition, we devise a novel social-aware motion attention mechanism in SoMoFormer to further optimize dynamics representations and capture interaction dependencies simultaneously via motion similarity calculation across time and social dimensions. On both short- and long-term horizons, we empirically evaluate our framework on multi-person motion datasets and demonstrate that our method greatly outperforms state-of-the-art methods of single- and multi-person motion prediction. Code will be made publicly available upon acceptance.

cs.CV↗

arXiv2023

More comprehensive facial inversion for more effective expression recognition

Facial expression recognition (FER) plays a significant role in the ubiquitous application of computer vision. We revisit this problem with a new perspective on whether it can acquire useful representations that improve FER performance in the image generation process, and propose a novel generative method based on the image inversion mechanism for the FER task, termed Inversion FER (IFER). Particularly, we devise a novel Adversarial Style Inversion Transformer (ASIT) towards IFER to comprehensively extract features of generated facial images. In addition, ASIT is equipped with an image inversion discriminator that measures the cosine similarity of semantic features between source and generated images, constrained by a distribution alignment loss. Finally, we introduce a feature modulation module to fuse the structural code and latent codes from ASIT for the subsequent FER work. We extensively evaluate ASIT on facial datasets such as FFHQ and CelebA-HQ, showing that our approach achieves state-of-the-art facial inversion performance. IFER also achieves competitive results in facial expression recognition datasets such as RAF-DB, SFEW and AffectNet. The code and models are available at https://github.com/Talented-Q/IFER-master.

cs.CV↗

arXiv2022

PointCMC: Cross-Modal Multi-Scale Correspondences Learning for Point Cloud Understanding

Some self-supervised cross-modal learning approaches have recently demonstrated the potential of image signals for enhancing point cloud representation. However, it remains a question on how to directly model cross-modal local and global correspondences in a self-supervised fashion. To solve it, we proposed PointCMC, a novel cross-modal method to model multi-scale correspondences across modalities for self-supervised point cloud representation learning. In particular, PointCMC is composed of: (1) a local-to-local (L2L) module that learns local correspondences through optimized cross-modal local geometric features, (2) a local-to-global (L2G) module that aims to learn the correspondences between local and global features across modalities via local-global discrimination, and (3) a global-to-global (G2G) module, which leverages auxiliary global contrastive loss between the point cloud and image to learn high-level semantic correspondences. Extensive experiment results show that our approach outperforms existing state-of-the-art methods in various downstream tasks such as 3D object classification and segmentation. Code will be made publicly available upon acceptance.

cs.CV↗

arXiv2020

Deciphering exciton-generation processes in quantum-dot electroluminescence

Electroluminescence (EL) of colloidal nanocrystals promises a new generation of high-performance and solution-processable light-emitting diodes (LEDs). The operation of nanocrystal-based LEDs relies on the recombination of electrically-generated excitons. However, a fundamental question, i.e, how excitons are electrically generated in individual nanocrystals, remains unanswered. Here, we reveal a molecular mechanism of sequential electron-hole injection for the exciton generation in nanocrystal-based EL devices. To decipher the corresponding elementary processes, we develop electrically-pumped single-nanocrystal spectroscopy. While hole injection into neutral quantum dots (QDs) is generally-considered to be inefficient, we find that the intermediate negatively-charged state of QD triggers confinement-enhanced Coulomb interactions, which simultaneously accelerate hole injection and hinder excessive electron injection. In-situ/operando spectroscopy on state-of-the-art QD-LEDs demonstrate that exciton generation at the ensemble level is consistent with the charge-confinement-enabled sequential electron-hole injection mechanism revealed at the single-nanocrystal level. Our findings provide a universal mechanism for enhancing charge balance in nanocrystal-based EL devices.

physics.optics↗

arXiv2019

Electrochemically-stable ligands bridge photoluminescence-electroluminescence gap of quantum dots

Colloidal quantum dots (QDs) are promising emitters for electroluminescence devices (QD light-emitting-diodes, QLEDs). Though QDs have been synthesized with efficient and stable photoluminescence, inheriting their superior luminescence in QLEDs remains challenging. This is commonly attributed to unbalanced charge injection and/or interfacial exciton quenching in the devices, instead of lack of suited QD materials. Here, a general but previously overlooked degradation channel in QLEDs, i.e., operando electrochemical reactions of surface ligands with injected charge carriers, is identified after systematic studies of various combination of core/shell QDs and ligands. Applying electrochemically-inert ligands to highly photoluminescent QDs is developed to bridge their photoluminescence-electroluminescence gap. This material-design principle is general for boosting electroluminescence efficiency and lifetime of the QLEDs, resulting in record-long operational lifetimes for both red-emitting QLEDs (T95 > 3800 hours at 1000 cd m-2) and blue-emitting QLEDs (T50 >10,000 hours at 100 cd m-2). Our study provides a critical guideline for the QDs to be used in optoelectronic and electronic devices.

physics.app-ph↗

arXiv2017

Deciphering Charging Status, Absolute Quantum Efficiency, and Absorption Cross Section of MultiCarrier States in Single Colloidal Quantum Dot

Upon photo- or electrical-excitation, colloidal quantum dots (QDs) are often found in multi-carrier states due to multi-photon absorption and photo-charging of the QDs. While many of these multi-carrier states are observed in single-dot spectroscopy, their properties are not well studied due to random charging/discharging, emission intensity intermittency, and uncontrolled surface defects of single QD. Here we report in-situ deciphering the charging status, and precisely assessing the absorption cross section, and determining the absolute emission quantum yield of mono-exciton and biexciton states for neutral, positively-charged, and negatively-charged single core/shell CdSe/CdS QD. We uncover very different photon statistics of the three charge states in single QD and unambiguously identify their charge sign together with the information of their photoluminescence decay dynamics. We then show their distinct photoluminescence saturation behaviors and evaluated the absolute values of absorption cross sections and quantum efficiencies of monoexcitons and biexcitons. We demonstrate that addition of an extra hole or electron in a QD changes not only its emission properties but also varies its absorption cross section.

physics.optics↗

arXiv2005

Spin dynamics and level structure of quantum-dot quantum wells

We have characterized CdS/CdSe/CdS quantum-dot quantum wells using time-resolved Faraday rotation (TRFR). The spin dynamics show that the electron g-factor varies as a function of quantum well width and the transverse spin lifetime of several nano-seconds is robust up to room temperature. As a function of probe energy, the amplitude of the TRFR signal shows pronounced resonances, which allow one to identify individual exciton transitions. While the TRFR data are inconsistent with the conduction and valence band level scheme of spherical quantum-dot quantum wells, a model in which broken spherical symmetry is taken into account captures the essential features.

cond-mat.mes-hall↗