SearcharxivSearch

arXiv subjects

Cuong Le

Publications and source records attributed to Cuong Le.

15 recordsLinked to original sources

CQF-HMR: Continuous Quaternion Flows for Probabilistic 3D Human Mesh Recovery from a Single Image

Recovering 3D digital humans from a single 2D image is an ill-posed computer vision problem due to the loss of depth information. Probabilistic 3D human pose estimation compensates for this by estimating a set of 3D hypotheses from a prior distribution via generative models. However, most prior work focuses only on 3D keypoints, which often leads to implausible poses that are difficult to apply to downstream tasks, e.g. animation or digital humans. SMPL-based methods are more scalable thanks to the explicit body priors, but it requires more complex modeling of the generation process due to the non-additive nature of the joint rotations. In this work, we propose a novel approach for probabilistic 3D humans using quaternion-constrained continuous normalizing flows conditioned on 2D pose estimations. Our proposed quaternion flows show significant advantages over approaches using other rotation representations. Experiments demonstrate state-of-the-art results of our method on Human3.6M, particularly in ambiguous settings, and comparable pose estimation accuracy on challenging 3DPW and EMDB benchmarks.

cs.CV

Gravity-guided Contact Dynamics Estimation from 3D Human Motions

Ground contact forces acting on the human body, are crucial for biomechanics studies or sport performance analysis. Prior methods rely on force plates or pressure mats to collect ground contact dynamics, limiting their applicability to carefully controlled settings. A more scalable solution is to estimate the dynamics directly from motion capture data. Recent approaches only roughly estimate the ground contact dynamics from the vertical distance between the body and the ground plane, which cannot capture the complex pressure distribution of all contact points. To this end, we propose GraCE -- Gravity-guided Contact Dynamics Estimation, a novel full-body contact dynamics model for human motions using a realistic influence of body mass distribution and gravity. We use the human's center of gravity to estimate the ground contacts based on its relative distance to the human body. The applied force on each contact is estimated via the product of predicted contact probabilities and the total exterior force computed from the center of mass trajectory. We outperform related work on the GroundLink dataset for ground reaction force estimation, and on the MOYO dataset for detailed contact pressure prediction. The code is published upon acceptance.

cs.CV

QuaMo: Quaternion Motions for Vision-based 3D Human Kinematics Capture

Vision-based 3D human motion capture from videos remains a challenge in computer vision. Traditional 3D pose estimation approaches often ignore the temporal consistency between frames, causing implausible and jittery motion. The emerging field of kinematics-based 3D motion capture addresses these issues by estimating the temporal transitioning between poses instead. A major drawback in current kinematics approaches is their reliance on Euler angles. Despite their simplicity, Euler angles suffer from discontinuity that leads to unstable motion reconstructions, especially in online settings where trajectory refinement is unavailable. Contrarily, quaternions have no discontinuity and can produce continuous transitions between poses. In this paper, we propose QuaMo, a novel Quaternion Motions method using quaternion differential equations (QDE) for human kinematics capture. We utilize the state-space model, an effective system for describing real-time kinematics estimations, with quaternion state and the QDE describing quaternion velocity. The corresponding angular acceleration is computed from a meta-PD controller with a novel acceleration enhancement that adaptively regulates the control signals as the human quickly changes to a new pose. Unlike previous work, our QDE is solved under the quaternion unit-sphere constraint that results in more accurate estimations. Experimental results show that our novel formulation of the QDE with acceleration enhancement accurately estimates 3D human kinematics with no discontinuity and minimal implausibilities. QuaMo outperforms comparable state-of-the-art methods on multiple datasets, namely Human3.6M, Fit3D, SportsPose and AIST. The code is available at https://github.com/cuongle1206/QuaMo

cs.CV

Flow Matching for Probabilistic Monocular 3D Human Pose Estimation

Recovering 3D human poses from a monocular camera view is a highly ill-posed problem due to the depth ambiguity. Earlier studies on 3D human pose lifting from 2D often contain incorrect-yet-overconfident 3D estimations. To mitigate the problem, emerging probabilistic approaches treat the 3D estimations as a distribution, taking into account the uncertainty measurement of the poses. Falling in a similar category, we proposed FMPose, a probabilistic 3D human pose estimation method based on the flow matching generative approach. Conditioned on the 2D cues, the flow matching scheme learns the optimal transport from a simple source distribution to the plausible 3D human pose distribution via continuous normalizing flows. The 2D lifting condition is modeled via graph convolutional networks, leveraging the learnable connections between human body joints as the graph structure for feature aggregation. While trade-offs between processing time and precision exist, already in the equal-accuracy comparison, FMPose exhibits significantly faster processing time than the diffusion model, and also offers another faster and more accurate configuration. Experimental results show major improvements of our FMPose over current state-of-the-art methods on two common benchmarks for 3D human pose estimation, namely Human3.6M, MPI-INF-3DHP. Additionally, FMPose shows competitive performance on the more challenging 3DPW dataset. The code implementation is available at https://github.com/cuongle1206/FMPose

cs.CV

Constrained MARL for Coexisting TN-NTN Resource Allocation: Scalability and Flexibility

This paper considers the joint TN-NTN constrained resource allocation, where terrestrial base stations and non-terrestrial base stations coexist in the spectrum. We focus on large-scale and practical scenarios characterized by large numbers of transmission channels and users, alongside highly dynamic user behaviors. As common learning solutions fail to address these challenges, we propose a decomposition solution based on the special properties of the cross-segment interference, and then tackle the original problem via solving subproblems in a sequential learning manner. Furthermore, to enhance the flexibility of the learned policies, we design a stochastic training environment that captures the key characteristics of real-world systems. Simulation results tested on the full 20MHz bandwidth with various numerologies show that our solution significantly improves scalability compared to existing solutions and remains robust in highly dynamic scenarios.

cs.IT

On the Role of Rotation Equivariance in Monocular 2D-to-3D Human Pose Lifting

We consider monocular 3D human pose estimation (HPE), where the goal is to predict 3D human skeletal joints from a single 2D image, typically via 2D keypoint detection followed by 2D-to-3D lifting. Despite their success, we find that current lifting models exhibit strong performance degradation under rotations. We systematically study rotation equivariance across lifting architectures with increasing degrees of equivariant inductive bias, and investigate whether equivariance should be enforced by architectural design or learned from data, given the inherent ambiguity of monocular 2D-to-3D lifting. Utilising common HPE benchmarks, we demonstrate that rotation equivariance can be effectively learned via rotation-based data augmentation applied jointly to input and output poses. Compared with non-augmented models, this reduces error on rotated poses by over 30% across the benchmarks, with reductions of up to 72%. Moreover, augmentation-based models achieve up to 15% lower error than methods that are fully equivariant by design, while providing up to 37$\times$ faster inference. We further show that these findings generalise to real-world human pose sequences involving full-body rotations and to a state-of-the-art HPE model.

cs.CV

Cooperative UAVs for Remote Data Collection under Limited Communications: An Asynchronous Multiagent Learning Framework

This paper addresses the joint optimization of trajectories and bandwidth allocation for multiple Unmanned Aerial Vehicles (UAVs) to enhance energy efficiency in the cooperative data collection problem. We focus on an important yet underestimated aspect of the system, where action synchronization across all UAVs is impossible. Since most existing learning-based solutions are not designed to learn in this asynchronous environment, we formulate the trajectory planning problem as a Decentralized Partially Observable Semi-Markov Decision Process and introduce an asynchronous multi-agent learning algorithm to learn UAVs' cooperative policies. Once the UAVs' trajectory policies are learned, the bandwidth allocation can be optimally solved based on local observations at each collection point. Comprehensive empirical results demonstrate the superiority of the proposed method over other learning-based and heuristic baselines in terms of both energy efficiency and mission completion time. Additionally, the learned policies exhibit robustness under varying environmental conditions.

cs.MA

Ultrafast electron diffractive imaging of the dissociation of pre-excited molecules

Gas phase ultrafast electron diffraction (GUED) has become a powerful technique to directly observe the structural dynamics of photoexcited molecules. GUED reveals information about the nuclear motions that is complementary to the information on the electronic states provided by spectroscopic measurements. GUED experiments so far have utilized a single laser pulse to excite the molecules and an electron pulse to probe the dynamics. This limits the excited states which can be studied to only those that can be reached by absorption of a photon from the ground state or in some cases simultaneous absorption of multiple photons. A broader class of experiments and dynamics can be accessed using two time-delayed laser pulses to access unexplored regions of the potential energy surfaces. As a proof-of-principle experiment using a double excitation, we studied the photodissociation of trifluoroiodomethane molecules that are pre-excited with an infrared (800 nm) femtosecond laser pulse before photo-dissociation is triggered with an ultraviolet (266 nm) femtosecond laser pulse. We have observed significant differences in the dissociation dynamics, with pre-excitation resulting in a slower dissociation process. This new capability can offer new insights on the evolution of nuclear wavepackets in regions of the excited potential energy surface which are inaccessible in single photon excitation. We present a methodology to carry out the measurement, analyze and interpret the data that could be applied to a broad class of experiments.

physics.chem-ph

Physics-informed Ground Reaction Dynamics from Human Motion Capture

Body dynamics are crucial information for the analysis of human motions in important research fields, ranging from biomechanics, sports science to computer vision and graphics. Modern approaches collect the body dynamics, external reactive force specifically, via force plates, synchronizing with human motion capture data, and learn to estimate the dynamics from a black-box deep learning model. Being specialized devices, force plates can only be installed in laboratory setups, imposing a significant limitation on the learning of human dynamics. To this end, we propose a novel method for estimating human ground reaction dynamics directly from the more reliable motion capture data with physics laws and computational simulation as constrains. We introduce a highly accurate and robust method for computing ground reaction forces from motion capture data using Euler's integration scheme and PD algorithm. The physics-based reactive forces are used to inform the learning model about the physics-informed motion dynamics thus improving the estimation accuracy. The proposed approach was tested on the GroundLink dataset, outperforming the baseline model on: 1) the ground reaction force estimation accuracy compared to the force plates measurement; and 2) our simulated root trajectory precision. The implementation code is available at https://github.com/cuongle1206/Phys-GRD

cs.CV

Ultrafast dynamics of ferroelectric polarization of NbOI$_{2}$ captured with femtosecond electron diffraction

Two-dimensional (2D) ferroelectric materials like NbOI$_{2}$ have garnered significant interest, yet their temporal response and synergetic interaction with light remain underexplored. Previous studies on the polarization of oxide ferroelectrics have relied on time-resolved optical second harmonic generation or ultrafast X-ray scattering. Here, we probe the laser-induced polarization dynamics of 2D NbOI$_{2}$ nanocrystals using ultrafast transmission electron diffraction and deflectometry. The deflection of the electron pulses is directly sensitive to the changes in the polarization, while the diffraction signal captures the structural evolution. Excited with a UV laser pulse, the polarization of NbOI$_{2}$ is initially suppressed for two picoseconds, then it recovers and overshoots, leading to a transiently enhanced polarization persisting for over 200 ps. This recovery coincides with coherent acoustic phonon generation, triggering a piezoresponse in the NbOI$_{2}$ nanocrystals. Our results offer a new method for sensing the ferroelectric order parameter in femtosecond time scales.

cond-mat.mtrl-sci

Competitive Facility Location with Market Expansion and Customer-centric Objective

We study a competitive facility location problem, where customer behavior is modeled and predicted using a discrete choice random utility model. The goal is to strategically place new facilities to maximize the overall captured customer demand in a competitive marketplace. In this work, we introduce two novel considerations. First, the total customer demand in the market is not fixed but is modeled as an increasing function of the customers' total utilities. Second, we incorporate a new term into the objective function, aiming to balance the firm's benefits and customer satisfaction. Our new formulation exhibits a highly nonlinear structure and is not directly solved by existing approaches. To address this, we first demonstrate that, under a concave market expansion function, the objective function is concave and submodular, allowing for a $(1-1/e)$ approximation solution by a simple polynomial-time greedy algorithm. We then develop a new method, called Inner-approximation, which enables us to approximate the mixed-integer nonlinear problem (MINLP), with arbitrary precision, by an MILP without introducing additional integer variables. We further demonstrate that our inner-approximation method consistently yields lower approximations than the outer-approximation methods typically used in the literature. Moreover, we extend our settings by considering a\textit{ general (non-concave)} market-expansion function and show that the Inner-approximation mechanism enables us to approximate the resulting MINLP, with arbitrary precision, by an MILP. To further enhance this MILP, we show how to significantly reduce the number of additional binary variables by leveraging concave areas of the objective function. Extensive experiments demonstrate the efficiency of our approaches.

math.OC

Fast Beam Placement for Ultra-Dense LEO Networks

Low Earth orbit (LEO) satellites has brought about significant improvements in wireless communications, characterized by low latency and reduced transmission loss compared to geostationary orbit (GSO) satellites. Ultra-dense LEO satellites can serve many users by generating active beams effective to their locations. The beam placement problem is challenging but important for efficiently allocating resources with a large number of users. This paper formulates and solves a fast beam placement optimization problem for ultra-dense satellite systems to enhance the link budget with a minimum number of active beams (NABs). To achieve this goal and balance load among beams within polynomial time, we propose two algorithms for large user groups exploiting the modified K-means clustering and the graph theory. Numerical results illustrate the effectiveness of the proposals in terms of the statistical channel gain-to-noise ratio and computation time over state-of-the-art benchmarks.

cs.IT

Optimal-state Dynamics Estimation for Physics-based Human Motion Capture from Videos

Human motion capture from monocular videos has made significant progress in recent years. However, modern approaches often produce temporal artifacts, e.g. in form of jittery motion and struggle to achieve smooth and physically plausible motions. Explicitly integrating physics, in form of internal forces and exterior torques, helps alleviating these artifacts. Current state-of-the-art approaches make use of an automatic PD controller to predict torques and reaction forces in order to re-simulate the input kinematics, i.e. the joint angles of a predefined skeleton. However, due to imperfect physical models, these methods often require simplifying assumptions and extensive preprocessing of the input kinematics to achieve good performance. To this end, we propose a novel method to selectively incorporate the physics models with the kinematics observations in an online setting, inspired by a neural Kalman-filtering approach. We develop a control loop as a meta-PD controller to predict internal joint torques and external reaction forces, followed by a physics-based motion simulation. A recurrent neural network is introduced to realize a Kalman filter that attentively balances the kinematics input and simulated motion, resulting in an optimal-state dynamics prediction. We show that this filtering step is crucial to provide an online supervision that helps balancing the shortcoming of the respective input motions, thus being important for not only capturing accurate global motion trajectories but also producing physically plausible human poses. The proposed approach excels in the physics-based human pose estimation task and demonstrates the physical plausibility of the predictive dynamics, compared to state of the art. The code is available on https://github.com/cuongle1206/OSDCap

cs.CV

O$n$ Learning Deep O($n$)-Equivariant Hyperspheres

In this paper, we utilize hyperspheres and regular $n$-simplexes and propose an approach to learning deep features equivariant under the transformations of $n$D reflections and rotations, encompassed by the powerful group of O$(n)$. Namely, we propose O$(n)$-equivariant neurons with spherical decision surfaces that generalize to any dimension $n$, which we call Deep Equivariant Hyperspheres. We demonstrate how to combine them in a network that directly operates on the basis of the input points and propose an invariant operator based on the relation between two points and a sphere, which as we show, turns out to be a Gram matrix. Using synthetic and real-world data in $n$D, we experimentally verify our theoretical contributions and find that our approach is superior to the competing methods for O$(n)$-equivariant benchmark datasets (classification and regression), demonstrating a favorable speed/performance trade-off. The code is available at https://github.com/pavlo-melnyk/equivariant-hyperspheres.

cs.LG

Constrained Assortment Optimization under the Cross-Nested Logit Model

We study the assortment optimization problem under general linear constraints, where the customer choice behavior is captured by the Cross-Nested Logit model. In this problem, there is a set of products organized into multiple subsets (or nests), where each product can belong to more than one nest. The aim is to find an assortment to offer to customers so that the expected revenue is maximized. We show that, under the Cross-Nested Logit model, the assortment problem is NP-hard, even without any constraints. To tackle the assortment optimization problem, we develop a new discretization mechanism to approximate the problem by a linear fractional program with a performance guarantee of $\frac{1 - ε}{1+ε}$, for any accuracy level $ε>0$. We then show that optimal solutions to the approximate problem can be obtained by solving mixed-integer linear programs. We further show that our discretization approach can also be applied to solve a joint assortment optimization and pricing problem, as well as an assortment problem under a mixture of Cross-Nested Logit models to account for multiple classes of customers. Our empirical results on a large number of randomly generated test instances demonstrate that, under a performance guarantee of 90%, the percentage gaps between the objective values obtained from our approximation methods and the optimal expected revenues are no larger than 1.2%.

math.OC