arXiv subjects

Changsheng Lv

Publications and source records attributed to Changsheng Lv.

13 recordsLinked to original sources

arXiv2026

SGFormer++: Semantic Graph Transformer for Incremental 3D Scene Graph Generation

In this paper, we propose SGFormer++, a novel Semantic Graph Transformer for 3D scene graph generation (SGG), which aims to parse point cloud scenes into semantic structural graphs, where nodes denote detected object instances and edges encode their pairwise relationships, with the core challenge lying in modeling complex global scene structure. While existing graph convolutional network (GCN)-based methods suffer from over-smoothing and limited receptive fields, SGFormer++ leverages Transformer layers as its backbone to enable global message passing. Specifically, we introduce two key components tailored for 3D SGG: (1) a Graph Embedding Layer++ that efficiently integrates edge-aware global context with linear computational complexity, and (2) a Semantic Injection Layer++ that enriches visual features with linguistic priors from large language models (LLMs) and vision-language models (VLMs), boosting semantic representation without introducing extra trainable parameters. To further address the practical challenge of incremental SGG (I-SGG), where new relationship categories arrive sequentially, we equip SGFormer++ with a novel Spatial-guided Feature Adapter, which calibrates predicate features using subject-object spatial geometry to counter scale variation, and a Cascaded Binary Prediction Head that mitigates catastrophic forgetting via task-incremental classifier expansion and logit distillation. Extensive experiments on the 3DSSG benchmark demonstrate that SGFormer++ achieves state-of-the-art performance in both standard and incremental settings: it yields a significant 4.49% absolute improvement in Predicate A@1 under the incremental setting. Code and data are available at: https://github.com/Andy20178/SGFormer.

cs.CV↗

arXiv2025

Multi-Modal Scene Graph with Kolmogorov-Arnold Experts for Audio-Visual Question Answering

In this paper, we propose a novel Multi-Modal Scene Graph with Kolmogorov-Arnold Expert Network for Audio-Visual Question Answering (SHRIKE). The task aims to mimic human reasoning by extracting and fusing information from audio-visual scenes, with the main challenge being the identification of question-relevant cues from complex audio-visual content. Existing methods fail to capture the structural information within videos and suffer from insufficient fine-grained modeling of multi-modal features. To address these issues, we are the first to introduce a new multi-modal scene graph that explicitly models objects and their relationships as a visually grounded, structured representation of the audio-visual scene, yielding 461,292 relation triplets over 9,288 musical performance videos. Furthermore, we design a Kolmogorov-Arnold Network (KAN)-based Mixture of Experts (MoE) to enhance the expressive power of the temporal integration stage. This enables more fine-grained modeling of cross-modal interactions within the question-aware fused audio-visual representation, capturing richer and more nuanced patterns and improving temporal reasoning performance. We evaluate the model on the established MUSIC-AVQA and MUSIC-AVQA v2 benchmarks, where it achieves state-of-the-art performance, reaching 78.14% average accuracy on MUSIC-AVQA, surpassing the previous best method QA-TIGER and ranking first under all four configurations of MUSIC-AVQA v2.0. Code and model checkpoints will be publicly released at https://github.com/feel12348/SHRIKE.

cs.AI↗

arXiv2025

Robo-SGG: Exploiting Layout-Oriented Normalization and Restitution Can Improve Robust Scene Graph Generation

In this paper, we propose Robo-SGG, a plug-and-play module for robust scene graph generation (SGG). Unlike standard SGG, the robust scene graph generation aims to perform inference on a diverse range of corrupted images, with the core challenge being the domain shift between the clean and corrupted images. Existing SGG methods suffer from degraded performance due to shifted visual features (e.g., corruption interference or occlusions). To obtain robust visual features, we leverage layout information, representing the global structure of an image, which is robust to domain shift, to enhance the robustness of SGG methods under corruption. Specifically, we employ Instance Normalization (IN) to alleviate the domain-specific variations and recover the robust structural features (i.e., the positional and semantic relationships among objects) by the proposed Layout-Oriented Restitution. Furthermore, under corrupted images, we introduce a Layout-Embedded Encoder (LEE) that adaptively fuses layout and visual features via a gating mechanism, enhancing the robustness of positional and semantic representations for objects and predicates. Note that our proposed Robo-SGG module is designed as a plug-and-play component, which can be easily integrated into any baseline SGG model. Extensive experiments demonstrate that by integrating the state-of-the-art method into our proposed Robo-SGG, we achieve relative improvements of 6.3%, 11.1%, and 8.0% in mR@50 for PredCls, SGCls, and SGDet tasks on the VG-C benchmark, respectively, and achieve new state-of-the-art performance in the corruption scene graph generation benchmark (VG-C and GQA-C). We will release our source code and model.

cs.CV↗

arXiv2025

Robust Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoning

In this paper, we propose a new Robust Disentangled Counterfactual Learning (RDCL) approach for physical audiovisual commonsense reasoning. The task aims to infer objects' physics commonsense based on both video and audio input, with the main challenge being how to imitate the reasoning ability of humans, even under the scenario of missing modalities. Most of the current methods fail to take full advantage of different characteristics in multi-modal data, and lacking causal reasoning ability in models impedes the progress of implicit physical knowledge inferring. To address these issues, our proposed RDCL method decouples videos into static (time-invariant) and dynamic (time-varying) factors in the latent space by the disentangled sequential encoder, which adopts a variational autoencoder (VAE) to maximize the mutual information with a contrastive loss function. Furthermore, we introduce a counterfactual learning module to augment the model's reasoning ability by modeling physical knowledge relationships among different objects under counterfactual intervention. To alleviate the incomplete modality data issue, we introduce a robust multimodal learning method to recover the missing data by decomposing the shared features and model-specific features. Our proposed method is a plug-and-play module that can be incorporated into any baseline including VLMs. In experiments, we show that our proposed method improves the reasoning accuracy and robustness of baseline methods and achieves the state-of-the-art performance.

cs.CV↗

arXiv2024

T2SG: Traffic Topology Scene Graph for Topology Reasoning in Autonomous Driving

Understanding the traffic scenes and then generating high-definition (HD) maps present significant challenges in autonomous driving. In this paper, we defined a novel Traffic Topology Scene Graph, a unified scene graph explicitly modeling the lane, controlled and guided by different road signals (e.g., right turn), and topology relationships among them, which is always ignored by previous high-definition (HD) mapping methods. For the generation of T2SG, we propose TopoFormer, a novel one-stage Topology Scene Graph TransFormer with two newly designed layers. Specifically, TopoFormer incorporates a Lane Aggregation Layer (LAL) that leverages the geometric distance among the centerline of lanes to guide the aggregation of global information. Furthermore, we proposed a Counterfactual Intervention Layer (CIL) to model the reasonable road structure ( e.g., intersection, straight) among lanes under counterfactual intervention. Then the generated T2SG can provide a more accurate and explainable description of the topological structure in traffic scenes. Experimental results demonstrate that TopoFormer outperforms existing methods on the T2SG generation task, and the generated T2SG significantly enhances traffic topology reasoning in downstream tasks, achieving a state-of-the-art performance of 46.3 OLS on the OpenLane-V2 benchmark. We will release our source code and model.

cs.CV↗

arXiv2023

SGFormer: Semantic Graph Transformer for Point Cloud-based 3D Scene Graph Generation

In this paper, we propose a novel model called SGFormer, Semantic Graph TransFormer for point cloud-based 3D scene graph generation. The task aims to parse a point cloud-based scene into a semantic structural graph, with the core challenge of modeling the complex global structure. Existing methods based on graph convolutional networks (GCNs) suffer from the over-smoothing dilemma and can only propagate information from limited neighboring nodes. In contrast, SGFormer uses Transformer layers as the base building block to allow global information passing, with two types of newly-designed layers tailored for the 3D scene graph generation task. Specifically, we introduce the graph embedding layer to best utilize the global information in graph edges while maintaining comparable computation costs. Furthermore, we propose the semantic injection layer to leverage linguistic knowledge from large-scale language model (i.e., ChatGPT), to enhance objects' visual features. We benchmark our SGFormer on the established 3DSSG dataset and achieve a 40.94% absolute improvement in relationship prediction's R@50 and an 88.36% boost on the subset with complex scenes over the state-of-the-art. Our analyses further show SGFormer's superiority in the long-tail and zero-shot scenarios. Our source code is available at https://github.com/Andy20178/SGFormer.

cs.CV↗

arXiv2023

Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoning

In this paper, we propose a Disentangled Counterfactual Learning~(DCL) approach for physical audiovisual commonsense reasoning. The task aims to infer objects' physics commonsense based on both video and audio input, with the main challenge is how to imitate the reasoning ability of humans. Most of the current methods fail to take full advantage of different characteristics in multi-modal data, and lacking causal reasoning ability in models impedes the progress of implicit physical knowledge inferring. To address these issues, our proposed DCL method decouples videos into static (time-invariant) and dynamic (time-varying) factors in the latent space by the disentangled sequential encoder, which adopts a variational autoencoder (VAE) to maximize the mutual information with a contrastive loss function. Furthermore, we introduce a counterfactual learning module to augment the model's reasoning ability by modeling physical knowledge relationships among different objects under counterfactual intervention. Our proposed method is a plug-and-play module that can be incorporated into any baseline. In experiments, we show that our proposed method improves baseline methods and achieves state-of-the-art performance. Our source code is available at https://github.com/Andy20178/DCL.

cs.CV↗

arXiv2014

Reconstructing human organ cross-sectional imaging along any axis

Cross-sectional imaging of human organ serves as a critical tool to provide diagnostic results of many diseases. Based on a unique body coordinate system, we present a method that we use to reconstruct any cross-sectional imaging of organ regardless of its original section going along which scanning or cutting axis. In clinical medicine, this method enables a patient to undergo only one scanning, and then the doctor can observe the structure of lesion sections along any axis, and it can help find changes of lesions at the same section from different scanning results and thus quantify diagnosis by cross-sectional imaging. Significant progress has thus been made towards quantitative diagnosis cross-sectional imaging.

physics.med-ph↗

arXiv2013

Three-Dimensional Reconstruction of Erythrocyte in the Capillary

The dynamic analysis of erythrocyte deformability is used as an important means for early diagnosis of blood diseases and blood rheology. Yet no effective method is available in terms of three-dimensional reconstruction of erythrocytes in a capillary. In this study, ultrathin serial sections of skeletal muscle tissue are obtained from the ultramicrotome, the tomographic images of an erythrocyte in a capillary are captured by the transmission electron microscope, and then a method to position and restore is devised to demonstrate the physiological relationship between two adjacent tomographic images of an erythrocyte. Both the modeling and the physical verification reveal that this method is effective, which means that it can be used to make three-dimensional reconstruction of an erythrocyte in a capillary. An example of reconstructed deformation of erythrocyte based on the serial ultrathin sections is shown at the end of this paper.

physics.med-ph↗

arXiv2011

Natural gaits of the non-pathological flat foot and high-arched foot

There has been a controversy as to whether or not the non-pathological flat foot and high-arched foot have an effect on human walking activities. The 3D foot scanning system was employed to obtain static footprints from subjects adopting a half-weight-bearing stance. Based upon their footprints, the subjects were divided into two groups: the flat-footed and the high-arched. The plantar pressure measurement system was used to measure and record the subjects' successive natural gaits. Two indices were proposed: distribution of vertical ground reaction force (VGRF) of plantar and the rate of the footprint areas. Using these two indices to compare the natural gaits of the two subject groups, we found that (1) in stance phase, there is a significant difference (p<0.01) in the distributions of VGRF of plantar; (2) in a stride cycle, there is also a significant difference (p<0.01) in the rates of the footprint areas. Our analysis suggests that when walking, the VGRF of the plantar brings greater muscle tension to the flat-footed while a smaller rate of the footprint areas brings greater stability to the high-arched.

physics.med-ph↗

arXiv2010

Bone in vivo: Surface mapping technique

Bone surface mapping technique is proposed on the bases of two kinds of uniqueness of bone in vivo, (i) magnitude of the principal moments of inertia, (ii) the direction cosines of principal axes of inertia relative to inertia reference frame. We choose the principal axes of inertia as the bone coordinate system axes. The geographical marks such as the prime meridian of the bone in vivo are defined and methods such as tomographic reconstruction and boundary development are employed so that the surface of bone in vivo can be mapped. Experimental results show that the surface mapping technique can both reflect the shape and help study the surface changes of bone in vivo. The prospect of such research into the surface shape and changing laws of organ, tissue or cell will be promising.

physics.bio-ph↗

arXiv2010

Foot Bone in Vivo: Its Center of Mass and Centroid of Shape

This paper studies foot bone geometrical shape and its mass distribution and establishes an assessment method of bone strength. Using spiral CT scanning, with an accuracy of sub-millimeter, we analyze the data of 384 pieces of foot bones in vivo and investigate the relationship between the bone's external shape and internal structure. This analysis is explored on the bases of the bone's center of mass and its centroid of shape. We observe the phenomenon of superposition of center of mass and centroid of shape fairly precisely, indicating a possible appearance of biomechanical organism. We investigate two aspects of the geometrical shape, (i) distance between compact bone's centroid of shape and that of the bone and (ii) the mean radius of the same density bone issue relative to the bone's centroid of shape. These quantities are used to interpret the influence of different physical exercises imposed on bone strength, thereby contributing to an alternate assessment technique to bone strength.

physics.bio-ph↗

arXiv2010

Dynamic Principles of Center of Mass in Human Walking

We present results of an analytic and numerical calculation that studies the relationship between the time of initial foot contact and the ground reaction force of human gait and explores the dynamic principle of center of mass. Assuming the ground reaction force of both feet to be the same in the same phase of a stride cycle, we establish the relationships between the time of initial foot contact and the ground reaction force, acceleration, velocity, displacement and average kinetic energy of center of mass. We employ the dispersion to analyze the effect of the time of the initial foot contact that imposes upon these physical quantities. Our study reveals that when the time of one foot's initial contact falls right in the middle of the other foot's stride cycle, these physical quantities reach extrema. An action function has been identified as the dispersion of the physical quantities and optimized analysis used to prove the least-action principle in gait. In addition to being very significant to the research domains such as clinical diagnosis, biped robot's gait control, the exploration of this principle can simplify our understanding of the basic properties of gait.

physics.bio-ph↗