arXiv subjects

Jiajia Luo

Publications and source records attributed to Jiajia Luo.

17 recordsLinked to original sources

arXiv2026

High-Fidelity 3D Geometric Reconstruction of Pelvic Organs from MRI: A Hybrid Deep Learning and Iterative Optimization Approach

Patient-specific 3D reconstruction of pelvic organ geometry from MRI is important for pelvic floor modeling and downstream patient-specific analysis. However, while previous studies have focused primarily on either image segmentation or downstream use of 3D models, the reconstruction of high-fidelity, high-quality geometries remains labor-intensive and poorly standardized. The study introduced a hybrid deformable shape modeling framework that integrates deep learning prediction with iterative optimization for the reconstruction of the bladder, uterus, and rectum. The framework consists of three core components: a geometry-aware multi-level deep learning architecture that preserves topological consistency of pelvic organs; a two-stage amortized optimization training strategy that balances global shape capture and local surface refinement; and a holistic synergy mechanism--where iterative optimization provides supervision for deep learning during the training phase, and during inference, deep learning rapidly predicts the global organ morphology, followed by iterative optimization to refine local surfaces and mesh quality. This framework demonstrated marked superiority in geometric fidelity than current mainstream deep learning-based organ reconstruction models. For individual anatomical structures, the reconstructed 3D geometries for the bladder, rectum, and uterus achieved significantly lower Chamfer Distance values and higher Dice Similarity Coefficient scores. In addition, while maintaining high computational efficiency, the proposed architecture yielded superior overall volumetric mesh quality. At the patient level, the framework achieved higher mean values for the 10 worst elements for both minSICN and minSIGE compared to traditional geometric post-processing algorithms.

cs.CV↗

arXiv2026

Evaluating Deep Surrogate Models for Knee Joint Contact Mechanics Under Input-Limited Conditions

Background and Objective: Accurate surrogate modeling of knee joint contact mechanics is important for reconstructing stress distributions and identifying risk-relevant regions, yet the relative suitability of different modeling paradigms under practically relevant input-limited conditions remains unclear. Methods: Nine male soccer players performed 90° change-of-direction trials. Finite element simulations driven by subject-specific joint posture and reaction forces were converted into graph-structured samples. Five surrogate architectures representing local diffusion, history-context enhancement, hierarchical multi-scale modeling, explicit global interaction, and local-global hybridization were compared using three-fold cross-subject validation under full, pose-corrupted, load-corrupted, and minimal-input conditions. Performance was evaluated using full-field error, high-stress error, high-risk region overlap, and hotspot localization metrics. Results: The hybrid model achieved the best overall performance under full inputs and remained the most robust under pose- and load-corrupted conditions. Under minimal inputs, no single model dominated all metrics: the history-context model yielded lower overall and high-stress errors, the hybrid model better preserved high-risk region reconstruction, and the hierarchical model showed an advantage in hotspot localization. Conclusion: Evaluation of surrogate models for knee joint contact mechanics should shift from accuracy comparisons under ideal inputs to a comprehensive assessment of the preservation of risk-relevant information under realistic input constraints. Although the local-global hybrid model showed the best overall robustness, the optimal model under minimal-input conditions remained task-dependent.

q-bio.QM↗

arXiv2026

Characterizing Long-Range Dependencies in Knee Joint Contact Mechanics: A Comparison of Topology Diffusion, Global Routing, and Hybrid Graph Neural Networks

Finite element analysis of knee joint contact mechanics is computationally expensive, which has motivated the development of graph neural network surrogate models. However, effectively representing long-range dependencies in joint mechanical responses remains challenging. This study systematically compared topology diffusion, global routing, and their hybridization for surrogate modeling of knee joint contact mechanics. Using kinematic and force data from nine soccer players performing change-of-direction maneuvers, finite element simulations were used to generate graph-structured samples for training and evaluation under a grouped three-fold cross-subject evaluation framework. Five architectures were compared: standard MeshGraphNet, hierarchical MeshGraphNet, a routing-only transformer, a topology-biased routing transformer, and a hybrid model. The hybrid model achieved the best overall performance, yielding the lowest full-field error and peak stress error, together with the highest spatial agreement for high-risk regions. Among the non-hybrid models, the standard topology-diffusion model performed best overall, whereas routing-only strategies were less effective. These findings indicate that topology diffusion provides a robust basis for surrogate modeling of knee joint contact mechanics within the present benchmark, while the addition of global routing can further improve reconstruction of clinically relevant high-stress patterns.

q-bio.QM↗

arXiv2026

Towards Structure-Aware Surrogate Modeling: Explicit Region Interaction Improves Knee Contact Stress Prediction

Knee contact-stress hotspots are closely linked to meniscal/cartilage injury risk. Still, high-fidelity subject-specific FEA is too computationally expensive for large-cohort, multi-condition, near-real-time use. Existing MeshGraphNet-style surrogates mainly rely on stacked local message passing, which is often insufficient for modeling long-range dependencies and limits interpretability. This study benchmarked a deep-stacked baseline model against three explicit region-interaction architectures. Using a 90° change-of-direction task and a strict cross-subject evaluation framework, we assessed whole-field error, peak stress fidelity, and hotspot spatial consistency under matched computational budgets. Region-interaction models significantly reduced whole-field nodal stress errors compared to the purely stacked baseline. Crucially, they achieved markedly higher accuracy in reconstructing the high-stress tail and demonstrated superior spatial consistency and temporal robustness in localizing high-risk stress hotspots. Explicit region-level interaction provides a more structure-aligned surrogate modeling paradigm for knee contact mechanics and yields stronger risk-relevant stress phenotype recovery under comparable computational budgets, while supporting more interpretable injury-risk assessment.

q-bio.TO↗

arXiv2026

Disentangling History and Propagation Dependencies in Cross-Subject Knee Contact Stress Prediction Using a Shared MeshGraphNet Backbone

Background:Subject-specific finite element analysis accurately characterizes knee joint mechanics but is computationally expensive. Deep surrogate models provide a rapid alternative, yet their generalization across subjects under limited pose and load inputs remains unclear. It remains unclear whether the dominant source of prediction uncertainty arises from temporal history dependence or spatial propagation dependence. Methods:To disentangle these factors, we employed a shared MGN backbone with a fixed mesh topology. A dataset of running trials from nine subjects was constructed using an OpenSim-FEBio workflow. We developed four model variants to isolate specific dependencies: (1) a baseline MGN; (2) CT-MGN, incorporating a Control Transformer to encode short-horizon history; (3) MsgModMGN, applying state-conditioned modulation to message passing for adaptive propagation; (4) CT-MsgModMGN, combining both mechanisms. Models were evaluated using a rigorous grouped 3-fold cross-validation on unseen subjects.Results:The models incorporating history encoding significantly outperformed the baseline MGN and MsgModMGN in global accuracy and spatial consistency. Crucially, the CT module effectively mitigated the peak-shaving defect common in deep surrogates, significantly reducing peak stress prediction errors. In contrast, the spatial propagation modulation alone yielded no significant improvement over the baseline, and combining it with CT provided no additional benefit.Conclusion:Temporal history dependence, rather than spatial propagation modulation, is the primary driver of prediction uncertainty in cross-subject knee contact mechanics. Explicitly encoding short-horizon driver sequences enables the surrogate model to recover implicit phase information, thereby achieving superior fidelity in peak-stress capture and high-risk localization compared to purely state-based approaches.

q-bio.QM↗

arXiv2025

GNN-Based Deep Surrogate Modeling of Knee Contact Mechanics: Generalizing Neuromuscular Control Patterns Across Subjects

Background: Accumulation of abnormal contact stress is a primary biomechanical driver of acute meniscal tears and chronic osteoarthritis. While Finite Element Analysis (FEA) provides the necessary fidelity to quantify these injury-inducing loads, its high computational cost precludes clinical utility. Emerging deep surrogate models promise real-time assessment but suffer a critical blind spot: they predominantly focus on learning anatomical variations, largely overlooking the neuromuscular control patterns. These dynamic, subject-specific motor strategies fundamentally dictate potentially injurious stress distributions inside the knee. Methods: This study investigates the generalization capability of the topology-aware MeshGraphNet regarding cross-subject neuromuscular control patterns under fixed anatomical conditions. We constructed a dataset using gait data from nine subjects via an OpenSim-FEBio co-simulation platform. The MGN was compared against a structure-agnostic Node-wise MLP using a rigorous grouped 3-fold cross-validation on unseen subjects. Results: The MGN demonstrated superior fidelity, achieving a correlation of 0.94 with ground truth (vs. 0.88 for MLP). In contrast to the MLP, which exhibited the "peak shaving" defect common in deep learning, MGN significantly reduced peak-stress prediction errors and achieved higher spatial overlap in high-risk regions. This indicates that MGN effectively captured the non-local force-transmission pathways unique to each subject's movement strategy. Conclusion: By mimicking the propagation of physical stress through message passing, MGN successfully decodes the heterogeneity of human neuromuscular control, even under fixed anatomy. This establishes GNNs as robust clinical tools capable of identifying functional injury risks that are invisible to purely geometry-based surrogate models.

q-bio.QM↗

arXiv2023

ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation

Large-scale Pre-Training Vision-Language Model such as CLIP has demonstrated outstanding performance in zero-shot classification, e.g. achieving 76.3% top-1 accuracy on ImageNet without seeing any example, which leads to potential benefits to many tasks that have no labeled data. However, while applying CLIP to a downstream target domain, the presence of visual and text domain gaps and cross-modality misalignment can greatly impact the model performance. To address such challenges, we propose ReCLIP, the first source-free domain adaptation method for vision-language models, which does not require any source data or target labeled data. ReCLIP first learns a projection space to mitigate the misaligned visual-text embeddings and learns pseudo labels, and then deploys cross-modality self-training with the pseudo labels, to update visual and text encoders, refine labels and reduce domain gaps and misalignments iteratively. With extensive experiments, we demonstrate ReCLIP reduces the average error rate of CLIP from 30.17% to 25.06% on 22 image classification benchmarks. Code available at https://github.com/michiganleon/ReCLIP_WACV.

cs.CV↗

arXiv2023

Pelvic floor MRI segmentation based on semi-supervised deep learning

The semantic segmentation of pelvic organs via MRI has important clinical significance. Recently, deep learning-enabled semantic segmentation has facilitated the three-dimensional geometric reconstruction of pelvic floor organs, providing clinicians with accurate and intuitive diagnostic results. However, the task of labeling pelvic floor MRI segmentation, typically performed by clinicians, is labor-intensive and costly, leading to a scarcity of labels. Insufficient segmentation labels limit the precise segmentation and reconstruction of pelvic floor organs. To address these issues, we propose a semi-supervised framework for pelvic organ segmentation. The implementation of this framework comprises two stages. In the first stage, it performs self-supervised pre-training using image restoration tasks. Subsequently, fine-tuning of the self-supervised model is performed, using labeled data to train the segmentation model. In the second stage, the self-supervised segmentation model is used to generate pseudo labels for unlabeled data. Ultimately, both labeled and unlabeled data are utilized in semi-supervised training. Upon evaluation, our method significantly enhances the performance in the semantic segmentation and geometric reconstruction of pelvic organs, Dice coefficient can increase by 2.65% averagely. Especially for organs that are difficult to segment, such as the uterus, the accuracy of semantic segmentation can be improved by up to 3.70%.

cs.CV↗

arXiv2023

Frequency Domain Decomposition Translation for Enhanced Medical Image Translation Using GANs

Medical Image-to-image translation is a key task in computer vision and generative artificial intelligence, and it is highly applicable to medical image analysis. GAN-based methods are the mainstream image translation methods, but they often ignore the variation and distribution of images in the frequency domain, or only take simple measures to align high-frequency information, which can lead to distortion and low quality of the generated images. To solve these problems, we propose a novel method called frequency domain decomposition translation (FDDT). This method decomposes the original image into a high-frequency component and a low-frequency component, with the high-frequency component containing the details and identity information, and the low-frequency component containing the style information. Next, the high-frequency and low-frequency components of the transformed image are aligned with the transformed results of the high-frequency and low-frequency components of the original image in the same frequency band in the spatial domain, thus preserving the identity information of the image while destroying as little stylistic information of the image as possible. We conduct extensive experiments on MRI images and natural images with FDDT and several mainstream baseline models, and we use four evaluation metrics to assess the quality of the generated images. Compared with the baseline models, optimally, FDDT can reduce Fréchet inception distance by up to 24.4%, structural similarity by up to 4.4%, peak signal-to-noise ratio by up to 5.8%, and mean squared error by up to 31%. Compared with the previous method, optimally, FDDT can reduce Fréchet inception distance by up to 23.7%, structural similarity by up to 1.8%, peak signal-to-noise ratio by up to 6.8%, and mean squared error by up to 31.6%.

eess.IV↗

arXiv2023

CameraPose: Weakly-Supervised Monocular 3D Human Pose Estimation by Leveraging In-the-wild 2D Annotations

To improve the generalization of 3D human pose estimators, many existing deep learning based models focus on adding different augmentations to training poses. However, data augmentation techniques are limited to the "seen" pose combinations and hard to infer poses with rare "unseen" joint positions. To address this problem, we present CameraPose, a weakly-supervised framework for 3D human pose estimation from a single image, which can not only be applied on 2D-3D pose pairs but also on 2D alone annotations. By adding a camera parameter branch, any in-the-wild 2D annotations can be fed into our pipeline to boost the training diversity and the 3D poses can be implicitly learned by reprojecting back to 2D. Moreover, CameraPose introduces a refinement network module with confidence-guided loss to further improve the quality of noisy 2D keypoints extracted by 2D pose estimators. Experimental results demonstrate that the CameraPose brings in clear improvements on cross-scenario datasets. Notably, it outperforms the baseline method by 3mm on the most challenging dataset 3DPW. In addition, by combining our proposed refinement network module with existing 3D pose estimators, their performance can be improved in cross-scenario evaluation.

cs.CV↗

arXiv2022

De-Noising of Photoacoustic Microscopy Images by Deep Learning

As a hybrid imaging technology, photoacoustic microscopy (PAM) imaging suffers from noise due to the maximum permissible exposure of laser intensity, attenuation of ultrasound in the tissue, and the inherent noise of the transducer. De-noising is a post-processing method to reduce noise, and PAM image quality can be recovered. However, previous de-noising techniques usually heavily rely on mathematical priors as well as manually selected parameters, resulting in unsatisfactory and slow de-noising performance for different noisy images, which greatly hinders practical and clinical applications. In this work, we propose a deep learning-based method to remove complex noise from PAM images without mathematical priors and manual selection of settings for different input images. An attention enhanced generative adversarial network is used to extract image features and remove various noises. The proposed method is demonstrated on both synthetic and real datasets, including phantom (leaf veins) and in vivo (mouse ear blood vessels and zebrafish pigment) experiments. The results show that compared with previous PAM de-noising methods, our method exhibits good performance in recovering images qualitatively and quantitatively. In addition, the de-noising speed of 0.016 s is achieved for an image with $256\times256$ pixels. Our approach is effective and practical for the de-noising of PAM images.

eess.IV↗

arXiv2020

MEBOW: Monocular Estimation of Body Orientation In the Wild

Body orientation estimation provides crucial visual cues in many applications, including robotics and autonomous driving. It is particularly desirable when 3-D pose estimation is difficult to infer due to poor image resolution, occlusion or indistinguishable body parts. We present COCO-MEBOW (Monocular Estimation of Body Orientation in the Wild), a new large-scale dataset for orientation estimation from a single in-the-wild image. The body-orientation labels for around 130K human bodies within 55K images from the COCO dataset have been collected using an efficient and high-precision annotation pipeline. We also validated the benefits of the dataset. First, we show that our dataset can substantially improve the performance and the robustness of a human body orientation estimation model, the development of which was previously limited by the scale and diversity of the available training data. Additionally, we present a novel triple-source solution for 3-D human pose estimation, where 3-D pose labels, 2-D pose labels, and our body-orientation labels are all used in joint training. Our model significantly outperforms state-of-the-art dual-source solutions for monocular 3-D human pose estimation, where training only uses 3-D pose labels and 2-D pose labels. This substantiates an important advantage of MEBOW for 3-D human pose estimation, which is particularly appealing because the per-instance labeling cost for body orientations is far less than that for 3-D poses. The work demonstrates high potential of MEBOW in addressing real-world challenges involving understanding human behaviors. Further information of this work is available at https://chenyanwu.github.io/MEBOW/.

cs.CV↗

arXiv2020

Adherent Mist and Raindrop Removal from a Single Image Using Attentive Convolutional Network

Temperature difference-induced mist adhered to the glass, such as windshield, camera lens, is often inhomogeneous and obscure, easily obstructing the vision and severely degrading the image. Together with adherent raindrops, they bring considerable challenges to various vision systems but without enough attention. Recent methods for other similar problems typically use hand-crafted priors to generate spatial attention maps. In this work, we newly present a problem of image degradation caused by adherent mist and raindrops. An attentive convolutional network is adopted to visually remove the adherent mist and raindrop from a single image. A baseline architecture with general channel-wise attention, spatial attention, and multilevel feature fusion is used. Considering the variations and regional characteristics of adherent mist and raindrops, we apply interpolation-based pyramid-attention blocks to perceive spatial information at different scales. Experiments show that the proposed method can improve severely degraded images' visibility, both qualitatively and quantitatively. More applied experiments show that this underrated practical problem is critical to high-level vision scenes. Our method also achieves state-of-the-art performance on conventional dehazing and pure de-raindrop problems, in addition to our task of handling adherent mist and raindrops.

cs.CV↗

arXiv2020

Photoacoustic Microscopy with Sparse Data Enabled by Convolutional Neural Networks for Fast Imaging

Photoacoustic microscopy (PAM) has been a promising biomedical imaging technology in recent years. However, the point-by-point scanning mechanism results in low-speed imaging, which limits the application of PAM. Reducing sampling density can naturally shorten image acquisition time, which is at the cost of image quality. In this work, we propose a method using convolutional neural networks (CNNs) to improve the quality of sparse PAM images, thereby speeding up image acquisition while keeping good image quality. The CNN model utilizes both squeeze-and-excitation blocks and residual blocks to achieve the enhancement, which is a mapping from a 1/4 or 1/16 low-sampling sparse PAM image to a latent fully-sampled image. The perceptual loss function is applied to keep the fidelity of images. The model is mainly trained and validated on PAM images of leaf veins. The experiments show the effectiveness of our proposed method, which significantly outperforms existing methods quantitatively and qualitatively. Our model is also tested using in vivo PAM images of blood vessels of mouse ears and eyes. The results show that the model can enhance the image quality of the sparse PAM image of blood vessels from several aspects, which may help fast PAM and facilitate its clinical applications.

eess.IV↗

arXiv2019

Adaptive Weighting Depth-variant Deconvolution of Fluorescence Microscopy Images with Convolutional Neural Network

Fluorescence microscopy plays an important role in biomedical research. The depth-variant point spread function (PSF) of a fluorescence microscope produces low-quality images especially in the out-of-focus regions of thick specimens. Traditional deconvolution to restore the out-of-focus images is usually insufficient since a depth-invariant PSF is assumed. This article aims at handling fluorescence microscopy images by learning-based depth-variant PSF and reducing artifacts. We propose adaptive weighting depth-variant deconvolution (AWDVD) with defocus level prediction convolutional neural network (DelpNet) to restore the out-of-focus images. Depth-variant PSFs of image patches can be obtained by DelpNet and applied in the afterward deconvolution. AWDVD is adopted for a whole image which is patch-wise deconvolved and appropriately cropped before deconvolution. DelpNet achieves the accuracy of 98.2%, which outperforms the best-ever one using the same microscopy dataset. Image patches of 11 defocus levels after deconvolution are validated with maximum improvement in the peak signal-to-noise ratio and structural similarity index of 6.6 dB and 11%, respectively. The adaptive weighting of the patch-wise deconvolved image can eliminate patch boundary artifacts and improve deconvolved image quality. The proposed method can accurately estimate depth-variant PSF and effectively recover out-of-focus microscopy images. To our acknowledge, this is the first study of handling out-of-focus microscopy images using learning-based depth-variant PSF. Facing one of the most common blurs in fluorescence microscopy, the novel method provides a practical technology to improve the image quality.

eess.IV↗

arXiv2019

CSSegNet: Fine-Grained Cardiac Structures Segmentation Using Dilated Pyramid Pooling in U-net

Cardiac structure segmentation plays an important role in medical analysis procedures. Images' blurred boundaries issue always limits the segmentation performance. To address this difficult problem, we presented a novel network structure which embedded dilated pyramid pooling block in the skip connections between networks' encoding and decoding stage. A dilated pyramid pooling block is made up of convolutions and pooling operations with different vision scopes. Equipped the model with such module, it could be endowed with multi-scales vision ability. Together combining with other techniques, it included a multi-scales initial features extraction and a multi-resolutions' prediction aggregation module. As for backbone feature extraction network, we referred to the basic idea of Xception network which benefited from separable convolutions. Evaluated on the Post 2017 MICCAI-ACDC challenge phase data, our proposed model could achieve state-of-the-art performance in left ventricle (LVC) cavities and right ventricle cavities (RVC) segmentation tasks. Results revealed that our method has advantages on both geometrical (Dice coefficient, Hausdorff distance) and clinical evaluation (Ejection Fraction, Volume), which represent closer boundaries and more statistically significant separately.

cs.CV↗

arXiv2018

Self-Aligned van der Waals Heterojunction Diodes and Transistors

A general self-aligned fabrication scheme is reported here for a diverse class of electronic devices based on van der Waals materials and heterojunctions. In particular, self-alignment enables the fabrication of source-gated transistors in monolayer MoS2 with near-ideal current saturation characteristics and channel lengths down to 135 nm. Furthermore, self-alignment of van der Waals p-n heterojunction diodes achieves complete electrostatic control of both the p-type and n-type constituent semiconductors in a dual-gated geometry, resulting in gate-tunable mean and variance of anti-ambipolar Gaussian characteristics. Through finite-element device simulations, the operating principles of source-gated transistors and dual-gated anti-ambipolar devices are elucidated, thus providing design rules for additional devices that employ self-aligned geometries. For example, the versatility of this scheme is demonstrated via contact-doped MoS2 homojunction diodes and mixed-dimensional heterojunctions based on organic semiconductors. The scalability of this approach is also shown by fabricating self-aligned short-channel transistors with sub-diffraction channel lengths in the range of 150 nm to 800 nm using photolithography on large-area MoS2 films grown by chemical vapor deposition. Overall, this self-aligned fabrication method represents an important step towards the scalable integration of van der Waals heterojunction devices into more sophisticated circuits and systems.

physics.app-ph↗