Searcharxiv⌕ Search

arXiv subjects

Shuai Jiang

Publications and source records attributed to Shuai Jiang.

At least 37 records · Page 2Linked to original sources
arXiv2024

A Structure-Preserving Domain Decomposition Method for Data-Driven Modeling

We present a domain decomposition strategy for developing structure-preserving finite element discretizations from data when exact governing equations are unknown. On subdomains, trainable Whitney form elements are used to identify structure-preserving models from data, providing a Dirichlet-to-Neumann map which may be used to globally construct a mortar method. The reduced-order local elements may be trained offline to reproduce high-fidelity Dirichlet data in cases where first principles model derivation is either intractable, unknown, or computationally prohibitive. In such cases, particular care must be taken to preserve structure on both local and mortar levels without knowledge of the governing equations, as well as to ensure well-posedness and stability of the resulting monolithic data-driven system. This strategy provides a flexible means of both scaling to large systems and treating complex geometries, and is particularly attractive for multiscale problems with complex microstructure geometry. While consistency is traditionally obtained in finite element methods via quasi-optimality results and the Bramble-Hilbert lemma as the local element diameter $h\rightarrow0$, our analysis establishes notions of accuracy and stability for finite h with accuracy coming from matching data. Numerical experiments and analysis establish properties for $H(\operatorname{div})$ problems in small data limits ($\mathcal{O}(1)$ reference solutions).

math.NA↗
arXiv2024

A Splice Method for Local-to-Nonlocal Coupling of Weak Forms

We propose a method to couple local and nonlocal diffusion models. By inheriting desirable properties such as patch tests, asymptotic compatibility and unintrusiveness from related splice and optimization-based coupling schemes, it enables the use of weak (or variational) formulations, is computationally efficient and straightforward to implement. We prove well-posedness of the coupling scheme and demonstrate its properties and effectiveness in a variety of numerical examples.

math.NA↗
arXiv2023

Automatic laminectomy cutting plane planning based on artificial intelligence in robot assisted laminectomy surgery

Objective: This study aims to use artificial intelligence to realize the automatic planning of laminectomy, and verify the method. Methods: We propose a two-stage approach for automatic laminectomy cutting plane planning. The first stage was the identification of key points. 7 key points were manually marked on each CT image. The Spatial Pyramid Upsampling Network (SPU-Net) algorithm developed by us was used to accurately locate the 7 key points. In the second stage, based on the identification of key points, a personalized coordinate system was generated for each vertebra. Finally, the transverse and longitudinal cutting planes of laminectomy were generated under the coordinate system. The overall effect of planning was evaluated. Results: In the first stage, the average localization error of the SPU-Net algorithm for the seven key points was 0.65mm. In the second stage, a total of 320 transverse cutting planes and 640 longitudinal cutting planes were planned by the algorithm. Among them, the number of horizontal plane planning effects of grade A, B, and C were 318(99.38%), 1(0.31%), and 1(0.31%), respectively. The longitudinal planning effects of grade A, B, and C were 622(97.18%), 1(0.16%), and 17(2.66%), respectively. Conclusions: In this study, we propose a method for automatic surgical path planning of laminectomy based on the localization of key points in CT images. The results showed that the method achieved satisfactory results. More studies are needed to confirm the reliability of this approach in the future.

eess.IV↗
arXiv2023

GlanceSeg: Real-time microaneurysm lesion segmentation with gaze-map-guided foundation model for early detection of diabetic retinopathy

Early-stage diabetic retinopathy (DR) presents challenges in clinical diagnosis due to inconspicuous and minute microangioma lesions, resulting in limited research in this area. Additionally, the potential of emerging foundation models, such as the segment anything model (SAM), in medical scenarios remains rarely explored. In this work, we propose a human-in-the-loop, label-free early DR diagnosis framework called GlanceSeg, based on SAM. GlanceSeg enables real-time segmentation of microangioma lesions as ophthalmologists review fundus images. Our human-in-the-loop framework integrates the ophthalmologist's gaze map, allowing for rough localization of minute lesions in fundus images. Subsequently, a saliency map is generated based on the located region of interest, which provides prompt points to assist the foundation model in efficiently segmenting microangioma lesions. Finally, a domain knowledge filter refines the segmentation of minute lesions. We conducted experiments on two newly-built public datasets, i.e., IDRiD and Retinal-Lesions, and validated the feasibility and superiority of GlanceSeg through visualized illustrations and quantitative measures. Additionally, we demonstrated that GlanceSeg improves annotation efficiency for clinicians and enhances segmentation performance through fine-tuning using annotations. This study highlights the potential of GlanceSeg-based annotations for self-model optimization, leading to enduring performance advancements through continual learning.

eess.IV↗
arXiv2023

VisionFM: a Multi-Modal Multi-Task Vision Foundation Model for Generalist Ophthalmic Artificial Intelligence

We present VisionFM, a foundation model pre-trained with 3.4 million ophthalmic images from 560,457 individuals, covering a broad range of ophthalmic diseases, modalities, imaging devices, and demography. After pre-training, VisionFM provides a foundation to foster multiple ophthalmic artificial intelligence (AI) applications, such as disease screening and diagnosis, disease prognosis, subclassification of disease phenotype, and systemic biomarker and disease prediction, with each application enhanced with expert-level intelligence and accuracy. The generalist intelligence of VisionFM outperformed ophthalmologists with basic and intermediate levels in jointly diagnosing 12 common ophthalmic diseases. Evaluated on a new large-scale ophthalmic disease diagnosis benchmark database, as well as a new large-scale segmentation and detection benchmark database, VisionFM outperformed strong baseline deep neural networks. The ophthalmic image representations learned by VisionFM exhibited noteworthy explainability, and demonstrated strong generalizability to new ophthalmic modalities, disease spectrum, and imaging devices. As a foundation model, VisionFM has a large capacity to learn from diverse ophthalmic imaging data and disparate datasets. To be commensurate with this capacity, in addition to the real data used for pre-training, we also generated and leveraged synthetic ophthalmic imaging data. Experimental results revealed that synthetic data that passed visual Turing tests, can also enhance the representation learning capability of VisionFM, leading to substantial performance gains on downstream ophthalmic AI tasks. Beyond the ophthalmic AI applications developed, validated, and demonstrated in this work, substantial further applications can be achieved in an efficient and cost-effective manner using VisionFM as the foundation.

eess.IV↗
arXiv2023

A Visual Interpretation-Based Self-Improved Classification System Using Virtual Adversarial Training

The successful application of large pre-trained models such as BERT in natural language processing has attracted more attention from researchers. Since the BERT typically acts as an end-to-end black box, classification systems based on it usually have difficulty in interpretation and low robustness. This paper proposes a visual interpretation-based self-improving classification model with a combination of virtual adversarial training (VAT) and BERT models to address the above problems. Specifically, a fine-tuned BERT model is used as a classifier to classify the sentiment of the text. Then, the predicted sentiment classification labels are used as part of the input of another BERT for spam classification via a semi-supervised training manner using VAT. Additionally, visualization techniques, including visualizing the importance of words and normalizing the attention head matrix, are employed to analyze the relevance of each component to classification accuracy. Moreover, brand-new features will be found in the visual analysis, and classification performance will be improved. Experimental results on Twitter's tweet dataset demonstrate the effectiveness of the proposed model on the classification task. Furthermore, the ablation study results illustrate the effect of different components of the proposed model on the classification results.

cs.CL↗
arXiv2023

Mutually Guided Few-shot Learning for Relational Triple Extraction

Knowledge graphs (KGs), containing many entity-relation-entity triples, provide rich information for downstream applications. Although extracting triples from unstructured texts has been widely explored, most of them require a large number of labeled instances. The performance will drop dramatically when only few labeled data are available. To tackle this problem, we propose the Mutually Guided Few-shot learning framework for Relational Triple Extraction (MG-FTE). Specifically, our method consists of an entity-guided relation proto-decoder to classify the relations firstly and a relation-guided entity proto-decoder to extract entities based on the classified relations. To draw the connection between entity and relation, we design a proto-level fusion module to boost the performance of both entity extraction and relation classification. Moreover, a new cross-domain few-shot triple extraction task is introduced. Extensive experiments show that our method outperforms many state-of-the-art methods by 12.6 F1 score on FewRel 1.0 (single-domain) and 20.5 F1 score on FewRel 2.0 (cross-domain).

cs.CL↗
arXiv2023

Masked Pre-Training of Transformers for Histology Image Analysis

In digital pathology, whole slide images (WSIs) are widely used for applications such as cancer diagnosis and prognosis prediction. Visual transformer models have recently emerged as a promising method for encoding large regions of WSIs while preserving spatial relationships among patches. However, due to the large number of model parameters and limited labeled data, applying transformer models to WSIs remains challenging. Inspired by masked language models, we propose a pretext task for training the transformer model without labeled data to address this problem. Our model, MaskHIT, uses the transformer output to reconstruct masked patches and learn representative histological features based on their positions and visual features. The experimental results demonstrate that MaskHIT surpasses various multiple instance learning approaches by 3% and 2% on survival prediction and cancer subtype classification tasks, respectively. Furthermore, MaskHIT also outperforms two of the most recent state-of-the-art transformer-based methods. Finally, a comparison between the attention maps generated by the MaskHIT model with pathologist's annotations indicates that the model can accurately identify clinically relevant histological structures in each task.

cs.CV↗
arXiv2023

Infinitely many solutions for Kirchhoff equations with indefinite potential

We obtain a sequence of solutions converging to zero for the Kirchhoff equation $$-\left( 1+\int_Ω\left\vert \nabla u\right\vert^2\right) Δu+V(x)u=f(u)\text{,\qquad}u\in H_{0}^{1}(Ω)$$ via truncating technique and a variant of Clark's theorem due to Liu--Wang, where $Ω$ is a bounded smooth domain $Ω\subset\mathbb{R}^N$. Similar result for Schrödinger-Poisson system on a bounded smooth domain $Ω\subset\mathbb{R}^3$ is also presented.

math.AP↗
arXiv2022

1st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Results

The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection. The subchallenges were based on the SeaDronesSee and MODS benchmarks. This report summarizes the main findings of the individual subchallenges and introduces a new benchmark, called SeaDronesSee Object Detection v2, which extends the previous benchmark by including more classes and footage. We provide statistical and qualitative analyses, and assess trends in the best-performing methodologies of over 130 submissions. The methods are summarized in the appendix. The datasets, evaluation code and the leaderboard are publicly available at https://seadronessee.cs.uni-tuebingen.de/macvi.

cs.CV↗
arXiv2022

Interactive Physically-Based Simulation of Roadheader Robot

Roadheader is an engineering robot widely used in underground engineering and mining industry. Interactive dynamics simulation of roadheader is a fundamental problem in unmanned excavation and virtual reality training. However, current research is only based on traditional animation techniques or commercial game engines. There are few studies that apply real-time physical simulation of computer graphics to the field of roadheader robot. This paper aims to present an interactive physically-based simulation system of roadheader robot. To this end, an improved multibody simulation method based on generalized coordinates is proposed. First, our simulation method describes robot dynamics based on generalized coordinates. Compared to state-of-the-art methods, our method is more stable and accurate. Numerical simulation results showed that our method has significantly less error than the game engine in the same number of iterations. Second, we adopt the symplectic Euler integrator instead of the conventional fourth-order Runge-Kutta (RK4) method for dynamics iteration. Compared with other integrators, our method is more stable in energy drift during long-term simulation. The test results showed that our system achieved real-time interaction performance of 60 frames per second (fps). Furthermore, we propose a model format for geometric and robotics modeling of roadheaders to implement the system. Our interactive simulation system of roadheader meets the requirements of interactivity, accuracy and stability.

cs.RO↗
arXiv2021

Towards fully ab initio simulation of atmospheric aerosol nucleation

Atmospheric aerosol nucleation contributes to more than half of cloud condensation nuclei globally. The emissions, properties and concentrations of atmospheric aerosols or aerosol precursors could respond significantly to climate change. Despite the importance for climate, the detailed nucleation mechanisms are still poorly understood. The ultimate goal of theoretical understanding aerosol nucleation is to simulate nucleation in ambient condition, hindered by lack of accurate reactive force field. Here we propose the reactive force field for nucleation systems with good size scalability based on deep neural network. The huge computational costs from direct molecular dynamics in ambient conditions are surmounted by bridging the simulation in the limited box with cluster kinetics, facilitating the aerosol nucleation simulation to be fully ab initio. We found that the acid-base formation rates previously based on hard sphere collision rate constants tend to be underestimated up to several times. These findings show that the widely recognized acid-base nucleation observed in the CLOUD (Cosmics Leaving OUtdoor Droplets) chamber experiments, pristine and polluted environments should be revisited to considering the contribution of collision enhancement. Besides, the framework here is transferable to other nucleation systems, potentially boosting the nucleation parameterizations accuracy generally to effectively advance the climate model predictions reliability.

physics.atm-clus↗
arXiv2021

MHAttnSurv: Multi-Head Attention for Survival Prediction Using Whole-Slide Pathology Images

In pathology, whole-slide images (WSI) based survival prediction has attracted increasing interest. However, given the large size of WSIs and the lack of pathologist annotations, extracting the prognostic information from WSIs remains a challenging task. Previous studies have used multiple instance learning approaches to combine the information from multiple randomly sampled patches, but different visual patterns may contribute differently to prognosis prediction. In this study, we developed a multi-head attention approach to focus on various parts of a tumor slide, for more comprehensive information extraction from WSIs. We evaluated our approach on four cancer types from The Cancer Genome Atlas database. Our model achieved an average c-index of 0.640, outperforming two existing state-of-the-art approaches for WSI-based survival prediction, which have an average c-index of 0.603 and 0.619 on these datasets. Visualization of our attention maps reveals each attention head focuses synergistically on different morphological patterns.

eess.IV↗
arXiv2021

Unsupervised clothing change adaptive person ReID

Clothing changes and lack of data labels are both crucial challenges in person ReID. For the former challenge, people may occur multiple times at different locations wearing different clothing. However, most of the current person ReID research works focus on the benchmarks in which a person's clothing is kept the same all the time. For the last challenge, some researchers try to make model learn information from a labeled dataset as a source to an unlabeled dataset. Whereas purely unsupervised training is less used. In this paper, we aim to solve both problems at the same time. We design a novel unsupervised model, Sync-Person-Cloud ReID, to solve the unsupervised clothing change person ReID problem. We developer a purely unsupervised clothing change person ReID pipeline with person sync augmentation operation and same person feature restriction. The person sync augmentation is to supply additional same person resources. These same person's resources can be used as part supervised input by same person feature restriction. The extensive experiments on clothing change ReID datasets show the out-performance of our methods.

cs.CV↗
arXiv2021

Resolution-invariant Person ReID Based on Feature Transformation and Self-weighted Attention

Person Re-identification (ReID) is a critical computer vision task which aims to match the same person in images or video sequences. Most current works focus on settings where the resolution of images is kept the same. However, the resolution is a crucial factor in person ReID, especially when the cameras are at different distances from the person or the camera's models are different from each other. In this paper, we propose a novel two-stream network with a lightweight resolution association ReID feature transformation (RAFT) module and a self-weighted attention (SWA) ReID module to evaluate features under different resolutions. RAFT transforms the low resolution features to corresponding high resolution features. SWA evaluates both features to get weight factors for the person ReID. Both modules are jointly trained to get a resolution-invariant representation. Extensive experiments on five benchmark datasets show the effectiveness of our method. For instance, we achieve Rank-1 accuracy of 43.3% and 83.2% on CAVIAR and MLR-CUHK03, outperforming the state-of-the-art.

cs.CV↗
arXiv2020

Extended Self-similar Solution for Circumstellar Material-Supernova Ejecta Interaction

In this note, we present a detailed self-similar solution to the interaction of a uniformly expanding gas and a stationary ambient medium, with an application to supernovae interacting with preexisting circumstellar media (Type IIn SNe). We implement the generalized solution into the Modular Open Source Fitter for Transients (MOSFiT), an open-source Python package for fitting extragalactic transient light curves.

astro-ph.HE↗
arXiv2020

RGB-IR Cross-modality Person ReID based on Teacher-Student GAN Model

RGB-Infrared (RGB-IR) person re-identification (ReID) is a technology where the system can automatically identify the same person appearing at different parts of a video when light is unavailable. The critical challenge of this task is the cross-modality gap of features under different modalities. To solve this challenge, we proposed a Teacher-Student GAN model (TS-GAN) to adopt different domains and guide the ReID backbone to learn better ReID information. (1) In order to get corresponding RGB-IR image pairs, the RGB-IR Generative Adversarial Network (GAN) was used to generate IR images. (2) To kick-start the training of identities, a ReID Teacher module was trained under IR modality person images, which is then used to guide its Student counterpart in training. (3) Likewise, to better adapt different domain features and enhance model ReID performance, three Teacher-Student loss functions were used. Unlike other GAN based models, the proposed model only needs the backbone module at the test stage, making it more efficient and resource-saving. To showcase our model's capability, we did extensive experiments on the newly-released SYSU-MM01 RGB-IR Re-ID benchmark and achieved superior performance to the state-of-the-art with 49.8% Rank-1 and 47.4% mAP.

cs.CV↗