arXiv subjects

Cheng Jiang

Publications and source records attributed to Cheng Jiang.

At least 55 records · Page 3Linked to original sources

arXiv2022

OpenSRH: optimizing brain tumor surgery using intraoperative stimulated Raman histology

Accurate intraoperative diagnosis is essential for providing safe and effective care during brain tumor surgery. Our standard-of-care diagnostic methods are time, resource, and labor intensive, which restricts access to optimal surgical treatments. To address these limitations, we propose an alternative workflow that combines stimulated Raman histology (SRH), a rapid optical imaging method, with deep learning-based automated interpretation of SRH images for intraoperative brain tumor diagnosis and real-time surgical decision support. Here, we present OpenSRH, the first public dataset of clinical SRH images from 300+ brain tumors patients and 1300+ unique whole slide optical images. OpenSRH contains data from the most common brain tumors diagnoses, full pathologic annotations, whole slide tumor segmentations, raw and processed optical imaging data for end-to-end model development and validation. We provide a framework for patch-based whole slide SRH classification and inference using weak (i.e. patient-level) diagnostic labels. Finally, we benchmark two computer vision tasks: multiclass histologic brain tumor classification and patch-based contrastive representation learning. We hope OpenSRH will facilitate the clinical translation of rapid optical imaging and real-time ML-based surgical decision support in order to improve the access, safety, and efficacy of cancer surgery in the era of precision medicine. Dataset access, code, and benchmarks are available at opensrh.mlins.org.

eess.IV↗

arXiv2022

Rapid Automated Analysis of Skull Base Tumor Specimens Using Intraoperative Optical Imaging and Artificial Intelligence

Background: Accurate diagnosis of skull base tumors is essential for providing personalized surgical treatment strategies. Intraoperative diagnosis can be challenging due to tumor diversity and lack of intraoperative pathology resources. Objective: To develop an independent and parallel intraoperative pathology workflow that can provide rapid and accurate skull base tumor diagnoses using label-free optical imaging and artificial intelligence. Method: We used a fiber laser-based, label-free, non-consumptive, high-resolution microscopy method ($<$ 60 sec per 1 $\times$ 1 mm$^\text{2}$), called stimulated Raman histology (SRH), to image a consecutive, multicenter cohort of skull base tumor patients. SRH images were then used to train a convolutional neural network (CNN) model using three representation learning strategies: cross-entropy, self-supervised contrastive learning, and supervised contrastive learning. Our trained CNN models were tested on a held-out, multicenter SRH dataset. Results: SRH was able to image the diagnostic features of both benign and malignant skull base tumors. Of the three representation learning strategies, supervised contrastive learning most effectively learned the distinctive and diagnostic SRH image features for each of the skull base tumor types. In our multicenter testing set, cross-entropy achieved an overall diagnostic accuracy of 91.5%, self-supervised contrastive learning 83.9%, and supervised contrastive learning 96.6%. Our trained model was able to identify tumor-normal margins and detect regions of microscopic tumor infiltration in whole-slide SRH images. Conclusion: SRH with trained artificial intelligence models can provide rapid and accurate intraoperative analysis of skull base tumor specimens to inform surgical decision-making.

cs.CV↗

arXiv2022

3D Shuffle-Mixer: An Efficient Context-Aware Vision Learner of Transformer-MLP Paradigm for Dense Prediction in Medical Volume

Dense prediction in medical volume provides enriched guidance for clinical analysis. CNN backbones have met bottleneck due to lack of long-range dependencies and global context modeling power. Recent works proposed to combine vision transformer with CNN, due to its strong global capture ability and learning capability. However, most works are limited to simply applying pure transformer with several fatal flaws (i.e., lack of inductive bias, heavy computation and little consideration for 3D data). Therefore, designing an elegant and efficient vision transformer learner for dense prediction in medical volume is promising and challenging. In this paper, we propose a novel 3D Shuffle-Mixer network of a new Local Vision Transformer-MLP paradigm for medical dense prediction. In our network, a local vision transformer block is utilized to shuffle and learn spatial context from full-view slices of rearranged volume, a residual axial-MLP is designed to mix and capture remaining volume context in a slice-aware manner, and a MLP view aggregator is employed to project the learned full-view rich context to the volume feature in a view-aware manner. Moreover, an Adaptive Scaled Enhanced Shortcut is proposed for local vision transformer to enhance feature along spatial and channel dimensions adaptively, and a CrossMerge is proposed to skip-connects the multi-scale feature appropriately in the pyramid architecture. Extensive experiments demonstrate the proposed model outperforms other state-of-the-art medical dense prediction methods.

cs.CV↗

arXiv2022

MixFormer: End-to-End Tracking with Iterative Mixed Attention

Tracking often uses a multi-stage pipeline of feature extraction, target information integration, and bounding box estimation. To simplify this pipeline and unify the process of feature extraction and target information integration, we present a compact tracking framework, termed as MixFormer, built upon transformers. Our core design is to utilize the flexibility of attention operations, and propose a Mixed Attention Module (MAM) for simultaneous feature extraction and target information integration. This synchronous modeling scheme allows to extract target-specific discriminative features and perform extensive communication between target and search area. Based on MAM, we build our MixFormer tracking framework simply by stacking multiple MAMs with progressive patch embedding and placing a localization head on top. In addition, to handle multiple target templates during online tracking, we devise an asymmetric attention scheme in MAM to reduce computational cost, and propose an effective score prediction module to select high-quality templates. Our MixFormer sets a new state-of-the-art performance on five tracking benchmarks, including LaSOT, TrackingNet, VOT2020, GOT-10k, and UAV123. In particular, our MixFormer-L achieves NP score of 79.9% on LaSOT, 88.9% on TrackingNet and EAO of 0.555 on VOT2020. We also perform in-depth ablation studies to demonstrate the effectiveness of simultaneous feature extraction and information integration. Code and trained models are publicly available at https://github.com/MCG-NJU/MixFormer.

cs.CV↗

arXiv2021

Fusion of medical imaging and electronic health records with attention and multi-head machanisms

Doctors often make diagonostic decisions based on patient's image scans, such as magnetic resonance imaging (MRI), and patient's electronic health records (EHR) such as age, gender, blood pressure and so on. Despite a lot of automatic methods have been proposed for either image or text analysis in computer vision or natural language research areas, much fewer studies have been developed for the fusion of medical image and EHR data for medical problems. Among existing early or intermediate fusion methods, concatenation of features from both modalities is still a mainstream. For a better exploiting of image and EHR data, we propose a multi-modal attention module which use EHR data to help the selection of important regions during image feature extraction process conducted by traditional CNN. Moreover, we propose to incorporate multi-head machnism to gated multimodal unit (GMU) to make it able to parallelly fuse image and EHR features in different subspaces. With the help of the two modules, existing CNN architecture can be enhanced using both modalities. Experiments on predicting Glasgow outcome scale (GOS) of intracerebral hemorrhage patients and classifying Alzheimer's Disease showed the proposed method can automatically focus on task-related areas and achieve better results by making better use of image and EHR features.

cs.CV↗

arXiv2021

Fully Convolutional Online Tracking

Online learning has turned out to be effective for improving tracking performance. However, it could be simply applied for classification branch, but still remains challenging to adapt to regression branch due to its complex design and intrinsic requirement for high-quality online samples. To tackle this issue, we present the fully convolutional online tracking framework, coined as FCOT, and focus on enabling online learning for both classification and regression branches by using a target filter based tracking paradigm. Our key contribution is to introduce an online regression model generator (RMG) for initializing weights of the target filter with online samples and then optimizing this target filter weights based on the groundtruth samples at the first frame. Based on the online RGM, we devise a simple anchor-free tracker (FCOT), composed of a feature backbone, an up-sampling decoder, a multi-scale classification branch, and a multi-scale regression branch. Thanks to the unique design of RMG, our FCOT can not only be more effective in handling target variation along temporal dimension thus generating more precise results, but also overcome the issue of error accumulation during the tracking procedure. In addition, due to its simplicity in design, our FCOT could be trained and deployed in a fully convolutional manner with a real-time running speed. The proposed FCOT achieves the state-of-the-art performance on seven benchmarks, including VOT2018, LaSOT, TrackingNet, GOT-10k, OTB100, UAV123, and NFS. Code and models of our FCOT have been released at: \url{https://github.com/MCG-NJU/FCOT}.

cs.CV↗

arXiv2021

Two-stream Convolutional Networks for Multi-frame Face Anti-spoofing

Face anti-spoofing is an important task to protect the security of face recognition. Most of previous work either struggle to capture discriminative and generalizable feature or rely on auxiliary information which is unavailable for most of industrial product. Inspired by the video classification work, we propose an efficient two-stream model to capture the key differences between live and spoof faces, which takes multi-frames and RGB difference as input respectively. Feature pyramid modules with two opposite fusion directions and pyramid pooling modules are applied to enhance feature representation. We evaluate the proposed method on the datasets of Siw, Oulu-NPU, CASIA-MFSD and Replay-Attack. The results show that our model achieves the state-of-the-art results on most of datasets' protocol with much less parameter size.

cs.CV↗

arXiv2021

Controllable double optical bistability via photon and phonon interaction in a hybrid optomechanical system

The optical bistability have been studied theoretically in a multi-mode optomechanical system with two mechanical oscillators independently coupled to two cavities in addition to direct tunnel coupling between cavities. It is proved that the bistable behavior of mean intracavity photon number in the right cavity can be tuned by adjusting the strength of the pump laser beam driving the left cavity. And the mean intracavity photon number is relatively larger in the red sideband regime than that in the blue sideband regime. Moreover, we have shown that the double optical bistability of intracavity photon in the right cavity and the two steady-state positions of mechanical resonators can be observed when the control field power is increased to a critical value. Besides, the critical values for observing bistability and double bistability can be tuned by adjusting the coupling coefficient between two cavities and the coupling rates between cavities mode and mechanical mode.

quant-ph↗

arXiv2021

Target Transformed Regression for Accurate Tracking

Accurate tracking is still a challenging task due to appearance variations, pose and view changes, and geometric deformations of target in videos. Recent anchor-free trackers provide an efficient regression mechanism but fail to produce precise bounding box estimation. To address these issues, this paper repurposes a Transformer-alike regression branch, termed as Target Transformed Regression (TREG), for accurate anchor-free tracking. The core to our TREG is to model pair-wise relation between elements in target template and search region, and use the resulted target enhanced visual representation for accurate bounding box regression. This target contextualized representation is able to enhance the target relevant information to help precisely locate the box boundaries, and deal with the object deformation to some extent due to its local and dense matching mechanism. In addition, we devise a simple online template update mechanism to select reliable templates, increasing the robustness for appearance variations and geometric deformations of target in time. Experimental results on visual tracking benchmarks including VOT2018, VOT2019, OTB100, GOT10k, NFS, UAV123, LaSOT and TrackingNet demonstrate that TREG obtains the state-of-the-art performance, achieving a success rate of 0.640 on LaSOT, while running at around 30 FPS. The code and models will be made available at https://github.com/MCG-NJU/TREG.

cs.CV↗

arXiv2020

Energy-level-attraction and heating-resistant-cooling of mechanical resonators with exceptional points

We study the energy-level evolution and ground-state cooling of mechanical resonators under a synthetic phononic gauge field. The tunable gauge phase is mediated by the phase difference between the $\mathcal{PT}$- and anti-$\mathcal{PT}$-symmetric mechanical couplings in a multimode optomechanical system. The transmission spectrum then exhibits the asymmetric Fano line shape or double optomechanically induced transparency by modulating the gauge phase. Moreover, the eigenvalues will collapse and become degenerate although the mechanical coupling is continuously increased. Such counterintuitive energy-attraction, instead of anti-crossing, attributes to destructive interferences between $\mathcal{PT}$- and anti-$\mathcal{PT}$-symmetric couplings. We find that the energy-attraction, as well as the accompanied exceptional points (EPs), can be more intuitively observed in the cavity output power spectrum where the mechanical eigenvalues correspond to the peaks. For mechanical cooling, the average phonon occupation number becomes minimum at these EPs. Especially, phonon transport becomes nonreciprocal and even ideally unidirectional at the EPs. Finally, we propose a heating-resistant ground-state cooling based on the nonreciprocal phonon transport, which is mediated by the gauge field. Towards the quantum regime of macroscopic mechanical resonators, most optomechanical systems are ultimately limited by their intrinsic cavity or mechanical heating. Our work revealed that the thermal energy transfer can be blocked by tuning the gauge phase, which supports a promising route to overpass the notorious heating limitations.

quant-ph↗

arXiv2020

Blind deblurring for microscopic pathology images using deep learning networks

Artificial Intelligence (AI)-powered pathology is a revolutionary step in the world of digital pathology and shows great promise to increase both diagnosis accuracy and efficiency. However, defocus and motion blur can obscure tissue or cell characteristics hence compromising AI algorithms'accuracy and robustness in analyzing the images. In this paper, we demonstrate a deep-learning-based approach that can alleviate the defocus and motion blur of a microscopic image and output a sharper and cleaner image with retrieved fine details without prior knowledge of the blur type, blur extent and pathological stain. In this approach, a deep learning classifier is first trained to identify the image blur type. Then, two encoder-decoder networks are trained and used alone or in combination to deblur the input image. It is an end-to-end approach and introduces no corrugated artifacts as traditional blind deconvolution methods do. We test our approach on different types of pathology specimens and demonstrate great performance on image blur correction and the subsequent improvement on the diagnosis outcome of AI algorithms.

eess.IV↗

arXiv2020

Optimal unidirectional amplification induced by optical gain in optomechanical systems

We propose a three-mode optomechanical system to realize optical nonreciprocal transmission with unidirectional amplification, where the system consists of two coupled cavities and one mechanical resonator which interacts with only one of the cavities. Additionally, the optical gain is introduced into the optomechanical cavity. It is found that for a strong optical input, the optical transmission coefficient can be greatly amplified in a particular direction and suppressed in the opposite direction. The expressions of the optimal transmission coefficient and the corresponding isolation ratio are given analytically. Our results pave a way to design high-quality nonreciprocal devices based on optomechanical systems.

quant-ph↗

arXiv2020

Switchable bipartite and genuine tripartite entanglement via an optoelectromechanical interface

Controllable multipartite entanglement is a crucial element in quantum information processing. Here we present a scheme that generates switchable bipartite and genuine tripartite entanglement between microwave and optical photons via an optoelectromechanical interface, where microwave and optical cavities are coupled to a mechanical mode with controllable coupling constants. We show that by tuning an effective gauge phase between the coupling constants to the "sweet spots", bipartite entanglement can be generated and switched between designated output photons. The bipartite entanglement is robust against the mechanical noise and the signal loss to the mechanical mode when the couplings are chosen to satisfy the impedance matching condition. When the gauge phase is tuned away from the "sweet spots", genuine tripartite entanglement can be generated and verified with homodyne measurement on the quadratures of the output fields. Our result can lead to the implementation of controllable and robust multipartite entanglement in hybrid quantum systems operated in distinctively different frequencies.

quant-ph↗

arXiv2018

Directional amplifier in an optomechanical system with optical gain

Directional amplifiers are crucial nonreciprocal devices in both classical and quantum information processing. Here we propose a scheme for realizing a directional amplifier between optical and microwave fields based on an optomechanical system with optical gain, where an active optical cavity and two passive microwave cavities are, respectively, coupled to a common mechanical resonator via radiation pressure. The two passive cavities are coupled via hopping interaction to facilitate the directional amplification between the active and passive cavities. We obtain the condition of achieving optical directional amplification and find that the direction of amplification can be controlled by the phase differences between the effective optomechanical couplings. The effects of the gain rate of the active cavity and the effective coupling strengths on the maximum gain of the amplifier are discussed. We show that the noise added to this amplifier can be greatly suppressed in the large cooperativity limit.

quant-ph↗

arXiv2014

Control of microwave signals using bichromatic electromechanically induced transparency in three-mode circuit electromechanical systems

We theoretically investigate the tunable slowing and advancing of microwave signals based on bichromatic electromechanically induced transparency in a three-mode circuit electromechanical system, where two mechanical oscillators with closely spaced frequencies are independently coupled to a common microwave cavity. In the presence of a strong microwave pump field, we obtain two transparency windows accompanied by steep phase dispersion in the transmitted microwave probe field. The width of the transparency window and the group delay of the probe field can be controlled effectively by the power of the pump field. It is shown that the maximum group delay of 0.12 ms and the advance of 0.27 ms can be obtained in the current experiments.

quant-ph↗

arXiv2014

Mechanically-mediated optical response in hybrid opto-electromechanical systems

We theoretically investigate the analog of electromagnetically induced transparency, absorption and parametric amplification in a hybrid opto-electromechanical system consisting of an optical cavity and a microwave cavity coupled to a common nanomechanical resonator. When the two cavity modes are driven on their respective red sidebands by two pump beams and a probe beam is applied to the optical cavity to monitor the optical response of the hybrid system, we find that a transparency window appears in the probe transmission spectrum due to destructive interference. When the optical cavity is pumped on its blue sideband, the analog of electromagnetically induced absorption and parametric amplification occur due to constructive interference.

quant-ph↗

arXiv2013

Controllable optical bistability based on photons and phonons in a two-mode optomechanical system

We explore theoretically the bistable behavior of intracavity photon number in a two-mode cavity optomechanical system, where two cavity modes are coupled to a common mechanical resonator. When the two cavity modes are driven by two pump laser beams, respectively, we find that the optical bistability can be controlled by tuning the power and frequency of the pump beams. The common interaction to a mechanical mode enables one to control the bistable behavior in one cavity by adjusting the pump laser beam driving another cavity. We also show that both branches of optical bistability at photon numbers below unity can be observed in this two-mode optomechanical system. This phenomenon can find potential applications in controllable optical switch.

quant-ph↗

arXiv2013

Electromagnetically induced transparency and slow light in two-mode optomechanics

We theoretically demonstrate the mechanically mediated electromagnetically induced transparency in a two-mode cavity optomechanical system, where two cavity modes are coupled to a common mechanical resonator. When the two cavity modes are driven on their respective red sidebands by two pump beams, a transparency window appears in the probe transmission spectrum due to destructive interference. Under this situation the transmitted probe beam can be delayed as much as 4 us, which can be easily controlled by the power of the pump beams. In addition, we also investigate the amplification of the transmitted probe beam owing to constructive interference when one cavity is driven on its blue sideband while another one is driven on its red sideband.

quant-ph↗