arXiv subjects

Jie Luo

Publications and source records attributed to Jie Luo.

At least 109 records · Page 6Linked to original sources

arXiv2020

Unsupervised Multimodal Image Registration with Adaptative Gradient Guidance

Multimodal image registration (MIR) is a fundamental procedure in many image-guided therapies. Recently, unsupervised learning-based methods have demonstrated promising performance over accuracy and efficiency in deformable image registration. However, the estimated deformation fields of the existing methods fully rely on the to-be-registered image pair. It is difficult for the networks to be aware of the mismatched boundaries, resulting in unsatisfactory organ boundary alignment. In this paper, we propose a novel multimodal registration framework, which leverages the deformation fields estimated from both: (i) the original to-be-registered image pair, (ii) their corresponding gradient intensity maps, and adaptively fuses them with the proposed gated fusion module. With the help of auxiliary gradient-space guidance, the network can concentrate more on the spatial relationship of the organ boundary. Experimental results on two clinically acquired CT-MRI datasets demonstrate the effectiveness of our proposed approach.

cs.CV↗

arXiv2020

Pseudo-Hermitian Systems Constructed by Transformation Optics with Robustly Balanced Loss and Gain

Non-Hermitian systems with parity-time symmetry have been found to exhibit real spectra of eigenvalues, indicating a balance between the loss and gain. However, such a balance is not only dependent on the magnitude of loss and gain, but also easily broken due to external disturbance. Here, the authors propose a transformation-optics approach to construct a unique class of non-Hermitian systems with robustly balanced loss and gain, irrespective of the magnitude of loss/gain and the environmental disturbance. Through transformation-optics operators like space folding and stretching, loss and gain can be generated and separated in the real space. While in the virtual space, the loss and gain are still combined to each other, rendering a balance of energy that is far more robust than other non-Hermitian systems. This amazing feature is verified by finite-element simulations. This work reveals a class of non-Hermitian systems in which loss and gain are balanced robustly, thereby denoted as pseudo-Hermitian systems.

physics.optics↗

arXiv2020

Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration

Deformable image registration between Computed Tomography (CT) images and Magnetic Resonance (MR) imaging is essential for many image-guided therapies. In this paper, we propose a novel translation-based unsupervised deformable image registration method. Distinct from other translation-based methods that attempt to convert the multimodal problem (e.g., CT-to-MR) into a unimodal problem (e.g., MR-to-MR) via image-to-image translation, our method leverages the deformation fields estimated from both: (i) the translated MR image and (ii) the original CT image in a dual-stream fashion, and automatically learns how to fuse them to achieve better registration performance. The multimodal registration network can be effectively trained by computationally efficient similarity metrics without any ground-truth deformation. Our method has been evaluated on two clinical datasets and demonstrates promising results compared to state-of-the-art traditional and learning-based methods.

eess.IV↗

arXiv2020

On the Applicability of Registration Uncertainty

Estimating the uncertainty in (probabilistic) image registration enables, e.g., surgeons to assess the operative risk based on the trustworthiness of the registered image data. If surgeons receive inaccurately calculated registration uncertainty and misplace unwarranted confidence in the alignment solutions, severe consequences may result. For probabilistic image registration (PIR), the predominant way to quantify the registration uncertainty is using summary statistics of the distribution of transformation parameters. The majority of existing research focuses on trying out different summary statistics as well as a means to exploit them. Distinctively, in this paper, we study two rarely examined topics: (1) whether those summary statistics of the transformation distribution most informatively represent the registration uncertainty; (2) Does utilizing the registration uncertainty always be beneficial. We show that there are two types of uncertainties: the transformation uncertainty, Ut, and label uncertainty Ul. The conventional way of using Ut to quantify Ul is inappropriate and can be misleading. By a real data experiment, we also share a potentially critical finding that making use of the registration uncertainty may not always be an improvement.

cs.CV↗

arXiv2020

Are Registration Uncertainty and Error Monotonically Associated

In image-guided neurosurgery, current commercial systems usually provide only rigid registration, partly because it is harder to predict, validate and understand non-rigid registration error. For instance, when surgeons see a discrepancy in aligned image features, they may not be able to distinguish between registration error and actual tissue deformation caused by tumor resection. In this case, the spatial distribution of registration error could help them make more informed decisions, e.g., ignoring the registration where the estimated error is high. However, error estimates are difficult to acquire. Probabilistic image registration (PIR) methods provide measures of registration uncertainty, which could be a surrogate for assessing the registration error. It is intuitive and believed by many clinicians that high uncertainty indicates a large error. However, the monotonic association between uncertainty and error has not been examined in image registration literature. In this pilot study, we attempt to address this fundamental problem by looking at one PIR method, the Gaussian process (GP) registration. We systematically investigate the relation between GP uncertainty and error based on clinical data and show empirically that there is a weak-to-moderate positive monotonic correlation between point-wise GP registration uncertainty and non-rigid registration error.

cs.CV↗

arXiv2020

Do Public Datasets Assure Unbiased Comparisons for Registration Evaluation?

With the increasing availability of new image registration approaches, an unbiased evaluation is becoming more needed so that clinicians can choose the most suitable approaches for their applications. Current evaluations typically use landmarks in manually annotated datasets. As a result, the quality of annotations is crucial for unbiased comparisons. Even though most data providers claim to have quality control over their datasets, an objective third-party screening can be reassuring for intended users. In this study, we use the variogram to screen the manually annotated landmarks in two datasets used to benchmark registration in image-guided neurosurgeries. The variogram provides an intuitive 2D representation of the spatial characteristics of annotated landmarks. Using variograms, we identified potentially problematic cases and had them examined by experienced radiologists. We found that (1) a small number of annotations may have fiducial localization errors; (2) the landmark distribution for some cases is not ideal to offer fair comparisons. If unresolved, both findings could incur bias in registration evaluation.

cs.CV↗

arXiv2020

Unbiased Scene Graph Generation via Rich and Fair Semantic Extraction

Extracting graph representation of visual scenes in image is a challenging task in computer vision. Although there has been encouraging progress of scene graph generation in the past decade, we surprisingly find that the performance of existing approaches is largely limited by the strong biases, which mainly stem from (1) unconsciously assuming relations with certain semantic properties such as symmetric and (2) imbalanced annotations over different relations. To alleviate the negative effects of these biases, we proposed a new and simple architecture named Rich and Fair semantic extraction network (RiFa for short), to not only capture rich semantic properties of the relations, but also fairly predict relations with different scale of annotations. Using pseudo-siamese networks, RiFa embeds the subject and object respectively to distinguish their semantic differences and meanwhile preserve their underlying semantic properties. Then, it further predicts subject-object relations based on both the visual and semantic features of entities under certain contextual area, and fairly ranks the relation predictions for those with a few annotations. Experiments on the popular Visual Genome dataset show that RiFa achieves state-of-the-art performance under several challenging settings of scene graph task. Especially, it performs significantly better on capturing different semantic properties of relations, and obtains the best overall per relation performance.

cs.CV↗

arXiv2020

An Investigation of Feature-based Nonrigid Image Registration using Gaussian Process

For a wide range of clinical applications, such as adaptive treatment planning or intraoperative image update, feature-based deformable registration (FDR) approaches are widely employed because of their simplicity and low computational complexity. FDR algorithms estimate a dense displacement field by interpolating a sparse field, which is given by the established correspondence between selected features. In this paper, we consider the deformation field as a Gaussian Process (GP), whereas the selected features are regarded as prior information on the valid deformations. Using GP, we are able to estimate the both dense displacement field and a corresponding uncertainty map at once. Furthermore, we evaluated the performance of different hyperparameter settings for squared exponential kernels with synthetic, phantom and clinical data respectively. The quantitative comparison shows, GP-based interpolation has performance on par with state-of-the-art B-spline interpolation. The greatest clinical benefit of GP-based interpolation is that it gives a reliable estimate of the mathematical uncertainty of the calculated dense displacement map.

cs.CV↗

arXiv2019

mvn2vec: Preservation and Collaboration in Multi-View Network Embedding

Multi-view networks are broadly present in real-world applications. In the meantime, network embedding has emerged as an effective representation learning approach for networked data. Therefore, we are motivated to study the problem of multi-view network embedding with a focus on the optimization objectives that are specific and important in embedding this type of network. In our practice of embedding real-world multi-view networks, we explicitly identify two such objectives, which we refer to as preservation and collaboration. The in-depth analysis of these two objectives is discussed throughout this paper. In addition, the mvn2vec algorithms are proposed to (i) study how varied extent of preservation and collaboration can impact embedding learning and (ii) explore the feasibility of achieving better embedding quality by modeling them simultaneously. With experiments on a series of synthetic datasets, a large-scale internal Snapchat dataset, and two public datasets, we confirm the validity and importance of preservation and collaboration as two objectives for multi-view network embedding. These experiments further demonstrate that better embedding can be obtained by simultaneously modeling the two objectives, while not over-complicating the model or requiring additional supervision. The code and the processed datasets are available at http://yushi2.web.engr.illinois.edu/.

cs.SI↗

arXiv2019

Two-Dimensional Optomechanical Crystal Cavity with High Quantum Cooperativity

Optomechanical systems offer new opportunities in quantum information processing and quantum sensing. Many solid-state quantum devices operate at millikelvin temperatures -- however, it has proven challenging to operate nanoscale optomechanical devices at these ultralow temperatures due to their limited thermal conductance and parasitic optical absorption. Here, we demonstrate a two-dimensional optomechanical crystal resonator capable of achieving large cooperativity $C$ and small effective bath occupancy $n_b$, resulting in a quantum cooperativity $C_{\text{eff}}\equiv C/n_b \approx 1.3 > 1$ under continuous-wave optical driving. This is realized using a two-dimensional phononic bandgap structure to host the optomechanical cavity, simultaneously isolating the acoustic mode of interest in the bandgap while allowing heat to be removed by phonon modes outside of the bandgap. This achievement paves the way for a variety of applications requiring quantum-coherent optomechanical interactions, such as transducers capable of bi-directional conversion of quantum states between microwave frequency superconducting quantum circuits and optical photons in a fiber optic network.

quant-ph↗

arXiv2019

I Know You'll Be Back: Interpretable New User Clustering and Churn Prediction on a Mobile Social Application

As online platforms are striving to get more users, a critical challenge is user churn, which is especially concerning for new users. In this paper, by taking the anonymous large-scale real-world data from Snapchat as an example, we develop \textit{ClusChurn}, a systematic two-step framework for interpretable new user clustering and churn prediction, based on the intuition that proper user clustering can help understand and predict user churn. Therefore, \textit{ClusChurn} firstly groups new users into interpretable typical clusters, based on their activities on the platform and ego-network structures. Then we design a novel deep learning pipeline based on LSTM and attention to accurately predict user churn with very limited initial behavior data, by leveraging the correlations among users' multi-dimensional activities and the underlying user types. \textit{ClusChurn} is also able to predict user types, which enables rapid reactions to different types of user churn. Extensive data analysis and experiments show that \textit{ClusChurn} provides valuable insight into user behaviors, and achieves state-of-the-art churn prediction performance. The whole framework is deployed as a data analysis pipeline, delivering real-time data analysis and prediction results to multiple relevant teams for business intelligence uses. It is also general enough to be readily adopted by any online systems with user behavior data.

cs.SI↗

arXiv2019

Temporal Registration in Application to In-utero MRI Time Series

We present a robust method to correct for motion in volumetric in-utero MRI time series. Time-course analysis for in-utero volumetric MRI time series often suffers from substantial and unpredictable fetal motion. Registration provides voxel correspondences between images and is commonly employed for motion correction. Current registration methods often fail when aligning images that are substantially different from a template (reference image). To achieve accurate and robust alignment, we make a Markov assumption on the nature of motion and take advantage of the temporal smoothness in the image data. Forward message passing in the corresponding hidden Markov model (HMM) yields an estimation algorithm that only has to account for relatively small motion between consecutive frames. We evaluate the utility of the temporal model in the context of in-utero MRI time series alignment by examining the accuracy of propagated segmentation label maps. Our results suggest that the proposed model captures accurately the temporal dynamics of transformations in in-utero MRI time series.

cs.CV↗

arXiv2019

UAV-aided urban target tracking system based on edge computing

Target tracking is an important issue of social security. In order to track a target, traditionally a large amount of surveillance video data need to be uploaded into the cloud for processing and analysis, which put stremendous bandwidth pressure on communication links in access networks and core networks. At the same time, the long delay in wide area network is very likely to cause a tracking system to lose its target. Often, unmanned aerial vehicle (UAV) has been adopted for target tracking due to its flexibility, but its limited flight time due to battery constraint and the blocking by various obstacles in the field pose two major challenges to its target tracking task, which also very likely results in the loss of target. A novel target tracking model that coordinates the tracking by UAV and ground nodes in an edge computing environment is proposed in this study. The model can effectively reduce the communication cost and the long delay of the traditional surveillance camera system that relies on cloud computing, and it can improve the probability of finding a target again after an UAV loses the tracing of that target. It has been demonstrated that the proposed system achieved a significantly better performance in terms of low latency, high reliability, and optimal quality of experience (QoE).

eess.SP↗

arXiv2019

Three-dimensional acoustic double-zero-index medium with a Dirac-like point

We report a design and experimental realization of a three-dimensional (3D) acoustic double-zero-index medium (DZIM), whose effective mass density and compressibility are nearly zero simultaneously. The DZIM is constructed from a cubic lattice of three orthogonally-aligned metal rods in air. The combination of lattice symmetry and accidental degeneracy yields a four-fold degenerate point with conical dispersion at the Brillouin zone center, where the material becomes a 3D DZIM. Though occupying a finite volume, the 3D DZIM maintains the wave properties of a "void space," and enables rich applications. For demonstration, we fabricate an acoustic "periscope" by placing the designed 3D DZIM inside a 3D bending waveguide, and observe the unusual wave tunneling effect through this waveguide with undisturbed planar wavefront. Our findings establish a practical route to realize 3D DZIM as an effective acoustic "void space," which offers unprecedented opportunities for advanced sound manipulation.

physics.app-ph↗

arXiv2019

Phononic bandgap nano-acoustic cavity with ultralong phonon lifetime

We present measurements at millikelvin temperatures of the microwave-frequency acoustic properties of a crystalline silicon nanobeam cavity incorporating a phononic bandgap clamping structure for acoustic confinement. Utilizing pulsed laser light to excite a co-localized optical mode of the nanobeam cavity, we measure the dynamics of cavity acoustic modes with single-phonon sensitivity. Energy ringdown measurements for the fundamental $5$~GHz acoustic mode of the cavity shows an exponential increase in phonon lifetime versus number of periods in the phononic bandgap shield, increasing up to $τ\approx 1.5$~seconds. This ultralong lifetime, corresponding to an effective phonon propagation length of several kilometers, is found to be consistent with damping from non-resonant two-level system defects on the surface of the silicon device. Potential applications of these ultra-coherent nanoscale mechanical resonators range from tests of various collapse models of quantum mechanics to miniature quantum memory elements in hybrid superconducting quantum circuits.

cond-mat.mes-hall↗

arXiv2018

Deep Information Theoretic Registration

This paper establishes an information theoretic framework for deep metric based image registration techniques. We show an exact equivalence between maximum profile likelihood and minimization of joint entropy, an important early information theoretic registration method. We further derive deep classifier-based metrics that can be used with iterated maximum likelihood to achieve Deep Information Theoretic Registration on patches rather than pixels. This alleviates a major shortcoming of previous information theoretic registration approaches, namely the implicit pixel-wise independence assumptions. Our proposed approach does not require well-registered training data; this brings previous fully supervised deep metric registration approaches to the realm of weak supervision. We evaluate our approach on several image registration tasks and show significantly better performance compared to mutual information, specifically when images have substantially different contrasts. This work enables general-purpose registration in applications where current methods are not successful.

cs.CV↗

arXiv2018

Deep learning based automatic segmentation of lumbosacral nerves on non-contrast CT for radiographic evaluation: a pilot study

Background and objective: Combined evaluation of lumbosacral structures (e.g. nerves, bone) on multimodal radiographic images is routinely conducted prior to spinal surgery and interventional procedures. Generally, magnetic resonance imaging is conducted to differentiate nerves, while computed tomography (CT) is used to observe bony structures. The aim of this study is to investigate the feasibility of automatically segmenting lumbosacral structures (e.g. nerves & bone) on non-contrast CT with deep learning. Methods: a total of 50 cases with spinal CT were manually labeled for lumbosacral nerves and bone with Slicer 4.8. The ratio of training: validation: testing is 32:8:10. A 3D-Unet is adopted to build the model SPINECT for automatically segmenting lumbosacral structures. Pixel accuracy, IoU, and Dice score are used to assess the segmentation performance of lumbosacral structures. Results: the testing results reveals successful segmentation of lumbosacral bone and nerve on CT. The average pixel accuracy is 0.940 for bone and 0.918 for nerve. The average IoU is 0.897 for bone and 0.827 for nerve. The dice score is 0.945 for bone and 0.905 for nerve. Conclusions: this pilot study indicated that automatic segmenting lumbosacral structures (nerves and bone) on non-contrast CT is feasible and may have utility for planning and navigating spinal interventions and surgery.

cs.CV↗

arXiv2018

Distributed Multiple Access with A General Link Layer Channel

This paper investigates the problem of distributed medium access control in a time slotted wireless multiple access network with an unknown finite number of homogeneous users. Assume that each user has a single transmission option. In each time slot, a user chooses either to idle or to transmit a packet. Under a general channel model, a distributed medium access control framework is proposed to adapt transmission probabilities of all users to a value that maximizes an arbitrarily chosen symmetric network utility. Probability target of each user in the proposed algorithm is calculated based upon a channel contention measure, which is defined as the success probability of a virtual packet. It is shown that the proposed algorithm falls into the classical stochastic approximation framework with guaranteed convergence when the contention measure can be directly obtained from the receiver. On the other hand, when the contention measure is not directly available, computer simulations show that a revised medium access control algorithm can still help the system to converge to the same designed equilibrium.

cs.IT↗