arXiv subjects

George Barbastathis

Publications and source records attributed to George Barbastathis.

At least 37 records · Page 2Linked to original sources

arXiv2022

AI-based Clinical Assessment of Optic Nerve Head Robustness Superseding Biomechanical Testing

$\mathbf{Purpose}$: To use artificial intelligence (AI) to: (1) exploit biomechanical knowledge of the optic nerve head (ONH) from a relatively large population; (2) assess ONH robustness from a single optical coherence tomography (OCT) scan of the ONH; (3) identify what critical three-dimensional (3D) structural features make a given ONH robust. $\mathbf{Design}$: Retrospective cross-sectional study. $\mathbf{Methods}$: 316 subjects had their ONHs imaged with OCT before and after acute intraocular pressure (IOP) elevation through ophthalmo-dynamometry. IOP-induced lamina-cribrosa deformations were then mapped in 3D and used to classify ONHs. Those with LC deformations superior to 4% were considered fragile, while those with deformations inferior to 4% robust. Learning from these data, we compared three AI algorithms to predict ONH robustness strictly from a baseline (undeformed) OCT volume: (1) a random forest classifier; (2) an autoencoder; and (3) a dynamic graph CNN (DGCNN). The latter algorithm also allowed us to identify what critical 3D structural features make a given ONH robust. $\mathbf{Results}$: All 3 methods were able to predict ONH robustness from 3D structural information alone and without the need to perform biomechanical testing. The DGCNN (area under the receiver operating curve [AUC]: 0.76 $\pm$ 0.08) outperformed the autoencoder (AUC: 0.70 $\pm$ 0.07) and the random forest classifier (AUC: 0.69 $\pm$ 0.05). Interestingly, to assess ONH robustness, the DGCNN mainly used information from the scleral canal and the LC insertion sites. $\mathbf{Conclusions}$: We propose an AI-driven approach that can assess the robustness of a given ONH solely from a single OCT scan of the ONH, and without the need to perform biomechanical testing. Longitudinal studies should establish whether ONH robustness could help us identify fast visual field loss progressors.

eess.IV↗

arXiv2022

Physics-assisted Generative Adversarial Network for X-Ray Tomography

X-ray tomography is capable of imaging the interior of objects in three dimensions non-invasively, with applications in biomedical imaging, materials science, electronic inspection, and other fields. The reconstruction process can be an ill-conditioned inverse problem, requiring regularization to obtain satisfactory results. Recently, deep learning has been adopted for tomographic reconstruction. Unlike iterative algorithms which require a distribution that is known a priori, deep reconstruction networks can learn a prior distribution through sampling the training distributions. In this work, we develop a Physics-assisted Generative Adversarial Network (PGAN), a two-step algorithm for tomographic reconstruction. In contrast to previous efforts, our PGAN utilizes maximum-likelihood estimates derived from the measurements to regularize the reconstruction with both known physics and the learned prior. Compared with methods with less physics assisting in training, PGAN can reduce the photon requirement with limited projection angles to achieve a given error rate. The advantages of using a physics-assisted learned prior in X-ray tomography may further enable low-photon nanoscale imaging.

eess.IV↗

arXiv2022

Geometric Deep Learning to Identify the Critical 3D Structural Features of the Optic Nerve Head for Glaucoma Diagnosis

Purpose: The optic nerve head (ONH) undergoes complex and deep 3D morphological changes during the development and progression of glaucoma. Optical coherence tomography (OCT) is the current gold standard to visualize and quantify these changes, however the resulting 3D deep-tissue information has not yet been fully exploited for the diagnosis and prognosis of glaucoma. To this end, we aimed: (1) To compare the performance of two relatively recent geometric deep learning techniques in diagnosing glaucoma from a single OCT scan of the ONH; and (2) To identify the 3D structural features of the ONH that are critical for the diagnosis of glaucoma. Methods: In this study, we included a total of 2,247 non-glaucoma and 2,259 glaucoma scans from 1,725 subjects. All subjects had their ONHs imaged in 3D with Spectralis OCT. All OCT scans were automatically segmented using deep learning to identify major neural and connective tissues. Each ONH was then represented as a 3D point cloud. We used PointNet and dynamic graph convolutional neural network (DGCNN) to diagnose glaucoma from such 3D ONH point clouds and to identify the critical 3D structural features of the ONH for glaucoma diagnosis. Results: Both the DGCNN (AUC: 0.97$\pm$0.01) and PointNet (AUC: 0.95$\pm$0.02) were able to accurately detect glaucoma from 3D ONH point clouds. The critical points formed an hourglass pattern with most of them located in the inferior and superior quadrant of the ONH. Discussion: The diagnostic accuracy of both geometric deep learning approaches was excellent. Moreover, we were able to identify the critical 3D structural features of the ONH for glaucoma diagnosis that tremendously improved the transparency and interpretability of our method. Consequently, our approach may have strong potential to be used in clinical applications for the diagnosis and prognosis of a wide range of ophthalmic disorders.

eess.IV↗

arXiv2022

Unified treatment of exact and approximate scalar electromagnetic wave scattering

Under conditions of strong scattering, a dilemma often arises regarding the best numerical method to use. Main competitors are the Born series, the Beam Propagation Method, and direct solution of the Lippmann-Schwinger equation. However, analytical relationships between the three methods have not yet, to our knowledge, been explicitly stated. Here, we bridge this gap in the literature. In addition to overall insight about aspects of optical scattering that are best numerically captured by each method, our approach allows us to derive approximate error bounds to be expected under various scattering conditions.

physics.optics↗

arXiv2021

Advantage of Machine Learning over Maximum Likelihood in Limited-Angle Low-Photon X-Ray Tomography

Limited-angle X-ray tomography reconstruction is an ill-conditioned inverse problem in general. Especially when the projection angles are limited and the measurements are taken in a photon-limited condition, reconstructions from classical algorithms such as filtered backprojection may lose fidelity and acquire artifacts due to the missing-cone problem. To obtain satisfactory reconstruction results, prior assumptions, such as total variation minimization and nonlocal image similarity, are usually incorporated within the reconstruction algorithm. In this work, we introduce deep neural networks to determine and apply a prior distribution in the reconstruction process. Our neural networks learn the prior directly from synthetic training samples. The neural nets thus obtain a prior distribution that is specific to the class of objects we are interested in reconstructing. In particular, we used deep generative models with 3D convolutional layers and 3D attention layers which are trained on 3D synthetic integrated circuit (IC) data from a model dubbed CircuitFaker. We demonstrate that, when the projection angles and photon budgets are limited, the priors from our deep generative models can dramatically improve the IC reconstruction quality on synthetic data compared with maximum likelihood estimation. Training the deep generative models with synthetic IC data from CircuitFaker illustrates the capabilities of the learned prior from machine learning. We expect that if the process were reproduced with experimental data, the advantage of the machine learning would persist. The advantages of machine learning in limited angle X-ray tomography may further enable applications in low-photon nanoscale imaging.

eess.IV↗

arXiv2020

Machine Learning Regularized Solution of the Lippmann-Schwinger Equation

Solution of the discretized Lippmann-Schwinger equation in the spatial frequency domain involves the inversion of a linear operator specified by the scattering potential. To regularize this inevitably ill-conditioned problem, we propose a machine learning approach: a recurrent neural network with long short-term memory (LSTM) and with the null space projection of the Lippmann-Schwinger kernel on the recurrence path. The learning method is trained using examples of typical scattering potentials and their corresponding scattered fields. We test the proposed method in two cases: electromagnetic scattering by dielectric objects, and electron scattering by multiple screened Coulomb potentials. In both cases the solutions to test examples, disjoint from the training set, were obtained with fewer iterations and were accurate compared to linear solvers. We also observed surprising generalization ability: in the electromagnetic case, an LSTM trained with random arrangements of dielectric spheres was able to obtain the correct solutions for general topologically similar objects, such as polygons. This suggests that the LSTM successfully incorporates the physics of scattering into the inversion algorithm.

physics.comp-ph↗

arXiv2020

A machine learning aided global diagnostic and comparative tool to assess effect of quarantine control in Covid-19 spread

We have developed a globally applicable diagnostic Covid-19 model by augmenting the classical SIR epidemiological model with a neural network module. Our model does not rely upon previous epidemics like SARS/MERS and all parameters are optimized via machine learning algorithms employed on publicly available Covid-19 data. The model decomposes the contributions to the infection timeseries to analyze and compare the role of quarantine control policies employed in highly affected regions of Europe, North America, South America and Asia in controlling the spread of the virus. For all continents considered, our results show a generally strong correlation between strengthening of the quarantine controls as learnt by the model and actions taken by the regions' respective governments. Finally, we have hosted our quarantine diagnosis results for the top 70 affected countries worldwide, on a public platform, which can be used for informed decision making by public health officials and researchers alike.

physics.soc-ph↗

arXiv2020

Limited-angle tomographic reconstruction of dense layered objects by dynamical machine learning

Limited-angle tomography of strongly scattering quasi-transparent objects is a challenging, highly ill-posed problem with practical implications in medical and biological imaging, manufacturing, automation, and environmental and food security. Regularizing priors are necessary to reduce artifacts by improving the condition of such problems. Recently, it was shown that one effective way to learn the priors for strongly scattering yet highly structured 3D objects, e.g. layered and Manhattan, is by a static neural network [Goy et al, Proc. Natl. Acad. Sci. 116, 19848-19856 (2019)]. Here, we present a radically different approach where the collection of raw images from multiple angles is viewed analogously to a dynamical system driven by the object-dependent forward scattering operator. The sequence index in angle of illumination plays the role of discrete time in the dynamical system analogy. Thus, the imaging problem turns into a problem of nonlinear system identification, which also suggests dynamical learning as better fit to regularize the reconstructions. We devised a recurrent neural network (RNN) architecture with a novel split-convolutional gated recurrent unit (SC-GRU) as the fundamental building block. Through comprehensive comparison of several quantitative metrics, we show that the dynamic method improves upon previous static approaches with fewer artifacts and better overall reconstruction fidelity.

eess.IV↗

arXiv2020

On the interplay between physical and content priors in deep learning for computational imaging

Deep learning (DL) has been applied extensively in many computational imaging problems, often leading to superior performance over traditional iterative approaches. However, two important questions remain largely unanswered: first, how well can the trained neural network generalize to objects very different from the ones in training? This is particularly important in practice, since large-scale annotated examples similar to those of interest are often not available during training. Second, has the trained neural network learnt the underlying (inverse) physics model, or has it merely done something trivial, such as memorizing the examples or point-wise pattern matching? This pertains to the interpretability of machine-learning based algorithms. In this work, we use the Phase Extraction Neural Network (PhENN), a deep neural network (DNN) for quantitative phase retrieval in a lensless phase imaging system as the standard platform and show that the two questions are related and share a common crux: the choice of the training examples. Moreover, we connect the strength of the regularization effect imposed by a training set to the training process with the Shannon entropy of images in the dataset. That is, the higher the entropy of the training images, the weaker the regularization effect can be imposed. We also discover that weaker regularization effect leads to better learning of the underlying propagation model, i.e. the weak object transfer function, applicable for weakly scattering objects under the weak object approximation. Finally, simulation and experimental results show that better cross-domain generalization performance can be achieved if DNN is trained on a higher-entropy database, e.g. the ImageNet, than if the same DNN is trained on a lower-entropy database, e.g. MNIST, as the former allows the underlying physics model be learned better than the latter.

eess.IV↗

arXiv2020

Neural Network aided quarantine control model estimation of global Covid-19 spread

Since the first recording of what we now call Covid-19 infection in Wuhan, Hubei province, China on Dec 31, 2019, the disease has spread worldwide and met with a wide variety of social distancing and quarantine policies. The effectiveness of these responses is notoriously difficult to quantify as individuals travel, violate policies deliberately or inadvertently, and infect others without themselves being detected. In this paper, we attempt to interpret and extrapolate from publicly available data using a mixed first-principles epidemiological equations and data-driven neural network model. Leveraging our neural network augmented model, we focus our analysis on four locales: Wuhan, Italy, South Korea and the United States of America, and compare the role played by the quarantine and isolation measures in each of these countries in controlling the effective reproduction number $R_{t}$ of the virus. Our results unequivocally indicate that the countries in which rapid government interventions and strict public health measures for quarantine and isolation were implemented were successful in halting the spread of infection and prevent it from exploding exponentially. We test the predictive ability of our model by matching predictions in the duration 3 March - 1 April 2020 for Wuhan and in the duration 25 March - 1 April 2020 for Italy and South Korea. In the case of the US, our model captures well the current infected curve growth and predicts a halting of infection spread by 20 April 2020. We further demonstrate that relaxing or reversing quarantine measures right now will lead to an exponential explosion in the infected case count, thus nullifying the role played by all measures implemented in the US since mid March 2020.

q-bio.PE↗

arXiv2020

Neural Network aided quarantine control model estimation of COVID spread in Wuhan, China

In a move described as unprecedented in public health history, starting 24 January 2020, China imposed quarantine and isolation restrictions in Wuhan, a city of more than 10 million people. This raised the question: is mass quarantine and isolation effective as a social tool in addition to its scientific use as a medical tool? In an effort to address this question, using a epidemiological model driven approach augmented by machine learning, we show that the quarantine and isolation measures implemented in Wuhan brought down the effective reproduction number R(t) of the CoVID-19 spread from R(t) > 1 to R(t) <1 within a month after the imposition of quarantine control measures in Wuhan, China. This ultimately resulted in a stagnation phase in the infected case count in Wuhan. Our results indicate that the strict public health policies implemented in Wuhan may have played a crucial role in halting down the spread of infection and such measures should potentially be implemented in other highly affected countries such as South Korea, Italy and Iran to curtail spread of the disease. Finally, our forecasting results predict a stagnation in the quarantine control measures implemented in Wuhan towards the end of March 2020; this would lead to a subsequent stagnation in the effective reproduction number at R(t) <1. We warn that immediate relaxation of the quarantine measures in Wuhan may lead to a relapse in the infection spread and a subsequent increase in the effective reproduction number to R(t) >1. Thus, it may be wise to relax quarantine measures after sufficient time has elapsed, during which maximum of the quarantined/isolated individuals are recovered.

q-bio.PE↗

arXiv2019

Scalable additive manufacturing via integral image formation

Additive manufacturing techniques enable the fabrication of functional microstructures with mechanical and chemical properties tailored to their intended use. One common technique is stereolithography, which has recently been augmented in response to modern demands to support smaller features, larger build areas, and/or faster speeds. However, a limitation is that such systems typically utilize a single-aperture imaging configuration, which restricts their ability to produce microstructures at large volumes due to the tradeoff between the image resolution and image field area. In this paper, we demonstrate three versatile imaging functions based on the coupling a planar lens array, namely, parallel image transfer, kaleidoscopic superposition, and integral reconstruction, the combination of which is termed integral lithography. In this approach, the individual microlenses in the planar lens array maintain a high numerical aperture and are employed in the creation of digital light patterns that can expand the printable area by the number of microlenses (103-104). The proposed lens array-based integral imaging system provides the concurrent ability to scale-up and scale-down incoming image fields, thereby enabling the scalable stereolithographic fabrication of three-dimensional features that surpass the resolution-to-area scaling limit. The proposed system opens up new possibilities for producing periodic microarchitectures spanning four orders of magnitude from micrometers to centimeters that can be applied to biological scaffolds, metastructures, chemical reactors, and functional surfaces.

physics.app-ph↗

arXiv2019

Learning to Synthesize: Robust Phase Retrieval at Low Photon counts

The quality of inverse problem solutions obtained through deep learning [Barbastathis et al, 2019] is limited by the nature of the priors learned from examples presented during the training phase. In the case of quantitative phase retrieval [Sinha et al, 2017, Goy et al, 2019], in particular, spatial frequencies that are underrepresented in the training database, most often at the high band, tend to be suppressed in the reconstruction. Ad hoc solutions have been proposed, such as pre-amplifying the high spatial frequencies in the examples [Li et al, 2018]; however, while that strategy improves resolution, it also leads to high-frequency artifacts as well as low-frequency distortions in the reconstructions. Here, we present a new approach that learns separately how to handle the two frequency bands, low and high; and also learns how to synthesize these two bands into the full-band reconstructions. We show that this "learning to synthesize" (LS) method yields phase reconstructions of high spatial resolution and artifact-free; and it is also resilient to high-noise conditions, e.g. in the case of very low photon flux. In addition to the problem of quantitative phase retrieval, the LS method is applicable, in principle, to any inverse problem where the forward operator treats different frequency bands unevenly, i.e. is ill-posed.

eess.IV↗

arXiv2019

Low Photon Budget Phase Retrieval with Perceptual Loss Trained Deep Neural Networks

Deep neural networks (DNNs) are efficient solvers for ill-posed problems and have been shown to outperform classical optimization techniques in several computational imaging problems. DNNs are trained by solving an optimization problem implies the choice of an appropriate loss function, i.e., the function to optimize. In a recent paper [A. Goy \textit{et al.}, Phys. Rev. Lett. 121(24), 243902 (2018)], we showed that DNNs trained with the negative Pearson correlation coefficient as the loss function are particularly fit for photon-starved phase retrieval problems, which are fundamentally affected by strong Poison noise. In this paper we demonstrate that the use of a perceptual loss function significantly improves the reconstructions.

eess.IV↗

arXiv2018

High-Resolution Limited-Angle Phase Tomography of Dense Layered Objects Using Deep Neural Networks

We present a Machine Learning-based method for tomographic reconstruction of dense layered objects, with range of projection angles limited to $\pm $10$^\circ$. Whereas previous approaches to phase tomography generally require two steps, first to retrieve phase projections from intensity projections and then perform tomographic reconstruction on the retrieved phase projections, in our work a physics-informed pre-processor followed by a Deep Neural Network (DNN) conduct the three-dimensional reconstruction directly from the intensity projections. We demonstrate this single-step method experimentally in the visible optical domain on a scaled up integrated circuit phantom. We show that even under conditions of highly attenuated photon fluxes a DNN trained only on synthetic data can be used to successfully reconstruct physical samples disjoint from the synthetic training set. Thus, the need of producing a large number of physical examples for training is ameliorated. The method is generally applicable to tomography with electromagnetic or other types of radiation at all bands.

eess.IV↗

arXiv2018

Learning to synthesize: splitting and recombining low and high spatial frequencies for image recovery

Deep Neural Network (DNN)-based image reconstruction, despite many successes, often exhibits uneven fidelity between high and low spatial frequency bands. In this paper we propose the Learning Synthesis by DNN (LS-DNN) approach where two DNNs process the low and high spatial frequencies, respectively, and, improving over [30], the two DNNs are trained separately and a third DNN combines them into an image with high fidelity at all bands. We demonstrate LS-DNN in two canonical inverse problems: super-resolution (SR) in diffraction-limited imaging (DLI), and quantitative phase retrieval (QPR). Our results also show comparable or improved performance over perceptual-loss based SR [21], and can be generalized to a wider range of image recovery problems.

cs.CV↗

arXiv2018

Spectral pre-modulation of training examples enhances the spatial resolution of the Phase Extraction Neural Network (PhENN)

The Phase Extraction Neural Network (PhENN) is a computational architecture, based on deep machine learning, for lens-less quantitative phase retrieval from raw intensity data. PhENN is a deep convolutional neural network trained through examples consisting of pairs of true phase objects and their corresponding intensity diffraction patterns; thereafter, given a test raw intensity pattern PhENN is capable of reconstructing the original phase object robustly, in many cases even for objects outside the database where the training examples were drawn from. Here, we show that the spatial frequency content of the training examples is an important factor limiting PhENN's spatial frequency response. For example, if the training database is relatively sparse in high spatial frequencies, as most natural scenes are, PhENN's ability to resolve fine spatial features in test patterns will be correspondingly limited. To combat this issue, we propose "flattening" the power spectral density of the training examples before presenting them to PhENN. For phase objects following the statistics of natural scenes, we demonstrate experimentally that the spectral pre-modulation method enhances the spatial resolution of PhENN by a factor of 2.

eess.IV↗

arXiv2018

Low Photon Count Phase Retrieval Using Deep Learning

Imaging systems' performance at low light intensity is affected by shot noise, which becomes increasingly strong as the power of the light source decreases. In this paper we experimentally demonstrate the use of deep neural networks to recover objects illuminated with weak light and demonstrate better performance than with the classical Gerchberg-Saxton phase retrieval algorithm for equivalent signal over noise ratio. Prior knowledge about the object is implicitly contained in the training data set and feature detection is possible for a signal over noise ratio close to one. We apply this principle to a phase retrieval problem and show successful recovery of the object's most salient features with as little as one photon per detector pixel on average in the illumination beam. We also show that the phase reconstruction is significantly improved by training the neural network with an initial estimate of the object, as opposed as training it with the raw intensity measurement.

eess.IV↗