arXiv subjects

Yoav Shechtman

Publications and source records attributed to Yoav Shechtman.

18 recordsLinked to original sources

arXiv2026

High-Density Monocular 3D Particle Image Velocimetry by Wavefront Shaping and Deep Learning

Three-dimensional (3D) Particle Image Velocimetry (PIV) measures flow velocities by imaging laser-illuminated tracer particles seeded in a fluid, and is widely used in both academia and industry. Many applications require a compact setup for optical accessibility, ideally with a single camera, while also demanding high seeding densities for accurate velocimetry. These requirements, however, are typically incompatible: monocular methods break down at high densities; high-density measurements instead generally rely on multi-camera tomographic systems. Here, we introduce Point-spread-function-Engineering Training-based PIV (PET-PIV), a compact monocular 3D velocimetry approach that resolves this long-standing compactness-density trade-off through a minimal optical modification and deep-learning-based algorithms. PET-PIV requires only the insertion of a single thin phase mask to an otherwise conventional PIV setup, beneath the objective in microscopy or at the lens iris in macro-scale imaging, and calibrates the resulting imaging system in situ, making the approach straightforward to implement and readily scalable. Computationally, PET-PIV operates in two complementary regimes: a tracking/Lagrangian mode that localizes and links individual particles, and, more importantly, a field mode for ultra-high densities which directly reconstructs 3D velocity fields from 2D image sequences. In realistic experimental validations using a tomographic PIV system, PET-PIV demonstrates strong agreement with ground-truth references, with correlation coefficients (CC) exceeding 0.97, alongside an order-of-magnitude improvement in computational speed. Notably, PET-PIV enables monocular 3D macro-scale PIV at densities previously achievable only with multi-camera setups, dramatically expanding the applicability of 3D velocimetry in space-constrained and optically restricted environments.

physics.optics↗

arXiv2023

On the Capacity of DNA Labeling

DNA labeling is a powerful tool in molecular biology and biotechnology that allows for the visualization, detection, and study of DNA at the molecular level. Under this paradigm, a DNA molecule is being labeled by specific k patterns and is then imaged. Then, the resulted image is modeled as a (k + 1)- ary sequence in which any non-zero symbol indicates on the appearance of the corresponding label in the DNA molecule. The primary goal of this work is to study the labeling capacity, which is defined as the maximal information rate that can be obtained using this labeling process. The labeling capacity is computed for any single label and several results are provided for multiple labels as well. Moreover, we provide the optimal minimal number of labels of length one or two that are needed in order to gain labeling capacity of 2.

cs.IT↗

arXiv2023

Fluorescence Microscopy: a statistics-optics perspective

Fundamental properties of light unavoidably impose features on images collected using fluorescence microscopes. Modeling these features is ever more important in quantitatively interpreting microscopy images collected at scales on par or smaller than light's wavelength. Here we review the optics responsible for generating fluorescent images, fluorophore properties, microscopy modalities leveraging properties of both light and fluorophores, in addition to the necessarily probabilistic modeling tools imposed by the stochastic nature of light and measurement.

physics.optics↗

arXiv2023

Additive manufacturing of solid diffractive optical elements via near index matching

Diffractive optical elements (DOEs) have a wide range of applications in optics and photonics, thanks to their capability to perform complex wavefront shaping in a compact form. However, widespread applicability of DOEs is still limited, because existing fabrication methods are cumbersome and expensive. Here, we present a simple and cost-effective fabrication approach for solid, high-performance DOEs. The method is based on conjugating two nearly refractive index-matched solidifiable transparent materials. The index matching allows for extreme scaling up of the elements in the axial dimension, which enables simple fabrication of a template using commercially available 3D printing at tens-of-micrometer resolution. We demonstrated the approach by fabricating and using DOEs serving as microlens arrays, vortex plates, including for highly sensitive applications such as vector beam generation and super-resolution microscopy using MINSTED, and phase-masks for three-dimensional single-molecule localization microscopy. Beyond the advantage of making DOEs widely accessible by drastically simplifying their production, the method also overcomes difficulties faced by existing methods in fabricating highly complex elements, such as high-order vortex plates, and spectrum-encoding phase masks for microscopy.

physics.optics↗

arXiv2023

Roadmap on Deep Learning for Microscopy

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

physics.optics↗

arXiv2022

Diffractive optical system design by cascaded propagation

Modern design of complex optical systems relies heavily on computational tools. These typically utilize geometrical optics as well as Fourier optics, which enables the use of diffractive elements to manipulate light with features on the scale of a wavelength. Fourier optics is typically used for designing thin elements, placed in the system's aperture, generating a shift-invariant Point Spread Function (PSF). A major bottleneck in applying Fourier Optics in many cases of interest, e.g. when dealing with multiple, or out-of-aperture elements, comes from numerical complexity. In this work, we propose and implement an efficient and differentiable propagation model based on the Collins integral, which enables the optimization of diffraction optical systems with unprecedented design freedom using backpropagation. We demonstrate the applicability of our method, numerically and experimentally, by engineering shift-variant PSFs via thin plate elements placed in arbitrary planes inside complex imaging systems, performing cascaded optimization of multiple planes, and designing optimal machine-vision systems by deep learning.

physics.optics↗

arXiv2021

Programmable thermocapillary shaping of thin liquid films

We present a method that leverages projected light patterns as a mechanism for freeform deformations of a thin liquid film via the thermocapillary effect. We developed a closed-form solution for the inverse problem of the thin-film evolution equation, allowing to obtain the projection pattern required in order to achieve a desired topography. We experimentally implement the method using a computer controlled light projector, which illuminates any desired pattern onto the bottom of a fluidic chamber patterned with heat absorbing metal pads. The resulting heat map induces surface tension gradients in the liquid-air interface, giving rise to thermocapillary flow that deforms the liquid surface. If a polymer is used for the liquid film, it can then be photocured to yield a solid device. Based on the inverse problem solutions and using this system, we demonstrate the fabrication of several diffractive optical elements (DOEs), including phase masks for extended depth of field imaging, and for 3D localization microscopy. The entire process, from projection to solidification, is completed in less than five minutes, and yields a sub-nanometric surface quality without any post-processing.

physics.optics↗

arXiv2021

Multiplexed PSF engineering for 3D multicolor particle tracking

Three-dimensional spatiotemporal tracking of microscopic particles in multiple colors is a challenging optical imaging task. Existing approaches require a trade-off between photon-efficiency, field of view, mechanical complexity, spectral specificity and speed. Here, we introduce multiplexed point-spread function engineering that achieves photon efficient, 3D, multicolor particle tracking over a large field of view. This is accomplished by first chromatically splitting the emission path of a microscope to different channels, engineering the point-spread function of each, and then recombining them onto the same region of the camera. We demonstrate our technique for simultaneously tracking five types of emitters in-vitro, as well as co-localization of DNA loci in live yeast cells.

physics.optics↗

arXiv2020

Learning an optimal PSF-pair for ultra-dense 3D localization microscopy

A long-standing challenge in multiple-particle-tracking is the accurate and precise 3D localization of individual particles at close proximity. One established approach for snapshot 3D imaging is point-spread-function (PSF) engineering, in which the PSF is modified to encode the axial information. However, engineered PSFs are challenging to localize at high densities due to lateral PSF overlaps. Here we suggest using multiple PSFs simultaneously to help overcome this challenge, and investigate the problem of engineering multiple PSFs for dense 3D localization. We implement our approach using a bifurcated optical system that modifies two separate PSFs, and design the PSFs using three different approaches including end-to-end learning. We demonstrate our approach experimentally by volumetric imaging of fluorescently labelled telomeres in cells.

eess.IV↗

arXiv2019

DeepSTORM3D: dense three dimensional localization microscopy and point spread function design by deep learning

Localization microscopy is an imaging technique in which the positions of individual nanoscale point emitters (e.g. fluorescent molecules) are determined at high precision from their images. This is the key ingredient in single/multiple-particle-tracking and several super-resolution microscopy approaches. Localization in three-dimensions (3D) can be performed by modifying the image that a point-source creates on the camera, namely, the point-spread function (PSF). The PSF is engineered using additional optical elements to vary distinctively with the depth of the point-source. However, localizing multiple adjacent emitters in 3D poses a significant algorithmic challenge, due to the lateral overlap of their PSFs. Here, we train a neural network to receive an image containing densely overlapping PSFs of multiple emitters over a large axial range and output a list of their 3D positions. Furthermore, we then use the network to design the optimal PSF for the multi-emitter case. We demonstrate our approach numerically as well as experimentally by 3D STORM imaging of mitochondria, and volumetric imaging of dozens of fluorescently-labeled telomeres occupying a mammalian nucleus in a single snapshot.

eess.IV↗

arXiv2018

Multicolor localization microscopy by deep learning

Deep learning has become an extremely effective tool for image classification and image restoration problems. Here, we apply deep learning to microscopy, and demonstrate how neural networks can exploit the chromatic dependence of the point-spread function to classify the colors of single emitters imaged on a grayscale camera. While existing single-molecule methods for spectral classification require additional optical elements in the emission path, e.g. spectral filters, prisms, or phase masks, our neural net correctly identifies static as well as mobile emitters with high efficiency using a standard, unmodified single-channel configuration. Furthermore, we demonstrate how deep learning can be used to design phase-modulating elements that, when implemented into the imaging path, result in further improved color differentiation between species.

physics.optics↗

arXiv2018

Fundamental precision bounds for three-dimensional optical localization microscopy with Poisson statistics

Point source localization is a problem of persistent interest in optical imaging. In particular, a number of widely used biological microscopy techniques rely on precise three-dimensional localization of single fluorophores. As emitter depth localization is more challenging than lateral localization, considerable effort has been spent on engineering the response of the microscope in a way that reveals increased depth information. Here we consider the theoretical limits of such approaches by deriving the quantum Cram\'{e}r-Rao bound (QCRB). We show that existing methods for depth localization with single-objective detection exceed the QCRB by a factor $>\sqrt{2}$, and propose an interferometer arrangement that approaches the bound. We also show that for detection with two opposed objectives, established interferometric measurement techniques globally reach the QCRB.

physics.optics↗

arXiv2018

Deep-STORM: super-resolution single-molecule microscopy by deep learning

We present an ultra-fast, precise, parameter-free method, which we term Deep-STORM, for obtaining super-resolution images from stochastically-blinking emitters, such as fluorescent molecules used for localization microscopy. Deep-STORM uses a deep convolutional neural network that can be trained on simulated data or experimental measurements, both of which are demonstrated. The method achieves state-of-the-art resolution under challenging signal-to-noise conditions and high emitter densities, and is significantly faster than existing approaches. Additionally, no prior information on the shape of the underlying structure is required, making the method applicable to any blinking data-set. We validate our approach by super-resolution image reconstruction of simulated and experimentally obtained data.

physics.optics↗

arXiv2017

Measurement-based estimation of global pupil functions in 3D localization microscopy

We report the use of a phase retrieval procedure based on maximum likelihood estimation (MLE) to produce an improved, experimentally calibrated model of a point spread function (PSF) for use in three-dimensional (3D) localization microscopy experiments. The method estimates a global pupil phase function (which includes both the PSF and system aberrations) over the full axial range from a simple calibration scan. The pupil function is used to refine the PSF model and hence enable superior localizations from experimental data. To demonstrate the utility of the procedure, we apply it to experimental data acquired with a microscope employing a tetrapod PSF with a 6 micron axial range. The phase-retrieved model demonstrates significant improvements in both accuracy and precision of 3D localizations relative to the model based on scalar diffraction theory. The localization precision of the phase-retrieved model is shown to be near the limits imposed by estimation theory, and the reproducibility of the procedure is characterized and discussed. Code which performs the phase retrieval algorithm is provided.

physics.optics↗

arXiv2014

Structure-Based Super-Resolution Recovery of Three-Photon Quantum States from Two-Fold Coincidences

The field of quantum information has been growing fast over the past decade. Optical quantum computation, based on the concepts of KLM and cluster states, has witnessed experimental realizations of larger and more complex systems in terms of photon number. Quantum optical systems, which offer long coherence times and easy manipulation of single qubits, allow us to probe quantum properties of the light itself and of the physical systems around it. Recently, a linear scheme for quantum computing, relying on the bosonic nature of particles, has been proposed and realized experimentally with photons. The ability to efficiently measure superpositions of quantum states consisting of several photons is essential to the characterization of such systems. In fact, the entire field of quantum information completely relies on the ability to recover quantum states from measurements. However, the characterization of quantum states requires many measurements, and often necessitates complicated measurements schemes; for example, characterizing qubits requires measurements. Here, we utilize structure, inherent to physically interesting quantum states of light, in order to reduce the complexity in the recovery of a quantum state. In particular, we devise a method enabling the recovery of three-photon quantum states from only two-fold correlation measurements in a single setting. The ability to take two-fold coincidences instead of three-fold offers the recovery of the quantum states in far less measurements and in a considerably higher SNR, because detection of two-photon events is much more likely than that of three-photon ones. The concept suggested here paves the way to further ideas on structure-based super resolution in quantum state tomography, such as recovering a quantum state in an unknown basis in a single setup and recovering the state of several photons without number resolving detectors.

quant-ph↗

arXiv2014

Phase Retrieval with Application to Optical Imaging

This review article provides a contemporary overview of phase retrieval in optical imaging, linking the relevant optical physics to the information processing methods and algorithms. Its purpose is to describe the current state of the art in this area, identify challenges, and suggest vision and areas where signal processing methods can have a large impact on optical imaging and on the world of imaging at large, with applications in a variety of fields ranging from biology and chemistry to physics and engineering.

cs.IT↗

arXiv2013

GESPAR: Efficient Phase Retrieval of Sparse Signals

We consider the problem of phase retrieval, namely, recovery of a signal from the magnitude of its Fourier transform, or of any other linear transform. Due to the loss of the Fourier phase information, this problem is ill-posed. Therefore, prior information on the signal is needed in order to enable its recovery. In this work we consider the case in which the signal is known to be sparse, i.e., it consists of a small number of nonzero elements in an appropriate basis. We propose a fast local search method for recovering a sparse signal from measurements of its Fourier transform (or other linear transform) magnitude which we refer to as GESPAR: GrEedy Sparse PhAse Retrieval. Our algorithm does not require matrix lifting, unlike previous approaches, and therefore is potentially suitable for large scale problems such as images. Simulation results indicate that GESPAR is fast and more accurate than existing techniques in a variety of settings.

cs.IT↗

arXiv2011

Sparsity based sub-wavelength imaging with partially incoherent light via quadratic compressed sensing

We demonstrate that sub-wavelength optical images borne on partially-spatially-incoherent light can be recovered, from their far-field or from the blurred image, given the prior knowledge that the image is sparse, and only that. The reconstruction method relies on the recently demonstrated sparsity-based sub-wavelength imaging. However, for partially-spatially-incoherent light, the relation between the measurements and the image is quadratic, yielding non-convex measurement equations that do not conform to previously used techniques. Consequently, we demonstrate new algorithmic methodology, referred to as quadratic compressed sensing, which can be applied to a range of other problems involving information recovery from partial correlation measurements, including when the correlation function has local dependencies. Specifically for microscopy, this method can be readily extended to white light microscopes with the additional knowledge of the light source spectrum.

cs.IT↗