arXiv subjects

Demetri Psaltis

Publications and source records attributed to Demetri Psaltis.

At least 19 recordsLinked to original sources

arXiv2025

High resolution Fluorescence lifetime IMaging Micro-Endoscopy (FLIMME) using a single multimode fiber

Endoscopic optical imaging using a single multimode fiber (MMF) has emerged as a promising approach for highly compact, minimally invasive, and high-resolution imaging. Unlike conventional fiber bundles, MMF-based endomicroscopes exploit the controlled excitation of multiple spatially overlapping modes in a single MMF. of core diameters of tens of micrometers. to deliver and collect light to form images with sub-micrometer resolution. Here, we introduce a fluorescence lifetime imaging microscopy (FLIM) modality to the MMF endomicroscope. We use amplitude modulation of a 405 nm single-mode light source at radio frequency (RF) and lock-in detection of autofluorescence to obtain intensity and lifetime images at sub-micrometer resolution. We experimentally demonstrate the capability of the ultrathin endomicroscope to perform label-free imaging in thick ex vivo murine submandibular gland tissue. With a temporal resolution of 0.03 ns, the FLIM images show distinguished structures of lifetime differences down to 0.5 ns. The combination of sub-micrometer fluorescence intensity and lifetime images in a minimally invasive endomicroscope opens new avenues for label-free cancer detection.

physics.optics↗

arXiv2025

Solar Forecasting with Causality: A Graph-Transformer Approach to Spatiotemporal Dependencies

Accurate solar forecasting underpins effective renewable energy management. We present SolarCAST, a causally informed model predicting future global horizontal irradiance (GHI) at a target site using only historical GHI from site X and nearby stations S - unlike prior work that relies on sky-camera or satellite imagery requiring specialized hardware and heavy preprocessing. To deliver high accuracy with only public sensor data, SolarCAST models three classes of confounding factors behind X-S correlations using scalable neural components: (i) observable synchronous variables (e.g., time of day, station identity), handled via an embedding module; (ii) latent synchronous factors (e.g., regional weather patterns), captured by a spatio-temporal graph neural network; and (iii) time-lagged influences (e.g., cloud movement across stations), modeled with a gated transformer that learns temporal shifts. It outperforms leading time-series and multimodal baselines across diverse geographical conditions, and achieves a 25.9% error reduction over the top commercial forecaster, Solcast. SolarCAST offers a lightweight, practical, and generalizable solution for localized solar forecasting.

cs.LG↗

arXiv2025

Exploring Expert Specialization through Unsupervised Training in Sparse Mixture of Experts

Understanding the internal organization of neural networks remains a fundamental challenge in deep learning interpretability. We address this challenge by exploring a novel Sparse Mixture of Experts Variational Autoencoder (SMoE-VAE) architecture. We test our model on the QuickDraw dataset, comparing unsupervised expert routing against a supervised baseline guided by ground-truth labels. Surprisingly, we find that unsupervised routing consistently achieves superior reconstruction performance. The experts learn to identify meaningful sub-categorical structures that often transcend human-defined class boundaries. Through t-SNE visualizations and reconstruction analysis, we investigate how MoE models uncover fundamental data structures that are more aligned with the model's objective than predefined labels. Furthermore, our study on the impact of dataset size provides insights into the trade-offs between data quantity and expert specialization, offering guidance for designing efficient MoE architectures.

cs.LG↗

arXiv2025

Solar Multimodal Transformer: Intraday Solar Irradiance Predictor using Public Cameras and Time Series

Accurate intraday solar irradiance forecasting is crucial for optimizing dispatch planning and electricity trading. For this purpose, we introduce a novel and effective approach that includes three distinguishing components from the literature: 1) the uncommon use of single-frame public camera imagery; 2) solar irradiance time series scaled with a proposed normalization step, which boosts performance; and 3) a lightweight multimodal model, called Solar Multimodal Transformer (SMT), that delivers accurate short-term solar irradiance forecasting by combining images and scaled time series. Benchmarking against Solcast, a leading solar forecasting service provider, our model improved prediction accuracy by 25.95%. Our approach allows for easy adaptation to various camera specifications, offering broad applicability for real-world solar forecasting challenges.

cs.CV↗

arXiv2025

Training Hybrid Neural Networks with Multimode Optical Nonlinearities Using Digital Twins

The ability to train ever-larger neural networks brings artificial intelligence to the forefront of scientific and technical discoveries. However, their exponentially increasing size creates a proportionally greater demand for energy and computational hardware. Incorporating complex physical events in networks as fixed, efficient computation modules can address this demand by decreasing the complexity of trainable layers. Here, we utilize ultrashort pulse propagation in multimode fibers, which perform large-scale nonlinear transformations, for this purpose. Training the hybrid architecture is achieved through a neural model that differentiably approximates the optical system. The training algorithm updates the neural simulator and backpropagates the error signal over this proxy to optimize layers preceding the optical one. Our experimental results achieve state-of-the-art image classification accuracies and simulation fidelity. Moreover, the framework demonstrates exceptional resilience to experimental drifts. By integrating low-energy physical systems into neural networks, this approach enables scalable, energy-efficient AI models with significantly reduced computational demands.

physics.optics↗

arXiv2024

Optical Diffusion Models for Image Generation

Diffusion models generate new samples by progressively decreasing the noise from the initially provided random distribution. This inference procedure generally utilizes a trained neural network numerous times to obtain the final output, creating significant latency and energy consumption on digital electronic hardware such as GPUs. In this study, we demonstrate that the propagation of a light beam through a semi-transparent medium can be programmed to implement a denoising diffusion model on image samples. This framework projects noisy image patterns through passive diffractive optical layers, which collectively only transmit the predicted noise term in the image. The optical transparent layers, which are trained with an online training approach, backpropagating the error to the analytical model of the system, are passive and kept the same across different steps of denoising. Hence this method enables high-speed image generation with minimal power consumption, benefiting from the bandwidth and energy efficiency of optical information processing.

physics.optics↗

arXiv2024

Training of Physical Neural Networks

Physical neural networks (PNNs) are a class of neural-like networks that leverage the properties of physical systems to perform computation. While PNNs are so far a niche research area with small-scale laboratory demonstrations, they are arguably one of the most underappreciated important opportunities in modern AI. Could we train AI models 1000x larger than current ones? Could we do this and also have them perform inference locally and privately on edge devices, such as smartphones or sensors? Research over the past few years has shown that the answer to all these questions is likely "yes, with enough research": PNNs could one day radically change what is possible and practical for AI systems. To do this will however require rethinking both how AI models work, and how they are trained - primarily by considering the problems through the constraints of the underlying hardware physics. To train PNNs at large scale, many methods including backpropagation-based and backpropagation-free approaches are now being explored. These methods have various trade-offs, and so far no method has been shown to scale to the same scale and performance as the backpropagation algorithm widely used in deep learning today. However, this is rapidly changing, and a diverse ecosystem of training techniques provides clues for how PNNs may one day be utilized to create both more efficient realizations of current-scale AI models, and to enable unprecedented-scale models.

physics.app-ph↗

arXiv2024

Second-harmonic optical diffraction tomography

Optical diffraction tomography (ODT) has emerged as an important label-free tool in biomedicine to measure the three-dimensional (3D) structure of a biological sample. In this paper, we describe ODT using second-harmonic generation (SHG) which is a coherent nonlinear optical process with a strict symmetry selectivity and has several advantages over traditional fluorescence methods. We report the tomographic retrieval of the 3D second-order nonlinear optical susceptibility using two-dimensional holographic measurements of the SHG fields at different illumination angles and polarization states. The method is a generalization of the conventional linear ODT to the nonlinear scenario. We demonstrate the method with a numerically simulated nanoparticle distribution and an experiment with muscle tissue fibers. Our results show that SHG ODT does not only provide an effective contrast mechanism for label-free imaging but also due to the symmetry requirement enables the visualization of properties that are not otherwise accessible.

physics.optics↗

arXiv2024

Programming the scalable optical learning operator with spatial-spectral optimization

Electronic computers have evolved drastically over the past years with an ever-growing demand for improved performance. However, the transfer of information from memory and high energy consumption have emerged as issues that require solutions. Optical techniques are considered promising solutions to these problems with higher speed than their electronic counterparts and with reduced energy consumption. Here, we use the optical reservoir computing framework we have previously described (Scalable Optical Learning Operator or SOLO) to program the spatial-spectral output of the light after nonlinear propagation in a multimode fiber. The novelty in the current paper is that the system is programmed through an output sampling scheme, similar to that used in hyperspectral imaging in astronomy. Linear and nonlinear computations are performed by light in the multimode fiber and the high dimensional spatial-spectral information at the fiber output is optically programmed before it reaches the camera. We then used a digital computer to classify the programmed output of the multi-mode fiber using a simple, single layer network. When combining front-end programming and the proposed spatial-spectral programming, we were able to achieve 89.9% classification accuracy on the dataset consisting of chest X-ray images from COVID-19 patients. At the same time, we obtained a decrease of 99% in the number of tunable parameters compared to an equivalently performing digital neural network. These results show that the performance of programmed SOLO is comparable with cutting-edge electronic computing platforms, albeit with a much-reduced number of electronic operations.

physics.optics↗

arXiv2024

Multicasting Optical Reconfigurable Switch

Artificial Intelligence (AI) demands large data flows within datacenters, heavily relying on multicasting data transfers. As AI models scale, the requirement for high-bandwidth and low-latency networking compounds. The common use of electrical packet switching faces limitations due to optical-electrical-optical conversion bottlenecks. Optical switches, while bandwidth-agnostic and low-latency, suffer from having only unicast or non-scalable multicasting capability. This paper introduces an optical switching technique addressing this challenge. Our approach enables arbitrarily programmable simultaneous unicast and multicast connectivity, eliminating the need for optical splitters that hinder scalability due to optical power loss. We use phase modulation in multiple layers, tailored to implement any multicast connectivity map. Phase modulation also enables wavelength selectivity on top of spatial selectivity, resulting in an optical switch that implements space-wavelength routing. We conducted simulations and experiments to validate our approach. Our results affirm the concept's feasibility, effectiveness, and scalability, as a multicasting switch by experimentally demonstrating 16 spatial ports using 2 wavelength channels. Numerically, 64 spatial ports with 4 wavelength channels each were simulated, with approximately constant efficiency (< 3 dB) as ports and wavelength channels scale.

physics.optics↗

arXiv2024

Nonlinear Processing with Linear Optics

Deep neural networks have achieved remarkable breakthroughs by leveraging multiple layers of data processing to extract hidden representations, albeit at the cost of large electronic computing power. To enhance energy efficiency and speed, the optical implementation of neural networks aims to harness the advantages of optical bandwidth and the energy efficiency of optical interconnections. In the absence of low-power optical nonlinearities, the challenge in the implementation of multilayer optical networks lies in realizing multiple optical layers without resorting to electronic components. In this study, we present a novel framework that uses multiple scattering that is capable of synthesizing programmable linear and nonlinear transformations concurrently at low optical power by leveraging the nonlinear relationship between the scattering potential, represented by data, and the scattered field. Theoretical and experimental investigations show that repeating the data by multiple scattering enables non-linear optical computing at low power continuous wave light. Moreover, we empirically found that scaling of this optical framework follows the power law as in state-of-the-art deep digital networks.

physics.optics↗

arXiv2024

Subwavelength Imaging using a Solid-Immersion Diffractive Optical Processor

Phase imaging is widely used in biomedical imaging, sensing, and material characterization, among other fields. However, direct imaging of phase objects with subwavelength resolution remains a challenge. Here, we demonstrate subwavelength imaging of phase and amplitude objects based on all-optical diffractive encoding and decoding. To resolve subwavelength features of an object, the diffractive imager uses a thin, high-index solid-immersion layer to transmit high-frequency information of the object to a spatially-optimized diffractive encoder, which converts/encodes high-frequency information of the input into low-frequency spatial modes for transmission through air. The subsequent diffractive decoder layers (in air) are jointly designed with the encoder using deep-learning-based optimization, and communicate with the encoder layer to create magnified images of input objects at its output, revealing subwavelength features that would otherwise be washed away due to diffraction limit. We demonstrate that this all-optical collaboration between a diffractive solid-immersion encoder and the following decoder layers in air can resolve subwavelength phase and amplitude features of input objects in a highly compact design. To experimentally demonstrate its proof-of-concept, we used terahertz radiation and developed a fabrication method for creating monolithic multi-layer diffractive processors. Through these monolithically fabricated diffractive encoder-decoder pairs, we demonstrated phase-to-intensity transformations and all-optically reconstructed subwavelength phase features of input objects by directly transforming them into magnified intensity features at the output. This solid-immersion-based diffractive imager, with its compact and cost-effective design, can find wide-ranging applications in bioimaging, endoscopy, sensing and materials characterization.

physics.optics↗

arXiv2023

Volume holograms with linear diffraction efficiency relation by (3+1)D printing

We demonstrate the fabrication of volume holograms using 2-photon polymerization with dynamic control of light exposure. We refer to our method as (3+1)D printing. Volume holograms that are recorded by interfering reference and signal beams have a diffraction efficiency relation that is inversely proportional with the square of the number of superimposed holograms. By using (3+1)D printing for fabrication, the refractive index of each voxel is created independently and thus by, digitally filtering the undesired interference terms, the diffraction efficiency is now inversely proportional to the number of multiplexed gratings. We experimentally demonstrated this linear dependence by recording M=50 volume gratings. To the best of our knowledge, this is the first experimental demonstration of distributed volume holograms that overcome the 1/M^2 limit.

physics.optics↗

arXiv2023

Forward-Forward Training of an Optical Neural Network

Neural networks (NN) have demonstrated remarkable capabilities in various tasks, but their computation-intensive nature demands faster and more energy-efficient hardware implementations. Optics-based platforms, using technologies such as silicon photonics and spatial light modulators, offer promising avenues for achieving this goal. However, training multiple trainable layers in tandem with these physical systems poses challenges, as they are difficult to fully characterize and describe with differentiable functions, hindering the use of error backpropagation algorithm. The recently introduced Forward-Forward Algorithm (FFA) eliminates the need for perfect characterization of the learning system and shows promise for efficient training with large numbers of programmable parameters. The FFA does not require backpropagating an error signal to update the weights, rather the weights are updated by only sending information in one direction. The local loss function for each set of trainable weights enables low-power analog hardware implementations without resorting to metaheuristic algorithms or reinforcement learning. In this paper, we present an experiment utilizing multimode nonlinear wave propagation in an optical fiber demonstrating the feasibility of the FFA approach using an optical system. The results show that incorporating optical transforms in multilayer NN architectures trained with the FFA, can lead to performance improvements, even with a relatively small number of trainable weights. The proposed method offers a new path to the challenge of training optical NNs and provides insights into leveraging physical transformations for enhancing NN performance.

cs.LG↗

arXiv2022

Predicting nonlinear optical scattering with physics-driven neural networks

Deep neural networks trained on physical losses are emerging as promising surrogates of nonlinear numerical solvers. These tools can predict solutions of Maxwell's equations and compute gradients of output fields with respect to the material and geometrical properties in millisecond times which makes them attractive for inverse design or inverse scattering applications. Here we develop a tunable version of MaxwellNet, a physics driven neural network able to compute light scattering from inhomogenous media with a size comparable with the incident wavelength in the presence of the optical Kerr effect. The weights of the network are dynamically adjusted to take into account the intensity-dependent refractive index of the material.

physics.optics↗

arXiv2022

Nonlinear Optical Data Transformer for Machine Learning

Modern machine learning models use an ever-increasing number of parameters to train (175 billion parameters for GPT-3) with large datasets to obtain better performance. Bigger is better has been the norm. Optical computing has been reawakened as a potential solution to large-scale computing through optical accelerators that carry out linear operations while reducing electrical power. However, to achieve efficient computing with light, creating and controlling nonlinearity optically rather than electronically remains a challenge. This study explores a reservoir computing (RC) approach whereby a 14 mm long few-mode waveguide in LiNbO3 on insulator is used as a complex nonlinear optical processor. A dataset is encoded digitally on the spectrum of a femtosecond pulse which is then launched in the waveguide. The output spectrum depends nonlinearly on the input. We experimentally show that a simple digital linear classifier with 784 parameters using the output spectrum from the waveguide as input increased the classification accuracy of several databases compared to non-transformed data, approximately 10$\%$. In comparison, a deep digital neural network (NN) with 40000 parameters was necessary to achieve the same accuracy. Reducing the number of parameters by a factor of $\sim$50 illustrates that a compact optical RC approach can perform on par with a deep digital NN.

physics.optics↗

arXiv2022

Programming Nonlinear Propagation for Efficient Optical Learning Machines

The ever-increasing demand for processing data with larger machine learning models requires more efficient hardware solutions due to limitations such as power dissipation and scalability. Optics is a promising contender for providing lower power computation since light propagation through a non-absorbing medium is a lossless operation. However, to carry out useful and efficient computations with light, generating and controlling nonlinearity optically is a necessity that is still elusive. Multimode fibers (MMF) have been shown that they can provide nonlinear effects with microwatts of average power while maintaining parallelism and low loss. In this work, we propose an optical neural network architecture, which performs nonlinear optical computation by controlling the propagation of ultrashort pulses in MMF by wavefront shaping. With a surrogate model, optimal sets of parameters are found to program this optical computer for different tasks with minimal utilization of an electronic computer. We show a remarkable decrease of 97% in the number of model parameters, which leads to an overall 99% digital operation reduction compared to an equivalently performing digital neural network. We further demonstrate that a fully optical implementation can also be performed with competitive accuracies.

cs.ET↗

arXiv2022

Physics-informed neural networks for diffraction tomography

We propose a physics-informed neural network as the forward model for tomographic reconstructions of biological samples. We demonstrate that by training this network with the Helmholtz equation as a physical loss, we can predict the scattered field accurately. It will be shown that a pretrained network can be fine-tuned for different samples and used for solving the scattering problem much faster than other numerical solutions. We evaluate our methodology with numerical and experimental results. Our physics-informed neural networks can be generalized for any forward and inverse scattering problem.

physics.optics↗