Searcharxiv⌕ Search

arXiv subjects

Maxim Ziatdinov

Publications and source records attributed to Maxim Ziatdinov.

At least 37 records · Page 2Linked to original sources
arXiv2023

When the atoms dance: exploring mechanisms of electron-beam induced modifications of materials with machine-learning assisted high temporal resolution electron microscopy

Directed atomic fabrication using an aberration-corrected scanning transmission electron microscope (STEM) opens new pathways for atomic engineering of functional materials. In this approach, the electron beam is used to actively alter the atomic structure through electron beam induced irradiation processes. One of the impediments that has limited widespread use thus far has been the ability to understand the fundamental mechanisms of atomic transformation pathways at high spatiotemporal resolution. Here, we develop a workflow for obtaining and analyzing high-speed spiral scan STEM data, up to 100 fps, to track the atomic fabrication process during nanopore milling in monolayer MoS2. An automated feedback-controlled electron beam positioning system combined with deep convolution neural network (DCNN) was used to decipher fast but low signal-to-noise datasets and classify time-resolved atom positions and nature of their evolving atomic defect configurations. Through this automated decoding, the initial atomic disordering and reordering processes leading to nanopore formation was able to be studied across various timescales. Using these experimental workflows a greater degree of speed and information can be extracted from small datasets without compromising spatial resolution. This approach can be adapted to other 2D materials systems to gain further insights into the defect formation necessary to inform future automated fabrication techniques utilizing the STEM electron beam.

cond-mat.mtrl-sci↗
arXiv2023

Physics-driven discovery and bandgap engineering of hybrid perovskites

The unique aspect of the hybrid perovskites is their tunability, allowing to engineer the bandgap via substitution. From application viewpoint, this allows creation of the tandem cells between perovskites and silicon, or two or more perovskites, with associated increase of efficiency beyond single-junction Schokley-Queisser limit. However, the concentration dependence of optical bandgap in the hybrid perovskite solid solutions can be non-linear and even non-monotonic, as determined by the band alignments between endmembers, presence of the defect states and Urbach tails, and phase separation. Exploring new compositions brings forth the joint problem of the discovery of the composition with the desired band gap, and establishing the physical model of the band gap concentration dependence. Here we report the development of the experimental workflow based on structured Gaussian Process (sGP) models and custom sGP (c-sGP) that allow the joint discovery of the experimental behavior and the underpinning physical model. This approach is verified with simulated data sets with known ground truth, and was found to accelerate the discovery of experimental behavior and the underlying physical model. The d/c-sGP approach utilizes a few calculated thin film bandgap data points to guide targeted explorations, minimizing the number of thin film preparations. Through iterative exploration, we demonstrate that the c-sGP algorithm that combined 5 bandgap models converges rapidly, revealing a relationship in the bandgap diagram of MA1-xGAxPb(I1-xBrx)3. This approach offers a promising method for efficiently understanding the physical model of band gap concentration dependence in the binary systems, this method can also be extended to ternary or higher dimensional systems.

cond-mat.mtrl-sci↗
arXiv2023

Human-in-the-loop: The future of Machine Learning in Automated Electron Microscopy

Machine learning methods are progressively gaining acceptance in the electron microscopy community for de-noising, semantic segmentation, and dimensionality reduction of data post-acquisition. The introduction of the APIs by major instrument manufacturers now allows the deployment of ML workflows in microscopes, not only for data analytics but also for real-time decision-making and feedback for microscope operation. However, the number of use cases for real-time ML remains remarkably small. Here, we discuss some considerations in designing ML-based active experiments and pose that the likely strategy for the next several years will be human-in-the-loop automated experiments (hAE). In this paradigm, the ML learning agent directly controls beam position and image and spectroscopy acquisition functions, and human operator monitors experiment progression in real- and feature space of the system and tunes the policies of the ML agent to steer the experiment towards specific objectives.

cond-mat.mtrl-sci↗
arXiv2023

Towards Lightweight Data Integration using Multi-workflow Provenance and Data Observability

Modern large-scale scientific discovery requires multidisciplinary collaboration across diverse computing facilities, including High Performance Computing (HPC) machines and the Edge-to-Cloud continuum. Integrated data analysis plays a crucial role in scientific discovery, especially in the current AI era, by enabling Responsible AI development, FAIR, Reproducibility, and User Steering. However, the heterogeneous nature of science poses challenges such as dealing with multiple supporting tools, cross-facility environments, and efficient HPC execution. Building on data observability, adapter system design, and provenance, we propose MIDA: an approach for lightweight runtime Multi-workflow Integrated Data Analysis. MIDA defines data observability strategies and adaptability methods for various parallel systems and machine learning tools. With observability, it intercepts the dataflows in the background without requiring instrumentation while integrating domain, provenance, and telemetry data at runtime into a unified database ready for user steering queries. We conduct experiments showing end-to-end multi-workflow analysis integrating data from Dask and MLFlow in a real distributed deep learning use case for materials science that runs on multiple environments with up to 276 GPUs in parallel. We show near-zero overhead running up to 100,000 tasks on 1,680 CPU cores on the Summit supercomputer.

cs.DC↗
arXiv2023

Cyber Framework for Steering and Measurements Collection Over Instrument-Computing Ecosystems

We propose a framework to develop cyber solutions to support the remote steering of science instruments and measurements collection over instrument-computing ecosystems. It is based on provisioning separate data and control connections at the network level, and developing software modules consisting of Python wrappers for instrument commands and Pyro server-client codes that make them available across the ecosystem network. We demonstrate automated measurement transfers and remote steering operations in a microscopy use case for materials research over an ecosystem of Nion microscopes and computing platforms connected over site networks. The proposed framework is currently under further refinement and being adopted to science workflows with automated remote experiments steering for autonomous chemistry laboratories and smart energy grid simulations.

cs.OH↗
arXiv2023

Combining Variational Autoencoders and Physical Bias for Improved Microscopy Data Analysis

Electron and scanning probe microscopy produce vast amounts of data in the form of images or hyperspectral data, such as EELS or 4D STEM, that contain information on a wide range of structural, physical, and chemical properties of materials. To extract valuable insights from these data, it is crucial to identify physically separate regions in the data, such as phases, ferroic variants, and boundaries between them. In order to derive an easily interpretable feature analysis, combining with well-defined boundaries in a principled and unsupervised manner, here we present a physics augmented machine learning method which combines the capability of Variational Autoencoders to disentangle factors of variability within the data and the physics driven loss function that seeks to minimize the total length of the discontinuities in images corresponding to latent representations. Our method is applied to various materials, including NiO-LSMO, BiFeO3, and graphene. The results demonstrate the effectiveness of our approach in extracting meaningful information from large volumes of imaging data. The fully notebook containing implementation of the code and analysis workflow is available at https://github.com/arpanbiswas52/PaperNotebooks

cs.LG↗
arXiv2023

A processing and analytics system for microscopy data workflows: the Pycroscopy ecosystem of packages

Major advancements in fields as diverse as biology and quantum computing have relied on a multitude of microscopic techniques. All optical, electron and scanning probe microscopy advanced with new detector technologies and integration of spectroscopy, imaging, and diffraction. Despite the considerable proliferation of these instruments, significant bottlenecks remain in terms of processing, analysis, storage, and retrieval of acquired datasets. Aside from the lack of file standards, individual domain-specific analysis packages are often disjoint from the underlying datasets. Thus, keeping track of analysis and processing steps remains tedious for the end-user, hampering reproducibility. Here, we introduce the pycroscopy ecosystem of packages, an open-source python-based ecosystem underpinned by a common data model. Our data model, termed the N-dimensional spectral imaging data format, is realized in pycroscopy's sidpy package. This package is built on top of dask arrays, thus leveraging dask array attributes but expanding them to accelerate microscopy-relevant analysis and visualization. Several examples of the use of the pycroscopy ecosystem to create workflows for data ingestion and analysis are shown. Adoption of such standardized routines will be critical to usher in the next generation of autonomous instruments where processing, computation, and meta-data storage will be critical to overall experimental operations.

physics.ins-det↗
arXiv2023

Learning the right channel in multimodal imaging: automated experiment in Piezoresponse Force Microscopy

We report the development and experimental implementation of the automated experiment workflows for the identification of the best predictive channel for a phenomenon of interest in spectroscopic measurements. The approach is based on the combination of ensembled deep kernel learning for probabilistic predictions and a basic reinforcement learning policy for channel selection. It allows the identification of which of the available observational channels, sampled sequentially, are most predictive of selected behaviors, and hence have the strongest correlations. We implement this approach for multimodal imaging in Piezoresponse Force Microscopy (PFM), with the behaviors of interest manifesting in piezoresponse spectroscopy. We illustrate the best predictive channel for polarization-voltage hysteresis loop and frequency-voltage hysteresis loop areas is amplitude in the model samples. The same workflow and code are universal and applicable for any multimodal imaging and local characterization methods.

cond-mat.mtrl-sci↗
arXiv2023

Post-Experiment Forensics and Human-in-the-Loop Interventions in Explainable Autonomous Scanning Probe Microscopy

The broad adoption of machine learning (ML)-based automated and autonomous experiments (AE) in physical characterization and synthesis requires development of strategies for understanding and intervention in the experimental workflow. Here, we introduce and realize strategies for post-acquisition forensic analysis applied to the deep kernel learning based AE scanning probe microscopy. This approach yields real-time and post-acquisition indicators of the progression of an active learning process interacting with an experimental system. We further illustrate that this approach can be extended towards human-in-the-loop autonomous experiments, where human operators make high-level decisions at high latencies setting the policies for AE, and the ML algorithm performs low-level fast decisions. The proposed approach is universal and can be extended to other physical and chemical imaging techniques and applications such as combinatorial library analysis. The full forensic analysis notebook is publicly available on GitHub at https://github.com/yongtaoliu/Forensics-DKL-BEPS.

cond-mat.mtrl-sci↗
arXiv2023

Designing Workflows for Materials Characterization

Experimental science is enabled by the combination of synthesis, imaging, and functional characterization. Synthesis of a new material is typically followed by a set of characterization methods aiming to provide feedback for optimization or discover fundamental mechanisms. However, the sequence of synthesis and characterization methods and their interpretation, or research workflow, has traditionally been driven by human intuition and is highly domain specific. Here we explore concepts of scientific workflows that emerge at the interface between theory, characterization, and imaging. We discuss the criteria by which these workflows can be constructed for special cases of multi-resolution structural imaging and structural and functional characterization. Some considerations for theory-experiment workflows are provided. We further pose that the emergence of user facilities and cloud labs disrupt the classical progression from ideation, orchestration, and execution stages of workflow development and necessitate development of universal frameworks for workflow design, including universal hyper-languages describing laboratory operation, reward functions and their integration between domains, and policy development for workflow optimization. These tools will enable knowledge-based workflow optimization, enable lateral instrumental networks, sequential and parallel orchestration of characterization between dissimilar facilities, and empower distributed research.

cond-mat.mtrl-sci↗
arXiv2022

Exploring the microstructural origins of conductivity and hysteresis in metal halide perovskites via active learning driven automated scanning probe microscopy

Electronic transport and hysteresis in metal halide perovskites (MHPs) are key to the applications in photovoltaics, light emitting devices, and light and chemical sensors. These phenomena are strongly affected by the materials microstructure including grain boundaries, ferroic domain walls, and secondary phase inclusions. Here, we demonstrate an active machine learning framework for 'driving' an automated scanning probe microscope (SPM) to discover the microstructures responsible for specific aspects of transport behavior in MHPs. In our setup, the microscope can discover the microstructural elements that maximize the onset of conduction, hysteresis, or any other characteristic that can be derived from a set of current-voltage spectra. This approach opens new opportunities for exploring the origins of materials functionality in complex materials by SPM and can be integrated with other characterization techniques either before (prior knowledge) or after (identification of locations of interest for detail studies) functional probing.

cond-mat.mtrl-sci↗
arXiv2022

Physics discovery in nanoplasmonic systems via autonomous experiments in Scanning Transmission Electron Microscopy

Physics-driven discovery in an autonomous experiment has emerged as a dream application of machine learning in physical sciences. Here we develop and experimentally implement a deep kernel learning workflow combining the correlative prediction of the target functional response and its uncertainty from the structure, and physics-based selection of acquisition function, which autonomously guides the navigation of the image space. Compared to classical Bayesian optimization methods, this approach allows to capture the complex spatial features present in the images of realistic materials, and dynamically learn structure-property relationships. In combination with the flexible scalarizer function that allows to ascribe the degree of physical interest to predicted spectra, this enables physical discovery in automated experiment. Here, this approach is illustrated for nanoplasmonic studies of nanoparticles and experimentally implemented in a truly autonomous fashion for bulk- and edge plasmon discovery in MnPS3, a lesser-known beam-sensitive layered 2D material. This approach is universal, can be directly used as-is with any specimen, and is expected to be applicable to any probe-based microscopic techniques including other STEM modalities, Scanning Probe Microscopies, chemical, and optical imaging.

cond-mat.mtrl-sci↗
arXiv2022

Disentangling electronic transport and hysteresis at individual grain boundaries in hybrid perovskites via automated scanning probe microscopy

Underlying the rapidly increasing photovoltaic efficiency and stability of metal halide perovskites (MHPs) is the advance in the understanding of the microstructure of polycrystalline MHP thin film. Over the past decade, intense efforts have aimed to understand the effect of microstructure on MHP properties, including chemical heterogeneity, strain disorder, phase impurity, etc. It has been found that grain and grain boundary (GB) are tightly related to lots of microscale and nanoscale behavior in MHP thin film. Atomic force microscopy (AFM) is widely used to observe grain and boundary structures in topography and subsequently to study the correlative surface potential and conductivity of these structures. For now, most AFM measurements have been performed in imaging mode to study the static behavior, in contrast, AFM spectroscopy mode allows us to investigate the dynamic behavior of materials, e.g. conductivity under sweeping voltage. However, a major limitation of AFM spectroscopy measurements is that it requests manual operation by human operators, as such only limited data can be obtained, hindering systematic investigations of these microstructures. In this work, we designed a workflow combining the conductive AFM measurement with a machine learning (ML) algorithm to systematically investigate grain boundaries in MHPs. The trained ML model can extract GBs locations from the topography image, and the workflow drives the AFM probe to each GB location to perform a current-voltage (IV) curve automatically. Then, we are able to IV curves at all GB locations, allowing us to systematically understand the property of GBs. Using this method, we discover that the GB junction points are more photoactive, while most previous works only focused on the difference between GB and grains.

cond-mat.mtrl-sci↗
arXiv2022

Enabling Autonomous Electron Microscopy for Networked Computation and Steering

Advanced electron microscopy workflows require an ecosystem of microscope instruments and computing systems possibly located at different sites to conduct remotely steered and automated experiments. Current workflow executions involve manual operations for steering and measurement tasks, which are typically performed from control workstations co-located with microscopes; consequently, their operational tempo and effectiveness are limited. We propose an approach based on separate data and control channels for such an ecosystem of Scanning Transmission Electron Microscopes (STEM) and computing systems, for which no general solutions presently exist, unlike the neutron and light source instruments. We demonstrate automated measurement transfers and remote steering of Nion STEM physical instruments over site networks. We propose a Virtual Infrastructure Twin (VIT) of this ecosystem, which is used to develop and test our steering software modules without requiring access to the physical instrument infrastructure. Additionally, we develop a VIT for a multiple laboratory scenario, which illustrates the applicability of this approach to ecosystems connected over wide-area networks, for the development and testing of software modules and their later field deployment.

cs.DC↗
arXiv2022

Microscopy is All You Need

We pose that microscopy offers an ideal real-world experimental environment for the development and deployment of active Bayesian and reinforcement learning methods. Indeed, the tremendous progress achieved by machine learning (ML) and artificial intelligence over the last decade has been largely achieved via the utilization of static data sets, from the paradigmatic MNIST to the bespoke corpora of text and image data used to train large models such as GPT3, DALLE and others. However, it is now recognized that continuous, minute improvements to state-of-the-art do not necessarily translate to advances in real-world applications. We argue that a promising pathway for the development of ML methods is via the route of domain-specific deployable algorithms in areas such as electron and scanning probe microscopy and chemical imaging. This will benefit both fundamental physical studies and serve as a test bed for more complex autonomous systems such as robotics and manufacturing. Favorable environment characteristics of scanning and electron microscopy include low risk, extensive availability of domain-specific priors and rewards, relatively small effects of exogeneous variables, and often the presence of both upstream first principles as well as downstream learnable physical models for both statics and dynamics. Recent developments in programmable interfaces, edge computing, and access to APIs facilitating microscope control, all render the deployment of ML codes on operational microscopes straightforward. We discuss these considerations and hope that these arguments will lead to creating a novel set of development targets for the ML community by accelerating both real-world ML applications and scientific progress.

cond-mat.dis-nn↗
arXiv2022

Bayesian Active Learning for Scanning Probe Microscopy: from Gaussian Processes to Hypothesis Learning

Recent progress in machine learning methods, and the emerging availability of programmable interfaces for scanning probe microscopes (SPMs), have propelled automated and autonomous microscopies to the forefront of attention of the scientific community. However, enabling automated microscopy requires the development of task-specific machine learning methods, understanding the interplay between physics discovery and machine learning, and fully defined discovery workflows. This, in turn, requires balancing the physical intuition and prior knowledge of the domain scientist with rewards that define experimental goals and machine learning algorithms that can translate these to specific experimental protocols. Here, we discuss the basic principles of Bayesian active learning and illustrate its applications for SPM. We progress from the Gaussian Process as a simple data-driven method and Bayesian inference for physical models as an extension of physics-based functional fits to more complex deep kernel learning methods, structured Gaussian Processes, and hypothesis learning. These frameworks allow for the use of prior data, the discovery of specific functionalities as encoded in spectral data, and exploration of physical laws manifesting during the experiment. The discussed framework can be universally applied to all techniques combining imaging and spectroscopy, SPM methods, nanoindentation, electron microscopy and spectroscopy, and chemical imaging methods, and can be particularly impactful for destructive or irreversible measurements.

cond-mat.mtrl-sci↗
arXiv2022

Learning and predicting photonic responses of plasmonic nanoparticle assemblies via dual variational autoencoders

We demonstrate the application of machine learning for rapid and accurate extraction of plasmonic particles cluster geometries from hyperspectral image data via a dual variational autoencoder (dual-VAE). In this approach, the information is shared between the latent spaces of two VAEs acting on the particle shape data and spectral data, respectively, but enforcing a common encoding on the shape-spectra pairs. We show that this approach can establish the relationship between the geometric characteristics of nanoparticles and their far-field photonic responses, demonstrating that we can use hyperspectral darkfield microscopy to accurately predict the geometry (number of particles, arrangement) of a multiparticle assemblies below the diffraction limit in an automated fashion with high fidelity (for monomers (0.96), dimers (0.86), and trimers (0.58). This approach of building structure-property relationships via shared encoding is universal and should have applications to a broader range of materials science and physics problems in imaging of both molecular and nanomaterial systems.

cond-mat.dis-nn↗
arXiv2022

Probing electron beam induced transformations on a single defect level via automated scanning transmission electron microscopy

The robust approach for real-time analysis of the scanning transmission electron microscopy (STEM) data streams, based on the ensemble learning and iterative training (ELIT) of deep convolutional neural networks, is implemented on an operational microscope, enabling the exploration of the dynamics of specific atomic configurations under electron beam irradiation via an automated experiment in STEM. Combined with beam control, this approach allows studying beam effects on selected atomic groups and chemical bonds in a fully automated mode. Here, we demonstrate atomically precise engineering of single vacancy lines in transition metal dichalcogenides and the creation and identification of topological defects graphene. The ELIT-based approach opens the pathway toward the direct on-the-fly analysis of the STEM data and engendering real-time feedback schemes for probing electron beam chemistry, atomic manipulation, and atom by atom assembly.

cond-mat.dis-nn↗