Searcharxiv⌕ Search

arXiv subjects

Maxim Ziatdinov

Publications and source records attributed to Maxim Ziatdinov.

At least 73 records · Page 4Linked to original sources
arXiv2021

Semi-supervised learning of images with strong rotational disorder: assembling nanoparticle libraries

The proliferation of optical, electron, and scanning probe microscopies gives rise to large volumes of imaging data of objects as diversified as cells, bacteria, pollen, to nanoparticles and atoms and molecules. In most cases, the experimental data streams contain images having arbitrary rotations and translations within the image. At the same time, for many cases, small amounts of labeled data are available in the form of prior published results, image collections, and catalogs, or even theoretical models. Here we develop an approach that allows generalizing from a small subset of labeled data with a weak orientational disorder to a large unlabeled dataset with a much stronger orientational (and positional) disorder, i.e., it performs a classification of image data given a small number of examples even in the presence of a distribution shift between the labeled and unlabeled parts. This approach is based on the semi-supervised rotationally invariant variational autoencoder (ss-rVAE) model consisting of the encoder-decoder "block" that learns a rotationally (and translationally) invariant continuous latent representation of data and a classifier that encodes data into a finite number of discrete classes. The classifier part of the trained ss-rVAE inherits the rotational (and translational) invariances and can be deployed independently of the other parts of the model. The performance of the ss-rVAE is illustrated using the synthetic data sets with known factors of variation. We further demonstrate its application for experimental data sets of nanoparticles, creating nanoparticle libraries and disentangling the representations defining the physical factors of variation in the data. The code reproducing the results is available at https://github.com/ziatdinovmax/Semi-Supervised-VAE-nanoparticles.

cs.LG↗
arXiv2021

AtomAI: A Deep Learning Framework for Analysis of Image and Spectroscopy Data in (Scanning) Transmission Electron Microscopy and Beyond

AtomAI is an open-source software package bridging instrument-specific Python libraries, deep learning, and simulation tools into a single ecosystem. AtomAI allows direct applications of the deep convolutional neural networks for atomic and mesoscopic image segmentation converting image and spectroscopy data into class-based local descriptors for downstream tasks such as statistical and graph analysis. For atomically-resolved imaging data, the output is types and positions of atomic species, with an option for subsequent refinement. AtomAI further allows the implementation of a broad range of image and spectrum analysis functions, including invariant variational autoencoders (VAEs). The latter consists of VAEs with rotational and (optionally) translational invariance for unsupervised and class-conditioned disentanglement of categorical and continuous data representations. In addition, AtomAI provides utilities for mapping structure-property relationships via im2spec and spec2im type of encoder-decoder models. Finally, AtomAI allows seamless connection to the first principles modeling with a Python interface, including molecular dynamics and density functional theory calculations on the inferred atomic position. While the majority of applications to date were based on atomically resolved electron microscopy, the flexibility of AtomAI allows straightforward extension towards the analysis of mesoscopic imaging data once the labels and feature identification workflows are established/available. The source code and example notebooks are available at https://github.com/pycroscopy/atomai.

physics.data-an↗
arXiv2021

Disentangling ferroelectric wall dynamics and identification of pinning mechanisms via deep learning

Field-induced domain wall dynamics in ferroelectric materials underpins multiple applications ranging from actuators to information technology devices and necessitates a quantitative description of the associated mechanisms including giant electromechanical couplings, controlled non-linearities, or low coercive voltages. While the advances in dynamic Piezoresponse Force Microscopy measurements over the last two decades have rendered visualization of polarization dynamics relatively straightforward, the associated insights into the local mechanisms have been elusive. Here we explore the domain dynamics in model polycrystalline materials using a workflow combining deep learning-based segmentation of the domain structures with non-linear dimensionality reduction using multilayer rotationally-invariant autoencoders (rVAE). The former allows unambiguous identification and classification of the ferroelectric and ferroelastic domain walls. The rVAE discover the latent representations of the domain wall geometries and their dynamics, thus providing insight into the intrinsic mechanisms of polarization switching, that can further be compared to simple physical models. The rVAE disentangles the factors affecting the pinning efficiency of ferroelectric walls, offering insights into the correlation of ferroelastic wall distribution and ferroelectric wall pinning.

cond-mat.dis-nn↗
arXiv2021

Exploring particle dynamics during self-organization processes via rotationally invariant latent representations

The dynamic of complex ordering systems with active rotational degrees of freedom exemplified by protein self-assembly is explored using a machine learning workflow that combines deep learning-based semantic segmentation and rotationally invariant variational autoencoder-based analysis of orientation and shape evolution. The latter allows for disentanglement of the particle orientation from other degrees of freedom and compensates for shifts. The disentangled representations in the latent space encode the rich spectrum of local transitions that can now be visualized and explored via continuous variables. The time dependence of ensemble averages allows insight into the time dynamics of the system, and in particular, illustrates the presence of the potential ordering transition. Finally, analysis of the latent variables along the single-particle trajectory allows tracing these parameters on a single particle level. The proposed approach is expected to be universally applicable for the description of the imaging data in optical, scanning probe, and electron microscopy seeking to understand the dynamics of complex systems where rotations are a significant part of the process.

cond-mat.soft↗
arXiv2021

Exploring Electron Beam Induced Atomic Assembly via Reinforcement Learning in a Molecular Dynamics Environment

Atom-by-atom assembly of functional materials and devices is perceived as one of the ultimate targets of nanoscience and nanotechnology. While traditionally implemented via scanning probe microscopy techniques, recently it has been shown that the beam of a scanning transmission electron microscope can be used for targeted manipulation of individual atoms. However, the process is highly dynamic in nature and proceeds via a large number of weakly-understood individual steps. Hence, harnessing an electron beam towards atomic assembly requires automated methods to control the parameters and positioning of the beam in such a way as to fabricate atomic-scale structures reliably. Here, we create a molecular dynamics environment wherein individual atom velocities can be modified, effectively simulating a beam-induced interaction, and apply reinforcement learning to model construction of specific atomic units consisting of Si dopant atoms on a graphene lattice. We find that it is possible to engineer the reward function of the agent in such a way as to encourage formation of local clusters of dopants, whilst at the same time minimizing the amplitude of momentum changes. Inspection of the learned policies indicates that of fundamental importance is the component of velocity perpendicular to the material plane, and further, that the high stochasticity of the environment leads to conservative policies. This study shows the potential for reinforcement learning agents trained in simulated environments for potential use as atomic scale fabricators, and further, that the dynamics learned by agents encode specific elements of important physics that can be learned.

cond-mat.mes-hall↗
arXiv2021

Robust Feature Disentanglement in Imaging Data via Joint Invariant Variational Autoencoders: from Cards to Atoms

Recent advances in imaging from celestial objects in astronomy visualized via optical and radio telescopes to atoms and molecules resolved via electron and probe microscopes are generating immense volumes of imaging data, containing information about the structure of the universe from atomic to astronomic levels. The classical deep convolutional neural network architectures traditionally perform poorly on the data sets having a significant orientational disorder, that is, having multiple copies of the same or similar object in arbitrary orientation in the image plane. Similarly, while clustering methods are well suited for classification into discrete classes and manifold learning and variational autoencoders methods can disentangle representations of the data, the combined problem is ill-suited to a classical non-supervised learning paradigm. Here we introduce a joint rotationally (and translationally) invariant variational autoencoder (j-trVAE) that is ideally suited to the solution of such a problem. The performance of this method is validated on several synthetic data sets and extended to high-resolution imaging data of electron and scanning probe microscopy. We show that latent space behaviors directly comport to the known physics of ferroelectric materials and quantum systems. We further note that the engineering of the latent space structure via imposed topological structure or directed graph relationship allows for applications in topological discovery and causal physical learning.

physics.data-an↗
arXiv2021

Decoding the shift-invariant data: applications for band-excitation scanning probe microscopy

A shift-invariant variational autoencoder (shift-VAE) is developed as an unsupervised method for the analysis of spectral data in the presence of shifts along the parameter axis, disentangling the physically-relevant shifts from other latent variables. Using synthetic data sets, we show that the shift-VAE latent variables closely match the ground truth parameters. The shift VAE is extended towards the analysis of band-excitation piezoresponse force microscopy (BE-PFM) data, disentangling the resonance frequency shifts from the peak shape parameters in a model-free unsupervised manner. The extensions of this approach towards denoising of data and model-free dimensionality reduction in imaging and spectroscopic data are further demonstrated. This approach is universal and can also be extended to analysis of X-ray diffraction, photoluminescence, Raman spectra, and other data sets.

cond-mat.dis-nn↗
arXiv2021

Disentangling ferroelectric domain wall geometries and pathways in dynamic piezoresponse force microscopy via unsupervised machine learning

Domain switching pathways in ferroelectric materials visualized by dynamic Piezoresponse Force Microscopy (PFM) are explored via variational autoencoder (VAE), which simplifies the elements of the observed domain structure, crucially allowing for rotational invariance, thereby reducing the variability of local polarization distributions to a small number of latent variables. For small sampling window sizes the latent space is degenerate, and variability is observed only in the direction of a single latent variable that can be identified with the presence of domain wall. For larger window sizes, the latent space is 2D, and the disentangled latent variables can be generally interpreted as the degree of switching and complexity of domain structure. Applied to multiple consecutive PFM images acquired while monitoring domain switching, the polarization switching mechanism can thus be visualized in the latent space, providing insight into domain evolution mechanisms and their correlation with the microstructure.

cond-mat.dis-nn↗
arXiv2021

Unsupervised Machine Learning Discovery of Chemical and Physical Transformation Pathways from Imaging Data

We show that unsupervised machine learning can be used to learn physical and chemical transformation pathways from the observational microscopic data, as demonstrated for atomically resolved images in Scanning Transmission Electron Microscopy (STEM) and ferroelectric domain structures in Piezoresponse Force Microscopy (PFM). To enable this analysis in STEM, we assumed the existence of atoms, a discreteness of atomic classes, and the presence of an explicit relationship between the observed STEM contrast and the presence of atomic units. In PFM, we assumed the uniquely-defined relationship between the measured signal and polarization distribution. With only these postulates, we developed a machine learning method leveraging a rotationally-invariant variational autoencoder (rVAE) that can identify the existing structural units observed within a material. The approach encodes the information contained in image sequences using a small number of latent variables, allowing the exploration of chemical and physical transformation pathways via the latent space of the system. The results suggest that the high-veracity imaging data can be used to derive fundamental physical and chemical mechanisms involved, by providing encodings of the observed structures that act as bottom-up equivalents of structural order parameters. The approach also demonstrates the potential of variational (i.e., Bayesian) methods for physical sciences and will stimulate the development of new ways to encode physical constraints in the encoder-decoder architectures, and generative physical laws, topological invariances, and causal relationships in the latent space of VAEs.

cond-mat.mtrl-sci↗
arXiv2021

Exploring order parameters and dynamic processes in disordered systems via variational autoencoders

We suggest and implement an approach for the bottom-up description of systems undergoing large-scale structural changes and chemical transformations from dynamic atomically resolved imaging data, where only partial or uncertain data on atomic positions are available. This approach is predicated on the synergy of two concepts, the parsimony of physical descriptors and general rotational invariance of non-crystalline solids, and is implemented using a rotationally-invariant extension of the variational autoencoder applied to semantically segmented atom-resolved data seeking the most effective reduced representation for the system that still contains the maximum amount of original information. This approach allowed us to explore the dynamic evolution of electron beam-induced processes in a silicon-doped graphene system, but it can be also applied for a much broader range of atomic-scale and mesoscopic phenomena to introduce the bottom-up order parameters and explore their dynamics with time and in response to external stimuli.

cond-mat.mtrl-sci↗
arXiv2021

Revealing the chemical bonding in adatoms arrays via machine learning of 3D scanning tunneling spectroscopy data

The adatom arrays on surfaces offer an ideal playground to explore the mechanisms of chemical bonding via changes in the local electronic tunneling spectra. While this information is readily available in hyperspectral scanning tunneling spectroscopy data, its analysis has been considerably impeded by a lack of suitable analytical tools. Here we develop a machine learning based workflow combining supervised feature identification in the spatial domain and un-supervised clustering in the energy domain to reveal the details of structure-dependent changes of the electronic structure in adatom arrays on the Co3Sn2S2 cleaved surface. This approach, in combination with first-principles calculations, provides insight for using artificial neural networks to detect adatoms and classifies each based on their local neighborhood comprised of other adatoms. These structurally classified adatoms are further spectrally deconvolved. The unexpected inhomogeneity of electronic structures among adatoms in similar configurations is unveiled using this method, suggesting there is not a single atomic species of adatoms, but rather multiple types of adatoms on the Co3Sn2S2 surface. This is further supported by a slight contrast difference in the images (or slight size variation) of the topography of the adatoms.

cond-mat.mtrl-sci↗
arXiv2021

Deep learning polarization distributions in ferroelectrics from STEM data: with and without atom finding

Over the last decade, scanning transmission electron microscopy (STEM) has emerged as a powerful tool for probing atomic structures of complex materials with picometer precision, opening the pathway toward exploring ferroelectric, ferroelastic, and chemical phenomena on the atomic-scale. Analyses to date extracting a polarization signal from lattice coupled distortions in STEM imaging rely on discovery of atomic positions from intensity maxima/minima and subsequent calculation of polarization and other order parameter fields from the atomic displacements. Here, we explore the feasibility of polarization mapping directly from the analysis of STEM images using deep convolutional neural networks (DCNNs). In this approach, the DCNN is trained on the labeled part of the image (i.e., for human labelling), and the trained network is subsequently applied to other images. We explore the effects of the choice of the descriptors (centered on atomic columns and grid-based), the effects of observational bias, and whether the network trained on one composition can be applied to a different one. This analysis demonstrates the tremendous potential of the DCNN for the analysis of high-resolution STEM imaging and spectral data and highlights the associated limitations.

physics.data-an↗
arXiv2021

Toward Decoding the Relationship between Domain Structure and Functionality in Ferroelectrics via Hidden Latent Variables

Polarization switching mechanisms in ferroelectric materials are fundamentally linked to local domain structure and presence of the structural defects, which both can act as nucleation and pinning centers and create local electrostatic and mechanical depolarization fields affecting wall dynamics. However, the general correlative mechanisms between domain structure and polarization dynamics are only weakly explored, precluding insight into the associated physical mechanisms. Here, the correlation between local domain structures and switching behavior in ferroelectric materials is explored using the convolutional encoder-decoder networks, enabling the image to spectral (im2spec) and spectral to image (spec2im) translations via encoding latent variables. The latter reflects the assumption that the relationship between domain structure and polarization switching is parsimonious, i.e. is based upon a small number of local mechanisms. The analysis of latent variables distributions and their real space representations provides insight into the predictability of the local switching behavior, and hence associated physical mechanisms. We further pose that the regions where these correlative relationships are violated, i.e. predictability of the polarization dynamics from domain structure is reduced, represent the obvious target for detailed studies, e.g. in the context of automated experiments. This approach provides a workflow to establish the presence of correlation between local spectral responses and local structure and can be universally applied to spectral imaging techniques such as PFM, scanning tunneling microscopy (STM) and spectroscopy, and electron energy loss spectroscopy (EELS) in scanning transmission electron microscopy (STEM).

cond-mat.dis-nn↗
arXiv2021

Predictability as a probe of manifest and latent physics: The case of atomic scale structural, chemical, and polarization behaviors in multiferroic Sm-doped BiFeO3

The predictability of a certain effect or phenomenon is often equated with the knowledge of relevant physical laws, typically understood as a functional or numerically derived relationship between the observations and known states of the system. Correspondingly, observations inconsistent with prior knowledge can be used to derive new knowledge on the nature of the system or indicate the presence of yet unknown mechanisms. Here we explore the applicability of Gaussian Processes (GP) to establish predictability and uncertainty of local behaviors from multimodal observations, providing an alternative to this classical paradigm. Using atomic-resolution Scanning Transmission Electron Microscopy (STEM) of multiferroic Sm-doped BiFeO3 across a broad composition range, we directly visualize the atomic structure and structural, physical, and chemical order parameter fields for the material. GP regression is used to establish the predictability of the local polarization field from different groups of parameters, including the adjacent polarization values and several combinations of physical and chemical descriptors, including lattice parameters, column intensities, etc. We observe that certain elements of microstructure including charged and uncharged domain walls and interfaces with the substrate are best predicted with specific combinations of descriptors, and this predictability and their associated uncertainties are consistent across the composition series. The associated generative physical mechanisms are discussed. We argue that predictability and uncertainty in observational data offers a new pathway to probe the physics of condensed matter systems from multimodal local observations.

cond-mat.mtrl-sci↗
arXiv2021

Exploring physics of ferroelectric domain walls via Bayesian analysis of atomically resolved STEM data

The physics of ferroelectric domain walls is explored using the Bayesian inference analysis of atomically resolved STEM data. We demonstrate that domain wall profile shapes are ultimately sensitive to the nature of the order parameter in the material, including the functional form of Ginzburg-Landau-Devonshire expansion, and numerical value of the corresponding parameters. The preexisting materials knowledge naturally folds in the Bayesian framework in the form of prior distributions, with the different order parameters forming competing (or hierarchical) models. Here, we explore the physics of the ferroelectric domain walls in BiFeO3 using this method, and derive the posterior estimates of relevant parameters. More generally, this inference approach both allows learning materials physics from experimental data with associated uncertainty quantification, and establishing guidelines for instrumental development answering questions on what resolution and information limits are necessary for reliable observation of specific physical mechanisms of interest.

cond-mat.mtrl-sci↗
arXiv2021

Ensemble learning and iterative training (ELIT) machine learning: applications towards uncertainty quantification and automated experiment in atom-resolved microscopy

Deep learning has emerged as a technique of choice for rapid feature extraction across imaging disciplines, allowing rapid conversion of the data streams to spatial or spatiotemporal arrays of features of interest. However, applications of deep learning in experimental domains are often limited by the out-of-distribution drift between the experiments, where the network trained for one set of imaging conditions becomes sub-optimal for different ones. This limitation is particularly stringent in the quest to have an automated experiment setting, where retraining or transfer learning becomes impractical due to the need for human intervention and associated latencies. Here we explore the reproducibility of deep learning for feature extraction in atom-resolved electron microscopy and introduce workflows based on ensemble learning and iterative training to greatly improve feature detection. This approach both allows incorporating uncertainty quantification into the deep learning analysis and also enables rapid automated experimental workflows where retraining of the network to compensate for out-of-distribution drift due to subtle change in imaging conditions is substituted for a human operator or programmatic selection of networks from the ensemble. This methodology can be further applied to machine learning workflows in other imaging areas including optical and chemical imaging.

physics.data-an↗
arXiv2020

Bayesian learning of adatom interactions from atomically-resolved imaging data

Atomic structures and adatom geometries of surfaces encode information about the thermodynamics and kinetics of the processes that lead to their formation, and which can be captured by a generative physical model. Here we develop a workflow based on a machine learning-based analysis of scanning tunneling microscopy images to reconstruct the atomic and adatom positions, and a Bayesian optimization procedure to minimize statistical distance between the chosen physical models and experimental observations. We optimize the parameters of a 2- and 3-parameter Ising model describing surface ordering and use the derived generative model to make predictions across the parameter space. For concentration dependence, we compare the predicted morphologies at different adatom concentrations with the dissimilar regions on the sample surfaces that serendipitously had different adatom concentrations. The proposed workflow is universal and can be used to reconstruct the thermodynamic models and associated uncertainties from the experimental observations of materials microstructures. The code used in the manuscript is available at https://github.com/saimani5/Adatom_interactions.

cond-mat.mtrl-sci↗
arXiv2020

Deep Bayesian Local Crystallography

The advent of high-resolution electron and scanning probe microscopy imaging has opened the floodgates for acquiring atomically resolved images of bulk materials, 2D materials, and surfaces. This plethora of data contains an immense volume of information on materials structures, structural distortions, and physical functionalities. Harnessing this knowledge regarding local physical phenomena necessitates the development of the mathematical frameworks for extraction of relevant information. However, the analysis of atomically resolved images is often based on the adaptation of concepts from macroscopic physics, notably translational and point group symmetries and symmetry lowering phenomena. Here, we explore the bottom-up definition of structural units and symmetry in atomically resolved data using a Bayesian framework. We demonstrate the need for a Bayesian definition of symmetry using a simple toy model and demonstrate how this definition can be extended to the experimental data using deep learning networks in a Bayesian setting, namely rotationally invariant variational autoencoders.

physics.comp-ph↗