arXiv subjects

Amit Sharma

Publications and source records attributed to Amit Sharma.

At least 73 records · Page 4Linked to original sources

arXiv2023

Machine learning of twin/matrix interfaces from local stress field

Twinning is an important deformation mode in plastically deformed hexagonal close-packed materials. The extremely high twin growth rates at the nanoscale make atomistic simulations an attractive method for investigating the role of individual twin/matrix interfaces such as twin boundaries and basal-prismatic interfaces in twin growth kinetics. Unfortunately, there is no single framework that allows researchers to differentiate such interfaces automatically, neither in experimental in-situ transmission electron microscopy analysis images nor in atomistic simulations. Moreover, the presence of alloying elements introduces substantial noise to local atomic environments, making it nearly impossible to identify which atoms belong to which interface. Here, with the help of advanced machine learning methods, we provide a proof-of-concept way of using the local stress field distribution as an indicator for the presence of interfaces and for determining their types. We apply such an analysis to the growth of twin embryos in Mg-10 at.% Al alloys under constant stress and constant strain conditions, corresponding to two extremes of high and low strain rates, respectively. We discover that the kinetics of such growth is driven by high-energy basal-prismatic interfaces, in line with our experimental observations for pure Mg.

cond-mat.mtrl-sci↗

arXiv2023

Improving the crystallinity and texture of oblique-angle-deposited AlN thin films using reactive synchronized HiPIMS

Many technologies require highly-oriented and textured functional thin films. The most common synthe-sis approaches use on-axis sputter geometries. However, in some scenarios, on-axis sputtering is not feasible. During ionized physical vapor deposition (PVD), in contrast to conventional PVD, the film-forming species can be accelerated onto the growing film using substrate-bias potentials. This increas-es the ad-atom mobility, but also deflects the trajectory of ions towards the substrate increasing the texture of the growing film. However, potential gas-ion incorporation in the films limits the feasibility of such approaches for the deposition of defect-sensitive materials. In this work, we report on the oblique-angle deposition of highly c-axis oriented AlN (0002) films, enabled by reactive metal-ion syn-chronized HiPIMS. The effect of critical deposition parameters, such as the magnetic configuration, ion kinetic energies and substrate biasing are investigated. The films deposited using HiPIMS show a more pronounced texture and orientation compared to DCMS films. We find that combining the HiPIMS dep-ositions with a moderate substrate bias of -30 V is sufficient to improve the crystalline quality and tex-ture of the films significantly. To reduce process-gas incorporation, and the formation of point defects, the negative substrate-bias potential is synchronized to the Al-rich fraction of each HiPIMS pulse. This leads to reduced Ar-Ion incorporation and improves the structural properties. The films also show uni-form polarization of the grains making this synthesis route suitable for piezoelectric applications. While the compressive stress in the films is still high, the results demonstrate, that synchronized HiPIMS can yield promising results for the synthesis under oblique-angle deposition conditions - even with low substrate-bias potentials.

cond-mat.mtrl-sci↗

arXiv2022

Counterfactual Generation Under Confounding

A machine learning model, under the influence of observed or unobserved confounders in the training data, can learn spurious correlations and fail to generalize when deployed. For image classifiers, augmenting a training dataset using counterfactual examples has been empirically shown to break spurious correlations. However, the counterfactual generation task itself becomes more difficult as the level of confounding increases. Existing methods for counterfactual generation under confounding consider a fixed set of interventions (e.g., texture, rotation) and are not flexible enough to capture diverse data-generating processes. Given a causal generative process, we formally characterize the adverse effects of confounding on any downstream tasks and show that the correlation between generative factors (attributes) can be used to quantitatively measure confounding between generative factors. To minimize such correlation, we propose a counterfactual generation method that learns to modify the value of any attribute in an image and generate new images given a set of observed attributes, even when the dataset is highly confounded. These counterfactual images are then used to regularize the downstream classifier such that the learned representations are the same across various generative factors conditioned on the class label. Our method is computationally efficient, simple to implement, and works well for any number of generative factors and confounding variables. Our experimental results on both synthetic (MNIST variants) and real-world (CelebA) datasets show the usefulness of our approach.

cs.LG↗

arXiv2022

The Counterfactual-Shapley Value: Attributing Change in System Metrics

Given an unexpected change in the output metric of a large-scale system, it is important to answer why the change occurred: which inputs caused the change in metric? A key component of such an attribution question is estimating the counterfactual: the (hypothetical) change in the system metric due to a specified change in a single input. However, due to inherent stochasticity and complex interactions between parts of the system, it is difficult to model an output metric directly. We utilize the computational structure of a system to break up the modelling task into sub-parts, such that each sub-part corresponds to a more stable mechanism that can be modelled accurately over time. Using the system's structure also helps to view the metric as a computation over a structural causal model (SCM), thus providing a principled way to estimate counterfactuals. Specifically, we propose a method to estimate counterfactuals using time-series predictive models and construct an attribution score, CF-Shapley, that is consistent with desirable axioms for attributing an observed change in the output metric. Unlike past work on causal shapley values, our proposed method can attribute a single observed change in output (rather than a population-level effect) and thus provides more accurate attribution scores when evaluated on simulated datasets. As a real-world application, we analyze a query-ad matching system with the goal of attributing observed change in a metric for ad matching density. Attribution scores explain how query volume and ad demand from different query categories affect the ad matching density, leading to actionable insights and uncovering the role of external events (e.g., "Cheetah Day") in driving the matching density.

cs.LG↗

arXiv2022

Matching Learned Causal Effects of Neural Networks with Domain Priors

A trained neural network can be interpreted as a structural causal model (SCM) that provides the effect of changing input variables on the model's output. However, if training data contains both causal and correlational relationships, a model that optimizes prediction accuracy may not necessarily learn the true causal relationships between input and output variables. On the other hand, expert users often have prior knowledge of the causal relationship between certain input variables and output from domain knowledge. Therefore, we propose a regularization method that aligns the learned causal effects of a neural network with domain priors, including both direct and total causal effects. We show that this approach can generalize to different kinds of domain priors, including monotonicity of causal effect of an input variable on output or zero causal effect of a variable on output for purposes of fairness. Our experiments on twelve benchmark datasets show its utility in regularizing a neural network model to maintain desired causal effects, without compromising on accuracy. Importantly, we also show that a model thus trained is robust and gets improved accuracy on noisy inputs.

cs.LG↗

arXiv2022

Deep End-to-end Causal Inference

Causal inference is essential for data-driven decision making across domains such as business engagement, medical treatment and policy making. However, research on causal discovery has evolved separately from inference methods, preventing straight-forward combination of methods from both fields. In this work, we develop Deep End-to-end Causal Inference (DECI), a single flow-based non-linear additive noise model that takes in observational data and can perform both causal discovery and inference, including conditional average treatment effect (CATE) estimation. We provide a theoretical guarantee that DECI can recover the ground truth causal graph under standard causal discovery assumptions. Motivated by application impact, we extend this model to heterogeneous, mixed-type data with missing values, allowing for both continuous and discrete treatment decisions. Our results show the competitive performance of DECI when compared to relevant baselines for both causal discovery and (C)ATE estimation in over a thousand experiments on both synthetic datasets and causal machine learning benchmarks across data-types and levels of missingness.

stat.ML↗

arXiv2022

coCartesian fibrations and homotopy colimits

The main objective of this paper is to show that the homotopy colimit of a diagram of quasi-categories and indexed by a small category is a localization of Lurie's higher Grothendieck construction of the diagram. We thereby generalize Thomason's classical result which states that the homotopy colimit of a diagram of categories has the homotopy type of (the classifying space of) the Grothendieck construction of the diagram of categories.

math.CT↗

arXiv2022

Evaluating and Mitigating Bias in Image Classifiers: A Causal Perspective Using Counterfactuals

Counterfactual examples for an input -- perturbations that change specific features but not others -- have been shown to be useful for evaluating bias of machine learning models, e.g., against specific demographic groups. However, generating counterfactual examples for images is non-trivial due to the underlying causal structure on the various features of an image. To be meaningful, generated perturbations need to satisfy constraints implied by the causal model. We present a method for generating counterfactuals by incorporating a structural causal model (SCM) in an improved variant of Adversarially Learned Inference (ALI), that generates counterfactuals in accordance with the causal relationships between attributes of an image. Based on the generated counterfactuals, we show how to explain a pre-trained machine learning classifier, evaluate its bias, and mitigate the bias using a counterfactual regularizer. On the Morpho-MNIST dataset, our method generates counterfactuals comparable in quality to prior work on SCM-based counterfactuals (DeepSCM), while on the more complex CelebA dataset our method outperforms DeepSCM in generating high-quality valid counterfactuals. Moreover, generated counterfactuals are indistinguishable from reconstructed images in a human evaluation experiment and we subsequently use them to evaluate the fairness of a standard classifier trained on CelebA data. We show that the classifier is biased w.r.t. skin and hair color, and how counterfactual regularization can remove those biases.

cs.CV↗

arXiv2021

Technology Readiness Levels for Machine Learning Systems

The development and deployment of machine learning (ML) systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. The lack of diligence can lead to technical debt, scope creep and misaligned objectives, model misuse and failures, and expensive consequences. Engineering systems, on the other hand, follow well-defined processes and testing standards to streamline development for high-quality, reliable results. The extreme is spacecraft systems, where mission critical measures and robustness are ingrained in the development process. Drawing on experience in both spacecraft engineering and ML (from research through product across domain areas), we have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" (MLTRL) framework defines a principled process to ensure robust, reliable, and responsible systems while being streamlined for ML workflows, including key distinctions from traditional software engineering. Even more, MLTRL defines a lingua franca for people across teams and organizations to work collaboratively on artificial intelligence and machine learning technologies. Here we describe the framework and elucidate it with several real world use-cases of developing ML methods from basic research through productization and deployment, in areas such as medical diagnostics, consumer computer vision, satellite imagery, and particle physics.

cs.LG↗

arXiv2021

Sayer: Using Implicit Feedback to Optimize System Policies

We observe that many system policies that make threshold decisions involving a resource (e.g., time, memory, cores) naturally reveal additional, or implicit feedback. For example, if a system waits X min for an event to occur, then it automatically learns what would have happened if it waited <X min, because time has a cumulative property. This feedback tells us about alternative decisions, and can be used to improve the system policy. However, leveraging implicit feedback is difficult because it tends to be one-sided or incomplete, and may depend on the outcome of the event. As a result, existing practices for using feedback, such as simply incorporating it into a data-driven model, suffer from bias. We develop a methodology, called Sayer, that leverages implicit feedback to evaluate and train new system policies. Sayer builds on two ideas from reinforcement learning -- randomized exploration and unbiased counterfactual estimators -- to leverage data collected by an existing policy to estimate the performance of new candidate policies, without actually deploying those policies. Sayer uses implicit exploration and implicit data augmentation to generate implicit feedback in an unbiased form, which is then used by an implicit counterfactual estimator to evaluate and train new policies. The key idea underlying these techniques is to assign implicit probabilities to decisions that are not actually taken but whose feedback can be inferred; these probabilities are carefully calculated to ensure statistical unbiasedness. We apply Sayer to two production scenarios in Azure, and show that it can evaluate arbitrary policies accurately, and train new policies that outperform the production policies.

cs.LG↗

arXiv2021

The Connection between Out-of-Distribution Generalization and Privacy of ML Models

With the goal of generalizing to out-of-distribution (OOD) data, recent domain generalization methods aim to learn "stable" feature representations whose effect on the output remains invariant across domains. Given the theoretical connection between generalization and privacy, we ask whether better OOD generalization leads to better privacy for machine learning models, where privacy is measured through robustness to membership inference (MI) attacks. In general, we find that the relationship does not hold. Through extensive evaluation on a synthetic dataset and image datasets like MNIST, Fashion-MNIST, and Chest X-rays, we show that a lower OOD generalization gap does not imply better robustness to MI attacks. Instead, privacy benefits are based on the extent to which a model captures the stable features. A model that captures stable features is more robust to MI attacks than models that exhibit better OOD generalization but do not learn stable features. Further, for the same provable differential privacy guarantees, a model that learns stable features provides higher utility as compared to others. Our results offer the first extensive empirical study connecting stable features and privacy, and also have a takeaway for the domain generalization community; MI attack can be used as a complementary metric to measure model quality.

cs.LG↗

arXiv2021

DoWhy: Addressing Challenges in Expressing and Validating Causal Assumptions

Estimation of causal effects involves crucial assumptions about the data-generating process, such as directionality of effect, presence of instrumental variables or mediators, and whether all relevant confounders are observed. Violation of any of these assumptions leads to significant error in the effect estimate. However, unlike cross-validation for predictive models, there is no global validator method for a causal estimate. As a result, expressing different causal assumptions formally and validating them (to the extent possible) becomes critical for any analysis. We present DoWhy, a framework that allows explicit declaration of assumptions through a causal graph and provides multiple validation tests to check a subset of these assumptions. Our experience with DoWhy highlights a number of open questions for future research: developing new ways beyond causal graphs to express assumptions, the role of causal discovery in learning relevant parts of the graph, and developing validation tests that can better detect errors, both for average and conditional treatment effects. DoWhy is available at https://github.com/microsoft/dowhy.

cs.LG↗

arXiv2021

Noise2Fast: Fast Self-Supervised Single Image Blind Denoising

In the last several years deep learning based approaches have come to dominate many areas of computer vision, and image denoising is no exception. Neural networks can learn by example to map noisy images to clean images. However, access to noisy/clean or even noisy/noisy image pairs isn't always readily available in the desired domain. Recent approaches have allowed for the denoising of single noisy images without access to any training data aside from that very image. But since they require both training and inference to be carried out on each individual input image, these methods require significant computation time. As such, they are difficult to integrate into automated microscopy pipelines where denoising large datasets is essential but needs to be carried out in a timely manner. Here we present Noise2Fast, a fast single image blind denoiser. Our method is tailored for speed by training on a four-image dataset produced using a unique form of downsampling we refer to as "checkerboard downsampling". Noise2Fast is faster than all tested approaches and is more accurate than all except Self2Self, which takes well over 100 times longer to denoise an image. This allows for a combination of speed and flexibility that was not previously attainable using any other method.

eess.IV↗

arXiv2021

Domain Generalization using Causal Matching

In the domain generalization literature, a common objective is to learn representations independent of the domain after conditioning on the class label. We show that this objective is not sufficient: there exist counter-examples where a model fails to generalize to unseen domains even after satisfying class-conditional domain invariance. We formalize this observation through a structural causal model and show the importance of modeling within-class variations for generalization. Specifically, classes contain objects that characterize specific causal features, and domains can be interpreted as interventions on these objects that change non-causal features. We highlight an alternative condition: inputs across domains should have the same representation if they are derived from the same object. Based on this objective, we propose matching-based algorithms when base objects are observed (e.g., through data augmentation) and approximate the objective when objects are not observed (MatchDG). Our simple matching-based algorithms are competitive to prior work on out-of-domain accuracy for rotated MNIST, Fashion-MNIST, PACS, and Chest-Xray datasets. Our method MatchDG also recovers ground-truth object matches: on MNIST and Fashion-MNIST, top-10 matches from MatchDG have over 50% overlap with ground-truth matches.

cs.LG↗

arXiv2021

Towards Unifying Feature Attribution and Counterfactual Explanations: Different Means to the Same End

Feature attributions and counterfactual explanations are popular approaches to explain a ML model. The former assigns an importance score to each input feature, while the latter provides input examples with minimal changes to alter the model's predictions. To unify these approaches, we provide an interpretation based on the actual causality framework and present two key results in terms of their use. First, we present a method to generate feature attribution explanations from a set of counterfactual examples. These feature attributions convey how important a feature is to changing the classification outcome of a model, especially on whether a subset of features is necessary and/or sufficient for that change, which attribution-based methods are unable to provide. Second, we show how counterfactual examples can be used to evaluate the goodness of an attribution-based explanation in terms of its necessity and sufficiency. As a result, we highlight the complementarity of these two approaches. Our evaluation on three benchmark datasets - Adult-Income, LendingClub, and German-Credit - confirms the complementarity. Feature attribution methods like LIME and SHAP and counterfactual explanation methods like Wachter et al. and DiCE often do not agree on feature importance rankings. In addition, by restricting the features that can be modified for generating counterfactual examples, we find that the top-k features from LIME or SHAP are often neither necessary nor sufficient explanations of a model's prediction. Finally, we present a case study of different explanation methods on a real-world hospital triage problem

cs.LG↗

arXiv2021

Causally Constrained Data Synthesis for Private Data Release

Making evidence based decisions requires data. However for real-world applications, the privacy of data is critical. Using synthetic data which reflects certain statistical properties of the original data preserves the privacy of the original data. To this end, prior works utilize differentially private data release mechanisms to provide formal privacy guarantees. However, such mechanisms have unacceptable privacy vs. utility trade-offs. We propose incorporating causal information into the training process to favorably modify the aforementioned trade-off. We theoretically prove that generative models trained with additional causal knowledge provide stronger differential privacy guarantees. Empirically, we evaluate our solution comparing different models based on variational auto-encoders (VAEs), and show that causal information improves resilience to membership inference, with improvements in downstream utility.

cs.LG↗

arXiv2021

Compact closed categories and $Γ$-categories (with an appendix by André Joyal)

In this paper we study compact closed categories within the context of homotopical algebra. We construct two new model category structures by localizing two (Quillen equivalent) model categories of symmetric monoidal categories with the objective of establishing the free compact closed category on one generator as a fibrant replacement of the free symmetric monoidal category on one generator, in our localized model categories. We go on to show that the fibrant objects in our model categories are compact closed categories.

math.CT↗

arXiv2021

On Cofibrations of Permutative categories

In this note we introduce a notion of free cofibrations of permutative categories. We show that each cofibration of permutative categories is a retract of a free cofibration.

math.CT↗