Searcharxiv⌕ Search

arXiv subjects

Vivek Natarajan

Publications and source records attributed to Vivek Natarajan.

53 records · Page 3Linked to original sources

Transient and Asymptotic Properties of Robust Adaptive Controllers in the Presence of Non-Coercive Lyapunov Functions

Adaptive control architectures often make use of Lyapunov functions to design adaptive laws. We are specifically interested in adaptive control methods, such as the well-known L1 adaptive architecture, which employ a parameter observer for this purpose. In such architectures, the observation error plays a critical role in determining analytical bounds on the tracking error as well as robustness. In this paper, we show how the non-existence of coercive Lyapunov operators can impact the analytical bounds, and with it the performance and the robustness of such adaptive systems.

eess.SY↗

Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions

We develop and rigorously evaluate a deep learning based system that can accurately classify skin conditions while detecting rare conditions for which there is not enough data available for training a confident classifier. We frame this task as an out-of-distribution (OOD) detection problem. Our novel approach, hierarchical outlier detection (HOD) assigns multiple abstention classes for each training outlier class and jointly performs a coarse classification of inliers vs. outliers, along with fine-grained classification of the individual classes. We demonstrate the effectiveness of the HOD loss in conjunction with modern representation learning approaches (BiT, SimCLR, MICLe) and explore different ensembling strategies for further improving the results. We perform an extensive subgroup analysis over conditions of varying risk levels and different skin types to investigate how the OOD detection performance changes over each subgroup and demonstrate the gains of our framework in comparison to baselines. Finally, we introduce a cost metric to approximate downstream clinical impact. We use this cost metric to compare the proposed method against a baseline system, thereby making a stronger case for the overall system effectiveness in a real-world deployment scenario.

cs.CV↗

Big Self-Supervised Models Advance Medical Image Classification

Self-supervised pretraining followed by supervised fine-tuning has seen success in image recognition, especially when labeled examples are scarce, but has received limited attention in medical image analysis. This paper studies the effectiveness of self-supervised learning as a pretraining strategy for medical image classification. We conduct experiments on two distinct tasks: dermatology skin condition classification from digital camera images and multi-label chest X-ray classification, and demonstrate that self-supervised learning on ImageNet, followed by additional self-supervised learning on unlabeled domain-specific medical images significantly improves the accuracy of medical image classifiers. We introduce a novel Multi-Instance Contrastive Learning (MICLe) method that uses multiple images of the underlying pathology per patient case, when available, to construct more informative positive pairs for self-supervised learning. Combining our contributions, we achieve an improvement of 6.7% in top-1 accuracy and an improvement of 1.1% in mean AUC on dermatology and chest X-ray classification respectively, outperforming strong supervised baselines pretrained on ImageNet. In addition, we show that big self-supervised models are robust to distribution shift and can learn efficiently with a small number of labeled medical images.

eess.IV↗

Supervised Transfer Learning at Scale for Medical Imaging

Transfer learning is a standard technique to improve performance on tasks with limited data. However, for medical imaging, the value of transfer learning is less clear. This is likely due to the large domain mismatch between the usual natural-image pre-training (e.g. ImageNet) and medical images. However, recent advances in transfer learning have shown substantial improvements from scale. We investigate whether modern methods can change the fortune of transfer learning for medical imaging. For this, we study the class of large-scale pre-trained networks presented by Kolesnikov et al. on three diverse imaging tasks: chest radiography, mammography, and dermatology. We study both transfer performance and critical properties for the deployment in the medical domain, including: out-of-distribution generalization, data-efficiency, sub-group fairness, and uncertainty estimation. Interestingly, we find that for some of these properties transfer from natural to medical images is indeed extremely effective, but only when performed at sufficient scale.

cs.CV↗

Stability of the integral control of stable nonlinear systems

PI controllers are the most widespread type of controllers and there is an intuitive understanding that if their gains are sufficiently small and of the correct sign, then they always work. In this paper we try to give some rigorous backing to this claim, under specific assumptions. Let $\bf P$ be a nonlinear system described by $\dot x=f(x,u)$, $y=g(x)$, where the state trajectory $x$ takes values in $R^n$, $u$ and $y$ are scalar and $f,g$ are of class $C^1$. We assume that there is a Lipschitz function $Ξ:[u_{min},u_{max}]\rightarrow R^n$ such that for every constant input $u_0\in[u_{min},u_{max}]$, $Ξ(u_0)$ is an exponentially stable equilibrium point of $\bf P$. We also assume that $G(u)=g(Ξ(u))$, which is the steady state input-output map of $\bf P$, is strictly increasing. Denoting $y_{min}=G(u_{min})$ and $y_{max}=G(u_{max})$, we assume that the reference value $r$ is in $(y_{min},y_{max})$. Our aim is that $y$ should track $r$, i.e., $y\rightarrow r$ as $t\rightarrow\infty$, while the input of $P$ is only allowed to be in $[u_{min},u_{max}]$. For this, we introduce a variation of the integrator, called the saturating integrator, and connect it in feedback with $\bf P$ in the standard way, with gain $k>0$. We show that for any small enough $k$, the closed-loop system is (locally) exponentially stable around an equilibrium point $(Xi(u_r),u_r)$, with a large region of attraction $X_T\subset R^n\times[u_{min},u_{max}]$. When the state $(x(t),u(t))$ of the closed-loop system converges to $(Ξ(u_r),u_r)$, then the tracking error $r-y$ tends to zero. The compact set $X_T$ can be made larger by choosing a larger parameter $T>0$, resulting in smaller $k$.

math.OC↗

Underspecification Presents Challenges for Credibility in Modern Machine Learning

ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline is underspecified when it can return many predictors with equivalently strong held-out performance in the training domain. Underspecification is common in modern ML pipelines, such as those based on deep learning. Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains. This ambiguity can lead to instability and poor model behavior in practice, and is a distinct failure mode from previously identified issues arising from structural mismatch between training and deployment domains. We show that this problem appears in a wide variety of practical ML pipelines, using examples from computer vision, medical imaging, natural language processing, clinical risk prediction based on electronic health records, and medical genomics. Our results show the need to explicitly account for underspecification in modeling pipelines that are intended for real-world deployment in any domain.

cs.LG↗

Addressing the Real-world Class Imbalance Problem in Dermatology

Class imbalance is a common problem in medical diagnosis, causing a standard classifier to be biased towards the common classes and perform poorly on the rare classes. This is especially true for dermatology, a specialty with thousands of skin conditions but many of which have low prevalence in the real world. Motivated by recent advances, we explore few-shot learning methods as well as conventional class imbalance techniques for the skin condition recognition problem and propose an evaluation setup to fairly assess the real-world utility of such approaches. We find the performance of few-show learning methods does not reach that of conventional class imbalance techniques, but combining the two approaches using a novel ensemble improves model performance, especially for rare classes. We conclude that ensembling can be useful to address the class imbalance problem, yet progress can further be accelerated by real-world evaluation setups for benchmarking new methods.

cs.CV↗

Compensating PDE actuator and sensor dynamics using Sylvester equation

We consider the problem of stabilizing PDE-ODE cascade systems in which the input is applied to the PDE system whose output drives the ODE system. We also consider the dual problem of constructing an observer for ODE-PDE cascade systems in which the output of the ODE system drives the PDE system, whose output is measured. The PDE in these problems is stable and the ODE is unstable. While the ODE system models the plant in both the problems, the PDE system models the actuator in the stabilization problem and the sensor in the dual problem. In the literature, these problems have been solved for specific PDE models using the backstepping approach. In contrast, in the present work we consider these problems in an abstract framework by letting the PDE system be any regular linear system. Using a state transformation obtained by solving a Sylvester equation with unbounded operators, we first diagonalize the state operator corresponding to the cascade systems. We then solve the stabilization problem and the dual estimation problem, provided they are solvable, by solving certain finite-dimensional counterparts. We also derive necessary and sufficient conditions for verifying the solvability of these problems. We show that the controller which solves the stabilization problem is robust to certain unbounded perturbations. We illustrate our theory by designing a stabilizing controller for a PDE-ODE cascade in which the PDE is a 1D diffusion equation and an observer for a ODE-PDE cascade in which the PDE is a 1D wave equation.

eess.SY↗

Modelling, Controllability and Gait Design for a Spherical Flexible Swimmer

This paper discusses modelling, controllability and gait design for a spherical flexible swimmer. We first present a kinematic model of a low Reynolds number spherical flexible swimming mechanism with periodic surface deformations in the radial and azimuthal directions. The model is then converted to a finite dimensional driftless, affine-in-control principal kinematic form by representing the surface deformations as a linear combination of finitely many Legendre polynomials. A controllability analysis is then done for this swimmer to conclude that the swimmer is locally controllable on $\mathbb{R}^3$ for certain combinations of the Legendre polynomials. The rates of the coefficients of the polynomials are considered as the control inputs for surface deformation. Finally, the Abelian nature of the structure group of the swimmer's configuration space is exploited to synthesize a curvature based gait for the spherical flexile swimmer and a rigid-link swimmer.

eess.SY↗

Contrastive Training for Improved Out-of-Distribution Detection

Reliable detection of out-of-distribution (OOD) inputs is increasingly understood to be a precondition for deployment of machine learning systems. This paper proposes and investigates the use of contrastive training to boost OOD detection performance. Unlike leading methods for OOD detection, our approach does not require access to examples labeled explicitly as OOD, which can be difficult to collect in practice. We show in extensive experiments that contrastive training significantly helps OOD detection performance on a number of common benchmarks. By introducing and employing the Confusion Log Probability (CLP) score, which quantifies the difficulty of the OOD detection task by capturing the similarity of inlier and outlier datasets, we show that our method especially improves performance in the `near OOD' classes -- a particularly challenging setting for previous methods.

cs.LG↗

DermGAN: Synthetic Generation of Clinical Skin Images with Pathology

Despite the recent success in applying supervised deep learning to medical imaging tasks, the problem of obtaining large and diverse expert-annotated datasets required for the development of high performant models remains particularly challenging. In this work, we explore the possibility of using Generative Adverserial Networks (GAN) to synthesize clinical images with skin condition. We propose DermGAN, an adaptation of the popular Pix2Pix architecture, to create synthetic images for a pre-specified skin condition while being able to vary its size, location and the underlying skin color. We demonstrate that the generated images are of high fidelity using objective GAN evaluation metrics. In a Human Turing test, we note that the synthetic images are not only visually similar to real images, but also embody the respective skin condition in dermatologists' eyes. Finally, when using the synthetic images as a data augmentation technique for training a skin condition classifier, we observe that the model performs comparably to the baseline model overall while improving on rare but malignant conditions.

cs.CV↗

A deep learning system for differential diagnosis of skin diseases

Skin conditions affect an estimated 1.9 billion people worldwide. A shortage of dermatologists causes long wait times and leads patients to seek dermatologic care from general practitioners. However, the diagnostic accuracy of general practitioners has been reported to be only 0.24-0.70 (compared to 0.77-0.96 for dermatologists), resulting in referral errors, delays in care, and errors in diagnosis and treatment. In this paper, we developed a deep learning system (DLS) to provide a differential diagnosis of skin conditions for clinical cases (skin photographs and associated medical histories). The DLS distinguishes between 26 skin conditions that represent roughly 80% of the volume of skin conditions seen in primary care. The DLS was developed and validated using de-identified cases from a teledermatology practice serving 17 clinical sites via a temporal split: the first 14,021 cases for development and the last 3,756 cases for validation. On the validation set, where a panel of three board-certified dermatologists defined the reference standard for every case, the DLS achieved 0.71 and 0.93 top-1 and top-3 accuracies respectively. For a random subset of the validation set (n=963 cases), 18 clinicians reviewed the cases for comparison. On this subset, the DLS achieved a 0.67 top-1 accuracy, non-inferior to board-certified dermatologists (0.63, p<0.001), and higher than primary care physicians (PCPs, 0.45) and nurse practitioners (NPs, 0.41). The top-3 accuracy showed a similar trend: 0.90 DLS, 0.75 dermatologists, 0.60 PCPs, and 0.55 NPs. These results highlight the potential of the DLS to augment general practitioners to accurately diagnose skin conditions by suggesting differential diagnoses that may not have been considered. Future work will be needed to prospectively assess the clinical impact of using this tool in actual clinical workflows.

eess.IV↗

Towards VQA Models That Can Read

Studies have shown that a dominant class of questions asked by visually impaired users on images of their surroundings involves reading text in the image. But today's VQA models can not read! Our paper takes a first step towards addressing this problem. First, we introduce a new "TextVQA" dataset to facilitate progress on this important problem. Existing datasets either have a small proportion of questions about text (e.g., the VQA dataset) or are too small (e.g., the VizWiz dataset). TextVQA contains 45,336 questions on 28,408 images that require reasoning about text to answer. Second, we introduce a novel model architecture that reads text in the image, reasons about it in the context of the image and the question, and predicts an answer which might be a deduction based on the text and the image or composed of the strings found in the image. Consequently, we call our approach Look, Read, Reason & Answer (LoRRA). We show that LoRRA outperforms existing state-of-the-art VQA models on our TextVQA dataset. We find that the gap between human performance and machine performance is significantly larger on TextVQA than on VQA 2.0, suggesting that TextVQA is well-suited to benchmark progress along directions complementary to VQA 2.0.

cs.CL↗

Minimal order controllers for output regulation of nonlinear systems

This paper is about the nonlinear local error feedback regulator problem. The plant is a nonlinear finite-dimensional system with a single control input and a single output and it is locally exponentially stable around the origin. The plant is driven, via a separate disturbance input, by a Lyapunov stable exosystem whose states are nonwandering. The reference signal that the plant output must track is a nonlinear function of the exosystem state. The local error feedback regulator problem is to design a dynamic feedback controller, with the tracking error as its input, such that (i) the closed-loop system of the plant and the controller is locally exponentially stable, and (ii) the tracking error tends to zero for all sufficiently small initial conditions of the plant, the controller and the exosystem. Under the assumption that the above regulator problem is solvable, we propose a nonlinear controller whose order is relatively small - typically equal to the order of the exosystem, and which solves the regulator problem. The emphasis is on the low order of the controller. The stability assumption on the plant (which can be relaxed to some extent) is crucial for making it possible to design a low order controller. We will show, under certain assumptions, that our proposed controller is of minimal order. Three examples are presented - the first illustrates our controller design procedure using an exosystem whose trajectories are periodic even though the state operator of the linearized exosystem contains a nontrivial Jordan block. The second example is more involved, and shows that sometimes a nontrivial immersion of the exosystem is needed in the design. The third example, based on output voltage regulation for a boost power converter, shows how the regulator equations may reduce to a first order PDE with no given boundary conditions, but which nevertheless has a locally unique solution.

math.OC↗

The global bifurcation picture for ground states in nonlinear Schrodinger equations

In this paper, we propose a method of finding all coherent structures supported by a given nonlinear wave equation. It relies on enhancing the recent global bifurcation theory as developed by Dancer, Toland, Buffoni and others, by determining all the limit points of the coherent structure manifolds at the boundary of the Fredholm domain. Local bifurcation theory is then used to trace back these manifolds from their limit points into the interior of the Fredholm domain identifying the singularities along them. This way all coherent structure manifold are discovered except may be the ones which form loops and hence never reach the boundary. The method is then applied to the Schrodinger equation with a power nonlinearity for which all ground states are identified.

math.AP↗

Pythia v0.1: the Winning Entry to the VQA Challenge 2018

This document describes Pythia v0.1, the winning entry from Facebook AI Research (FAIR)'s A-STAR team to the VQA Challenge 2018. Our starting point is a modular re-implementation of the bottom-up top-down (up-down) model. We demonstrate that by making subtle but important changes to the model architecture and the learning rate schedule, fine-tuning image features, and adding data augmentation, we can significantly improve the performance of the up-down model on VQA v2.0 dataset -- from 65.67% to 70.22%. Furthermore, by using a diverse ensemble of models trained with different features and on different datasets, we are able to significantly improve over the 'standard' way of ensembling (i.e. same model with different random seeds) by 1.31%. Overall, we achieve 72.27% on the test-std split of the VQA v2.0 dataset. Our code in its entirety (training, evaluation, data-augmentation, ensembling) and pre-trained models are publicly available at: https://github.com/facebookresearch/pythia

cs.CV↗

Almost global asymptotic stability of a grid-connected synchronous generator

We study the global asymptotic behavior of a grid-connected constant field current synchronous generator (SG). The grid is regarded as an "infinite bus", i.e. a three-phase AC voltage source. The generator does not include any controller other than the frequency droop loop. This means that the mechanical torque applied to this generator is an affine function of its angular velocity. The negative slope of this function is the frequency droop constant. We derive sufficient conditions on the SG parameters under which there exist exactly two periodic state trajectories for the SG, one stable and another unstable, and for almost all initial states, the state trajectory of the SG converges to the stable periodic trajectory (all the angles are measured modulo $2π$). Along both periodic state trajectories, the angular velocity of the SG is equal to the grid frequency. Our sufficient conditions are easy to check computationally. An important tool in our analysis is an integro-differential equation called the {\em exact swing equation}, which resembles a forced pendulum equation and is equivalent to our fourth order model of the grid-connected SG. Apart from our objective of providing an analytical proof for a global asymptotic behavior observed in a classical dynamical system, a key motivation for this work is the development of synchronverters which are inverters that mimic the behavior of SGs. Understanding the global dynamics of SGs can guide the choice of synchronverter parameters and operation. As an application we find a set of stable nominal parameters for a 500 kW synchronverter.

math.OC↗