arXiv subjects

Deepan Muthirayan

Publications and source records attributed to Deepan Muthirayan.

31 records · Page 2Linked to original sources

arXiv2021

Adaptive Gradient Online Control

In this work we consider the online control of a known linear dynamic system with adversarial disturbance and adversarial controller cost. The goal in online control is to minimize the regret, defined as the difference between cumulative cost over a period $T$ and the cumulative cost for the best policy from a comparator class. For the setting we consider, we generalize the previously proposed online Disturbance Response Controller (DRC) to the adaptive gradient online Disturbance Response Controller. Using the modified controller, we present novel regret guarantees that improves the established regret guarantees for the same setting. We show that the proposed online learning controller is able to achieve intermediate intermediate regret rates between $\sqrt{T}$ and $\log{T}$ for intermediate convex conditions, while it recovers the previously established regret results for general convex controller cost and strongly convex controller cost.

math.OC↗

arXiv2021

Memory Augmented Neural Network Adaptive Controllers: Performance and Stability

In this paper, we propose a novel control architecture, inspired from neuroscience, for adaptive control of continuous-time systems. The proposed architecture, in the setting of standard Neural Network (NN) based adaptive control, augments an external working memory to the NN. The controller, through a write operation, writes the hidden layer feature vector of the NN to the external working memory and can also update this information with the observed error in the output. Through a read operation, the controller retrieves information from the working memory to modify the final control signal. First, we consider a simpler estimation problem to theoretically study the effect of an external memory and prove that the estimation accuracy can be improved by incorporating memory. We then consider a model reference NN adaptive controller for linear systems with matched uncertainty to implement and illustrate our ideas. We prove that the resulting controller leads to a Uniformly Bounded (UB) stable closed loop system. Through extensive simulations and specific metrics, such as peak deviation and settling time, we show that memory augmentation improves learning significantly. Importantly, we also provide evidence for and insights on the mechanism by which this specific memory augmentation improves learning.

eess.SY↗

arXiv2021

Graph Learning for Cognitive Digital Twins in Manufacturing Systems

Future manufacturing requires complex systems that connect simulation platforms and virtualization with physical data from industrial processes. Digital twins incorporate a physical twin, a digital twin, and the connection between the two. Benefits of using digital twins, especially in manufacturing, are abundant as they can increase efficiency across an entire manufacturing life-cycle. The digital twin concept has become increasingly sophisticated and capable over time, enabled by rises in many technologies. In this paper, we detail the cognitive digital twin as the next stage of advancement of a digital twin that will help realize the vision of Industry 4.0. Cognitive digital twins will allow enterprises to creatively, effectively, and efficiently exploit implicit knowledge drawn from the experience of existing manufacturing systems. They also enable more autonomous decisions and control, while improving the performance across the enterprise (at scale). This paper presents graph learning as one potential pathway towards enabling cognitive functionalities in manufacturing digital twins. A novel approach to realize cognitive digital twins in the product design stage of manufacturing that utilizes graph learning is presented.

cs.LG↗

arXiv2021

Generative Adversarial Imitation Learning for Empathy-based AI

Generative adversarial imitation learning (GAIL) is a model-free algorithm that has been shown to provide strong results in imitating complex behaviors in high-dimensional environments. In this paper, we utilize the GAIL model for text generation to develop empathy-based context-aware conversational AI. Our model uses an expert trajectory of empathetic prompt-response dialogues which can accurately exhibit the correct empathetic emotion when generating a response. The Generator of the GAIL model uses the GPT-2 sequential pre-trained language model trained on 117 million parameters from 40 GB of internet data. We propose a novel application of an approach used in transfer learning to fine tune the GPT-2 model in order to generate concise, user-specific empathetic responses validated against the Discriminator. Our novel GAIL model utilizes a sentiment analysis history-based reinforcement learning approach to empathetically respond to human interactions in a personalized manner. We find that our model's response scores on various human-generated prompts collected from the Facebook Empathetic Dialogues dataset outperform baseline counterparts. Moreover, our model improves upon various history-based conversational AI models developed recently, as our model's performance over a sustained conversation of 3 or more interactions outperform similar conversational AI models.

cs.CL↗

arXiv2021

Neuroscience-Inspired Algorithms for the Predictive Maintenance of Manufacturing Systems

If machine failures can be detected preemptively, then maintenance and repairs can be performed more efficiently, reducing production costs. Many machine learning techniques for performing early failure detection using vibration data have been proposed; however, these methods are often power and data-hungry, susceptible to noise, and require large amounts of data preprocessing. Also, training is usually only performed once before inference, so they do not learn and adapt as the machine ages. Thus, we propose a method of performing online, real-time anomaly detection for predictive maintenance using Hierarchical Temporal Memory (HTM). Inspired by the human neocortex, HTMs learn and adapt continuously and are robust to noise. Using the Numenta Anomaly Benchmark, we empirically demonstrate that our approach outperforms state-of-the-art algorithms at preemptively detecting real-world cases of bearing failures and simulated 3D printer failures. Our approach achieves an average score of 64.71, surpassing state-of-the-art deep-learning (49.38) and statistical (61.06) methods.

cs.NE↗

arXiv2020

Cognitive Preadaptation for Resilient Adaptive Control

In this paper, we investigate a novel control architecture and algorithm for incorporating preadaption functions. We propose a preadaptation mechanism that can augment any adaptive control scheme and improve its resilience. Through simulations of a flight control system we illustrate the effectiveness of the preadaptation mechanism in improving the adaptation. We show that the preadaptation mechanism we propose can reduce the peak of the response by as much as $50\%$. The scenarios we present also show that the preadaptation mechanism is effective across a wide range of scenarios suggesting that the mechanism is reliable.

eess.SY↗

arXiv2020

Scene-Graph Augmented Data-Driven Risk Assessment of Autonomous Vehicle Decisions

Despite impressive advancements in Autonomous Driving Systems (ADS), navigation in complex road conditions remains a challenging problem. There is considerable evidence that evaluating the subjective risk level of various decisions can improve ADS' safety in both normal and complex driving scenarios. However, existing deep learning-based methods often fail to model the relationships between traffic participants and can suffer when faced with complex real-world scenarios. Besides, these methods lack transferability and explainability. To address these limitations, we propose a novel data-driven approach that uses scene-graphs as intermediate representations. Our approach includes a Multi-Relation Graph Convolution Network, a Long-Short Term Memory Network, and attention layers for modeling the subjective risk of driving maneuvers. To train our model, we formulate this task as a supervised scene classification problem. We consider a typical use case to demonstrate our model's capabilities: lane changes. We show that our approach achieves a higher classification accuracy than the state-of-the-art approach on both large (96.4% vs. 91.2%) and small (91.8% vs. 71.2%) synthesized datasets, also illustrating that our approach can learn effectively even from smaller datasets. We also show that our model trained on a synthesized dataset achieves an average accuracy of 87.8% when tested on a real-world dataset compared to the 70.3% accuracy achieved by the state-of-the-art model trained on the same synthesized dataset, showing that our approach can more effectively transfer knowledge. Finally, we demonstrate that the use of spatial and temporal attention layers improves our model's performance by 2.7% and 0.7% respectively, and increases its explainability.

cs.CV↗

arXiv2020

Selling Demand Response Using Options

Wholesale electricity markets in many jurisdictions use a two-settlement structure: a day-ahead market for bulk power transactions and a real-time market for fine-grain supply-demand balancing. This paper explores trading demand response assets within this two-settlement market structure. We consider two approaches for trading demand response assets: (a) an intermediate spot market with contingent pricing, and (b) an over-the-counter options contract. In the first case, we characterize the competitive equilibrium of the spot market, and show that it is socially optimal. Economic orthodoxy advocates spot markets, but these require expensive infrastructure and regulatory blessing. In the second case, we characterize competitive equilibria and compare its efficiency with the idealized spot market. Options contract are private bilateral over-the-counter transactions and do not require regulatory approval. We show that the optimal social welfare is, in general, not supported. We then design optimal option prices that minimize the social welfare gap. This optimal design serves to approximate the ideal spot market for demand response using options with modest loss of efficiency. Our results are validated through numerical simulations.

eess.SY↗

arXiv2020

Online Algorithms for Dynamic Matching Markets in Power Distribution Systems

This paper proposes online algorithms for dynamic matching markets in power distribution systems, which at any real-time operation instance decides about matching -- or delaying the supply of -- flexible loads with available renewable generation with the objective of maximizing the social welfare of the exchange in the system. More specifically, two online matching algorithms are proposed for the following generation-load scenarios: (i) when the mean of renewable generation is greater than the mean of the flexible load, and (ii) when the condition (i) is reversed. With the intuition that the performance of such algorithms degrades with increasing randomness of the supply and demand, two properties are proposed for assessing the performance of the algorithms. First property is convergence to optimality (CO) as the underlying randomness of renewable generation and customer loads goes to zero. The second property is deviation from optimality, is measured as a function of the standard deviation of the underlying randomness of renewable generation and customer loads. The algorithm proposed for the first scenario is shown to satisfy CO and a deviation from optimal that varies linearly with the variation in the standard deviation. But the same algorithm is shown to not satisfy CO for the second scenario. We then show that the algorithm proposed for the second scenario satisfies CO and a deviation from optimal that varies linearly with the variation in standard deviation plus an offset.

eess.SY↗

arXiv2020

Improved Attention Models for Memory Augmented Neural Network Adaptive Controllers

We introduced a {\it working memory} augmented adaptive controller in our recent work. The controller uses attention to read from and write to the working memory. Attention allows the controller to read specific information that is relevant and update its working memory with information based on its relevance. The retrieved information is used to modify the final control input computed by the controller. We showed that this modification speeds up learning. In the above work, we used a soft-attention mechanism for the adaptive controller. Controllers that use soft attention or hard attention mechanisms are limited either because they can forget the information or fail to shift attention when the information they are reading becomes less relevant. We propose an attention mechanism that comprises of (i) a hard attention mechanism and additionally (ii) an attention reallocation mechanism. The attention reallocation enables the controller to reallocate attention to a different location when the relevance of the location it is reading from diminishes. The reallocation also ensures that the information stored in the memory before the shift in attention is retained which can be lost in both soft and hard attention mechanisms. We illustrate through detailed simulations of various scenarios for two link robot and three link robot arm systems we illustrate the effectiveness of the proposed attention mechanism.

eess.SY↗

arXiv2020

Memory Augmented Neural Network Adaptive Controller for Strict Feedback Nonlinear Systems

In this work, we consider the adaptive nonlinear control problem for strict feedback nonlinear systems, where the functions that determine the dynamics of the system are completely unknown. We assume that certain upper bounds for the functions $g_i$s of the system are known. The objective of the control design is to design an adaptive controller that can adapt to changes in the unknown functions that are even abrupt. We propose a novel backstepping memory augmented NN (MANN) adaptive control method for the control of strict feedback non-linear systems. Here, each NN, in the backstepping NN adaptive controller, is augmented with an external working memory. The NN can write relevant information to its working memory and later retrieve them to modify its output, thus providing it with the capability to leverage past learned information effectively and improve its speed of learning. We propose a specific design for this external memory interface and show that the proposed control design achieves bounded stability for the closed loop system. We also provide substantial numerical evidence showing that the proposed memory augmentation improves the speed of learning by a significant margin.

eess.SY↗

arXiv2019

Mechanism Design for Demand Response Programs

Demand Response (DR) programs serve to reduce the consumption of electricity at times when the supply is scarce and expensive. The utility informs the aggregator of an anticipated DR event. The aggregator calls on a subset of its pool of recruited agents to reduce their electricity use. Agents are paid for reducing their energy consumption from contractually established baselines. Baselines are counter-factual consumption estimates of the energy an agent would have consumed if they were not participating in the DR program. Baselines are used to determine payments to agents. This creates an incentive for agents to inflate their baselines. We propose a novel self-reported baseline mechanism (SRBM) where each agent reports its baseline and marginal utility. These reports are strategic and need not be truthful. Based on the reported information, the aggregator selects or calls on agents to meet the load reduction target. Called agents are paid for observed reductions from their self-reported baselines. Agents who are not called face penalties for consumption shortfalls below their baselines. The mechanism is specified by the probability with which agents are called, reward prices for called agents, and penalty prices for agents who are not called. Under SRBM, we show that truthful reporting of baseline consumption and marginal utility is a dominant strategy. Thus, SRBM eliminates the incentive for agents to inflate baselines. SRBM is assured to meet the load reduction target. SRBM is also nearly efficient since it selects agents with the smallest marginal utilities, and each called agent contributes maximally to the load reduction target. Finally, we show that SRBM is almost optimal in the metric of average cost of DR provision faced by the aggregator.

cs.GT↗

arXiv2019

A Minimal Incentive-based Demand Response Program With Self Reported Baseline Mechanism

In this paper, we propose a novel incentive based Demand Response (DR) program with a self reported baseline mechanism. The System Operator (SO) managing the DR program recruits consumers or aggregators of DR resources. The recruited consumers are required to only report their baseline, which is the minimal information necessary for any DR program. During a DR event, a set of consumers, from this pool of recruited consumers, are randomly selected. The consumers are selected such that the required load reduction is delivered. The selected consumers, who reduce their load, are rewarded for their services and other recruited consumers, who deviate from their reported baseline, are penalized. The randomization in selection and penalty ensure that the baseline inflation is controlled. We also justify that the selection probability can be simultaneously used to control SO's cost. This allows the SO to design the mechanism such that its cost is almost optimal when there are no recruitment costs or at least significantly reduced otherwise. Finally, we also show that the proposed method of self-reported baseline outperforms other baseline estimation methods commonly used in practice.

eess.SY↗