arXiv subjects

Deepan Muthirayan

Publications and source records attributed to Deepan Muthirayan.

At least 19 recordsLinked to original sources

arXiv2026

OpenLanguageModel: Readable and Composable Small-Language-Model Pretraining for Education and Research

OpenLanguageModel (OLM) is an open-source PyTorch library for building and pretraining small language models while keeping their machinery visible. In OLM, model code reads like the architecture: components are ordinary modules, while Block, Residual, Repeat, and Parallel describe how they are wired. The resulting model can move unchanged from a teaching notebook to a complete pretraining run or a research ablation. OLM connects this readable model layer to tokenizers, local and streaming datasets, optimization, mixed precision, callbacks, checkpoints, and hardware-aware CPU, single-GPU, and single-node multi-GPU execution. We demonstrate the full path by tracing GPT-2 from diagram to code, launching a FineWeb-Edu training script, replacing one attention component, and letting AutoTrainer configure the available machine. The package includes 27 presets across nine familiar model families and documentation that progresses from LM fundamentals to architecture research. Validation shows close agreement with independent reference implementations, 90.6% four-GPU weak-scaling efficiency for a 348M-parameter workload, compact architecture edits, and positive early usability results. OLM is MIT-licensed and available through PyPI, GitHub, and its documentation site.

cs.CL↗

arXiv2025

Emissions and Performance Trade-off Between Small and Large Language Models

The advent of Large Language Models (LLMs) has raised concerns about their enormous carbon footprint, starting with energy-intensive training and continuing through repeated inference. This study investigates the potential of using fine-tuned Small Language Models (SLMs) as a sustainable alternative for predefined tasks. Here, we present a comparative analysis of the performance-emissions trade-off between LLMs and fine-tuned SLMs across selected tasks under Natural Language Processing, Reasoning and Programming. Our results show that in four out of the six selected tasks, SLMs maintained comparable performances for a significant reduction in carbon emissions during inference. Our findings demonstrate the viability of smaller models in mitigating the environmental impact of resource-heavy LLMs, thus advancing towards sustainable, green AI.

cs.CL↗

arXiv2024

Long-term Fairness For Real-time Decision Making: A Constrained Online Optimization Approach

Machine learning (ML) has demonstrated remarkable capabilities across many real-world systems, from predictive modeling to intelligent automation. However, the widespread integration of machine learning also makes it necessary to ensure machine learning-driven decision-making systems do not violate ethical principles and values of society in which they operate. As ML-driven decisions proliferate, particularly in cases involving sensitive attributes such as gender, race, and age, to name a few, the need for equity and impartiality has emerged as a fundamental concern. In situations demanding real-time decision-making, fairness objectives become more nuanced and complex: instantaneous fairness to ensure equity in every time slot, and long-term fairness to ensure fairness over a period of time. There is a growing awareness that real-world systems that operate over long periods and require fairness over different timelines. However, existing approaches mainly address dynamic costs with time-invariant fairness constraints, often disregarding the challenges posed by time-varying fairness constraints. To bridge this gap, this work introduces a framework for ensuring long-term fairness within dynamic decision-making systems characterized by time-varying fairness constraints. We formulate the decision problem with fairness constraints over a period as a constrained online optimization problem. A novel online algorithm, named LoTFair, is presented that solves the problem 'on the fly'. We prove that LoTFair can make overall fairness violations negligible while maintaining the performance over the long run.

cs.LG↗

arXiv2023

Online Learning for Incentive-Based Demand Response

In this paper, we consider the problem of learning online to manage Demand Response (DR) resources. A typical DR mechanism requires the DR manager to assign a baseline to the participating consumer, where the baseline is an estimate of the counterfactual consumption of the consumer had it not been called to provide the DR service. A challenge in estimating baseline is the incentive the consumer has to inflate the baseline estimate. We consider the problem of learning online to estimate the baseline and to optimize the operating costs over a period of time under such incentives. We propose an online learning scheme that employs least-squares for estimation with a perturbation to the reward price (for the DR services or load curtailment) that is designed to balance the exploration and exploitation trade-off that arises with online learning. We show that, our proposed scheme is able to achieve a very low regret of $\mathcal{O}\left((\log{T})^2\right)$ with respect to the optimal operating cost over $T$ days of the DR program with full knowledge of the baseline, and is individually rational for the consumers to participate. Our scheme is significantly better than the averaging type approach, which only fetches $\mathcal{O}(T^{1/3})$ regret.

cs.LG↗

arXiv2023

Scalable Grid-Aware Dynamic Matching using Deep Reinforcement Learning

This paper proposes a two-level hierarchical matching framework for Integrated Hybrid Resources (IHRs) with grid constraints. An IHR is a collection of Renewable Energy Sources (RES) and flexible customers within a certain power system zone, endowed with an agent to match. The key idea is to pick the IHR zones so that the power loss effects within the IHRs can be neglected. This simplifies the overall matching problem into independent IHR-level matching problems and an upper-level optimal power flow problem to meet the IHR-level upstream flow requirements while respecting the grid constraints. Within each IHR, the agent employs a scalable Deep Reinforcement Learning algorithm to identify matching solutions such that the customer's service constraints are met. The central agent then solves an optimal power flow problem with the IHRs as the nodes, with their active power flow and reactive power {capacities}, and grid constraints to scalably determine the final flows such that matched power can be delivered to the extent the grid constraints are satisfied. The proposed framework is implemented on a test power distribution system, and multiple case studies are presented to substantiate the welfare efficiency of the proposed solution and the satisfaction of the grid and customers' servicing constraints.

eess.SY↗

arXiv2022

Online Convex Optimization with Long Term Constraints for Predictable Sequences

In this paper, we investigate the framework of Online Convex Optimization (OCO) for online learning. OCO offers a very powerful online learning framework for many applications. In this context, we study a specific framework of OCO called {\it OCO with long term constraints}. Long term constraints are introduced typically as an alternative to reduce the complexity of the projection at every update step in online optimization. While many algorithmic advances have been made towards online optimization with long term constraints, these algorithms typically assume that the sequence of cost functions over a certain $T$ finite steps that determine the cost to the online learner are adversarially generated. In many circumstances, the sequence of cost functions may not be unrelated, and thus predictable from those observed till a point of time. In this paper, we study the setting where the sequences are predictable. We present a novel online optimization algorithm for online optimization with long term constraints that can leverage such predictability. We show that, with a predictor that can supply the gradient information of the next function in the sequence, our algorithm can achieve an overall regret and constraint violation rate that is strictly less than the rate that is achievable without prediction.

cs.LG↗

arXiv2022

Change Point Detection Approach for Online Control of Unknown Time Varying Dynamical Systems

We propose a novel change point detection approach for online learning control with full information feedback (state, disturbance, and cost feedback) for unknown time-varying dynamical systems. We show that our algorithm can achieve a sub-linear regret with respect to the class of Disturbance Action Control (DAC) policies, which are a widely studied class of policies for online control of dynamical systems, for any sub-linear number of changes and very general class of systems: (i) matched disturbance system with general convex cost functions, (ii) general system with linear cost functions. Specifically, a (dynamic) regret of $\Gamma_T^{1/5}T^{4/5}$ can be achieved for these class of systems, where $\Gamma_T$ is the number of changes of the underlying system and $T$ is the duration of the control episode. That is, the change point detection approach achieves a sub-linear regret for any sub-linear number of changes, which other previous algorithms such as in \cite{minasyan2021online} cannot. Numerically, we demonstrate that the change point detection approach is superior to a standard restart approach \cite{minasyan2021online} and to standard online learning approaches for time-invariant dynamical systems. Our work presents the first regret guarantee for unknown time-varying dynamical systems in terms of a stronger notion of variability like the number of changes in the underlying system. The extension of our work to state and output feedback controllers is a subject of future work.

eess.SY↗

arXiv2022

Competing Bandits in Time Varying Matching Markets

We study the problem of online learning in two-sided non-stationary matching markets, where the objective is to converge to a stable match. In particular, we consider the setting where one side of the market, the arms, has fixed known set of preferences over the other side, the players. While this problem has been studied when the players have fixed but unknown preferences, in this work we study the problem of how to learn when the preferences of the players are time varying and unknown. Our contribution is a methodology that can handle any type of preference structure and variation scenario. We show that, with the proposed algorithm, each player receives a uniform sub-linear regret of {$\widetilde{\mathcal{O}}(L^{1/2}_TT^{1/2})$} up to the number of changes in the underlying preferences of the agents, $L_T$. Therefore, we show that the optimal rates for single-agent learning can be achieved in spite of the competition up to a difference of a constant factor. We also discuss extensions of this algorithm to the case where the number of changes need not be known a priori.

cs.LG↗

arXiv2022

Meta-Learning Online Control for Linear Dynamical Systems

In this paper, we consider the problem of finding a meta-learning online control algorithm that can learn across the tasks when faced with a sequence of $N$ (similar) control tasks. Each task involves controlling a linear dynamical system for a finite horizon of $T$ time steps. The cost function and system noise at each time step are adversarial and unknown to the controller before taking the control action. Meta-learning is a broad approach where the goal is to prescribe an online policy for any new unseen task exploiting the information from other tasks and the similarity between the tasks. We propose a meta-learning online control algorithm for the control setting and characterize its performance by \textit{meta-regret}, the average cumulative regret across the tasks. We show that when the number of tasks are sufficiently large, our proposed approach achieves a meta-regret that is smaller by a factor $D/D^{*}$ compared to an independent-learning online control algorithm which does not perform learning across the tasks, where $D$ is a problem constant and $D^{*}$ is a scalar that decreases with increase in the similarity between tasks. Thus, when the sequence of tasks are similar the regret of the proposed meta-learning online control is significantly lower than that of the naive approaches without meta-learning. We also present experiment results to demonstrate the superior performance achieved by our meta-learning algorithm.

cs.LG↗

arXiv2021

Online Learning for Predictive Control with Provable Regret Guarantees

We study the problem of online learning in predictive control of an unknown linear dynamical system with time varying cost functions which are unknown apriori. Specifically, we study the online learning problem where the control algorithm does not know the true system model and has only access to a fixed-length (that does not grow with the control horizon) preview of the future cost functions. The goal of the online algorithm is to minimize the dynamic regret, defined as the difference between the cumulative cost incurred by the algorithm and that of the best sequence of actions in hindsight. We propose two different online Model Predictive Control (MPC) algorithms to address this problem, namely Certainty Equivalence MPC (CE-MPC) algorithm and Optimistic MPC (O-MPC) algorithm. We show that under the standard stability assumption for the model estimate, the CE-MPC algorithm achieves $\mathcal{O}(T^{2/3})$ dynamic regret. We then extend this result to the setting where the stability assumption holds only for the true system model by proposing the O-MPC algorithm. We show that the O-MPC algorithm also achieves $\mathcal{O}(T^{2/3})$ dynamic regret, at the cost of some additional computation. We also present numerical studies to demonstrate the performance of our algorithm.

cs.LG↗

arXiv2021

Online Robust Control of Linear Dynamical Systems with Limited Prediction

We study the online robust control problem for linear dynamical systems with disturbances and uncertainties in the cost functions, with limited preview of the future disturbances and the cost functions, $N$. Our goal is to find an online control policy that can minimize the disturbance gain, defined as the ratio of the cumulative cost and the cumulative energy in the disturbances over a period of time, in the face of the uncertainties, and characterize its achievable gain in terms of the system relevant parameters. Our goals contrast with prior online control works for the same problem, which either focus on minimizing the static regret, a weaker performance metric, or assume a very large preview of the future uncertainties. Specifically, we consider a class of cost functions characterized by $\beta$ ($\beta < 1$), a number whose inverse bounds the variation of the cost functions. We propose a novel variation of the Receding Horizon Control as the online control policy. We show that, under standard system assumptions, when $N > 4/\beta^3$, the proposed algorithm can achieve a disturbance gain $(2/\beta+\rho(N)) \overline{\gamma}^2$, where $\overline{\gamma}^2$ is the best (minimum) possible disturbance gain for an oracle policy with full knowledge of the cost functions and disturbances, with $\rho(N) = O(1/N)$. We also demonstrate through simulations that the proposed policy satisfies the derived bounds and is consistently better than the standard RHC approach.

math.OC↗

arXiv2021

Spatio-Temporal Scene-Graph Embedding for Autonomous Vehicle Collision Prediction

In autonomous vehicles (AVs), early warning systems rely on collision prediction to ensure occupant safety. However, state-of-the-art methods using deep convolutional networks either fail at modeling collisions or are too expensive/slow, making them less suitable for deployment on AV edge hardware. To address these limitations, we propose sg2vec, a spatio-temporal scene-graph embedding methodology that uses Graph Neural Network (GNN) and Long Short-Term Memory (LSTM) layers to predict future collisions via visual scene perception. We demonstrate that sg2vec predicts collisions 8.11% more accurately and 39.07% earlier than the state-of-the-art method on synthesized datasets, and 29.47% more accurately on a challenging real-world collision dataset. We also show that sg2vec is better than the state-of-the-art at transferring knowledge from synthetic datasets to real-world driving datasets. Finally, we demonstrate that sg2vec performs inference 9.3x faster with an 88.0% smaller model, 32.4% less power, and 92.8% less energy than the state-of-the-art method on the industry-standard Nvidia DRIVE PX 2 platform, making it more suitable for implementation on the edge.

cs.CV↗

arXiv2021

Graph Learning for Cognitive Digital Twins in Manufacturing Systems

Future manufacturing requires complex systems that connect simulation platforms and virtualization with physical data from industrial processes. Digital twins incorporate a physical twin, a digital twin, and the connection between the two. Benefits of using digital twins, especially in manufacturing, are abundant as they can increase efficiency across an entire manufacturing life-cycle. The digital twin concept has become increasingly sophisticated and capable over time, enabled by rises in many technologies. In this paper, we detail the cognitive digital twin as the next stage of advancement of a digital twin that will help realize the vision of Industry 4.0. Cognitive digital twins will allow enterprises to creatively, effectively, and efficiently exploit implicit knowledge drawn from the experience of existing manufacturing systems. They also enable more autonomous decisions and control, while improving the performance across the enterprise (at scale). This paper presents graph learning as one potential pathway towards enabling cognitive functionalities in manufacturing digital twins. A novel approach to realize cognitive digital twins in the product design stage of manufacturing that utilizes graph learning is presented.

cs.LG↗

arXiv2021

Online Algorithms for Network Robustness under Connectivity Constraints

In this paper, we present algorithms for designing networks that are robust to node failures with minimal or limited number of links. We present algorithms for both the static network setting and the dynamic network setting; setting where new nodes can arrive in the future. For the static setting, we present algorithms for constructing the optimal network in terms of the number of links used for a given node size and the number of nodes that can fail. We then consider the dynamic setting where it is disruptive to remove any of the older links. For this setting, we present online algorithms for two cases: (i) when the number of nodes that can fail remains constant and (ii) when only the proportion of the nodes that can fail remains constant. We show that the proposed algorithm for the first case saves nearly $3/4$th of the total possible links at any point of time. We then present algorithms for various levels of the fraction of the nodes that can fail and characterize their link usage. We show that when $1/2$ the number of nodes can fail at any point of time, the proposed algorithm saves nearly $1/2$ of the total possible links at any point of time. We show that when the number of nodes that can fail is limited to the fraction $1/(2m)$ ($m \in \mathbb{N}$), the proposed algorithm saves nearly as much as $(1-1/2m)$ of the total possible links at any point of time. We also show that when the number of nodes that can fail at any point of time is $1/2$ of the number of nodes plus $n$, $n \in \mathbb{N}$, the number of links saved by the proposed algorithm reduces only linearly in $n$. We conjecture that the saving ratio achieved by the algorithms we present is optimal for the dynamic setting.

cs.DS↗

arXiv2021

Online Learning Robust Control of Nonlinear Dynamical Systems

In this work we address the problem of the online robust control of nonlinear dynamical systems perturbed by disturbance. We study the problem of attenuation of the total cost over a duration $T$ in response to the disturbances. We consider the setting where the cost function (at a particular time) is a general continuous function and adversarial, the disturbance is adversarial and bounded at any point of time. Our goal is to design a controller that can learn and adapt to achieve a certain level of attenuation. We analyse two cases (i) when the system is known and (ii) when the system is unknown. We measure the performance of the controller by the deviation of the controller's cost for a sequence of cost functions with respect to an attenuation $\gamma$, $R^p_t$. We propose an online controller and present guarantees for the metric $R^p_t$ when the maximum possible attenuation is given by $\overline{\gamma}$, which is a system constant. We show that when the controller has preview of the cost functions and the disturbances for a short duration of time and the system is known $R^p_T(\gamma) = O(1)$ when $\gamma \geq \gamma_c$, where $\gamma_c = \mathcal{O}(\overline{\gamma})$. We then show that when the system is unknown the proposed controller with a preview of the cost functions and the disturbances for a short horizon achieves $R^p_T(\gamma) = \mathcal{O}(N) + \mathcal{O}(1) + \mathcal{O}((T-N)g(N))$, when $\gamma \geq \gamma_c$, where $g(N)$ is the accuracy of a given nonlinear estimator and $N$ is the duration of the initial estimation period. We also characterize the lower bound on the required prediction horizon for these guarantees to hold in terms of the system constants.

eess.SY↗

arXiv2021

Generative Adversarial Imitation Learning for Empathy-based AI

Generative adversarial imitation learning (GAIL) is a model-free algorithm that has been shown to provide strong results in imitating complex behaviors in high-dimensional environments. In this paper, we utilize the GAIL model for text generation to develop empathy-based context-aware conversational AI. Our model uses an expert trajectory of empathetic prompt-response dialogues which can accurately exhibit the correct empathetic emotion when generating a response. The Generator of the GAIL model uses the GPT-2 sequential pre-trained language model trained on 117 million parameters from 40 GB of internet data. We propose a novel application of an approach used in transfer learning to fine tune the GPT-2 model in order to generate concise, user-specific empathetic responses validated against the Discriminator. Our novel GAIL model utilizes a sentiment analysis history-based reinforcement learning approach to empathetically respond to human interactions in a personalized manner. We find that our model's response scores on various human-generated prompts collected from the Facebook Empathetic Dialogues dataset outperform baseline counterparts. Moreover, our model improves upon various history-based conversational AI models developed recently, as our model's performance over a sustained conversation of 3 or more interactions outperform similar conversational AI models.

cs.CL↗

arXiv2021

Dynamic Matching Markets in Power Grid: Concepts and Solution using Deep Reinforcement Learning

Traditional bulk load flexibility options, such as load shifting and load curtailment, for managing uncertainty in power markets limit the diversity of options and ignore the preferences of the individual loads, thus reducing efficiency and welfare. This paper proposes an alternative to bulk load flexibility options for managing uncertainty in power markets: a reinforcement learning based dynamic matching market. We propose a novel hybrid learning-based model for maximizing social welfare in the dynamic matching market. The key features of our model is a fixed rule-based function and a learnable component that can be trained by data gathered online with no prior knowledge or expert supervision. The output of the learnable component is a probability distribution over the matching decisions for the individual customers. The proposed hybrid model enables the learning algorithm to find an effective matching policy that simultaneously satisfies the customers' servicing preferences. The simulations show that the learning algorithm learns an effective matching policy for different generation-consumption profiles and exhibits better performance compared to standard online matching heuristics such as Match on Arrival, Match to the Highest, and Match to the Earliest Deadline policies.

eess.SY↗

arXiv2021

Adaptive Gradient Online Control

In this work we consider the online control of a known linear dynamic system with adversarial disturbance and adversarial controller cost. The goal in online control is to minimize the regret, defined as the difference between cumulative cost over a period $T$ and the cumulative cost for the best policy from a comparator class. For the setting we consider, we generalize the previously proposed online Disturbance Response Controller (DRC) to the adaptive gradient online Disturbance Response Controller. Using the modified controller, we present novel regret guarantees that improves the established regret guarantees for the same setting. We show that the proposed online learning controller is able to achieve intermediate intermediate regret rates between $\sqrt{T}$ and $\log{T}$ for intermediate convex conditions, while it recovers the previously established regret results for general convex controller cost and strongly convex controller cost.

math.OC↗