arXiv subjects

Abhinav Sinha

Publications and source records attributed to Abhinav Sinha.

51 records · Page 3Linked to original sources

arXiv2023

Fairness in Preference-based Reinforcement Learning

In this paper, we address the issue of fairness in preference-based reinforcement learning (PbRL) in the presence of multiple objectives. The main objective is to design control policies that can optimize multiple objectives while treating each objective fairly. Toward this objective, we design a new fairness-induced preference-based reinforcement learning or FPbRL. The main idea of FPbRL is to learn vector reward functions associated with multiple objectives via new welfare-based preferences rather than reward-based preference in PbRL, coupled with policy learning via maximizing a generalized Gini welfare function. Finally, we provide experiment studies on three different environments to show that the proposed FPbRL approach can achieve both efficiency and equity for learning effective and fair policies.

cs.LG↗

arXiv2022

Three-agent Time-constrained Cooperative Pursuit-Evasion

This paper considers a pursuit-evasion scenario among three agents -- an evader, a pursuer, and a defender. We design cooperative guidance laws for the evader and the defender team to safeguard the evader from an attacking pursuer. Unlike differential games, optimal control formulations, and other heuristic methods, we propose a novel perspective on designing effective nonlinear feedback control laws for the evader-defender team using a time-constrained guidance approach. The evader lures the pursuer on the collision course by offering itself as bait. At the same time, the defender protects the evader from the pursuer by exercising control over the engagement duration. Depending on the nature of the mission, the defender may choose to take an aggressive or defensive stance. Such consideration widens the applicability of the proposed methods in various three-agent motion planning scenarios such as aircraft defense, asset guarding, search and rescue, surveillance, and secure transportation. We use a fixed-time sliding mode control strategy to design the control laws for the evader-defender team and a nonlinear finite-time disturbance observer to estimate the pursuer's maneuver. Finally, we present simulations to demonstrate favorable performance under various engagement geometries, thus vindicating the efficacy of the proposed designs.

eess.SY↗

arXiv2021

Cooperative Target Capture using Predefined-time Consensus over Fixed and Switching Networks

We propose predefined-time consensus-based cooperative guidance laws for a swarm of interceptors to simultaneously capture a target capable of executing various kinds of motions. Unlike leader-follower cooperative guidance techniques, the swarm of interceptors has no leader and each interceptor executes its own guidance command in a distributive fashion, thus obviating the residency of the mission over a single interceptor. We first design cooperative guidance commands subject to target's mobility, followed by the same when the target is stationary, and also account for interceptors' changing topologies. We further show through rigorous analysis that the proposed cooperative guidance laws guarantee consensus in the interceptors' time-to-go values within a predefined-time. The proposed design allows a feasible time of consensus in time-to-go to be set arbitrarily at will during the design regardless of the interceptors' initial time-to-go values, thereby ensuring a simultaneous target interception. We also demonstrate the efficacy of the proposed design via simulations for interceptors connected over static and dynamic networks.

eess.SY↗

arXiv2018

Control of a nonlinear continuous stirred tank reactor via event triggered sliding modes

Continuous Stirred Tank Reactors (CSTR) are the most important and central equipment in many chemical and biochemical industry that exhibit second order complex nonlinear dynamics. The nonlinear dynamics of CSTR poses many design and control challenges. The proposed controller guarantees a stable closed loop behavior over multiple operating points even in presence of disturbances and parametric uncertainties. An event driven sliding mode control is presented in this work to regulate the temperature and concentration very close to the equilibrium points of CSTR. The control is executed only when a predefined condition gets violated and hence the controller is relaxed when the system is operating under tolerable limits in terms of closed loop performance. A novel dynamic event triggering rule is presented to maintain desired performance with minimum computational cost. The inter event execution time is shown to be lower bounded by a finite positive quantity to exclude Zeno behavior. Sliding mode control (SMC) combined with event triggering scheme retains the inherent robustness of traditional SMC and aids in reducing computational load on the controller involved. Simulation results validate the efficiency of the proposed controller.

eess.SY↗

arXiv2018

A systematic process for evaluating structured perfect Bayesian equilibria in dynamic games with asymmetric information

We consider finite-horizon and infinite-horizon versions of a dynamic game with $N$ selfish players who observe their types privately and take actions that are publicly observed. Players' types evolve as conditionally independent Markov processes, conditioned on their current actions. Their actions and types jointly determine their instantaneous rewards. In dynamic games with asymmetric information, a widely used concept of equilibrium is perfect Bayesian equilibrium (PBE), which consists of a strategy and belief pair that simultaneously satisfy sequential rationality and belief consistency. In general, there does not exist a universal algorithm that decouples the interdependence of strategies and beliefs over time in calculating PBE. In this paper, for the finite-horizon game with independent types we develop a two-step backward-forward recursive algorithm that sequentially decomposes the problem (w.r.t. time) to obtain a subset of PBEs, which we refer to as structured Bayesian perfect equilibria (SPBE). In such equilibria, a player's strategy depends on its history only through a common public belief and its current private type. The backward recursive part of this algorithm defines an equilibrium generating function. Each period in the backward recursion involves solving a fixed-point equation on the space of probability simplexes for every possible belief on types. Using this function, equilibrium strategies and beliefs are generated through a forward recursion. We then extend this methodology to the infinite-horizon model, where we propose a time-invariant single-shot fixed-point equation, which in conjunction with a forward recursive step, generates the SPBE. Sufficient conditions for the existence of SPBE are provided. With our proposed method, we find equilibria that exhibit signaling behavior. This is illustrated with the help of a concrete public goods example.

math.OC↗

arXiv2017

Consensus tracking in multi agent system with nonlinear and non identical dynamics via event driven sliding modes

In this work, leader follower consensus objective has been addressed with the synthesis of an event based controller utilizing sliding mode robust control. The schema has been partitioned into two parts viz. finite time consensus problem and event triggered control mechanism. A nonlinear multi agent system with non identical dynamics has been put forward to illustrate the robust capabilities of the proposed control. The first part incorporates matching of states of the followers with those of the leader via consensus tracking algorithm. In the subsequent part, an event triggered rule is devised to save computational power and restrict periodic updating of the controller involved while ensuring desired closed loop performance of the system. Switching of the event based controller is achieved via sliding mode control. Advantage of using switched controller like sliding mode is that it retains its inherent robustness as well as event triggering approach aids in saving energy expenditure. Efficacy of the proposed scheme is confirmed via numerical simulations.

eess.SY↗

arXiv2017

Cooperative control of multi-agent systems to locate source of an odor

This work targets the problem of odor source localization by multi-agent systems. A hierarchical cooperative control has been put forward to solve the problem of locating source of an odor by driving the agents in consensus when at least one agent obtains information about location of the source. Synthesis of the proposed controller has been carried out in a hierarchical manner of group decision making, path planning and control. Decision making utilizes information of the agents using conventional Particle Swarm Algorithm and information of the movement of filaments to predict the location of the odor source. The predicted source location in the decision level is then utilized to map a trajectory and pass that information to the control level. The distributed control layer uses sliding mode controllers known for their inherent robustness and the ability to reject matched disturbances completely. Two cases of movement of agents towards the source, i.e., under consensus and formation have been discussed herein. Finally, numerical simulations demonstrate the efficacy of the proposed hierarchical distributed control.

eess.SY↗

arXiv2017

Distributed Mechanism Design with Learning Guarantees

Mechanism design for fully strategic agents commonly assumes broadcast nature of communication between agents of the system. Moreover, for mechanism design, the stability of Nash equilibrium (NE) is demonstrated by showing convergence of specific pre-designed learning dynamics, rather than for a class of learning dynamics. In this paper we consider two common resource allocation problems: sharing $ K $ infinitely divisible resources among strategic agents for their private consumption (private goods), and determining the level for an infinitely divisible public good with $ P $ features, that is shared between strategic agents. For both cases, we present a distributed mechanism for a set of agents who communicate through a given network. In a distributed mechanism, agents' messages are not broadcast to all other agents as in the standard mechanism design framework, but are exchanged only in the local neighborhood of each agent. The presented mechanisms produce a unique NE and fully implement the social welfare maximizing allocation. In addition, the mechanisms are budget-balanced at NE. It is also shown that the mechanisms induce a game with contractive best-response, leading to guaranteed convergence for all learning dynamics within the Adaptive Best-Response dynamics class, including dynamics such as Cournot best-response, $ k- $period best-response and Fictitious Play. We also present a numerically study of convergence under repeated play, for various communication graphs and learning dynamics.

cs.GT↗

arXiv2017

Mechanism design for resource allocation in networks with intergroup competition and intragroup sharing

We consider a network where strategic agents, who are contesting for allocation of resources, are divided into fixed groups. The network control protocol is such that within each group agents get to share the resource and across groups they contest for it. A prototypical example is the allocation of data rate on a network with multicast/multirate architecture. Compared to the unicast architecture (which is a special case), the multicast/multirate architecture can result in substantial bandwidth savings. However, design of a market mechanism in such a scenario requires dealing with both private and public good problems as opposed to just private goods in unicast. The mechanism proposed in this work ensures that social welfare maximizing allocation on such a network is realized at all Nash equilibria (NE) i.e., full implementation in NE. In addition it is individually rational, i.e., agents have an incentive to participate in the mechanism. The mechanism, which is constructed in a quasi-systematic way starting from the dual of the centralized problem, has a number of useful properties. Specifically, due to a novel allocation scheme, namely "radial projection", the proposed mechanism results in feasible allocation even off equilibrium. This is a practical necessity for any realistic mechanism since agents have to "learn" the NE through a dynamic process. Finally, it is shown how strong budget balance at equilibrium can be achieved with a minimal increase in message space as an add-on to a weakly budget balanced mechanism.

cs.GT↗

arXiv2016

Structured Perfect Bayesian Equilibrium in Infinite Horizon Dynamic Games with Asymmetric Information

In dynamic games with asymmetric information structure, the widely used concept of equilibrium is perfect Bayesian equilibrium (PBE). This is expressed as a strategy and belief pair that simultaneously satisfy sequential rationality and belief consistency. Unlike symmetric information dynamic games, where subgame perfect equilibrium (SPE) is the natural equilibrium concept, to date there does not exist a universal algorithm that decouples the interdependence of strategies and beliefs over time in calculating PBE. In this paper we find a subset of PBE for an infinite horizon discounted reward asymmetric information dynamic game. We refer to it as Structured PBE or SPBE; in SPBE, any agents' strategy depends on the public history only through a common public belief and on private history only through the respective agents' latest private information (his private type). The public belief acts as a summary of all the relevant past information and it's dimension does not increase with time. The motivation for this comes the common information approach proposed in Nayyar et al. (2013) for solving decentralized team (non-strategic) resource allocation problems with asymmetric information. We calculate SPBE by solving a single-shot fixed-point equation and a corresponding forward recursive algorithm. We demonstrate our methodology by means of a public goods example.

cs.GT↗

arXiv2015

Mechanism Design for Fair Allocation

Mechanism design for a social utility being the sum of agents' utilities (SoU) is a well-studied problem. There are, however, a number of problems of theoretical and practical interest where a designer may have a different objective than maximization of the SoU. One motivation for this is the desire for more equitable allocation of resources among agents. A second, more subtle, motivation is the fact that a fairer allocation indirectly implies less variation in taxes which can be desirable in a situation where (implicit) individual agent budgetary constraints make payment of large taxes unrealistic. In this paper we study a family of social utilities that provide fair allocation (with SoU being subsumed as an extreme case) and derive conditions under which Bayesian and Dominant strategy implementation is possible. Furthermore, it is shown how a simple modification of the above mechanism can guarantee full Bayesian implementation. Through a numerical example it is shown that the proposed method can result in significant gains both in allocation fairness and tax reduction.

cs.GT↗

arXiv2015

A General Mechanism Design Methodology for Social Utility Maximization with Linear Constraints

Social utility maximization refers to the process of allocating resources in such a way that the sum of agents' utilities is maximized under the system constraints. Such allocation arises in several problems in the general area of communications, including unicast (and multicast multi-rate) service on the Internet, as well as in applications with (local) public goods, such as power allocation in wireless networks, spectrum allocation, etc. Mechanisms that implement such allocations in Nash equilibrium have also been studied but either they do not possess full implementation property, or are given in a case-by-case fashion, thus obscuring fundamental understanding of these problems. In this paper we propose a unified methodology for creating mechanisms that fully implement, in Nash equilibria, social utility maximizing functions arising in various contexts where the constraints are convex. The construction of the mechanism is done in a systematic way by considering the dual optimization problem. In addition to the required properties of efficiency and individual rationality that such mechanisms ought to satisfy, three additional design goals are the focus of this paper: a) the size of the message space scaling linearly with the number of agents (even if agents' types are entire valuation functions), b) allocation being feasible on and off equilibrium, and c) strong budget balance at equilibrium and also off equilibrium whenever demand is feasible.

cs.GT↗

arXiv2014

Online Linear Optimization via Smoothing

We present a new optimization-theoretic approach to analyzing Follow-the-Leader style algorithms, particularly in the setting where perturbations are used as a tool for regularization. We show that adding a strongly convex penalty function to the decision rule and adding stochastic perturbations to data correspond to deterministic and stochastic smoothing operations, respectively. We establish an equivalence between "Follow the Regularized Leader" and "Follow the Perturbed Leader" up to the smoothness properties. This intuition leads to a new generic analysis framework that recovers and improves the previous known regret bounds of the class of algorithms commonly known as Follow the Perturbed Leader.

cs.LG↗

arXiv2014

Generalized Proportional Allocation Mechanism Design for Unicast Service on the Internet

In this report we construct two mechanisms that fully implement social welfare maximising allocation in Nash equilibria for the case of a single infinitely divisible good subject to multiple inequality constraints. The first mechanism achieves weak budget balance, while the second is an extension of the first, and achieves strong budget balance. One important application of this mechanism is unicast service on the Internet where a network operator wishes to allocate rates among strategic users in such a way that maximise overall user satisfaction while respecting capacity constraints on every link in the network. The emphasis of this work is on full implementation, which means that all Nash equilibria of the induced game result in the optimal allocations of the centralized allocation problem.

cs.GT↗

arXiv2011

Optimal Power Allocation for Renewable Energy Source

Battery powered transmitters face energy constraint, replenishing their energy by a renewable energy source (like solar or wind power) can lead to longer lifetime. We consider here the problem of finding the optimal power allocation under random channel conditions for a wireless transmitter, such that rate of information transfer is maximized. Here a rechargeable battery, which is periodically charged by renewable source, is used to power the transmitter. All of above is formulated as a Markov Decision Process. Structural properties like the monotonicity of the optimal value and policy derived in this paper will be of vital importance in understanding the kind of algorithms and approximations needed in real-life scenarios. The effect of curse of dimensionality which is prevalent in Dynamic programming problems can thus be reduced. We show our results under the most general of assumptions.

eess.SY↗