arXiv subjects

Diwas Paudel

Publications and source records attributed to Diwas Paudel.

3 recordsLinked to original sources

arXiv2024

Tacit algorithmic collusion in deep reinforcement learning guided price competition: A study using EV charge pricing game

Players in pricing games with complex structures are increasingly adopting artificial intelligence (AI) aided learning algorithms to make pricing decisions for maximizing profits. This is raising concern for the antitrust agencies as the practice of using AI may promote tacit algorithmic collusion among otherwise independent players. Recent studies of games in canonical forms have shown contrasting claims ranging from none to a high level of tacit collusion among AI-guided players. In this paper, we examine the concern for tacit collusion by considering a practical game where EV charging hubs compete by dynamically varying their prices. Such a game is likely to be commonplace in the near future as EV adoption grows in all sectors of transportation. The hubs source power from the day-ahead (DA) and real-time (RT) electricity markets as well as from in-house battery storage systems. Their goal is to maximize profits via pricing and efficiently managing the cost of power usage. To aid our examination, we develop a two-step data-driven methodology. The first step obtains the DA commitment by solving a stochastic model. The second step generates the pricing strategies by solving a competitive Markov decision process model using a multi-agent deep reinforcement learning (MADRL) framework. We evaluate the resulting pricing strategies using an index for the level of tacit algorithmic collusion. An index value of zero indicates no collusion (perfect competition) and one indicates full collusion (monopolistic behavior). Results from our numerical case study yield collusion index values between 0.14 and 0.45, suggesting a low to moderate level of collusion.

cs.LG↗

arXiv2024

Joint operation of a fast-charging EV hub with a stand-alone independent battery storage system under fairness considerations

The need for larger-scale fast-charging electric vehicle (EV) hubs is on the rise due to the growth in EV adoption. Another area of power infrastructure growth is the proliferation of independently operated stand-alone battery storage systems (BSS), which is fueled by improvements and cost reductions in battery technology. Many possible uses of the stand-alone BSS are being explored including participation in the energy and ancillary markets, load balancing for renewable generations, and supporting large-scale load-consuming entities like hospitals. In this paper, we study a novel usage of the stand-alone BSS whereby in addition to participating in the electricity reserve market, it allows an EV hub to use a part of its storage capacity, when profitable. The hub uses the BSS storage capacity for arbitrage consequently reducing its operating cost. We formulate this joint operation as a bi-objective optimization model. We then reformulate it into a second-order cone Nash bargaining problem, the solution of which guarantees fairness to both the hub and the BSS. A sample numerical case study is formulated using actual prices of electricity and simulated data for the reserve market and EV charging demand. The Nash bargaining solution shows that both participants can benefit from the joint operation.

eess.SY↗

arXiv2023

Deep PackGen: A Deep Reinforcement Learning Framework for Adversarial Network Packet Generation

Recent advancements in artificial intelligence (AI) and machine learning (ML) algorithms, coupled with the availability of faster computing infrastructure, have enhanced the security posture of cybersecurity operations centers (defenders) through the development of ML-aided network intrusion detection systems (NIDS). Concurrently, the abilities of adversaries to evade security have also increased with the support of AI/ML models. Therefore, defenders need to proactively prepare for evasion attacks that exploit the detection mechanisms of NIDS. Recent studies have found that the perturbation of flow-based and packet-based features can deceive ML models, but these approaches have limitations. Perturbations made to the flow-based features are difficult to reverse-engineer, while samples generated with perturbations to the packet-based features are not playable. Our methodological framework, Deep PackGen, employs deep reinforcement learning to generate adversarial packets and aims to overcome the limitations of approaches in the literature. By taking raw malicious network packets as inputs and systematically making perturbations on them, Deep PackGen camouflages them as benign packets while still maintaining their functionality. In our experiments, using publicly available data, Deep PackGen achieved an average adversarial success rate of 66.4\% against various ML models and across different attack types. Our investigation also revealed that more than 45\% of the successful adversarial samples were out-of-distribution packets that evaded the decision boundaries of the classifiers. The knowledge gained from our study on the adversary's ability to make specific evasive perturbations to different types of malicious packets can help defenders enhance the robustness of their NIDS against evolving adversarial attacks.

cs.CR↗