arXiv subjects

Morteza Hashemi

Publications and source records attributed to Morteza Hashemi.

At least 37 records · Page 2Linked to original sources

arXiv2024

Model-free Resilient Controller Design based on Incentive Feedback Stackelberg Game and Q-learning

In the swift evolution of Cyber-Physical Systems (CPSs) within intelligent environments, especially in the industrial domain shaped by Industry 4.0, the surge in development brings forth unprecedented security challenges. This paper explores the intricate security issues of Industrial CPSs (ICPSs), with a specific focus on the unique threats presented by intelligent attackers capable of directly compromising the controller, thereby posing a direct risk to physical security. Within the framework of hierarchical control and incentive feedback Stackelberg game, we design a resilient leading controller (leader) that is adaptive to a compromised following controller (follower) such that the compromised follower acts cooperatively with the leader, aligning its strategies with the leader's objective to achieve a team-optimal solution. First, we provide sufficient conditions for the existence of an incentive Stackelberg solution when system dynamics are known. Then, we propose a Q-learning-based Approximate Dynamic Programming (ADP) approach, and corresponding algorithms for the online resolution of the incentive Stackelberg solution without requiring prior knowledge of system dynamics. Last but not least, we prove the convergence of our approach to the optimum.

eess.SY↗

arXiv2024

Semantic-Aware and Goal-Oriented Communications for Object Detection in Wireless End-to-End Image Transmission

Semantic communication is focused on optimizing the exchange of information by transmitting only the most relevant data required to convey the intended message to the receiver and achieve the desired communication goal. For example, if we consider images as the information and the goal of the communication is object detection at the receiver side, the semantic of information would be the objects in each image. Therefore, by only transferring the semantics of images we can achieve the communication goal. In this paper, we propose a design framework for implementing semantic-aware and goal-oriented communication of images. To achieve this, we first define the baseline problem as a set of mathematical problems that can be optimized to improve the efficiency and effectiveness of the communication system. We consider two scenarios in which either the data rate or the error at the receiver is the limiting constraint. Our proposed system model and solution is inspired by the concept of auto-encoders, where the encoder and the decoder are respectively implemented at the transmitter and receiver to extract semantic information for specific object detection goals. Our numerical results validate the proposed design framework to achieve low error or near-optimal in a goal-oriented communication system while reducing the amount of data transfers.

cs.IT↗

arXiv2024

Interference-Aware Queuing Analysis for Distributed Transmission Control in UAV Networks

In this paper, we investigate the problem of distributed transmission control for unmanned aerial vehicles (UAVs) operating in unlicensed spectrum bands. We develop a rigorous interference-aware queuing analysis framework that jointly considers two inter-dependent factors: (i) limited-size queues with delay-constrained packet arrival, and (ii) in-band interference introduced by other ground/aerial users. We aim to optimize the expected throughput by jointly analyzing these factors. In the queuing analysis, we explore two packet loss probabilities including, buffer overflow model and time threshold model. For interference analysis, we investigate the outage probability and packet losses due to low signal-to-interference-plus-noise ratio (SINR). We introduce two algorithms namely, Interference-Aware Transmission Control (IA-TC), and Interference-Aware Distributed Transmission Control (IA-DTC). These algorithms maximize the expected throughput by adjusting transmission policies to balance the trade-offs between packet drop from queues vs. transmission errors due to low SINRs. We implement the proposed algorithms and demonstrate that the optimal transmission policy under various scenarios is found.

cs.IT↗

arXiv2023

Exploring the Interplay of Interference and Queues in Unlicensed Spectrum Bands for UAV Networks

In this paper, we present an analytical framework to explore the interplay of signal interference and transmission queue management, and their impacts on the performance of unmanned aerial vehicles (UAVs) when operating in the unlicensed spectrum bands. In particular, we develop a comprehensive framework to investigate the impact of other interference links on the UAV as it communicates with the ground users. To this end, we provide closed-form expressions for packet drop probabilities in the queue due to buffer overflow or large queuing delay, which are expressed in terms of a transmission policy as a function of the channel fading threshold $β$. The overall packet loss caused either by interference signals or queuing packet drop is obtained, which, in turn, yields in obtaining the expected throughput performance. Through extensive numerical results, we investigate the impact of the channel fading threshold $β$, which plays an important role in balancing the trade-offs between packet loss due to queue drop or transmission error due to large interference levels.

cs.IT↗

arXiv2023

Efficient Cluster Selection for Personalized Federated Learning: A Multi-Armed Bandit Approach

Federated learning (FL) offers a decentralized training approach for machine learning models, prioritizing data privacy. However, the inherent heterogeneity in FL networks, arising from variations in data distribution, size, and device capabilities, poses challenges in user federation. Recognizing this, Personalized Federated Learning (PFL) emphasizes tailoring learning processes to individual data profiles. In this paper, we address the complexity of clustering users in PFL, especially in dynamic networks, by introducing a dynamic Upper Confidence Bound (dUCB) algorithm inspired by the multi-armed bandit (MAB) approach. The dUCB algorithm ensures that new users can effectively find the best cluster for their data distribution by balancing exploration and exploitation. The performance of our algorithm is evaluated in various cases, showing its effectiveness in handling dynamic federated learning scenarios.

cs.LG↗

arXiv2023

An Efficient Distributed Multi-Agent Reinforcement Learning for EV Charging Network Control

The increasing trend in adopting electric vehicles (EVs) will significantly impact the residential electricity demand, which results in an increased risk of transformer overload in the distribution grid. To mitigate such risks, there are urgent needs to develop effective EV charging controllers. Currently, the majority of the EV charge controllers are based on a centralized approach for managing individual EVs or a group of EVs. In this paper, we introduce a decentralized Multi-agent Reinforcement Learning (MARL) charging framework that prioritizes the preservation of privacy for EV owners. We employ the Centralized Training Decentralized Execution-Deep Deterministic Policy Gradient (CTDE-DDPG) scheme, which provides valuable information to users during training while maintaining privacy during execution. Our results demonstrate that the CTDE framework improves the performance of the charging network by reducing the network costs. Moreover, we show that the Peak-to-Average Ratio (PAR) of the total demand is reduced, which, in turn, reduces the risk of transformer overload during the peak hours.

cs.MA↗

arXiv2023

Energy-Efficient Deadline-Aware Edge Computing: Bandit Learning with Partial Observations in Multi-Channel Systems

In this paper, we consider a task offloading problem in a multi-access edge computing (MEC) network, in which edge users can either use their local processing unit to compute their tasks or offload their tasks to a nearby edge server through multiple communication channels each with different characteristics. The main objective is to maximize the energy efficiency of the edge users while meeting computing tasks deadlines. In the multi-user multi-channel offloading scenario, users are distributed with partial observations of the system states. We formulate this problem as a stochastic optimization problem and leverage \emph{contextual neural multi-armed bandit} models to develop an energy-efficient deadline-aware solution, dubbed E2DA. The proposed E2DA framework only relies on partial state information (i.e., computation task features) to make offloading decisions. Through extensive numerical analysis, we demonstrate that the E2DA algorithm can efficiently learn an offloading policy and achieve close-to-optimal performance in comparison with several baseline policies that optimize energy consumption and/or response time. Furthermore, we provide a comprehensive set of results on the MEC system performance for various applications such as augmented reality (AR) and virtual reality (VR).

cs.NI↗

arXiv2023

Collaborative Wideband Spectrum Sensing and Scheduling for Networked UAVs in UTM Systems

In this paper, we propose a data-driven framework for collaborative wideband spectrum sensing and scheduling for networked unmanned aerial vehicles (UAVs), which act as the secondary users to opportunistically utilize detected spectrum holes. To this end, we propose a multi-class classification problem for wideband spectrum sensing to detect vacant spectrum spots based on collected I/Q samples. To enhance the accuracy of the spectrum sensing module, the outputs from the multi-class classification by each individual UAV are fused at a server in the unmanned aircraft system traffic management (UTM) ecosystem. In the spectrum scheduling phase, we leverage reinforcement learning (RL) solutions to dynamically allocate the detected spectrum holes to the secondary users (i.e., UAVs). To evaluate the proposed methods, we establish a comprehensive simulation framework that generates a near-realistic synthetic dataset using MATLAB LTE toolbox by incorporating base-station~(BS) locations in a chosen area of interest, performing ray-tracing, and emulating the primary users channel usage in terms of I/Q samples. This evaluation methodology provides a flexible framework to generate large spectrum datasets that could be used for developing ML/AI-based spectrum management solutions for aerial devices.

eess.SP↗

arXiv2023

Combating Uncertainties in Wind and Distributed PV Energy Sources Using Integrated Reinforcement Learning and Time-Series Forecasting

Renewable energy sources, such as wind and solar power, are increasingly being integrated into smart grid systems. However, when compared to traditional energy resources, the unpredictability of renewable energy generation poses significant challenges for both electricity providers and utility companies. Furthermore, the large-scale integration of distributed energy resources (such as PV systems) creates new challenges for energy management in microgrids. To tackle these issues, we propose a novel framework with two objectives: (i) combating uncertainty of renewable energy in smart grid by leveraging time-series forecasting with Long-Short Term Memory (LSTM) solutions, and (ii) establishing distributed and dynamic decision-making framework with multi-agent reinforcement learning using Deep Deterministic Policy Gradient (DDPG) algorithm. The proposed framework considers both objectives concurrently to fully integrate them, while considering both wholesale and retail markets, thereby enabling efficient energy management in the presence of uncertain and distributed renewable energy sources. Through extensive numerical simulations, we demonstrate that the proposed solution significantly improves the profit of load serving entities (LSE) by providing a more accurate wind generation forecast. Furthermore, our results demonstrate that households with PV and battery installations can increase their profits by using intelligent battery charge/discharge actions determined by the DDPG agents.

eess.SY↗

arXiv2022

Minimum Overhead Beamforming and Resource Allocation in D2D Edge Networks

Device-to-device (D2D) communications is expected to be a critical enabler of distributed computing in edge networks at scale. A key challenge in providing this capability is the requirement for judicious management of the heterogeneous communication and computation resources that exist at the edge to meet processing needs. In this paper, we develop an optimization methodology that considers the network topology jointly with device and network resource allocation to minimize total D2D overhead, which we quantify in terms of time and energy required for task processing. Variables in our model include task assignment, CPU allocation, subchannel selection, and beamforming design for multiple-input multiple-output (MIMO) wireless devices. We propose two methods to solve the resulting non-convex mixed integer program: semi-exhaustive search optimization, which represents a "best-effort" at obtaining the optimal solution, and efficient alternate optimization, which is more computationally efficient. As a component of these two methods, we develop a novel coordinated beamforming algorithm which we show obtains the optimal beamformer for a common receiver characteristic. Through numerical experiments, we find that our methodology yields substantial improvements in network overhead compared with local computation and partially optimized methods, which validates our joint optimization approach. Further, we find that the efficient alternate optimization scales well with the number of nodes, and thus can be a practical solution for D2D computing in large networks.

eess.SP↗

arXiv2022

QoE-Centric Multi-User mmWave Scheduling: A Beam Alignment and Buffer Predictive Approach

In this paper, we consider the multi-user scheduling problem in millimeter wave (mmWave) video streaming networks, which comprise a streaming server and several users, each requesting a video stream with a different resolution. The main objective is to optimize the long-term average quality of experience (QoE) for all users. We tackle this problem by considering the physical layer characteristics of the mmWave network, including the beam alignment overhead due to pencil-beams. To develop an efficient scheduling policy, we leverage the contextual multi-armed bandit (MAB) models to propose a beam alignment overhead and buffer predictive streaming solution, dubbed B2P-Stream. The proposed B2P-Stream algorithm optimally balances the trade-off between the overhead and users' buffer levels and improves the QoE by reducing the beam alignment overhead for users of higher resolutions. We also provide a theoretical guarantee for our proposed method and prove that it guarantees a sub-linear regret bound. Finally, we examine our proposed framework through extensive simulations. We provide a detailed comparison of the B2P-Stream against uniformly random and Round-robin (RR) policies and show that it outperforms both of them in providing a better QoE and fairness. We also analyze the scalability and robustness of the B2P-Stream algorithm with different network configurations.

cs.NI↗

arXiv2020

Multi-hop Routing with Proactive Route Refinement for 60 GHz Millimeter-Wave Networks

Fundamental requirements of mmWave systems are peak data rates of multiple Gbps and latencies of the order of at most a few milliseconds. However, highly directional mmWave links are susceptible to frequent link failures under stress conditions such as mobility and human blockage. Under these conditions, multi-hop routing can achieve reliable and robust performance. In this paper, we consider multi-hop millimeter wave (mmWave) wireless systems and propose proactive route refinement schemes that are particularly important under dynamic scenarios. First, we consider the AODV-type protocols and propose a cross-layer approach that integrates sectorized communication at the MAC layer with on-demand multi-hop routing at the network layer. Next, we consider Backpressure routing protocol, and enhance this protocol with periodic HELLO status messages. System-level simulation results based on the IEEE 802.11ad standard are provided that confirm the benefits of proactive route refinement for the ADOV-type and Backpressure routing protocols.

cs.NI↗

arXiv2020

Demand Responsive Dynamic Pricing Framework for Prosumer Dominated Microgrids using Multiagent Reinforcement Learning

Demand Response (DR) has a widely recognized potential for improving grid stability and reliability while reducing customers energy bills. However, the conventional DR techniques come with several shortcomings, such as inability to handle operational uncertainties and incurring customer disutility, impeding their wide spread adoption in real-world applications. This paper proposes a new multiagent Reinforcement Learning (RL) based decision-making environment for implementing a Real-Time Pricing (RTP) DR technique in a prosumer dominated microgrid. The proposed technique addresses several shortcomings common to traditional DR methods and provides significant economic benefits to the grid operator and prosumers. To show its better efficacy, the proposed DR method is compared to a baseline traditional operation scenario in a small-scale microgrid system. Finally, investigations on the use of prosumers energy storage capacity in this microgrid highlight the advantages of the proposed method in establishing a balanced market setup.

eess.SY↗

arXiv2020

A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids

This paper presents a Reinforcement Learning (RL) based energy market for a prosumer dominated microgrid. The proposed market model facilitates a real-time and demanddependent dynamic pricing environment, which reduces grid costs and improves the economic benefits for prosumers. Furthermore, this market model enables the grid operator to leverage prosumers storage capacity as a dispatchable asset for grid support applications. Simulation results based on the Deep QNetwork (DQN) framework demonstrate significant improvements of the 24-hour accumulative profit for both prosumers and the grid operator, as well as major reductions in grid reserve power utilization.

eess.SY↗

arXiv2020

On the Benefits of Multi-hop Communication for Indoor 60 GHz Wireless Networks

The spectrum-rich millimeter wave (mmWave) frequencies have the potential to alleviate the spectrum crunch that the wireless and cellular operators are already experiencing. However, compared with traditional wireless communication in the sub-6 GHz bands, due to small wavelengths most objects such as human body, cause significant additional path losses (up to 20dB), which can entirely break the mmWave link. Also, mmwave links suffer from limited range of communication. In this paper, we resort to network layer solutions to demonstrate the benefits of multi-hop routing in mitigating the blockage issue and extending communication range in mmWave band. To this end, we develop a hop-by-hop multi-path routing protocol that finds one primary and one backup next-hop per destination in order to guarantee reliable and robust communication under extreme stress conditions. System-level simulations based on the IEEE802.11ad specifications demonstrate that the proposed routing protocol provides a reliable end-to-end throughput performance, while satisfying the latency requirements.

cs.NI↗

arXiv2020

Delay-Efficient and Reliable Data Relaying in Ultra Dense Networks using Rateless Codes

We investigate the problem of delay-efficient and reliable data delivery in ultra-dense networks (UDNs) that constitute macro base stations (MBSs), small base stations (SBSs), and mobile users. Considering a two-hop data delivery system, we propose a partial decode-and-forward (PDF) relaying strategy together with a simple and intuitive amicable encoding scheme for rateless codes to significantly improve user experience in terms of end-to-end delay. Simulation results verify that our amicable encoding scheme is efficient in improving the intermediate performance of rateless codes. It also verifies that our proposed PDF significantly improves the performance of the decode-and-forward (DF) strategy, and that PDF is much more robust against channel degradation. Overall, the proposed strategy and encoding scheme are efficient towards delay-sensitive data delivery in the UDN scenarios.

cs.IT↗

arXiv2020

Joint Optimization of Signal Design and Resource Allocation in Wireless D2D Edge Computing

In this paper, we study the distributed computational capabilities of device-to-device (D2D) networks. A key characteristic of D2D networks is that their topologies are reconfigurable to cope with network demands. For distributed computing, resource management is challenging due to limited network and communication resources, leading to inter-channel interference. To overcome this, recent research has addressed the problems of wireless scheduling, subchannel allocation, power allocation, and multiple-input multiple-output (MIMO) signal design, but has not considered them jointly. In this paper, unlike previous mobile edge computing (MEC) approaches, we propose a joint optimization of wireless MIMO signal design and network resource allocation to maximize energy efficiency. Given that the resulting problem is a non-convex mixed integer program (MIP) which is prohibitive to solve at scale, we decompose its solution into two parts: (i) a resource allocation subproblem, which optimizes the link selection and subchannel allocations, and (ii) MIMO signal design subproblem, which optimizes the transmit beamformer, transmit power, and receive combiner. Simulation results using wireless edge topologies show that our method yields substantial improvements in energy efficiency compared with cases of no offloading and partially optimized methods and that the efficiency scales well with the size of the network.

cs.NI↗

arXiv2019

Energy-Aware Multi-Server Mobile Edge Computing: A Deep Reinforcement Learning Approach

We investigate the problem of computation offloading in a mobile edge computing architecture, where multiple energy-constrained users compete to offload their computational tasks to multiple servers through a shared wireless medium. We propose a multi-agent deep reinforcement learning algorithm, where each server is equipped with an agent, observing the status of its associated users and selecting the best user for offloading at each step. We consider computation time (i.e., task completion time) and system lifetime as two key performance indicators, and we numerically demonstrate that our approach outperforms baseline algorithms in terms of the trade-off between computation time and system lifetime.

cs.IT↗