arXiv subjects

M. Cenk Gursoy

Publications and source records attributed to M. Cenk Gursoy.

At least 55 records · Page 3Linked to original sources

arXiv2020

Energy Harvesting in Unmanned Aerial Vehicle Networks with 3D Antenna Radiation Patterns

In this paper, an analytical framework is provided to analyze the energy coverage performance of unmanned aerial vehicle (UAV) energy harvesting networks with clustered user equipments (UEs). Locations of UEs are modeled as a Poisson Cluster Process (PCP), and UAVs are assumed to be located at a certain height above the center of user clusters. Hence, user-centric UAV deployments are addressed. Two different models are considered for the line-of-sight (LOS) probability function to compare their effects on the network performance. Moreover, antennas with doughnut-shaped radiation patterns are employed at both UAVs and UEs, and the impact of practical 3D antenna radiation patterns on the network performance is also investigated. Initially, the path loss of each tier is statistically described by deriving the complementary cumulative distribution function and probability density function. Following this, association probabilities with each tier are determined, and energy coverage probability of the UAV network is characterized in terms of key system and network parameters for UAV deployments both at a single height level and more generally at multiple heights. Through numerical results, we have shown that cluster size and UAV height play crucial roles on the energy coverage performance. Furthermore, energy coverage probability is significantly affected by the antenna orientation and number of UAVs in the network.

cs.IT↗

arXiv2020

Adversarial jamming attacks and defense strategies via adaptive deep reinforcement learning

As the applications of deep reinforcement learning (DRL) in wireless communications grow, sensitivity of DRL based wireless communication strategies against adversarial attacks has started to draw increasing attention. In order to address such sensitivity and alleviate the resulting security concerns, we in this paper consider a victim user that performs DRL-based dynamic channel access, and an attacker that executes DRLbased jamming attacks to disrupt the victim. Hence, both the victim and attacker are DRL agents and can interact with each other, retrain their models, and adapt to opponents' policies. In this setting, we initially develop an adversarial jamming attack policy that aims at minimizing the accuracy of victim's decision making on dynamic channel access. Subsequently, we devise defense strategies against such an attacker, and propose three defense strategies, namely diversified defense with proportional-integral-derivative (PID) control, diversified defense with an imitation attacker, and defense via orthogonal policies. We design these strategies to maximize the attacked victim's accuracy and evaluate their performances.

eess.SP↗

arXiv2020

Anomaly Detection Under Controlled Sensing Using Actor-Critic Reinforcement Learning

We consider the problem of detecting anomalies among a given set of processes using their noisy binary sensor measurements. The noiseless sensor measurement corresponding to a normal process is 0, and the measurement is 1 if the process is anomalous. The decision-making algorithm is assumed to have no knowledge of the number of anomalous processes. The algorithm is allowed to choose a subset of the sensors at each time instant until the confidence level on the decision exceeds the desired value. Our objective is to design a sequential sensor selection policy that dynamically determines which processes to observe at each time and when to terminate the detection algorithm. The selection policy is designed such that the anomalous processes are detected with the desired confidence level while incurring minimum cost which comprises the delay in detection and the cost of sensing. We cast this problem as a sequential hypothesis testing problem within the framework of Markov decision processes, and solve it using the actor-critic deep reinforcement learning algorithm. This deep neural network-based algorithm offers a low complexity solution with good detection accuracy. We also study the effect of statistical dependence between the processes on the algorithm performance. Through numerical experiments, we show that our algorithm is able to adapt to any unknown statistical dependence pattern of the processes.

eess.SP↗

arXiv2020

Age-Energy Tradeoff in Fading Channels with Packet-Based Transmissions

The optimal transmission strategy to minimize the weighted combination of age of information (AoI) and total energy consumption is studied in this paper. It is assumed that the status update information is obtained and transmitted at fixed rate over a Rayleigh fading channel in a packet-based wireless communication system. A maximum transmission round on each packet is enforced to guarantee certain reliability of the update packets. Given fixed average transmission power, the age-energy tradeoff can be formulated as a constrained Markov decision process (CMDP) problem considering the sensing power consumption as well. Employing the Lagrangian relaxation, the CMDP problem is transformed into a Markov decision process (MDP) problem. An algorithm is proposed to obtain the optimal power allocation policy. Through simulation results, it is shown that both age and energy efficiency can be improved by the proposed optimal policy compared with two benchmark schemes. Also, age can be effectively reduced at the expense of higher energy cost, and more emphasis on energy consumption leads to higher average age at the same energy efficiency. Overall, the tradeoff between average age and energy efficiency is identified.

cs.IT↗

arXiv2020

Mission-Aware Spatio-Temporal Deep Learning Model for UAS Instantaneous Density Prediction

The number of daily sUAS operations in uncontrolled low altitude airspace is expected to reach into the millions in a few years. Therefore, UAS density prediction has become an emerging and challenging problem. In this paper, a deep learning-based UAS instantaneous density prediction model is presented. The model takes two types of data as input: 1) the historical density generated from the historical data, and 2) the future sUAS mission information. The architecture of our model contains four components: Historical Density Formulation module, UAS Mission Translation module, Mission Feature Extraction module, and Density Map Projection module. The training and testing data are generated by a python based simulator which is inspired by the multi-agent air traffic resource usage simulator (MATRUS) framework. The quality of prediction is measured by the correlation score and the Area Under the Receiver Operating Characteristics (AUROC) between the predicted value and simulated value. The experimental results demonstrate outstanding performance of the deep learning-based UAS density predictor. Compared to the baseline models, for simplified traffic scenario where no-fly zones and safe distance among sUASs are not considered, our model improves the prediction accuracy by more than 15.2% and its correlation score reaches 0.947. In a more realistic scenario, where the no-fly zone avoidance and the safe distance among sUASs are maintained using A* routing algorithm, our model can still achieve 0.823 correlation score. Meanwhile, the AUROC can reach 0.951 for the hot spot prediction.

cs.CV↗

arXiv2020

Simulation of Real-time Routing for UAS traffic Management with Communication and Airspace Safety Considerations

Small Unmanned Aircraft Systems (sUAS) will be an important component of the smart city and intelligent transportation environments of the near future. The demand for sUAS related applications, such as commercial delivery and land surveying, is expected to grow rapidly in next few years. In general, sUAS traffic routing and management functions are needed to coordinate the launching of sUAS from different launch sites and determine their trajectories to avoid conflict while considering several other constraints such as expected arrival time, minimum flight energy, and availability of communication resources. However, as the airborne sUAS density grows in a certain area, it is difficult to foresee the potential airspace and communications resource conflicts and make immediate decisions to avoid them. To address this challenge, we present a temporal and spatial routing algorithm and simulation platform for sUAS trajectory management in a high density urban area that plans sUAS movements in a spatial and temporal maze taking into account obstacles that are either static or dynamic in time. The routing allows the sUAS to avoid static no-fly areas (i.e. static obstacles) or other in-flight sUAS and areas that have congested communication resources (i.e. dynamic obstacles). The algorithm is evaluated using an agent-based simulation platform. The simulation results show that the proposed algorithm outperforms other route management algorithms in many areas, especially in processing speed and memory efficiency. Detailed comparisons are provided for the sUAS flight time, the overall throughput, conflict rate and communication resource utilization. The results demonstrate that our proposed algorithm can be used to address the airspace and communication resource utilization needs for a next generation smart city and smart transportation.

cs.MA↗

arXiv2020

Optimal Resource Allocation in Ground Wireless Networks Supporting Unmanned Aerial Vehicle Transmissions

We consider a fully-loaded ground wireless network supporting unmanned aerial vehicle (UAV) transmission services. To enable the overload transmissions to a ground user (GU) and a UAV, two transmission schemes are employed, namely non-orthogonal multiple access (NOMA) and relaying, depending on whether or not the GU and UAV are served simultaneously. Under the assumption of the system operating with infinite blocklength (IBL) codes, the IBL throughputs of both the GU and the UAV are derived under the two schemes. More importantly, we also consider the scenario in which data packets are transmitted via finite blocklength (FBL) codes, i.e., data transmission to both the UAV and the GU is performed under low-latency and high reliability constraints. In this setting, the FBL throughputs are characterized again considering the two schemes of NOMA and relaying. Following the IBL and FBL throughput characterizations, optimal resource allocation designs are subsequently proposed to maximize the UAV throughput while guaranteeing the throughput of the cellular user.Moreover, we prove that the relaying scheme is able to provide transmission service to the UAV while improving the GU's performance, and that the relaying scheme potentially offers a higher throughput to the UAV in the FBL regime than in the IBL regime. On the other hand, the NOMA scheme provides a higher UAV throughput (than relaying) by slightly sacrificing the GU's performance.

cs.IT↗

arXiv2019

Coverage Analysis for Energy-Harvesting UAV-assisted mmWave Cellular Networks

In this paper, we jointly consider the downlink simultaneous wireless information and power transfer (SWIPT) and uplink information transmission in unmanned aerial vehicle (UAV)-assisted millimeter wave (mmWave) cellular networks, in which the user equipment (UE) locations are modeled using Poisson cluster processes (e.g., Thomas cluster processes or Matérn cluster processes). Distinguishing features of mmWave communications, such as different path loss models for line-of-sight (LOS) and non-LOS (NLOS) links and directional transmissions are taken into account. In the downlink phase, the association probability, and energy coverages of different tier UAVs and ground base stations (GBSs) are investigated. Moreover, we define a successful transmission probability to jointly present the energy and signal-to-interference-plus-noise ratio (SINR) coverages and provide general expressions. In the uplink phase, we consider the scenario that each UAV receives information from its own cluster member UEs. We determine the Laplace transform of the interference components and characterize the uplink SINR coverage. In addition, we formulate the average uplink throughput, with the goal to identify the optimal time division multiplexing between the donwlink and uplink phases. Through numerical results we investigate the impact of key system parameters on the performance. We show that the network performance is improved when the cluster size becomes smaller. In addition, we analyze the optimal height of UAVs, optimal power splitting value and optimal time division multiplexing that maximizes the network performance.

eess.SP↗

arXiv2019

Deep Actor-Critic Reinforcement Learning for Anomaly Detection

Anomaly detection is widely applied in a variety of domains, involving for instance, smart home systems, network traffic monitoring, IoT applications and sensor networks. In this paper, we study deep reinforcement learning based active sequential testing for anomaly detection. We assume that there is an unknown number of abnormal processes at a time and the agent can only check with one sensor in each sampling step. To maximize the confidence level of the decision and minimize the stopping time concurrently, we propose a deep actor-critic reinforcement learning framework that can dynamically select the sensor based on the posterior probabilities. We provide simulation results for both the training phase and testing phase, and compare the proposed framework with the Chernoff test in terms of claim delay and loss.

cs.LG↗

arXiv2019

A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access

To make efficient use of limited spectral resources, we in this work propose a deep actor-critic reinforcement learning based framework for dynamic multichannel access. We consider both a single-user case and a scenario in which multiple users attempt to access channels simultaneously. We employ the proposed framework as a single agent in the single-user case, and extend it to a decentralized multi-agent framework in the multi-user scenario. In both cases, we develop algorithms for the actor-critic deep reinforcement learning and evaluate the proposed learning policies via experiments and numerical results. In the single-user model, in order to evaluate the performance of the proposed channel access policy and the framework's tolerance against uncertainty, we explore different channel switching patterns and different switching probabilities. In the case of multiple users, we analyze the probabilities of each user accessing channels with favorable channel conditions and the probability of collision. We also address a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons (in terms of both the average reward and time efficiency) between the proposed actor-critic deep reinforcement learning framework, Deep-Q network (DQN) based approach, random access, and the optimal policy when the channel dynamics are known.

cs.LG↗

arXiv2019

Age-Optimal Power Control for Status Update Systems with Packet Based Transmissions

This paper investigates the average age of information (AoI) minimization in status update systems in which the update packets are transmitted with fixed rate. It is assumed that the source avoids queue-induced delay by generating and sending a new status update packet in each time slot. Assuming that the transmit power can be adapted based on the number of successive NACKs (negative acknowledgements) received, a closed-form expression for the average AoI is derived considering transmissions over block fading channel models. An optimization problem for minimizing the average AoI under average power constraints is formulated and a stochastic optimization algorithm for obtaining a feasible solution is proposed. Numerical results on the proposed power control policy show that more power is preferred to be allocated to packets after a certain number of failure packets in the low-power regime and the proposed power control reduces the average AoI compared to the constant power policy. An alternative on-off power control policy is also proposed and shown to achieve satisfactory performance.

cs.IT↗

arXiv2019

RSS-Based Detection of Drones in the Presence of RF Interferers

Drones will have extensive use cases across various commercial, government, and military sectors, ranging from delivery of consumer goods to search and rescue operations. To maintain the safety and security of people and infrastructure, it becomes critically important to quickly and accurately detect non-cooperating drones. In this paper we formulate a received signal strength (RSS) based detector, leveraging the existing wireless infrastructures that might already be serving other devices. Thus the detector can detect the presence of a drone signal buried in radio frequency (RF) interference and thermal noise, in a mixed line-of-sight (LOS) and non-LOS (NLOS) environment. We develop analytical expressions for the probability of false alarm and the probability of detection of a drone, which quantify the impact of aggregate interference and air-to-ground (A2G) propagation characteristics on the detection performance of individual sensors. We also provide analytical expressions for the average network probability of detection, which capture the impact of sensor density on a network's detection coverage. Finally, we find the critical sensor density that maximizes the average network probability of detection for a given requirement of the probability of false alarm.

eess.SP↗

arXiv2019

Deep Multi-Agent Reinforcement Learning Based Cooperative Edge Caching in Wireless Networks

The growing demand on high-quality and low-latency multimedia services has led to much interest in edge caching techniques. Motivated by this, we in this paper consider edge caching at the base stations with unknown content popularity distributions. To solve the dynamic control problem of making caching decisions, we propose a deep actor-critic reinforcement learning based multi-agent framework with the aim to minimize the overall average transmission delay. To evaluate the proposed framework, we compare the learning-based performance with three other caching policies, namely least recently used (LRU), least frequently used (LFU), and first-in-first-out (FIFO) policies. Through simulation results, performance improvements of the proposed framework over these three caching algorithms have been identified and its superior ability to adapt to varying environments is demonstrated.

cs.IT↗

arXiv2019

Power Control for Wireless VBR Video Streaming: From Optimization to Reinforcement Learning

In this paper, we investigate the problem of power control for streaming variable bit rate (VBR) videos over wireless links. A system model involving a transmitter (e.g., a base station) that sends VBR video data to a receiver (e.g., a mobile user) equipped with a playout buffer is adopted, as used in dynamic adaptive streaming video applications. In this setting, we analyze power control policies considering the following two objectives: 1) the minimization of the transmit power consumption, and 2) the minimization of the transmission completion time of the communication session. In order to play the video without interruptions, the power control policy should also satisfy the requirement that the VBR video data is delivered to the mobile user without causing playout buffer underflow or overflows. A directional water-filling algorithm, which provides a simple and concise interpretation of the necessary optimality conditions, is identified as the optimal offline policy. Following this, two online policies are proposed for power control based on channel side information (CSI) prediction within a short time window. Dynamic programming is employed to implement the optimal offline and the initial online power control policies that minimize the transmit power consumption in the communication session. Subsequently, reinforcement learning (RL) based approach is employed for the second online power control policy. Via simulation results, we show that the optimal offline power control policy that minimizes the overall power consumption leads to substantial energy savings compared to the strategy of minimizing the time duration of video streaming. We also demonstrate that the RL algorithm performs better than the dynamic programming based online grouped water-filling (GWF) strategy unless the channel is highly correlated.

cs.IT↗

arXiv2019

A Simulation Framework for Fast Design Space Exploration of Unmanned Air System Traffic Management Policies

The number of daily small Unmanned Aircraft Systems (sUAS) operations in uncontrolled low altitude airspace is expected to reach into the millions. UAS Traffic Management (UTM) is an emerging concept aiming at the safe and efficient management of such very dense traffic, but few studies are addressing the policies to accommodate such demand and the required ground infrastructure in suburban or urban environments. Searching for the optimal air traffic management policy is a combinatorial optimization problem with intractable complexity when the number of sUAS and the constraints increases. As the demands on the airspace increase and traffic patterns get complicated, it is difficult to forecast the potential low altitude airspace hotspots and the corresponding ground resource requirements. This work presents a Multi-agent Air Traffic and Resource Usage Simulation (MATRUS) framework that aims for fast evaluation of different air traffic management policies and the relationship between policy, environment and resulting traffic patterns. It can also be used as a tool to decide the resource distribution and launch site location in the planning of a next-generation smart city. As a case study, detailed comparisons are provided for the sUAS flight time, conflict ratio, cellular communication resource usage, for a managed (centrally coordinated) and unmanaged (free flight) traffic scenario.

eess.SY↗

arXiv2018

Deep Learning Based Power Control for Quality-Driven Wireless Video Transmissions

In this paper, wireless video transmission to multiple users under total transmission power and minimum required video quality constraints is studied. In order to provide the desired performance levels to the end-users in real-time video transmissions while using the energy resources efficiently, we assume that power control is employed. Due to the presence of interference, determining the optimal power control is a non-convex problem but can be solved via monotonic optimization framework. However, monotonic optimization is an iterative algorithm and can often entail considerable computational complexity, making it not suitable for real-time applications. To address this, we propose a learning-based approach that treats the input and output of a resource allocation algorithm as an unknown nonlinear mapping and a deep neural network (DNN) is employed to learn this mapping. This learned mapping via DNN can provide the optimal power level quickly for given channel conditions.

cs.LG↗

arXiv2018

Actor-Critic Deep Reinforcement Learning for Dynamic Multichannel Access

We consider the dynamic multichannel access problem, which can be formulated as a partially observable Markov decision process (POMDP). We first propose a model-free actor-critic deep reinforcement learning based framework to explore the sensing policy. To evaluate the performance of the proposed sensing policy and the framework's tolerance against uncertainty, we test the framework in scenarios with different channel switching patterns and consider different switching probabilities. Then, we consider a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons with the Deep-Q network (DQN) based framework proposed in [1], in terms of both average reward and the time efficiency.

cs.IT↗

arXiv2018

Learning-Based Delay-Aware Caching in Wireless D2D Caching Networks

Recently, wireless caching techniques have been studied to satisfy lower delay requirements and offload traffic from peak periods. By storing parts of the popular files at the mobile users, users can locate some of their requested files in their own caches or the caches at their neighbors. In the latter case, when a user receives files from its neighbors, device-to-device(D2D) communication is performed. D2D communication underlaid with cellular networks is also a new paradigm for the upcoming wireless systems. By allowing a pair of adjacent D2D users to communicate directly, D2D communication can achieve higher throughput, better energy efficiency and lower traffic delay. In this work, we propose an efficient learning-based caching algorithm operating together with a non-parametric estimator to minimize the average transmission delay in D2D-enabled cellular networks. It is assumed that the system does not have any prior information regarding the popularity of the files, and the non-parametric estimator is aimed at learning the intensity function of the file requests. An algorithm is devised to determine the best pairs that provide the best delay improvement in each loop to form a caching policy with very low transmission delay and high throughput. This algorithm is also extended to address a more general scenario, in which the distributions of fading coefficients and values of system parameters potentially change over time. Via numerical results, the superiority of the proposed algorithm is verified by comparing it with a naive algorithm, in which all users simply cache their favorite files, and by comparing with a probabilistic algorithm, in which the users cache a file with a probability that is proportional to its popularity.

cs.IT↗