arXiv subjects

Ram Rajagopal

Publications and source records attributed to Ram Rajagopal.

At least 73 records · Page 4Linked to original sources

arXiv2019

Distributed generation of privacy preserving data with user customization

Distributed devices such as mobile phones can produce and store large amounts of data that can enhance machine learning models; however, this data may contain private information specific to the data owner that prevents the release of the data. We wish to reduce the correlation between user-specific private information and data while maintaining the useful information. Rather than learning a large model to achieve privatization from end to end, we introduce a decoupling of the creation of a latent representation and the privatization of data that allows user-specific privatization to occur in a distributed setting with limited computation and minimal disturbance on the utility of the data. We leverage a Variational Autoencoder (VAE) to create a compact latent representation of the data; however, the VAE remains fixed for all devices and all possible private labels. We then train a small generative filter to perturb the latent representation based on individual preferences regarding the private and utility information. The small filter is trained by utilizing a GAN-type robust optimization that can take place on a distributed device. We conduct experiments on three popular datasets: MNIST, UCI-Adult, and CelebA, and give a thorough evaluation including visualizing the geometry of the latent embeddings and estimating the empirical mutual information to show the effectiveness of our approach.

cs.LG↗

arXiv2018

PaToPaEM: A Data-Driven Parameter and Topology Joint Estimation Framework for Time Varying System in Distribution Grids

Grid topology and line parameters are essential for grid operation and planning, which may be missing or inaccurate in distribution grids. Existing data-driven approaches for recovering such information usually suffer from ignoring 1) input measurement errors and 2) possible state changes among historical measurements. While using the errors-in-variables (EIV) model and letting the parameter and topology estimation interact with each other (PaToPa) can address input and output measurement error modeling, it only works when all measurements are from a single system state. To solve the two challenges simultaneously, we propose the PaToPaEM framework for joint line parameter and topology estimation with historical measurements from different unknown states. We improve the static framework that only works when measurements are from one single state, by further treating state changes in historical measurements as an unobserved latent variable. We then systematically analyze the new mathematical modeling, decouple the optimization problem, and incorporate the expectation-maximization (EM) algorithm to recover different hidden states in measurements. Combining these, PaToPaEM framework enables joint topology and line parameter estimation using noisy measurements from multiple system states. It lays a solid foundation for data-driven system identification in distribution grids. Superior numerical results validate the practicability of the PaToPaEM framework.

eess.SY↗

arXiv2018

Sequential Multiple Structural Damage Detection and Localization: A Distributed Approach

As essential components of the modern urban system, the health conditions of civil structures are the foundation of urban system sustainability and need to be continuously monitored. In Structural Health Monitoring (SHM), many existing works will have limited performance in the sequential damage diagnosis process because 1) the damage events needs to be reported with short delay, 2) multiple damage locations have to be identified simultaneously, and 3) the computational complexity is intractable in large-scale wireless sensor networks (WSNs). To address these drawbacks, we propose a new damage identification approach that utilizes the time-series of damage sensitive features extracted from multiple sensors' measurements and the optimal change point detection theory to find damage occurrence time and identify the number of damage locations. As the existing change point detection methods require to centralize the sensor data, which is impracticable in many applications, we use the probabilistic graphical model to formulate WSNs and the targeting structure and propose a distributed algorithm for structural damage identification. Validation results show highly accurate damage identification in a shake table experiment and American Society of Civil Engineers benchmark structure. Also, we demonstrate that the detection delay is reduced significantly by utilizing multiple sensors' data.

stat.AP↗

arXiv2018

Congestion management within a multi-service scheduling coordination scheme for large battery storage systems

There is a growing interest in the use of largescale battery storage systems for grid services. This technology has been deployed in several countries to increase transmission systems capabilities and reliability. In particular, large-scale battery storage systems could be used for congestion management and the French Transmission System Operator (RTE) is currently installing 3 large batteries for 2020 at the sub-transmission grid level for this purpose. The battery operation for congestion management does not require the full storage capacities at all times. Thus, the residual capacities can be offered to other services to increase batteries profitability. This paper presents the framework which will be used by RTE for battery operation scheduling to combine congestion management with other services by computing day-ahead bandwidths defining available storage capacities. The bandwidths represent safe domains for grid operation scheduling: as long as the battery operation is performed within these bandwidths, there will be no grid congestion or grid congestions will be managed.

math.OC↗

arXiv2018

Fast Distribution Grid Line Outage Identification with $μ$PMU

The growing integration of distributed energy resources (DERs) in urban distribution grids raises various reliability issues due to DER's uncertain and complex behaviors. With a large-scale DER penetration, traditional outage detection methods, which rely on customers making phone calls and smart meters' "last gasp" signals, will have limited performance, because the renewable generators can supply powers after line outages and many urban grids are mesh so line outages do not affect power supply. To address these drawbacks, we propose a data-driven outage monitoring approach based on the stochastic time series analysis from micro phasor measurement unit ($μ$PMU). Specifically, we prove via power flow analysis that the dependency of time-series voltage measurements exhibits significant statistical changes after line outages. This makes the theory on optimal change-point detection suitable to identify line outages via $μ$PMUs with fast and accurate sampling. However, existing change point detection methods require post-outage voltage distribution unknown in distribution systems. Therefore, we design a maximum likelihood-based method to directly learn the distribution parameters from $μ$PMU data. We prove that the estimated parameters-based detection still achieves the optimal performance, making it extremely useful for distribution grid outage identifications. Simulation results show highly accurate outage identification in eight distribution grids with 14 configurations with and without DERs using $μ$PMU data.

eess.SY↗

arXiv2018

Structural Damage Detection and Localization with Unknown Post-Damage Feature Distribution Using Sequential Change-Point Detection Method

The high structural deficient rate poses serious risks to the operation of many bridges and buildings. To prevent critical damage and structural collapse, a quick structural health diagnosis tool is needed during normal operation or immediately after extreme events. In structural health monitoring (SHM), many existing works will have limited performance in the quick damage identification process because 1) the damage event needs to be identified with short delay and 2) the post-damage information is usually unavailable. To address these drawbacks, we propose a new damage detection and localization approach based on stochastic time series analysis. Specifically, the damage sensitive features are extracted from vibration signals and follow different distributions before and after a damage event. Hence, we use the optimal change point detection theory to find damage occurrence time. As the existing change point detectors require the post-damage feature distribution, which is unavailable in SHM, we propose a maximum likelihood method to learn the distribution parameters from the time-series data. The proposed damage detection using estimated parameters also achieves the optimal performance. Also, we utilize the detection results to find damage location without any further computation. Validation results show highly accurate damage identification in American Society of Civil Engineers benchmark structure and two shake table experiments.

stat.AP↗

arXiv2018

A Two-layer Decentralized Control Architecture for DER Coordination

This paper presents a two-layer distributed energy resource (DER) coordination architecture that allows for separate ownership of data, operates with data subjected to a large buffering delay, and employs a new measure of power quality. The two-layer architecture comprises a centralized model predictive controller (MPC) and several decentralized MPCs each operating independently with no direct communication between them and with infrequent communication with the centralized controller. The goal is to minimize a combination of total energy cost and a measure of power quality while obeying cyber-physical constraints. The global controller utilizes a fast AC optimal power flow (OPF) solver and extensive parallelization to scale the solution to large networks. Each local controller attempts to maximize arbitrage profit while following the load profile and constraints dictated by the global controller. Extensive simulations are performed for two distribution networks under a wide variety of possible storage and solar penetrations enabled by the controller speed. The simulations show that (i) the two-layer architecture can achieve tenfold improvement in power quality relative to no coordination, while capturing nearly all of the available arbitrage profit for a moderate amount of storage penetration, and (ii) both power quality and arbitrage profits are optimized when the solar and storage are distributed more widely over the network, hence it is more effective to install storage closer to the consumer.

eess.SP↗

arXiv2018

Siamese Generative Adversarial Privatizer for Biometric Data

State-of-the-art machine learning algorithms can be fooled by carefully crafted adversarial examples. As such, adversarial examples present a concrete problem in AI safety. In this work we turn the tables and ask the following question: can we harness the power of adversarial examples to prevent malicious adversaries from learning identifying information from data while allowing non-malicious entities to benefit from the utility of the same data? For instance, can we use adversarial examples to anonymize biometric dataset of faces while retaining usefulness of this data for other purposes, such as emotion recognition? To address this question, we propose a simple yet effective method, called Siamese Generative Adversarial Privatizer (SGAP), that exploits the properties of a Siamese neural network to find discriminative features that convey identifying information. When coupled with a generative model, our approach is able to correctly locate and disguise identifying information, while minimally reducing the utility of the privatized dataset. Extensive evaluation on a biometric dataset of fingerprints and cartoon faces confirms usefulness of our simple yet effective method.

cs.CV↗

arXiv2018

Data-Driven Load Modeling and Forecasting of Residential Appliances

The expansion of residential demand response programs and increased deployment of controllable loads will require accurate appliance-level load modeling and forecasting. This paper proposes a conditional hidden semi-Markov model to describe the probabilistic nature of residential appliance demand, and an algorithm for short-term load forecasting. Model parameters are estimated directly from power consumption data using scalable statistical learning methods. Case studies performed using sub-metered 1-minute power consumption data from several types of appliances demonstrate the effectiveness of the model for load forecasting and anomaly detection.

stat.AP↗

arXiv2018

Understanding Compressive Adversarial Privacy

Designing a data sharing mechanism without sacrificing too much privacy can be considered as a game between data holders and malicious attackers. This paper describes a compressive adversarial privacy framework that captures the trade-off between the data privacy and utility. We characterize the optimal data releasing mechanism through convex optimization when assuming that both the data holder and attacker can only modify the data using linear transformations. We then build a more realistic data releasing mechanism that can rely on a nonlinear compression model while the attacker uses a neural network. We demonstrate in a series of empirical applications that this framework, consisting of compressive adversarial privacy, can preserve sensitive information.

cs.LG↗

arXiv2018

Peer to Peer Sharing of Distributed Energy Resources

As the penetration of distributed energy resources in the residential sector increases, the scope for sharing arrangements expands. We model a peer-to-peer rental market for rooftop solar and energy storage in the residential sector, with households seeking to minimize their electricity costs. For varying adoption levels, we characterize the market rental price, quantity, and participation rate. We find that up to 15% adoption, the peer-to-peer market generates a surplus comparable to that attainable though a centralized sharing model. The peer-to-peer market can incentivize an increase in total adoption in the long run. We find that direct subsidies would be a cheaper way to increase adoption if enabling the peer-to-peer market increases distribution grid costs by more than a few percent. This cost increase would be related to how locally the peer-to-peer market can match renters and owners. We compute metrics of this localness and find that the market clears quite locally for a wide range of adoption rates.

cs.CE↗

arXiv2018

Urban MV and LV Distribution Grid Topology Estimation via Group Lasso

The increasing penetration of distributed energy resources poses numerous reliability issues to the urban distribution grid. The topology estimation is a critical step to ensure the robustness of distribution grid operation. However, the bus connectivity and grid topology estimation are usually hard in distribution grids. For example, it is technically challenging and costly to monitor the bus connectivity in urban grids, e.g., underground lines. It is also inappropriate to use the radial topology assumption exclusively because the grids of metropolitan cities and regions with dense loads could be with many mesh structures. To resolve these drawbacks, we propose a data-driven topology estimation method for MV and LV distribution grids by only utilizing the historical smart meter measurements. Particularly, a probabilistic graphical model is utilized to capture the statistical dependencies amongst bus voltages. We prove that the bus connectivity and grid topology estimation problems, in radial and mesh structures, can be formulated as a linear regression with a least absolute shrinkage regularization on grouped variables (\textit{group lasso}). Simulations show highly accurate results in eight MV and LV distribution networks at different sizes and 22 topology configurations using PG\&E residential smart meter data.

stat.ML↗

arXiv2018

The Value of Distributed Energy Resources for Heterogeneous Residential Consumers

The presence of behind-the-meter rooftop photovoltaics and storage in the residential sector is poised to increase significantly. Here we quantify in detail the value of these technologies to consumers and service providers. We characterize the heterogeneity in household electricity cost savings under time-varying prices due to consumption behavior differences. Different pricing policies significantly alter how households fare with respect to one another. Furthermore, household savings in absolute terms are not strongly correlated with savings normalized by PV and storage system size. We characterize the financial value of improved forecasting capabilities for a household, finding that it is a relatively small fraction of a household's cost savings. Coordination services that combine the resources available at all households can reduce costs by an additional 10% to 15% of the original total cost. Surprisingly, coordination service providers will not encourage adoption beyond 35-55% within a group. We present a simple model that explains the value of coordination and its relationship to the pricing of distribution services.

eess.SY↗

arXiv2017

Parking Sensing and Information System: Sensors, Deployment, and Evaluation

This paper describes a smart parking sensing and information system that disseminates the parking availability information for public users in a cost-effective and efficient manner. The hardware framework of the system is built on advanced wireless sensor networks and cloud service over the Internet, and the system is highly scalable. The parking information provided to the users is set in the form of occupancy rates and expected cruising time. Both are obtained from our analytical algorithm processing both historical and real-time data, and are thereafter visualized in a color theme. The entire parking system is deployed and extensively evaluated at Stanford University Parking Structure-1.

cs.CY↗

arXiv2017

Context-Aware Generative Adversarial Privacy

Preserving the utility of published datasets while simultaneously providing provable privacy guarantees is a well-known challenge. On the one hand, context-free privacy solutions, such as differential privacy, provide strong privacy guarantees, but often lead to a significant reduction in utility. On the other hand, context-aware privacy solutions, such as information theoretic privacy, achieve an improved privacy-utility tradeoff, but assume that the data holder has access to dataset statistics. We circumvent these limitations by introducing a novel context-aware privacy framework called generative adversarial privacy (GAP). GAP leverages recent advancements in generative adversarial networks (GANs) to allow the data holder to learn privatization schemes from the dataset itself. Under GAP, learning the privacy mechanism is formulated as a constrained minimax game between two players: a privatizer that sanitizes the dataset in a way that limits the risk of inference attacks on the individuals' private variables, and an adversary that tries to infer the private variables from the sanitized dataset. To evaluate GAP's performance, we investigate two simple (yet canonical) statistical dataset models: (a) the binary data model, and (b) the binary Gaussian mixture model. For both models, we derive game-theoretically optimal minimax privacy mechanisms, and show that the privacy mechanisms learned from data (in a generative adversarial fashion) match the theoretically optimal ones. This demonstrates that our framework can be easily applied in practice, even in the absence of dataset statistics.

cs.LG↗

arXiv2017

Distribution System Topology Detection Using Consumer Load and Line Flow Measurements

This work presents a topology detection method combining home smart meter information and sparse line flow measurements. The problem is formulated as a spanning tree detection problem over a graph given partial nodal and edge flow information in a deterministic and stochastic setting. In the deterministic case of known nodal power consumption and edge flows we provide sensor placement criterion which guarantees correct identification of all spanning trees. We then present a detection method which is polynomial in complexity to the size of the graph. In the stochastic case where loads are given by forecasts derived from delayed smart meter data, we provide a combinatorial Maximum a Posteriori (MAP) detector and a polynomial complexity approximate MAP detector which is shown to work near optimum in low noise regime numerical cases and moderately well in higher noise regime.

eess.SY↗

arXiv2017

Short Term Electricity Load Forecasting on Varying Levels of Aggregation

We propose a simple empirical scaling law that describes load forecasting accuracy at different levels of aggregation. The model is justified based on a simple decomposition of individual consumption patterns. We show that for different forecasting methods and horizons, aggregating more customers improves the relative forecasting performance up to specific point. Beyond this point, no more improvement in relative performance can be obtained.

stat.AP↗

arXiv2017

VADER: Visualization and Analytics for Distributed Energy Resources

Enabling deep penetration of distributed energy resources (DERs) requires comprehensive monitoring and control of the distribution network. Increasing observability beyond the substation and extending it to the edge of the grid is required to achieve this goal. The growing availability of data from measurements from inverters, smart meters, EV chargers, smart thermostats and other devices provides an opportunity to address this problem. Integration of these new data poses many challenges since not all devices are connected to the traditional supervisory control and data acquisition (SCADA) networks and can be novel types of information, collected at various sampling rates and with potentially missing values. Visualization and analytics for distributed energy resources (VADER) system and workflow is introduced as an approach and platform to fuse these different streams of data from utilities and third parties to enable comprehensive situational awareness, including scenario analysis and system state estimation. The system leverages modern large scale computing platforms, machine learning and data analytics and can be used alongside traditional advanced distribution management system (ADMS) systems to provide improved insights for distribution system management in the presence of DERs.

cs.CY↗