arXiv subjects

Kibaek Kim

Publications and source records attributed to Kibaek Kim.

At least 55 records · Page 3Linked to original sources

arXiv2023

Enabling End-to-End Secure Federated Learning in Biomedical Research on Heterogeneous Computing Environments with APPFLx

Facilitating large-scale, cross-institutional collaboration in biomedical machine learning projects requires a trustworthy and resilient federated learning (FL) environment to ensure that sensitive information such as protected health information is kept confidential. In this work, we introduce APPFLx, a low-code FL framework that enables the easy setup, configuration, and running of FL experiments across organizational and administrative boundaries while providing secure end-to-end communication, privacy-preserving functionality, and identity management. APPFLx is completely agnostic to the underlying computational infrastructure of participating clients. We demonstrate the capability of APPFLx as an easy-to-use framework for accelerating biomedical studies across institutions and healthcare systems while maintaining the protection of private medical data in two case studies: (1) predicting participant age from electrocardiogram (ECG) waveforms, and (2) detecting COVID-19 disease from chest radiographs. These experiments were performed securely across heterogeneous compute resources, including a mixture of on-premise high-performance computing and cloud computing, and highlight the role of federated learning in improving model generalizability and performance when aggregating data from multiple healthcare systems. Finally, we demonstrate that APPFLx serves as a convenient and easy-to-use framework for accelerating biomedical studies across institutions and healthcare system while maintaining the protection of private medical data.

cs.DC↗

arXiv2023

Privacy-Preserving Load Forecasting via Personalized Model Obfuscation

The widespread adoption of smart meters provides access to detailed and localized load consumption data, suitable for training building-level load forecasting models. To mitigate privacy concerns stemming from model-induced data leakage, federated learning (FL) has been proposed. This paper addresses the performance challenges of short-term load forecasting models trained with FL on heterogeneous data, emphasizing privacy preservation through model obfuscation. Our proposed algorithm, Privacy Preserving Federated Learning (PPFL), incorporates personalization layers for localized training at each smart meter. Additionally, we employ a differentially private mechanism to safeguard against data leakage from shared layers. Simulations on the NREL ComStock dataset corroborate the effectiveness of our approach.

cs.CR↗

arXiv2023

GPU-Accelerated Sequential Quadratic Programming Algorithm for Solving ACOPF

Sequential quadratic programming (SQP) is widely used in solving nonlinear optimization problem, with advantages of warm-starting solutions, as well as finding high-accurate solution and converging quadratically using second-order information, such as the Hessian matrix. In this study we develop a scalable SQP algorithm for solving the alternate current optimal power flow problem (ACOPF), leveraging the parallel computing capabilities of graphics processing units (GPUs). Our methodology incorporates the alternating direction method of multipliers (ADMM) to initialize and decompose the quadratic programming subproblems within each SQP iteration into independent small subproblems for each electric grid component. We have implemented the proposed SQP algorithm using our portable Julia package ExaAdmm.jl, which solves the ADMM subproblems in parallel on all major GPU architectures. For numerical experiments, we compared three solution approaches: (i) the SQP algorithm with a GPU-based ADMM subproblem solver, (ii) a CPU-based ADMM solver, and (iii) the QP solver Ipopt (the state-of-the-art interior point solver) and observed that for larger instances our GPU-based SQP solver efficiently leverages the GPU many-core architecture, dramatically reducing the solution time.

math.OC↗

arXiv2023

On Solving Unit Commitment with Alternating Current Optimal Power Flow on GPU

We consider the unit commitment (UC) problem that employs the alternating current optimal power flow (ACOPF) constraints, which is formulated as a mixed-integer nonlinear programming problem and thus challenging to solve in practice. We develop a new scalable algorithm based on alternating direction method of multiplier (ADMM), which enables quickly finding a good feasible solution of the UC-ACOPF problem. Our algorithm employs the component-based decomposition that solves a large number of independent small subproblems, each of which represents an individual grid component (e.g., generator, bus, and transmission line), as well as the UC subproblem. We implement the algorithm in Julia, as part of the existing package \texttt{ExaAdmm.jl}, which can efficiently run on GPUs as well as CPUs. The numerical results are reported by using IEEE test instances.

math.OC↗

arXiv2023

A GPU-based Distributed Algorithm for Linearized Optimal Power Flow in Distribution Systems

We propose a GPU-based distributed optimization algorithm, aimed at controlling optimal power flow in multi-phase and unbalanced distribution systems. Typically, conventional distributed optimization algorithms employed in such scenarios rely on parallel execution with multiple CPUs. However, this often leads to significant computation time primarily due to the need for optimization solvers to solve subproblems for every iteration of the algorithms. To address this computational challenge, we propose a distributed optimization algorithm that eliminates solver dependencies and harnesses GPU acceleration. The central idea involves decomposing networks to yield subproblems with closed-form solutions based on matrix operations that GPU can efficiently handle. We demonstrate the computational performance of our approach through numerical experiments on four IEEE test instances ranging from 13 to 8500 buses. Our results affirm the scalability and superior speed of our GPU-based approach compared to the CPU-based counterpart.

math.OC↗

arXiv2023

Federated Short-Term Load Forecasting with Personalization Layers for Heterogeneous Clients

The advent of smart meters has enabled pervasive collection of energy consumption data for training short-term load forecasting (STLF) models. In response to privacy concerns, federated learning (FL) has been proposed as a privacy-preserving approach for training, but the quality of trained models degrades as client data becomes heterogeneous. In this paper we alleviate this drawback using personalization layers, wherein certain layers of an STLF model in an FL framework are trained exclusively on the clients' own data. To that end, we propose a personalized FL algorithm (PL-FL) enabling FL to handle personalization layers. The PL-FL algorithm is implemented by using the Argonne Privacy-Preserving Federated Learning package. We test the forecast performance of models trained on the NREL ComStock dataset, which contains heterogeneous energy consumption data of multiple commercial buildings. Superior performance of models trained with PL-FL demonstrates that personalization layers enable classical FL algorithms to handle clients with heterogeneous data.

cs.LG↗

arXiv2023

APPFLx: Providing Privacy-Preserving Cross-Silo Federated Learning as a Service

Cross-silo privacy-preserving federated learning (PPFL) is a powerful tool to collaboratively train robust and generalized machine learning (ML) models without sharing sensitive (e.g., healthcare of financial) local data. To ease and accelerate the adoption of PPFL, we introduce APPFLx, a ready-to-use platform that provides privacy-preserving cross-silo federated learning as a service. APPFLx employs Globus authentication to allow users to easily and securely invite trustworthy collaborators for PPFL, implements several synchronous and asynchronous FL algorithms, streamlines the FL experiment launch process, and enables tracking and visualizing the life cycle of FL experiments, allowing domain experts and ML practitioners to easily orchestrate and evaluate cross-silo FL under one platform. APPFLx is available online at https://appflx.link

cs.LG↗

arXiv2023

Differentially Private Distributed Convex Optimization

This paper considers distributed optimization (DO) where multiple agents cooperate to minimize a global objective function, expressed as a sum of local objectives, subject to some constraints. In DO, each agent iteratively solves a local optimization model constructed by its own data and communicates some information (e.g., a local solution) with its neighbors until a global solution is obtained. Even though locally stored data are not shared with other agents, it is still possible to reconstruct the data from the information communicated among agents, which could limit the practical usage of DO in applications with sensitive data. To address this issue, we propose a privacy-preserving DO algorithm for constrained convex optimization models, which provides a statistical guarantee of data privacy, known as differential privacy, and a sequence of iterates that converges to an optimal solution in expectation. The proposed algorithm generalizes a linearized alternating direction method of multipliers by introducing a multiple local updates technique to reduce communication costs and incorporating an objective perturbation method in the local optimization models to compute and communicate randomized feasible local solutions that cannot be utilized to reconstruct the local data, thus preserving data privacy. Under the existence of convex constraints, we show that, while both algorithms provide the same level of data privacy, the objective perturbation used in the proposed algorithm can provide better solutions than does the widely adopted output perturbation method that randomizes the local solutions by adding some noise. We present the details of privacy and convergence analyses and numerically demonstrate the effectiveness of the proposed algorithm by applying it in two different applications, namely, distributed control of power flow and federated learning, where data privacy is of concern.

math.OC↗

arXiv2023

Polytopic Superset Algorithm for Nonlinear Robust Optimization

Nonlinear robust optimization (NRO) is widely used in different applications, including energy, control, and economics, to make robust decisions under uncertainty. One of the classical solution methods in NRO is an outer approximation method that iteratively solves a sample-based nonlinear problem and updates the sample set by solving an auxiliary problem subject to the uncertainty set. Although it guarantees convergence under certain assumptions, its solution iterates are generally infeasible in the original NRO problem, and it provides only a lower bound on the optimal objective value. We propose a new algorithm for a class of NRO problems that generates feasible solution iterates and provides both lower and upper bounds to the optimal objective value. In each iteration, the algorithm solves the reformulation of an NRO subproblem with respect to the polytopic supersets of the original uncertainty set and uses a cutting plane method to improve the supersets over iteration. If the NRO subproblem is infeasible, we provide a feasibility restoration step to detect whether the original NRO problem is infeasible or construct a new superset to restore the feasibility of the NRO subproblem. Further, we prove that our superset algorithm converges to the optimal solution of the original NRO problem. In numerical studies, we use application instances from portfolio optimization and production cost minimization and compare the performance between the superset algorithm and an outer approximation method called Polak's algorithm. Our result shows that the superset algorithm is more advantageous than Polak's algorithm when the number of robust constraints is large.

math.OC↗

arXiv2022

A Reinforcement Learning Approach to Parameter Selection for Distributed Optimal Power Flow

With the increasing penetration of distributed energy resources, distributed optimization algorithms have attracted significant attention for power systems applications due to their potential for superior scalability, privacy, and robustness to a single point-of-failure. The Alternating Direction Method of Multipliers (ADMM) is a popular distributed optimization algorithm; however, its convergence performance is highly dependent on the selection of penalty parameters, which are usually chosen heuristically. In this work, we use reinforcement learning (RL) to develop an adaptive penalty parameter selection policy for the AC optimal power flow (ACOPF) problem solved via ADMM with the goal of minimizing the number of iterations until convergence. We train our RL policy using deep Q-learning, and show that this policy can result in significantly accelerated convergence (up to a 59% reduction in the number of iterations compared to existing, curvature-informed penalty parameter selection methods). Furthermore, we show that our RL policy demonstrates promise for generalizability, performing well under unseen loading schemes as well as under unseen losses of lines and generators (up to a 50% reduction in iterations). This work thus provides a proof-of-concept for using RL for parameter selection in ADMM for power systems applications.

eess.SY↗

arXiv2022

APPFL: Open-Source Software Framework for Privacy-Preserving Federated Learning

Federated learning (FL) enables training models at different sites and updating the weights from the training instead of transferring data to a central location and training as in classical machine learning. The FL capability is especially important to domains such as biomedicine and smart grid, where data may not be shared freely or stored at a central location because of policy challenges. Thanks to the capability of learning from decentralized datasets, FL is now a rapidly growing research field, and numerous FL frameworks have been developed. In this work, we introduce APPFL, the Argonne Privacy-Preserving Federated Learning framework. APPFL allows users to leverage implemented privacy-preserving algorithms, implement new algorithms, and simulate and deploy various FL algorithms with privacy-preserving techniques. The modular framework enables users to customize the components for algorithms, privacy, communication protocols, neural network models, and user data. We also present a new communication-efficient algorithm based on an inexact alternating direction method of multipliers. The algorithm requires significantly less communication between the server and the clients than does the current state of the art. We demonstrate the computational capabilities of APPFL, including differentially private FL on various test datasets and its scalability, by using multiple algorithms and datasets on different computing environments.

cs.LG↗

arXiv2022

Differentially Private Federated Learning via Inexact ADMM with Multiple Local Updates

Differential privacy (DP) techniques can be applied to the federated learning model to statistically guarantee data privacy against inference attacks to communication among the learning agents. While ensuring strong data privacy, however, the DP techniques hinder achieving a greater learning performance. In this paper we develop a DP inexact alternating direction method of multipliers algorithm with multiple local updates for federated learning, where a sequence of convex subproblems is solved with the objective perturbation by random noises generated from a Laplace distribution. We show that our algorithm provides $\barε$-DP for every iteration, where $\barε$ is a privacy budget controlled by the user. We also present convergence analyses of the proposed algorithm. Using MNIST and FEMNIST datasets for the image classification, we demonstrate that our algorithm reduces the testing error by at most $31\%$ compared with the existing DP algorithm, while achieving the same level of data privacy. The numerical experiment also shows that our algorithm converges faster than the existing algorithm.

cs.LG↗

arXiv2021

A mixed complementarity problem approach for steady-state voltage and frequency stability analysis

We present a mixed complementarity problem (MCP) approach for a steady-state stability analysis of voltage and frequency of electrical grids. We perform a theoretical analysis providing conditions for the global convergence and local quadratic convergence of our solution procedure, enabling fast computation time. Moreover, algebraic equations for power flow, voltage control, and frequency control are compactly and incrementally formulated as a single MCP that subsequently is solved by a highly efficient and robust solution method. Experimental results over large grids demonstrate that our approach is as fast as the existing Newton method with heuristics while showing more robust performance.

math.OC↗

arXiv2021

On the Tightness of the Lagrangian Dual Bound for Alternating Current Optimal Power Flow

We study tightness properties of a Lagrangian dual (LD) bound for the nonconvex alternating current optimal power flow (ACOPF) problem. We show an LD bound that can be computed in a parallel, decentralized manner. Specifically, the proposed approach partitions the network into a set of subnetworks, dualizes the coupling constraints (giving the LD function), and maximizes the LD function with respect to the dual variables of the coupling constraints (giving the desired LD bound). The dual variables that maximize the LD are obtained by using a proximal bundle method. We show that the bound is less tight than the popular semidefinite programming relaxation but as tight as the second-order cone programming relaxation. We demonstrate our developments using PGLib-OPF test instances.

math.OC↗

arXiv2021

Accelerated Computation and Tracking of AC Optimal Power Flow Solutions using GPUs

We present a scalable solution method based on an alternating direction method of multipliers and graphics processing units (GPUs) for rapidly computing and tracking a solution of alternating current optimal power flow (ACOPF) problem. Such a fast computation is particularly useful for mitigating the negative impact of frequent load and generation fluctuations on the optimal operation of a large electrical grid. To this end, we decompose a given ACOPF problem by grid components, resulting in a large number of small independent nonlinear nonconvex optimization subproblems. The computation time of these subproblems is significantly accelerated by employing the massive parallel computing capability of GPUs. In addition, the warm-start ability of our method leads to faster convergence, making the method particularly suitable for fast tracking of optimal solutions. We demonstrate the performance of our method on grids having up to 70,000 buses by solving associated optimal power flow problems with both cold start and warm start.

math.OC↗

arXiv2021

A Privacy-Preserving Distributed Control of Optimal Power Flow

We consider a distributed optimal power flow formulated as an optimization problem that maximizes a nondifferentiable concave function. Solving such a problem by the existing distributed algorithms can lead to data privacy issues because the solution information exchanged within the algorithms can be utilized by an adversary to infer the data. To preserve data privacy, in this paper we propose a differentially private projected subgradient (DP-PS) algorithm that includes a solution encryption step. We show that a sequence generated by DP-PS converges in expectation, in probability, and with probability 1. Moreover, we show that the rate of convergence in expectation is affected by a target privacy level of DP-PS chosen by the user. We conduct numerical experiments that demonstrate the convergence and data privacy preservation of DP-PS.

math.OC↗

arXiv2021

Differentially Private Federated Learning via Inexact ADMM

Differential privacy (DP) techniques can be applied to the federated learning model to protect data privacy against inference attacks to communication among the learning agents. The DP techniques, however, hinder achieving a greater learning performance while ensuring strong data privacy. In this paper we develop a DP inexact alternating direction method of multipliers algorithm that solves a sequence of subproblems with the objective perturbation by random noises generated from a Laplace distribution. We show that our algorithm provides $\barε$-DP for every iteration, where $\barε$ is a privacy parameter controlled by a user. Using MNIST and FEMNIST datasets for the image classification, we demonstrate that our algorithm reduces the testing error by at most $22\%$ compared with the existing DP algorithm, while achieving the same level of data privacy. The numerical experiment also shows that our algorithm converges faster than the existing algorithm.

cs.LG↗

arXiv2021

Numerical Performance of Different Formulations for Alternating Current Optimal Power Flow

Alternating current optimal power flow (ACOPF) problems are nonconvex and nonlinear optimization problems. Utilities and independent service operators (ISO) require ACOPF to be solved in almost real time. Interior point methods (IPMs) are one of the powerful methods for solving large-scale nonlinear optimization problems and are a suitable approach for solving ACOPF with large-scale real-world transmission networks. Moreover, the choice of the formulation is as important as choosing the algorithm for solving an ACOPF problem. In this paper, different ACOPF formulations with various linear solvers and the impact of employing box constraints are evaluated for computational viability and best performance when using IPMs. Different optimization structures are used in these formulations to model the ACOPF problem representing a range of sparsity. The numerical experiments suggest that the least sparse ACOPF formulations with polar voltages yield the best computational results. Additionally, nodal injected models and current-based branch flow models are improved by enforcing box constraints. A wide range of test cases, ranging from 500-bus systems to 9591-bus systems, are used to verify the test results.

math.OC↗