arXiv subjects

Nikolaos M. Freris

Publications and source records attributed to Nikolaos M. Freris.

At least 19 recordsLinked to original sources

arXiv2025

Curvature accelerated decentralized non-convex optimization for high-dimensional machine learning problems

We consider distributed optimization as motivated by machine learning in a multi-agent system: each agent holds local data and the goal is to minimize an aggregate loss function over a common model, via an interplay of local training and distributed communication. In the most interesting case of training a neural network, the loss functions are non-convex and the high dimension of the model poses challenges in terms of communication and computation. We propose a primal-dual method that leverages second order information in the local training sub-problems in order to accelerate the algorithm. To ease the computational burden, we invoke a quasi-Newton local solver with linear cost in the model dimension. Besides, our method is communication efficient in the sense of requiring to broadcast the local model only once per round. We rigorously establish the convergence of the algorithm and demonstrate its merits by numerical experiments.

math.OC↗

arXiv2024

Communication efficient quasi-Newton distributed optimization based on the Douglas-Rachford envelope

We consider distributed optimization in the client-server setting. By use of Douglas-Rachford splitting to the dual of the sum problem, we design a BFGS method that requires minimal communication (sending/receiving one vector per round for each client). Our method is line search free and achieves superlinear convergence. Experiments are also used to demonstrate the merits in decreasing communication and computation costs.

math.OC↗

arXiv2024

SpiRobs: Logarithmic Spiral-shaped Robots for Versatile Grasping Across Scales

Realizing a soft manipulator with biologically comparable flexibility and versatility often requires careful selection of materials and actuation, as well as attentive design of its structure, perception, and control. Here, we report a new class of soft robots (SpiRobs) that morphologically replicates the logarithmic spiral pattern observed in natural appendages (e.g., octopus arms, elephant trunks, etc.). This allows for a common design principle across different scales and a speedy and inexpensive fabrication process. We further present a grasping strategy inspired by the octopus that can automatically adapt to a target object's size and shape. We illustrate the dexterity of SpiRobs and the ability to tightly grasp objects that vary in size by more than two orders of magnitude and up to 260 times self-weight. We demonstrate scalability via three additional variants: a miniaturized gripper (mm), a one-meter-long manipulator, and an array of SpiRobs that can tangle up various objects.

cs.RO↗

arXiv2024

Anteumbler: Non-Invasive Antenna Orientation Error Measurement for WiFi APs

The performance of WiFi-based localization systems is affected by the spatial accuracy of WiFi AP. Compared with the imprecision of AP location and antenna separation, the imprecision of AP's or antenna's orientation is more important in real scenarios, including AP rotation and antenna irregular tilt. In this paper, we propose Anteumbler that non-invasively, accurately and efficiently measures the orientation of each antenna in physical space. Based on the fact that the received power is maximized when a Tx-Rx antenna pair is perfectly aligned, we construct a spatial angle model that can obtain the antennas' orientations without prior knowledge. However, the sampling points of traversing the spatial angle need to cover the entire space. We use the orthogonality of antenna directivity and polarization and adopt an iterative algorithm to reduce the sampling points by hundreds of times, which greatly improves the efficiency. To achieve the required antenna orientation accuracy, we eliminate the influence of propagation distance using a dual plane intersection model and filter out ambient noise. Our real-world experiments with six antenna types, two antenna layouts and two antenna separations show that Anteumbler achieves median errors below 6 degree for both elevation and azimuth angles, and is robust to NLoS and dynamic environments. Last but not least, for the reverse localization system, we deploy Anteumbler over LocAP and reduce the antenna separation error by 10 mm, while for the user localization system, we deploy Anteumbler over SpotFi and reduce the user localization error by more than 1 m.

cs.AR↗

arXiv2023

Communication-efficient distributed optimization with adaptability to system heterogeneity

We consider the setting of agents cooperatively minimizing the sum of local objectives plus a regularizer on a graph. This paper proposes a primal-dual method in consideration of three distinctive attributes of real-life multi-agent systems, namely: (i)expensive communication, (ii)lack of synchronization, and (iii)system heterogeneity. In specific, we propose a distributed asynchronous algorithm with minimal communication cost, in which users commit variable amounts of local work on their respective sub-problems. We illustrate this both theoretically and experimentally in the machine learning setting, where the agents hold private data and use a stochastic Newton method as the local solver. Under standard assumptions on Lipschitz continuous gradients and strong convexity, our analysis establishes linear convergence in expectation and characterizes the dependency of the rate on the number of local iterations. We proceed a step further to propose a simple means for tuning agents' hyperparameters locally, so as to adjust to heterogeneity and accelerate the overall convergence. Last, we validate our proposed method on a benchmark machine learning dataset to illustrate the merits in terms of computation, communication, and run-time saving as well as adaptability to heterogeneity.

math.OC↗

arXiv2023

TPMDP: Threshold Personalized Multi-party Differential Privacy via Optimal Gaussian Mechanism

In modern distributed computing applications, such as federated learning and AIoT systems, protecting privacy is crucial to prevent adversarial parties from colluding to steal others' private information. However, guaranteeing the utility of computation outcomes while protecting all parties' data privacy can be challenging, particularly when the parties' privacy requirements are highly heterogeneous. In this paper, we propose a novel privacy framework for multi-party computation called Threshold Personalized Multi-party Differential Privacy (TPMDP), which addresses a limited number of semi-honest colluding adversaries. Our framework enables each party to have a personalized privacy budget. We design a multi-party Gaussian mechanism that is easy to implement and satisfies TPMDP, wherein each party perturbs the computation outcome in a secure multi-party computation protocol using Gaussian noise. To optimize the utility of the mechanism, we cast the utility loss minimization problem into a linear programming (LP) problem. We exploit the specific structure of this LP problem to compute the optimal solution after O(n) computations, where n is the number of parties, while a generic solver may require exponentially many computations. Extensive experiments demonstrate the benefits of our approach in terms of low utility loss and high efficiency compared to existing private mechanisms that do not consider personalized privacy requirements or collusion thresholds.

cs.CR↗

arXiv2023

Distributed optimization on directed graphs based on inexact ADMM with partial participation

We consider the problem of minimizing the sum of cost functions pertaining to agents over a network whose topology is captured by a directed graph (i.e., asymmetric communication). We cast the problem into the ADMM setting, via a consensus constraint, for which both primal subproblems are solved inexactly. In specific, the computationally demanding local minimization step is replaced by a single gradient step, while the averaging step is approximated in a distributed fashion. Furthermore, partial participation is allowed in the implementation of the algorithm. Under standard assumptions on strong convexity and Lipschitz continuous gradients, we establish linear convergence and characterize the rate in terms of the connectivity of the graph and the conditioning of the problem. Our line of analysis provides a sharper convergence rate compared to Push-DIGing. Numerical experiments corroborate the merits of the proposed solution in terms of superior rate as well as computation and communication savings over baselines.

math.OC↗

arXiv2023

Communication Efficient Curvature Aided Primal-dual Algorithms for Decentralized Optimization

This paper presents a family of algorithms for decentralized convex composite problems. We consider the setting of a network of agents that cooperatively minimize a global objective function composed of a sum of local functions plus a regularizer. Through the use of intermediate consensus variables, we remove the need for inner communication loops between agents when computing curvature-guided updates. A general scheme is presented which unifies the analysis for a plethora of computing choices, including gradient descent, Newton updates, and BFGS updates. Our analysis establishes sublinear convergence rates under convex objective functions with Lipschitz continuous gradients, as well as linear convergence rates when the local functions are further assumed to be strongly convex. Moreover, we explicitly characterize the acceleration due to curvature information. Last but not the least, we present an asynchronous implementation for the proposed algorithms, which removes the need for a central clock, with linear convergence rates established in expectation under strongly convex objectives. We ascertain the effectiveness of the proposed methods with numerical experiments on benchmark datasets.

math.OC↗

arXiv2022

A Novel Sequential Coreset Method for Gradient Descent Algorithms

A wide range of optimization problems arising in machine learning can be solved by gradient descent algorithms, and a central question in this area is how to efficiently compress a large-scale dataset so as to reduce the computational complexity. {\em Coreset} is a popular data compression technique that has been extensively studied before. However, most of existing coreset methods are problem-dependent and cannot be used as a general tool for a broader range of applications. A key obstacle is that they often rely on the pseudo-dimension and total sensitivity bound that can be very high or hard to obtain. In this paper, based on the ''locality'' property of gradient descent algorithms, we propose a new framework, termed ''sequential coreset'', which effectively avoids these obstacles. Moreover, our method is particularly suitable for sparse optimization whence the coreset size can be further reduced to be only poly-logarithmically dependent on the dimension. In practice, the experimental results suggest that our method can save a large amount of running time compared with the baseline algorithms.

cs.LG↗

arXiv2022

Distributed Primal-dual Optimization for Heterogeneous Multi-agent Systems

Heterogeneous networks comprise agents with varying capabilities in terms of computation, storage, and communication. In such settings, it is crucial to factor in the operating characteristics in allowing agents to choose appropriate updating schemes, so as to better distribute computational tasks and utilize the network more efficiently. We consider the multi-agent optimization problem of cooperatively minimizing the sum of local strongly convex objectives. We propose an asynchronous distributed primal-dual protocol, which allows for the primal update steps to be agent-dependent (an agent can opt between first-order or Newton updates). Our analysis introduces a unifying framework for such hybrid optimization scheme and establishes global linear convergence in expectation, under strongly convex objectives and general agent activation schemes. Numerical experiments on real life datasets attest to the merits of the proposed algorithm.

math.OC↗

arXiv2022

FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity

Federated Learning (FL) is an emerging framework for distributed processing of large data volumes by edge devices subject to limited communication bandwidths, heterogeneity in data distributions and computational resources, as well as privacy considerations. In this paper, we introduce a new FL protocol termed FedADMM based on primal-dual optimization. The proposed method leverages dual variables to tackle statistical heterogeneity, and accommodates system heterogeneity by tolerating variable amount of work performed by clients. FedADMM maintains identical communication costs per round as FedAvg/Prox, and generalizes them via the augmented Lagrangian. A convergence proof is established for nonconvex objectives, under no restrictions in terms of data dissimilarity or number of participants per round of the algorithm. We demonstrate the merits through extensive experiments on real datasets, under both IID and non-IID data distributions across clients. FedADMM consistently outperforms all baseline methods in terms of communication efficiency, with the number of rounds needed to reach a prescribed accuracy reduced by up to 87%. The algorithm effectively adapts to heterogeneous data distributions through the use of dual variables, without the need for hyperparameter tuning, and its advantages are more pronounced in large-scale systems.

cs.LG↗

arXiv2022

Dynamical Modeling and Control of Soft Robots with Non-constant Curvature Deformation

The Piecewise Constant Curvature (PCC) model is the most widely used soft robotic modeling and control. However, the PCC fails to accurately describe the deformation of the soft robots when executing dynamic tasks or interacting with the environment. This paper presents a simple threedimensional (3D) modeling method for a multi-segment soft robotic manipulator with non-constant curvature deformation. We devise kinematic and dynamical models for soft manipulators by modeling each segment of the manipulator as two stretchable links connected by a universal joint. Based on that, we present two controllers for dynamic trajectory tracking in confguration space and pose control in task space, respectively. Model accuracy is demonstrated with simulations and experimental data. The controllers are implemented on a four-segment soft robotic manipulator and validated in dynamic motions and pose control with unknown loads. The experimental results show that the dynamic controller enables a stable reference trajectory tracking at speeds up to 7m/s.

cs.RO↗

arXiv2022

Unified Kinematic and Dynamical Modeling of a Soft Robotic Arm by a Piecewise Universal Joint Model

The compliance of soft robotic arms renders the development of accurate kinematic & dynamical models especially challenging. The most widely used model in soft robotic kinematics assumes Piecewise Constant Curvature (PCC). However, PCC fails to effectively handle external forces, or even the influence of gravity, since the robot does not deform with a constant curvature under these conditions. In this paper, we establish three-dimensional (3D) modeling of a multi-segment soft robotic arm under the less restrictive assumption that each segment of the arm is deformed on a plane without twisting. We devise a kinematic and dynamical model for the soft arm by deriving equivalence to a serial universal joint robot. Numerous experiments on the real robot platform along with simulations attest to the modeling accuracy of our approach in 3D motion with load. The maximum position/rotation error of the proposed model is verified 6.7x/4.6x lower than the PCC model considering gravity and external forces.

cs.RO↗

arXiv2022

Control of a Soft Robotic Arm Using a Piecewise Universal Joint Model

The 'infinite' passive degrees of freedom of soft robotic arms render their control especially challenging. In this paper, we leverage a previously developed model, which drawing equivalence of the soft arm to a series of universal joints, to design two closed-loop controllers: a configuration space controller for trajectory tracking and a task space controller for position control of the end effector. Extensive experiments and simulations on a four-segment soft arm attest to substantial improvement in terms of: a) superior tracking accuracy of the configuration space controller and b) reduced settling time and steady-state error of the task space controller. The task space controller is also verified to be effective in the presence of interactions between the soft arm and the environment.

cs.RO↗

arXiv2022

A Communication Efficient Quasi-Newton Method for Large-scale Distributed Multi-agent Optimization

We propose a communication efficient quasi-Newton method for large-scale multi-agent convex composite optimization. We assume the setting of a network of agents that cooperatively solve a global minimization problem with strongly convex local cost functions augmented with a non-smooth convex regularizer. By introducing consensus variables, we obtain a block-diagonal Hessian and thus eliminate the need for additional communication when approximating the objective curvature information. Moreover, we reduce computational costs of existing primal-dual quasi-Newton methods from $\mathcal{O}(d^3)$ to $\mathcal{O}(cd)$ by storing $c$ pairs of vectors of dimension $d$. An asynchronous implementation is presented that removes the need for coordination. Global linear convergence rate in expectation is established, and we demonstrate the merit of our algorithm numerically with real datasets.

math.OC↗

arXiv2021

BFGS-ADMM for Large-Scale Distributed Optimization

We consider a class of distributed optimization problem where the objective function consists of a sum of strongly convex and smooth functions and a (possibly nonsmooth) convex regularizer. A multi-agent network is assumed, where each agent holds a private cost function and cooperates with its neighbors to compute the optimum of the aggregate objective. We propose a quasi-Newton Alternating Direction Method of Multipliers (ADMM) where the primal update is solved inexactly with approximated curvature information. By introducing an intermediate consensus variable, we achieve a block diagonal Hessian which eliminates the need for inner communication loops within the network when computing the update direction. We establish global linear convergence to the optimal primal-dual solution without the need for backtracking line search, under the assumption that component cost functions are strongly convex with Lipschitz continuous gradients. Numerical simulations on real datasets demonstrate the advantages of the proposed method over state of the art.

math.OC↗

arXiv2021

DN-ADMM: Distributed Newton ADMM for Multi-agent Optimization

In a multi-agent network, we consider the problem of minimizing an objective function that is expressed as the sum of private convex and smooth functions, and a (possibly) non-differentiable convex regularizer. We propose a novel distributed second-order method based on the framework of Alternating Direction Method of Multipliers (ADMM), by invoking approximate Newton iterations to the primal update corresponding to the differentiable part. In order to achieve a distributed implementation, the total Hessian matrix is split into a diagonal component (locally computable) and an off-diagonal component (that requires communication between neighboring agents). Subsequently, the Hessian inverse is approximated by a truncation of the Taylor expansion to $K$ terms: this amounts to fully distributed updates entailing $K$ distributed communication rounds. We establish global linear convergence to the primal-dual optimal solution under the assumption that the private functions are strongly convex and have Lipschitz continuous gradient. Numerical experiments demonstrate the merits of the approach comparatively with state-of-the-art methods.

math.OC↗

arXiv2019

A New Randomized Block-Coordinate Primal-Dual Proximal Algorithm for Distributed Optimization

This paper proposes TriPD, a new primal-dual algorithm for minimizing the sum of a Lipschitz-differentiable convex function and two possibly nonsmooth convex functions, one of which is composed with a linear mapping. We devise a randomized block-coordinate version of the algorithm which converges under the same stepsize conditions as the full algorithm. It is shown that both the original as well as the block-coordinate scheme feature linear convergence rate when the functions involved are either piecewise linear-quadratic, or when they satisfy a certain quadratic growth condition (which is weaker than strong convexity). Moreover, we apply the developed algorithms to the problem of multi-agent optimization on a graph, thus obtaining novel synchronous and asynchronous distributed methods. The proposed algorithms are fully distributed in the sense that the updates and the stepsizes of each agent only depend on local information. In fact, no prior global coordination is required. Finally, we showcase an application of our algorithm in distributed formation control.

math.OC↗