arXiv subjects

Nitin H. Vaidya

Publications and source records attributed to Nitin H. Vaidya.

At least 37 records · Page 2Linked to original sources

arXiv2019

Global Stabilization for Causally Consistent Partial Replication

Causally consistent distributed storage systems have received significant attention recently due to the potential for providing high throughput and causality guarantees. {\em Global stabilization} is a technique established for achieving causal consistency in distributed multi-version key-value store systems, adopted by the previous work such as GentleRain \cite{Du2014GentleRainCA} and Cure \cite{akkoorath2016cure}. Intuitively, this approach serializes all updates by their physical time and computes the ``Global Stable Time'' which is a time point $t$ such that versions with timestamp $\leq t$ can be returned to the client without violating causality. However, all previous designs with global stabilization assume {\em full replication}, where each data center stores a full copy of data, and each client is restricted to access servers within one data center. In this paper, we propose a theoretical framework to support {\em general partial replication} with causal consistency via global stabilization, where each server can store an arbitrary subset of the data, and each client is allowed to communicate with any subset of the servers and migrate among them without extra delays. We propose an algorithm that implements causal consistency for distributed multi-version key-value stores with general partially replication. We prove the optimality of the Global Stable Time computation in our algorithm regarding the remote update visibility latency, i.e. how fast update from a remote server is visible to the client, under general partial replication. We also provide trade-offs to further optimize the remote update visibility by introducing extra delays during client's migration. Simulation results on the performance of our algorithm compared to the previous work are also provided.

cs.DC↗

arXiv2019

Byzantine Fault Tolerant Distributed Linear Regression

This paper considers the problem of Byzantine fault tolerance in distributed linear regression in a multi-agent system. However, the proposed algorithms are given for a more general class of distributed optimization problems, of which distributed linear regression is a special case. The system comprises of a server and multiple agents, where each agent is holding a certain number of data points and responses that satisfy a linear relationship (could be noisy). The objective of the server is to determine this relationship, given that some of the agents in the system (up to a known number) are Byzantine faulty (aka. actively adversarial). We show that the server can achieve this objective, in a deterministic manner, by robustifying the original distributed gradient descent method using norm based filters, namely 'norm filtering' and 'norm-cap filtering', incurring an additional log-linear computation cost in each iteration. The proposed algorithms improve upon the existing methods on three levels: i) no assumptions are required on the probability distribution of data points, ii) system can be partially asynchronous, and iii) the computational overhead (in order to handle Byzantine faulty agents) is log-linear in number of agents and linear in dimension of data points. The proposed algorithms differ from each other in the assumptions made for their correctness, and the gradient filter they use.

cs.LG↗

arXiv2019

Byzantine Consensus under Local Broadcast Model: Tight Sufficient Condition

In this work we consider Byzantine Consensus on undirected communication graphs under the local broadcast model. In the classical point-to-point communication model the messages exchanged between two nodes $u, v$ on an edge $uv$ of $G$ are private. This allows a faulty node to send conflicting information to its different neighbours, a property called equivocation. In contrast, in the local broadcast communication model considered here, a message sent by node $u$ is received identically by all of its neighbours. This restriction to broadcast messages provides non-equivocation even for faulty nodes. In prior results [10, 11] it was shown that in the local broadcast model the communication graph must be $(\lfloor 3f/2 \rfloor +1)$-connected and have degree at least $2f$ to achieve Byzantine Consensus. In this work we show that this network condition is tight.

cs.DC↗

arXiv2019

Distributed Learning with Adversarial Agents Under Relaxed Network Condition

This work studies the problem of non-Bayesian learning over multi-agent network when there are some adversarial (faulty) agents in the network. At each time step, each non-faulty agent collects partial information about an unknown state of the world and tries to estimate true state of the world by iteratively sharing information with its neighbors. Existing algorithms in this setting require that all non-faulty agents in the network should be able to achieve consensus via local information exchange. In this work, we present an analysis of a distributed algorithm which does not require the network to achieve consensus. We show that if every non-faulty agent can receive enough information (via iteratively communicating with neighbors) to differentiate the true state of the world from other possible states then it can indeed learn the true state.

cs.DC↗

arXiv2018

Exact Byzantine Consensus Under Local-Broadcast Model

This paper considers the problem of achieving exact Byzantine consensus in a synchronous system under a local-broadcast communication model. The nodes communicate with each other via message-passing. The communication network is modeled as an undirected graph, with each vertex representing a node in the system. Under the local-broadcast communication model, when any node transmits a message, all its neighbors in the communication graph receive the message reliably. This communication model is motivated by wireless networks. In this work, we present necessary and sufficient conditions on the underlying communication graph to achieve exact Byzantine consensus under the local-broadcast communication model.

cs.DC↗

arXiv2018

Optimal Record and Replay under Causal Consistency

We investigate the minimum record needed to replay executions of processes that share causally consistent memory. For a version of causal consistency, we identify optimal records under both offline and online recording setting. Under the offline setting, a central authority has information about every process' view of the execution and can decide what information to record for each process. Under the online setting, each process has to decide on the record at runtime as the operations are observed.

cs.DC↗

arXiv2018

Effects of Topology Knowledge and Relay Depth on Asynchronous Consensus

Consider a point-to-point message-passing network. We are interested in the asynchronous crash-tolerant consensus problem in incomplete networks. We study the feasibility and efficiency of approximate consensus under different restrictions on topology knowledge and the relay depth, i.e., the maximum number of hops any message can be relayed. These two constraints are common in large-scale networks, and are used to avoid memory overload and network congestion respectively. Specifically, for different values of integers k, k , we consider that each node knows all its neighbors of at most k-hop distance (k-hop topology knowledge), and the relay depth is k . We consider both directed and undirected graphs. More concretely, we answer the following main question in asynchronous systems: What is a tight condition on the underlying communication graphs for achieving approximate consensus if each node has only a k-hop topology knowledge and relay depth k? To prove that the necessary conditions presented in the paper are also sufficient, we have developed algorithms that achieve consensus in graphs satisfying those conditions: -The first class of algorithms requires k-hop topology knowledge and relay depth k. Unlike prior algorithms, these algorithms do not flood the network, and each node does not need the full topology knowledge. We show how the convergence time and the message complexity of those algorithms is affected by k, providing the respective upper bounds. -The second set of algorithms requires only one-hop neighborhood knowledge, i.e., immediate incoming and outgoing neighbors, but needs to flood the network (i.e., relay depth is n, where n is the number of nodes). One result that may be of independent interest is a topology discovery mechanism to learn and "estimate" the topology in asynchronous directed networks with crash faults.

cs.DC↗

arXiv2017

Private Learning on Networks: Part II

This paper considers a distributed multi-agent optimization problem, with the global objective consisting of the sum of local objective functions of the agents. The agents solve the optimization problem using local computation and communication between adjacent agents in the network. We present two randomized iterative algorithms for distributed optimization. To improve privacy, our algorithms add "structured" randomization to the information exchanged between the agents. We prove deterministic correctness (in every execution) of the proposed algorithms despite the information being perturbed by noise with non-zero mean. We prove that a special case of a proposed algorithm (called function sharing) preserves privacy of individual polynomial objective functions under a suitable connectivity condition on the network topology.

cs.DC↗

arXiv2016

Timestamps for Partial Replication

Maintaining causal consistency in distributed shared memory systems using vector timestamps has received a lot of attention from both theoretical and practical prospective. However, most of the previous literature focuses on full replication where each data is stored in all replicas, which may not be scalable due to the increasing amount of data. In this report, we investigate how to achieve causal consistency in partial replicated systems, where each replica may store different set of data. We propose an algorithm that tracks causal dependencies via vector timestamp in client-server model for partial replication. The cost of our algorithm in terms of timestamps size varies as a function of the manner in which the replicas share data, and the set of replicas accessed by each client. We also establish a connection between our algorithm with the previous work on full replication.

cs.DC↗

arXiv2016

Distributed Optimization for Client-Server Architecture with Negative Gradient Weights

Availability of both massive datasets and computing resources have made machine learning and predictive analytics extremely pervasive. In this work we present a synchronous algorithm and architecture for distributed optimization motivated by privacy requirements posed by applications in machine learning. We present an algorithm for the recently proposed multi-parameter-server architecture. We consider a group of parameter servers that learn a model based on randomized gradients received from clients. Clients are computational entities with private datasets (inducing a private objective function), that evaluate and upload randomized gradients to the parameter servers. The parameter servers perform model updates based on received gradients and share the model parameters with other servers. We prove that the proposed algorithm can optimize the overall objective function for a very general architecture involving $C$ clients connected to $S$ parameter servers in an arbitrary time varying topology and the parameter servers forming a connected network.

cs.DC↗

arXiv2016

Private Learning on Networks

Continual data collection and widespread deployment of machine learning algorithms, particularly the distributed variants, have raised new privacy challenges. In a distributed machine learning scenario, the dataset is stored among several machines and they solve a distributed optimization problem to collectively learn the underlying model. We present a secure multi-party computation inspired privacy preserving distributed algorithm for optimizing a convex function consisting of several possibly non-convex functions. Each individual objective function is privately stored with an agent while the agents communicate model parameters with neighbor machines connected in a network. We show that our algorithm can correctly optimize the overall objective function and learn the underlying model accurately. We further prove that under a vertex connectivity condition on the topology, our algorithm preserves privacy of individual objective functions. We establish limits on the what a coalition of adversaries can learn by observing the messages and states shared over a network.

cs.DC↗

arXiv2016

Distributed Optimization of Convex Sum of Non-Convex Functions

We present a distributed solution to optimizing a convex function composed of several non-convex functions. Each non-convex function is privately stored with an agent while the agents communicate with neighbors to form a network. We show that coupled consensus and projected gradient descent algorithm proposed in [1] can optimize convex sum of non-convex functions under an additional assumption on gradient Lipschitzness. We further discuss the applications of this analysis in improving privacy in distributed optimization.

cs.DC↗

arXiv2016

Defending Non-Bayesian Learning against Adversarial Attacks

This paper addresses the problem of non-Bayesian learning over multi-agent networks, where agents repeatedly collect partially informative observations about an unknown state of the world, and try to collaboratively learn the true state. We focus on the impact of the adversarial agents on the performance of consensus-based non-Bayesian learning, where non-faulty agents combine local learning updates with consensus primitives. In particular, we consider the scenario where an unknown subset of agents suffer Byzantine faults -- agents suffering Byzantine faults behave arbitrarily. Two different learning rules are proposed.

cs.DC↗

arXiv2016

Efficient Timestamps for Capturing Causality

Consider an asynchronous system consisting of processes that communicate via message-passing. The processes communicate over a potentially {\em incomplete} communication network consisting of reliable bidirectional communication channels. Thus, not every pair of processes is necessarily able to communicate with each other directly. % For instance, when the communication network is a {\em star} graph, there is a {\em central} process % that can communicate with all the remaining processes (which are called {\em radial} processes), % but the radial processes cannot communicate with each other directly. The goal of the algorithms discussed in this paper is to assign timestamps to the events at all the processes such that (a) distinct events are assigned distinct timestamps, and (b) the happened-before relationship between the events can be inferred from the timestamps. We consider three types of algorithms for assigning timestamps to events: (i) Online algorithms that must (greedily) assign a timestamp to each event when the event occurs. (ii) Offline algorithms that assign timestamps to event after a finite execution is complete. (iii) Inline algorithms that assign a timestamp to each event when it occurs, but may modify some elements of a timestamp again at a later time. For specific classes of graphs, particularly {\em star} graphs and graphs with connectivity $\geq 1$, the paper presents bounds on the length of vector timestamps assigned by an {\em online} algorithm. The paper then presents an {\em inline} algorithm, which typically assigns substantially smaller timestamps than the optimal-length {\em online} vector timestamps. In particular, the inline algorithm assigns timestamp in the form of a tuple containing $2c+2$ integer elements, where $c$ is the size of the vertex cover for the underlying communication graph.

cs.DC↗

arXiv2016

Asynchronous Distributed Hypothesis Testing in the Presence of Crash Failures

This paper addresses the problem of distributed hypothesis testing in multi-agent networks, where agents repeatedly collect local observations about an unknown state of the world, and try to collaboratively detect the true state through information exchange. We focus on the impact of failures and asynchrony (two fundamental factors in distributed systems) on the performance of consensus-based non-Bayesian learning. In particular, we consider the scenario where the networked agents may suffer crash faults, and messages delay can be arbitrarily long but finite. We identify the minimal global detectability of the network for non-Bayesian rule to succeed. In addition, we obtain a generalization of a celebrated result by Wolfowitz and Hajnal to submatrices, which might be of independent interest.

cs.DC↗

arXiv2016

Relaxed Byzantine Vector Consensus

Exact Byzantine consensus problem requires that non-faulty processes reach agreement on a decision (or output) that is in the convex hull of the inputs at the non-faulty processes. It is well-known that exact consensus is impossible in an asynchronous system in presence of faults, and in a synchronous system, n>=3f+1 is tight on the number of processes to achieve exact Byzantine consensus with scalar inputs, in presence of up to f Byzantine faulty processes. Recent work has shown that when the inputs are d-dimensional vectors of reals, n>=max(3f+1,(d+1)f+1) is tight to achieve exact Byzantine consensus in synchronous systems, and n>= (d+2)f+1 for approximate Byzantine consensus in asynchronous systems. Due to the dependence of the lower bound on vector dimension d, the number of processes necessary becomes large when the vector dimension is large. With the hope of reducing the lower bound on n, we consider two relaxed versions of Byzantine vector consensus: k-Relaxed Byzantine vector consensus and (delta,p)-Relaxed Byzantine vector consensus. In k-relaxed consensus, the validity condition requires that the output must be in the convex hull of projection of the inputs onto any subset of k-dimensions of the vectors. For (delta,p)-consensus the validity condition requires that the output must be within distance delta of the convex hull of the inputs of the non-faulty processes, where L_p norm is used as the distance metric. For (delta,p)-consensus, we consider two versions: in one version, delta is a constant, and in the second version, delta is a function of the inputs themselves. We show that for k-relaxed consensus and (delta,p)-consensus with constant delta>=0, the bound on n is identical to the bound stated above for the original vector consensus problem. On the other hand, when delta depends on the inputs, we show that the bound on n is smaller when d>=3.

cs.DC↗

arXiv2015

Fault-Tolerant Distributed Optimization (Part IV): Constrained Optimization with Arbitrary Directed Networks

We study the problem of constrained distributed optimization in multi-agent networks when some of the computing agents may be faulty. In this problem, the system goal is to have all the non-faulty agents collectively minimize a global objective given by weighted average of local cost functions, each of which is initially known to a non-faulty agent only. In particular, we are interested in the scenario when the computing agents are connected by an arbitrary directed communication network, some of the agents may suffer from crash faults or Byzantine faults, and the estimate of each agent is restricted to lie in a common constraint set. This problem finds its applications in social computing and distributed large-scale machine learning. The fault-tolerant multi-agent optimization problem was first formulated by Su and Vaidya, and is solved when the local functions are defined over the whole real line, and the networks are fully-connected. In this report, we consider arbitrary directed communication networks and focus on the scenario where, local estimates at the non-faulty agents are constrained, and only local communication and minimal memory carried across iterations are allowed. In particular, we generalize our previous results on fully-connected networks and unconstrained optimization to arbitrary directed networks and constrained optimization. As a byproduct, we provide a matrix representation for iterative approximate crash consensus. The matrix representation allows us to characterize the convergence rate for crash iterative consensus.

cs.DC↗

arXiv2015

Application-Aware Consistency: An Application to Social Network

This work weakens well-known consistency models using graphs that capture applications' characteristics. The weakened models not only respect application semantic, but also yield a performance benefit. We introduce a notion of dependency graphs, which specify relations between events that are important with respect to application semantic, and then weaken traditional consistency models (e.g., causal consistency) using these graphs. Particularly, we consider two types of graphs: intra-process and inter-process dependency graphs, where intra-process dependency graphs specify how events in a single process are related, and inter-process dependency graphs specify how events across multiple processes are related. Then, based on these two types of graphs, we define new consistency model, namely {\em application-aware} consistency. We also discuss why such application-aware consistency can be useful in social network applications. This is a work in progress, and we present early ideas regarding application-aware consistency here.

cs.DC↗