arXiv subjects

Johan Pouwelse

Publications and source records attributed to Johan Pouwelse.

At least 19 recordsLinked to original sources

arXiv2026

ZeroRAG: Adaptive Semantic Peer Routing for Decentralized Retrieval-Augmented Generation

Retrieval-augmented generation (RAG) is a standard approach to enhance the reliability of large language model (LLM) outputs by retrieving documents relevant to a query and supplying them in the LLM prompt. Today's RAG systems collect, index, and serve these documents from centralized stores. In practice, however, documents are owned by organizations, devices, and individuals who cannot or will not surrender custody of them to a central party, e.g., because of privacy, copyright or regulatory compliance. Existing modern decentralized retrieval techniques either reintroduce a coordinating authority, or rely on a global data structure that requires continuous network-wide coordination (e.g., DHTs). We present ZeroRAG, a decentralized routing layer for document retrieval in RAG that performs semantic query routing across an overlay of peers with- out global data structures or centralized orchestration. Documents remain with the peers that own them, and each peer keeps only lightweight local state: a bounded set of neighbor advertisements and a cache of recently routed queries. Using these, ZeroRAG steers each query through a best-first walk that progressively rewires the overlay toward semantically relevant neighbors, so recall improves over time as the network self-organizes. We evaluate ZeroRAG using three real-world workloads and two decentralized baselines. We find that ZeroRAG outperforms baselines in terms of recall and required network messages, approaches the recall and LLM answer accuracy of a centralized document store, self-heals after catastrophic peer failures, and scales to 100 000 peers.

cs.DC↗

arXiv2026

Benchmarking Content-Defined Chunking Algorithms for Data Deduplication

Data deduplication relies on content-defined chunking (CDC) algorithms to decide where data is split. We observe that most CDC algorithms are modular: a base chunking algorithm fixes the byte-level cut-point condition, while high-level techniques (size normalization, min/max thresholds, caching) layer on top. Existing benchmarks conflate the two, obscuring which design choices actually drive performance. We isolate and evaluate eight state-of-the-art base chunking algorithms across four realistic workloads and four metrics: throughput, deduplication ratio, mean chunk size, and chunk-size variance. Further, we derive new formulas relating algorithm parameters to expected chunk size for AE, RAM, and MII. In doing so, we uncover a longstanding error in the widely-used AE parameterization reproduced across multiple papers and evaluation tools that produces chunks 13-39% smaller than intended and inflates AE's reported deduplication. Across our evaluation, AE is the most balanced base chunking algorithm across all four metrics, RAM's similarity to AE does not carry over to low-entropy data, and MII and PCI show limited advantages over simpler alternatives in storage-oriented workloads. We release all implementations, datasets, and analysis scripts as open source.

cs.DB↗

arXiv2026

TwoStepDemocracy: Prototyping of self-evolving, democratic, and decentralized systems

Decentralised systems are often built to avoid central control, but their evolution almost always depends on centralised platforms, informal maintainer authority, and a surprising amount of unpaid goodwill. To address this uncomfortable mismatch, we introduce TwoStepDemocracy, a technical proof-of-concept for protocol-native software evolution. The prototype combines costly cryptographic identities, peer-to-peer dissemination, issue and solution voting, and Bitcoin-based funding campaigns. Users can express demand by proposing and voting on issues; developers can submit concrete solutions; and accepted work can be linked to voluntary, non-custodial funding. The design deliberately separates demand, approval, and payment. This way, money can support a solution, but it never buys more voting power. The prototype demonstrates that such a coordination layer can be built as a peer-to-peer implementation with local storage, signed governance objects, and Bitcoin integration. We studied performance, scalability, and costs across storage, identity management, and funding. The results show technical feasibility, but not yet social viability. A larger user study is still needed to evaluate whether real communities would, in practice, vote, fund, and coordinate through this mechanism.

cs.DC↗

arXiv2026

Self-replicating seedbox servers using programmable money

Centralized content distribution makes availability depend on a single operator's survival and willingness to serve. EternalSeedBox replaces the operator and network with inherited economic parameters: each node is a VPS that seeds media over BitTorrent, holds a Bitcoin wallet, and autonomously decides every twelve hours whether to renew its lease, spawn a child, or sweep its funds to a healthier peer before expiring. A single genesis node seeds the fleet, and every node thereafter is provisioned, funded, and retired autonomously. We validate the design against faithful replicas of both the Bitcoin payment network and the SporeStack VPS marketplace by running the unmodified node code. A lump sum of EUR 10,000 grew the fleet to 33 nodes before capital exhausted at day 153. With simulated income, the fleet held 40--80 live nodes across 510 days, recording 268 births and 190 deaths. A heritable caution trait was introduced to diverge across generations: low-caution lineages reproduced faster during high-income phases, while the survival advantage expected of high-caution lineages during income pauses did not appear, leaving selection in favor of low-caution nodes. The fleet tolerates high node turnover because reproduction depends on any node holding a surplus, not any single node surviving. EternalSeedBox shows that a content distribution network can lease, pay for, and replenish its own hardware without a human operator after genesis, provided income exceeds per-node rent.

physics.soc-ph↗

arXiv2026

Robust and Automated Reconfiguration of Byzantine Wide-Area Replication

Distributed systems handle adversarial nodes through redundancy, which imposes a significant performance overhead. In blockchain systems, Byzantine fault-tolerant state-machine replication (BFT-SMR) is the replicated service that totally orders client transactions before execution. While prior research has primarily focused on designing novel consensus algorithms with improved performance, recent studies have shown that further gains can be achieved through configuration optimization. More precisely, replicas can monitor network latency to dynamically assign the leader role and tune voting weights, thereby improving consensus performance. However, we identify three vulnerabilities in this process that Byzantine nodes can exploit. To address these weaknesses, we propose Beware, a reconfiguration framework that filters out falsified latency reports, computes robust weight distributions, and applies machine learning to converge towards Byzantine-resilient configurations. Our evaluation shows that Beware reduces consensus latency by up to 45% compared to existing solutions.

cs.DC↗

arXiv2026

Efficient and Robust Online Learning to Rank in Decentralized Systems

In Online Learning to Rank (OLTR), ranking models are trained directly from live user interactions, but existing systems rely on a trusted central server to collect and process these interactions. This leaves operators free to introduce biases that conflict with user interests. Decentralized learning offers an attractive alternative, allowing users to collaboratively train a shared ranking model by exchanging model updates directly with one another, without any central authority. In such settings, however, malicious nodes can send poisoned model updates that degrade the ranking quality of honest nodes. We introduce RankGuard, a decentralized OLTR framework in which users collaboratively train ranking models and exchange model updates directly with other nodes. RankGuard defends against poisoning attacks by carefully evaluating incoming models against the user's own private click history, corrected for position bias. An incoming model is only aggregated if it better explains the user's past interactions than the current local model, making it fundamentally hard for malicious nodes to craft updates that pass this test without also genuinely helping the user. We derive a theoretical convergence guarantee of RankGuard. To the best of our knowledge, this is the first formal convergence analysis of a decentralized OLTR algorithm. We evaluate RankGuard against four poisoning attacks, including a powerful adaptive attack, using four standard benchmarks and three click models. RankGuard outperforms all baselines in most settings while being up to 62x more efficient than its closest competitors.

cs.DC↗

arXiv2025

StarveSpam: Mitigating Spam with Local Reputation in Permissionless Blockchains

Spam poses a growing threat to blockchain networks. Adversaries can easily create multiple accounts to flood transaction pools, inflating fees and degrading service quality. Existing defenses against spam, such as fee markets and staking requirements, primarily rely on economic deterrence, which fails to distinguish between malicious and legitimate users and often exclude low-value but honest activity. To address these shortcomings, we present StarveSpam, a decentralized reputation-based protocol that mitigates spam by operating at the transaction relay layer. StarveSpam combines local behavior tracking, peer scoring, and adaptive rate-limiting to suppress abusive actors, without requiring global consensus, protocol changes, or trusted infrastructure. We evaluate StarveSpam using real Ethereum data from a major NFT spam event and show that it outperforms existing fee-based and rule-based defenses, allowing each node to block over 95% of spam while dropping just 3% of honest traffic, and reducing the fraction of the network exposed to spam by 85% compared to existing rule-based methods. StarveSpam offers a scalable and deployable alternative to traditional spam defenses, paving the way toward more resilient and equitable blockchain infrastructure.

cs.CR↗

arXiv2025

SwarmSearch: Decentralized Search Engine with Self-Funding Economy

Centralized search engines control what we see, read, believe, and vote. Consequently, they raise concerns over information control, censorship, and bias. Decentralized search engines offer a remedy to this problem, but their adoption has been hindered by their inferior quality and lack of a self-sustaining economic framework. We present SwarmSearch, a fully decentralized, AI-powered search engine with a self-funding architecture. Our system is designed for deployment within the decentralized file-sharing software Tribler. SwarmSearch integrates volunteer-based with profit-driven mechanisms to foster an implicit marketplace for resources. Employing the state-of-the-art of AI-based retrieval and relevance ranking, we also aim to close the quality gap between decentralized search and centralized alternatives. Our system demonstrates high retrieval accuracy while showing robustness in the presence of 50% adversarial nodes.

cs.DC↗

arXiv2025

A Large-Scale Web Search Dataset for Federated Online Learning to Rank

The centralized collection of search interaction logs for training ranking models raises significant privacy concerns. Federated Online Learning to Rank (FOLTR) offers a privacy-preserving alternative by enabling collaborative model training without sharing raw user data. However, benchmarks in FOLTR are largely based on random partitioning of classical learning-to-rank datasets, simulated user clicks, and the assumption of synchronous client participation. This oversimplifies real-world dynamics and undermines the realism of experimental results. We present AOL4FOLTR, a large-scale web search dataset with 2.6 million queries from 10,000 users. Our dataset addresses key limitations of existing benchmarks by including user identifiers, real click data, and query timestamps, enabling realistic user partitioning, behavior modeling, and asynchronous federated learning scenarios.

cs.IR↗

arXiv2025

SmartphoneDemocracy: Privacy-Preserving E-Voting on Decentralized Infrastructure using Novel European Identity

The digitization of democratic processes promises greater accessibility but presents challenges in terms of security, privacy, and verifiability. Existing electronic voting systems often rely on centralized architectures, creating single points of failure and forcing too much trust in authorities, which contradicts democratic principles. This research addresses the challenge of creating a secure, private e-voting system with minimized trust dependencies designed for the most versatile personal device: the smartphone. We introduce SmartphoneDemocracy, a novel e-voting protocol that combines three key technologies: the emerging European Digital Identity (EUDI) Wallet for Sybil-resistant identity verification, Zero-Knowledge Proofs for privacy-preserving validation, and a peer-to-peer blockchain (TrustChain) for a resilient, serverless public bulletin board. Our protocol enables voters to register and cast ballots anonymously and verifiably directly from their smartphones. We provide a detailed protocol design, a security analysis against a defined threat model, and a performance evaluation demonstrating that the computational and network overhead is feasible for medium- to large-scale elections. By developing and prototyping this system, we demonstrate a viable path to empower citizens with a trustworthy, accessible, and user-controlled digital voting experience.

cs.CR↗

arXiv2025

MeritRank: Sybil Tolerant Reputation for Merit-based Tokenomics

Decentralized reputation systems are emerging as promising mechanisms to enhance the effectiveness of token-based economies. Unlike traditional monetary incentives, these systems reward participants based on the actual value of their contributions to the network. However, the advantages and challenges associated with such systems remain largely unexplored. In this work, we investigate the inherent trade-offs in designing a decentralized reputation system that is simultaneously generalizable, trustless, and Sybil-resistant. Specifically, `generalizable' means that the system can assess various types of contributions across different contexts, `trustless' indicates that it functions without the need for a central authority to oversee reputations, and `Sybil-resistant' refers to its ability to withstand manipulations by fake identities, i.e., Sybil attacks. We propose MeritRank, a Sybil-tolerant reputation system based on feedback aggregation from participants. Instead of entirely preventing Sybil attacks, our approach effectively limits the benefits that attackers can gain from such strategies. This is achieved by reducing the perceived value of the attacker's and Sybil nodes' contributions through the application of decay mechanisms -- specifically, transitivity decay, connectivity decay, and epoch decay. Using a dataset of participant interactions in MakerDAO, we conducted experiments to demonstrate the Sybil tolerance of MeritRank.

cs.DC↗

arXiv2025

Practical Federated Learning without a Server

Federated Learning (FL) enables end-user devices to collaboratively train ML models without sharing raw data, thereby preserving data privacy. In FL, a central parameter server coordinates the learning process by iteratively aggregating the trained models received from clients. Yet, deploying a central server is not always feasible due to hardware unavailability, infrastructure constraints, or operational costs. We present Plexus, a fully decentralized FL system for large networks that operates without the drawbacks originating from having a central server. Plexus distributes the responsibilities of model aggregation and sampling among participating nodes while avoiding network-wide coordination. We evaluate Plexus using realistic traces for compute speed, pairwise latency and network capacity. Our experiments on three common learning tasks and with up to 1000 nodes empirically show that Plexus reduces time-to-accuracy by 1.4-1.6x, communication volume by 15.8-292x and training resources needed for convergence by 30.5-77.9x compared to conventional decentralized learning algorithms.

cs.DC↗

arXiv2025

TrustZero -- open, verifiable and scalable zero-trust

We present a passport-level trust token for Europe. In an era of escalating cyber threats fueled by global competition in economic, military, and technological domains, traditional security models are proving inadequate. The rise of advanced attacks exploiting zero-day vulnerabilities, supply chain infiltration, and system interdependencies underscores the need for a paradigm shift in cybersecurity. Zero Trust Architecture (ZTA) emerges as a transformative framework that replaces implicit trust with continuous verification of identity and granular access control. This thesis introduces TrustZero, a scalable layer of zero-trust security built around a universal "trust token" - a non-revocable self-sovereign identity with cryptographic signatures to enable robust, mathematically grounded trust attestations. By integrating ZTA principles with cryptography, TrustZero establishes a secure web-of-trust framework adaptable to legacy systems and inter-organisational communication.

cs.CR↗

arXiv2025

Semantica: Decentralized Search using a LLM-Guided Semantic Tree Overlay

Centralized search engines are key for the Internet, but lead to undesirable concentration of power. Decentralized alternatives fail to offer equal document retrieval accuracy and speed. Nevertheless, Semantic Overlay Networks can come close to the performance of centralized solutions when the semantics of documents are properly captured. This work uses embeddings from Large Language Models to capture semantics and fulfill the promise of Semantic Overlay Networks. Our proposed algorithm, called Semantica, constructs a prefix tree (trie) utilizing document embeddings calculated by a language model. Users connect to each other based on the embeddings of their documents, ensuring that semantically similar users are directly linked. Thereby, this construction makes it more likely for user searches to be answered by the users that they are directly connected to, or by the users they are close to in the network connection graph. The implementation of our algorithm also accommodates the semantic diversity of individual users by spawning "clone" user identifiers in the tree. Our experiments use emulation with a real-world workload to show Semantica's ability to identify and connect to similar users quickly. Semantica finds up to ten times more semantically similar users than current state-of-the-art approaches. At the same time, Semantica can retrieve more than two times the number of relevant documents given the same network load. We also make our code publicly available to facilitate further research in the area.

cs.IR↗

arXiv2024

A Thorough Investigation of Content-Defined Chunking Algorithms for Data Deduplication

Data deduplication emerged as a powerful solution for reducing storage and bandwidth costs in cloud settings by eliminating redundancies at the level of chunks. This has spurred the development of numerous Content-Defined Chunking (CDC) algorithms over the past two decades. Despite advancements, the current state-of-the-art remains obscure, as a thorough and impartial analysis and comparison is lacking. We conduct a rigorous theoretical analysis and impartial experimental comparison of several leading CDC algorithms. Using four realistic datasets, we evaluate these algorithms against four key metrics: throughput, deduplication ratio, average chunk size, and chunk-size variance. Our analyses, in many instances, extend the findings of their original publications by reporting new results and putting existing ones into context. Moreover, we highlight limitations that have previously gone unnoticed. Our findings provide valuable insights that inform the selection and optimization of CDC algorithms for practical applications in data deduplication.

cs.DC↗

arXiv2024

Offline Digital Euro: a Minimum Viable CBDC using Groth-Sahai proofs

Current digital payment solutions are fragile and offer less privacy than traditional cash. Their critical dependency on an online service used to perform and validate transactions makes them void if this service is unreachable. Moreover, no transaction can be executed during server malfunctions or power outages. Due to climate change, the likelihood of extreme weather increases. As extreme weather is a major cause of power outages, the frequency of power outages is expected to increase. The lack of privacy is an inherent result of their account-based design or the use of a public ledger. The critical dependency and lack of privacy can be resolved with a Central Bank Digital Currency that can be used offline. This thesis proposes a design and a first implementation for an offline-first digital euro. The protocol offers complete privacy during transactions using zero-knowledge proofs. Furthermore, transactions can be executed offline without third parties and retroactive double-spending detection is facilitated. To protect the users' privacy, but also guard against money laundering, we have added the following privacy-guarding mechanism. The bank and trusted third parties for law enforcement must collaborate to decrypt transactions, revealing the digital pseudonym used in the transaction. Importantly, the transaction can be decrypted without decrypting prior transactions attached to the digital euro. The protocol has a working initial implementation showcasing its usability and demonstrating functionality.

cs.CR↗

arXiv2024

Decentralized Learning Made Practical with Client Sampling

Decentralized learning (DL) leverages edge devices for collaborative model training while avoiding coordination by a central server. Due to privacy concerns, DL has become an attractive alternative to centralized learning schemes since training data never leaves the device. In a round of DL, all nodes participate in model training and exchange their model with some other nodes. Performing DL in large-scale heterogeneous networks results in high communication costs and prolonged round durations due to slow nodes, effectively inflating the total training time. Furthermore, current DL algorithms also assume all nodes are available for training and aggregation at all times, diminishing the practicality of DL. This paper presents Plexus, an efficient, scalable, and practical DL system. Plexus (1) avoids network-wide participation by introducing a decentralized peer sampler that selects small subsets of available nodes that train the model each round and, (2) aggregates the trained models produced by nodes every round. Plexus is designed to handle joining and leaving nodes (churn). We extensively evaluate Plexus by incorporating realistic traces for compute speed, pairwise latency, network capacity, and availability of edge devices in our experiments. Our experiments on four common learning tasks empirically show that Plexus reduces time-to-accuracy by 1.2-8.3x, communication volume by 2.4-15.3x and training resources needed for convergence by 6.4-370x compared to baseline DL algorithms.

cs.DC↗

arXiv2024

De-DSI: Decentralised Differentiable Search Index

This study introduces De-DSI, a novel framework that fuses large language models (LLMs) with genuine decentralization for information retrieval, particularly employing the differentiable search index (DSI) concept in a decentralized setting. Focused on efficiently connecting novel user queries with document identifiers without direct document access, De-DSI operates solely on query-docid pairs. To enhance scalability, an ensemble of DSI models is introduced, where the dataset is partitioned into smaller shards for individual model training. This approach not only maintains accuracy by reducing the number of data each model needs to handle but also facilitates scalability by aggregating outcomes from multiple models. This aggregation uses a beam search to identify top docids and applies a softmax function for score normalization, selecting documents with the highest scores for retrieval. The decentralized implementation demonstrates that retrieval success is comparable to centralized methods, with the added benefit of the possibility of distributing computational complexity across the network. This setup also allows for the retrieval of multimedia items through magnet links, eliminating the need for platforms or intermediaries.

cs.IR↗