SearcharxivSearch

arXiv subjects

Stefan Tai

Publications and source records attributed to Stefan Tai.

18 recordsLinked to original sources

Enabling Reuse for Data-Sharing Pipelines in Federated Environments

Data mesh architectures enable decentralized data sharing through domain-owned data products, but supporting diverse consumers in federated settings often requires customized data-sharing pipelines. As the number of consumers grows, this leads to a proliferation of pipelines, increasing design and maintenance complexity. We observe that such pipelines frequently exhibit substantial structural overlap. In this paper, we argue that reuse should serve as a guiding principle to address this challenge. We define reuse in data-sharing pipelines as the systematic use of existing data assets and transformation logic across pipelines, and identify reuse opportunities at both design time and runtime. We analyze the associated challenges and outline a reuse-oriented design approach, supported by a reference architecture. A preliminary evaluation demonstrates the potential of reuse to reduce redundancy and improve manageability, providing a pathway toward more scalable and sustainable federated data sharing.

cs.DC

An Evidence-driven Protocol for Trustworthy CI Pipelines

Enterprise software supply chains are increasingly vulnerable to infrastructure attacks, resulting in financial and reputational damage. Ensuring the integrity and provenance of software artifacts remains a significant challenge, where re-execution of the build and tests by every consumer to guarantee provenance produces a verification bottleneck and credibility reduction. This paper presents an evidence-driven protocol for trustworthy Continuous Integration (CI) pipelines that combines Deterministic Build Systems (DBS) with Trusted Execution Environments (TEEs). The approach provides cryptographically verifiable guarantees of integrity, authenticity, and attestation for CI artifacts in distributed environments, reducing implicit trust without requiring costly re-execution by consumers. We introduce a protocol that binds deterministic builds with TEE-based attestations, formalizing the evidence life cycle, together with a practical implementation using Nix and Intel TDX. Experimental results show that artifact verification is reduced from redundant computation to lightweight signature and policy checks. These findings demonstrate that evidence-driven CI pipelines establish scalable and verifiable trust in digital infrastructure, effectively amortizing the initial computational overhead introduced by TEEs.

cs.CR

Service-Level Energy Modeling and Experimentation for Cloud-Native Microservices

Microservice architectures have become the dominant paradigm for cloud-native systems, offering flexibility and scalability. However, this shift has also led to increased demand for cloud resources, contributing to higher energy consumption and carbon emissions. While existing research has focused on measuring fine-grained energy usage of CPU and memory at the container level, or on system-wide assessments, these approaches often overlook the energy impact of cross-container service interactions, especially those involving network and storage for auxiliary services such as observability and system monitoring. To address this gap, we introduce a service-level energy model that captures the distributed nature of microservice execution across containers. Our model is supported by an experimentation tool that accounts for energy consumption not just in CPU and memory, but also in network and storage components. We validate our approach through extensive experimentation with diverse experiment configurations of auxiliary services for a popular open-source cloud-native microservice application. Results show that omitting network and storage can lead to an underestimation of auxiliary service energy use by up to 63%, highlighting the need for more comprehensive energy assessments in the design of energy-efficient microservice architectures.

cs.DC

A Non-Intrusive Framework for Deferred Integration of Cloud Patterns in Energy-Efficient Data-Sharing Pipelines

As data mesh architectures gain traction in federated environments, organizations are increasingly building consumer-specific data-sharing pipelines using modular, cloud-native transformation services. Prior work has shown that structuring these pipelines with reusable transformation stages enhances both scalability and energy efficiency. However, integrating traditional cloud design patterns into such pipelines poses a challenge: predefining and embedding patterns can compromise modularity, reduce reusability, and conflict with the pipelines dynamic, consumer-driven nature. To address this, we introduce a Kubernetes-based tool that enables the deferred and non-intrusive application of selected cloud design patterns without requiring changes to service source code. The tool supports automated pattern injection and collects energy consumption metrics, allowing developers to make energy-aware decisions while preserving the flexible, composable structure of reusable data-sharing pipelines.

cs.DC

Energy Profiling of Data-Sharing Pipelines: Modeling, Estimation, and Reuse Strategies

Data-sharing pipelines involve a series of stages that apply policy-based data transformations to enable secure and effective data exchange among organizations. Although numerous tools and platforms exist to manage governance and enforcement in these pipelines, energy efficiency in data exchange has received limited attention. This paper introduces a novel method to model and estimate the energy consumption of different execution configurations in data-sharing pipelines. Additionally, this method identifies reuse potential in shared stages across pipelines that hold the key to reducing energy in large data-sharing federations. We validate this method through simulation experiments, revealing promising potential for cross-organizational pipeline optimization and laying a foundation for energy-conscious execution strategies.

cs.DB

Trusted Compute Units: A Framework for Chained Verifiable Computations

Blockchain and distributed ledger technologies (DLTs) facilitate decentralized computations across trust boundaries. However, ensuring complex computations with low gas fees and confidentiality remains challenging. Recent advances in Confidential Computing -- leveraging hardware-based Trusted Execution Environments (TEEs) -- and Proof-carrying Data -- employing cryptographic Zero-Knowledge Virtual Machines (zkVMs) -- hold promise for secure, privacy-preserving off-chain and layer-2 computations. On the other side, a homogeneous reliance on a single technology, such as TEEs or zkVMs, is impractical for decentralized environments with heterogeneous computational requirements. This paper introduces the Trusted Compute Unit (TCU), a unifying framework that enables composable and interoperable verifiable computations across heterogeneous technologies. Our approach allows decentralized applications (dApps) to flexibly offload complex computations to TCUs, obtaining proof of correctness. These proofs can be anchored on-chain for automated dApp interactions, while ensuring confidentiality of input data, and integrity of output data. We demonstrate how TCUs can support a prominent blockchain use case, such as federated learning. By enabling secure off-chain interactions without incurring on-chain confirmation delays or gas fees, TCUs significantly improve system performance and scalability. Experimental insights and performance evaluations confirm the feasibility and practicality of this unified approach, advancing the state of the art in verifiable off-chain services for the blockchain ecosystem.

cs.CR

A Comprehensive Experimentation Framework for Energy-Efficient Design of Cloud-Native Applications

Current approaches to designing energy-efficient applications typically rely on measuring individual components using readily available local metrics, like CPU utilization. However, these metrics fall short when applied to cloud-native applications, which operate within the multi-tenant, shared environments of distributed cloud providers. Assessing and optimizing the energy efficiency of cloud-native applications requires consideration of the complex, layered nature of modern cloud stacks. To address this need, we present a comprehensive, automated, and extensible experimentation framework that enables developers to measure energy efficiency across all relevant layers of a cloud-based application and evaluate associated quality trade-offs. Our framework integrates a suite of service quality and sustainability metrics, providing compatibility with any Kubernetes-based application. We demonstrate the feasibility and effectiveness of this approach through initial experimental results, comparing architectural design alternatives for a widely used open-source cloud-native application.

cs.SE

Informed and Assessable Observability Design Decisions in Cloud-native Microservice Applications

Observability is important to ensure the reliability of microservice applications. These applications are often prone to failures, since they have many independent services deployed on heterogeneous environments. When employed "correctly", observability can help developers identify and troubleshoot faults quickly. However, instrumenting and configuring the observability of a microservice application is not trivial but tool-dependent and tied to costs. Architects need to understand observability-related trade-offs in order to weigh between different observability design alternatives. Still, these architectural design decisions are not supported by systematic methods and typically just rely on "professional intuition". In this paper, we argue for a systematic method to arrive at informed and continuously assessable observability design decisions. Specifically, we focus on fault observability of cloud-native microservice applications, and turn this into a testable and quantifiable property. Towards our goal, we first model the scale and scope of observability design decisions across the cloud-native stack. Then, we propose observability metrics which can be determined for any microservice application through so-called observability experiments. We present a proof-of-concept implementation of our experiment tool OXN. OXN is able to inject arbitrary faults into an application, similar to Chaos Engineering, but also possesses the unique capability to modify the observability configuration, allowing for the assessment of design decisions that were previously left unexplored. We demonstrate our approach using a popular open source microservice application and show the trade-offs involved in different observability design decisions.

cs.SE

Servicifying zk-SNARKs Execution for Verifiable Off-chain Computations

Zk-SNARKs help scale blockchains with Verifiable Off-chain Computations (VOC). zk-SNARK DSL toolkits are key when designing arithmetic circuits but fall short of automating the subsequent proof-generation step in an automated manner. We emphasize the need for portability, interoperability, and manageability in VOC-based solutions and introduce a Proving Service that is designed to provide a scalable and reusable solution for generating zk-SNARK proofs leveraging clouds.

cs.SE

End-to-End Verifiable Decentralized Federated Learning

Verifiable decentralized federated learning (FL) systems combining blockchains and zero-knowledge proofs (ZKP) make the computational integrity of local learning and global aggregation verifiable across workers. However, they are not end-to-end: data can still be corrupted prior to the learning. In this paper, we propose a verifiable decentralized FL system for end-to-end integrity and authenticity of data and computation extending verifiability to the data source. Addressing an inherent conflict of confidentiality and transparency, we introduce a two-step proving and verification (2PV) method that we apply to central system procedures: a registration workflow that enables non-disclosing verification of device certificates and a learning workflow that extends existing blockchain and ZKP-based FL systems through non-disclosing data authenticity proofs. Our evaluation on a prototypical implementation demonstrates the technical feasibility with only marginal overheads to state-of-the-art solutions.

cs.LG

Non-Disclosing Credential On-chaining for Blockchain-based Decentralized Applications

Many service systems rely on verifiable identity-related information of their users. Manipulation and unwanted exposure of this privacy-relevant information, however, must at the same time be prevented and avoided. Peer-to-peer blockchain-based decentralization with a smart contract-based execution model and verifiable off-chain computations leveraging zero-knowledge proofs promise to provide the basis for next-generation, non-disclosing credential management solutions. In this paper, we propose a novel credential on-chaining system that ensures blockchain-based transparency while preserving pseudonymity. We present a general model compliant to the W3C verifiable credential recommendation and demonstrate how it can be applied to solve existing problems that require computational identity-related attribute verification. Our zkSNARKs-based reference implementation and evaluation show that, compared to related approaches based on, e.g., CL-signatures, our approach provides significant performance advantages and more flexible proof mechanisms, underpinning our vision of increasingly decentralized, transparent, and trustworthy service systems.

cs.CR

Advancing Blockchain-based Federated Learning through Verifiable Off-chain Computations

Federated learning may be subject to both global aggregation attacks and distributed poisoning attacks. Blockchain technology along with incentive and penalty mechanisms have been suggested to counter these. In this paper, we explore verifiable off-chain computations using zero-knowledge proofs as an alternative to incentive and penalty mechanisms in blockchain-based federated learning. In our solution, learning nodes, in addition to their computational duties, act as off-chain provers submitting proofs to attest computational correctness of parameters that can be verified on the blockchain. We demonstrate and evaluate our solution through a health monitoring use case and proof-of-concept implementation leveraging the ZoKrates language and tools for smart contract-based on-chain model management. Our research introduces verifiability of correctness of learning processes, thus advancing blockchain-based federated learning.

cs.CR

Synthesizing Configuration Tactics for Exercising Hidden Options in Serverless Systems

A proper configuration of an information system can ensure accuracy and efficiency, among other system objectives. Conversely, a poor configuration can have a significant negative impact on the system's performance, reliability, and cost. Serverless systems, which are comprised of many functions and managed services, especially risk exposure to misconfigurations, with many provider- and platform-specific, often intransparent and 'hidden' settings. In this paper, we argue to pay close attention to the configuration of serverless systems to exercise options with known accuracy, cost and time. Based on a literature study and long-term serverless systems development experience, we present nine tactics to unlock potentially neglected and unknown options in serverless systems.

cs.SE

Trustworthy Pre-Processing of Sensor Data in Data On-chaining Workflows for Blockchain-based IoT Applications

Prior to provisioning sensor data to smart contracts, a pre-processing of the data on intermediate off-chain nodes is often necessary. When doing so, originally constructed cryptographic signatures cannot be verified on-chain anymore. This exposes an opportunity for undetected manipulation and presents a problem for applications in the Internet of Things where trustworthy sensor data is required on-chain. In this paper, we propose trustworthy pre-processing as enabler for end-to-end sensor data integrity in data on-chaining workflows. We define requirements for trustworthy pre-processing, present a model and common workflow for data on-chaining, select off-chain computation utilizing Zero-knowledge Proofs (ZKPs) and Trusted Execution Environments (TEEs) as promising solution approaches, and discuss both our proof-of-concept implementations and initial experimental, comparative evaluation results. The importance of trustworthy pre-processing and principle solution approaches are presented, addressing the major problem of end-to-end sensor data integrity in blockchain-based IoT applications.

cs.DC

Application-Platform Co-Design for Serverless Data Processing

"Application-platform co-design" refers to the phenomenon of new platforms being created in response to changing application needs, followed by application design and development changing due to the emergence (and the specifics, limitations) of the new platforms, therefore creating, again, new application and platform requirements. This continuous process of application and platform (re-)design describes an engineering and management responsibility to constantly evaluate any given platform for application fit and platform-specific application design, and to consider a new or evolutionary platform development project due to evolving and changing application needs. In this paper, we study this phenomenon in the context of serverless computing and (big) data processing needs, and thus, for application-platform co-design for serverless data processing (SDP). We present an analysis of the state-of-the-art of function-as-a-service (FaaS) platforms, which reveals several configuration, deployment, execution, and measurement differences between popular platforms happening at-speed. These differences indicate already ongoing platform (re-)design processes resulting in more specialized serverless platforms and new, platform-specific challenges for application design. We discuss data processing needs of applications using the serverless model and present common initial (and undesirable) workaround solutions on the application level, giving additional argument to the creation of new SDP platforms. We present critical SDP requirements and possible new platform augmentations, but identify the need for engineering methods and tooling to better guide application-platform co-design. We argue to pay appropriate attention to the phenomenon of continuous application-platform co-design to better anticipate and to control future platform and application developments.

cs.SE

Messaging with Purpose Limitation -- Privacy-Compliant Publish-Subscribe Systems

Purpose limitation is an important privacy principle to ensure that personal data may only be used for the declared purposes it was originally collected for. Ensuring compliance with respective privacy regulations like the GDPR, which codify purpose limitation as an obligation, consequently, is a major challenge in real-world enterprise systems. Technical solutions under the umbrella of purpose-based access control (PBAC), however, focus mostly on data being held at-rest in databases, while PBAC for communication and publish-subscribe messaging in particular has received only little attention. In this paper, we argue for PBAC to be also applied to data-in-transit and introduce and study a concrete proof-of-concept implementation, which extends a popular MQTT message broker with purpose limitation. On this basis, purpose limitation as a core privacy principle can be addressed in enterprise IoT and message-driven integration architectures that do not focus on databases but event-driven communication and integration instead.

cs.CR

Blockchain-based Data Provenance for the Internet of Things

As more and more applications and services depend on data collected and provided by Internet of Things (IoT) devices, it is of importance that such data can be trusted. Data provenance solutions together with blockchain technology are one way to make data more trustworthy. However, current solutions do not address the heterogeneous nature of IoT applications and their data. In this work, we identify functional and non-functional requirements for a generic IoT data provenance framework, and conceptualise the framework as a layered architecture. Using a proof-of-concept implementation based on Ethereum smart contracts, data provenance can be realised for a wide range of IoT use cases. Benefits of a generic framework include simplified adoption and a more rapid implementation of data provenance for the IoT.

cs.CR

(MC2)2: A Generic Decision-Making Framework and its Application to Cloud Computing

Cloud computing is a disruptive technology, representing a new model for information technology (IT) solution engineering and management that promises to introduce significant cost savings and other benefits. The adoption of Cloud computing requires a detailed comparison of infrastructure alternatives, taking a number of aspects into careful consideration. Existing methods of evaluation, however, limit decision making to the relative costs of cloud computing, but do not take a broader range of criteria into account. In this paper, we introduce a generic, multi-criteria-based decision framework and an application for Cloud Computing, the Multi-Criteria Comparison Method for Cloud Computing ((MC2)2). The framework and method allow organizations to determine what infrastructure best suits their needs by evaluating and ranking infrastructure alternatives using multiple criteria. Therefore, (MC2)2 offers a way to differentiate infrastructures not only by costs, but also in terms of benefits, opportunities and risks. (MC2)2 can be adapted to facilitate a wide array of decision-making scenarios within the domain of information technology infrastructures, depending on the criteria selected to support the framework.

cs.DC