arXiv subjects

Youssef Moawad

Publications and source records attributed to Youssef Moawad.

6 recordsLinked to original sources

arXiv2026

How Accurately Can the Energy Use of Spark Applications Be Estimated Based on Resource Utilisation?

Distributed batch data processing applications are widely executed on cloud-based resources where restricted user access to node-level hardware energy counters hinders transparent sustainability accounting. Energy and carbon attribution methodologies therefore depend on power models and available resource utilisation traces, yet the accuracy of these estimates has to be validated while direct counters are available. In this work, we use Apache Spark running on Kubernetes as a case-study dataflow runtime and cluster resource manager to compare model-based energy estimates to Intel RAPL package and DRAM energy on an AWS bare-metal cloud and an on-premises cluster, comparing different CPU usage signals and memory coefficients. We show that external monitoring improves signed package-energy error relative to Spark task traces, reducing underestimation from -29.58% to -24.41% on AWS and from -24.00% to -16.22% on-premises.

cs.DC↗

arXiv2026

Ichnos+: Estimating the Carbon Footprint of Scientific Workflows Using Fitted Power Models

As data-intensive scientific workflows scale to facilitate the automation of analysis of increasing amounts of data, their resource-intensive and long-running execution incurs significant energy consumption and carbon emissions. Given the already significant and rising emissions from the ICT sector, it is crucial to quantify and understand the carbon footprint of scientific workflows. However, existing tooling is commonly not usable in shared, virtualized environments or resorts to power models that are based on only one or two generic data points. To address this gap, this paper presents Ichnos+, a novel system to quantify the environmental footprint of Nextflow scientific workflows. Ichnos+ enables post-hoc footprint estimation based on existing workflow traces, node-specific power models for the computational resources utilized, and carbon intensity data aligned with the execution time. We evaluate Ichnos+ against hardware-level energy measurements obtained using Intel RAPL, and the nf-core co2footprint plugin, which implements the Green Algorithms methodology. We find that Ichnos+ is capable of estimating workflow energy consumption with an estimation error of 10.8% across three compute clusters, significantly outperforming the nf-core plugin. We further show that Ichnos+ extends beyond operational carbon to estimate embodied emissions as well as water and land use. Finally, we demonstrate how Ichnos+ can be extended for another workflow system, Apache Airflow, maintaining a similarly high degree of estimation accuracy.

cs.DC↗

arXiv2025

Spatio-Temporal Shifting to Reduce Carbon, Water, and Land-Use Footprints of Cloud Workloads

In this paper, we investigate the potential of spatial and temporal cloud workload shifting to reduce carbon, water, and land use footprints. Specifically, we perform a simulation study leveraging publicly available data on the cloud infrastructure of major providers (AWS and Azure) as well as real-world workload traces (big data analytics and FaaS) and grid mix data to consider two different scenarios. Our simulation results indicate that spatial shifting can substantially lower carbon, water, and land use footprints. In the FaaS applications, shifting the spatiotemporal workload achieves carbon savings of up to 85%, water savings of around 50%, and reductions in land use of up to 45%, all while optimizing for the respective factors. Mixed optimization yields results comparable to those of land use alone. For big data workloads, spatiotemporal shifting delivers reductions of up to 45% in carbon emissions, 40% in water consumption, and nearly 40% in land use when optimized for the respective factors. Temporal shifting also decreases the footprint, though to a lesser extent. When applied together, the two strategies yield the greatest overall reduction, driven mainly by spatial shifting with temporal adjustments providing an additional, incremental benefit. Sensitivity analysis demonstrates that such shifting is robust to prediction errors in grid mix data and to variations across different seasons.

cs.DC↗

arXiv2025

A Systematic Evaluation of the Potential of Carbon-Aware Execution for Scientific Workflows

Scientific workflows are critical to scientific data analysis and often involve computationally intensive processing of large datasets on compute clusters. As such, their execution tends to be long-running and resource-intensive, resulting in significant energy consumption and carbon emissions. While carbon-aware computing methods have received considerable attention in general cloud contexts, their application to scientific data analysis workflows remains a critical research gap. Our study addresses this oversight by showing how the delay tolerance, interruptibility, and scalability of scientific workflows can be leveraged for a significantly more sustainable execution model. In this study, we first quantify the problem of carbon emissions associated with running scientific workflows, and then demonstrate the transformative potential for carbon-aware workflow execution. We estimate the carbon footprint of seven real-world Nextflow workflows executed on diverse dedicated cluster and public cloud resources using high-resolution average and marginal grid carbon intensity data from open and commercial data providers. Furthermore, we conduct a systematic evaluation of the impact of carbon-aware temporal shifting, and the dynamic pausing and resuming of the workflow. Moreover, we investigate the impact of resource scaling at both workflow and workflow task levels. Finally, we report substantial potential reductions in overall carbon emissions, with temporal shifting capable of decreasing emissions by over 80%, and resource scaling by 67%.

cs.DC↗

arXiv2024

Optimising Iteration Scheduling for Full-State Vector Simulation of Quantum Circuits on FPGAs

As the field of quantum computing grows, novel algorithms which take advantage of quantum phenomena need to be developed. As we are currently in the NISQ (noisy intermediate scale quantum) era, quantum algorithm researchers cannot reliably test their algorithms on real quantum hardware, which is still too limited. Instead, quantum computing simulators on classical computing systems are used. In the quantum circuit model, quantum bits (qubits) are operated on by quantum gates. A quantum circuit is a sequence of such quantum gates operating on some number of qubits. A quantum gate applied to a qubit can be controlled by other qubits in the circuit. This applies the gate only to the states which satisfy the required control qubit state. We particularly target FPGAs as our main simulation platform, as these offer potential energy savings when compared to running simulations on CPUs/GPUs. In this work, we present a memory access pattern to optimise the number of iterations that need to be scheduled to execute a quantum gate such that only the iterations which access the required pairs (determined according to the control qubits imposed on the gate) are scheduled. We show that this approach results in a significant reduction in the time required to simulate a gate for each added control qubit. We also show that this approach benefits the simulation time on FPGAs more than CPUs and GPUs and allows to outperform both CPU and GPU platforms in terms of energy efficiency, which is the main factor for scalability of the simulations.

quant-ph↗

arXiv2022

Transformations for accelerator-based quantum circuit simulation in Haskell

For efficient hardware-accelerated simulations of quantum circuits, we can define hardware-specific quantum-circuit transformations. We use a functional programming approach to create a quantum-circuit analysis and transformation method implemented in Haskell. This tool forms a key part of our larger quantum-computing simulation toolchain. As an example of hardware acceleration, we discuss FPGA-based simulations of selected quantum arithmetic circuits, including the transformation steps to optimise the hardware utilisation. Future development steps in the Haskell-based analysis and transformation tool are outlined. The described toolchain can be found on GitHub: https://github.com/DevdudeSami/fqt.

quant-ph↗