arXiv subjects

Ben Carrillo

Publications and source records attributed to Ben Carrillo.

3 recordsLinked to original sources

arXiv2025

denet, A lightweight command-line tool for process monitoring in benchmarking and beyond

Summary: denet is a lightweight process monitoring tool providing real-time resource profiling of running processes. It reports CPU, memory, disk I/O, network activity, and thread usage, including recursive child monitoring, with adaptive sampling rates. denet exposes two interfaces: a CLI with colorized output and a Python API. Output format is JSONL. Each record carries performance metrics and process metadata (PID and the executed command). Structured output makes denet suitable for benchmarking and tuning data-intensive pipelines in bioinformatics and beyond. We provide CLI and API examples, including a bioinformatics workflow in Snakemake, showcasing denet's diagnostic properties. Availability and implementation: denet is open-source software released under the GPLv3 terms, and maintained at https://github.com/btraven00/denet. It is implemented in Rust, with Python bindings provided via maturin, and installs from Cargo (cargo install denet) or PyPI (pip install denet). Most functionality does not require administrative privileges, enabling use on cloud platforms, HPC clusters, and standard Linux workstations. Advanced features such as eBPF support may require elevated permissions.

cs.PF↗

arXiv2024

Omnibenchmark: transparent, reproducible, extensible and standardized orchestration of solo and collaborative benchmarks

Benchmarking involves designing, running and disseminating rigorous performance assessments of methods, most often for data analysis and software tools, but the process can also be applied to experimental systems. Ideally, a benchmarking system is used to facilitate the benchmarking process by providing a structured entrypoint to design, coordinate, execute, and store standardized benchmarks. We describe a novel benchmarking system, Omnibenchmark, that facilitates benchmark formalization and execution in both solo and community efforts. Omnibenchmark provides a flexible benchmark plan syntax (i.e., a configuration YAML file), dynamic workflow generation based on Snakemake, S3-compatible storage handling, and reproducible software environments using environment modules, Apptainer or Conda. Such a setup provides an unprecedented flexibility such that existing benchmark designs can be forked and extended, run separately or collaboratively, giving versioned and standardized result outputs and therefore much-needed transparency to the analysis and interpretation of benchmark results. Tutorials and installation instructions are available from https://omnibenchmark.org.

q-bio.OT↗

arXiv2024

Building a continuous benchmarking ecosystem in bioinformatics

Benchmarking, which involves collecting reference datasets and demonstrating method performances, is a requirement for the development of new computational tools, but also becomes a domain of its own to achieve neutral comparisons of methods. Although a lot has been written about how to design and conduct benchmark studies, this Perspective sheds light on a wish list for a computational platform to orchestrate benchmark studies. We discuss various ideas for organizing reproducible software environments, formally defining benchmarks, orchestrating standardized workflows, and how they interface with computing infrastructure.

q-bio.OT↗