arXiv subjects

Dirk Eddelbuettel

Publications and source records attributed to Dirk Eddelbuettel.

11 recordsLinked to original sources

arXiv2023

CRAN Task Views: The Next Generation

CRAN Task Views have been available on the Comprehensive R Archive Network since 2005. They provide guidance about which CRAN packages are relevant for tasks related to a certain topic, and can also facilitate automatic installation of all corresponding packages. Motivated by challenges from the growth of CRAN and the R community as a whole since 2005, all of the task views infrastructure and workflows were rethought and relaunched in 2021/22 in order to facilitate maintenance and to foster deeper interactions with the R community. The redesign encompasses the establishment of a group of CRAN Task View Editors, moving all task view sources to dedicated GitHub repositories, adopting well-documented workflows with a code of conduct, and leveraging R/Markdown files (rather than XML) for the content of the task views.

stat.CO↗

arXiv2023

mlpack 4: a fast, header-only C++ machine learning library

For over 15 years, the mlpack machine learning library has served as a "swiss army knife" for C++-based machine learning. Its efficient implementations of common and cutting-edge machine learning algorithms have been used in a wide variety of scientific and industrial applications. This paper overviews mlpack 4, a significant upgrade over its predecessor. The library has been significantly refactored and redesigned to facilitate an easier prototyping-to-deployment pipeline, including bindings to other languages (Python, Julia, R, Go, and the command line) that allow prototyping to be seamlessly performed in environments other than C++. mlpack is open-source software, distributed under the permissive 3-clause BSD license; it can be obtained at https://mlpack.org

cs.MS↗

arXiv2022

Redis for Market Monitoring

This note shows how to use Redis cache (near-)real-time market data, and utilise its publish/subscribe ("pub/sub") facility to distribute the data.

stat.CO↗

arXiv2022

A Brief Introduction to Redis

This note provides a brief introduction to Redis highlighting its usefulness in multi-lingual statistical computing.

stat.CO↗

arXiv2021

Binary R Packages for Linux: Past, Present and Future

Pre-compiled binary packages provide a convenient way of efficiently distributing software that has been adopted by most Linux package management systems. However, the heterogeneity of the Linux ecosystem, combined with the growing number of R extensions available, poses a scalability problem. As a result, efforts to bring binary R packages to Linux have been scattered, and lack a proper mechanism to fully integrate them with R's package manager. This work reviews past and present of binary distribution for Linux, and presents a path forward by showcasing the `cran2copr' project, an RPM-based proof-of-concept implementation of an automated scalable binary distribution system with the capability of building, maintaining and distributing thousands of packages, while providing a portable and extensible bridge to the system package manager.

stat.CO↗

arXiv2020

The Rockerverse: Packages and Applications for Containerization with R

The Rocker Project provides widely used Docker images for R across different application scenarios. This article surveys downstream projects that build upon the Rocker Project images and presents the current state of R packages for managing Docker images and controlling containers. These use cases cover diverse topics such as package development, reproducible research, collaborative work, cloud-based data processing, and production deployment of services. The variety of applications demonstrates the power of the Rocker Project specifically and containerisation in general. Across the diverse ways to use containers, we identified common themes: reproducible environments, scalability and efficiency, and portability across clouds. We conclude that the current growth and diversification of use cases is likely to continue its positive impact, but see the need for consolidating the Rockerverse ecosystem of packages, developing common practices for applications, and exploring alternative containerisation software.

cs.SE↗

arXiv2020

Parallel Computing With R: A Brief Review

Parallel computing has established itself as another standard method for applied research and data analysis. The R system, being internally constrained to mostly singly-threaded operations, can nevertheless be used along with different parallel computing approaches. This brief review covers OpenMP and Intel TBB at the cpu- and compiler level, moves to process-parallel approaches before discussing message-passing parallelism and big data technologies for parallel processing such as Spark, Docker and Kubernetes before concluding with a focus on the future package integrating many of these approaches.

stat.CO↗

arXiv2020

An R Autograder for PrairieLearn

We describe how we both use and extend the PrarieLearn framework by taking advantage of its built-in support for external auto-graders. By using a custom Docker container, we can match our course requirements perfectly. Moreover, by relying on the flexibility of the interface we can customize our Docker container. A specific extension for unit testing is described which creates context-dependent difference between student answers and reference solution providing a more comprehensive response at test time.

stat.OT↗

arXiv2019

Thirteen Simple Steps for Creating An R Package with an External C++ Library

We desribe how we extend R with an external C++ code library by using the Rcpp package. Our working example uses the recent machine learning library and application 'Corels' providing optimal yet easily interpretable rule lists which we bring to R in the form of the 'RcppCorels' package. We discuss each step in the process, and derive a set of simple rules and recommendations which are illustrated with the concrete example.

stat.CO↗

arXiv2017

An Introduction to Rocker: Docker Containers for R

We describe the Rocker project, which provides a widely-used suite of Docker images with customized R environments for particular tasks. We discuss how this suite is organized, and how these tools can increase portability, scaling, reproducibility, and convenience of R users and developers.

cs.SE↗

arXiv2014

RProtoBuf: Efficient Cross-Language Data Serialization in R

Modern data collection and analysis pipelines often involve a sophisticated mix of applications written in general purpose and specialized programming languages. Many formats commonly used to import and export data between different programs or systems, such as CSV or JSON, are verbose, inefficient, not type-safe, or tied to a specific programming language. Protocol Buffers are a popular method of serializing structured data between applications - while remaining independent of programming languages or operating systems. They offer a unique combination of features, performance, and maturity that seems particularly well suited for data-driven applications and numerical computing. The RProtoBuf package provides a complete interface to Protocol Buffers from the R environment for statistical computing. This paper outlines the general class of data serialization requirements for statistical computing, describes the implementation of the RProtoBuf package, and illustrates its use with example applications in large-scale data collection pipelines and web services.

stat.CO↗