The Crew Pkg Apr 2026

It is, in essence, a . And it changes the game for production-level R code. The Problem crew Solves (That You Didn't Know You Had) Traditional parallel backends in R share a common flaw: they are often too "chatty" or too fragile. foreach with doParallel works, but it forks processes, which can crash on Windows or with large objects. future is elegant, but its nested parallelism and persistent-worker logic can be tricky to debug.

But crew (which stands for oordinated R esource E xecution W orker) isn't just another entry in the parallel-processing catalog. Created by William Landau, the author of the targets package, crew is a fundamental rethink of how R should talk to background jobs.

That’s it. The controller sits in your main R session. You push tasks to it, and it distributes them to persistent, resilient R sessions running in the background. # Non-blocking push controller$push( name = "long_compute", command = slow_function(data) ) Collect results later result <- controller$pop() the crew pkg

With crew :

controller <- crew_controller_local(workers = 8) controller$start() for (file in all_files) { controller$push( name = file, command = process_file(file) ) } results <- list() while (controller$pop()$name != "done") { Crew auto-replaces crashed workers results <- c(results, controller$pop()$result) } It is, in essence, a

For HPC users: Replace crew_controller_local() with crew_controller_slurm() and define your job submission template. The API remains identical.

library(crew) controller <- crew_controller_local( name = "my_cluster", workers = 4, tasks_max = 100 # Auto-restart workers after 100 tasks ) Start the workers controller$start() foreach with doParallel works, but it forks processes,

For analysts running one-off scripts, the overhead of learning crew might not be worth it. But for data scientists building automated reports, for bioinformaticians processing thousands of genomes, and for production pipelines that must run at 3 AM without failing— crew is quietly becoming the gold standard.

tar_option_set( controller = crew_controller_local(workers = 10) ) Suddenly, your pipeline is running across a fleet of auto-healing workers without changing a single analysis step. crew is not a parallel engine itself. It is a controller specification that leverages two incredibly fast lower-level packages: mirai (for asynchronous task execution) and nanonext (for low-level networking).