logvogel

Overview

Tracking progress from parallel processes is hard, e.g. when running a foreach::foreach() loop. One recommended solution is to print the status to an external logfile. This is the strategy of logvogel: it provides an easy way to monitor long-running loops from an external R process using logfiles. Simultaneous write access to the logfile from parallel processes is prevented using the lock mechanism from seagull.

Installation

devtools::install_github("cszang/logvogel")

Usage

In the foreach construct

Use logfile() to create a new logfile. This creates an R6 class instance, whose handling is similar to dplyr::progress_estimated(). The resulting logfile can be printed, updated with .$update(), and removed with .$remove().

Example:

library(logvogel)
library(foreach)
library(parallel)
library(doSNOW)

cl <- makeCluster(detectCores())
registerDoSNOW(cl)

n <- 100
loop_log <- logfile("loop.log", n)

foreach(i = 1:n, .packages = c("logvogel")) %dopar% {
  Sys.sleep(rnorm(1, mean = 4))
  loop_log$update()
}

loop_log$remove()

In the monitoring process

Start another R process, preferably in the same root directory. Then use logvogel::status() to track the status of the loop.

Example:

logvogel::status("loop.log")

Alternatively, autostatus() finds all files with the extension .log in the project root and asks for which one the status should be tracked.

Package name

The name logvogel is a German pun on the word "Lockvogel", which means "decoy".