R: Apply-like function for distribution across MPI-based...

srapply {ShortRead}

R Documentation

Apply-like function for distribution across MPI-based clusters.

Description

This lapply like function evaluates locally or, if Rmpi or parallel is loaded (and Rmpi workers spawned), across nodes in a cluster. Errors in evaluation of FUN generate warnings; results are trimmed to exclude results where the error occurs.

Usage

srapply(X, FUN, ..., fapply = .fapply(), reduce = .reduce(),
        USE.NAMES = FALSE, verbose = FALSE)

Arguments

`X`	Tasks to be distributed. `X` should be an object for which `lapply` or `sapply` are defined (more precisely, `mpi.parLapply`, `mpi.parSapply`, or `mclapply`). Performance is best when these objects are relatively small, e.g., file names, compared to the work to be done on each by `FUN`.
`FUN`	A function to be applied to each element of `X`. The function must have `...` or named argument `verbose` in its signature. It is best if it makes no reference to variables other than those in its argument list. or in loaded packages (the ShortRead package is available on remote nodes).
`...`	Additional arguments, passed to `FUN`.
`fapply`	An optional argument defining an `lapply`-like function to be used in partitioning `X`. See details, below.
`reduce`	Optional function accepting a list (the result of `fapply` and summarizing this. The default reports errors in function evaluation as warnings, returning the remaining values as elements of a list. See details below for additional hints.
`USE.NAMES`	If `TRUE` and if `X` is character, use `X` as `names` for the result unless it had names already.
`verbose`	Report whether evaluation is local or mpi-based; also forwarded to `FUN`, allowing detailed reports from remote instances.

Details

The default value for fapply is available with ShortRead:::.fapply(). It tests the getOption("srapply_fapply") for value “Rmpi” or “parallel”.

If Rmpi is indicated, fapply ensures that ShortRead is required on all workers, and then invokes mpi.parLapply with arguments X, FUN, ..., and verbose. The function FUN is wrapped so that errors are returned as objects of class SRError with type RemoteError. If no workers are available, the code evaluates FUN so that errors are reported as with remote evaluation.

If parallel is indicated, fapply invokes mclapply with arguments as for mpi.parLapply.

Custom reduce functions might be written as reduce=function(lst) unlist(lst, use.names=TRUE).

Value

The returned value depends on the value of reduce, but by default is a list with elements containing the results of FUN applied to each of X. Evaluations resulting in an error have been removed, and a warning generated.

Author(s)

Martin Morgan <mtmorgan@fhcrc.org>

Examples

## ... or 'verbose' required in argument, 
srapply(1:10, function(i, ...) i)
## collapse result to vector
srapply(1:10, function(i, ...) i, reduce=unlist)
x <- srapply(1:10, function(i, ...) {
    if (runif(1)<.2) stop("oops") else i
})
length(x) ## trimmed to exclude errors

[Package ShortRead version 1.12.4 Index]