par_multirasters {chopin} | R Documentation |
Parallelize spatial computation over multiple raster files
Description
Large raster files usually exceed the memory capacity in size.
This function can be helpful to process heterogenous raster files with
homogeneous summary functions. Heterogenous raster files refer to
rasters with different spatial extents and resolutions.
Cropping a large raster into a small subset even consumes
a lot of memory and adds processing time.
This function leverages terra
SpatRaster
to distribute computation jobs over multiple threads.
It is assumed that users have multiple large raster files
in their disk, then each file path is assigned to a thread.
Each thread will directly read raster values from
the disk using C++ pointers that operate in terra functions.
For use, it is strongly recommended to use vector data with
small and confined spatial extent for computation to avoid
out-of-memory error. y
argument in fun_dist
will be used as-is.
That means no preprocessing or subsetting will be
applied. Please be aware of the spatial extent and size of the
inputs.
Usage
par_multirasters(filenames, fun_dist, ..., .debug = FALSE)
Arguments
filenames |
character. A vector or list of full file paths of raster files. n is the total number of raster files. |
fun_dist |
terra or chopin functions that accept |
... |
Arguments passed to the argument |
.debug |
logical(1). Default is |
Value
a data.frame object with computation results.
For entries of the results,
consult the function used in fun_dist
argument.
Author(s)
Insang Song geoissong@gmail.com
See Also
future::multisession
, future::multicore
, future::cluster
,
future.mirai::mirai_multisession
, future::plan
, par_convert_f
Other Parallelization:
par_cut_coords()
,
par_grid()
,
par_grid_mirai()
,
par_hierarchy()
,
par_hierarchy_mirai()
,
par_make_grid()
,
par_merge_grid()
,
par_multirasters_mirai()
,
par_pad_balanced()
,
par_pad_grid()
,
par_split_list()
Examples
lastpar <- par(mfrow = c(1, 1))
library(terra)
library(sf)
library(future)
library(future.mirai)
options(sf_use_s2 = FALSE)
future::plan(future.mirai::mirai_multisession, workers = 2)
nccnty <- sf::st_read(
system.file("shape/nc.shp", package = "sf")
)
nccnty <- sf::st_transform(nccnty, "EPSG:5070")
nccntygrid <- sf::st_make_grid(nccnty, n = c(200, 100))
nccntygrid <- sf::st_as_sf(nccntygrid)
nccntygrid$GEOID <- sprintf("%05d", seq_len(nrow(nccntygrid)))
nccntygrid <- sf::st_intersection(nccntygrid, nccnty)
rrast <- terra::rast(nccnty, nrow = 600, ncol = 1320)
terra::values(rrast) <- rgamma(7.92e5, 4, 2)
tdir <- tempdir(check = TRUE)
testpath1 <- file.path(tdir, "test1.tif")
testpath2 <- file.path(tdir, "test2.tif")
terra::writeRaster(rrast, testpath1, overwrite = TRUE)
terra::writeRaster(rrast, testpath2, overwrite = TRUE)
testfiles <- list.files(tdir, pattern = "tif$", full.names = TRUE)
res <- par_multirasters(
filenames = testfiles,
fun_dist = extract_at,
x = testpath1,
y = nccnty,
id = "GEOID",
func = "mean"
)
future::plan(future::sequential)
mirai::daemons(0)
par(lastpar)