Download several datasets given their URLs and destination path

Source: R/mf_download_data.R

This function enables to download datasets. In a data import workflow, this function is typically used after a call to the mf_get_url function. The output value of mf_get_url can be used as input of parameter df_to_dl of mf_download_data.

The download can the parallelized.

mf_download_data(
  df_to_dl,
  path = tempfile("modisfast_"),
  parallel = FALSE,
  num_workers = parallel::detectCores() - 1,
  credentials = NULL,
  verbose = TRUE,
  min_filesize = 5000,
  ...
)

Arguments

df_to_dl

data.frame. Urls and destination files of dataset to download. Typically output of mf_get_url. See Details for the structure

path

string. Target folder for the data to download. Default : temporary folder.

parallel

boolean. Parallelize the download ? Default to FALSE

num_workers

integer. Number of workers in case of parallel download. Default to number of workers available in the machine minus one.

credentials

vector string of length 2 with username and password. optional if the function mf_login was previously executed.

verbose

boolean. optional. Verbose (default TRUE)

min_filesize

integer. Minimum file size expected (in bites) for one file downloaded. If files downloaded are less that this value, the files will be downloaded again. Default 5000.

...

not used

Value

a data.frame with the same structure of the input data.frame df_to_dl + columns providing details of the data downloaded. The additional columns are :

fileDl

Booloean (dataset downloaded or failure)

dlStatus

Download status : 1 = download ok ; 2 = download error ; 3 = dataset was already existing in destination file

fileSize

File size on disk

Details

Parameter df_to_dl must be a data.frame with the following minimal structure :

id_roi

An id for the ROI (character string)

collection

Collection (character string)

name
url

URL of the file to download (character string)

Examples


if (FALSE) {

### Login to EOSDIS Earthdata with your username and password
log <- mf_login(credentials = c("earthdata_un","earthdata_pw"))

### Set-up parameters of interest
coll <- "MOD11A1.061"

bands <- c("LST_Day_1km","LST_Night_1km")

time_range <- as.Date(c("2017-01-01","2017-01-30"))

roi <- sf::st_as_sf(data.frame(
id = "roi_test",
geom="POLYGON ((-5.82 9.54, -5.42 9.55, -5.41 8.84, -5.81 8.84, -5.82 9.54))"),
wkt="geom",crs = 4326)

### Get the URLs of the data
(urls_mod11a1 <- mf_get_url(
collection = coll,
variables = bands,
roi = roi,
time_range = time_range
))

### Download the data
res_dl <- mf_download_data(urls_mod11a1)

### Import the data as terra::SpatRast
modis_ts <- mf_import_data(dirname(res_dl$destfile[1]), collection = coll)

### Plot the data
terra::plot(modis_ts)

}