In this example we want to download, import and plot over the 3500 km2 wide region of interest (mapped below) :
- a 1 year-long time series of weekly land surface temperature from VIIRS (spatial resolution : 1 km ; temporal resolution : 8 day),
- the same 1 year-long times series of monthly precipitations from Global Precipitation Measurement (GPM) (spatial resolution : 1° ; temporal resolution : 1 month)
Check which collections are available
Before starting, let’s identify our collections of interest among the
collections that are available for download, using the function
mf_list_collections() :
collections <- mf_list_collections()
print(str(collections))
#> 'data.frame': 95 obs. of 27 variables:
#> $ collection : chr "GPM_3IMERGDE.06" "GPM_3IMERGDF.06" "GPM_3IMERGDF.07" "GPM_3IMERGDL.06" ...
#> $ collection_id : chr "" "" "" "" ...
#> $ access_type : chr "GES DISC OPeNDAP" "GES DISC OPeNDAP" "GES DISC OPeNDAP" "GES DISC OPeNDAP" ...
#> $ source : chr "GPM" "GPM" "GPM" "GPM" ...
#> $ long_name : chr "GPM IMERG Early Precipitation L3 1 day 0.1 degree x 0.1 degree V06" "GPM IMERG Final Precipitation L3 1 day 0.1 degree x 0.1 degree V06" "GPM IMERG Final Precipitation L3 1 day 0.1 degree x 0.1 degree V07" "GPM IMERG Late Precipitation L3 1 day 0.1 degree x 0.1 degree V06" ...
#> $ nature : chr "Rainfall" "Rainfall" "Rainfall" "Rainfall" ...
#> $ provider : chr "NASA GES DISC" "NASA GES DISC" "NASA GES DISC" "NASA GES DISC" ...
#> $ url_metadata : chr "https://disc.gsfc.nasa.gov/datasets/GPM_3IMERGDE_06/summary" "https://disc.gsfc.nasa.gov/datasets/GPM_3IMERGDF_06/summary" "https://disc.gsfc.nasa.gov/datasets/GPM_3IMERGDF_07/summary" "https://disc.gsfc.nasa.gov/datasets/GPM_3IMERGDL_06/summary" ...
#> $ doi : chr "https://doi.org/10.5067/GPM/IMERGDE/DAY/06" "https://doi.org/10.5067/GPM/IMERGDF/DAY/06" "https://doi.org/10.5067/GPM/IMERGDF/DAY/07" "https://doi.org/10.5067/GPM/IMERGDL/DAY/06" ...
#> $ version : num 6.1 6.1 7 6.1 6.1 7 6.1 6.1 6.1 7 ...
#> $ spatial_resolution_m : num 10000 10000 10000 10000 10000 10000 10000 10000 10000 10000 ...
#> $ temporal_resolution : num 1 1 1 1 30 30 30 30 1 1 ...
#> $ temporal_resolution_unit: chr "day" "day" "day" "day" ...
#> $ spatial_coverage : chr "Global" "Global" "Global" "Global" ...
#> $ start_date : chr "2000-06-01" "2000-06-01" "2000-06-01" "2000-06-01" ...
#> $ end_date : chr "ongoing" "ongoing" "ongoing" "ongoing" ...
#> $ indicative_latency_days : num 1 100 100 1 100 100 0 1 100 100 ...
#> $ url_manual_access : chr "https://search.earthdata.nasa.gov/search?q=GPM_3IMERGDE_06" "https://search.earthdata.nasa.gov/search?q=GPM_3IMERGDF_06" "https://search.earthdata.nasa.gov/search?q=GPM_3IMERGDF_07" "https://search.earthdata.nasa.gov/search?q=GPM_3IMERGDL_06" ...
#> $ status : chr "Implemented" "Implemented" "Implemented" "Implemented" ...
#> $ login : chr "earthdata" "earthdata" "earthdata" "earthdata" ...
#> $ url_programmatic_access : chr "https://gpm1.gesdisc.eosdis.nasa.gov/opendap/GPM_L3/GPM_3IMERGDE.06/" "https://gpm1.gesdisc.eosdis.nasa.gov/opendap/GPM_L3/GPM_3IMERGDF.06/" "https://gpm1.gesdisc.eosdis.nasa.gov/opendap/GPM_L3/GPM_3IMERGDF.07/" "https://gpm1.gesdisc.eosdis.nasa.gov/opendap/GPM_L3/GPM_3IMERGDL.06/" ...
#> $ url_stac : chr "" "" "" "" ...
#> $ dim_lon : chr "lon" "lon" "lon" "lon" ...
#> $ dim_lat : chr "lat" "lat" "lat" "lat" ...
#> $ dim_time : chr "time" "time" "time" "time" ...
#> $ dim_proj : chr "" "" "" "" ...
#> $ crs : chr "+proj=longlat +datum=WGS84 +no_defs +type=crs" "+proj=longlat +datum=WGS84 +no_defs +type=crs" "+proj=longlat +datum=WGS84 +no_defs +type=crs" "+proj=longlat +datum=WGS84 +no_defs +type=crs" ...
#> NULLSo our collections of interest are :
- VJ221A2.002 : VIIRS/JPSS2 Land Surface Temperature/Emissivity 8-Day L3 Global 1km SIN Grid V002
- GPM_3IMERGM.07 : GPM IMERG Final Precipitation L3 1 month 0.1 degree x 0.1 degree V07
Setup the region and time range of interest
First we prepare the script : define the ROI, the time frame, and the folder were the data will be downloaded :
# Set ROI and time range of interest
roi <- st_as_sf(data.frame(id="Korhogo", geom="POLYGON ((-5.82 9.54, -5.42 9.55, -5.41 8.84, -5.81 8.84, -5.82 9.54))"), wkt="geom", crs = 4326)
time_range <- as.Date(c("2024-01-01","2024-12-31"))Configure an Earthdata token
Create a token in your Earthdata profile and set it in R:
Sys.setenv(EARTHDATA_TOKEN = "your Earthdata bearer token")Get the URLs of the data to download
With the function mf_get_url(), we get the https URLs
for our collections of interest (VJ221A2.002 and
GPM_3IMERGDF.07) given our ROI and time frame.
:arrow_right: For each collection, we can select a subset only of
variables (sometimes called “dimensions”, or “bands”) that are of
interest for us, through the parameter variables. By
default, this parameter is set to NULL, which means that
all the available variables for the specified collection are included.
To get the variables available for a given collection along with
information for each of them (description, etc.), use the function
mf_list_variables().
For more information on the variables in the context of OPeNDAP, see the OPeNDAP Data model documentation.
## Get the URLs of VIIRS LST datacubes
urls_viirs <- mf_get_url(
collection = "VJ221A2.002",
variables = c("LST_Day_1KM","LST_Night_1KM"), # get the available variables with : mf_list_variables("VJ221A2.002")
roi = roi,
time_range = time_range )
#> Building the URLs... This may take a few moments.
## Get the URLs of GPM daily
urls_gpm <- mf_get_url(
collection = "GPM_3IMERGM.07",
variables = c("precipitation"), # get the available variables with : mf_list_variables("GPM_3IMERGM.07")
roi = roi,
time_range = time_range )
#> Building the URLs... This may take a few moments.
#> Estimated maximum size of data to be downloaded is 1 Mb
nrow(urls_viirs)
#> [1] 46
head(urls_viirs,3)
#> id_roi time_start collection
#> 1 Korhogo 2024-01-01 VJ221A2.002
#> 2 Korhogo 2024-01-09 VJ221A2.002
#> 3 Korhogo 2024-01-17 VJ221A2.002
#> name
#> 1 VJ221A2.A2024001.h17v08.002.2025285041924_LST_Day_1KM-LST_Night_1KM.nc4
#> 2 VJ221A2.A2024009.h17v08.002.2025285060600_LST_Day_1KM-LST_Night_1KM.nc4
#> 3 VJ221A2.A2024017.h17v08.002.2025296200949_LST_Day_1KM-LST_Night_1KM.nc4
#> url
#> 1 https://opendap.earthdata.nasa.gov/collections/C2831655321-LPCLOUD/granules/VJ221A2.A2024001.h17v08.002.2025285041924.nc4?/HDFEOS/GRIDS/VIIRS_Grid_8Day_1km_LST21/Data_Fields/LST_Day_1KM%5B53:139%5D%5B511:558%5D,/HDFEOS/GRIDS/VIIRS_Grid_8Day_1km_LST21/Data_Fields/LST_Night_1KM%5B53:139%5D%5B511:558%5D
#> 2 https://opendap.earthdata.nasa.gov/collections/C2831655321-LPCLOUD/granules/VJ221A2.A2024009.h17v08.002.2025285060600.nc4?/HDFEOS/GRIDS/VIIRS_Grid_8Day_1km_LST21/Data_Fields/LST_Day_1KM%5B53:139%5D%5B511:558%5D,/HDFEOS/GRIDS/VIIRS_Grid_8Day_1km_LST21/Data_Fields/LST_Night_1KM%5B53:139%5D%5B511:558%5D
#> 3 https://opendap.earthdata.nasa.gov/collections/C2831655321-LPCLOUD/granules/VJ221A2.A2024017.h17v08.002.2025296200949.nc4?/HDFEOS/GRIDS/VIIRS_Grid_8Day_1km_LST21/Data_Fields/LST_Day_1KM%5B53:139%5D%5B511:558%5D,/HDFEOS/GRIDS/VIIRS_Grid_8Day_1km_LST21/Data_Fields/LST_Night_1KM%5B53:139%5D%5B511:558%5D
#> grid_nrow grid_ncol maxFileSizeEstimated
#> 1 1200 1200 33408
#> 2 1200 1200 33408
#> 3 1200 1200 33408
nrow(urls_gpm)
#> [1] 12
head(urls_gpm,3)
#> id_roi time_start collection
#> 1 Korhogo 2024-01-01 GPM_3IMERGM.07
#> 2 Korhogo 2024-02-01 GPM_3IMERGM.07
#> 3 Korhogo 2024-03-01 GPM_3IMERGM.07
#> name
#> 1 3B-MO.MS.MRG.3IMERG.20240101-S000000-E235959.01.V07B.nc4
#> 2 3B-MO.MS.MRG.3IMERG.20240201-S000000-E235959.02.V07B.nc4
#> 3 3B-MO.MS.MRG.3IMERG.20240301-S000000-E235959.03.V07B.nc4
#> url
#> 1 https://gpm1.gesdisc.eosdis.nasa.gov/opendap/GPM_L3/GPM_3IMERGM.07/2024/3B-MO.MS.MRG.3IMERG.20240101-S000000-E235959.01.V07B.HDF5.nc4?precipitation%5B0:0%5D%5B1742:1746%5D%5B989:996%5D,time%5B0:0%5D,lon%5B1742:1746%5D,lat%5B989:996%5D
#> 2 https://gpm1.gesdisc.eosdis.nasa.gov/opendap/GPM_L3/GPM_3IMERGM.07/2024/3B-MO.MS.MRG.3IMERG.20240201-S000000-E235959.02.V07B.HDF5.nc4?precipitation%5B0:0%5D%5B1742:1746%5D%5B989:996%5D,time%5B0:0%5D,lon%5B1742:1746%5D,lat%5B989:996%5D
#> 3 https://gpm1.gesdisc.eosdis.nasa.gov/opendap/GPM_L3/GPM_3IMERGM.07/2024/3B-MO.MS.MRG.3IMERG.20240301-S000000-E235959.03.V07B.HDF5.nc4?precipitation%5B0:0%5D%5B1742:1746%5D%5B989:996%5D,time%5B0:0%5D,lon%5B1742:1746%5D,lat%5B989:996%5D
#> grid_nrow grid_ncol maxFileSizeEstimated
#> 1 NA NA 112
#> 2 NA NA 112
#> 3 NA NA 112Download the data
Now we download the data with the function
mf_download_data().
The destination file for each dataset is specified in the column destfile of the dataframes urls_viirs and urls_gpm. A destination file is specified by default but it can of course be modified.
Setting the argument parallel to TRUE will
parallelize - therefore fasten - the download in case numerous datasets
need to be downloaded.
# Hint : to fasten the download in case of multiple collections, bind them in a single data.frame (as shown below) before executing the function 'mf_download_data()' :
df_to_dl <- rbind(urls_viirs, urls_gpm)
# Then download the data (by default, in temporary directory, but the target folder can be set with the argument 'path'
res_dl <- mf_download_data(df_to_dl = df_to_dl,
parallel = TRUE,
num_workers = 3) # too many workers may crash the download
#> 58 datasets in total : 0 already downloaded and 58 datasets to download
#> Downloading the data (destination folder: /tmp/RtmpCKkpmn/modisfast_5bd82d02dd51 ) ...
#> Estimated maximum size of data to be downloaded is ~ 2 Mb
#>
#> Actual size of downloaded data is 2 Mb
print(str(res_dl))
#> 'data.frame': 58 obs. of 12 variables:
#> $ id_roi : chr "Korhogo" "Korhogo" "Korhogo" "Korhogo" ...
#> $ time_start : Date, format: "2024-01-01" "2024-01-09" ...
#> $ collection : chr "VJ221A2.002" "VJ221A2.002" "VJ221A2.002" "VJ221A2.002" ...
#> $ name : chr "VJ221A2.A2024001.h17v08.002.2025285041924_LST_Day_1KM-LST_Night_1KM.nc4" "VJ221A2.A2024009.h17v08.002.2025285060600_LST_Day_1KM-LST_Night_1KM.nc4" "VJ221A2.A2024017.h17v08.002.2025296200949_LST_Day_1KM-LST_Night_1KM.nc4" "VJ221A2.A2024025.h17v08.002.2025296214251_LST_Day_1KM-LST_Night_1KM.nc4" ...
#> $ url : chr "https://opendap.earthdata.nasa.gov/collections/C2831655321-LPCLOUD/granules/VJ221A2.A2024001.h17v08.002.2025285"| __truncated__ "https://opendap.earthdata.nasa.gov/collections/C2831655321-LPCLOUD/granules/VJ221A2.A2024009.h17v08.002.2025285"| __truncated__ "https://opendap.earthdata.nasa.gov/collections/C2831655321-LPCLOUD/granules/VJ221A2.A2024017.h17v08.002.2025296"| __truncated__ "https://opendap.earthdata.nasa.gov/collections/C2831655321-LPCLOUD/granules/VJ221A2.A2024025.h17v08.002.2025296"| __truncated__ ...
#> $ grid_nrow : int 1200 1200 1200 1200 1200 1200 1200 1200 1200 1200 ...
#> $ grid_ncol : int 1200 1200 1200 1200 1200 1200 1200 1200 1200 1200 ...
#> $ maxFileSizeEstimated: num 33408 33408 33408 33408 33408 ...
#> $ destfile : chr "/tmp/RtmpCKkpmn/modisfast_5bd82d02dd51/data/Korhogo/VJ221A2.002/VJ221A2.A2024001.h17v08.002.2025285041924_LST_D"| __truncated__ "/tmp/RtmpCKkpmn/modisfast_5bd82d02dd51/data/Korhogo/VJ221A2.002/VJ221A2.A2024009.h17v08.002.2025285060600_LST_D"| __truncated__ "/tmp/RtmpCKkpmn/modisfast_5bd82d02dd51/data/Korhogo/VJ221A2.002/VJ221A2.A2024017.h17v08.002.2025296200949_LST_D"| __truncated__ "/tmp/RtmpCKkpmn/modisfast_5bd82d02dd51/data/Korhogo/VJ221A2.002/VJ221A2.A2024025.h17v08.002.2025296214251_LST_D"| __truncated__ ...
#> $ fileDl : logi TRUE TRUE TRUE TRUE TRUE TRUE ...
#> $ fileSize : num 45393 45456 45263 45286 45401 ...
#> $ dlStatus : num 1 1 1 1 1 1 1 1 1 1 ...
#> NULLImport the data in R
Various packages and related classes can be used to read the data
downloaded through OPeNDAP. If terra
(successor of raster)
is surely the most famous class for raster objects, many packages
facilitate the use of spatiotemporal data cubes in formats such as those
proposed through modisfast (e.g. NetCDF). For instance,
MODIS or VIIRS products can be imported as a stars object
from the excellent stars
package for data cubes manipulation. All the data can also be imported
as ncdf4 objects using e.g. the ncdf4
package.
:warning: :warning: :warning: :warning:
Important warning regarding the use of the data after download
through modisfast
Care must be taken when importing data that was downloaded through
the OPeNDAP data providers servers. Depending on the collection, some
“issues” were raised. These issues are independent from
modisfast : they result most of time of a kind of lack of
full implementation of the OPeNDAP framework by the data providers.
End warning
These issues can easily be dealt at the import phase in R using the
function mf_import_data(). This function enables to open
the data in R as a SpatRaster object (from the package terra)
:
# import VJ221A2.002
# argument 'path' is the folder where the data are stored (here, in a temporary directory)
viirs_ts <- mf_import_data(path = dirname(list.files(path = tempdir(), pattern = "VJ221A2", recursive = TRUE, full.names = TRUE)[1]),
collection = "VJ221A2.002",
proj_epsg = 4326)
#> Importing the dataset as a SpatRaster object...
# import GPM
gpm_ts <- mf_import_data(path = dirname(list.files(path = tempdir(), pattern = "3IMERG", recursive = TRUE, full.names = TRUE)[1]),
collection = "GPM_3IMERGM.07",
proj_epsg = 4326)
#> Importing the dataset as a SpatRaster object...
viirs_ts
#> class : SpatRaster
#> size : 86, 50, 92 (nrow, ncol, nlyr)
#> resolution : 0.008415894, 0.008415894 (x, y)
#> extent : -5.8225, -5.401705, 8.834566, 9.558333 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84 (EPSG:4326)
#> source(s) : memory
#> names : 2024-~y_1KM, 2024-~t_1KM, 2024-~y_1KM, 2024-~t_1KM, 2024-~y_1KM, 2024-~t_1KM, ...
#> min values : 307.620697, 287.07901, 310.001068, 288.411194, 309.903412, 286.311157, ...
#> max values : 320.594025, 298.429474, 320.995361, 297.353882, 320.342346, 298.213135, ...
#> time (days) : 2024-01-01 to 2024-12-26 (46 steps)
gpm_ts
#> class : SpatRaster
#> size : 8, 5, 12 (nrow, ncol, nlyr)
#> resolution : 0.1, 0.09999997 (x, y)
#> extent : -5.8, -5.3, 8.9, 9.7 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84 (EPSG:4326)
#> sources : 3B-MO.MS.MRG.3IMERG.20240101-S000000-E235959.01.V07B.nc4
#> 3B-MO.MS.MRG.3IMERG.20240201-S000000-E235959.02.V07B.nc4
#> 3B-MO.MS.MRG.3IMERG.20240301-S000000-E235959.03.V07B.nc4
#> ... and 9 more sources
#> varnames : precipitation
#> precipitation
#> precipitation
#> precipitation
#> precipitation
#> ...
#> names : preci~ation, preci~ation, preci~ation, preci~ation, preci~ation, preci~ation, ...
#> unit : mm/hr
#> time : 2024-01-01 00:00:00zUTC to 2024-12-01 00:00:00zUTC (12 steps)
