---
title: "Accessing elevation data in R with the elevatr package"
author: "Jeffrey W. Hollister"
date: '`r Sys.Date()`'
output:
  html_document:
    theme: readable
    toc: yes
    toc_float: yes
vignette: >
  %\VignetteIndexEntry{Introduction to elevatr} 
  %\VignetteEngine{knitr::rmarkdown} 
  %\VignetteEncoding{UTF-8}
---


```{r setup, include=FALSE, echo=FALSE}
################################################################################
#Load packages
################################################################################
library("sp")
library("raster")
library("knitr")
library("elevatr")
library("httr")
NOT_CRAN <- identical(tolower(Sys.getenv("NOT_CRAN")), "true")
knitr::opts_chunk$set(purl = NOT_CRAN, 
                      eval = NOT_CRAN,
                      fig.width = 5, 
                      fig.height = 5, 
                      tidy = TRUE)
```

```{r environ, echo=FALSE}
key <- readRDS("../tests/testthat/key_file.rds")
Sys.setenv(mapzen_key=key)
```

# Introduction to `elevatr`

Elevation data is used for a wide array of applications, including, for example, visualization, hydrology, and ecological modelling.  Gaining access to these data in R has not had a single interface, is made available through functions across many packages, or requires local access to the data.  This is no longer required as a variety of APIs now exist that provide programmatic access to elevation data.  The `elevatr` package was written to standarize access to elevation data from web APIs.  This introductory vignette provides details on how to use `elevatr` to access elevation data and provides a bit of detail on the source data it accesses.

There are currently four endpoints that `elevatr` accesses.  For point elevation data it uses USGS Elevation Point Query Service and Mapzen Elevation Service and to access raster elevation data (e.g. a DEM) it uses Mapzen Terrain Service, and the Amazon Web Services Terrain Tiles.

# Get Point Elevation Data

Point elevation is accesses from `get_elev_point()`.  This function takes either a data.frame with  x (longitude) and y (latitude) locations as the first two columns  or a SpatialPoints/SpatialPointsDataFrame as input and then fetches the reported elevation for that location.  As mentioned there are two services that provide this information.  Details and use for those are provided below.

## Mapzen Elevation Service 

The [Mapzen Elevation Service](https://mapzen.com/documentation/elevation/elevation-service/) provides point elevations that are currently derived from the  [SRTM](http://www2.jpl.nasa.gov/srtm/), [GMTED](http://topotools.cr.usgs.gov/gmted_viewer/) and [GEBCO](http://www.gebco.net/data_and_products/gridded_bathymetry_data/) digital elevation models.  The web API provides a variety of input and output sources, but `elevatr` only uses the shape list and returns only the elevation.  The [range parameter](https://mapzen.com/documentation/elevation/elevation-service/#get-height-and-distance-with-the-range-parameter) may be implemented at some point in the future.

### API Keys

A Mapzen API Key is not required to use this service, but the rate limit is 1000 requests per day, 6 per minute, and 1 per second which is impractical for typical uses, thus it is suggested that you get and use an API Key.  With the API Key the rate limit is 2 queries per second and 20000 queries per day.

Get an API Key from <https://mapzen.com/developers>.  Since `elevatr` will search the environment for the appropriate key, it is suggested that you store this in your `.Renviron` file.  To do this:

```{r api_key, eval=FALSE}
cat("mapzen_key=mapzen-XXXXXXX\n",
    file=file.path(normalizePath("~/"), ".Renviron"),
    append=TRUE)
```

Once this is set for a given machine there is no need to include the API key in the functions themselves as the default is to use the `mapzen_key` environment variable.

### Using `get_elev_point()` to Access The Mapzen Elevation Service

Usage of `get_elev_point()` requires an input SpatialPoints, SpatialPointsDataFrame, or a two-column data frame with column one containing the x (e.g. longitude) coordinates and the second column containing the y coordinates (e.g. latitude).  The source data are global and also include estimates of depth for oceans.

Example usage of each is included below.  For these examples, we can create a dataset to use.

```{r example_dataframe}
# Create an example data.frame
set.seed(65.7)
examp_df <- data.frame(x = runif(10, min = -73, max = -71), 
                       y = runif(10, min = 41 , max = 45))
prj_dd <- "+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs"

# Create and example data.frame with additional columns
cats <- data.frame(category = c("H", "H", "L", "L", "L", 
                                "M", "H", "L", "M", "M"))

examp_df2 <- data.frame(examp_df, cats)

# Create an example SpatialPoints
examp_sp <- SpatialPoints(examp_df, proj4string = CRS(prj_dd))

# Create an example SpatialPointsDataFrame
examp_spdf <- SpatialPointsDataFrame(examp_sp, proj4string = CRS(prj_dd), data = cats )
```

If a data frame is used it may have additional columns beyond the first two, which must contain the coordinates.  The additional columns, along with the returned elevation, will be part of the output SpatialPointsDataFrame.  Similarly, an elevation column is added to the data slot of a SpatialPointsDataFrame.

```{r examples_df1}
# Example using data.frame with longitude and latitude
df_elev <- get_elev_point(examp_df, prj = prj_dd, src = "mapzen")

# Compare
examp_df
data.frame(df_elev)
```

```{r sleep1, echo=F}
Sys.sleep(2)
```

```{r examples_df2}
# Example using data.frame with longitude, latitude and an additional column
df2_elev <- get_elev_point(examp_df2, prj = prj_dd, src = "mapzen")

# Compare
examp_df2
data.frame(df2_elev)
```

```{r sleep2, echo=F}
Sys.sleep(2)
```

The process is the same for a SpatialPoints and a SpatialPointsDataFrame object.  The SpatialPointsDataFrame example is shown below.

```{r examples_sp2}
# Example using SpatialPointsDataFrame
# prj is taken from the SpatialPointsDataFrame object
# api_key is taken from environment variable mapzen_key
spdf_elev <- get_elev_point(examp_spdf)

# Compare
examp_spdf
spdf_elev
```

## USGS Elevation Point Query Service 

The [USGS Elevation Point Query Service](http://ned.usgs.gov/epqs/) is also accessible from `elevatr`.  This service is included as it provides elevation from a single source (as opposed to the [muliple sources from Mapzen](https://mapzen.com/documentation/elevation/elevation-service/#data-sources-and-known-issues)) and provides higher precision elevation.  It is only available for the United States (including Alaska and Hawaii).  Points that fall within the United States but are not on land return a value of zero.  Points outside the United States boundaries return a value of -1000000.

### Using `get_elev_point()` to Access The USGS Elevation Point Query Service

The USGS Elevation Point Query Service returns a single point at a time.  The implemntation in `get_elev_point()` will loop through each point, thus can be slow for large number of requests.  In those instances, the Mapzen source would be preferred.

Accessing data from this service is done by setting the `src` to `"epqs"`.  No API key is required and there are no rate limits.

```{r}
df_elev_epqs <- get_elev_point(examp_df, prj = prj_dd, src = "epqs")
data.frame(df_elev_epqs)
df2_elev_epqs <- get_elev_point(examp_df2, prj = prj_dd, src = "epqs")
data.frame(df2_elev_epqs)
sp_elev_epqs <- get_elev_point(examp_sp, src = "epqs")
sp_elev_epqs
spdf_elev_epqs <- get_elev_point(examp_spdf, src = "epqs")
spdf_elev_epqs
```

# Get Raster Elevation Data

While point elevations are useful, they will not provide the information required for most elevation based analysis such as hydrologic modeling, viewsheds, etc.  To do that requires a raster digital elevation model (DEM).  There are several sources for digital elevation models such as the Shuttle Radar Topography Mission (SRTM), the USGS National Elevation Dataset (NED), Global DEM (GDEM), and others.  Each of these DEMs has pros and cons for their use.  Recently, Mapzen has combined [several of these sources](https://mapzen.com/documentation/terrain-tiles/data-sources/) to create a synthesis elevation product that utilizes the best available elevation data for a given region at given zoom level.  Additionally, the elevation data are enhanced with the inclusion of bathymetry in oceans from ETOPO1.  These data are made available through two separate APIs: the [Mapzen Terrain Tile Service](https://mapzen.com/documentation/terrain-tiles/) and the [Terrain Tiles on Amazon Web Services](https://aws.amazon.com/public-datasets/terrain/).  

The input for `get_elev_raster()` is a data.frame with  x (longitude) and y (latitude) locations as the first two columns, any `sp` object, or any `raster` object and it returns a RasterLayer of the tiles that overlap the bounding box of the input.  If multiple tiles are retrieved, the resultant output is a merged Raster Layer.  Details for each service and their usage via `get_elev_raster()` are provided below.

## Mapzen Terrain Tile Service

The [Mapzen Terrain Tile Service](https://mapzen.com/documentation/terrain-tiles/) provides access to global terrain tiles in 4 separate formats: terrarium, normal, geotiff, and skadi.  Details on each is provided in the [API documentation](https://mapzen.com/documentation/terrain-tiles/formats/).  The only format accessed via `elevatr` is the geotiff format.  

### Caching, API Keys, and rate limits

The Mapzen Terrain Tile Service is cached and as such it will provide quickest access to the data in most cases.  Additionally, you can access the service with or without the use of a key, but using a key is preferred and will allow you to view your usage of the service via <https://mapzen.com/developers>.   Additionally, the key will allow Mapzen to better understand and support use of this service.  There are no rate limits.

### Using `get_elev_raster()` to access the Mapzen Terrain Tile Service

As mentioned a data frame with x and y columns, a `sp` object, or a `raster` object needs be the input and the `src` needs to be set to "mapzen" (this is the default).  As with the Elevation service, the `api_key` will be taken from the `mapzen_key` environment variable and if that is not available it will be set to `NULL`.  

There is no difference in using the `sp` and `raster` input data types.  The data frame requires a `prj`.   We show examples using a `SpatialPolygonsDataFrame` and a data frame.  The zoom level (`z`) defaults to 9 (a trade off between resolution and time for download), but different zoom levels are often desired.  For example:  

```{r get_raster}
# SpatialPolygonsDataFrame example
data(lake)
elevation <- get_elev_raster(lake,z = 9)
plot(elevation)
plot(lake, add=TRUE)

# data.frame example
elevation_df <- get_elev_raster(examp_df,prj=prj_dd, z = 5)
plot(elevation_df)
plot(examp_sp, add = TRUE)
```

The zoom level determines the resolution of the output raster.  More details on resolution and zoom level is available in the [Mapzen Documentation on ground resolution](https://mapzen.com/documentation/terrain-tiles/data-sources/#what-is-the-ground-resolution).

In addition the the required arguments (`locations`, `z`, and  `prj` for data frames), and arguments with default values (`api_key`), several additional arguments may be passsed to `get_elev_raster()`.  First, the `expand` argument is provided to expand the size of the bounding box by a given value in map units.  This is useful when bounding box coordinates are near the edge of an xyz tile. For example:

```{r expand}
# Bounding box on edge
elev_edge<-get_elev_raster(lake, z = 10, api_key = key)
plot(elev_edge)
plot(lake, add = TRUE)

# Use expand to grab additional tiles
elev_expand<-get_elev_raster(lake, z = 10, expand = 1500, api_key = key)
plot(elev_expand)
plot(lake, add = TRUE)
```

Lastly, `...` provides the ability to pass additional arguments to `httr::GET` which is used to access the API endpoints.  While any `httr::GET` arguments may be used, this will most likely be used to pass on configuration arguments such as `httr::timeout()` or `httr::verbose()` via a named argument, `config` to `httr::GET`.  The `httr::timeout()` can be used to increase the timeout if downloads are timing out.  For instance:

```{r timeout}
# Increase timeout:
get_elev_raster(lake, z = 5, config = timeout(5), api_key = key)
```

Lastly, multiple configurations may be passed.  Below is an example combining `httr::timeout()` with `httr::verbose()`.

```{r timeout_verbose}
# Increase timeout:
get_elev_raster(lake, z = 5, config = c(verbose(),timeout(5)), api_key = key)
```

## Terrain Tiles on Amazon Web Services (AWS)

The data behind [Terrain Tiles on Amazon Web Services (AWS)](https://aws.amazon.com/public-datasets/terrain/) is the same as that available through Mapzen; however, the two services were built with different use cases in mind and thus have a few differences.  First, the AWS Terrain Tiles are not cached and in most cases can take longer to access than the Mapzen tiles.  Although, if you are accessing the data from AWS it will be faster as the data are served up through [S3](https://aws.amazon.com/s3/). There are also no API keys required for using the AWS Terrain Tiles. 

### Using `get_elev_raster()` to access the Terrain Tiles on AWS

The only difference between using the AWS Terrain Tiles and Mapzen is in specifying the source.  All other functionality and optional arguments are the same as with Mapzen.  So to use the AWS service:

```{r aws}
elevation <- get_elev_raster(lake,z = 9, src = "aws")
plot(elevation)
plot(lake, add=TRUE)
```

