---
title: "varycoef: An R Package to Model Spatially Varying Coefficients"
author: "Jakob A. Dambon"
date: "October 2019"
output: 
  rmarkdown::html_vignette:
    toc: true
vignette: >
  %\VignetteIndexEntry{First Example}
  %\VignetteEngine{knitr::rmarkdown}
  %\usepackage[utf8]{inputenc}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE, message = FALSE, 
                      fig.width=7, fig.height=4)
```


## Introduction

With the release of the R package `varycoef` on CRAN ([see](https://CRAN.R-project.org/package=varycoef)) we enable you to analyze your spatial data and in a simple, yet versatile way. The underlying idea are *spatially varying coefficients*, short SVC, which have been studied over the last decades. These models are based on the linear regression model and are therefore very easy to interpret. Due to their design, SVC models offer a high flexibility. These two properties allow for better understanding of the data, gaining new insights and, ultimately, better predictive performance. 

In this blog post, we will show what SVC models are and how to define them. Afterwards, we give a short and illustrative example on how to apply these tools in `varycoef` using the well known data set `meuse` from the package `sp`. 

**Disclaimer**
This analysis is meant to be a show case of what is possible using the package `varycoef`, not to write the most rigorous statistical analysis of a data set.


But first, let us start with the set up and let me show you where to find help: 

```{r install}
# install from CRAN
# install.packages("varycoef")

# attach package
library(varycoef)

# general package help file (check out examples with manual vignette!!)
help("varycoef")
```

There is also a version of `varycoef` on [Github](https://github.com/jakobdambon/varycoef) from which you can get the latest devel version. 

### Preliminaries
Before we start, we want to give some prerequisites in order to understand the analysis below. Beside a classical linear regression model, we require the knowledge of geostatistics and tools thereof, like:

- what is the advantage of geostatistics?
- two-dimensional Gaussian processes, i.e. Gaussian random fields
- covariance functions and how the range and variance parameter influence them
- maximum likelihood estimation




### What are Spatially Varying Coefficient Models?

SVC models are a generalization of the classical linear regression model. In matrix notation with a response vector $\mathbf y$, a data matrix $\mathbf X$, and a vector $\boldsymbol \varepsilon$ containing the errors, a linear regression model is defined as

$$\mathbf y = \mathbf X \boldsymbol \beta + \boldsymbol \varepsilon.$$

In geostatistics, the error term usually is called the nugget. SVC models are introduced through a random effect $\boldsymbol \eta(s)$ that is depending on the location $s$. The underlying model takes the form 

$$\mathbf y = \mathbf X \boldsymbol \beta + \mathbf W \boldsymbol \eta (s) + \boldsymbol  \varepsilon$$
 where $\mathbf W$ parses the covariates to the random effect $\boldsymbol \eta(s)$. Specifically, each SVC is defined by a Gaussian process, more specifically a  Gaussian random field (GRF), with mean zero and some stationary covariance function. This function usually depends on some marginal variance (sill) and a range. An example of 3 SVC, i.e. 3 GRFs, and the nugget $\boldsymbol \varepsilon$ on a regular grid is given in the figure below.
 
 
```{r SVC example}
# setting seed
set.seed(123)
# number of SVC
p <- 3
# sqrt of total number of observations
m <- 20
# covariance parameters
(pars <- data.frame(var = c(0.1, 0.2, 0.3), 
                    scale = c(0.3, 0.1, 0.2)))
nugget.var <- 0.05

# function to sample SVCs
sp.SVC <- fullSVC_reggrid(m = m, p = p, 
                          cov_pars = pars, 
                          nugget = nugget.var)

library(sp)
# visualization of sampled SVC
spplot(sp.SVC, colorkey = TRUE)
```

Just by looking at the figure, one can see that

1. SVC_1, SVC_2 and SVC_3 have some spatial clustering...
2. ..., whereas the nugget does not.
3. The largest variance can be observed in SVC_2 and SVC_3.
4. The largest spatial clusters are in SVC_1.
5. They all vary around 0, which is expected as they are all zero mean GRF.

### Goal of SVC Modelling

Given the response $\mathbf y$, the data matrices $\mathbf X$ and $\mathbf W$, we want to estimate the coefficients $\boldsymbol \beta$ as we would with a linear regression model, and further, try to understand the properties of $\boldsymbol \eta(s)$. Another important aspect is that we are able to predict $\boldsymbol \eta(s')$ for some new locations $s'$. This is what we are about to do in the following sections. 

## Meuse Data Set Example

### The Data Set

As mentioned, the data set `meuse` from the package `sp` is quite well known. 

```{r meuse intro}
# attach sp and load data
library(sp)
data("meuse")

# documentation
help("meuse")

# overview
summary(meuse)
dim(meuse)
```

What we are interested in are the `cadmium` measurements. On the plots below, the data is skewed, which is why we are going to work on the natural logarithm of `cadmium`.

```{r hist plot}
par(mfrow = 1:2)
# histogram of (log) cadmium 
hist(meuse$cadmium); hist(log(meuse$cadmium))
par(mfrow = c(1, 1))

meuse$l_cad <- log(meuse$cadmium)
```

Adding the coordinate reference system (CRS, see `help(meuse)`) and the sampling locations, one can obtain a spatial plot of the log cadmium measurements.

```{r spatial plot}
# construct spatial object
sp.meuse <- meuse
coordinates(sp.meuse) <- ~x+y
proj4string(sp.meuse) <- CRS("+init=epsg:28992")

# using package tmap
library(tmap)
# producing an interactive map
tmap_leaflet(tm_shape(sp.meuse) + tm_dots("l_cad", style = "cont"))
```

Generally, the values for `l_cad` are highest close to the river Meuse. However, there is some spatial clustering comparing the center of all observations to the Northern part. We will now try model the `l_cad` based on some of the covariates given in the data set `meuse`. 

### The Covariates

As independent variables, we choose the following:

- `dist`, i.e. the normalized distance to the river Meuse. 
- `lime`, which is a 2-level factor indicating the presence of lime. 
- `elev`, i.e. the relative elevation above the local river bed.

I am not a geologist, but these covariates seem like a sensible choice, if I try to predict cadmium at a given location without taking further measurements of cadmium or other highly correlated heavy metals like copper, lead, or zinc. 

## First Models

Given the covariates, we continue to fit the data with different models and corresponding methods. Hence, we work our way up starting with a linear regression and geostatistical model.


### Linear Regression Model

We start with a linear regression model (LM) and ordinary least squares (OLS).

```{r linear regression}
# linear model (LM)
lm.fit <- lm(l_cad ~ 1+dist+lime+elev, data = meuse)
summary(lm.fit)
```

The residual analysis shows:

```{r LM residuals}
oldpar <- par(mfrow = c(1, 2))
plot(lm.fit, which = 1:2)
par(oldpar)
```

The spatial distribution of the residuals is the following:

```{r LM spatial residuals}
# add LM residuals to data frame
sp.meuse$LM_res <- resid(lm.fit)
head(sp.meuse)
# plot residuals at corresponding locations
tmap_leaflet(tm_shape(sp.meuse) + tm_dots("LM_res", style = "cont"))
```


One can observe that there is a spatial clustering of the residuals. This motivates us to use geostatistics. 


### Geostatistical Model

The classical geostatistical model is a good reference and starting point for SVC models. Geostatistical models using a GRF are a special case of SVC models. More specifically, we want to model a single random effect and the data matrix $\mathbf W$ only contains the intercept, such that the SVC model simplifies to the known geostatistical model 

$$\mathbf y = \mathbf X \boldsymbol \beta +  \boldsymbol \eta (s) + \boldsymbol  \varepsilon.$$

Usually, we start by computing the empirical (semi-) variogramm and estimating a suitable model. Trying the exponential covariance function, we get the following:

```{r geostatistical}
library(gstat)
# empirical variogram
eV <- variogram(LM_res ~ 1, sp.meuse)
# define variogram model with initial values
mV <- vgm(0.2, "Exp", 300, 0.4)
# fit model 
(fV <- fit.variogram( eV, mV))
# plot empirical and fitted
plot(eV, model=fV)
```


The fit looks reasonable and the estimated values are (literally) good starting points for our methodology coming up in the next section. So we can do ordinary kriging on the residuals.

```{r ordinary kriging}
# study area
data("meuse.grid")
coordinates(meuse.grid) <- ~x+y
proj4string(meuse.grid) <- proj4string(sp.meuse)
# kriging
GS.fit <- krige(LM_res ~ 1, sp.meuse, 
                newdata = meuse.grid, model = fV)
# output
tmap_leaflet(tm_shape(GS.fit) + tm_dots("var1.pred", style = "cont"))
```

This seems to fit the residuals pretty good.


## The SVC Model

Now we want to see how the SVC model performs. We will use the same covariates. However, as for the locations, we will transform the Easting and Northing given in meters to kilometers. This is due to computational stability.

```{r varycoef preparation}
# response variable
y <- meuse$l_cad
# covariates for fixed effects
X <- model.matrix(~1+dist+lime+elev, data = meuse)
# locations
locs <- as.matrix(meuse[, 1:2])/1000
```

Our SVC model describes the data as follows: The fixed effects are the same as in the LM. For the random effects, we consider an intercept (as a geostatistical model would) as well as SVCs for the covariates `dist` and `lime`. The reason is to show that our methodology also applies to SVC models where we assume that some coefficients are spatially constant, i.e. we do not associate an SVC. In our case, this is the case for the covariate `elev`.

```{r varycoef random effect}
# covariates for SVC (random effects)
W <- model.matrix(~1+dist+lime, data = meuse)
```

The methodology we use is maximum likelihood estimation (MLE). It is implemented as a numerical optimization of the likelihood function. More specifically, we optimize over the covariance parameters associated to the SVC as well as the error variance. The optimization is therefore performed on the profile likelihood. 
As the MLE is a numerical optimization, we need starting values. Here, the parameters estimated in the geostatistical model come in handy and we will use them across all SVC.

```{r varycoef control}
# construct initial value (recall transformation from meters to kilometers)
(init <- c(
  # 3 times for 3 SVC
  rep(c(
    # range
    fV$range[2]/1000,
    # variance
    fV$psill[2]),     
  3), 
  # nugget
  fV$psill[1]
))
# control settings vor MLE
control <- SVC_mle_control(
  # profile likelihood optimization
  profileLik = TRUE,
  # initial values
  init = init
)
```

Now, we can start the MLE:

```{r varycoef fit}
# MLE
VC.fit <- SVC_mle(y = y, X = X, W = W, locs = locs,
                  control = control)
# outcome
summary(VC.fit)

# residuals
oldpar <- par(mfrow = c(1, 2))
plot(VC.fit, which = 1:2)
par(mfrow = c(1, 1))
plot(VC.fit, which = 3)
par(oldpar)
```

If we want to make some kind of predictions, we will have to transform the locations, as we did with the initial `meuse` data. 

```{r varycoef predict locations}
newlocs <- coordinates(meuse.grid)/1000
# prediciton
VC.pred <- predict(VC.fit, newlocs = newlocs)
# outcome
head(VC.pred)
# transformation to a spatial points data frame
colnames(VC.pred)[1:3] <- c("Intercept", "dist", "lime")
sp.VC.pred <- VC.pred
coordinates(sp.VC.pred) <- coordinates(meuse.grid)
proj4string(sp.VC.pred) <- proj4string(sp.meuse)

# points of interest (POI), we come back to them later
POI1.id <- 235
POI2.id <- 2016
POI1 <- meuse.grid[POI1.id, ]
POI2 <- meuse.grid[POI2.id, ]

tm_POIs <- 
  tm_shape(POI1) +
  # square (pch = 15)
  tm_symbols(shape = 15, col = "black") +
  tm_shape(POI2) +
  # triangle (pch = 17)
  tm_symbols(shape = 17, col = "black")
```


```{r,fig.height=6}
# tm.GS: GS fit
tm.GS <- tm_shape(GS.fit) + tm_dots("var1.pred", style = "cont")

# tm1: Intercept
tm1 <- tm_shape(sp.VC.pred) + 
  tm_dots("Intercept", style = "cont") +
  tm_POIs
# tm2: dist
tm2 <- tm_shape(sp.VC.pred) + 
  tm_dots("dist", style = "cont") +
  tm_POIs
# tm1: Intercept
tm3 <- tm_shape(sp.VC.pred) + 
  tm_dots("lime", style = "cont") +
  tm_POIs

tmap_arrange(list(tm.GS, tm1, tm2, tm3), ncol = 2, nrow = 2)
```

**For now, please ignore the black triangles and squares.**

In the upper left panel, one can see the GRF estimated and predicted by the classical geostatistical model. This can be compared to the upper right panel, i.e. the SVC associated with the intercept. In the bottom row we have the other two SVCs for `dist` and `lime`. I leave the comparison and interpretation of the results to the reader. What I want to point out is that the GRFs representing the SVCs are all inherently different. For instance, the ranges and variances differ substantially across all 3, as one can observe in the summary output. Further, as they all are zero mean Gaussian processes, the SVC are to be interpreted as deviations from the estimated means, which are given in the summary output above, or can be retrieved by:

```{r coef means}
coef(VC.fit)
```

So in the example above, this translates to the following. We have, say, two points of interest (POI) where we would like to receive the exact prediction equation for cadmium. These POIs are indicated with a black triangles and squares. We can look up the SVC estimates at thods locations to get the following two regression equations. 


```{r POI equations}
# POI1:
# SVC deviations at POI1
VC.pred[POI1.id, 1:3]
# combined
(coefs.poi1 <- coef(VC.fit) + c(as.numeric(VC.pred[POI1.id, 1:3]), 0))
# POI2
# SVC deviations at POI2
VC.pred[POI2.id, 1:3]
# combined
(coefs.poi2 <- coef(VC.fit) + c(as.numeric(VC.pred[POI2.id, 1:3]), 0))

options(digits = 3)
```

So, our two equations would be:

$$ y(s_{POI1}) = `r coefs.poi1[1]` + (`r coefs.poi1[2]`) \cdot dist + `r coefs.poi1[3]` \cdot lime1 + (`r coefs.poi1[4]`) \cdot elev$$

$$ y(s_{POI2}) = `r coefs.poi2[1]` + (`r coefs.poi2[2]`) \cdot dist + `r coefs.poi2[3]` \cdot lime1 + (`r coefs.poi2[4]`) \cdot elev$$

The coefficients are indeed different except for `elev`, as we did not associate an SVC in the model to it. For all other coefficients, we see that the estimated marginal effects of the covariates change over space. This is a very nice example why SVC models are easy to interpret. For a fixed location, they are linear regression models. As a reference, we give the regression equation as estimated with OLS:

$$ y = `r coef(lm.fit)[1]` + (`r coef(lm.fit)[2]`) \cdot dist + `r coef(lm.fit)[3]` \cdot lime1 + (`r coef(lm.fit)[4]`) \cdot elev.$$

## Conclusion

I hope that this example was easy to follow and showed the possibilities of how to model SVCs. I will continue my work on `varycoef` to add new and helpful features. On [Twitter](https://twitter.com/JakobDambon), I will use \#varycoef to announce updates. If there is a key feature missing or you have questions, please contact me by mailing to jakob.dambon (at) math.uzh.ch.

*Kind regards,*

*Jakob Dambon*

