---
title: "Time Series Machine Learning"
author: "Matt Dancho"
date: "`r Sys.Date()`"
output: 
  rmarkdown::html_vignette:
    toc: true
    toc_depth: 2
vignette: >
  %\VignetteIndexEntry{Time Series Machine Learning}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, echo = FALSE, message = FALSE, warning = FALSE}
knitr::opts_chunk$set(
    message = FALSE,
    warning = FALSE,
    fig.width = 8, 
    fig.height = 4.5,
    fig.align = 'center',
    out.width='95%', 
    dpi = 200
)

# devtools::load_all() # Travis CI fails on load_all()
```

> A collection of tools for working with time series in R

The time series signature is a collection of useful features that describe the time series index of a time-based data set. It contains a wealth of features that can be used to forecast time series that contain patterns. In this vignette, the user will learn methods to implement machine learning to predict future outcomes in a time-based data set. The vignette example uses a well known time series dataset, the Bike Sharing Dataset, from the UCI Machine Learning Repository. The vignette follows an example where we'll use `timetk` to build a basic Machine Learning model to predict future values using the time series signature. The objective is to build a model and predict the next six months of Bike Sharing daily counts.  


# Prerequisites

Before we get started, load the following packages.

```{r, message = FALSE}
library(workflows)
library(parsnip)
library(recipes)
library(yardstick)
library(glmnet)
library(tidyverse)
library(tidyquant)
library(timetk)
```

# Data

We'll be using the [Bike Sharing Dataset](https://archive.ics.uci.edu/ml/datasets/bike+sharing+dataset) from the UCI Machine Learning Repository. Download the data and select the "day.csv" file which is aggregated to daily periodicity.

_Source: Fanaee-T, Hadi, and Gama, Joao, 'Event labeling combining ensemble detectors and background knowledge', Progress in Artificial Intelligence (2013): pp. 1-15, Springer Berlin Heidelberg_

```{r}
# Read data
bikes <- read_csv("day.csv")

# Select date and count
bikes_tbl <- bikes %>%
    select(dteday, cnt) %>%
    rename(date  = dteday,
           value = cnt)
```

A visualization will help understand how we plan to tackle the problem of forecasting the data. We'll split the data into two regions: a training region and a testing region. 

```{r}
# Visualize data and training/testing regions
bikes_tbl %>%
    ggplot(aes(x = date, y = value)) +
    geom_rect(xmin = as.numeric(ymd("2012-07-01")),
              xmax = as.numeric(ymd("2013-01-01")),
              ymin = 0, ymax = 10000,
              fill = palette_light()[[4]], alpha = 0.01) +
    annotate("text", x = ymd("2011-10-01"), y = 7800,
             color = palette_light()[[1]], label = "Train Region") +
    annotate("text", x = ymd("2012-10-01"), y = 1550,
             color = palette_light()[[1]], label = "Test Region") +
    geom_point(alpha = 0.5, color = palette_light()[[1]]) +
    labs(title = "Bikes Sharing Dataset: Daily Scale", x = "") +
    theme_tq()
```

Split the data into train and test sets at "2012-07-01". 

```{r}
# Split into training and test sets
train_tbl <- bikes_tbl %>% filter(date < ymd("2012-07-01"))
test_tbl  <- bikes_tbl %>% filter(date >= ymd("2012-07-01"))
```


# Modeling

Start with the training set, which has the "date" and "value" columns.  

```{r}
# Training set
train_tbl
```

## Recipe Preprocessing Specification

The first step is to add the _time series signature_ to the training set, which will be used this to learn the patterns. New in `timetk` 0.1.3 is integration with the `recipes` R package:

- The `recipes` package allows us to add preprocessing steps that are applied sequentially as part of a data transformation pipeline. 

- The `timetk` has `step_timeseries_signature()`, which is used to add a number of features that can help machine learning models. 

```{r}
# Add time series signature
recipe_spec_timeseries <- recipe(value ~ ., data = train_tbl) %>%
    step_timeseries_signature(date) 
```

We can see what happens when we apply a prepared recipe `prep()` using the `bake()` function. Many new columns were added from the timestamp "date" feature. These are features we can use in our machine learning models. 

```{r}
bake(prep(recipe_spec_timeseries), new_data = train_tbl)
```

Next, I apply various preprocessing steps to improve the modeling behavior. If you wish to learn more, I have an [Advanced Time Series course](https://mailchi.mp/business-science/time-series-forecasting-course-coming-soon) that will help you learn these techniques.  

```{r}
recipe_spec_final <- recipe_spec_timeseries %>%
    step_rm(date) %>%
    step_rm(contains("iso"), contains("minute"), contains("hour"),
            contains("am.pm"), contains("xts")) %>%
    step_normalize(contains("index.num"), date_year) %>%
    step_dummy(contains("lbl"), one_hot = TRUE) %>%
    step_ns(date_index.num, deg_free = 3)

bake(prep(recipe_spec_final), new_data = train_tbl)
```


## Model Specification

Next, let's create a model specification. We'll use a `glmnet`. 

```{r}
model_spec_glmnet <- linear_reg(mode = "regression", penalty = 0.001, mixture = 0.5) %>%
    set_engine("glmnet")
```

## Workflow

We can mary up the preprocessing recipe and the model using a `workflow()`.

```{r}
workflow_glmnet <- workflow() %>%
    add_recipe(recipe_spec_final) %>%
    add_model(model_spec_glmnet)

workflow_glmnet
```

## Training

The workflow can be trained with the `fit()` function. 

```{r}
workflow_trained <- workflow_glmnet %>% fit(data = train_tbl)
```



## Test (Validation)

With a suitable model in hand, we can forecast using the "test" set for validation purposes. 

```{r}
prediction_tbl <- workflow_trained %>% 
    predict(test_tbl) %>%
    bind_cols(test_tbl) 

prediction_tbl
```


Visualize the results using `ggplot()`.

```{r}
ggplot(aes(x = date), data = bikes_tbl) +
    geom_rect(xmin = as.numeric(ymd("2012-07-01")),
              xmax = as.numeric(ymd("2013-01-01")),
              ymin = 0, ymax = 10000,
              fill = palette_light()[[4]], alpha = 0.01) +
    annotate("text", x = ymd("2011-10-01"), y = 7800,
             color = palette_light()[[1]], label = "Train Region") +
    annotate("text", x = ymd("2012-10-01"), y = 1550,
             color = palette_light()[[1]], label = "Test Region") + 
    geom_point(aes(x = date, y = value),  
               alpha = 0.5, color = palette_light()[[1]]) +
    # Add predictions
    geom_point(aes(x = date, y = .pred), data = prediction_tbl, 
               alpha = 0.5, color = palette_light()[[2]]) +
    theme_tq() 
    
```



## Validation Accuracy (Out of Sample)

The Out-of-Sample Forecast Accuracy can be measured with `yardstick`. 

```{r}
# Calculating forecast error
prediction_tbl %>% metrics(value, .pred)
```

Next we can visualize the residuals of the test set. The residuals of the model aren't perfect, but we can work with it. The residuals show that the model predicts low in October and high in December.    

```{r}
prediction_tbl %>%
    ggplot(aes(x = date, y = value - .pred)) +
    geom_hline(yintercept = 0, color = "red") +
    geom_point(color = palette_light()[[1]], alpha = 0.5) +
    geom_smooth() +
    theme_tq() +
    labs(title = "Test Set: GLM Model Residuals", x = "") +
    scale_y_continuous(limits = c(-5000, 5000))
```

At this point you might go back to the model and try tweaking features using interactions or polynomial terms, adding other features that may be known in the future (e.g. temperature of day can be forecasted relatively accurately within 7 days), or try a completely different modeling technique with the hope of better predictions on the test set. Once you feel that your model is optimized, move on the final step of forecasting. 

# Forecasting Future Data

Let's use our model to predict What are the expected future values for the next six months. The first step is to create the date sequence. Let's use `tk_get_timeseries_summary()` to review the summary of the dates from the original dataset, "bikes". 

```{r}
# Extract bikes index
idx <- bikes_tbl %>% tk_index()

# Get time series summary from index
bikes_summary <- idx %>% tk_get_timeseries_summary()
```

The first six parameters are general summary information.

```{r}
bikes_summary[1:6]
```

The second six parameters are the periodicity information.

```{r}
bikes_summary[7:12]
```

From the summary, we know that the data is 100% regular because the median and mean differences are 86400 seconds or 1 day. We don't need to do any special inspections when we use `tk_make_future_timeseries()`. If the data was irregular, meaning weekends or holidays were excluded, you'd want to account for this. Otherwise your forecast would be inaccurate.

```{r}
idx_future <- idx %>% tk_make_future_timeseries(n_future = 180)

future_tbl <- tibble(date = idx_future) 

future_tbl
```

Retrain the model specification on the full data set, then predict the next 6-months. 

```{r}
future_predictions_tbl <- workflow_glmnet %>% 
    fit(data = bikes_tbl) %>%
    predict(future_tbl) %>%
    bind_cols(future_tbl)
```



Visualize the forecast.

```{r}
bikes_tbl %>%
    ggplot(aes(x = date, y = value)) +
    geom_rect(xmin = as.numeric(ymd("2012-07-01")),
              xmax = as.numeric(ymd("2013-01-01")),
              ymin = 0, ymax = 10000,
              fill = palette_light()[[4]], alpha = 0.01) +
    geom_rect(xmin = as.numeric(ymd("2013-01-01")),
              xmax = as.numeric(ymd("2013-07-01")),
              ymin = 0, ymax = 10000,
              fill = palette_light()[[3]], alpha = 0.01) +
    annotate("text", x = ymd("2011-10-01"), y = 7800,
             color = palette_light()[[1]], label = "Train Region") +
    annotate("text", x = ymd("2012-10-01"), y = 1550,
             color = palette_light()[[1]], label = "Test Region") +
    annotate("text", x = ymd("2013-4-01"), y = 1550,
             color = palette_light()[[1]], label = "Forecast Region") +
    geom_point(alpha = 0.5, color = palette_light()[[1]]) +
    # future data
    geom_point(aes(x = date, y = .pred), data = future_predictions_tbl,
               alpha = 0.5, color = palette_light()[[2]]) +
    geom_smooth(aes(x = date, y = .pred), data = future_predictions_tbl,
                method = 'loess') + 
    labs(title = "Bikes Sharing Dataset: 6-Month Forecast", x = "") +
    theme_tq()
    
```

# Forecast Error

A forecast is never perfect. We need prediction intervals to account for the variance from the model predictions to the actual data. There's a number of methods to achieve this. We'll follow the [prediction interval](https://www.otexts.org/fpp/2/7) methodology from Forecasting: Principles and Practice.

```{r}
# Calculate standard deviation of residuals
test_resid_sd <- prediction_tbl %>%
    summarize(stdev = sd(value - .pred))

future_predictions_tbl <- future_predictions_tbl %>%
    mutate(
        lo.95 = .pred - 1.96 * test_resid_sd$stdev,
        lo.80 = .pred - 1.28 * test_resid_sd$stdev,
        hi.80 = .pred + 1.28 * test_resid_sd$stdev,
        hi.95 = .pred + 1.96 * test_resid_sd$stdev
    )
```

Now, plotting the forecast with the prediction intervals.

```{r}
bikes_tbl %>%
    ggplot(aes(x = date, y = value)) +
    geom_point(alpha = 0.5, color = palette_light()[[1]]) +
    geom_ribbon(aes(y = .pred, ymin = lo.95, ymax = hi.95), 
                data = future_predictions_tbl, 
                fill = "#D5DBFF", color = NA, size = 0) +
    geom_ribbon(aes(y = .pred, ymin = lo.80, ymax = hi.80, fill = key), 
                data = future_predictions_tbl,
                fill = "#596DD5", color = NA, size = 0, alpha = 0.8) +
    geom_point(aes(x = date, y = .pred), data = future_predictions_tbl,
               alpha = 0.5, color = palette_light()[[2]]) +
    geom_smooth(aes(x = date, y = .pred), data = future_predictions_tbl,
                method = 'loess', color = "white") + 
    labs(title = "Bikes Sharing Dataset: 6-Month Forecast with Prediction Intervals", x = "") +
    theme_tq()
```

 


# Parting Thoughts

Forecasting using the time series signature can be very accurate especially when time-based patterns are present in the underlying data. As with most machine learning applications, the prediction is only as good as the patterns in the data. Forecasting using this approach may _not_ be suitable when patterns are not present or when the future is highly uncertain (i.e. past is not a suitable predictor of future performance). However, in may situations the time series signature can provide an accurate forecast.

One benefit to the machine learning approach that was not covered in this vignette but is an significant advantage is that other features (including non-time-based) can be included in the analysis if the values are present in the training and test sets and can be determined with some level of accuracy in the future. For example, one can expect that experts in Bike Sharing analytics have access to historical temperature and weather patterns, wind speeds, and so on that could have a significant affect on bicycle sharing. The beauty of this method is these features can easily be incorporated into the model and prediction.

__Last, a few points on the modeling process - The following modeling steps are absolutely critical to developing forecasts that will return ROI to your company:__

- Preprocessing 
- Feature engineering using lagged variables 
- Hyperparameter Tuning
- Time series cross validation
- Using Multiple Modeling Techniques
- and more. 

These will be covered in my upcoming [__Advanced Time Series Course (Register Here)__](https://mailchi.mp/business-science/time-series-forecasting-course-coming-soon). 


