---
title: "Investigating selected specifications"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Investigating selected specifications}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  #fig.path = "man/figures/README-",
  out.width = "100%",
  fig.retina = 2
)
```

Per default, `run_specs()` summarizes individual specifications by using `broom::tidy()`. For most cases, this provides "good-enough" summaries of the relationship of interest. Sometimes, however, it might be useful to investigate specific models in more detail or to investigate a specific parameter that is not provided by `broom::tidy()` (e.g., r-square, AIC). This vignette shows how to access individual models.

## 1. Run specification curve analysis

If we want to investigate individual models and particularly all aspects of that model, we need to use `run_specs()` with the additional argument `keep.results = TRUE`. 

```{r, message=F, warning = F}
library(specr)
library(dplyr)
library(purrr)
library(ggplot2)
library(performance)

# Run specification curve analysis
results <- run_specs(df = example_data, 
                     y = c("y1", "y2"), 
                     x = c("x1", "x2"), 
                     model = c("lm"), 
                     controls = c("c1", "c2"), 
                     subsets = list(group1 = unique(example_data$group1),
                                    group2 = unique(example_data$group2)),
                     keep.results = TRUE)
```


## 2. Identify model(s) of interest

For this example, we are going to look at two specific models (same independent variables, all controls, all participants, but different dependent variables).

```{r}
(y_models <- results %>%
  filter(x == "x1", 
         controls == "c1 + c2",
         subsets == "all"))
```

As you can see, the resulting tibble includes an additional column called "res". This column includes the entire "model object" and we can use it to further investigate each model. 

## 3. Investigate models

For example, we can now easily get a full summary of the models and compare individual coefficients and statistics. 

```{r}
y_models %>%
  pull(res) %>%
  map(summary) %>%
  map(coef)
```

Or we could get r-squared values for both models.

```{r}
y_models %>%
  pull(res) %>% 
  map(r2)       # r2 is include in the package "performance"
```


## 4. Some more examples

This way, we can analyze or compare such statistics across several models.

```{r, fig.height=8, fig.width=10, message=F, warning = F}
r2_results <- results %>%
  filter(subsets == "all") %>%
  mutate(r2 = map(res, r2), 
         r2 = map_dbl(r2, 1)) %>%
  arrange(r2)

r2_results %>%
  select(x:controls, r2)
```

And we can plot comparisons.

```{r}
r2_results %>%
  arrange(r2) %>%
  mutate(rank = 1:n()) %>%
  ggplot(aes(x = rank, 
             y = r2)) +
  geom_point() +
  geom_line() +
  theme_minimal() +
  theme(strip.text = element_blank(),
        axis.line = element_line("black", size = .5),
        axis.text = element_text(colour = "black"))
```

