---
title: "The Outlier Model"
author: "Mads Lindskou"
output: 
  rmarkdown::html_vignette:
    highligh: zenburn
    toc: true
    fig_height: 4
    fig_width: 4
    fig_align: "center"
bibliography: refs.bib
vignette: >
  %\VignetteIndexEntry{The Outlier Model}
  %\VignetteEngine{knitr::rmarkdown}
  \usepackage[utf8]{inputenc}
---

```{r global_options, include=FALSE}
knitr::opts_chunk$set(echo = TRUE, warning = FALSE, message = FALSE, include = TRUE, eval = TRUE)
```

# Background Theory on The Outlier Detection Model

This vignette is an introduction to the "behind-the-scenes" in the paper @lindskououtlier

### Contingency Tables
The outlier detection model can only be used on discrete data; i.e data for which all variables can only take on a finite set of values (also called _levels_). We also say, that such variables are _discrete variables_ or _categorical variables_. 

<!-- Examples of such variables are  -->
<!-- ```{r echo = FALSE} -->
<!-- Variables <- c("gender", -->
<!--   "seasons", -->
<!--   "DNA markers", -->
<!--   "grades", -->
<!--   "race", -->
<!--   "treatment" -->
<!-- ) -->
<!-- Levels  <- c("male/female", -->
<!--   "spring/summer/autumn/winther", -->
<!--   "A/T/C/G", -->
<!--   "pass/fail", -->
<!--   "white/hispanic/black", -->
<!--   "active/placebo" -->
<!-- ) -->
<!-- discrete_examples <- data.frame(Variables, Levels) -->
<!-- pander::pander(discrete_examples) -->
<!-- ``` -->

Consider for example the built in discrete data set in `R` on `4526` applicants to graduate school at Berkeley for the six largest departments in 1973 classified by admission and sex. Below, a small sample of the data is seen.

```{r admission, echo = FALSE, message = FALSE, warning = FALSE}
## https://stat.ethz.ch/R-manual/R-patched/library/datasets/html/UCBAdmissions.html
library(dplyr)

adm_ <- as.data.frame(UCBAdmissions) 
adm <- data.frame(Admit = c(), Gender = c(), Dept = c())

for (i in 1:nrow(adm_)) {
  freq_i <- adm_[i, "Freq"]
  for (j in 1:freq_i) {
    adm <- rbind(adm, adm_[i, c("Admit", "Gender", "Dept")])
  }
}

adm <- adm[sample(nrow(adm)), ] %>%
  as_tibble() %>%
  mutate_all(.funs = as.character)

pander::pander(head(adm))
```

Another and more appropriate way of showing the entire data, is via a _contingency table_ which summarizes the counts of all combinations of `Admit`, `Gender` and `Dept`

```{r cont_adm, echo = FALSE}
pander::pander(ftable(adm))
```

This is much more informative and we can e.g. say immediately that `207` `Male` applicants were `Rejected` at department `B`. A particular entry in the table is called a `cell`. So `(Male, Rejected, B)` is a cell with _cellcount_ `207`. The _dimension_ of the table is the number of variables (here `3`). For high-dimensional tables things get more complicated and it may not be feasible to show the table (think of hundreds of DNA markers) and there may be many cells with zero cellcount.

The outlier detection method (described shortly) uses such contingency table data. However, it should be mentioned one more time, that a contingency table is in one-to-one correspondence with a "regular" categorical data set; contingency tables are nothing more than a convenient way of describing categorical data sets.

### Outliers and Novelties
An outlier can be regarded as an observation which deviates so much from the other observations in a database as to arouse suspicions that it was generated by a different mechanism. The outlier detection method directly adapts this definition by specifying a hypothesis of an outlier being distributed differently than all other observations in a given database. An outlier is also a very case-specific unit, since it may be interpreted as natural extreme noise in some applications and in other applications it might be the most interesting observation. 

Outlier detection is usually divided into \textit{outlier detection} (yes the same word) where the task is to detect observations that are extreme within some dataset. In combination to categorical data this amounts to detect unusual combinations between all variables in consideration. On the other hand, we might be interested in detecting if a new (novel) observation is an outlier in a homogeneous dataset which is sometimes referred to as \textit{novelty detection} in the litterature. **molic** handles both of these cases through a single function called `fit_outlier`.

For high-dimensional data, it is a tedious task to determine anomalies such as outliers and novelties. The method described here is a multivariate method for outlier detection in high-dimensional contingency tables, i.e. data with discrete variables only. 

The method relies on the class of decomposable graphical models to model the relationship among the variables of interest, which can be depicted by an undirected graph called the interaction graph. **molic** requires the [ess](https://github.com/mlindsk/ess) package to be installed. The **ess** package offers the simple `fit_graph` function that conveniently fits a decomposable graph to data.

### Decomposable Graphical Models
In order to model the interaction structure between variables, we need to construct a _decomposable graphical model_. A graphical model is a statistical model for which an undirected graph represents the interaction between the vertices in the model. An undirected graph is a pair $G = (V,E)$ where $V$ is a set of vertices and $E$ is a set of edges connecting elements in $V$. An edge connecting two vertices indicates that these two are dependend on each other; this is also called a two-way interaction. A threeway interaction occurs when three vertices are all mutually connected (e.g.\ can be visualized as a triangle). An undirected graph is _decomposable_ if there are no cycles of length greater than four without a chord (an edge between two non-adjacent vertices in the cycle). The _subgraph_ $G_{A} = (A, E_{A})$ consist of vertices $A\subseteq V$ from $G$ and the corresponding edges $E_A$ between them. A graph is _complete_ if there is an edge between all pairs of vertices and a complete subgraph is called a (maximal) _clique_ if it is not contained in any other complete subgraph. A subset of vertices is complete if it induces a complete subgraph. Two sets $A,B \subseteq V$ are separated by a third set $C \subseteq V$ if all paths between vertices in $A$ and $B$ go through $C$. If $C$ is the smallest set such that $A$ and $B$ are separated, we say that $C$ is a (minimal) _separator_ for $A$ and $B$.

\

Consider the undirected graph $G$ in the figure below. 

\

```{r undirected_graph, echo = FALSE, message = FALSE, warning = FALSE, fig.align = "center", fig.cap = "\\label{fig:DG} $G$: An undirected decomposable graph."}
# library(igraph)
el <- matrix( c("a", "b",
  "b", "c",
  "b", "d",
  "d", "e",
  "d", "c"),
  nc = 2,
  byrow = TRUE
)
G <- igraph::graph_from_edgelist(el, directed = FALSE)
set.seed(7)
plot(G, layout = igraph::layout_nicely(G), 
     edge.arrow.size=1, 
     vertex.label.cex=1.5, 
     # vertex.label.family="Helvetica",
     vertex.label.font=1,
     vertex.shape="circle", 
     vertex.size=40, 
     vertex.label.color="black", 
     edge.width=5)
```

\

The set of vertices is $V = \{a,b,c,d,e\}$ and the set of edges is \(E = \{ab, bc, bd, cd,de\}\). The cliques are $C_1 = \{a, b\}$, \(C_2 = \{b, c, d\}\) and $C_3 = \{d, e\}$. The minimal separators are $S_{2} = \{b\}$ and $S_3 = \{d\}$ where $S_{2}$ separates $C_{1}$ and $C_{2}$ and $S_{3}$ separates $C_{2}$ and $C_{3}$. Since $G$ has no cycles of length greater than three, the graph is decomposable. Notice, that $a$ is only connected to $\{c,d,e\}$ through $\{b\}$; we interpret this as "$a$ is independent of $\{c,d,e\}$ when we know the value of $b$". Such statements can be used to gain insight into complex structures.

Finally, we can associate a probability measure with an _interaction graph_; an undirected graph with each vertex being a random variable. For decomposable graphs, the probability density function can be written in terms of the cliques and separators. Collectively, models for which the interaction graph is decomposable are called decomposable graphical models (DGM). For more details on graphical models, see for example [@whittaker2009graphical; @lauritzen1996graphical].

Tree graphs are per definition decomposable graphs. Notice that the graph $G$ is not a tree because of the presence of the cycle $\{b,c,d\}$. However, the graph is indeed decomposable.

# The Test Statistic
Assume that the _likelihood_ $L_0$ expresses how likely it is that $z_{new}$ belongs to the database $D$. We can also specify an alternative likelihood, $L_{1}$, specifying how likely it is that $z_{new}$ do not belong to $D$. We then define the _likelihood ratio_ as
\begin{equation*}
  \label{eq:LR}
  LR = \frac{L_{0}}{L_{1}},
\end{equation*}
which can be shown to be completely specified through the counts of observations in cliques and separators for the given interaction graph. We can therefore test if $z_{new}$ is an outlier in $D$ by calculating $LR$ and determine if the value of $LR$ is "too large" in which case we would reject that $z_{new}$ comes from $D$.

### A note on novelty detection
It should be noted, that the likelihood ratio defined above, applies to novelty detection; i.e. we test if the novel observation $z_{new}$ is an outlier in $D$. If the goal is to find the outliers in $D$ one should, one at a time, remove the observations from $D$ and regard the observations as novel observations and apply the above test statistic. All of this is taken care of in **molic** via the `fit_outlier` function.

# References
