%\VignetteEngine{knitr::knitr} 
%\VignetteIndexEntry{vdmR: Generating web-based visual data mining tools with R}
%\VignetteAuthor{Tomokazu Fujino}
\documentclass{article}

\usepackage{graphicx}

\newcommand{\pkg}[1]{{\fontseries{b}\selectfont #1}}

\pagestyle{empty}
\author{Tomokazu Fujino}
\title{\pkg{vdmR}: Generating web-based visual data mining tools with R}


\begin{document}
\maketitle
\section{Outline of usage}
\subsection{Installation and Sample dataset}
The vdmR pakcage can be loaded as following command.
<<setup, results="hide">>=
library(vdmR)
@
Several packages are required for \pkg{vdmR}, which would be installed all together if you didn't install them previously. vdmR provides the sample dataset for demonstration and illustration of the package.

<<>>=
data(vsfuk2012)
head(vsfuk2012[,1:5])
@

The \texttt{vsfuk2012} data set gives the result of the vital statistics in Fukuoka prefecture of Japan from 2008 to 2012. The data set consists of 72 rows and 17 columns. Each row indicates the result of one municipality of Fukuoka prefecture. Each column shows basic information of the municipalities such as code, name or type (city, town, village) and indices such as population, fertility rate, mortality rate and so on. Information of all variables are given in the help document of the data set.

\subsection{Creating VDM tools}
Basically, to create VDM tools from \pkg{vdmR} are achieved in following two steps.
\begin{itemize}
\item create each statistical plots in html files with SVG and javascript
\item create main window for launching each plot and open the window
\end{itemize}
Here, we will show the example for generating two plots.
<<warning=FALSE, message=FALSE>>=
vscat(MortalityRate, FertilityRate, vsfuk2012, "scat01", "vsfuk2012")
#vhist(MarriageRate, vsfuk2012, "hist01", "vsfuk2012", bins=30)
@
Each command generates some files in the current working directory, which are named in the following rule:
\begin{quote}
\{\textit{plot name}\}.\{\textit{tag name}\}.(\texttt{svg}.)\{\textit{extension}\}
\end{quote}
All html files include \texttt{<embed>} tag for the related SVG file which contain the graphical information such as axes and sizes of the shapes in the plot. When opening the html file with a Web browser, the plot will appear. Note that the interactive functions will not enabled yet at this stage. To enable interactive functions, you need to generate "launcher" using following command.

<<eval=FALSE>>=
vlaunch(vsfuk2012, "main", "vsfuk2012")
@
This command generate html file named like
\begin{quote}
\{\textit{launcher name}\}.\{\textit{tag name}\}.\texttt{html}
\end{quote}
with some related files (\texttt{.js, .css}), and then launch the default Web browser and display the html file (Figure~\ref{vlaunch_ss}). This VDM tool currently doesn't support Internet Explorer, while supporting latest versions of Google Chrome, Mozilla Firefox and Safari.

\subsection{Manipulating VDM tools}
When clicking one of the button located on the upper part of the main window, corresponding plot window will be opened (Figure~\ref{vhist_ss}, \ref{vscat_ss}). You can see the gray rectangle on the top left of the window. Moving this rectangle selection tool, in the case of scatter plot, the data points in the rectangle region will be highlighted, and then the histogram of the selected data will be drawn on the histogram window by highlighted color. In addition, corresponding rows in the data table of the main window will be also highlighted (Figure~\ref{highlight_ss}). 

The persistent selection is supported in \pkg{vdmR} package, which means that once the objects such as points in the scatter plot are selected, the selected objects continue to be selected even if these objects are out of the region of the selection tool to be moved. In our VDM tools, double-clicking selection tool enables the persistent selection mode. When double-clicking it again, it will return to normal (temporary) selection mode (Figure~\ref{persistent_ss}).

The selection tool is also resizable. When dragging the mouse on the bottom right of the selection tool, the region will be resized.

The sortable table of the data set is displayed on the main window.  When clicking one of the variable labels, the rows are sorted based on the descending or ascending order of the corresponding variable.

\begin{figure}[htbp]

\begin{tabular}{cc}

\multicolumn{2}{c}{
\begin{minipage}{\hsize}
\begin{center}
\includegraphics[width=0.8\linewidth]{./figs/vlaunch_ss.jpg}
\caption{Main window generated by \texttt{vlaunch()} function}
\label{vlaunch_ss}
\end{center}
\vspace{0.5cm}
\end{minipage}
}

\\

\begin{minipage}{0.5\hsize}
\begin{center}
\includegraphics[width=0.9\linewidth]{./figs/vhist_ss.jpg}
\caption{Histogram window generated by \texttt{vhist()} function}
\label{vhist_ss}
\end{center}
\end{minipage}

&

\begin{minipage}{0.5\hsize}
\begin{center}
\includegraphics[width=0.9\linewidth]{./figs/vscat_ss.jpg}
\caption{Scatter plot window generated by \texttt{vscat()} function}
\label{vscat_ss}
\end{center}
\end{minipage}

\end{tabular}

\end{figure}

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=0.7\linewidth]{./figs/highlight_ss.jpg}
\caption{Example of the multiple linked views}
\label{highlight_ss}
\end{center}
\end{figure}

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=0.7\linewidth]{./figs/persistent_ss.jpg}
\caption{Example of the persistent selection in a scatter plot}
\label{persistent_ss}
\end{center}
\end{figure}

\section[Detail illustration of vdmR]{Detail illustration of \pkg{vdmR}}
\subsection[Data table and launcher of other plots: vlaunch()]{data table and launcher of other plots: \texttt{vlaunch()}}
\texttt{vlaunch()} function generates a main window which opens each pre-generated windows including a statistical plot with interactivity. At first, the function finds pre-generated html and SVG files in the current working directory by using regular expressions based on the \texttt{tag} argument. In the example of section 2.2, \texttt{vlaunch()} function finds html files including a SVG file of a statistical plot which have a name like \texttt{*.vsfuk2012.svg.html}. Then the function generates buttons for opening these files in the html file. All of the plot window should be opened from main window, because the multiple linked views (linked-brushing) can be implemented by using the cross-document messaging through the main window, which is one of the HTML5 technologies.

The data table in the main window is implemented by \textsf{DataTables} which is a plug-in for the \textsf{jQuery} Javascript library. It provides many useful functions for data tables in web pages such as pagination, instant search, multi-column ordering and so on.

\subsection[Scatter plot: vscat()]{scatter plot: \texttt{vscat()}}
\texttt{vscat()} function generates a interactive scatter plot in \pkg{ggplot2} style. The first two arguments \texttt{x,y} are the column names in the dataframe given by third arguments \texttt{data}, which are mapped into x-axis and y-axis of the scatter plot respectively. The dots argument (\texttt{...}) is passed to \texttt{aes()} function of \pkg{ggplot2}. Thus, in the \texttt{vsfuk2012} dataset, it is possible to relate each city's population of male to its point size on the scatter plot, and each city's type to its color on it by following script:

<<>>=
vscat(MortalityRate, FertilityRate, vsfuk2012, "scat01", "vsfuk2012",
      color=Type, size=pop_male)
@
The output of this \texttt{vscat()} function is shown in Figure~\ref{scataes_ss}. If you need to set (not aesthetic mapping) the color of all of the points to the specific color, you can do it by setting the argument of \texttt{I()} function to the character of the color name or color code like following example.
<<>>=
vscat(MortalityRate, FertilityRate, vsfuk2012, "scat01", "vsfuk2012",
      color=I("darkgreen"), size=pop_male)
@
This is a same style as \texttt{qplot()} function of \pkg{ggplot2}.

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=0.7\linewidth]{./figs/scataes_ss.jpg}
\caption{Scatter plot with aesthetic mapping}
\label{scataes_ss}
\end{center}
\end{figure}

\subsection[Histogram: vhist()]{histogram: \texttt{vhist()}}
\texttt{vhist()} function generates a interactive histogram in \pkg{ggplot2} style. The arguments of the function is almost same as \texttt{vscat()} function except that the variable of \texttt{y} doesn't exsist. If you need to specify the color of the histogram, you can do it as follows:
<<message=FALSE>>=
vhist(MarriageRate, vsfuk2012, "hist01", "vsfuk2012",
      fill=I("darkgreen"), color=I("black"))
@

A histogram visualizes a frequency count table of the specific variable of the dataset. Thus each graphic objects (bars of histogram) of the plot doesn't have one-to-one correspondence with a row of the dataset. This makes a little difficulties to implement linked-brushing facility. We have solved this by embedding the mapping information between each data and a bar of histogram into generated \textsf{SVG} files in \textsf{JSON} (Javascript Object Notation) format using \pkg{rjson} package.

\subsection[Parallel coordinate plot: vpcp()]{parallel coordinate plot: \texttt{vpcp()}}
A parallel coordinate plot is well-known as a powerful tool for visualizing multivariate data. \texttt{vpcp()} function can generate a parallel coordinate plot with interactive facilities in \pkg{ggplot2} style. In \pkg{ggplot2}, \texttt{ggpcp()} function can draw the parallel coordinate plot, however it has been deprecated. Thus, we used \texttt{ggparcoord()} function provided by \pkg{GGally} package in the \texttt{vpcp()} function. So the most of arguments of the \texttt{vpcp()} function is same as \texttt{ggparcoord()} function. An example is shown in Figure~\ref{vpcp_ss} created with the following code.
<<>>=
vpcp(vsfuk2012, 4:17, "pcp1", "vsfuk2012",
     groupColumn="Type", scale="uniminmax", missing="min10")
@
Note that the argument of \texttt{groupColumn} require the character of the column name. It means that double quotation marks are needed.

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=0.7\linewidth]{./figs/vpcp_ss.jpg}
\caption{Parallel coordinate plot}
\label{vpcp_ss}
\end{center}
\end{figure}

\subsection[Choropleth map: vcmap()]{choropleth map: \texttt{vcmap()}}
The interactive version of a choropleth map in which polygons of the areas are painted in proportion to the value of the specific variable is one of the central feature of \pkg{vdmR} package. By using the choropleth map in the multiple linked views, it will be easy to see the relationship between the spatial characteristics and the multivariate characteristics. The interactive linked micromap plot is one of the interactive application of the choropleth map.

To generate the interactive choropleth map in \pkg{vdmR}, you have to prepare the ESRI shapefile including the region for the dataset in which attribute table has a common column of the id column of the data frame. The \pkg{vdmR} package provides a sample shapefile related to the data \texttt{vsfuk2012}. By using \pkg{rgdal} package it is possible to import the shapefile into \textsf{R} environment as \texttt{SpatialPolygonsDataFrame} object as follows:
<<>>=
library(rgdal)
shp.path <- file.path(system.file(package="vdmR"), "etc/shapes/fukuoka2012.shp")
vsfuk2012.spdf <- readOGR(shp.path)
head(vsfuk2012.spdf@data)
@
If the shapefile doesn't have the common column of the id column of the data frame, you need to edit the shapefile by using desktop GIS software such as \textsf{QGIS} or \textsf{ArcGIS} or edit the data frame.

\texttt{vcmap()} function provides the interactive choropleth map in \pkg{vdmR} package. For example, following codes will generate the interactive choropleth map of the fertility rate in Fukuoka prefecture:
<<>>=
frcol <- ggplot2::scale_fill_gradient2(low="blue", mid="white", high="red",
                              midpoint=median(vsfuk2012$FertilityRate))
vcmap(shp.path, vsfuk2012, "CityCode", "CityCode", "map1", "vsfuk2012",
      fill=FertilityRate, ggscale=frcol)
@
Figure~\ref{vcmap_ss} shows the result of the code. The first and second arguments take the path to the shapefile and the data frame, respectively. The third and fourth argument take column names of the common id for the attribute table and data frame, respectively. The argument \texttt{fill} takes a column name assigned to the color of polygons. If you need to use your own color scale, the color scale which is generated by \texttt{scale\_fill\_*()} function has to be passed to the argument \texttt{ggscale}.

Thu brushing operation is a little different from other \pkg{vdmR} outputs. The select box doesn't appear on the choropleth map, so you have to brush the polygons directly by the mouse pointer. The operation on the choropleth map is always persistent selection. Double-clicking out of the polygons' region  will reset the all of the selections.

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=0.7\linewidth]{./figs/vcmap_ss.jpg}
\caption{Choropleth map}
\label{vcmap_ss}
\end{center}
\end{figure}

\end{document}