\documentclass[12pt,letterpaper]{article}

\usepackage[T1]{fontenc}

\usepackage{Rd}
\usepackage{Sweave}
\usepackage{upquote}


% === for bad lm output ===
% this wasn't working with rbuild
%% so i took out the signficance stars
%\usepackage[utf8]{inputenc}

%\usepackage{Rd/Rd}
%\usepackage{Rd/Sweave}
%\usepackage{Rd/upquote}

\SweaveOpts{width=70}

% === graphic packages ===
\usepackage{graphicx}
\usepackage{epstopdf}
\DeclareGraphicsRule{.tif}{png}{.png}{`convert #1 `dirname #1`/`basename #1 .tif`.png}

% === bibliography package ===
\usepackage{natbib}
% === margin and formatting ===
\usepackage{setspace}
\usepackage{fullpage}
\usepackage{caption}
% === math packages ===
\usepackage[reqno]{amsmath}
\usepackage{amsthm}
\usepackage{amssymb,enumerate}
\newtheorem{Com} {Comment}
\newtheorem{prop}{Proposition}
\newtheorem{thm}{Theorem}
% === dcolumn package ===
\usepackage{dcolumn}
\newcolumntype{.}{D{.}{.}{-1}}
\newcolumntype{d}[1]{D{.}{.}{#1}}
% === additional packages ===
\usepackage{url}
\setcounter{tocdepth}{4}
\usepackage{tikz}

%\VignetteIndexEntry{Amelia II: A Package for Missing Data}
%\VignetteDepends{Zelig}
%\VignetteKeyWords{}
%\VignettePackage{Amelia}

\usepackage{color}
\usepackage[pdftex, bookmarksopen=true, bookmarksnumbered=true,
pdfstartview=FitH, breaklinks=true, urlbordercolor={0 1 0},
citebordercolor={0 0 1}]{hyperref}

\newcommand{\obs}{{\text{obs}}}
\newcommand{\mis}{{\text{mis}}}
%\newcommand{\Amelia}{\texttt{Amelia}}
\renewcommand{\R}{\textsf{R}}
\newcommand{\Amelia}{\ensuremath{\mathbb A}melia}
\newcommand{\AmeliaII}{\ensuremath{\mathbb A}melia~\ensuremath{\mathbb{II}}}
\newcommand{\AmeliaView}{\ensuremath{\mathbb A}melia\ensuremath{\mathbb{V}}iew}
\newcommand{\adjparbox}[2]{\parbox{#1}{\vspace{.35em}#2\vspace{.35em}}}
\usepackage{vmargin}
\topmargin=0in

\title{${\mathbb{AMELIA~II}}$: A Program for Missing Data}

% rbuild: replace 'Version ' '\\' Version


\author{James Honaker, Gary King, and Matthew Blackwell}

\begin{document}

<<echo=FALSE, print=FALSE>>=
ameliaVersion <- packageVersion("Amelia")
@


\date{Version \Sexpr{ameliaVersion}\\ \today}
\maketitle

\tableofcontents
\newpage



\section{Introduction}
\label{sec:intro}
Missing data is a ubiquitous problem in social science data.
Respondents do not answer every question, countries do not collect
statistics every year, archives are incomplete, subjects drop out of
panels. Most statistical analysis methods, however, assume the absence of
missing data, and are only able to include observations for which
every variable is measured. \AmeliaII\ allows users to impute (``fill in'' or
rectangularize) incomplete data sets so that analyses which require
complete observations can appropriately use all the information
present in a dataset with missingness, and avoid the biases,
inefficiencies, and incorrect uncertainty estimates that can result
from dropping all partially observed observations from the analysis.

\AmeliaII\ performs \emph{multiple imputation}, a general-purpose
approach to data with missing values. Multiple imputation has been
shown to reduce bias and increase efficiency compared to listwise
deletion. Furthermore, ad-hoc methods of imputation, such as mean
imputation, can lead to serious biases in variances and covariances.
Unfortunately, creating multiple imputations can be a burdensome
process due to the technical nature of algorithms involved. \Amelia\
provides users with a simple way to create and implement an imputation model,
generate imputed datasets, and check its fit using diagnostics.

The \AmeliaII\ program goes several significant steps beyond the
capabilities of the first version of \Amelia\ \citep*{HonJosKin98}.
For one, the bootstrap-based EMB algorithm included in \AmeliaII\ can
impute many more variables, with many more observations, in much less
time.  The great simplicity and power of the EMB algorithm made it
possible to write \AmeliaII\ so that it virtually never crashes ---
which to our knowledge makes it unique among all existing multiple
imputation software --- and is much faster than the alternatives too.
\AmeliaII\ also has features to make valid and much more accurate
imputations for cross-sectional, time-series, and
time-series-cross-section data, and allows the incorporation of
observation and data-matrix-cell level prior information. In addition
to all of this, \AmeliaII\ provides many diagnostic functions that
help users check the validity of their imputation model.  This
software implements the ideas developed in \citet*{HonKin10}.

\section[What Amelia Does]{What \Amelia\ Does}
\label{sec:what}

Multiple imputation involves imputing $m$ values for each missing cell
in your data matrix and creating $m$ ``completed'' data sets.  Across
these completed data sets, the observed values are the same, but the
missing values are filled in with a distribution of imputations that reflect
the uncertainty about the missing data.  After imputation with
\AmeliaII's EMB algorithm, you can apply whatever statistical method
you would have used if there had been no missing values to each of the
$m$ data sets, and use a simple procedure, described below, to combine
the results\footnote{You can combine the results automatically by
  doing your data analyses within Zelig for R, or within Clarify for
  Stata; see \url{http://gking.harvard.edu/stats.shtml}.}.  Under
normal circumstances, you only need to impute once and can then
analyze the $m$ imputed data sets as many times and for as many
purposes as you wish.  The advantage of \AmeliaII\ is that it combines
the comparative speed and ease-of-use of our algorithm with the power
of multiple imputation, to let you focus on your substantive research
questions rather than spending time developing complex
application-specific models for nonresponse in each new data set.
Unless the rate of missingness is very high, $m = 5$ (the program
default) is probably adequate.

\subsection{Assumptions}

The imputation model in \AmeliaII\ assumes that the complete data (that
is, both observed and unobserved) are multivariate normal. If we denote
the $(n \times k)$ dataset as $D$ (with observed part $D^{\obs}$ and
unobserved part $D^{\mis}$), then this assumption is
\begin{equation}
  \label{eq:norm}
  D \sim \mathcal{N}_k(\mu, \Sigma),
\end{equation}
which states that $D$ has a multivariate normal distribution with mean
vector $\mu$ and covariance matrix $\Sigma$. The multivariate normal
distribution is often a crude approximation to the true distribution of
the data, yet there is evidence that this model works as well as other,
more complicated models even in the face of categorical or mixed data
\citep[see][]{Schafer97, SchOls98}. Furthermore, transformations of many types of variables
can often make this normality assumption more plausible (see
\ref{sec:trans} for more information on how to implement this in \Amelia).

The essential problem of imputation is that we only observe
$D^{\obs}$, not the entirety of $D$. In order to gain traction, we need to make
the usual assumption in multiple imputation that the data are
\emph{missing at random} (MAR). This assumption means that the pattern
of missingness only depends on the observed data $D^{\obs}$, not the
unobserved data $D^{\mis}$. Let $M$ to be the missingness matrix, with
cells $m_{ij} = 1$ if $d_{ij} \in D^{\mis}$ and $m_{ij} = 0$
otherwise.  Put simply, $M$ is a matrix that indicates whether or not
a cell is missing in the data. With this, we can define the MAR
assumption as
\begin{equation}
  \label{eq:mar}
  p(M|D) = p(M|D^{\obs}).
\end{equation}
Note that MAR includes the case when missing values are created
randomly by, say, coin flips, but it also includes many more
sophisticated missingness models.  When missingness is not dependent
on the data at all, we say that the data are \emph{missing completely
  at random} (MCAR).  \Amelia\ requires both the multivariate
normality and the MAR assumption (or the simpler special case of
MCAR).  Note that the MAR assumption can be made more plausible by
including additional variables in the dataset $D$ in the imputation
dataset than just those eventually envisioned to be used in the analysis model.

\subsection{Algorithm}

In multiple imputation, we are concerned with the complete-data
parameters, $\theta = (\mu, \Sigma)$. When writing down a model of the
data, it is clear that our observed data is actually $D^{\obs}$ and $M$,
the missingness matrix. Thus, the likelihood of our observed data is
$p(D^{\obs}, M|\theta)$. Using the MAR assumption\footnote{There is an
  additional assumption hidden here that $M$ does not depend on the
  complete-data parameters.}, we can break this up,
\begin{align}
  p(D^{\obs},M|\theta) = p(M|D^{\obs})p(D^{\obs}|\theta).
\end{align}
As we only care about inference on the complete data parameters, we can
write the likelihood as
\begin{align}
  L(\theta|D^{\obs}) &\propto p(D^{\obs}|\theta),
\end{align}
which we can rewrite using the law of iterated expectations as
\begin{align}
  p(D^{\obs}|\theta) &= \int p(D|\theta) dD^{\mis}.
\end{align}
With this likelihood and a flat prior on $\theta$, we can see that the
posterior is
\begin{equation}
  p(\theta | D^{\obs}) \propto p(D^{\obs}|\theta) = \int p(D|\theta)
  dD^{\mis}.
\end{equation}
The main computational difficulty in the analysis of incomplete data
is taking draws from this posterior. The EM algorithm
\citep*{DemLaiRub77} is a simple computational approach to finding the
mode of the posterior. Our EMB algorithm combines the classic EM
algorithm with a bootstrap approach to take draws from this posterior.
For each draw, we bootstrap the data to simulate estimation
uncertainty and then run the EM algorithm to find the mode of the
posterior for the bootstrapped data, which gives us fundamental
uncertainty too (see \citet{HonKin10} for details of the EMB
algorithm).

Once we have draws of the posterior of the complete-data parameters,
we make imputations by drawing values of $D^{\mis}$ from its
distribution conditional on $D^{\obs}$ and the draws of $\theta$,
which is a linear regression with parameters that can be calculated
directly from $\theta$.

\subsection{Analysis}

In order to combine the results across $m$ data sets, first decide on
the quantity of interest to compute, such as a univariate mean,
regression coefficient, predicted probability, or first difference.
Then, the easiest way is to draw $1/m$ simulations of $q$ from each of
the $m$ data sets, combine them into one set of $m$ simulations, and
then to use the standard simulation-based methods of interpretation
common for single data sets \citep{KinTomWit00}.

Alternatively, you can combine directly and use as the multiple
imputation estimate of this parameter, $\bar{q}$, the average of the
$m$ separate estimates, $q_j$ $(j=1,\dots,m)$:
\begin{equation}
  \bar{q}=\frac{1}{m}\sum^{m}_{j=1}q_j.
\end{equation}

The variance of the point estimate is the average of the estimated
variances from \emph{within} each completed data set, plus the sample
variance in the point estimates \emph{across} the data sets
(multiplied by a factor that corrects for the bias because
$m<\infty$).  Let $SE(q_j)^2$ denote the estimated variance (squared
standard error) of $q_j$ from the data set $j$, and
$S^{2}_{q}=\Sigma^{m}_{j=1}(q_j-\bar{q})^2/(m-1)$ be the sample
variance across the $m$ point estimates.  The standard error of the
multiple imputation point estimate is the square root of
\begin{equation}
SE(q)^2=\frac{1}{m}\sum^{m}_{j=1}SE(q_j)^2+S^2_q(1+1/m).
\end{equation}

\begin{figure}
\centering
\begin{tikzpicture}
[inner sep = 2.5mm,
missdata/.style={rectangle, draw=red!50, fill=red!20, thick},
 impdata/.style={rectangle, draw=blue!50, fill=blue!20, thick},
results/.style= {circle, draw=blue!50, fill=blue!20, thick}]

\node  (odata) at (3,3) [missdata] {};
\node [red!80,right] at (5.25,3) {incomplete data};


\node (boot1) at (1,1) [missdata] {}
  edge [<-, bend left=40] (odata);
\node (boot2) at (2,1) [missdata] {}
  edge [<-, bend left=20] (odata);
\node (boot3) at (3,1) [missdata] {}
  edge [<-] (odata);
\node (boot4) at (4,1) [missdata] {}
  edge [<-, bend right=20] (odata);
\node (boot5) at (5,1) [missdata] {}
  edge [<-, bend right=40] (odata);



%% add "random question marks"
\node[red!50!black] at (2.95,3.1) {\tiny ?};
\node[red!50!black] at (2.85,2.9) {\tiny ?};
\node[red!50!black] at (3.15,2.95) {\tiny ?};

\node[red!50!black] at (2.85,1.1) {\tiny ?};
\node[red!50!black] at (2.95,0.9) {\tiny ?};
\node[red!50!black] at (3.15,1.1) {\tiny ?};

\node[red!50!black] at (0.85,0.9) {\tiny ?};
\node[red!50!black] at (1.15,0.95) {\tiny ?};


\node[red!50!black] at (1.85,1.1) {\tiny ?};
\node[red!50!black] at (1.95,0.9) {\tiny ?};


\node[red!50!black] at (3.85,1.1) {\tiny ?};

\node[red!50!black] at (4.95,0.9) {\tiny ?};
\node[red!50!black] at (5.15,1.1) {\tiny ?};


\node [black!80,right] at (5.25,2) {bootstrap};
\node [red!80, right] at (5.25, 1) {bootstrapped data};

\node (imp1) at (1,0) [impdata] {}
  edge [<-] (boot1);
\node (imp2) at (2,0) [impdata] {}
  edge [<-] (boot2);
\node (imp3) at (3,0) [impdata] {}
  edge [<-] (boot3);
\node (imp4) at (4,0) [impdata] {}
  edge [<-] (boot4);
\node (imp5) at (5,0) [impdata] {}
  edge [<-] (boot5);

\node [blue!80,right] at (5.25,0) {imputed datasets};
\node [black!80, right] at (5.25, 0.5) {EM};

\node (res1) at (1,-1) [results] {}
  edge [<-] (imp1);
\node (res2) at (2,-1) [results] {}
  edge [<-] (imp2);
\node (res3) at (3,-1) [results] {}
  edge [<-] (imp3);
\node (res4) at (4,-1) [results] {}
  edge [<-] (imp4);
\node (res5) at (5,-1) [results] {}
  edge [<-] (imp5);

\node [black!80, right] at (5.25, -0.5) {analysis};
\node [blue!80, right] at (5.25, -1) {separate results};

\node[circle, draw=green!50!black!50, fill=green!20, thick] at (3,-3) {}
  edge [<-, bend right=40] (res5)
  edge [<-, bend left=40] (res1)
  edge [<-, bend right=20] (res4)
  edge [<-, bend left=20] (res2)
  edge [<-] (res3);

\node [black!80, right] at (5.25, -2) {combination};
\node [green!80!black!80, right] at (5.25, -3) { final results};
\end{tikzpicture}
\caption{A schematic of our approach to multiple imputation with the EMB
algorithm. }
\end{figure}

\section[Versions of Amelia]{Versions of \Amelia\ }

\label{sec:versions}
Two versions of \AmeliaII\ are available, each with its own advantages
and drawbacks, but both of which use the same underlying code and algorithms.  First,
\AmeliaII\ exists as a package for the \R\ statistical software
package.  Users can utilize their knowledge of the \R\ language to run
\AmeliaII\ at the command line or to create scripts that will run
\AmeliaII\ and preserve the commands for future use.  Alternatively,
you may prefer \AmeliaView, where an interactive Graphical User
Interface (GUI) allows you to set options and run \Amelia\ without any
knowledge of the \R\ programming language.

Both versions of \AmeliaII\ are available on the Windows, Mac OS X,
and Linux platforms and \AmeliaII\ for \R\ runs in any environment that \R\
can.  All versions of \Amelia\ require the \R\ software, which
is freely available at \url{http://www.r-project.org/}.

Before installing \AmeliaII, you must have installed \R\ version 2.1.0
or higher, which is freely available at
\url{http://www.r-project.org/}.

\subsection{Installation and Updates from R}
\label{sec:install}

To install the \Amelia\ package on any platform, simply type the following at
the \R\ command prompt,
\begin{verbatim}
> install.packages("Amelia")
\end{verbatim}
and \R\ will automatically install the package to your system from CRAN. If
you wish to use the most current beta version of \Amelia\, feel free to
install the test version,
\begin{verbatim}
> install.packages("Amelia", repos = "http://gking.harvard.edu")
\end{verbatim}

In order to keep your copy of \Amelia\ completely up to date, you should
use the command
\begin{verbatim}
> update.packages()
\end{verbatim}


\subsection[Installation in Windows of AmeliaView as a Standalone Program]{Installation in Windows of \AmeliaView\ as a Standalone Program}
\label{sec:win-install}
To install a standalone version of \AmeliaView\ in the Windows environment,
simply download the installer \texttt{setup.exe} from
\url{http://gking.harvard.edu/amelia/} and run it.  The installer will ask
you to choose a location to install \AmeliaII.  If you have installed \R\
with the default options, \AmeliaII\ will automatically find the location
of \R.  If the installer cannot find \R, it will ask you to locate the
directory of the most current version of \R.  Make sure you choose the
directory name that includes the version number of \R\ (e.g. C:/Program
Files/R/R-2.9.0) and contains a subdirectory named \texttt{bin}.  The
installer will also put shortcuts on your Desktop and Start Menu.

Even users familiar with the \R\ language may find it useful to
utilize \AmeliaView\ to set options on variables, change arguments, or
run diagnostics.  From the command line, \AmeliaView\ can be brought up
with the call:
\begin{verbatim}
> library(Amelia)
> AmeliaView()
\end{verbatim}


\subsection{Linux (local installation)}
\label{sec:lin-install}
Installing \Amelia\ on a Linux system is slightly more complicated due to
user permissions. If you are running \R\ with root access, you can simply
run the above installation procedure. If you do not have root access, you
can install \Amelia\ to a local library. First, create a local directory to
house the packages,
\begin{verbatim}
w4:mblackwell [~]: mkdir ~/myrlibrary
\end{verbatim}
and then, in an \R\ session, install the package directing \R\ to this
location:

\begin{verbatim}
> install.packages("Amelia", lib = "~/myrlibrary")
\end{verbatim}

Once this is complete you need to edit or create your \R\ profile.
Locate or create \texttt{\~/.Rprofile} in your home directory and add
this line:
\begin{verbatim}
.libPath("~/myrlibrary")
\end{verbatim}
This will add your local library to the list of library paths that \R\
searches in when you load libraries.

Linux users can use \AmeliaView\ in the same way as Windows users of
\Amelia\ for \R.  From the command line, \AmeliaView\ can be brought up
with the call:
\begin{verbatim}
> AmeliaView()
\end{verbatim}



\section{A User's Guide}
\label{sec:guide}

\subsection{Data and Initial Results}

We now demonstrate how to use \Amelia\ using data from \citet{MilKub05}
which studies the effect of democracy on trade policy. For the purposes of
this user's guide, we will use a subset restricted to nine developing
countries in Asia from 1980 to 1999\footnote{We have artificially added
  some missingness to these data for presentational purposes. You can
  access the original data at
  \url{http://www.princeton.edu/~hmilner/Research.htm}}. This dataset
includes 9 variables: year (\code{year}), country (\code{country}),
average tariff rates (\code{tariff}), Polity IV score\footnote{The Polity score
  is a number between -10 and 10 indicating how democratic a country is. A
  fully autocratic country would be a -10 while a fully democratic country
  would be 1 10.} (\code{polity}), total population (\code{pop}), gross
domestic product per capita (\code{gdp.pc}), gross international reserves
(\code{intresmi}), a dummy variable signifying whether the country had signed an IMF
agreement in that year (\code{signed}), a measure of financial openness
(\code{fivop}), and a measure of US hegemony\footnote{This measure of US
  hegemony is the US imports and exports as a percent of the world total
  imports and exports.} (\code{usheg}). These variables correspond to the
variables used in the analysis model of \citet{MilKub05} in table 2.

<<echo=FALSE, print=FALSE>>=
options("digits"=4)
options("width"=70)
options("show.signif.stars" = FALSE)
set.seed(12345)
@

We first load the \Amelia\ and the data:

<<echo=TRUE, print=FALSE>>=
require(Amelia)
data(freetrade)
@

We can check the summary statistics of the data to see that there is
missingness on many of the variables:

<<<echo=TRUE, print=TRUE>>=
summary(freetrade)
@

In the presence of missing data, most statistical packages use
\emph{listwise deletion}, which removes any row that contains a missing
value from the analysis. Using the base model of \citet{MilKub05} table 2,
we run a simple linear model in \R, which uses listwise deletion:
<<echo=TRUE, print=TRUE>>=
summary(lm(tariff ~ polity + pop + gdp.pc + year + country,
          data = freetrade))
@

Note that 60 of the 171 original observations are deleted due to
missingness. These observations, however, are partially observed, and
contain valuable information about the relationships between those
variables which are present in the partially completed observations.
Multiple imputation will help us retrieve that information and make
better, more efficient, inferences.

\subsection{Multiple Imputation}


When performing multiple imputation, the first step is to identify the
variables to include in the imputation model.  It is crucial to include at least
as much information as will be used in the analysis model. That is, any variable
that will be in the analysis model should also be in the imputation
model. This includes any transformations or interactions of variables that will
appear in the analysis model.

In fact, it is often useful to add more information to the imputation model than
will be present when the analysis is run. Since
imputation is predictive, any variables that would increase predictive
power should be included in the model, even if including them in the
analysis model would produce bias in estimating a causal effect (such as for post-treatment
variables) or collinearity would preclude determining which variable had a relationship
with the dependent variable (such as including multiple alternate measures of GDP).
In our case, we include all the variables in
\code{freetrade} in the imputation model, even though our analysis
model focuses on \code{polity}, \code{pop} and
\code{gdp.pc}\footnote{Note that this specification does not utilize
  time or spatial data yet. The \texttt{ts} and \texttt{cs} arguments
  only have force when we also include \texttt{polytime} or
  \texttt{intercs}, discussed in section \ref{sec:tscs}}.

To create multiple imputations in \Amelia, we can simply run
<<echo=TRUE, print=FALSE>>=
a.out <- amelia(freetrade, m = 5, ts = "year", cs = "country")
a.out
@

Note that our example dataset is deliberately small both in variables and in
cross-sectional elements.  Typical datasets may often have hundreds or possibly a
couple thousand steps to the EM algorithm.  Long chains should remind the analyst
to consider whether transformations of the variables would more closely fit the
multivariate normal assumptions of the model (correct but omitted transformations
will shorten the number of steps and improve the fit of the imputations), but do not
necessarily denote problems with the imputation model.

The output gives some information about how the algorithm ran. Each of the
imputed datasets is now in the list \code{a.out\$imputations}.  Thus, we
could plot a histogram of the \code{tariff} variable from the 3rd
imputation,

<<label=hist1plot, include=FALSE>>=
hist(a.out$imputations[[3]]$tariff, col="grey", border="white")
@ %$



<<echo=FALSE, results = hide>>=
options(SweaveHooks = list(fig = function() par(mfrow=c(1,1))))
@

\begin{figure}
\begin{center}
<<label=hist1, fig=TRUE, echo=FALSE, width=7, height=7>>=
<<hist1plot>>
@

\caption{Histogram of the \texttt{tariff} variable from the 3rd imputed dataset.}
\label{fig:hist1}
\end{center}
\end{figure}

\subsubsection{Saving imputed datasets}
\label{sec:saving}
If you need to save your imputed datasets, one direct method is to save the output list
from \code{amelia},

\begin{verbatim}
> save(a.out, file = "imputations.RData")
\end{verbatim}

As in the previous example, the \emph{i}th imputed datasets can be retrieved from this list
as \code{a.out\$imputations[[i]]}.

In addition, you can save each of the imputed datasets to its own file
using the \code{write.amelia} command,
\begin{verbatim}
> write.amelia(obj=a.out, file.stem = "outdata")
\end{verbatim}
This will create one comma-separated value file for each imputed dataset
in the following manner:
\begin{verbatim}
outdata1.csv
outdata2.csv
outdata3.csv
outdata4.csv
outdata5.csv
\end{verbatim}
The \code{write.amelia} function can also save files in tab-delimited and
Stata (\code{.dta}) file formats. For instance, to save Stata files,
simply change the \code{format} argument to \code{"dta"},

\begin{verbatim}
> write.amelia(obj=a.out, file.stem = "outdata", format = "dta")
\end{verbatim}

Additionally, \code{write.amelia} can create a ``stacked'' version of
the imputed dataset which stacks each imputed dataset on top of one
another. This can be done by setting the \code{separate} argument to
\code{FALSE}. The resulting matrix is of size $(N \cdot m) \times p$
if the original dataset is excluded (\code{orig.data = FALSE}) and of
size $(N \cdot (m+1))\times p$ if it is included (\code{orig.data =
  TRUE}). The stacked dataset will include a variable (set with
\code{impvar}) that indicates to which imputed dataset the observation
belongs. See Section~\ref{sec:analysis} for a description of how to
use this stacked dataset with the \code{mi} commands in Stata.


\subsubsection[Combining Multiple Amelia Runs]{Combining Multiple \Amelia\ Runs}
The EMB algorithm is what computer scientists call \emph{embarrassingly
  parallel}, meaning that it is simple to separate each imputation into
parallel processes. With \Amelia\ it is simple to run subsets of the
imputations on different machines and then combine them after the
imputation for use in analysis model. This allows for a huge increase in
the speed of the algorithm.

Output lists from different \Amelia\ runs can be combined together into a
new list.  For instance, suppose that we wanted to add another ten imputed datasets
to our earlier call to \code{amelia}. First, run the function to get
these additional imputations,
<<>>=
a.out.more <- amelia(freetrade, m = 10, ts = "year", cs = "country", p2s=0)
a.out.more
@
then combine this output with our original output using the
\code{ameliabind} function,
<<>>=
a.out.more <- ameliabind(a.out, a.out.more)
a.out.more
@
This function binds the two outputs into the same output so that you can
pass the combined imputations easily to analysis models and
diagnostics. Note that \code{a.out.more} now has a total of 15
imputations.

A simple way to execute a parallel processing scheme with \Amelia\ would
be to run \code{amelia} with \code{m} set to 1 on $m$ different machines
or processors, save each output using the \code{save} function, load them all on the same
\R\ session using \code{load} command and then combine them using
\code{ameliabind}. In order to do this, however, make sure to name each of
the outputs a different name so that they do not overwrite each other when
loading into the same \R\ session.  Also, some parallel environments will
dump all generated files into a common directory, where they may overwrite
each other.  If it is convenient in a parallel environment to run a large number
of \code{amelia} calls from a single piece of code, one useful way to avoid
overwriting is to create the \code{file.stem} with a random suffix.  For example:

\begin{verbatim}
> b<-round(runif(1,min=1111,max=9999))
> random.name<-paste("am",b,sep="")
> amelia <- write.amelia(obj=a.out, file.stem = random.name)
\end{verbatim}


\subsubsection{Screen Output}
Screen output can be adjusted with the ``print to screen'' argument,
\code{p2s}.  At a value of 0, no screen printing will occur.  This may be
useful in large jobs or simulations where a very large number of
imputation models may be required.  The default value of 1, lists each
bootstrap, and displays the number of iterations required to reach
convergence in that bootstrapped dataset.  The value of 2 gives more
thorough screen output, including, at each iteration, the number of
parameters that have significantly changed since the last iteration. This
may be useful when the EM chain length is very long, as it can provide an
intuition for many parameters still need to converge in the EM chain, and
a sense of the time remaining.  However, it is worth noting that the last
several parameters can often take a significant fraction of the total
number of iterations to converge.  Setting \code{p2s} to 2 will also
generate information on how EM algorithm is behaving, such as a \code{!}
when the current estimated complete data covariance matrix is not
invertible and a \code{*} when the likelihood has not monotonically
increased in that step. Having many of these two symbols in the screen
output is an indication of a problematic imputation
model\footnote{Problems of non-invertible matrices often mean that current
  guess for the covariance matrix is singular. This is a sign that there
  may be two highly correlated variables in the model. One way to resolve
  is to use a ridge prior (see \ref{sec:prior})}.

An example of the output when \code{p2s} is 2 would be

<<echo=TRUE, print=FALSE>>=
amelia(freetrade, m = 1, ts = "year", cs = "country", p2s = 2)
@

\subsection{Parallel Imputation Using Multicore CPUs}
\label{sec:parallel}

Each imputation in the above EMB algorithm is completely independent
of any other imputation, a property called embarrassingly
parallel. This type of approach can take advantage of the multiple
-core infrastructure of modern CPUs. Each core in a multi-core
processor can execute independent operations in parallel. \Amelia\ can
utilize this parallel processing internally \emph{via} the
\code{parallel} and the \code{ncpus} arguments. The \code{parallel}
argument sets the parallel processing backend, either with
\code{"multicore"} or \code{"snow"} (or \code{"no"} for no parallel
processing). The \code{"multicore"} backend is not available on
Windows systems, but tends to be quicker at parallel processing. On a
Windows system, the \code{"snow"} backend provides parallel processing
through a cluster of worker processes across the CPUs. You can set the
default for this argument using the \code{"amelia.parallel"}
option. This allows you to run Amelia in parallel as the default for
an entire R session without setting arguments in the \code{amelia}
call.

For each of the parallel backends, \Amelia\ requires a number of CPUs
to use in parallel. This can be set using the \code{ncpus}
argument. It can be higher than the number of physical cores in
the system if hyperthreading or other technologies are available. You
can use the \code{parallel::detectCores} function to determine how
many cores are available on your machine. The default for this
argument can be set using the \code{"amelia.ncpus"} option.

On Unix-alike systems (such as Mac OS X and Linux distributions), the
\code{"multicore"} backend automatically sets up and stops the
parallel workers by forking the process. On Windows, the \code{"snow"}
backend requires more attention. \Amelia\ will attempt to create a
parallel cluster of worker processes (since Windows systems cannot
fork a process) and will stop this cluster after the
imputations are complete. Alternatively, \Amelia\ also has a \code{cl}
argument, which accepts a predefined cluster made using the
\code{parallel::makePSOCKcluster}. For more information about parallel
processing in R, see the documentation for the \pkg{parallel} package
that ships along with R.


\subsection{Imputation-improving Transformations}
\label{sec:trans}

Social science data commonly includes variables that fail to fit to
a multivariate normal distribution. Indeed, numerous models have been
introduced specifically to deal with the problems they present.  As it
turns out, much evidence in the literature (discussed in
\citealt{KinHonJos01}) indicates that the multivariate normal model
used in \Amelia\ usually works well for the imputation
stage even when discrete or non-normal variables are included and when
the analysis stage involves these limited dependent variable models.
Nevertheless, \Amelia\ includes some limited capacity to
deal directly with ordinal and nominal variables and to modify variables that
require other transformations.  In general nominal and log transform
variables should be declared to \Amelia, whereas ordinal (including
dichotomous) variables often need not be, as described below.  (For
harder cases, see \citep{Schafer97}, for specialized MCMC-based imputation
models for discrete variables.)\nocite{KinHonJos01}

Although these transformations are taken internally on these variables
to better fit the data to the multivariate normal assumptions of the
imputation model, all the imputations that are created will be
returned in the original untransformed form of the data.  If the user
has already performed transformations on their data (such as by taking
a log or square root prior to feeding the data to \code{amelia}) these
do not need to be declared, as that would result in the transformation
occurring \emph{doubly} in the imputation model.  The fully imputed
data sets that are returned will always be in the form of the original
data that is passed to the \code{amelia} routine.

\subsubsection{Ordinal}
\label{sec:ord}
In much statistical research, researchers treat independent ordinal
(including dichotomous) variables as if they were really continuous.
If the analysis model to be employed is of this type, then nothing
extra is required of the of the imputation model. Users are advised to
allow \Amelia\ to impute non-integer values for any
missing data, and to use these non-integer values in their analysis.
Sometimes this makes sense, and sometimes this defies intuition. One
particular imputation of 2.35 for a missing value on a seven point
scale carries the intuition that the respondent is between a 2 and a 3
and most probably would have responded 2 had the data been observed.
This is easier to accept than an imputation of 0.79 for a dichotomous
variable where a zero represents a male and a one represents a female
respondent. However, in both cases the non-integer imputations carry
more information about the underlying distribution than would be
carried if we were to force the imputations to be integers. Thus
whenever the analysis model permits, missing ordinal observations
should be allowed to take on continuously valued imputations.

In the \code{freetrade} data, one such ordinal variable is \code{polity}
which ranges from -10 (full autocracy) to 10 (full democracy). If we
tabulate this variable from one of the imputed datasets,


<<echo=TRUE, print=TRUE>>=
table(a.out$imputations[[3]]$polity)
@ %$
we can see that there is one imputation between -4 and -3 and one
imputation between 6 and 7. Again, the interpretation of these values is
rather straightforward even if they are not strictly in the coding of the
original Polity data.

Often, however, analysis models require some variables to be strictly
ordinal, as for example, when the dependent variable will be modeled in a logistical or
Poisson regression.  Imputations for variables set as ordinal are created
by taking the continuously valued imputation and using an appropriately
scaled version of this as the probability of success in a binomial
distribution. The draw from this binomial distribution is then translated
back into one of the ordinal categories.

For our data we can simply add \code{polity} to the \code{ords} argument:
<<echo=TRUE, print=FALSE>>=
a.out1 <- amelia(freetrade, m = 5, ts = "year", cs = "country", ords =
                 "polity", p2s = 0)
table(a.out1$imputations[[3]]$polity)
@
Now, we can see that all of the imputations fall into one of the original
polity categories.

\subsubsection{Nominal}
\label{sec:nom}
Nominal variables\footnote{Dichotomous (two category) variables are a
  special case of nominal variables. For these variables, the nominal and
  ordinal methods of transformation in \Amelia\ agree.} must be treated
quite differently than ordinal variables. Any multinomial variables in the
data set (such as religion coded 1 for Catholic, 2 for Jewish, and 3 for
Protestant) must be specified to \Amelia. In our
\code{freetrade} dataset, we have \code{signed} which is 1 if a country
signed an IMF agreement in that year and 0 if it did not. Of course, our
first imputation did not limit the imputations to these two categories

<<>>=
table(a.out1$imputations[[3]]$signed)
@

In order to fix this for a $ p$-category multinomial variable,\Amelia\ will
determine $ p$ (as long as your data contain at least one value in each
category), and substitute $ p-1$ binary variables to specify each possible
category. These new $ p-1$ variables will be treated as the other
variables in the multivariate normal imputation method chosen, and receive
continuous imputations. These continuously valued imputations will then be
appropriately scaled into probabilities for each of the $ p$ possible
categories, and one of these categories will be drawn, where upon the
original $ p$-category multinomial variable will be reconstructed and
returned to the user. Thus all imputations will be appropriately
multinomial.


For our data we can simply add \code{signed} to the \code{noms} argument:
<<echo=TRUE, print=FALSE>>=
a.out2 <- amelia(freetrade, m = 5, ts = "year", cs = "country", noms =
                 "signed", p2s = 0)
table(a.out2$imputations[[3]]$signed)
@
Note that \Amelia\ can only fit imputations into categories that exist in
the original data. Thus, if there was a third category of signed, say 2,
that corresponded to a different kind of IMF agreement, but it never
occurred in the original data, \Amelia\ could not match imputations to it.

Since \Amelia\ properly treats a $ p$-category multinomial
variable as $ p-1$ variables, one should understand the number of
parameters that are quickly accumulating if many multinomial variables
are being used. If the square of the number of real and constructed
variables is large relative to the number of observations, it is useful to
use a ridge prior as in section \ref{sec:prior}.

\subsubsection{Natural Log}
\label{sec:log}
If one of your variables is heavily skewed or has outliers that may
alter the imputation in an unwanted way, you can use a natural
logarithm transformation of that variable in order to normalize its
distribution.  This transformed distribution helps \Amelia\
to avoid imputing values that depend too heavily on outlying data
points.  Log transformations are common in expenditure and economic
variables where we have strong beliefs that the marginal relationship
between two variables decreases as we move across the range.

For instance, figure \ref{fig:logshist} show the \code{tariff} variable
clearly has positive (or, right) skew while its natural log transformation
has a roughly normal distribution.

<<label=logshist, include=FALSE, echo=FALSE>>=
hist(freetrade$tariff, col="grey", border="white")
hist(log(freetrade$tariff), col="grey", border="white")
@ %$

<<echo=FALSE, results = hide>>=
options(SweaveHooks = list(fig = function() par(mfrow=c(1,2))))
@

\begin{figure}
\begin{center}
<<label=hist2, fig=TRUE, echo=FALSE, eps=FALSE, width=10, height=6>>=
<<logshist>>
@
\caption{Histogram of \texttt{tariff} and \texttt{log(tariff)}.}
\label{fig:logshist}
\end{center}
\end{figure}


<<echo=FALSE, results = hide>>=
options(SweaveHooks = list(fig = function() par(mfrow=c(1,1))))
@

\subsubsection{Square Root}
\label{sec:sqrt}
Event count data is often heavily skewed and has nonlinear
relationships with other variables.  One common transformation to
tailor the linear model to count data is to take the square roots of
the counts.  This is a transformation that can be set as an option in
\Amelia.

\subsubsection{Logistic}
\label{sec:lgstc}
Proportional data is sharply bounded between 0 and 1.  A logistic
transformation is one possible option in \Amelia\ to make the
distribution symmetric and relatively unbounded.

\subsection{Identification Variables}
\label{sec:idvars}
Datasets often contain identification variables, such as country
names, respondent numbers, or other identification numbers, codes or
abbreviations.  Sometimes these are text and sometimes these are
numeric.  Often it is not appropriate to include these variables in
the imputation model, but it is useful to have them remain in the
imputed datasets (However, there are models that would include the ID
variables in the imputation model, such as fixed effects model for
data with repeated observations of the same countries).
Identification variables which are not to be included in the
imputation model can be identified with the argument \code{idvars}.
These variables will not be used in the imputation model, but will be
kept in the imputed datasets.

If the \code{year} and \code{country} contained no information except
labels, we could omit them from the imputation:

<<results = hide>>=
amelia(freetrade, idvars = c("year", "country"))
@
Note that Amelia will return with an error if your dataset contains a
factor or character variable that is not marked as a nominal or
identification variable. Thus, if we were to omit the factor
\code{country} from the \code{cs} or \code{idvars} arguments, we would
receive an error:
<<>>=
a.out2 <- amelia(freetrade, idvars = c("year"))
@

In order to conserve memory, it is wise to remove unnecessary
variables from a data set before loading it into \Amelia.
The only variables you should include in your data when running
\Amelia\ are variables you will use in the analysis stage
and those variables that will help in the imputation model.  While it
may be tempting to simply mark unneeded variables as IDs, it only
serves to waste memory and slow down the imputation procedure.

\subsection{Time Series, or Time Series Cross Sectional Data} \label{sec:tscs}

Many variables that are recorded over time within a cross-sectional unit
are observed to vary smoothly over time.  In such cases, knowing the
observed values of observations close in time to any missing value may
enormously aid the imputation of that value.  However, the exact pattern
may vary over time within any cross-section.  There may be periods of
growth, stability, or decline; in each of which the observed values would
be used in a different fashion to impute missing values.  Also, these
patterns may vary enormously across different cross-sections, or may exist
in some and not others.  \Amelia\ can build a general model of patterns
within variables across time by creating a sequence of polynomials of the
time index.  If, for example, tariffs vary smoothly over time, then we
make the modeling assumption that there exists some polynomial that
describes the economy in cross-sectional unit $i$ at time $t$ as:
\begin{equation}
\textrm{tariff}_{ti} = \beta_0 + \beta_1 t + \beta_1 t^2 + \beta_1 t^3 \ldots
\end{equation}
And thus if we include enough higher order terms of time then the pattern
between observed values of the tariff rate can be estimated.  \Amelia\
will create polynomials of time up to the user defined $k$-th order,
($k\leq3$).

We can implement this with the \code{ts} and \code{polytime} arguments. If
we thought that a second-order polynomial would help predict we could run
<<results = hide>>=
a.out2 <- amelia(freetrade, ts = "year", cs = "country", polytime = 2)
@
With this input, \Amelia\ will add covariates to the model that correspond
to time and its polynomials. These covariates will help better predict the
missing values.

If cross-sectional units are specified these polynomials can be interacted
with the cross-section unit to allow the patterns over time to vary
between cross-sectional units.  Unless you strongly believe all units have
the same patterns over time in all variables (including the same constant
term), this is a reasonable setting.  When $k$ is set to 0, this
interaction simply results in a model of \emph{fixed effects} where every
unit has a uniquely estimated constant term.  \Amelia\ does not smooth
the observed data, and only uses this functional form, or one you choose,
with all the other variables in the analysis and the uncertainty of the
prediction, to impute the missing values.

In order to impute with trends specific to each cross-sectional unit,
we can set \code{intercs} to \code{TRUE}:

<<echo=TRUE, results = hide>>=
a.out.time <- amelia(freetrade, ts = "year", cs = "country", polytime = 2,
                 intercs = TRUE, p2s = 2)
@

Note that attempting to use \code{polytime} without the \code{ts}
argument, or \code{intercs} without the \code{cs} argument will result in
an error.

Using the \code{tscsPlot} function (discussed below), we can see in figure
\ref{fig:tcomp} that we have a much better prediction about the missing
values when incorporating time than when we omit it:

<<tcomp1, include = FALSE>>=
tscsPlot(a.out, cs = "Malaysia", main = "Malaysia (no time settings)",
         var = "tariff", ylim = c(-10, 60))

tscsPlot(a.out.time, cs = "Malaysia", main = "Malaysia (with time settings)",
         var = "tariff", ylim = c(-10, 60))
@


<<echo=FALSE, results = hide>>=
options(SweaveHooks = list(fig = function() par(mfrow=c(1,2))))
@
\begin{figure}
\begin{center}
<<label = timecompare, echo=FALSE, fig=TRUE, eps=FALSE, width=10, height=6>>=
<<tcomp1>>
@
\caption{The increase in predictive power when using polynomials of
      time. The panels shows mean imputations with 95\% bands (in red) and
    observed data point (in black). The left panel shows an imputation
    without using time and the right panel includes polynomials of time.}
  \label{fig:tcomp}
  \end{center}
\end{figure}

\subsubsection{Lags and Leads}
\label{sec:lags}

An alternative way of handling time-series information is to include lags
and leads of certain variables into the imputation model. \emph{Lags} are
variables that take the value of another variable in the previous time
period while \emph{leads} take the value of another variable in the next
time period. Many analysis models use lagged variables to deal with issues
of endogeneity, thus using leads may seems strange. It is important to
remember, however, that imputation models are predictive, not causal.
Thus, since both past and future values of a variable are likely
correlated with the present value, both lags and leads should improve the
model.

If we wanted to include lags and leads of tariffs, for instance, we would
simply pass this to the \code{lags} and \code{leads} arguments:

<<results = hide>>=
a.out2 <- amelia(freetrade, ts = "year", cs = "country", lags = "tariff",
                 leads = "tariff")
@


\subsection{Including Prior Information}

\Amelia\ has a number of methods of setting priors within the imputation
model.  Two of these are commonly used and discussed below, ridge priors
and observational priors.

\subsubsection{Ridge Priors for High Missingness, Small $n$'s, or Large Correlations}
\label{sec:prior}

When the data to be analyzed contain a high degree of missingness or
very strong correlations among the variables, or when the number of
observations is only slightly greater than the number of parameters
$p(p+3)/2$ (where $p$ is the number of variables), results from your
analysis model will be more dependent on the choice of imputation
model.  This suggests more testing in these cases of alternative
specifications under \Amelia. This can happen when using the
polynomials of time interacted with the cross section are included in
the imputation model. In our running example, we used a polynomial of
degree 2 and there are 9 countries. This adds $3 \times 9 - 1= 17$
more variables to the imputation model (One of the constant ``fixed
effects'' will be dropped so the model will be identified). When these
are added, the EM algorithm can become unstable, as indicated by the
vastly differing chain lengths for each imputation:
<<>>=
a.out.time
@

In these circumstances, we recommend adding a ridge prior which will help
with numerical stability by shrinking the covariances among the variables
toward zero without changing the means or variances. This can be done by
including the \code{empri} argument. Including this prior as a positive
number is roughly equivalent to adding \code{empri} artificial
observations to the data set with the same means and variances as the
existing data but with zero covariances.  Thus, increasing the
\code{empri} setting results in more shrinkage of the covariances, thus
putting more a priori structure on the estimation problem: like many
Bayesian methods, it reduces variance in return for an increase in bias
that one hopes does not overwhelm the advantages in efficiency.  In
general, we suggest keeping the value on this prior relatively small and
increase it only when necessary.  A recommendation of 0.5 to 1 percent of
the number of observations, $n$, is a reasonable starting value, and often
useful in large datasets to add some numerical stability.  For example, in
a dataset of two thousand observations, this would translate to a prior
value of 10 or 20 respectively.  A prior of up to 5 percent is moderate in
most applications and 10 percent is reasonable upper bound.

For our data, it is easy to code up a 1 percent ridge prior:
<<>>=
a.out.time2 <- amelia(freetrade, ts = "year", cs = "country", polytime = 2,
                 intercs = TRUE, p2s = 0, empri = .01*nrow(freetrade))
a.out.time2
@
This new imputation model is much more stable and, as shown by using
\code{tscsPlot}, produces about the same imputations as the original
model (see figure \ref{fig:tcomp2}):
<<tcomp2, include = FALSE>>=
tscsPlot(a.out.time, cs = "Malaysia", main = "Malaysia (no ridge prior)",
         var = "tariff", ylim = c(-10, 60))

tscsPlot(a.out.time2, cs = "Malaysia", main = "Malaysia (with ridge prior)",
         var = "tariff", ylim = c(-10, 60))
@

<<echo=FALSE, results = hide>>=
options(SweaveHooks = list(fig = function() par(mfrow=c(1,2))))
@

\begin{figure}
  \begin{center}
<<label = timecomp2, echo=FALSE, fig=TRUE, eps=FALSE, width=10, height=6>>=
<<tcomp2>>
@
    \caption{The difference in imputations when using no ridge prior
      (left) and when using a ridge prior set to 1\% of the data (right).}
  \label{fig:tcomp2}
  \end{center}
\end{figure}


\subsubsection{Observation-level priors}
\label{sec:obspri}

Researchers often have additional prior information about missing data
values based on previous research, academic consensus, or personal
experience.  \Amelia\ can incorporate this information to produce vastly
improved imputations.  The \Amelia\ algorithm allows users to include
informative Bayesian priors about individual missing data cells
instead of the more general model parameters, many of which have
little direct meaning.

The incorporation of priors follows basic Bayesian analysis where the
imputation turns out to be a weighted average of the model-based
imputation and the prior mean, where the weights are functions of the
relative strength of the data and prior: when the model predicts very
well, the imputation will down-weight the prior, and vice versa
\citep{HonKin10}.

The priors about individual observations should describe the analyst's
belief about the distribution of the missing data cell.  This can either
take the form of a mean and a standard deviation or a confidence interval.
For instance, we might know that 1986 tariff rates in Thailand around
$40\%$, but we have some uncertainty as to the exact value.  Our prior
belief about the distribution of the missing data cell, then, centers on
$40$ with a standard deviation that reflects the amount of uncertainty we
have about our prior belief.

To input priors you must build a priors matrix with either four or five
columns. Each row of the matrix represents a prior on either one
observation or one variable.  In any row, the entry in the first column is
the row of the observation and the entry is the second column is the
column of the observation.  In the four column priors matrix the third and
fourth columns are the mean and standard deviation of the prior
distribution of the missing value.

For instance, suppose that we had some expert prior information about
tariff rates in Thailand. We know from the data that Thailand is missing
tariff rates in many years,

<<>>=
freetrade[freetrade$country == "Thailand", c("year","country","tariff")]
@  %$


Suppose that we had expert information that tariff rates were roughly 40\%
in Thailand between 1986 and 1988 with about a 6\% margin of error.  This
corresponds to a standard deviation of about 3. In order to include this
information, we must form the priors matrix:

<<>>=
pr <- matrix(c(158,159,160,3,3,3,40,40,40,3,3,3), nrow=3, ncol=4)
pr
@

The first column of this matrix corresponds to the row numbers of Thailand
in these three years, the second column refers to the column number of
\code{tariff} in the data and the last two columns refer to the actual
prior. Once we have this matrix, we can pass it to \code{amelia},
<<results = hide>>=
a.out.pr <- amelia(freetrade, ts = "year", cs = "country", priors = pr)
@

In the five column matrix, the last three columns describe a confidence
range of the data. The columns are a lower bound, an upper bound, and a
confidence level between 0 and 1, exclusive.  Whichever format you choose,
it must be consistent across the entire matrix.  We could get roughly the
same prior as above by utilizing this method. Our margin of error implies
that we would want imputations between 34 and 46, so our matrix would be
<<>>=
pr.2 <- matrix(c(158,159,160,3,3,3,34,34,34,46,46,46,.95,.95,.95), nrow=3, ncol=5)
pr.2
@
These priors indicate that we are 95\% confident that these missing values
are in the range 34 to 46.

If a prior has the value 0 in the first column, this prior will be applied
to all missing values in this variable, except for explicitly set priors.
Thus, we could set a prior for the entire \code{tariff} variable of 20,
but still keep the above specific priors with the following code:


<<>>=
pr.3 <- matrix(c(158,159,160,0,3,3,3,3,40,40,40,20,3,3,3,5), nrow=4, ncol=4)
pr.3
@

%% add section on bounds, get rid of sessions, etc.

\subsubsection{Logical bounds}

In some cases, variables in the social sciences have known logical
bounds. Proportions must be between 0 and 1 and duration data
must be greater than 0, for instance. Many of these logical bounds can be
handled by using the correct transformation for that type of variable (see \ref{sec:trans} for more details
on the transformations handled by \Amelia). In the occasional case that
imputations must satisfy certain logical bounds not handled by these
transformations, \Amelia\ can take draws from a truncated normal
distribution in order to achieve imputations that satisfy the
bounds. Note, however, that this procedure imposes extremely strong
restrictions on the imputations and can lead to lower variances than the
imputation model implies. The mean value across all the imputed values of
a missing cell is the best guess from the imputation model of that missing value.
The variance of the distribution across imputed datasets correctly reflects the
uncertainty in that imputation.  It is often the mean imputed value that should conform
to the any known bounds, even if individual imputations are drawn beyond
those bounds.  The mean imputed value can be checked with the diagnostics
presented in the next section.  In general, building a more predictive
imputation model will lead to better imputations than imposing bounds.

\Amelia\ implements these bounds by rejection sampling. When drawing the
imputations from their posterior, we repeatedly resample until we have a
draw that satisfies all of the logical constraints. You can set an upper
limit on the number of times to resample with the \code{max.resample}
arguments. Thus, if after \code{max.resample} draws, the imputations are
still outside the bounds, \Amelia\ will set the imputation at the edge of
the bounds. Thus, if the bounds were 0 and 100 and all of the draws were
negative, \Amelia\ would simply impute 0.

As an extreme example, suppose that we know, for certain that tariff rates
had to fall between 30 and 40. This, obviously, is not true, but we can
generate imputations from this model. In order to specify these bounds, we
need to generate a matrix of bounds to pass to the \code{bounds}
argument. This matrix will have 3 columns: the first is the column for the
bounded variable, the second is the lower bound and the third is the upper
bound. Thus, to implement our bound on tariff rates (the 3rd column of the
dataset), we would create the matrix,

<<>>=
bds <- matrix(c(3, 30, 40), nrow = 1, ncol = 3)
bds
@

which we can pass to the \code{bounds} argument,

<<>>=
a.out.bds <- amelia(freetrade, ts = "year", cs = "country", bounds = bds,
                    max.resample = 1000)
@

The difference in results between the bounded and unbounded model are not
obvious from the output, but inspection of the imputed tariff rates for
Malaysia in figure \ref{fig:bcomp} shows that there has been a drastic
restriction of the imputations to the desired range:


<<bounds, include=FALSE>>=
tscsPlot(a.out, cs = "Malaysia", main = "No logical bounds", var =
         "tariff", ylim = c(-10,60))

tscsPlot(a.out.bds, cs = "Malaysia", main = "Bounded between 30 and 40", var =
         "tariff", ylim = c(-10,60))
@

Again, analysts should be extremely cautious when using these bounds as
they can seriously affect the inferences from the imputation model, as
shown in this example. Even when logical bounds exist, we recommend simply
imputing variables normally, as the violation of the logical bounds
represents part of the true uncertainty of imputation.

<<echo=FALSE, results = hide>>=
options(SweaveHooks = list(fig = function() par(mfrow=c(1,2))))
@

\begin{figure}
  \begin{center}
<<label = boundscomp, echo=FALSE, fig=TRUE, eps=FALSE, width=10, height=6>>=
<<bounds>>
@
    \caption{On the left are the original imputations without logical
      bounds and on the right are the imputation after imposing the
      bounds.}
  \label{fig:bcomp}
  \end{center}
\end{figure}


\subsection{Diagnostics}\label{sec:diag}

\Amelia\ currently provides a number of diagnostic tools to inspect the
imputations that are created.

\subsubsection{Comparing Densities}

One check on the plausibility of the imputation model is check the
distribution of imputed values to the distribution of observed values.
Obviously we cannot expect, \emph{a priori}, that these distribution will
be identical as the missing values may differ systematically from the
observed value--this is fundamental reason to impute to begin with! Imputations
with strange distributions or those that are far from the observed data
may indicate that imputation model needs at least some investigation and
possibly some improvement.

The \code{plot} method works on output from \code{amelia} and, by default,
shows for each variable a plot of the relative frequencies of the
observed data with an overlay of the relative frequency of the imputed
values.
<<plotmeth, include = FALSE>>=
plot(a.out, which.vars = 3:6)
@
where the argument \code{which.vars} indicates which of the variables to
plot (in this case, we are taking the 3rd through the 6th variables).

\begin{figure}
  \begin{center}
<<plot1, echo=FALSE, fig=TRUE, eps=FALSE, height = 7, width = 7>>=
<<plotmeth>>
@
\caption{The output of the \texttt{plot} method as applied to output from
  \texttt{amelia}. In the upper panels, the distribution of mean imputations
  (in red) is overlayed on the distribution of observed values (in black)
  for each variable. In the lower panels, there are no missing values and
  the distribution of observed values is simply plotted (in blue). Note
  that now imputed tariff rates are very similar to observed tariff rates,
  but the imputation of the Polity score are quite different. This is
  plausible if different types of regimes tend to be missing at different
  rates.}
\end{center}
\end{figure}

The imputed curve (in red) plots the density of the \emph{mean} imputation
over the $m$ datasets.  That is, for each cell that is missing in the
variable, the diagnostic will find the mean of that cell across each of
the $m$ datasets and use that value for the density plot. The black
distributions are the those of the observed data.  When variables are
completely observed, their densities are plotted in blue. These graphs
will allow you to inspect how the density of imputations compares to the
density of observed data. Some discussion of these graphs can be found in
\citet*{AbaGelLev08}.  Minimally, these graphs can be used to check that
the mean imputation falls within known bounds, when such bounds exist in
certain variables or settings.

We can also use the function \code{compare.density} directly to make these
plots for an individual variable:

<<include=FALSE>>=
compare.density(a.out, var = "signed")
@

<<echo=FALSE, results = hide>>=
options(SweaveHooks = list(fig = function() par(mfrow=c(1,1))))
@
\subsubsection{Overimpute}
\label{sec:overimpute}

\emph{Overimputing} is a technique we have developed to judge the fit
of the imputation model.  Because of the nature of the missing data
mechanism, it is impossible to tell whether the mean prediction of the
imputation model is close to the unobserved value that is trying to be
recovered.  By definition this missing data does not exist to create
this comparison, and if it existed we would no longer need the
imputations or care about their accuracy.  However, a natural question
the applied researcher will often ask is how accurate are these
imputed values?

Overimputing involves sequentially treating each of the
\emph{observed} values as if they had actually been missing.  For each
observed value in turn we then generate several hundred imputed values
of that observed value, \emph{as if it had been missing}.  While $m=5$
imputations are sufficient for most analysis models, this large number
of imputations allows us to construct a confidence interval of what
the imputed value would have been, had any of the observed data been
missing.  We can then graphically inspect whether our observed data
tends to fall within the region where it would have been imputed had
it been missing.

For example, we can run the overimputation diagnostic on our data by
running

<<overimp, include=FALSE>>=
overimpute(a.out, var = "tariff")
@

<<echo=FALSE, results = hide>>=
options(SweaveHooks = list(fig = function() par(mfrow=c(1,1))))
@

\begin{figure}[htp!]
  \begin{center}
<<oi2, echo=FALSE, fig=TRUE, eps=FALSE, width = 7, height = 7>>=
<<overimp>>
@
  \caption{An example of the overimputation diagnostic graph.  Here
    ninety percent confidence intervals are constructed that detail
    where an observed value would have been imputed had it been
    missing from the dataset, given the imputation model.  The dots
    represent the mean imputation.  Around ninety percent of these
    confidence intervals contain the $y=x$ line, which means that the
    true observed value falls within this range.  The color of the
    line (as coded in the legend) represents the fraction of missing
    observations in the pattern of missingness for that observation.}
  \label{f:oi2}
\end{center}
\end{figure}

Our overimputation diagnostic, shown in Figure~\ref{f:oi2}, runs this procedure
through all of the observed values for a user selected variable.  We can
graph the estimates of each observation against the true values of the
observation.  On this graph, a $y=x$ line indicates the line of perfect
agreement; that is, if the imputation model was a perfect predictor of the
true value, all the imputations would fall on this line.  For each
observation, \Amelia\ also plots 90\% confidence intervals that allows the
user to visually inspect the behavior of the imputation model. By checking
how many of the confidence intervals cover the $y=x$ line, we can tell how
often the imputation model can confidently predict the true value of the
observation.
<<overimp-bad, echo=FALSE, results = hide>>=
dd <- Amelia:::rmvnorm(50, mu = c(0.5,0.5), vcv =
                       matrix(c(0.25^2,.06, .06,0.25^2),2,2))
ddmiss <- sample(1:50, replace = FALSE, size = 10)
is.na(dd) <- ddmiss
aa.out <- amelia(dd, m= 5)
overimpute(aa.out, var = 2, main = "Observed versus Imputed Values")
@

\begin{figure}[htp!]
  \begin{center}
<<oi, echo=FALSE, results=hide, fig=TRUE, eps=FALSE, width = 7, height = 7>>=
<<overimp-bad>>
@
  \caption{Another example of the overimpute diagnostic graph. Note that
    the red lines are those observations that have fewer covariates
    observed and have a higher variance across the imputed values. }
  \label{f:oi-bad}
\end{center}
\end{figure}

Occasionally, the overimputation can display unintuitive results.  For
example, different observations may have different numbers of observed
covariates.  If covariates that are useful to the prediction are
themselves missing, then the confidence interval for this observation
will be much larger.  In the extreme, there may be observations where
the observed value we are trying to overimpute is \emph{the only}
observed value in that observation, and thus there is nothing left to
impute that observation with when we pretend that it is missing, other
than the mean and variance of that variable.  In these cases, we
should correctly expect the confidence interval to be very large.

An example of this graph is shown in figure \ref{f:oi-bad}.  In this
simulated bivariate dataset, one variable is overimputed and the
results displayed.  The second variable is either observed, in which
case the confidence intervals are very small and the imputations
(yellow) are very accurate, or the second variable is missing in which
case this variable is being imputed simply from the mean and variance
parameters, and the imputations (red) have a very large and
encompassing spread.  The circles represent the mean of all the
imputations for that value.  As the amount of missing information in a
particular pattern of missingness increases, we expect the width of
the confidence interval to increase.  The color of the confidence
interval reflects the percent of covariates observed in that pattern
of missingness, as reflected in the legend at the bottom.


\subsubsection{Overdispersed Starting Values}
\label{sec:overdisperse}

If the data given to \Amelia\ has a poorly behaved likelihood, the EM
algorithm can have problems finding a global maximum of the likelihood
surface and starting values can begin to effect imputations.  Because the
EM algorithm is deterministic, the point in the parameter space where you
start it can impact where it ends, though this is irrelevant when the
likelihood has only one mode.  However, if the starting values of an EM
chain are close to a local maximum, the algorithm may find this maximum,
unaware that there is a global maximum farther away.  To make sure that
our imputations do not depend on our starting values, a good test is to
run the EM algorithm from multiple, dispersed starting values and check
their convergence.  In a well behaved likelihood, we will see all of these
chains converging to the same value, and reasonably conclude that this is
the likely global maximum.  On the other hand, we might see our EM chain
converging to multiple locations.  The algorithm may also wander around
portions of the parameter space that are not fully identified, such as a
ridge of equal likelihood, as would happen for example, if the same
variable were accidentally included in the imputation model twice.

\Amelia\ includes a diagnostic to run the EM chain from multiple
starting values that are overdispersed from the estimated maximum.
The overdispersion diagnostic will display a graph of the paths of
each chain.  Since these chains move through spaces that are in an
extremely high number of dimensions and can not be graphically
displayed, the diagnostic reduces the dimensionality of the EM paths
by showing the paths relative to the largest principle components of
the final mode(s) that are reached.  Users can choose between graphing
the movement over the two largest principal components, or more simply
the largest dimension with time (iteration number) on the $x$-axis.
The number of EM chains can also be adjusted.  Once the diagnostic
draws the graph, the user can visually inspect the results to check
that all chains convergence to the same point.

For our original model, this is a simple call to \code{disperse}:

<<disp1d>>=
disperse(a.out, dims = 1, m = 5)
disperse(a.out, dims = 2, m = 5)
@

where \code{m} designates the number of places to start EM chains from and
\code{dims} are the number of dimensions of the principal components to
show.

<<echo=FALSE, results = hide>>=
options(SweaveHooks = list(fig = function() par(mfrow=c(1,2))))
@
\begin{figure}[ht]
  \begin{center}
<<disp1dfig, echo=FALSE, fig=TRUE, eps=FALSE, width=9, height=5.5>>=
<<disp1d>>
@
   \caption{A plot from the overdispersion diagnostic where all EM chains
     are converging to the same mode, regardless of starting value.  On
     the left, the $y$-axis represents movement in the (very high
     dimensional) parameter space, and the $x$-axis represents the
     iteration number of the chain. On the right, we visualize the
     parameter space in two dimensions using the first two principal
     components of the end points of the EM chains.  The iteration number
     is no longer represented on the $y$-axis, although the distance
     between iterations is marked by the distance between arrowheads on
     each chain.}
   \label{f:overgood}
  \end{center}
\end{figure}

In one dimension, the diagnostic plots movement of the chain on the
$y$-axis and time, in the form of the iteration number, on the $x$-axis.
Figures~\ref{f:overgood} show two examples of these plots.  The
first shows a well behaved likelihood, as the starting values all converge
to the same point.  The black horizontal line is the point where \Amelia\
converges when it uses the default method for choosing the starting
values.  The diagnostic takes the end point of this chain as the possible
maximum and disperses the starting values away from it to see if the chain
will ever finish at another mode.

% A few of the iterations of this diagnostic can ending up in vastly
% different locations of the parameter space.  This can happen for a variety
% of reasons.  For instance, suppose that we created another dataset and
% accidentally included a linear function of another variable in this dataset:
%<<echo=FALSE, results=hide>>=
%set.seed(02138)
%@
%<<>>=
%freetrade2 <- freetrade
%freetrade2$tariff2 <- freetrade2$tariff*2+3
%@
%<<echo=FALSE, results=hide>>=
%## We add a little bit of noise here to avoid numeric problems
%freetrade2$tariff2 <- freetrade2$tariff*2+3 +
%  rnorm(nrow(freetrade), 0, 0.000001)
%@

%If we tried to impute this dataset, \Amelia\ could draw imputations without
%any problems:
%
%<<>>=
%a.out.bad <- amelia(freetrade2, ts = "year", cs = "country")
%a.out.bad
%@

%But if we were to run \code{disperse}, we would end up with the
%problematic Figure~\ref{f:overbad}:

%<<dispbad, include=FALSE>>=
%disperse(a.out.bad, dims = 1, m = 10)
%@

% While this is a special case of a problematic likelihood, situations very
% similar to this can go undetected without using the proper diagnostics.
% More generally, an unidentified imputation model will lead to non-unique
% ML estimates (see \citet{King89} for a more detailed discussion of
% identification and likelihoods).


% <<echo=FALSE, results = hide>>=
% options(SweaveHooks = list(fig = function() par(mfrow=c(1,1))))
% @
%\begin{figure}
%  \begin{center}
%<<dispbadfig, echo=FALSE, fig=TRUE, width = 7, height = 7>>=
%<<dispbad>>
%@
%   \caption{ A problematic plot from the overdispersion diagnostic
%     showing that EM chains are converging to one of two different
%     modes, depending upon the starting value of the chain.}
%   \label{f:overbad}
%   \end{center}
% \end{figure}


\subsubsection{Time-series plots}
\label{sec:tscsplots}
<<echo=FALSE, results = hide>>=
options(SweaveHooks = list(fig = function() par(mfrow=c(1,1))))
@

As discussed above, information about time trends and fixed effects can
help produce better imputations. One way to check the plausibility of our
imputation model is to see how it predicts missing values in a time
series. If the imputations for the Malaysian tariff rate were drastically
higher in 1990 than the observed years of 1989 or 1991, we might worry
that there is a problem in our imputation model. Checking these time
series is easy to do with the \code{tscsPlot} command. Simply choose the
variable (with the \code{var} argument) and the cross-section (with the
\code{cs} argument) to plot the observed time-series along with
distributions of the imputed values for each missing time period. For
instance, we can run
<<tsplot1, include=FALSE>>=
tscsPlot(a.out.time, cs = "Malaysia", main = "Malaysia (with time settings)",
         var = "tariff", ylim = c(-10, 60))
@
to get the plot in figure \ref{fig:tsplot1}. Here, the black point are
observed tariff rates for Malaysia from 1980 to 2000. The red points are
the mean imputation for each of the missing values, along with their 95\%
confidence bands. We draw these bands by imputing each of missing values
100 times to get the imputation distribution for that observation.

\begin{figure}
  \begin{center}
<<tsplot2, echo=FALSE, fig=TRUE, width = 7, height = 7>>=
<<tsplot1>>
@
  \caption{Tariff rates in Malaysia, 1980-2000. An example of the
    \texttt{tscsPlot} function, the black points are observed values of
    the time series and the red points are the mean of the imputation
    distributions. The red lines represent the 95\% confidence bands of
    the imputation distribution.}
  \label{fig:tsplot1}
  \end{center}
\end{figure}

to get the plot in figure \ref{fig:tsplot1}. Here, the black point are
observed tariff rates for Malaysia from 1980 to 2000. The red points are
the mean imputation for each of the missing values, along with their 95\%
confidence bands. We draw these bands by imputing each of missing values
100 times to get the imputation distribution for that observation.

In figure \ref{fig:tsplot1}, we can see that the imputed 1990 tariff rate
is quite in line with the values around it. Notice also that values toward
the beginning and end of the time series have higher imputation variance.
This occurs because the fit of the polynomials of time in the imputation
model have higher variance at the beginning and end of the time
series. This is intuitive because these points have fewer neighbors from
which to draw predictive power.

A word of caution is in order. As with comparing the histograms of imputed
and observed values, there could be reasons that the missing values are
systematically different than the observed time series. For instance, if
there had been a major financial crisis in Malaysia in 1990 which caused
the government to close off trade, then we would expect that the missing
tariff rates should be quite different than the observed time series. If
we have this information in our imputation model, we might expect to see
out-of-line imputations in these time-series plots. If, on the other hand,
we did not have this information, we might see ``good'' time-series plots
that fail to point out this violation of the MAR assumption. Our
imputation model would produce poor estimates of the missing values since
it would be unaware that both the missingness and the true unobserved
tariff rate depend on another variable. Hence, the \code{tscsPlot} is
useful for finding obvious problems in imputation model and comparing the
efficiency of various imputation models, but it cannot speak to the
untestable assumption of MAR.

\subsubsection{Missingness maps}
\label{sec:missmaps}

One useful tool for exploring the missingness in a dataset is a
\emph{missingness map}. This is a map that visualizes the dataset a grid
and colors the grid by missingness status. The column of the grid are the
variables and the rows are the observations, as in any spreadsheet
program. This tool allows for a quick summary of the patterns of
missingness in the data.

If we simply call the \code{missmap} function on our output from
\code{amelia},
<<mmap1, include=FALSE>>=
missmap(a.out)
@
we get the plot in figure \ref{fig:missmap}. The \code{missmap} function
arrange the columns so that the variables are in decreasing order of
missingness from left to right. If the \code{cs} argument was set in the
\code{amelia} function, the labels for the rows will indicate where each
of the cross-sections begin.


\begin{figure}
  \begin{center}
<<mmap2, echo=FALSE, fig=TRUE, width = 7, height = 7>>=
<<mmap1>>
@
  \caption{Missingness map of the \texttt{freetrade} data. Missing values
    are in tan and observed values are in red.}
  \label{fig:missmap}
  \end{center}
\end{figure}

In figure \ref{fig:missmap}, it is clear that the tariff rate is the
variable most missing in the data and it tends to be missing in blocks of
a few observations.  Gross international reserves (\code{intresmi}) and
financial openness (\code{fivop}), on the other hand, are missing mostly
at the end of each cross-section. This suggests \emph{missingness by
  merging}, when variables with different temporal coverages are merged to
make one dataset. Sometimes this kind of missingness is an artifact of the
date at which the data was merged and researchers can resolve it by
finding updated versions of the relevant variables.

The missingness map is an important tool for understanding the patterns of
missingness in the data and can often indicate potential ways to improve
the imputation model or data collection process.

\subsection{Post-imputation Transformations}
\label{sec:postimptrans}
In many cases, it is useful to create transformations of the imputed
varibles for use in further analysis. For instance, one may want to
create an interaction between two variables or perform a
log-transformation on the imputed data. To do this, \Amelia\ includes
a \code{transform} function for \code{amelia} output that adds or
overwrites variables in each of the imputed datasets. For instance, if we wanted to create a log-transformation of the \code{gdp.pc}
variable, we could use the following command:
<<>>=
a.out <- transform(a.out, lgdp = log(gdp.pc))
head(a.out$imputations[[1]][,c("country", "year","gdp.pc", "lgdp")])
@ %$
To create an interaction between two variables, we could simply use:
<<>>=
a.out <- transform(a.out, pol_gdp = polity * gdp.pc)
@
Each transformation is recorded and the \code{summary} command prints
out each transformation that has been performed:
<<>>=
summary(a.out)
@
Note the updated output is almost exactly the same as the fresh
\code{amelia} output. You can pass the transformed output back to
\code{amelia} and it will add imputations and update these imputations
with the transformations you have performed.
\subsection{Analysis Models}
\label{sec:analysis}

Imputation is most often a data processing step as opposed to a final
model in of itself. To this end, it is easy to pass output from
\code{amelia} to other functions. The easiest and most integrated way to
run an analysis model is to pass the output to the \code{zelig} function
from the \code{Zelig} package. For example, in \citet{MilKub05}, the
dependent variable was tariff rates. We can replicate table 5.1 from their
analysis with the original data simply by running

<<results = hide>>=
require(Zelig)
z5 <- zls$new()
z5$zelig(tariff ~ polity + pop + gdp.pc + year + country, data = freetrade)
@

<<>>=
z5
@

Running the same model with imputed data is almost identical. Simply
replace the original data set with the  \code{amelia} output:


<<>>=
z5 <- zls$new()
z5$zelig(tariff ~ polity + pop + gdp.pc + year + country, data = a.out)
@ %$

<<>>=
z5
@


Zelig is one way to run analysis models on imputed data, but certainly not
the only way. The \code{imputations} list in the \code{amelia} output
contains each of the imputed datasets. Thus, users could simply program a
loop over the number of imputations and run the analysis model on each
imputed dataset and combine the results using the rules described in
\citet{KinHonJos01} and \citet{Schafer97}. Furthermore, users can easily
export their imputations using the \code{write.amelia} function as
described in \ref{sec:saving} and use statistical packages other than {\sf
  R} for the analysis model.

\Amelia\ also has the ability combine quantities of interest from
arbitrary models using the \code{mi.meld} function. This command takes
in a matrix with columns for the quantity and its standard error in
each of the imputed datasets. It then uses the standard rules for
combining multiple imputations to create an overall estimated quantity.

<<>>=
b.out<-NULL
se.out<-NULL
for(i in 1:a.out$m) {
  ols.out <- lm(tariff ~ polity + pop + gdp.pc, data = a.out$imputations[[i]])
  b.out <- rbind(b.out, ols.out$coef)
  se.out <- rbind(se.out, coef(summary(ols.out))[,2])
}

combined.results <- mi.meld(q = b.out, se = se.out)
combined.results
@ %$

In addition to the resources available in {\sf R}, users can draw on
Stata to implement their analysis models. As of version 11,
  Stata has built-in handling of multiply imputed datasets. In order
to utilize this functionality, simply export the ``stacked''
imputations using the \code{write.amelia} function:

\begin{verbatim}
> write.amelia(a.out, separate = FALSE, file.stem = "outdata", format = "dta")
\end{verbatim}

Once this stacked dataset is open in Stata, you must tell Stata that
it is an imputed dataset using the \code{mi import flong} command:
\begin{verbatim}
mi import flong, m(imp) id(year country) imp(tariff-usheg)
\end{verbatim}
The command takes a few options: \code{m} designates the imputation
variable (set with \code{impvar} in \code{write.amelia}), \code{id}
 sets the identifying varibles, and \code{imp} sets the variables that
 were imputed (or included in the imputation). The \code{tariff-usheg}
 indicates that Stata should treat the range of variables between
 \code{tariff} and \code{usheg} as imputed. Once we have set the
 dataset as imputed, we can use the built-in \code{mi} commands to
 analyze the data:
\begin{verbatim}
. mi estimate: reg tariff polity pop gdp_pc

Multiple-imputation estimates                     Imputations     =          5
Linear regression                                 Number of obs   =        171
                                                  Average RVI     =     1.4114
                                                  Complete DF     =        167
DF adjustment:   Small sample                     DF:     min     =      10.36
                                                          avg     =      18.81
                                                          max     =      37.62
Model F test:       Equal FMI                     F(   2,   10.4) =      15.50
Within VCE type:          OLS                     Prob > F        =     0.0008

------------------------------------------------------------------------------
      tariff |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      polity |  -.2058115   .3911049    -0.53   0.610    -1.072968    .6613452
         pop |   3.21e-08   8.72e-09     3.68   0.004     1.27e-08    5.14e-08
      gdp_pc |  -.0027561    .000644    -4.28   0.000    -.0040602   -.0014519
       _cons |   32.70461   2.660091    12.29   0.000     27.08917    38.32005
------------------------------------------------------------------------------
\end{verbatim}


\subsection[The amelia class]{The \texttt{amelia} class}
\label{sec:out}

The output from the \code{amelia} function is an instance of the S3 class
``amelia.'' Instances of the \code{amelia} class contain much more than
simply the imputed datasets. The \code{mu} object of the class contains
the posterior draws of the means of the complete data. The
\code{covMatrices} contains the posterior draws of the covariance
matrices of the complete data. Note that these correspond to the variables
as they are sent to the EM algorithm. Namely, they refer to the variables
after being transformed, centered and scaled.

The \code{iterHist} object is a list of \code{m} 3-column matrices. Each
row of the matrices corresponds to an iteration of the EM algorithm. The
first column indicates how many parameters had yet to converge at that
iteration. The second column indicates if the EM algorithm made a step
that decreased the number of converged parameters. The third column
indicates whether the covariance matrix at this iteration was
singular. Clearly, the last two columns are meant to indicate when the EM
algorithm enters a problematic part of the parameter space.

% \section{Multiple Overimputation in Amelia}
% \label{sec:mo}

% \citet{BlaHonKin14a} introduces a method, called multiple overimputation (MO), for handling missing data and measurement error at the same time. This approach works by treating any mismeasured cell in the data matrix as missing and imputing it, with a prior distribution on the missing value centered at the observed value with some variance. Practically, this approach can be implemented in Amelia using the \code{overimp} and \code{priors} arguments, but the construction of these matrices can be tedious. Amelia provides helper functions both estimate the measurement error variance and to run MO. 

% For a given variable we want to overimpute, we assume the following:
% $$w_i = x_i^* + u_i$$
% $$u_i|x_i^* \sim \mathcal{N}(0,\sigma^2_u)$$
% Here, $w_i$ is the observed variable, $x_i^*$ is the unobserved latent
% variable, and $u_i$ is the measurement error. We can think of the
% amount of measurement error, under this simple model, as $\sigma^2_u$.
% We can visualize the set of all $\sigma^2_u$ values as a continuum, where when
% $\sigma^2_u=0$ we observe the latent variable perfectly. When $\sigma^2_u = 0$
% there is no uncertainty about the location of the true value, so our
% belief about the true value, $x_i^*$, is simply a spike at the
% observed location, $w_i$. In between, we have some uncertainty about the location of $x^*_i$, but we know that values closer to $w_i$ are more likely.

% The main input required to run MO is this measurement error variance. \citet{BlaHonKin14a} and \citet{BlaHonKin14b} present a few different ways to set or estimate the amount of measurement error and Amelia can implement some of these approaches in the \code{moPrep} function. The basic idea is to pass the data and the measurement error structure to \code{moPrep} and then pass the output of \code{moPrep} to \code{amelia}. This will automatically handling the setting of \code{priors} and \code{overimp} for \code{amelia} and replace the observed, noisy data with overimputations. 

% To demonstrate this, suppose that we thought that the \code{polity} variable had measurement error in it and that we suspected that the standard deviation of the measurement error was 0.1. We can pass this to \code{moPrep} using the following commands:
%<<>>=
%m.out <- moPrep(freetrade, polity ~ polity, error.sd = 0.1)
%a.mo.out <- amelia(m.out, ts = "year", cs = "country")
%
%head(freetrade$polity)
%head(a.mo.out$imputations$imp1$polity)
%head(a.mo.out$imputations$imp2$polity)
%@ 
% There are a few things to notice. First, we pass the name of the data as the first argument to \code{moPrep}. Second, we pass a formula that indicates which variable should be overimputed (the left-hand side) and which variable that contains the mean of the measurement distribution (the right-hand side). By and large, these two variables will be the same, but if there was some known bias or scaling, we could incorporate that into the formula. Finally, we give the standard deviation of the error to the \code{error.sd} argument. Note that we gave \code{moPrep} a single number, but it is also possible to pass an entire vector of values, one for each observation, that would allow for heteroskedastic measurement error.

% In addition to specifying the error standard deviation, there are other ways of implicitly or explicitly estimating the error variance. First, we can identify units in the data that have no measurement error, called \emph{gold-standard observations}, and use those to calculate the error variance for the other units. This is possible through the \code{gold.standard} argument. Second, we can specify what proportion of the overall variance in a variable is due to measurement error, for either all observations or a different error proportion for each unit. This can be set with the \code{error.proportion} argument. Finally, if there are two proxies/mismeasurements of the same variable, we can use the covariance between the two to estimate the error variance. To implement this in \code{moPrep}, simply pass the second proxy in the formula: \code{errvar ~ errvar | proxy}.

\clearpage


\section[AmeliaView Menu Guide]{\AmeliaView\ Menu Guide}
\label{sec:menu}
Below is a guide to the \AmeliaView\ menus with references back to the
users's guide. The same principles from the user's guide apply to
\AmeliaView. The only difference is how you interact with the program.
Whether you use the GUI or the command line versions, the same
underlying code is being called, and so you can read the command
line-oriented discussion above even if you intend to use the GUI.

\subsection[Loading AmeliaView]{Loading \AmeliaView}

The easiest way to load \AmeliaView\ is to open an \R\ session and type
the following two commands:
\begin{verbatim}
> library(Amelia)
> AmeliaView()
\end{verbatim}
This will bring up the \AmeliaView\ window on any platform.

On the Windows operating system, there is an alternative way to start
\AmeliaView\ from the Desktop. See section \ref{sec:win-install} for a
guide on how to install this version. Once installed, there should be a
Desktop icon for \AmeliaView. Simply double-click this icon and the
\AmeliaView\ window should appear. If, for some reason, this approach does
not work, simply open an \R\ session and use the approach above.

\subsection[Loading a data set into AmeliaView]{Loading a data set into \AmeliaView }
\label{sec:step1}
\begin{figure}[ht]
  \centering
    \includegraphics[width = 6in]{splash.png}
  \caption{\AmeliaView\ welcome screen.}
  \label{f:splash}
\end{figure}

\AmeliaView\ load with a welcome screen (Figure \ref{f:splash}) that
has buttons which can load a data in many of the common formats. Each
of these will bring up a window for choosing your dataset. Note that
these buttons are only a subset of the possible ways to load data in
\AmeliaView. Under the \underline{F}ile menu (shown in Figure
\ref{f:file}), you will find more options, including the datasets
included in the package (\code{africa} and \code{freetrade}). You will
also find import commands for Comma-Separated Values (.CSV),
Tab-Delimited Text (.TXT), Stata v.5-10 (.DTA), SPSS (.DAT), and SAS
Transport (.XPORT). Note that when using a CSV file, Amelia assumes
that your file has a header (that is, a row at the top of the data
indicating the variable names).

\begin{figure}[ht]
  \centering
    \includegraphics[width = 3in]{import.png}
  \caption{\AmeliaView File and import menu.}
  \label{f:file}
\end{figure}


You can also load data from an RData file. If the RData file contains
more than one \code{data.frame}, a pop-up window will ask to you find
the dataset you would like to load. In the file menu, you can also
change the underlying working directory. This is where \AmeliaView\
will look for data by default and where it will save imputed
datasets.

\subsection{Variable dashboard}

\label{sec:step2}
\begin{figure}[ht]
  \centering
    \includegraphics[width=6in]{main.png}
  \caption{Main variable dashboard in \AmeliaView.}
  \label{f:main}
\end{figure}

Once a dataset is loaded, \AmeliaView\ will show the variable
dashboard (Figure \ref{f:main}). In this mode, you will see a table of
variables, with the current options for each of them shown, along with
a few summary statistics. You can reorder this table by any of these
columns by clicking on the column headings. This might be helpful to,
say, order the variables by mean or amount of missingness.

\label{sec:step2}
\begin{figure}[ht]
  \centering
    \includegraphics[width = 3in]{context-menu.png}
  \caption{Variable options via right-click menu on the variable dashboard.}
  \label{f:context}
\end{figure}

You can set options for individual variables by the right-click
context menu (Figure \ref{f:context}) or through the
\underline{V}ariables menu. For instance, clicking ``Set as
Time-Series Variable'' will set the currently selected variable in the
dashboard as the time-series variable. Certain options are disabled
until other options are enabled. For instance, you cannot add a lagged
variable to the imputation until you have set the time-series
variable. Note that any \code{factor} in the data is marked as a ID
variable by default, since a \code{factor} cannot be included in the
imputation without being set as an ID variable, a nominal variable, or
the cross-section variable. If there is a \code{factor} that fails to
meet one of these conditions, a red flag will appear next to the
variable name.

\begin{enumerate}
\item \textbf{Set as Time-Series Variable} - Sets the currently
  selected variable to the time-series variable. Disabled when more
  than one variable is selected. Once this is set, you can add lags
  and leads and add splines of time. The time-series variable will
  have a clock icon next to it.
\item \textbf{Set as Cross-Section Variable} - Sets the currently
  selected variable to the cross-section variable. Disabled when more
  than one variable is selected. Once this is set, you can interact
  the splines of time with the cross-section. The cross-section
  variable will have a person icon next to it.
\item \textbf{Unset as Time-Series Variable} - Removes the time-series
  status of the variable. This will remove any lags, leads, or splines
  of time.
\item \textbf{Unset as Cross-Section Variable} - Removes the cross-section
  status of the variable. This will remove any intersection of the
  splines of time and the cross-section.
\item \textbf{Add Lag/Lead} - Adds versions of the
  selected variables either lagged back (``lag'') or
  forward(``lead''). See \ref{sec:lags} above.
\item \textbf{Remove Lag/Lead} - Removes any lags or leads on
  the selected variables.
\item \textbf{Plot Histogram of Selected} - Plots a histogram of the
  selected variables. This command will attempt to put all of the
  histograms on one page, but if more than nine histograms are
  requested, they will appear on multiple pages.
\item \textbf{Add Transformation...} - Adds a transformation setting
  for the selected variables. Note that each variable can only have
  one transformation and the time-series and cross-section variables
  cannot be transformed.
\item \textbf{Remove Transformation} - Removes any transformation for
  the selected variables.
\item \textbf{Add or Edit Bounds} - Opens a dialog box to set logical
  bounds for the selected variable.
\end{enumerate}


\subsection[Amelia Options]{\Amelia\ Options}
\label{sec:optsmenu}
\begin{figure}[ht]
  \centering
    \includegraphics[width = 3in]{options.png}
  \caption{Options menu.}
  \label{f:opts}
\end{figure}
The Variable menu and the variable dashboard are the place to set
variable-level options, but global options are set in the
\underline{O}ptions menu.
\begin{enumerate}

\item \textbf{Splines of Time with...} - This option, if activated, will have
  \Amelia\ use flexible trends of time with the specified number of
  knots in the imputation. The higher the number of knots the greater
  the variation in the trend structure, yet it will take more degrees
  of freedom to estimate. For more information see \ref{sec:tscs}
  above.
\item \textbf{Interact with Cross-Section?} - Include and interaction
  of the cross-section with the time trends. This interaction is way
  of allowing the trend of time to vary across cases as well.  Using a
  0-level spline of time and interacting with the cross section is the
  equivalent of using a fixed effects.  For more information see
  \ref{sec:tscs} above.
\item \textbf{Add Observational Priors...} - Brings a dialog window to
  set prior beliefs about ranges for individual missing observations.
  For more information about observational priors, see
  \ref{sec:obspri}.
\item \textbf{Numerical Options} - Brings a dialog window to set the
  tolerance of the EM algorithm, the seed of the random number
  generator, the ridge prior for numerical stability, and the maximum
  number of redraws for the logical bounds.
\item \textbf{Draw Missingness Map} - Draws a missingness map. See
  \ref{sec:missmaps} for more details on missingness maps.
\item \textbf{Output File Options} - Bring a dialog to set the stub of
  the prefix of the imputed data files and the number of
  imputations. If you set the prefix to ``mydata'', your output files
  will be \texttt{mydata1.csv, mydata2.csv...} etc.
\item \textbf{Output File Type} - Sets the format of imputed data.
  If you would like to not save any output data sets (if you wanted,
  for instance, to simply look at diagnostics), set this option to
  ``(no save).''  Currently, you can save the output data as: Comma
  Separated Values (.CSV), Tab Delimited Text (.TXT), Stata (.DTA), R save
  object (.RData), or to hold it in \R\ memory. This last option will only
  work if you have called \AmeliaView\ from an \R\ session and want to
  return to the \R\ command line to work with the output. Its name in
  R workspace will be the file prefix. The stacked version of the
  Stata output will work with their built-in \code{mi} tools.

\end{enumerate}


\subsubsection{Numerical Options}
\label{sec:numopts}
\begin{figure}[h]
  \centering
    \includegraphics[width=2in]{numopts.png}
  \caption{Options menu.}
  \label{f:numopts}
\end{figure}
\begin{enumerate}

\item \textbf{Seed} - Sets the seed for the random number generator
  used by \Amelia. Useful if you need to have the same output twice.

\item \textbf{Tolerance} - Adjust the level of tolerance that \Amelia\
  uses to check convergence of the EM algorithm.  In very large
  datasets, if your imputation chains run a long time without
  converging, increasing the tolerance will allow a lower threshold to
  judge convergence and end chains after fewer iterations.

\item \textbf{Empirical Prior} - A prior that adds observations to
  your data in order to shrink the covariances.  A useful place to
  start is around 0.5\% of the total number of observations in the
  dataset (see \ref{sec:prior}).

\item \textbf{Maximum Resample for Bounds} - Amelia fits logical bounds by
  rejecting any draws that do not fall within the bounds. This value
  sets the number of times Amelia should attempt to resample to fit
  the bounds before setting the imputation to the bound.
\end{enumerate}


\subsubsection{Add Distribution Prior}\label{sec:refdistpri}
\begin{figure}[h]
  \centering
    \includegraphics[width = 5in]{distpri.png}
  \caption{Detail for Add Distributional Prior dialog}
\end{figure}
\begin{enumerate}
\item \textbf{Current Priors} - A table of current priors in distributional
  form, with the variable and case name. You can remove priors by
  selecting them and using the right-click context menu.
\item \textbf{Case} - Select the case name or number you wish to set
  the prior about.  You can also choose to make the prior for the
  entire variable, which will set the prior for any missing cell in
  that variable.  The case names are generated from the row name of
  the observation, the value of the cross-section variable of the
  observation and the value of the time series variable of the
  observation.
\item \textbf{Variable} - The variable associated with the prior you would like
  specify.  The list provided only shows the missing variables for the currently
  selected observation.
\item \textbf{Mean} - The mean value of the prior. The textbox will not accept
  letters or out of place punctuation.
\item \textbf{Standard Deviation} - The standard deviation of the prior.  The
  textbox will only accept positive non-zero values.
\end{enumerate}


\subsubsection{Add Range Prior}\label{sec:refrangepri}
\begin{figure}[h]
  \centering
    \includegraphics[width = 5in]{rangepri.png}
  \caption{Detail for Add Range Prior dialog.}
\end{figure}
\begin{enumerate}
\item \textbf{Case} - Select the case name or number you wish to set
  the prior about. You can also choose to make the prior for the
  entire variable, which will set the prior for any missing cell in
  that variable.  The case names are generated from the row name of
  the observation, the value of the cross-section variable of the
  observation and the value of the time series variable of the
  observation.
\item \textbf{Variable} - The variable associated with the prior you would like
  specify.  The list provided only shows the missing variables for the currently
  selected observation.
\item \textbf{Minimum} - The minimum value of the prior. The textbox will not accept
  letters or out of place punctuation.
\item \textbf{Maximum} - The maximum value of the prior. The textbox will not accept
  letters or out of place punctuation.
\item \textbf{Confidence} - The confidence level of the prior.  This should be
  between 0 and 1, non-inclusive.  This value represents how certain your priors
  are.  This value cannot be 1, even if you are absolutely certain of a give
  range.  This is used to convert the range into an appropriate distributional
  prior.
\end{enumerate}


\subsection{Imputing and checking diagnostics}
\label{sec:step3}


\begin{figure}[ht]
  \centering
    \includegraphics[width = 5in]{output-log.png}
  \caption{Output log showing \Amelia\ output for a successful imputation.}
\end{figure}
Once you have set all the relevant options, you can impute your data
by clicking the ``Impute!'' button in the toolbar. In the bottom right
corner of the window, you will see a progress bar that indicates the
progress of the imputations. For large datasets this could take some
time. Once the imputations are complete, you should see a ``Successful
Imputation!'' message appear where the progress bar was. You can click
on this message to open the folder containing the imputed datasets.

If there was an error during the imputation, the output log will
pop-up and give you the error message along with some information
about how to fix the problem. Once you have fixed the problem, simply
click ``Impute!'' again. Even if there was no error, you may want to
view the output log to see how \Amelia\ ran. To do so, simply click
the ``Show Output Log'' button. The log also shows the call to the
\code{amelia} function in \R. You can use this code snippet to run the
same imputation from the \R\ command line.\footnote{You will have to
  replace the \code{x} argument in the \code{amelia} call to the name
  of you dataset in the \R\ session. }


\subsubsection{Diagnostics Dialog}
\label{sec:diagdiag}
\begin{figure}[ht]
  \centering
    \includegraphics[width = 3.5in]{diag.png}
  \caption{Detail for Diagnostics dialog.}
\end{figure}

Upon the successful completion of an imputation, the diagnostics menu
will become available. Here you can use all of the diagnostics
available at the command-line.
\begin{enumerate}
\item \textbf{Compare Plots} - This will display the relative
  densities of the observed (red) and imputed (black) data.  The
  density of the imputed values are the average imputations across all
  of the imputed datasets.
\item \textbf{Overimpute} - This will run \Amelia\ on the full data
  with one cell of the chosen variable artificially set to missing and
  then check the result of that imputation against the truth.  The
  resulting plot will plot average imputations against true values
  along with 90\% confidence intervals.  These are plotted over a
  $y=x$ line for visual inspection of the imputation model.
\item \textbf{Number of overdispersions} - When running the
  overdispersion diagnostic, you need to run the imputation algorithm
  from several overdispersed starting points in order to get a clear
  idea of how the chain are converging.  Enter the number of
  imputations here.
\item \textbf{Number of dimensions} - The overdispersion diagnostic
  must reduce the dimensionality of the paths of the imputation
  algorithm to either one or two dimensions due to graphical
  restraints.
\item \textbf{Overdisperse} - Run overdispersion diagnostic to
  visually inspect the convergence of the \Amelia\ algorithm from
  multiple start values that are drawn randomly.
\end{enumerate}

\subsection{Sessions}
\label{sec:sessions}

It is often useful to save a session of \AmeliaView\ to save time if
you have impute the same data again. Using the \textbf{Save Session}
button will do just that, saving all of the current settings
(including the original and any imputed data) to an RData file. You
can then reload your session, on the same computer or any other,
simply by clicking the \textbf{Load Session} button and finding the
relevant RData file. All of the settings will be restored, including
any completed imputations. Thus, if you save the session after
imputing, you can always load up those imputations and view their
diagnostics using the sessions feature of \AmeliaView.

%\clearpage

% \section{Reference to Amelia's Functions}

% \include{Rd/africa}
% \include{Rd/amelia}
% \include{Rd/compare.density}
% \include{Rd/disperse}
% \include{Rd/freetrade}
% \include{Rd/missmap}
% \include{Rd/overimpute}
% \include{Rd/plot.amelia}
% \include{Rd/summary.amelia}
% \include{Rd/tscsPlot}
% \include{Rd/write.amelia}

\bibliographystyle{apsr}
\bibsep=0in
\bibliography{amelia}
\end{document}
