\documentclass{article}

%\usepackage{hyperref}
\usepackage{url}
\usepackage{Sweave}
\usepackage{rotating}
\usepackage{enumitem}
\usepackage{color}
\usepackage{makebox}
\usepackage{float}
\usepackage{array}
\usepackage{fancyhdr}
\usepackage{multirow}
%\usepackage[usenames,dvipsnames]{color}

\pagestyle{fancy}
\fancyhead{}
\fancyfoot{}
\renewcommand{\headrulewidth}{0pt}
\fancyhead[C]{\leftmark}
\fancyfoot[C]{\thepage}

\renewcommand{\arraystretch}{1.5}
%\setlength{\parskip}{10pt}

\definecolor{gray50}{gray}{0.5}

\DefineVerbatimEnvironment{Sinput}{Verbatim} {xleftmargin=0em,
                                              frame=single,
                                              rulecolor=\color{gray50},
                                              framesep=3mm,
                                              label=\tiny{R Input},
                                              samepage=true}
\DefineVerbatimEnvironment{Soutput}{Verbatim}{xleftmargin=0em,
                                              frame=single,
                                              rulecolor=\color{gray50},
                                              framesep=3mm,
                                              label=\tiny{R Output},
                                              samepage=true}
                                              
\newcommand{\Lagr}{\mathop{\mathcal{L}}}

<<echo=FALSE, results=hide>>=
options(continue = "  ", width = 60, warn = -1)
pdf.options(pointsize = 10)
library(tcpl)
tbls <- unlist(tcplQuery("SELECT name FROM sqlite_master WHERE type='table';"))
keep <- grepl("methods|categories", tbls)
for (i in tbls[!keep]) tcplSendQuery(paste("DELETE FROM", i))
@

%\VignetteIndexEntry{tcpl Overview}

\begin{document}
\SweaveOpts{concordance=true}
\hyphenpenalty=10000
\tolerance=10000

\title{The ToxCast\texttrademark{} Analysis Pipeline:\\ An R Package for Processing and Modeling Chemical Screening Data \\ \hfill \newline \large Version \Sexpr{packageVersion("tcpl")}}
\author{Dayne L. Filer, Parth Kothiya, Woodrow R. Setzer,\\ Richard S. Judson, Matthew T. Martin}
\maketitle

\clearpage

\tableofcontents
%\listoffigures
%\listoftables

\clearpage
\section*{Introduction}
\label{sec:intro}
\markboth{Introduction}{}
\thispagestyle{plain}
\addcontentsline{toc}{section}{Introduction}
\subsection*{Overview}
\label{subsec:overview}
\addcontentsline{toc}{subsection}{Overview}
The \texttt{tcpl} package was developed to process high-throughput and high-content screening data generated by the U.S. Environmental Protection Agency (EPA) ToxCast\texttrademark{} program.\footnote{\url{<http://www.epa.gov/ncct/toxcast/>}} ToxCast is screening thousands of chemicals with hundreds of assays coming from numerous and diverse biochemical and cell-based technology platforms. The diverse data, received in heterogeneous formats from numerous vendors, are transformed to a standard computable format and loaded into the \texttt{tcpl} database by vendor-specific R scripts. Once data is loaded into the database, ToxCast utilizes the generalized processing functions provided in this package to process, normalize, model, qualify, flag, inspect, and visualize the data. While developed primarily for ToxCast, we have attempted to make the \texttt{tcpl} package generally applicable to chemical-screening community. 

The \texttt{tcpl} package includes processing functionality for two screening paradigms: (1) single-concentration screening and (2) multiple-concentration screening. Single-concentration screening consists of testing chemicals at one concentration, often for the purpose of identifying potentially active chemicals to test in the multiple-concentration format. Multiple-concentration screening consists of testing chemicals across a concentration range, such that the modeled activity can give an estimate of potency, efficacy, etc.

Prior to the pipeline processing provided in this package, all the data must go through pre-processing (level 0). Level 0 pre-processing utilizes dataset-specific R scripts to process the heterogeneous data into a uniform format and to load the uniform data into the \texttt{tcpl} database. Level 0 pre-processing is outside the scope of this package, but can be done for virtually any high-throughput or high-content chemical screening effort, provided the resulting data includes the minimum required information.

In addition to storing the data, the \texttt{tcpl} database stores every processing/analysis decision at the assay component or assay endpoint level to facilitate transparency and reproducibility. For the illustrative purposes of this vignette we have included a SQLite version of the \texttt{tcpl} database containing a small subset of data from the ToxCast program. Because of differences in database capabilities, not all functionality of the package will work with the SQLite version. To best utilize the package the user should work with a MySQL database and the \texttt{RMySQL} package. The package includes a SQL file to initialize the MySQL database on the user's server of choice. Additionally, the MySQL version of the ToxCast database containing all the publicly available ToxCast data is available for download at: \url{<http://epa.gov/ncct/toxcast/data.html>}.

\subsection*{Package Settings}
\label{subsec:pkgsettings}
\addcontentsline{toc}{subsection}{Package Settings}
First, it is highly recommended for users to utilize the \texttt{data.table} package. The \texttt{tcpl} package utilizes the \texttt{data.table} package for all data frame-like objects. 
\hfill \newline
<<results=verbatim, echo=true, eval=true>>=
library(data.table)
library(tcpl)
## Store the path the tcpl directory for loading data
pkg_dir <- system.file(package = "tcpl")
@
\hfill \par
Every time the package is loaded in a new R session, a message similar to the following will print showing the default package settings:
\hfill \newline
\begin{Schunk}
\begin{Soutput}
tcpl (v1.0) loaded with the following settings:
  TCPL_DB:    /usr/local/lib64/R/library/tcpl/sql/xmpl.sqlite
  TCPL_USER:  NA
  TCPL_HOST:  NA
  TCPL_DRVR:  SQLite
Default settings stored in TCPL.conf. See ?tcplConf for 
more information.
\end{Soutput}
\end{Schunk}
\hfill \par

The package consists of five settings: (1) \texttt{\$TCPL\_DB} points to the \texttt{tcpl} database (either the SQLite file, as in the given example above, or the name of the MySQL database), (2) \texttt{\$TCPL\_USER} stores the username for accessing the database, (3) \texttt{\$TCPL\_PASS} stores the password for accessing the database, (4) \texttt{\$TCPL\_HOST} points to the MySQL server host, and (5) \texttt{\$TCPL\_DRVR} indicates which database driver to use (either ``MySQL'' or ``SQLite'').

Refer to \texttt{?tcplConf} for more information. At any time users can check the settings using \texttt{tcplConfList()}. An example of database settings would be as follows:

\hfill \newline
<<eval=FALSE>>=
tcplConf(drvr = "MySQL", 
         user = "root", 
         pass = "", 
         host = "localhost",
         db   = "toxcastdb")
@
\hfill \par

Note, \texttt{tcplSetOpts} will only make changes to the parameters given. The package is always loaded with the settings stored in the TCPL.config file located within the package directory. The user can edit the file, such that the package loads with the desired settings, rather than having to call the \texttt{tcplSetOpts} function every time. The TCPL.config file has to be edited whenever the package is updated or re-installed.

\subsection*{Assay Structure}
\label{subsec:assaystruc}
\addcontentsline{toc}{subsection}{Assay Structure}
The definition of an ``assay'' is, for the purposes of this package, broken into:
\begin{description}[labelindent=1cm]
  \item[assay\_source] -- the vendor/origination of the data
  \item[assay] -- the procedure to generate the component data
  \item[assay\_component] -- the raw data readout(s)
  \item[assay\_component\_endpoint] -- the normalized component data
\end{description}
Each assay element is represented by a separate table in the \texttt{tcpl} database. In general, we refer to an ``assay\_component\_endpoint'' as an ``assay endpoint.'' As we move down the hierarchy, each additional layer has a one-to-many relationship with the previous layer. For example, an assay component can have multiple assay endpoints, but an assay endpoint can derive only from a single assay component.

All processing occurs by assay component or assay endpoint, depending on the processing type (single-concentration or multiple-concentration) and level. No data are stored at the assay or assay source level. The ``assay'' and ``assay\_source'' tables store annotations to help in the processing and down-stream understanding/analysis of the data. For more information about the assay annotations and the ToxCast assays please refer to \url{<http://www.epa.gov/ncct/toxcast/>}.

Throughout the package the levels of assay hierarchy are defined and referenced by their primary keys (IDs) in the \texttt{tcpl} database: $\mathit{asid}$ (assay source ID), $\mathit{aid}$ (assay ID), $\mathit{acid}$ (assay component ID), and $\mathit{aeid}$ (assay endpoint ID). In addition, the package abbreviates the fields for the assay hierarchy names. The abbreviations mirror the abbreviations for the IDs with ``nm'' in place of ``id'' in the abbreviations, e.g. assay\_component\_name is abbreviated $\mathit{acnm}$.

\clearpage
\section*{Register and Upload New Data}
\label{sec:newdata}
\markboth{Register and Upload New Data}{}
\thispagestyle{plain}
\addcontentsline{toc}{section}{Register and Upload New Data}
This section explains how to register and upload new data into the \texttt{tcpl} database using a small subset of ToxCast data showing changes intracellular cortisol hormone. The subset of data comes from an assay measuring steroidogenesis through cellular levels of mutliple steroid hormones. 

The \texttt{tcpl} package provides three functions for adding new data: (1) \texttt{tcplRegister} to register a new assay or chemical ID, (2) \texttt{tcplUpdate} to change or add additional information for existing assay or chemical IDs, and (3) \texttt{tcplWriteLvl0} for loading data. Before writing any data to the \texttt{tcpl} database, the user has to register the assay and chemical information.

The first step in registering new assays is to register the assay source. As discussed in the previous section, the package refers to the levels of the assay hierarchy by their ID names, e.g. $\mathit{asid}$ for assay source.The following code shows how to register an assay source, then ensure the assay source was properly registered.
\hfill \newline
<<>>=
## Add a new assay source, call it CTox,
## that produced the data
tcplRegister(what = "asid", flds = list(asnm = "CTox"))
tcplLoadAsid()
@
\hfill \par
The \texttt{tcplRegister} function takes the abbreviation for $\mathit{assay\_source\_name}$, but the function will also take the unabbreviated form. The same is true of the \texttt{tcplLoadA-} functions, which load the information for the assay annotations stored in the database. The next steps show how to register, in order, an assay, assay component, and assay endpoints. 
\hfill \newline
<<>>=
tcplRegister(what = "aid", 
             flds = list(asid = 1, 
                         anm = "Steroidogenesis", 
                         assay_footprint = "96 well"))
@
\hfill \par
When registering an assay ($\mathit{aid}$), the user must give an $\mathit{asid}$ to map the assay to the correct assay source. Registering an assay, in addition to an assay\_name ($\mathit{anm}$) and $\mathit{asid}$, requires $\mathit{assay\_footprint}$. The $\mathit{assay\_footprint}$ field is used in the assay plate visualization functions (discussed later) to define the appropriate plate size. The $\mathit{assay\_footprint}$ field can take most string values, but only the numeric value will be extracted, e.g. the text string ``hello 384'' would indicate to draw a 384-well microtitier plate. Values containing multiple numeric values in $\mathit{assay\_footprint}$ may cause errors in plotting plate diagrams.

With the assay registered, the next step is to register an assay component. The example data presented here only contains data for one of the many steroids measured and only requires one assay component, but at this step the user could add multiple assay components to the ``Steroidogenesis'' assay.
\hfill \newline
<<>>=
tcplRegister(what = "acid", 
             flds = list(aid = 1, acnm = "CTox_CORT"))
tcplRegister(what = "aeid", 
             flds = list(acid = c(1, 1), 
                         aenm = c("CTox_CORT_up", 
                                  "CTox_CORT_dn"),
                         normalized_data_type = 
                         rep("log2_fold_induction", 2),
                         export_ready = c(1, 1),
                         burst_assay = c(0, 0),
                         fit_all = c(0, 0)))
@
\hfill \par
In the example above two assay endpoints were assigned to the assay component. Multiple endpoints allow for different normalization approaches of the data, in this case to detect activity in both the positive and negative directions (up and down). Notice registering an assay endpoint also requires the $\mathit{normalized\_data\_type}$ field. The $\mathit{normalized\_data\_type}$ field gives some default values for plotting. Currently the package supports three $\mathit{normalized\_data\_type}$ values: (1) ``percent\_activity,'' (2) ``log2\_fold\_induction,'' and (3) ``log10\_fold\_induction.'' Any other values will be treated as  ``percent\_activity.''

The other three additional fields when registering an assay endpoint do not have to be explicitly defined when working in the MySQL environment and will default to the values given above. All three fields represent Boolean values (1 or 0, 1 being \texttt{TRUE}). The $\mathit{export\_ready}$ field indicates (1) the data is done and ready for export or (0) still in progress. The $\mathit{burst\_assay}$ field is specific to multiple-concentration processing and indicates (1) the assay endpoint is included in the burst distribution calculation or (0) not (Appendix \ref{app:cyto}). The $\mathit{fit\_all}$ field is specific to multiple-concentration processing and indicates (1) the package should try to fit every concentration series, or (0) only attempt to fit concentration series that show evidence of activity (page \pageref{subsec:mc4}).

The final piece of assay information needed is the assay component source name (abbreviated $\mathit{acsn}$), stored in the ``assay\_component\_map'' table. The assay component source name is intended to simplify level 0 pre-processing by defining unique character strings (concatenating information if necessary) from the source files that identify the specific assay components. The unique character strings ($\mathit{acsn}$) get mapped to $\mathit{acid}$. An example of how to register a new $\mathit{acsn}$ will be given later in this section. 

With the minimal assay information registered, the next step is to register the necessary chemical and sample information. The ``chdat.csv'' file included in the package contains the sample and chemical information for the data that will be loaded. The following shows an example of how to load chemical information. Similar to the order in registering assay information, the user must first register chemicals, then register samples that map to chemical.
\hfill \newline
<<>>=
ch <- fread(file.path(pkg_dir, "sql", "chdat.csv"))
head(ch)

## Register the unique chemicals
tcplRegister(what = "chid", 
             flds = ch[ , 
                        unique(.SD), 
                        .SDcols = c("casn", "chnm")])
@
\hfill \par
The ``chdat.csv'' file contains a map of sample to chemical information, but chemical and sample information have to be registered separately because a chemical could potentially have multiple samples. Registering chemicals only takes a chemical CAS registry number ($\mathit{casn}$) and name ($\mathit{chnm}$). In the above example only the unique chemicals were loaded. The $\mathit{casn}$ and $\mathit{chnm}$ fields have unique constraints; trying to register multiple chemicals with the same name or CAS registry number is not possible and will result in an error. With the chemicals loaded the samples can be registered by mapping the sample ID ($\mathit{spid}$) to chemical ID. Note, the user needs to load the chemical information to get the chemical IDs then merge the new chemical IDs with the sample IDs from the original file by chemical name or CASRN.
\hfill \newline
<<>>=
cmap <- tcplLoadChem()
tcplRegister(what = "spid", 
             flds = merge(ch[ , list(spid, casn)], 
                          cmap[ , list(casn, chid)], 
                          by = "casn")[ , list(spid, chid)])
@
\hfill \par
Optionally, the user user can subdivide the chemcial IDs into different groups or libraries. For illustration, the chemical IDs will be arbitrarily divided into two chemical libraries, with the even numbered chemical IDs in group 1 and the odd numbered chemicals IDs in group 2.
\hfill \newline
<<>>=
grp1 <- cmap[chid %% 2 == 0, unique(chid)]
grp2 <- cmap[chid %% 2 == 1, unique(chid)]
tcplRegister(what = "clib", 
             flds = list(clib = "group_1", chid = grp1))
tcplRegister(what = "clib", 
             flds = list(clib = "group_2", chid = grp2))
@
\hfill \par
Chemical IDs can belong to more than one library, and will be listed as seperate entries when loading chemical library information.
\hfill \newline
<<>>=
tcplRegister(what = "clib", 
             flds = list(clib = "other", chid = 1:2))
tcplLoadClib(field = "chid", val = 1:2)
@
\hfill \par
After registering the chemical and assay information the data can be loaded into the \texttt{tcpl} database. The package includes two files from the ToxCast program, ``scdat.csv'' and ``mcdat.csv,'' with a subset of single- and multiple-concentration data, respectively. The single- and multiple-concentration processing require the same level 0 fields; more information about level 0 pre-processing in Appendix \ref{app:l0}.
\hfill \newline
<<>>=
scdat <- fread(file.path(pkg_dir, "sql", "scdat.csv"))
mcdat <- fread(file.path(pkg_dir, "sql", "mcdat.csv"))
c(unique(scdat$acsn), unique(mcdat$acsn))
@
\hfill \par
As discussed above, the final step before loading data is mapping the assay component source name ($\mathit{acsn}$) to the correct $\mathit{acid}$. An assay component can have multiple $\mathit{acsn}$ values, but an $\mathit{acsn}$ must be unique to one assay component. Assay components can have multiple $\mathit{acsn}$ values to minimize the amount of data manipulation required (and therefore potential errors) during the level 0 pre-processing if assay source files change or are inconsistent. The example data presented here only has one $\mathit{acsn}$ value, ``cort.''
\hfill \newline
<<>>=
tcplRegister(what = "acsn", 
             flds = list(acid = 1, acsn = "cort"))
@
\hfill \par
The data are now ready to be loaded with the \texttt{tcplWriteLvl0} function.
\hfill \newline
<<results=hide>>=
tcplWriteLvl0(dat = scdat, type = "sc")
tcplWriteLvl0(dat = mcdat, type = "mc")
@
\hfill \par
The \texttt{type} argument is used throughout the package to distinguish the type of data/processing: ``sc'' indicates single-concentration; ``mc'' indicates multiple-concentration. The \texttt{tcplLoadData} function can be used to load the data from the database.
\hfill \newline
<<>>=
tcplLoadData(lvl = 0, fld = "acid", val = 1, type = "sc")
@
\hfill \par
Notice in the loaded data the \texttt{acsn} is replaced by the correct $\mathit{acid}$ and the $\mathit{s0id}$ field is added. The ``s\#'' fields, and corresponding ``m\#'' fields in the multiple-concentration data, are the primary keys for each level of data. These primary keys link the various levels of data. All of the keys are auto-generated and will change anytime data are reloaded or processed. Note, the primary keys only change for the levels affected, e.g. if the user reprocesses level 1, the level 0 keys will remain the same. 

\clearpage
\section*{Data Processing and the tcplRun Function}
\label{sec:tcplrun}
\markboth{Data Processing and the tcplRun Function}{}
\thispagestyle{plain}
\addcontentsline{toc}{section}{Data Processing and the tcplRun Function}
This section is intended help the user understand the general aspects of how the data is processed before diving into the specifics of each processing level for both screening paradigms. The details of the two screening paradigms are provided in later sections.

All processing in the \texttt{tcpl} package occurs at the assay component or assay endpoint level. There is no capability within either screening paradigm to do any processing which combines data from multiple assay components or assay endpoints. Any combining of data must occur before or after the pipeline processing. For example, a ratio of two values could be processed through the pipeline if the user calculated the ratio during the level 0 pre-processing and uploaded a single ``component.''

Once data are uploaded in the database, data processing occurs through the \texttt{tcplRun} function for both single- and multiple-concentration screening. The \texttt{tcplRun} function can either take a single ID ($\mathit{acid}$ or $\mathit{aeid}$, depending on the processing type and level) or an $\mathit{asid}$. If given an $\mathit{asid}$ the \texttt{tcplRun} function will attempt to process all corresponding components/endpoints. When processing by $\mathit{acid}$ or $\mathit{aeid}$, the user must know which ID to give for each level (Table \ref{tab:proccheck}).

The processing is sequential, and every level of processing requires successful processing at the antecedent level. Any processing changes will cause a ``delete cascade,'' removing any subsequent data affected by the processing change to ensure complete data fidelity at any given time. For example, processing level 3 data will cause the data from levels 4 through 6 to be deleted for the corresponding IDs. Changing any method assignments will also trigger a delete cascade for any corresponding data (more on method assignments below).

The user must give a start and end level when using the \texttt{tcplRun} function. If processing more than one assay component or endpoint, the function will not stop if one component or endpoint fails. If a component or endpoint fails while processing multiple levels, the function will not attempt to processes the failed component/endpoint in subsequent levels. When finished processing, the \texttt{tcplRun} function returns a list indicating the processing success of each id. For each level processed the list will contain two elements: (1) ``l\#'' a named Boolean vector where \texttt{TRUE} indicates successful processing, and (2) ``l\#\_failed'' containing the names of any ids that failed processing where ``\#'' is the processing level.

The processing functions print messages to the console indicating the four steps of the processing. First, data for the given assay component ID are loaded, the data are processed, data for the same ID in subsequent levels are deleted, then the processed data is written to the database. The `outfile' parameter in the \texttt{tcplRun} function gives the user the option of printing all of the output text to a file.

The \texttt{tcplRun} function will attempt to use multiple processors on Unix-based systems (does not include Windows). Depending on the system environment, or if the user is running into memory constraints, the user may wish to use less processing power and can do so by setting the `mc.cores' parameter in the \texttt{tcplRun} function.

\begin{table}[h!]
  \centering
  \caption{Processing checklist}
  \noindent\makebox[\textwidth]{%
  \begin{tabular}{c c c c}
  Type & Level & Input ID & Method ID\\
  SC & Lvl 1 & \texttt{acid} & \texttt{aeid} \\ \hline
  SC & Lvl 2 & \texttt{aeid} & \texttt{aeid} \\ \hline \hline
  MC & Lvl 1 & \texttt{acid} & \texttt{N/A} \\ \hline
  MC & Lvl 2 & \texttt{acid} & \texttt{acid} \\ \hline
  MC & Lvl 3 & \texttt{acid} & \texttt{aeid} \\ \hline
  MC & Lvl 4 & \texttt{aeid} & \texttt{N/A} \\ \hline
  MC & Lvl 5 & \texttt{aeid} & \texttt{aeid} \\ \hline
  MC & Lvl 6 & \texttt{aeid} & \texttt{aeid} \\ \hline
  \multicolumn{4}{p{8cm}}{\footnotesize{The Input ID column indicates the ID used for each processing step; Method ID indicates the ID used for assigning methods for data processing, when necessary. SC = single-concentration; MC = multiple-concentration.}}
  \end{tabular}}
  \label{tab:proccheck}
\end{table}

The processing requirements vary by screening paradigm and level. Later sections will cover the details, but in general, many of the processing steps require specific methods to accommodate different experimental designs or data processing approaches.

Notice from Table \ref{tab:proccheck} that level 1 single-concentration processing (SC1) requires an $\mathit{acid}$ input (Table \ref{tab:proccheck}), but the methods are assigned by $\mathit{aeid}$. The same is true for MC3 processing. SC1 and MC3 are the normalization steps and convert $\mathit{acid}$ to $\mathit{aeid}$. (Only MC2 has methods assigned by $\mathit{acid}$.) The normalization process is discussed in the following section.

To promote reproducibility, all method assignments must occur through the database. Methods cannot be passed to either the \texttt{tcplRun} function or the low-level processing functions called by \texttt{tcplRun}. 

In general, method data are stored in the ``\_methods'' and ``\_id'' tables that correspond to the data-storing tables. For example, the ``sc1'' table is accompanied by the ``sc1\_methods'' table which stores the available methods for SC1, and the ``sc1\_aeid'' table which stores the method assignments and execution order.

The \texttt{tcpl} package provides three functions for easily modifying and loading the method assignments for the given assay components or endpoints: (1) \texttt{tcplMthdAssign} allows the user to assign methods, (2) \texttt{tcplMthdClear} clears method assignments, and (3) \texttt{tcplMthdLoad} queries the \texttt{tcpl} database and returns the method assignments. The package also includes the \texttt{tcplMthdList} function that queries the \texttt{tcpl} database and returns the list of available methods.

The following code blocks will give some examples of how to use the method-related functions.

\hfill \newline
<<>>=
## For illustrative purposes, assign level 2 MC methods to 
## ACIDs 98, 99. First check for available methods.
mthds <- tcplMthdList(lvl = 2, type = "mc")
mthds[1:2]
## Assign some methods to ACID 97, 98 & 99
tcplMthdAssign(lvl = 2, 
               id = 97:99, 
               mthd_id = c(3, 4, 2), 
               ordr = 1:3, 
               type = "mc")
tcplMthdLoad(lvl = 2, id = 97:99, type = "mc")
## Methods can be cleared one at a time for the given id(s)
tcplMthdClear(lvl = 2, id = 99, mthd_id = 2, type = "mc")
tcplMthdLoad(lvl = 2, id = 99, type = "mc")
## Or all methods can be cleared for the given id(s)
tcplMthdClear(lvl = 2, id = 97:98, type = "mc")
tcplMthdLoad(lvl = 2, id = 97:98, type = "mc")
@
\hfill \par

\clearpage
\section*{Data Normalization}
\label{sec:datanorm}
\markboth{Data Normalization}{}
\thispagestyle{plain}
\addcontentsline{toc}{section}{Data Normalization}

Data normalization occurs in both single- and multiple-concentration processing at levels 1 and 3, respectively. While the two paradigms use different methods, the normalization approach is the same for both single- and multiple-concentration processing. Data normalization does not have to occur within the package, and normalized data can be loaded into the database at level 0. However, \textbf{data must be zero-centered and will only be fit in the positive direction}.

The \texttt{tcpl} package supports fold-change and a percent of control approaches to normalization. All data must be zero-centered so all fold-change data must be log-transformed. Normalizing to a control requires three normalization methods: (1) one to define the baseline value, (2) one to define the control value, and (3) one to calculate percent of control (``resp.pc''). Normalizing to fold-change also requires three methods: (1) one to define the baseline value, (2) one to calculate the fold-change, and (3) one to log-transform the fold-change values. Methods defining a baseline value ($\mathit{bval}$) have the ``bval'' prefix, methods defining the control value ($\mathit{pval}$) have the ``pval'' prefix, and methods that calculate or modify the final response value have the ``resp'' prefix. For example, ``resp.log2'' does a log-transformation of the response value using a base value of 2. The formluae for calculating the percent of control and fold-change response values are listed in equations \ref{eq:pc} and \ref{eq:fc}, respectively.

The percent of control and fold-change values, respectively:
\begin{equation}
\label{eq:pc} \mathit{resp} = \frac{\mathit{cval} - \mathit{bval}}{\mathit{pval} - \mathit{bval}}100
\end{equation}
\begin{equation}
\label{eq:fc} \mathit{resp} = \mathit{cval}/\mathit{bval}
\end{equation}

Order matters when assigning normalization methods. The $\mathit{bval}$, and $\mathit{pval}$ if normalizing as a percent of control, need to be calculated prior to calculating the response value. Table \ref{tab:normxmpl} shows some possible normalization schemes.

\begin{table}[h!]
  \centering
  \caption{Example normalization method assignments.}
  \noindent\makebox[\textwidth]{%
  \begin{tabular}{c|p{4cm}|p{4cm}|p{4cm}|}
  \cline{2-4}
  \parbox[t]{2mm}{\multirow{4}{*}{\rotatebox[origin=c]{90}{Fold-Change}}} & 1. bval.apid.nwlls.med & 1. bval.apid.lowconc.med & 1. none \\
   & 2. resp.fc & 2. resp.fc & 2. resp.log10 \\
   & 3. resp.log2 & 3. resp.log2 & 3. resp.blineshift.50.spid \\
   & 4. resp.mult.neg1 & 4. & 4. \\ \cline{2-4}
  \parbox[t]{2mm}{\multirow{4}{*}{\rotatebox[origin=c]{90}{\% Control}}} & 1. bval.apid.lowconc.med & 1. bval.spid.lowconc.med & 1. none \\
   & 2. pval.apid.pwlls.med & 2. pval.apid.mwlls.med & 2. resp.multneg1 \\
   & 3. resp.pc & 3. resp.pc & 3. \\
   & 4. resp.multneg1 & 4. & 4. \\ \cline{2-4}
  \end{tabular}}
  \label{tab:normxmpl}
\end{table}

If the data does not require any normalization the ``none'' method must be assigned for normalization. The ``none'' method simply copies the input data to the response field. Without assigning ``none'' the response field will not get generated and the processing will not complete.

To reiterate, the package only models response in the positive direction. Therefore, signal in the negative direction must transformed to the positive direction during normalization. Negative direction data are inverted by multiplying the final response values by ${-1}$ (see the ``resp.mult.neg`'' methods in Table \ref{tab:normxmpl}).

In addition to the required normalization methods, the user can add additional methods to transform the normalized values. For example, the third fold-change example in Table \ref{tab:normxmpl} includes ``resp.blineshift.50.spid,'' which corrects for baseline deviations by $\mathit{spid}$. A complete list of available methods, by processing type and level, can be listed with \texttt{tcplMthdList}. More information is available in the package documentation, and can be found by running \texttt{??tcpl::Methods}.

As discussed in the Assay Structure section (page \pageref{subsec:assaystruc}), an assay component can have more than one assay endpoint. Creating multiple endpoints for one component enables multiple normalization approaches. Multiple normalization approaches may become necessary when the assay component detects signal in both positive and negative directions.

\clearpage
\section*{Single-concentration Screening}
\label{sec:snglconc}
\markboth{Single-concentration Screening}{}
\thispagestyle{plain}
\addcontentsline{toc}{section}{Single-concentration Screening}
This section will cover the \texttt{tcpl} process for handling single-concentration data\footnote{This section assumes a working knowledge of the concepts covered in the Data Processing and Data Normalization sections (pages \pageref{sec:tcplrun} and \pageref{sec:datanorm}, respectively).}. The goal of single-concentration processing is to identify potentially active compounds from a broad screen at a single concentration. After the data is loaded into the \texttt{tcpl} database, the single-concentration processing consists of 2 levels (Table \ref{tab:scsummary}).

\begin{table}[h!]
  \centering
  \caption{Summary of the \texttt{tcpl} single-concentration pipeline}
  \noindent\makebox[\textwidth]{%
  \begin{tabular}{c p{10cm}}
   & Description \\ \hline
  Lvl 0 & Pre-processing: Vendor/dataset-specific pre-processing to organize heterogeneous raw data to the uniform format for processing by the \texttt{tcpl} package$^\dagger$ \\ \hline
  Lvl 1 & Normalize: Apply assay endpoint-specific normalization listed in the ``sc1\_aeid'' table to the raw data to define response \\ \hline
  Lvl 2 & Activity Call: Collapse replicates by median response, define the response cutoff based on methods in the ``sc2\_aeid'' table, and determine activity \\ \hline
  \multicolumn{2}{l}{\footnotesize{$^\dagger$Level 0 pre-processing is outside the scope of this package}}
  \end{tabular}}
  \label{tab:scsummary}
\end{table}

\subsection*{Level 1}
\label{subsec:sc1}
\addcontentsline{toc}{subsection}{Level 1}
Level 1 processing converts the assay component to assay endpoint(s) and defines the normalized-response value field ($\mathit{resp}$); logarithm-concentration field ($\mathit{logc}$); and optionally, the baseline value ($\mathit{bval}$) and positive control value ($\mathit{pval}$) fields. The purpose of level 1 is to normalize the raw values to either the percentage of a control or to fold-change from baseline. The normalization process is discussed in greater detail in the Data Normalization section (page \pageref{sec:datanorm}).

Before beginning the normalization process, all wells with well quality ($\mathit{wllq}$) equal to 0 are removed.

The first step in beginning the processing is to identify which assay endpoints stem from the assay component(s) being processed.
\hfill \newline
<<>>=
tcplLoadAeid(fld = "acid", val = 1)
@
\hfill \par

With the corresponding endpoints identified, the appropriate methods can be assigned.

\hfill \newline
<<>>=
tcplMthdAssign(lvl = 1, 
               id = 1:2,
               mthd_id = c(1, 11, 13), 
               ordr = 1:3,
               type = "sc")
tcplMthdAssign(lvl = 1, 
               id = 2,
               mthd_id = 16, 
               ordr = 4,
               type = "sc")
@
\hfill \par

Above, methods 1, 11, and 13 were assigned for both endpoints. The method assignments instruct the processing to: (1) calculate $\mathit{bval}$ for each assay plate ID by taking the median of all data where the well type equals ``n;'' (2) calculate a fold-change over $\mathit{bval}$; (3) log-transform the fold-change values with base 2. The second method assignment (only for AEID 2) indicates to multiply all response values by $-1$.

For a complete list of normalization methods see \texttt{tcplMthdList(lvl = 1, type = "sc")} or \texttt{?SC1\_Methods}. With the assay endpoints and normalization methods defined, the data are ready for level 1 processing. 
\hfill \newline
<<>>=
## Do level 1 processing for acid 1
sc1_res <- tcplRun(id = 1, slvl = 1, elvl = 1, type = "sc")
@
\hfill \par
\textbf{Notice that level 1 processing takes an assay component ID, not an assay endpoint ID, as the input ID.} As mentioned in previously, the user must assign normalization methods by assay endpoint, then do the processing by assay component. The level 1 processing will attempt to process all endpoints in the database for a given component. If one endpoint fails for any reason (e.g., does not have appropriate methods assigned), the processing for the entire component fails.

\subsection*{Level 2}
\label{subsec:sc2}
\addcontentsline{toc}{subsection}{Level 2}
Level 2 processing defines the baseline median absolute deviation ($\mathit{bmad}$), collapses any replicates by sample ID, and determines the activity. 

Before the data are collapsed by sample ID, the $\mathit{bmad}$ is calculated as the median absolute deviation of all wells with well type equal to ``t.'' The calculation to define $\mathit{bmad}$ is done once across the entire assay endpoint. \textbf{If additional data is added to the database for an assay component, the $\mathit{bmad}$ values for all associated assay endpoints will change.} Note, this $\mathit{bmad}$ definition is different from the $\mathit{bmad}$ definition used for multiple-concentration screening.

To collapse the data by sample ID, the median response value is calculated at each concentration. The data are then further collapsed by taking the maximum of those median values ($\mathit{max\_med}$).

Once the data are collapsed, such that each assay endpoint-sample pair only has one value, the activity is determined. For a sample to get an active hit-call, the $\mathit{max\_med}$ must be greater than an efficacy cutoff. The efficacy cutoff is determined by the level 2 methods. The efficacy cutoff value ($\mathit{coff}$) is defined as the maximum of all values given by the assigned level 2 methods. Failing to assign a level 2 method will result in every sample being called active. For a complete list of level 5 methods see \texttt{tcplMthdList(lvl = 2, type = "sc")} or \texttt{?SC2\_Methods}.
\hfill \newline
<<>>=
## Assign a cutoff value of log2(1.2)
tcplMthdAssign(lvl = 2,
               id = 1:2,
               mthd_id = 3,
               type = "sc")
@
\hfill \par
For the example data the cutoff value is $log_2(1.2)$. If the maximum median value ($\mathit{max\_med}$) is greater than or equal to the efficacy cutoff ($\mathit{coff}$), the sample ID is considered active and the hit-call ($\mathit{hitc}$) is set to 1.

With the methods assigned, the level 2 processing can be completed.

\hfill \newline
<<>>=
## Do level 1 processing for acid 1
sc2_res <- tcplRun(id = 1:2, slvl = 2, elvl = 2, type = "sc")
@
\hfill \par

\clearpage
\section*{Multiple-concentration Screening}
\label{sec:multiconc}
\markboth{Multiple-concentration Screening}{}
\thispagestyle{plain}
\addcontentsline{toc}{section}{Multiple-concentration Screening}
This section will cover the \texttt{tcpl} process for handling multiple-concentration data\footnote{This section assumes a working knowledge of the concepts covered in the Data Processing and Data Normalization sections (pages \pageref{sec:tcplrun} and \pageref{sec:datanorm}, respectively).}. The goal of multiple-concentration processing is to estimate the activity, potency, efficacy, and other parameters for sample-assay pairs. After the data is loaded into the \texttt{tcpl} database, the multiple-concentration processing consists of six levels (Table \ref{tab:mcsummary}).

\begin{table}[h!]
  \centering
  \caption{Summary of the \texttt{tcpl} multiple-concentration pipeline}
  \noindent\makebox[\textwidth]{%
  \begin{tabular}{c p{10cm}}
   & Description \\ \hline
  Lvl 0 & Pre-processing: Vendor/dataset-specific pre-processing to organize heterogeneous raw data to the uniform format for processing by the \texttt{tcpl} package$^\dagger$ \\ \hline
  Lvl 1 & Index: Define the replicate and concentration indices to facilitate all subsequent processing \\ \hline
  Lvl 2 & Transform: Apply assay component-specific transformations listed in the ``mc2\_acid'' table to the raw data to define the corrected data \\ \hline
  Lvl 3 & Normalize: Apply assay endpoint-specific normalization listed in the ``mc3\_aeid'' table to the corrected data to define response \\ \hline
  Lvl 4 & Fit: Model the concentration-response data utilizing three objective functions: (1) constant, (2) hill, and (3) gain-loss \\ \hline
  Lvl 5 & Model Selection/Acitivty Call: Select the winning model, define the response cutoff based on methods in the ``mc5\_aeid'' table, and determine activity \\ \hline
  Lvl 6 & Flag: Flag potential false positive and false negative findings based on methods in the ``mc6\_aeid'' table \\ \hline
  \multicolumn{2}{l}{\footnotesize{$^\dagger$Level 0 pre-processing is outside the scope of this package}}
  \end{tabular}}
  \label{tab:mcsummary}
\end{table}

\subsection*{Level 1}
\label{subsec:mc1}
\addcontentsline{toc}{subsection}{Level 1}
Level 1 processing defines the replicate and concentration index fields to facilitate downstream processing. Because of cost, availability, physicochemical, and technical constraints screening-level efforts utilize numerous experimental designs and test compound (sample) stock concentrations. The resulting data may contain inconsistent numbers of concentrations, concentration values, and technical replicates. To enable quick and uniform processing, level 1 processing explicitly defines concentration and replicate indices, giving integer values $1 \dots N$ to increasing concentrations and technical replicates, where $1$ represents the lowest concentration or first technical replicate.

To assign replicate and concentration indices we assume one of two experimental designs. The first design assumes samples are plated in multiple concentrations on each assay plate, such that the concentration series all falls on a single assay plate. The second design assumes samples are plated in a single concentration on each assay plate, such that the concentration series falls across many assay plates.

For both experimental designs, data are ordered by source file ($\mathit{srcf}$), assay plate ID ($\mathit{apid}$), column index ($\mathit{coli}$), row index ($\mathit{rowi}$), sample ID ($\mathit{spid}$), and concentration ($\mathit{conc}$). Concentration is rounded to three significant figures to correct for potential rounding errors. After ordering the data we create a temporary replicate ID, identifying an individual concentration series. For test compounds in experimental designs with the concentration series on a single plate and all control compounds, the temporary replicate ID consists of the sample ID, well type ($\mathit{wllt}$), source file, assay plate ID, and concentration. The temporary replicate ID for test compounds in experimental designs with concentration series that span multiple assay plates is defined similarly, but does not include assay plate ID.

Once the data are ordered, and the temporary replicate ID is defined, the data are scanned from top to bottom and increment the replicate index ($\mathit{repi}$) every time a replicate ID is duplicated. Then, for each replicate, the concentration index ($\mathit{cndx}$) is defined by ranking the unique concentrations, with the lowest concentration starting at 1.

The following demonstrates how to carry out the level 1 processing and look at the resulting data:
\hfill \newline
<<>>=
## Do level 1 processing for acid 1
mc1_res <- tcplRun(id = 1, slvl = 1, elvl = 1, type = "mc")
@
\hfill \par
With the processing complete, the resulting level 1 data can be loaded to check the processing:
\hfill \newline
<<>>=
## Load the level 1 data and look at the cndx and repi values
m1dat <- tcplLoadData(lvl = 1, 
                      fld = "acid", 
                      val = 1, 
                      type = "mc")
m1dat <- tcplPrepOtpt(m1dat)
setkeyv(m1dat, c("repi", "cndx"))
m1dat[chnm == "3-Phenylphenol", list(chnm, conc, cndx, repi)]
@
\hfill \par
3-phenylphenol contains two replicates, each with six distinct concentrations. The package also contains a tool for visualizing the data at the assay plate level.
\setkeys{Gin}{width=1\textwidth}
\begin{figure}[h!]
<<l1apid, fig=true, strip.white=true, echo=true, height=6.6667, width=10>>=
tcplPlotPlate(dat = m1dat, apid = "09Apr2014.Plate.17")
@
\caption{An assay plate diagram. The color indicates the raw values according to the key on the right. The bold lines on the key show the distribution of values for the plate on the scale of values across the entire assay. The text inside each well shows the well type and concentration index. For example, ``t4'' indicates a test compound at the fourth concentration. The wells with an ``X'' have a well quality of 0.}
\label{fig:l1apid}
\end{figure}
In Figure \ref{fig:l1apid} we see the results of \texttt{tcplPlotPlate}. The \texttt{tcplPlotPlate} function can be used to visualize the data at levels 1 to 3. The row and column indices are printed along the edge of the plate, with the values in each well represented by color. While the plate does not give sample ID information, the letter/number codes in the wells indicate the well type and concentration index, respectively. The plate display also shows the wells with poor quality (as defined by the well quality, $\mathit{wllq}$, field at level 0) with an ``X.'' Plotting plates in subsequent levels wells with poor quality will appear empty. The title of the plate display lists the assay component/assay endpoint and the assay plate ID ($\mathit{apid}$).

\subsection*{Level 2}
\label{subsec:mc2}
\addcontentsline{toc}{subsection}{Level 2}
Level 2 processing removes data where the well quality ($\mathit{wllq}$) equals 0 and defines the corrected value ($\mathit{cval}$) field. Level 2 processing allows for any transformation of the raw values at the assay component level. Examples of transformation methods could range from basic logarithm transformations, to complex spacial noise reduction algorithms. Currently the \texttt{tcpl} package only consists of basic transformations, but could be expanded in future releases. Level 2 processing does not include normalization methods; normalization should occur during level 3 processing.

For the example data used in this vignette, no transformations are necessary at level 2. To not apply any transformation methods, assign the ``none'' method:
\hfill \newline
<<>>=
tcplMthdAssign(lvl = 2,
               id = 1,
               mthd_id = 1, 
               ordr = 1, 
               type = "mc")
@
\hfill \par
Every assay component needs at least one transformation method assigned to complete level 2 processing. With the method assigned, the processing can be completed.
\hfill \newline
<<>>=
## Do level 2 processing for acid 1
mc2_res <- tcplRun(id = 1, slvl = 2, elvl = 2, type = "mc")
@
\hfill \par
For the complete list of level 2 transformation methods currently available, see \texttt{tcplMthdList(lvl = 2, type = "mc")} or \texttt{?MC2\_Methods} for more detail. The coding methodology used to implement the methods is beyond the scope of this vignette, but, in brief, the method names in the database correspond to a function name in the list of functions returned by \texttt{mc2\_mthds()} (the \texttt{mc2\_mthds} function is not exported, and not intended for use by the user). Each of the functions in the list given by \texttt{mc2\_mthds()} only return expression objects that processing function called by \texttt{tcplRun} executes in the local function environment to avoid making additional copies of the data in memory. We encourage suggestions for new methods.

\subsection*{Level 3}
\label{subsec:mc3}
\addcontentsline{toc}{subsection}{Level 3}
Level 3 processing converts the assay component to assay endpoint(s) and defines the normalized-response value field ($\mathit{resp}$); logarithm-concentration field ($\mathit{logc}$); and optionally, the baseline value ($\mathit{bval}$) and positive control value ($\mathit{pval}$) fields. The purpose of level 3 processing is to normalize the corrected values to either the percentage of a control or to fold-change from baseline. The normalization process is discussed in greater detail in the Data Normalization section (page \pageref{sec:datanorm}). The processing aspect of level 3 is almost completely analogous to level 2, except the user has to be careful about using assay component versus assay endpoint.

The user first needs to check which assay endpoints stem from the the assay component queued for processing.
\hfill \newline
<<>>=
## Look at the assay endpoints for acid 1
tcplLoadAeid(fld = "acid", val = 1)
@
\hfill \par
With the corresponding assay endpoints listed, the normalization methods can be assigned.
<<>>=
tcplMthdAssign(lvl = 3,
               id = 1:2,
               mthd_id = c(17, 9, 7), 
               ordr = 1:3, 
               type = "mc")
tcplMthdAssign(lvl = 3, 
               id = 2,
               mthd_id = 6, 
               ordr = 4, 
               type = "mc")
@
Above, methods 17, 9, and 7 were assigned for both endpoints. The method assignments instruct the processing to: (1) calculate $\mathit{bval}$ for each assay plate ID by taking the median of all data where the well type equals ``n'' or the well type equals ``t'' and the concentration index is 1 or 2; (2) calculate a fold-change over $\mathit{bval}$; (3) log-transform the fold-change values with base 2. The second method assignment (only for AEID 2) tells the processing to multiply all response values by $-1$.

For a complete list of normalization methods see \texttt{tcplMthdList(lvl = 3, type = "mc")} or \texttt{?MC3\_Methods}. With the assay endpoints and normalization methods defined, the data are ready for level 3 processing. 
\hfill \newline
<<>>=
## Do level 3 processing for acid 1
mc3_res <- tcplRun(id = 1, slvl = 3, elvl = 3, type = "mc")
@
\hfill \par
\textbf{Notice that level 3 processing takes an assay component ID, not an assay endpoint ID, as the input ID.} As mentioned in previous sections, the user must assign normalization methods by assay endpoint, then do the processing by assay component. The level 3 processing will attempt to process all endpoints in the database for a given component. If one endpoint fails for any reason (e.g., does not have appropriate methods assigned), the processing for the entire component fails.

\subsection*{Level 4}
\label{subsec:mc4}
\addcontentsline{toc}{subsection}{Level 4}
Level 4 processing splits the data into concentration series by sample and assay endpoint, then models the activity of each concentration series. Activity is modeled only in the positive direction. More information on readouts with both directions is available in the previous section.

The first step in level 4 processing is to remove the well types with only one concentration. To establish the noise-band for the assay endpoint, the baseline median absolute deviation ($\mathit{bmad}$) is calculated as the median absolute deviation of the response values for test compounds where the concentration index equals 1 or 2. The calculation to define $\mathit{bmad}$ is done once across the entire assay endpoint. \textbf{If additional data is added to the database for an assay component, the $\mathit{bmad}$ values for all associated assay endpoints will change.} Note, this $\mathit{bmad}$ definition is different from the $\mathit{bmad}$ definition used for single-concentration screening.

Before the model parameters are estimated, a set of summary values are calculated for each concentration series: the minimum and maximum response; minimum and maximum log concentration; the number of concentrations, points, and replicates; the maximum mean and median with the concentration at which they occur; and the number of medians greater than $3\mathit{bmad}$. When referring to the concentration series the ``mean'' and ``median'' values are defined as the mean or median of the response values at every concentration. In other words, the maximum median is the maximum of all median values across the concentration series.

Concentration series must have at least four concentrations to enter the fitting algorithm. By default, concentration series must additionally have at least one median value greater than $3\mathit{bmad}$ to enter the fitting algorithm. The median value above $3\mathit{bmad}$ requirement can be ignored by setting $\mathit{fit\_all}$ to 1 in the assay endpoint annotation. 

All models draw from the Student's t-distribution with four degrees of freedom. The wider tails in the t-distribution diminish the influence of outlier values, and produce more robust estimates than do the more commonly used normal distribution. The robust fitting removes the need for any outlier elimination before fitting. The fitting algorithm utilizes maximum likelihood estimates parameters for three models as defined below in equations \ref{eq:z} through \ref{eq:gnlsc6}.

Let $t(z,\nu)$ be the Student's t-distribution with $\nu$ degrees of freedom, $y_{i}$ be the observed response at the $i^{th}$ observation, and $\mu_{i}$ be the estimated response at the $i^{th}$ observation. We calculate $z_{i}$ as
\begin{equation}
\label{eq:z}
z_{i} = \frac{y_{i} - \mu_{i}}{\exp(\sigma)}\mathrm{,}
\end{equation}
where $\sigma$ is the scale term. Then the log-likelihood is
\begin{equation}
\label{eq:ll}
\sum_{i=1}^{n} [\ln\left(t(z_{i}, 4)\right) - \sigma]\mathrm{,}
\end{equation}
where $n$ is the number of observations.

The first model fit in the fitting algorithm is a constant model at 0, abbreviated ``cnst.'' The constant model only has one parameter, the scale term. For the constant model $\mu_{i}$ is given by
\begin{equation}
\label{eq:cnst}
\mu_{i} = 0\mathrm{.}
\end{equation}

The second model in the fitting algorithm is a constrained Hill model (hill), where the bottom asymptote is forced to 0. Including the scale parameter, the Hill model has four parameters. Let $\mathit{tp}$ be the top asymptote, $\mathit{ga}$ be the AC$_{50}$\footnote{The AC$_{50}$ is the activity concentration at 50\%, or the concentration where the modeled activity equals 50\% of the top asymptote.} in the gain direction, $\mathit{gw}$ be the Hill coefficient in the gain direction, and $x_{i}$ be the log concentration at the $i^{th}$ observation. Then $\mu_{i}$ for the Hill model is given by
\begin{equation}
\label{eq:hill}
\mu_{i} = \frac{tp}{1 + 10^{(\mathit{ga} - x_{i})\mathit{gw}}}\mathrm{,}
\end{equation}
with the constraints
\begin{equation}
\label{eq:hillc1}
0 \leq \mathit{tp} \leq 1.2\mathrm{max\;resp,}
\end{equation}
\begin{equation}
\label{eq:hillc2}
\mathrm{min\;logc} - 2 \leq \mathit{ga} \leq \mathrm{max\;logc} + 0.5\mathrm{,}
\end{equation}
and 
\begin{equation}
\label{eq:hillc3}
0.3 \leq \mathit{gw} \leq 8\mathit{.}
\end{equation}

The third model in the fitting algorithm is a constrained gain-loss model (gnls), defined as a product of two Hill models, with a shared top asymptote and both bottom asymptote values equal to 0. Including the scale term, the gain-loss model has six parameters. Let $\mathit{tp}$ be the shared top asymptote, $\mathit{ga}$ be the AC$_{50}$ in the gain direction, $\mathit{gw}$ be the Hill coefficient in the gain direction, $\mathit{la}$ be the AC$_{50}$ in the loss direction, $\mathit{lw}$ be the Hill coefficient in the loss direction, and $x_{i}$ be the log concentration at the $i^{th}$ observation. Then $\mu_{i}$ for the gain-loss model is given by
\begin{equation}
\label{eq:gnls}
\mu_{i} = \mathit{tp}\left(\frac{1}{1 + 10^{(\mathit{ga} - x_{i})\mathit{gw}}}\right)\left(\frac{1}{1 + 10^{(x_{i} - \mathit{la})\mathit{lw}}}\right)\mathrm{,}
\end{equation}
with the constraints
\begin{equation}
\label{eq:gnlsc1}
0 \leq \mathit{tp} \leq 1.2\mathrm{max\;resp,}
\end{equation}
\begin{equation}
\label{eq:gnlsc2}
\mathrm{min\;logc} - 2 \leq \mathit{ga} \leq \mathrm{max\;logc,}
\end{equation}
\begin{equation}
\label{eq:gnlsc3}
0.3 \leq \mathit{gw} \leq 8\mathrm{,}
\end{equation}
\begin{equation}
\label{eq:gnlsc4}
\mathrm{min\;logc} - 2 \leq \mathit{la} \leq \mathrm{max\;logc} + 2\mathrm{,}
\end{equation}
\begin{equation}
\label{eq:gnlsc5}
0.3 \leq \mathit{lw} \leq 18\mathrm{,}
\end{equation}
and
\begin{equation}
\label{eq:gnlsc6}
\mathit{la}-\mathit{ga} > 0.25\mathrm{.}
\end{equation}

Level 4 does not utilize any assay endpoint-specific methods; the user only needs to run the \texttt{tcplRun} function. \textbf{Level 4 processing and all subsequent processing is done by assay endpoint, not assay component}. The previous section showed how to find the assay endpoints for an assay component using the \texttt{tcplLoadAeid} function. The example dataset includes two assay endpoints with aeid values of 1 and 2.
\hfill \newline
<<>>=
## Do level 4 processing for aeid 1 and load the data
mc4_res <- tcplRun(id = 1:2, slvl = 4, elvl = 4, type = "mc")
@
\hfill \par
The level 4 data include 52 variables, including the ID fields. A complete list of level 4 fields is available in Appendix \ref{app:dbstruc}. The level 4 data include the fields $\mathit{cnst}$, $\mathit{hill}$, and $\mathit{gnls}$ indicating the convergence of the model where a value of 1 means the model converged and a value of 0 means the model did not converge. N/A values indicate the fitting algorithm did not attempt to fit the model. $\mathit{cnst}$ will be N/A when the concentration series had less than 4 concentrations; $\mathit{hill}$ and $\mathit{gnls}$ will be N/A when none of the medians were greater than or equal to $3\mathit{bmad}$. Similarly, the $\mathit{hcov}$ and $\mathit{gcov}$ fields indicate the success in inverting the Hessian matrix. Where the Hessian matrix did not invert, the parameter standard deviation estimates will be N/A. NaN values in the parameter standard deviation fields indicate the covariance matrix was not positive definite. In Figure \ref{fig:l4plt} the $\mathit{hill}$ field is used to find potentially active compounds to visualize with the \texttt{tcplPlotL4ID} function.
\hfill \newline
<<>>=
## Load the level 4 data 
m4dat <- tcplLoadData(lvl = 4, type = "mc")
## List the first m4ids where the hill model convered
## for AEID 1
m4dat[hill == 1 & aeid == 1, head(m4id)]
@
\hfill \par
\setkeys{Gin}{width=1\textwidth}
\begin{figure}[h!]
<<l4plt,fig=true,strip.white=true,echo=true,height=6,width=10>>=
## Plot a fit for m4id 21
tcplPlotM4ID(m4id = 686, lvl = 4)
@
\caption{An example level 4 plot for a single concentration series. The orange dashed line shows the constant model, the red dashed line shows the Hill model, and the blue dashed line shows the gain-loss model. The gray striped box shows the baseline region, $0 \pm 3\mathit{bmad}$. The summary panel shows assay endpoint and sample information, the parameter values (val) and standard deviations (sd) for the Hill and gain-loss models, and summary values for each model.} 
\label{fig:l4plt}
\end{figure}
\par
The model summary values in Figure \ref{fig:l4plt} include Akaike Information Criterion (AIC), probability, and the root mean square error (RMSE). Let $\log(\Lagr(\hat{\theta}, y))$ be the log-likelihood of the model $\hat{\theta}$ given the observed values $y$, and $K$ be the number of parameters in $\hat{\theta}$, then,
\begin{equation}
\label{eq:aic}
\mathrm{AIC} = -2\log(\Lagr(\hat{\theta}, y)) + 2K\mathrm{.}
\end{equation}
The probability, $\omega_{i}$, is defined as the weight of evidence that model $i$ is the best model, given that one of the models must be the best model. Let $\Delta_{i}$ be the difference $\mathrm{AIC}_{i} - \mathrm{AIC}_{min}$ for the $i^{th}$ model. If $R$ is the set of models, then $\omega_{i}$ is given by
\begin{equation}
\label{eq:prob}
\omega_{i} = \frac{\exp\left(-\frac{1}{2}\Delta_{i}\right)}{\sum_{i=1}^{R} \exp\left(-\frac{1}{2}\Delta_{r}\right)}\mathrm{.}
\end{equation}
The RMSE is given by
\begin{equation}
\label{eq:rmse}
\mathrm{RMSE} = \sqrt{\frac{\sum_{i=1}^{N} (y_{i} - \mu_{i})^2}{N}}\mathrm{,}
\end{equation}
where $N$ is the number of observations, and $\mu_{i}$ and $y_{i}$ are the estimated and observed values at the $i^{th}$ observation, respectively. 

\subsection*{Level 5}
\label{subsec:mc5}
\addcontentsline{toc}{subsection}{Level 5}
Level 5 processing determines the winning model and activity for the concentration series, bins all of the concentration series into categories, and calculates additional point-of-departure estimates based on the activity cutoff.

\textbf{The model with lowest AIC value is selected as the winning model ($\mathit{modl}$)}, and is used to determine the activity or hit-call for the concentration series. If two models have equal AIC values, the simpler model (the model with fewer parameters) wins the tie. All of the parameters for the winning model are stored at level 5 with the prefix ``modl\_'' to facilitate easier queries. For a concentration series to get an active hit-call, either the Hill or gain-loss must be selected as the winning model. In addition to selecting the Hill or gain-loss model, the modeled and observed response must meet an efficacy cutoff. 

The efficacy cutoff is defined by the level 5 methods. The efficacy cutoff value ($\mathit{coff}$) is defined as the maximum of all values given by the assigned level 5 methods. Failing to assign a level 5 method will result in every concentration series being called active. For a complete list of level 5 methods see \texttt{tcplMthdList(lvl = 5)} or \texttt{?MC5\_Methods}.
\hfill \newline
<<>>=
## Assign a cutoff value of bmad*6
tcplMthdAssign(lvl = 5,
               id = 1:2,
               mthd_id = 6, 
               type = "mc")
@
\hfill \par
For the example data the cutoff value is $6\mathit{bmad}$. If the Hill or gain-loss model wins, and the estimated top parameter for the winning model ($\mathit{modl\_tp}$) and the maximum median value ($\mathit{max\_med}$) are both greater than or equal to the efficacy cutoff ($\mathit{coff}$), the concentration series is considered active and the hit-call ($\mathit{hitc}$) is set to 1.

The hit-call can be 1, 0, or -1. A hit-call of 1 or 0 indicates the concentration series is active or inactive, respectively, according to the analysis; a hit-call of -1 indicates the concentration series had less than four concentrations.

For active concentration series, two additional point-of-departure estimates are calculated for the winning model: (1) the activity concentration at baseline (ACB or $\mathit{modl\_acb}$) and (2) the activity concentration at cutoff (ACC or $\mathit{modl\_acc}$). The ACB and ACC are defined as the concentration where the estimated model value equals $3\mathit{bmad}$ and the cutoff, respectively. The point-of-departure estimates are summarized in Figure \ref{fig:podplt}.

\hfill \newline
\setkeys{Gin}{width=1\textwidth}
\begin{figure}[h]
<<podplt,fig=true,strip.white=true,echo=FALSE,height=6,width=10>>=
par(family = "mono", mar = rep(1, 4), pty = "m")
plot.new()
plot.window(xlim = c(0, 30), ylim = c(-30, 100))
# axis(side = 2, lwd = 2, col = "gray35")
rect(xleft = par()$usr[1],
     xright = par()$usr[2], 
     ybottom = -15, 
     ytop = 15,
     border = NA, 
     col = "gray45",
     density = 15, 
     angle = 45)
abline(h = 26, lwd = 3, lty = "dashed", col = "gray30")
tmp <- list(modl = "gnls", gnls_ga = 12, gnls_tp = 80, 
            gnls_gw = 0.18, gnls_lw = 0.7, gnls_la = 25)
tcplAddModel(pars = tmp, lwd = 3, col = "dodgerblue2")

abline(v = 8.46, lwd = 3, lty = "solid", col = "firebrick")
text(x = 8.46, y = par()$usr[4]*0.9, 
     font = 2, labels = "ACB", cex = 2, pos = 2, srt = 90)
abline(v = 10.24, lwd = 3, lty = "solid", col = "yellow2")
text(x = 10.24, y = par()$usr[4]*0.9, 
     font = 2, labels = "ACC", cex = 2, pos = 2, srt = 90)
abline(v = 12, lwd = 3, lty = "solid", col = "dodgerblue2")
text(x = 12, y = par()$usr[4]*0.9, 
     font = 2, labels = "AC50", cex = 2, pos = 2, srt = 90)

points(x = c(8.46, 10.24, 12), y = c(15, 26, 40),
       pch = 21, cex = 2, col = "gray30", lwd = 2,
       bg = c("firebrick", "yellow2", "dodgerblue2"))
@
\caption{The point-of-departure estimates calculated by the \texttt{tcpl} package. The shaded rectangle represents the baseline region, $0 \pm 3\mathit{bmad}$. The dark stripped line represents the efficacy cutoff ($\mathit{coff}$). The vertical lines show where the point-of-departure estimates are defined: the red line shows the ACB, the yellow line shows the ACC, and the blue line shows the AC$_{50}$.} 
\label{fig:podplt}
\end{figure}
\par

All concentration series fall into a single fit category ($\mathit{fitc}$), defined by the leaves on the tree structure in Figure \ref{fig:fitc1}. Concentration series in the same category will have similar characteristics, and often look very similar. Categorizing all of the series enables faster quality control checking and easier identification of potential false results. The first split differentiates series by hit-call. Series with a hit-call of -1 go into fit category 2. The following two paragraphs will outline the logic for the active and inactive branches.
\setkeys{Gin}{width=1.2\textwidth}
\begin{sidewaysfigure}[!p]
\centering
<<fitc1, fig=true, strip.white=true, echo=FALSE, height=5, width=8.5>>=
tcplPlotFitc()
@
\caption{The categories used to bin each fit. Each fit falls into one leaf of the tree. The leaves are indicated by bold green font. (Figure created by calling \texttt{tcplPlotFitc()}.)}
\label{fig:fitc1}
\end{sidewaysfigure}

The first split in the active branch differentiates series by the model winner, Hill or gain-loss. For each model, the next split is defined by the efficacy of it's top parameter in relation to the cutoff. The top value is either less than $1.2\mathit{coff}$ or greater than or equal to $1.2\mathit{coff}$. Finally, series on the active branch go into leaves based on the position of the AC$_{50}$ parameter in relation to the tested concentration range. For comparison purposes, the activity concentration at  95\% (AC95) is calculated, but not stored.\footnote{The \texttt{tcplHill-} functions can be used to calculate values, concentrations, and activity concentrations for the Hill model.} Series with AC$_{50}$ values less than the minimum concentration tested ($\mathit{logc\_min}$) go into the ``$<=$'' leaves, series with AC$_{50}$ values greater than the minimum tested concentration and AC95 values less than maximum tested concentration ($\mathit{logc\_max}$) go into the ``$==$'' leaves, and series with AC95 values greater than the maximum concentration tested go into the ``$>=$'' leaves.

The inactive branch is first divided by whether any median values were greater than or equal to $3\mathit{bmad}$. Series with no evidence of activity go into fit category 4. Similar to the active branch, series with evidence for activity are separated by the model winner. The Hill and gain-loss portions of the inactive branch follow the same logic. First, series diverge by the efficacy of their top parameter in relation to the cutoff: $\mathit{modl\_tp < 0.8\mathit{coff}}$ or $\mathit{modl\_tp \geq 0.8\mathit{coff}}$. Then the same comparison is made on the top values of the losing model. If the losing model did not converge, then the series go into the ``DNC'' category. If the losing model top value is greater than or equal to $0.8\mathit{coff}$, then the series are split based on whether the losing model top surpassed the cutoff. On the constant model branch, if neither top parameter is greater than or equal to $0.8\mathit{bmad}$, then the series goes into fit category 7. If one of the top parameters is greater than or equal to $0.8\mathit{coff}$, the series goes into fit category 9 or 10 based on whether one of the top values surpassed the cutoff.

With the level 5 methods assigned, the data are ready for level 5 processing:
\hfill \newline
<<>>=
## Do level 5 processing for aeid 1 and load the data
mc5_res <- tcplRun(id = 1:2, slvl = 5, elvl = 5, type = "mc")
@
\hfill \par
\setkeys{Gin}{width=1\textwidth}
\begin{figure}[h]
<<l5plt1,fig=true,strip.white=true,echo=true,height=6,width=10>>=
tcplPlotM4ID(m4id = 370, lvl = 5)
@
\caption{An example level 5 plot for a single concentration series. The solid line and model highlighting indicate the model winner. The horizontal line shows the cutoff value. In addition to the information from the level 4 plots, the summary panel includes the cutoff ($\mathit{coff}$), hit-call ($\mathit{hitc}$), fit category ($\mathit{fitc}$) and activity probability ($\mathit{actp}$) values.} 
\label{fig:l5plt1}
\end{figure}
\par
Figure \ref{fig:l5plt1} shows an example of a concentration series in fit category 37, indicating the series is active and the Hill model won with a top value less than or equal to $1.2\mathit{coff}$, and an AC$_{50}$ value within the tested concentration range. The \texttt{tcplPlotFitc} function shows the distribution of concentration series across the fit category tree (Figure \ref{fig:fitc2}). 
\setkeys{Gin}{width=1\textwidth}
\begin{figure}[h]
<<fitc2,fig=true,strip.white=true,echo=true,height=6,width=10>>=
m5dat <- tcplLoadData(lvl = 5, type = "mc")
tcplPlotFitc(fitc = m5dat$fitc)
@
\caption{The distribution of concentration series by fit category for the example data. Both the size and color of the circles indicate the number of concentration series. The legend gives the range for number of concentration series by color.} 
\label{fig:fitc2}
\end{figure}
\hfill \par
The distribution in Figure \ref{fig:fitc2} shows 24-40 concentration series fell into fit category 21. Following the logic discussed previously, fit category 21 indicates an inactive series where the Hill model was selected, the top asymptote for the Hill model was greater than $0.8\mathit{coff}$, and the gain-loss top asymptote was greater than or equal to the cutoff. The series in fit category 21 can be found easily in the level 5 data.
\hfill \newline
<<>>=
head(m5dat[fitc == 21, 
           list(m4id, hill_tp, gnls_tp, 
                max_med, coff, hitc)])
@
\hfill \par
The plot in Figure \ref{fig:l5plt2} shows a concentration series in fit category 21. In the example given by Figure \ref{fig:l5plt2}, the $\mathit{hill\_tp}$ and $\mathit{gnls\_tp}$ parameters are equal and greater than $\mathit{coff}$; however, the maximum median value ($\mathit{max\_med}$) is not greater than the cutoff making the series inactive. 
\hfill \par
\setkeys{Gin}{width=1\textwidth}
\begin{figure}[h]
<<l5plt2,fig=true,strip.white=true,echo=true,height=6,width=10>>=
tcplPlotM4ID(m4id = 45, lvl = 5)
@
\caption{Level 5 plot for m4id 45 showing an example series in fit category 21.} 
\label{fig:l5plt2}
\end{figure}

\subsection*{Level 6}
\label{subsec:mc6}
\addcontentsline{toc}{subsection}{Level 6}
Level 6 processing uses various methods to identify concentration series with etiologies that may suggest false positive/false negative results or explain apparent anomalies in the data. Each flag has is defined by a level 6 method that has to be assigned to each assay endpoint. Similar to level 5, an assay endpoint does not need any level 6 methods assigned to complete processing. 
\hfill \newline
<<>>=
tcplMthdAssign(lvl = 6,
               id = 1:2,
               mthd_id = c(6:8, 10:12, 15:16), 
               type = "mc")
tcplMthdLoad(lvl = 6, id = 1, type = "mc")
@
\hfill \par
The example above assigns the most common flags. Some of the available flags only apply to specific experimental designs and do not apply to all data. For a complete list of normalization methods see \texttt{tcplMthdList(lvl = 6)} or \texttt{?MC6\_Methods}. 

The additional $\mathit{nddr}$ field in the ``mc6\_methods'' (and the output from \texttt{tcplMthdLoad()}/\texttt{tcplMthdList()} for level 6) indicates whether the method requires additional data. Methods with an $\mathit{nddr}$ value of 0 only require the modeled/summary information from levels 4 and 5. Methods with an $\mathit{nddr}$ value of 1 also require the individual response and concentration values from level 3. Methods requiring data from level 3 can greatly increase the processing time.
\hfill \newline
<<>>=
## Do level 6 processing
mc6_res <- tcplRun(id = 1:2, slvl = 6, elvl = 6, type = "mc")
m6dat <- tcplLoadData(lvl = 6, type = "mc")
@
\hfill \par
For the two assay endpoints, \Sexpr{nrow(m6dat)} out of the \Sexpr{nrow(m5dat)} concentration series were flagged in the level 6 processing. Series not flagged in the level 6 processing do not get stored at level 6. Each series-flag combination is a separate entry in the level 6 data. Or, in other words, if a series has multiple flags it will show up on multiple rows in the output. For example, consider the following results:
\hfill \newline
<<>>=
m6dat[m4id == 46]
@
\hfill \par
The data above lists two flags: ``Multiple points above baseline, inactive'' and ``Borderline inactive.'' Without knowing much about the flags one might assume this concentration series had some evidence of activity but was not called a hit, and could potentially be a false negative. In cases of borderline results, plotting the curve is often helpful.

\hfill \par
\setkeys{Gin}{width=1\textwidth}
\begin{figure}[h!]
<<l6plt,fig=true,strip.white=true,echo=true,height=6,width=10>>=
tcplPlotM4ID(m4id = 46, lvl = 6)
@
\caption{An example level 6 plot for a single concentration series. All level 6 method ID ($\mathit{l6\_mthd\_id}$) values are concatenated in the flags section. If flags have an associated value ($\mathit{fval}$), the value will be shown in parentheses to the right of the level 6 method ID.} 
\label{fig:l6plt}
\end{figure}
The evidence of true activity shown in Figure \ref{fig:l6plt} could be argued either way. Level 6 processing does not attempt to define truth in the matter of borderline compounds or data anomalies, but rather attempts to identify concentration series for closer consideration.

\appendix
\clearpage
\section{Field Explanation/Database Structure}
\markboth{Field Explanation/Database Structure}{}
\thispagestyle{plain}
\label{app:dbstruc}

This appendix contains reference tables that describe the structure and table fields found in the \texttt{tcpl} database. The first sections of this appendix describe the data-containing tables, followed by a section describing the additional annotation tables.

In general, the single-concentration data and accompanying methods are found in the ``sc\#'' tables, where the number indications the processing level. Likewise, the multiple-concentration data and accompanying methods are found in the ``mc\#'' tables. Each processing level that has accompanying methods will also have a tables with the ``\_methods'' and ``\_id'' naming scheme. For example, the database contains the following tables: ``mc5'' storing the data from multiple-concentration level 5 processing, ``mc5\_methods'' storing the available level 5 methods, and ``mc5\_aeid'' storing the method assignments for level 5. Note, the table storing the method assignments for level 2 multiple-concentration processing is called ``mc2\_acid'' because MC2 methods are assigned by assay component ID.

There are two additional tables, ``sc2\_agg'' and ``mc4\_agg,'' that link the data in tables ``sc2'' and ``mc4'' to the data in tables ``sc1'' and ``mc3,'' respectively. This is necessary because each entry in the database before SC2 and MC4 processing represents a single value; subsequent entries represent summary/modeled values that encompass many values. To know what values were used in calculating the summary/modeled values, the user must use the ``\_agg'' look-up tables.

Each of the methods tables have fields analogous to $\mathit{mc5\_mthd\_id}$, $\mathit{mc5\_mthd}$, and $\mathit{desc}$. These fields represent the unique key for the method, the abbreviated method name (used to call the method from the corresponding \texttt{mc5\_mthds} function), and a brief description of the method, respectively. The ``mc6\_methods'' table may also includes $\mathit{nddr}$ field. More information about $\mathit{nddr}$ is available in the discussion of multiple-concentration level 6 processing (page \pageref{subsec:mc6}).

The method assignment tables will have fields analogous to $\mathit{mc5\_mthd\_id}$ matching the method ID from the methods tables, an assay component or assay endpoint ID, and possibly an $\mathit{exec\_ordr}$ field indicating the order in which to execute the methods.

The method and method assignment tables will not be listed in the tables below to reduce redundancy. 

Many of the tables also include the $\mathit{created\_date}$, $\mathit{modified\_date}$, and $\mathit{modified\_by}$ fields that store information helpful for tracking changes to the data. These fields will not be discussed further or included in the tables below.

Many of the tables specific to the assay annotation are not utilized by the \texttt{tcpl} package. The full complexity of the assay annotation used by the ToxCast program is beyond the scope of this vignette and the \texttt{tcpl} package. More information about the ToxCast assay annotation can be found at: \url{<http://epa.gov/ncct/toxcast/data.html>}.

\clearpage

\subsection*{Single-concentration data-containing tables}
\label{subsec:sctabs}
\addcontentsline{toc}{subsection}{Single-concentration tables}

\begin{table}[H]
  \centering
  \caption{Fields in sc0 table.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  s0id & Level 0 ID \\ \hline
  acid & Assay component ID \\ \hline
  spid & Sample ID \\ \hline
  cpid & Chemical plate ID \\ \hline
  apid & Assay plate ID \\ \hline
  rowi & Assay plate row index \\ \hline
  coli & Assay plate column index \\ \hline
  wllt & Well type$^\dagger$\\ \hline
  wllq & 1 if the well quality was good, else 0$^\ddagger$\\ \hline
  conc & Concentration in micromolar \\ \hline
  rval & Raw assay component value/readout from vendor \\ \hline
  srcf & Filename of the source file containing the data \\ \hline
  \multicolumn{2}{l}{\footnotesize{$^\dagger$Information about the different well types is available in Appendix \ref{app:l0}.}} \\[-8pt]
  \end{tabular}}
\label{tab:sc0}
\end{table}

\begin{table}[H]
  \centering
  \caption{Fields in sc1 table.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  s1id & Level 1 ID \\ \hline
  s0id & Level 0 ID \\ \hline
  acid & Assay component ID \\ \hline
  aeid & Assay component endpoint ID \\ \hline
  logc & Log base 10 concentration \\ \hline
  bval & Baseline value \\ \hline
  pval & Positive control value \\ \hline 
  resp & Normalized response value \\ \hline
  \end{tabular}}
\label{tab:sc1}

\end{table}\begin{table}[H]
  \centering
  \caption{Fields in sc2\_agg table.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  aeid & Assay component endpoint ID \\ \hline
  s0id & Level 0 ID \\ \hline
  s1id & Level 1 ID \\ \hline
  s2id & Level 2 ID \\ \hline
  \end{tabular}}
\label{tab:sc2agg}
\end{table}

\begin{table}[H]
  \centering
  \caption{Fields in sc2 table.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  s2id & Level 2 ID \\ \hline
  aeid & Assay component endpoint ID \\ \hline
  spid & Sample ID \\ \hline
  bmad & Baseline median absolute deviation \\ \hline
  max\_med & Maximum median response value \\ \hline
  hitc & Hit-/activity-call, 1 if active, 0 if inactive\\ \hline
  coff & Efficacy cutoff value \\ \hline
  tmpi & Ignore, temporary index used for uploading purposes\\ \hline
  \end{tabular}}
\label{tab:sc2}
\end{table}

\clearpage

\subsection*{Multiple-concentration data-containing tables}
\label{subsec:mctabs}
\addcontentsline{toc}{subsection}{Multiple-concentration tables}

The ``mc0'' table, other than containing $\mathit{m0id}$ rather than $\mathit{s0id}$, is identical to the ``sc0'' described in the section above.

\begin{table}[H]
  \centering
  \caption{Fields in mc1 table.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  m1id & Level 1 ID \\ \hline
  m0id & Level 0 ID \\ \hline
  acid & Assay component ID \\ \hline
  cndx & Concentration index \\ \hline
  repi & Replicate index \\ \hline
  \end{tabular}}
\label{tab:mc1}
\end{table}
  
\begin{table}[H]
  \centering
  \caption{Fields in mc2 table.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  m2id & Level 2 ID \\ \hline
  m0id & Level 0 ID \\ \hline
  acid & Assay component ID \\ \hline
  m1id & Level 1 ID \\ \hline
  cval & Corrected value \\ \hline
  \end{tabular}}
\label{tab:mc2}
\end{table}

\begin{table}[H]
  \centering
  \caption{Fields in mc3 table.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  m3id & Level 3 ID \\ \hline
  aeid & Assay endpoint ID \\ \hline
  m0id & Level 0 ID \\ \hline
  acid & Assay component ID \\ \hline
  m1id & Level 1 ID \\ \hline
  m2id & Level 2 ID \\ \hline
  bval & Baseline value \\ \hline
  pval & Positive control value \\ \hline
  logc & Log base 10 concentration \\ \hline
  resp & Normalized response value \\ \hline
  \end{tabular}}
\label{tab:mc3}
\end{table}

\begin{table}[H]
  \centering
  \caption{Fields in mc4\_agg table.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  aeid & Assay endpoint ID \\ \hline
  m0id & Level 0 ID \\ \hline
  m1id & Level 1 ID \\ \hline
  m2id & Level 2 ID \\ \hline
  m3id & Level 3 ID \\ \hline
  m4id & Level 4 ID \\ \hline 
  \end{tabular}}
\label{tab:mc4agg}
\end{table}

\begin{table}[H]
  \centering
  \caption{Fields in mc4 table (Part 1).}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  m4id & Level 4 ID \\ \hline
  aeid & Assay endpoint ID \\ \hline
  spid & Sample ID \\ \hline
  bmad & Baseline median absolute deviation \\ \hline
  resp\_max & Maximum response value \\ \hline
  resp\_min & Minimum response value \\ \hline
  max\_mean & Maximum mean response value \\ \hline
  max\_mean\_conc & Log concentration at $\mathit{max\_mean}$ \\ \hline
  max\_med & Maximum median response value \\ \hline
  max\_med\_conc & Log concentration at $\mathit{max\_med}$ \\ \hline
  logc\_max & Maximum log concentration tested \\ \hline
  logc\_min & Minimum log concentration tested \\ \hline
  cnst & 1 if the constant model converged, 0 if it failed to converge, N/A if series had less than four concentrations\\ \hline
  hill & 1 if the Hill model converged, 0 if it failed to converge, N/A if series had less than four concentrations or if $\mathit{max\_med} < 3\mathit{bmad}$ \\ \hline
  hcov & 1 if the Hill model Hessian matrix could be inverted, else 0 \\ \hline
  gnls & 1 if the gain-loss model converged, 0 if it failed to converge, N/A if series had less than four concentrations or if $\mathit{max\_med} < 3\mathit{bmad}$ \\ \hline
  gcov & 1 if the gain-loss model Hessian matrix could be inverted, else 0\\ \hline
  cnst\_er & Scale term for the constant model \\ \hline
  cnst\_aic & AIC for the constant model  \\ \hline
  cnst\_rmse & RMSE for the constant model \\ \hline
  cnst\_prob & Probability the constant model is the true model \\ \hline
  hill\_tp & Top asymptote for the Hill model \\ \hline
  hill\_tp\_sd & Standard deviation for $\mathit{hill\_tp}$ \\ \hline
  hill\_ga & AC$_{50}$ for the Hill model \\ \hline
  hill\_ga\_sd & Standard deviation for $\mathit{hill\_ga}$ \\ \hline
  \end{tabular}}
\label{tab:mc4p1}
\end{table}

\begin{table}[H]
  \centering
  \caption{Fields in mc4 table (Part 2).}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  hill\_gw & Hill coefficient \\ \hline
  hill\_gw\_sd & Standard deviation for $\mathit{hill\_gw}$ \\ \hline
  hill\_er & Scale term for the Hill model \\ \hline
  hill\_er\_sd & Standard deviation for $\mathit{hill\_er}$ \\ \hline
  hill\_aic & AIC for the Hill model \\ \hline
  hill\_rmse & RMSE for the Hill model \\ \hline
  hill\_prob & Probability the Hill model is the true model \\ \hline
  gnls\_tp & Top asymptote for the gain-loss model \\ \hline
  gnls\_tp\_sd & Standard deviation for $\mathit{gnls\_tp}$ \\ \hline
  gnls\_ga & AC$_{50}$ in the gain direction for the gain-loss model \\ \hline
  gnls\_ga\_sd & Standard deviation for $\mathit{gnls\_ga}$ \\ \hline
  gnls\_gw & Hill coefficient in the gain direction \\ \hline
  gnls\_gw\_sd & Standard deviation for $\mathit{gnls\_gw}$ \\ \hline
  gnls\_la & AC$_{50}$ in the loss direction for the gain-loss model \\ \hline
  gnls\_la\_sd & Standard deviation for $\mathit{gnls\_la}$ \\ \hline
  gnls\_lw & Hill coefficient in the loss direction \\ \hline
  gnls\_lw\_sd & Standard deviation for $\mathit{gnls\_lw}$ \\ \hline
  gnls\_er & Scale term for the gain-loss model \\ \hline
  gnls\_er\_sd & Standard deviation for $\mathit{gnls\_er}$ \\ \hline
  gnls\_aic & AIC for the gain-loss model \\ \hline
  gnls\_rmse & RMSE for the gain-loss model \\ \hline
  gnls\_prob & Probability the gain-loss model is the true model \\ \hline
  nconc & Number of concentrations tested \\ \hline
  npts & Number of points in the concentration series \\ \hline
  nrep & Number of replicates in the concentration series \\ \hline
  nmed\_gtbl & Number of median values greater than $3\mathit{bmad}$ \\ \hline
  tmpi & Ignore, temporary index used for uploading purposes\\ \hline
  \end{tabular}}
\label{tab:mc4p2}
\end{table}

\begin{table}[H]
  \centering
  \caption{Fields in mc5 table.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  m5id & Level 5 ID \\ \hline
  m4id & Level 4 ID \\ \hline
  aeid & Assay endpoint ID \\ \hline
  modl & Winning model: ``cnst'', ``hill'', or ``gnls'' \\ \hline
  hitc & Hit-/activity-call, 1 if active, 0 if inactive, -1 if cannot determine\\ \hline
  fitc & Fit category \\ \hline
  coff & Efficacy cutoff value \\ \hline
  actp & Activity probability ($1 - \mathit{cnst\_prob}$)\\ \hline
  modl\_er & Scale term for the winning model \\ \hline
  modl\_tp & Top asymptote for the winning model \\ \hline
  modl\_ga & Gain AC$_{50}$ for the winning model \\ \hline
  modl\_gw & Gain Hill coefficient for the winning model \\ \hline
  modl\_la & Loss AC$_{50}$ for the winning model \\ \hline
  modl\_lw & Loss Hill coefficient for the winning model \\ \hline
  modl\_prob & Probability for the winning model \\ \hline
  modl\_rmse & RMSE for the winning model \\ \hline
  modl\_acc & Activity concentration at cutoff for the winning model \\ \hline
  modl\_acb & Activity concentration at baseline for the winning model \\ \hline
  modl\_ac10 & AC10 for the winning model \\ \hline
  \end{tabular}}
\label{tab:mc5}
\end{table}

\begin{table}[H]
  \centering
  \caption{Fields in mc6 table.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  m6id & Level 6 ID \\ \hline
  m5id & Level 5 ID \\ \hline
  m4id & Level 4 ID \\ \hline
  aeid & Assay endpoint ID \\ \hline
  m6\_mthd\_id & Level 6 method ID \\ \hline
  flag & Text text output for the level 6 method \\ \hline
  fval & Value from the flag method, if applicable \\ \hline
  fval\_unit & Units for $\mathit{fval}$, if applicable \\ \hline
  \end{tabular}}
\label{tab:mc6}
\end{table}

\clearpage

\subsection*{Auxiliary annotation tables}
\label{subsec:auxtabs}
\addcontentsline{toc}{subsection}{Auxiliary annotation tables}

As mentioned in the introduction to this appendix, a full description of the assay annotation is beyond the scope of this vignette. The fields pertinent to the \texttt{tcpl} package are listed in the tables below.

\begin{table}[H]
  \centering
  \caption{List of annotation tables.}
  \noindent\makebox[\textwidth]{%
  \begin{tabular}{l p{10cm}}
  Table Name & Description \\ \hline
  assay & Assay-level annotation \\ \hline
  assay\_component & Assay component-level annotation \\ \hline
  assay\_component\_endpoint & Assay endpoint-level annotation \\ \hline
  assay\_component\_map & Assay component source names and their corresponding assay component ids \\ \hline
  assay\_reagent* & Assay reagent information \\ \hline
  assay\_reference* & Map of citations to assay \\ \hline
  assay\_source & Assay source-level annotation \\ \hline
  chemical & List of chemicals and associated identifiers \\ \hline
  chemical\_library & Map of chemicals to different chemical libraries \\ \hline
  citations* & List of citations \\ \hline
  gene & Gene* identifiers and descriptions \\ \hline   
  intended\_target* & Intended assay target at the assay endpoint level \\ \hline
  mc5\_fit\_categories & The level 5 fit categories \\ \hline
  organism* & Organism identifiers and descriptions \\ \hline
  sample & Sample ID information and chemical ID mapping \\ \hline
  technological\_target* & Technological assay target at the assay component level \\ \hline
  \multicolumn{2}{p{15cm}}{\footnotesize{* indicates tables not currently used by the \texttt{tcpl} package}}
  \end{tabular}}
  \label{tab:datatbllist}
\end{table}

\begin{table}[H]
\centering
\caption{Fields in assay.}
\noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  aid & Assay ID \\ \hline
  asid & Assay source ID \\ \hline
  assay\_name & Assay name (abbreviated ``anm'' within the package) \\ \hline
  assay\_desc & Assay description \\ \hline
  timepoint\_hr & Treatment duration in hours \\ \hline
  assay\_footprint & Microtiter plate size$^\dagger$ \\ \hline
  \multicolumn{2}{p{12cm}}{\footnotesize{$^\dagger$ discussed further in the ``Register and Upload New Data'' section (page \pageref{sec:newdata})}}
  \end{tabular}}
\label{tab:assay}
\end{table}

\begin{table}[H]
\centering
\caption{Fields in assay\_component.}
\noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  acid & Assay component ID \\ \hline
  aid & Assay ID \\ \hline
  assay\_component\_name & Assay component name (abbreviated ``acnm'' within the package) \\ \hline
  assay\_component\_desc & Assay component description \\ \hline
  \end{tabular}}
\label{tab:assaycomp}
\end{table}

\begin{table}[H]
\centering
\caption{Fields in assay\_source.}
\noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  asid & Assay source ID \\ \hline
  assay\_source\_name & Assay source name (typically an abbreviation of the assay\_source\_long\_name, abbreviated ``asnm'' within the package) \\ \hline
  assay\_source\_long\_name & The full assay source name \\ \hline
  assay\_source\_description & Assay source description \\ \hline
  \end{tabular}}
\label{tab:assaysource}
\end{table}

\begin{table}[H]
\centering
\caption{Fields in assay\_component\_endpoint.}
\noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  aeid & Assay component endpoint ID \\ \hline
  acid & Assay component ID \\ \hline
  assay\_component\_endpoint\_name & Assay component endpoint name (abbreviated ``aenm'' within the package) \\ \hline
  assay\_component\_endpoint\_desc & Assay component endpoint description \\ \hline
  export\_ready & 0 or 1, used to flag data as ``done'' \\ \hline
  normalized\_data\_type & The units of the normalized data$^\dagger$ \\ \hline
  burst\_assay & 0 or 1, 1 indicates the assay results should be used in calculating the burst z-score \\ \hline
  fit\_all & 0 or 1, 1 indicates all results should be fit, regardless of whether the $\mathit{max\_med}$ surpasses $3\mathit{bmad}$ \\ \hline
  \multicolumn{2}{p{12cm}}{\footnotesize{$^\dagger$ discussed further in the ``Register and Upload New Data'' section (page \pageref{sec:newdata})}}
  \end{tabular}}
\label{tab:assaycompend}
\end{table}

\begin{table}[H]
\centering
\caption{Fields in assay\_component\_map table.}
\noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  acid & Assay component ID \\ \hline
  acsn & Assay component source name \\ \hline 
  \end{tabular}}
\label{tab:assaycompmap}
\end{table}

\begin{table}[H]
\centering
\caption{Fields in chemical.}
\noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  chid & Chemical ID$^\dagger$ \\ \hline
  casn & CAS Registry Number \\ \hline
  chnm & Chemical name \\ \hline
  \multicolumn{2}{p{10cm}}{\footnotesize{$^\dagger$ this is the DSSTox GSID within the ToxCast data, but can be any integer and will be auto-generated (if not explicitly defined) for newly registered chemicals}}
  \end{tabular}}
\label{tab:chemical}
\end{table}

\begin{table}[H]
\centering
\caption{Fields in chemical\_library.}
\noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  chid & Chemical ID \\ \hline
  clib & Chemical library \\ \hline
  \end{tabular}}
\label{tab:chemlib}
\end{table}

\begin{table}[H]
  \centering
  \caption{Fields in mc5\_fit\_categories table.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  fitc & Fit category \\ \hline
  parent\_fitc & Parent fit category \\ \hline
  name & Fit category name \\ \hline
  xloc & x-axis location for plotting purposes \\ \hline
  yloc & y-axis location for plotting purposes \\ \hline 
  \end{tabular}}
\label{tab:l5fitcategories}
\end{table}

\begin{table}[H]
\centering
\caption{Fields in sample.}
\noindent\makebox[\textwidth]{
  \begin{tabular}{l p{10cm}}
  Field & Description \\ \hline
  spid & Sample ID \\ \hline
  chid & Chemical ID \\ \hline
  stkc & Stock concentration \\ \hline
  stkc\_unit & Stock concentration unit \\ \hline
  tested\_conc\_unit & The concentration unit for the concentration values in the data-containing tables \\ \hline
  spid\_legacy & A place-holder for previous sample ID strings
  \end{tabular}}
\label{tab:sample}
\end{table}

The stock concentration fields in the ``sample'' table allow the user to track the original concentration when the neat sample is solubilized in vehicle before any serial dilutions for testing purposes. 

\clearpage
\section{Level 0 Pre-processing}
\label{app:l0}
\markboth{Level 0 Pre-processing}{}
\thispagestyle{plain}
Level 0 pre-processing can be done on virtually any high-throughput/high-content screening application. In the ToxCast program, level 0 processing is done in R by vendor/dataset-specific scripts. The individual R scripts act as the ``laboratory notebook'' for the data, with all pre-processing decisions clearly commented and explained. 

Level 0 pre-processing has to reformat the raw data into the standard format for the pipeline, and also can make manual changes to the data. All manual changes to the data should be very well documented with justification. Common examples of manual changes include fixing a sample ID typo, or changing well quality value(s) to 0 after finding obvious problems like a plate row/column missing an assay reagent. 

Each row in the level 0 pre-processing data represents one well-assay component combination, containing 11 fields (Table \ref{tab:preproc}). The only field in level 0 pre-processing not stored at level 0 is the assay component source name ($\mathit{acsn}$). The assay component source name should be some concatenation of data from the assay source file that identifies the unique assay components. When the data are loaded into the database, the assay component source name is mapped to assay component ID through the assay\_component\_map table in the \texttt{tcpl} database. Assay components can have multiple assay component source names, but each assay component source name can only map to a single assay component. 

\begin{table}[h!]
  \centering
  \caption{Required fields in level 0 pre-processing.}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{l p{8cm} c}
  Field & Description & N/A \\ \hline
  acsn & Assay component source name & No \\ \hline
  spid & Sample ID & No \\ \hline
  cpid & Chemical plate ID & Yes \\ \hline
  apid & Assay plate ID & Yes \\ \hline
  rowi & Assay plate row index, as an integer & Yes \\ \hline
  coli & Assay plate column index, as an integer & Yes \\ \hline
  wllt & Well type & No \\ \hline
  wllq & 1 if the well quality was good, else 0 & No \\ \hline
  conc & Concentration in micromolar & No$^\dagger$ \\ \hline
  rval & Raw assay component value/readout from vendor & Yes$^\ddagger$ \\ \hline
  srcf & Filename of the source file containing the data & No \\ \hline
  \multicolumn{3}{l}{\footnotesize{The N/A column indicates whether the field can be N/A in the pre-processed data.}} \\[-8pt]
  \multicolumn{3}{p{10cm}}{\footnotesize{$^\dagger$Concentration can be N/A for control values only tested at a single concentration. Concentration cannot be N/A for any test compound (well type of ``t'') data.}} \\[-8pt]
  \multicolumn{3}{p{10cm}}{\footnotesize{$^\ddagger$If the raw value is N/A, well type has to be 0.}}
  \end{tabular}}
\label{tab:preproc}
\end{table}

The well type field is used in the processing to differentiate controls from test compounds in numerous applications, including normalization and definition of the assay noise level. Currently, the \texttt{tcpl} package includes the eight well types in Table \ref{tab:wllt}. Package users are encouraged to suggest new well types and methods to better accommodate their data. 

 \begin{table}[h!]
  \centering
  \caption{Well types}
  \noindent\makebox[\textwidth]{
  \begin{tabular}{c p{10cm}}
  Well Type & Description \\ \hline
  t & Test compound \\ \hline
  c & Gain-of-signal control in multiple concentrations \\ \hline
  p & Gain-of-signal control in single concentration \\ \hline
  n & Neutral/negative control \\ \hline
  m & Loss-of-signal control in multiple concentrations \\ \hline
  o & Loss-of-signal control in single concentration \\ \hline
  b & Blank well \\ \hline
  v & Viability control \\ \hline 
  \end{tabular}}
\label{tab:wllt}
\end{table}

The final step in level 0 pre-processing is loading the data into the \texttt{tcpl} database. The \texttt{tcpl} package includes the \texttt{tcplWriteLvl0} function to load data into the database. The \texttt{tcplWriteLvl0} function maps the assay component source name to the appropriate assay component ID, checks each field for the correct class, and checks the database for the sample IDs with well type ``t.'' Each test compound sample ID must be included in the \texttt{tcpl} database before loading data. The \texttt{tcplWriteLvl0} also checks each test compound for concentration values.

\clearpage
\section{Cytotoxicity Distribution}
\label{app:cyto}
\markboth{Cytotoxicity Distribution}{}
\thispagestyle{plain}
Recognizing the susbtantial impact of cytotoxicity in confounding high-throughput and high-content screening results, the \texttt{tcpl} package includes methodology for defining chemical-specific cytotoxicity estimates. Our observations based on ToxCast data suggest a complex, and not-yet fully understood, cellular biology that includes non-specific activation of many targets as cells approach death. For example, a chemical may induce activity in an estrogen-related assay, but if that chemical also causes activity in hundreds of other assays at or around the same concentration as cytotoxicity, should the chemical be called an estrogen agonist? The \texttt{tcplCytoPt} function provides an estimate of chemical-specific cytotoxicity points to provide some context to the ``burst'' phenomenon.

The cytotoxicity point is simply the median AC$_{50}$ for a set of assay endpoints, either given by the user or defined within the \texttt{tcpl} database. By default, the \texttt{tcplCytoPt} function uses the assay endpoints listed in the $\mathit{burst\_assay}$ field of the ``assay\_component\_endpoint'' table, where 1 indicates to include the assay endpoint in the calculation. The ``burst'' assay endpoints can be indentified by running \texttt{tcplLoadAeid(fld = "burst\_assay", val = 1)}.

In addition to the cytotoxicity point, \texttt{tcplCytoPt} provides two additional estimates: (1) the MAD of the AC$_{50}$ ($\mathit{modl\_ga}$) values used to calculate the cytotoxicity point, and (2) the global MAD. Note, only active assay endpoints (where the hit-call, $\mathit{hitc}$, equals $1$) are included in the calculations. Once the burst distribution (cytotoxicity point and MAD) is defined for each chemical, the global burst MAD is defined as the median of the MAD values. Not every chemcial may be tested in every ``burst'' assay, so the user can determine the minimum number of tested assays as a condition for the MAD value for a particular chemical to be included in the global MAD calculation. By default, if ``aeid'' is the vector of assay endpoints used in the calculation, \texttt{tcplCytoPt} requires the chemical to tested in at least \texttt{floor(0.8 * length(aeid))} assay endpoints to be included in the calculation. The user can specify to include all calculated MAD values (note, there must be at least two active assay endpoints to calculate the MAD) by setting `min.test' to \texttt{FALSE}. The `min.test' parameter also accepts a number, allowing the user to explicitly set the requirement.

The global MAD gives an estimate of overall cytotoxicity window, and allows for a cytotoxicity distrubtion to be determined for chemicals with less than two active ``burst'' assay endpoints. The cytotoxicity point for chemicals with less two active ``burst'' endpoints is set to the value given to the `default.pt' parameter. By default the \texttt{tcplCytoPt} assigns `default.pt' to 3.\footnote{$10^3 = 1000$, therefore, when using micromolar units, $3$ is equivalent to $1$ millimolar. $1$ millimolar was chosen as an arbitrary high concentration (outside the testing range for ToxCast data), based on the principle all compounds are toxic if given in high enough concentration.}


\clearpage
\section{Build Variable Matrices}
\label{app:mats}
\markboth{Build Variable Matrices}{}
\thispagestyle{plain}
The \texttt{tcplVarMat} function creates chemical-by-assay matrices for the level 4 and level 5 data. When multiple sample-assay series exist for one chemical, a single series is selected by the \texttt{tcplSubsetChid} function. See \texttt{?tcplSubsetChid} for more information. 

\begin{enumerate}
  \item ``modl\_ga'' -- The $\log_{10}\mathit{AC_{50}}$ (in the gain direction) for the winning model. 
  \item ``hitc'' -- The hit-call for the winning model.
  \item ``m4id'' -- The m4id, listing the concentration series selected by \texttt{tcplSubsetChid}.
  \item ``zscore'' -- The z-score (described below).
  \item ``tested'' -- $1$ or $0$, $1$ indicating the chemical/assay pair was tested in either the single- or multiple-concentration format.
  \item ``tested\_sc'' -- $1$ or $0$, $1$ indicating the chemical/assay pair was tested in the single-concentration format
  \item ``tested\_mc'' -- $1$ or $0$, $1$ indicating the chemical/assay pair was tested in the multiple-concentration format
  \item ``ac50'' -- a modified AC$_{50}$ table (in non-log units) where assay/chemical pairs that were not tested, or tested and had a hitcall of $0$ or $-1$ have the value $1e6$. 
  \item ``neglogac50'' -- $-\log_{10}\frac{\mathit{AC_{50}}}{1e6}$ where assay/chemical pairs that were not tested, or tested and had a hitcall of $0$ or $-1$ have the value $0$. 
\end{enumerate}

The z-score calculation is based on the output from \texttt{tcplCytoPt} (Appendix \ref{app:cyto}), and is calculated for each AC$_{50}$ value as follows: 
\begin{equation}
\label{eq:zscore}
\mathit{z\-score} = -\frac{\mathit{modl\_ga} - \mathit{cyto\_pt}}{\mathit{global\_mad}}\mathrm{,}
\end{equation}
Note: the burst z-score values are multiplied by -1 to make values that are more potent relative to the burst distribution a higher positive z-score.

In addition the the standard matrices, additional matrices can be defined by the `add.vars' parameter in the \texttt{tcplVarMat} function. The `add.vars' function will take any level 4 or level 5 field and create the respective matrix. 


\end{document}




