---
title: "chTKCat Operations manual"
author: "Patrice Godard"
date:  "`r format(Sys.time(), '%B %d, %Y')`"
package: "TKCat (version `r packageVersion('TKCat')`)"
output:
   rmarkdown::html_document:
      number_sections: yes
      self_contained: false
      lib_dir: "libs"
      theme: cerulean
      toc: yes
      toc_float: yes
      fig_width: 7
      fig_height: 5
vignette: >
   %\VignetteIndexEntry{chTKCat Operations manual}
   %\VignetteEncoding{UTF-8}
   %\VignetteEngine{knitr::rmarkdown}
editor_options: 
  chunk_output_type: console
---

```{r setup, message=FALSE, echo=FALSE, include=FALSE, cache=FALSE}
library(knitr)
opts_chunk$set(
   include=TRUE,
   echo=TRUE,
   message=FALSE,
   warning=FALSE,
   cache=FALSE,
   cache.lazy=FALSE
)
## The following line is to avoid building errors on CRAN
knitr::opts_chunk$set(eval=Sys.getenv("USER") %in% c("pgodard"))
library(TKCat)
```

```{r child = 'supp/urls.Rmd'}
```

::: {style="width:200px;"}
![](libs/figs/TKCat.png){width="100%"}
:::

# Introduction

```{r child = 'supp/genIntro.Rmd'}
```

This vignette describes how to create and administrate a TKCat
[ClickHouse][clickhouse] instance. An [other vignette][chuguide] focuses how
TKCat can be used with a ClickHouse database. Users should also refer to the
[general TKCat user guide][uguide].

# Instantiating the ClickHouse database

## Install ClickHouse, initialize and configure the TKCat instance

The ClickHouse docker container supporting TKCat, its initialization and its
configuration procedures are implemented here:
[S01-install-and-init.R](https://github.com/patzaw/TKCat/blob/master/ClickHouse/S01-install-and-init.R)

This script should be adapted according to requirements and needs.

The data are stored in the `TKCAT_HOME` folder.

## Cleaning and removing a TKCat instance

Stop and remove the docker container.

```{sh, eval=FALSE}
# In shell
docker stop ucb_tbn_tkcat
docker rm ucb_tbn_tkcat
docker volume prune -f
# Remove the folder with all the data: `$TKCAT_HOME`.`
sudo rm -rf ~/Documents/Projects/TKCat_UCB_TBN

```

# User management

User management requires admin right on the database.

## Creation

```{r, eval=FALSE}
k <- chTKCat(user="pgodard")
create_chTKCat_user(k, login="lfrancois", contact=NA, admin=FALSE)
```

The function will require to setup a password for the new user.

## Drop

```{r, eval=FALSE}
drop_chTKCat_user(k, login="lfrancois")
```

# chMDB management

## chMDB Creation

Before MDB data can be uploaded, the database should be created.

```{r, eval=FALSE}
create_chMDB(k, "CHEMBL", public=FALSE)
```

By default chMDB are not public. It can be changed through the `public`
parameter when creating the chMDB or by using the `set_chMDB_access` afterward.

```{r, eval=FALSE}
set_chMDB_access(k, "CHEMBL", public=TRUE)
```

Then, users having access to the chMDB can be identified with or without admin
rights on the chMDB. Admin rights allow the user to update the chMDB data.

```{r, eval=FALSE}
add_chMDB_user(k, "CHEMBL", "lfrancois", admin=TRUE)
# remove_chMDB_user(k, "CHEMBL", "lfrancois")
list_chMDB_users(k, "CHEMBL")
```

## Populating chMDB

Each chMDB can be populated individualy using the `as_chMDB()` function. The
code chunk below shows how to scan a directory for all *fileMDB* it contains.
The `as_memoMDB()` function load all the data in memory and checks that all the
model constraints are fulfilled (this step is optional). The `overwrite`
parameter of the `as_chMDB()` function allows updating the data in the database.

```{r, eval=FALSE}
lc <- scan_fileMDBs("fileMDB_directory")
## The commented line below allows the exploration of the data models in lc.
# explore_MDBs(lc)
for(r in toFeed){
   message(r)
   lr <- as_memoMDB(lc[[r]])
   cr <- as_chMDB(lr, k, overwrite=TRUE)
}
```

## Deleting a chMDB

Any admin user of a chMDB can delete the corresponding data.

```{r, eval=FALSE}
empty_chMDB(k, "CHEMBL")
```

Only a system admin can delete the chMDB from the ClickHouse database.

```{r, eval=FALSE}
drop_chMDB(k, "CHEMBL")
```

# Collection management

The aim of collections is described in the [general user
guide](TKCat-User-guide.html#merging-with-collections)

Collections needs to be added to a chTKCat instance in order to support
collection members of the different chMDB. They can be taken from the TKCat
package environment, from a JSON file or directly from a JSON text variable.
Additional functions are available to list and remove chTKCat collections.

```{r, eval=FALSE}
add_chTKCat_collection(k, "BE")
list_chTKCat_collections(k)
remove_chTKCat_collection(k, "BE")
```

# Implementation

## Data models

### Default database

The default database stores information about chTKCat instance, users and user
access.

```{r, echo=FALSE}
plot(TKCat:::DEFAULT_DATA_MODEL)
```

### Modeled databases

Modeled databases (MDB) are stored in dedicated database in chTKCat. Their data
model is provided in dedicated tables described below.

```{r, echo=FALSE}
plot(TKCat:::CHMDB_DATA_MODEL)
```

# Acknowledgments

This work was entirely supported by [UCB Pharma](https://www.ucb.com/) (Early
Solutions department).
