<!--
%\VignetteEngine{knitr::knitr}
%\VignetteIndexEntry{Getting full text}
%\VignetteEncoding{UTF-8}
-->

```{r echo=FALSE}
knitr::opts_chunk$set(
  comment = "#>",
  collapse = TRUE,
  warning = FALSE, 
  message = FALSE
)
```

The main interface to fetching full text is through `ft_get()`.

## Load fulltext

```{r}
library("fulltext")
```

## Structure of the returned object from ft_get()

Simple call, pass in a DOI and say where you want to get data from (by default, it's _plos_)

```{r}
res <- ft_get('10.1371/journal.pone.0086169', from = 'plos')
```

The article text and metadata is stored in the output object (though soon they'll be the 
option to cache data on disk instead of in memory, but the interface will work the same 
for examining the text and metadata)

The `res` object is a list, with slots for each of the data sources, b/c you can request 
data from more than 1 data source.

```{r}
names(res)
```

Let's dig into the `plos` source object, which is another list, including metadata the 
text data itself (in the `data` slot).

```{r}
res$plos
```

Indexing to the `data` slot takes us to another list with metadata and the article

```{r}
res$plos$data
```

Going down one more index gets us the data object, using the DOI searched to get the text. 
However, since we're dealing with full text, this is a bit messy (just a snippet here)

```{r eval=FALSE}
res$plos$data$data$`10.1371/journal.pone.0086169`
#> [1] "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE article\n  
#> PUBLIC \"-//NLM//DTD Journal Publishing DTD v3.0 20080202//EN\"
#> \"http://dtd.nlm.nih.gov/publishing/3.0/journalpublishing3.dtd\">\n<article
#> xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"
...
```

## Fetching many articles

You can get a bunch of DOIs first, e.g., from PLOS using the `rplos` package

```{r}
library("rplos")
(dois <- searchplos(q = "*:*", fl = 'id',
   fq = list('doc_type:full', "article_type:\"research article\""), limit = 5)$data$id)
ft_get(dois, from = 'plos')
```

## Different data sources

### Articles from eLife

One article

```{r}
ft_get('10.7554/eLife.04300', from = 'elife')
ft_get(c('10.7554/eLife.04300','10.7554/eLife.03032'), from = 'elife')
```

Many articles

```{r}
ft_get(c('10.7554/eLife.04300','10.7554/eLife.03032'), from = 'elife')
```

### Articles from Frontiers in Pharmacology (publisher: Frontiers)

```{r}
doi <- '10.3389/fphar.2014.00109'
ft_get(doi, from = "entrez")
```

## Search using ft_search()

For example, search entrez, get some DOIs, then fetch some articles

```{r}
(res <- ft_search(query = 'ecology', from = 'entrez'))
res$entrez$data$doi
```

Get articles

```{r}
ft_get(res$entrez$data$doi[1:3], from = 'entrez')
```

## Caching

To cache results or not. If `cache=TRUE`, raw XML, or other format that article is 
in is written to disk, then pulled from disk when further manipulations are done on 
the data.

```{r}
cache_options_set(cache = TRUE)
(res <- ft_get('10.1371/journal.pone.0086169', from = 'plos'))
```

> Note how the [source] field has "rds - <path to file>" - indicating that the text is 
cached on disk, not in R'm memory.

Nothing changes from the normal workflow now that data is cached on disk - simply behave 
normally as above. For example, `collect()` reads text from disk into memory (although 
the printed object doesn't indicate it)

```{r}
res %>% collect
```
