---
title: "MARVEL"
author: "Sean Wen"
date: "`r Sys.Date()`"
output:
  html_document:
    toc: true
vignette: >
  %\VignetteIndexEntry{MARVEL}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

<style type="text/css">

body, td {
   font-size: 14px;
}
code.r{
  font-size: 12px;
}
pre {
  font-size: 12px
}
</style>

# 1. Introduction
Alternative splicing represents an additional and under-appreciated layer of complexity underlying gene expression profiles. More recently, technological advances in library preparation methodologies enabled capturing and amplification of full-length cDNAs from single cells. Thus, paving the way for splicing analysis at single-cell resolution.  

Nevertheless, single-cell splicing analysis comes with its own set of challenges including, but not limited to, low coverage of lowly-expressed genes, low capture rate of cDNA molecules, and amplification bias (*Wen et al., 2020*). To date, there remains a paucity of peer-reviewed softwares available for single-cell splicing analysis. Notable examples are BRIE (*Huang & Sanguinetti, 2017*) and Expedition (*Song et al., 2017*).  

Here, we introduce MARVEL (**M**odality **A**ssessment to **R**e**VE**a**L** alternative splicing dynamics at single-cell resolution). MARVEL includes features that complement existing softwares in order to more comprehensively describe and reveal splicing dynamics at single-cell resolution.  

The following is a quick comparison against two other leading published softwares for single-cell alternative splicing analysis:  

```{r message=FALSE, echo=FALSE}
# Check plot
output <- system.file("extdata", "Vignette_Fig_0_Software_Comparison.png", package="MARVEL")
knitr::include_graphics(output)
```

\* BRIE incorporates sequence features to model PSI values. These features were identified and trained on human and mouse data only.  

## Objectives
The main objectives of MARVEL are  

(1) Compute PSI values for all five main exon-level splicing events, i.e. skipped-exon (SE),  mutually-exclusive exons (MXE), retained-intron (RI), alternative 5' splice site (A5SS), and alternative 3' splice site (A3SS).  

(2) Stratify PSI distribution for each splicing event into the five main modalities, i.e. included, excluded, bimodal, middle, and multimodal. Further stratify included and excluded into primary and dispersed sub-modalities.  

(3) Perform differential splicing analysis and identify network of genes which are coordinately spliced.  

(4) Integrate both splicing and gene expression data to compare and contrast splicing and gene expression profiles.  

## Additional resource
The main aims of this vignette is to highlight the principles and technicalities behind MARVEL and to show case the main functionalities of MARVEL. We also provide results from our benchmarking exercise under the **Appendix** section. 

We refer our prospective users to the comprehensive tutorial on using MARVEL to extract biological insights from single-cell alternative splicing analysis here: https://wenweixiong.github.io/MARVEL.html  

## Example dataset
In the examples that follow, we will use published single induced pluripotent stem cells (iPSC) and endoderm cells. Single cells were prepared using Smart-seq2 and sequenced on HiSeq2000 on 125-bp paired-end (PE) mode (*Linker et al., 2018*).  

## Installing the package
Install package from Github
```{r eval=FALSE}
library(devtools)
install_github("wenweixiong/MARVEL")
```

Or install package from CRAN
```{r eval=FALSE}
install.packages("MARVEL")
```

## Loading the package
```{r message=FALSE}
# Load package
library(MARVEL)
```

# 2. Creating a Marvel object
MARVEL uses S3 object-oriented system (OOS) to allow convenient data manipulation by users. Therefore, the first step is to create the S3 object. We will use ```CreateMarvelObject``` to create our S3 object. Arguments required are:  
(1) ```SplicePheno``` Sample metadata. Mandatory columns are ```sample.id``` and ```cell.type```.  
(2) ```SpliceJunction``` Splice junction counts.  
(3) ```SpliceFeature``` Splicing event metadata. Each element in the list represents a data frame corresponding to a specific splicing event type. Mandatory columns are ```tran_id``` and ```gene_id```. Names of each element in the list should reflect the splicing event type, i.e. ```SE```, ```MXE```, ```RI```, ```A5SS```, and ```A3SS```.  

It is noteworthy that MARVEL is agnostic with regards to splice junction and splicing event detection. Hence, users have the freedom to use their preferred softwares to detect splice junctions and splicing events. Here, splice junction counts and splicing events were generated using STAR (*Dobin et al., 2013*) and Stringtie2-rMATS (*Kovaka et al., 2019*; *Shen et al., 2014*), respectively.  

Additionally, it is highly recommended to include gene expression data (normalised and log-transformed) as MARVEL has useful functionalities to compare and contrast gene expression profiles.  

```{r size="small"}
# Read splicing files
  # Sample metadata
  path_to_file <- system.file("extdata", "SE_phenoData.txt", package="MARVEL")
  df.pheno <- read.table(path_to_file, sep="\t", header=TRUE, stringsAsFactors=FALSE, na.strings="NA")

  # Subset samples that passed sequencing QC
  df.pheno <- df.pheno[which(df.pheno$qc.seq=="pass"), ]
  df.pheno[1:5,]
  
  # Splice junction file
  path_to_file <- system.file("extdata", "SJ.txt", package="MARVEL")
  sj <- read.table(path_to_file, sep="\t", header=TRUE, stringsAsFactors=FALSE, na.strings="NA")
  sj[1:5,1:5]
  
  # Splicing event metadata
  df.feature.list <- list()
  path_to_file <- system.file("extdata", "SE_featureData.txt", package="MARVEL")
  df.feature.list[[1]] <- read.table(path_to_file, sep="\t", header=TRUE, stringsAsFactors=FALSE, na.strings="NA")
  names(df.feature.list) <- "SE"
  df.feature.list[["SE"]][1:5,]

# Read gene files
  # featureData
  path_to_file <- system.file("extdata", "TPM_featureData.txt", package="MARVEL")
  df.tpm.feature <- read.table(path_to_file, sep="\t", header=TRUE, stringsAsFactors=FALSE, na.strings="NA")
  df.tpm.feature[1:5,]

  # phenoData
  path_to_file <- system.file("extdata", "SE_phenoData.txt", package="MARVEL")
  df.tpm.pheno <- read.table(path_to_file, sep="\t", header=TRUE, stringsAsFactors=FALSE, na.strings="NA")

  # Normalised expression matrix
  path_to_file <- system.file("extdata", "TPM.txt", package="MARVEL")
  df.tpm <- read.table(path_to_file, sep="\t", header=TRUE, stringsAsFactors=FALSE, na.strings="NA")
  
  # Log-transform values
  df.tpm[,-1] <- log2(df.tpm[,-1])
  df.tpm[,-1][df.tpm[,-1] < 1] <- 0

  df.tpm[1:5,1:5]
  
  # Subset samples that passed sequencing QC
  df.tpm.pheno <- df.tpm.pheno[which(df.pheno$qc.seq=="pass"), ]
  df.tpm.pheno[1:5,]
  df.tpm <- df.tpm[, which(names(df.tpm) %in% c("gene_id", df.tpm.pheno$sample.id))]
  
# Create Marvel object
marvel <- CreateMarvelObject(
            SplicePheno=df.pheno,          # Sample metadata
            SpliceJunction=sj,             # Splice junction counts 
            SpliceFeature=df.feature.list, # Splicing event metadata
            GenePheno=df.tpm.pheno,        # Sample metadata
            GeneFeature=df.tpm.feature,    # Gene metadata
            Exp=df.tpm                     # Gene expression matrix
            )
```

# 3. Calculate percent spliced-in (PSI)

## Principles
Percent spliced-in (PSI) is the fraction of reads supporting the included isoform over the number of reads supporting both included and excluded isoforms. Included isoforms are isoforms that include the alternative exon. Hence, PSI is a measure of alternative exon inclusion ("spliced-in").  

To date, peer-reviewed softwares focused only on SE and MXE events. MARVEL expands exon-level splicing analysis to include all five main exon-level splicing events, namely SE, MXE, RI, A5SS, and A3SS. Other than SE, the other 4 splicing types have been reported to play important roles in both health and disease too. For example, RI is known to regulate gene expression through nonsense-mediated decay (*Smart et al., 2018*).  

First, MARVEL will cross-check the splicing events with the splice junctions provided. Only splicing events whose splice junctions are found in the splice junction file are retained. This ensures only high-quality splicing events are included for PSI calculation and downstream analysis.  

Furthermore, MARVEL allows users to set a coverage threshold using the ```CoverageThreshold``` argument, above which the splicing event is included for PSI calculation. For example, if the coverage threshold is set at 10, then MARVEL will only include the splicing event if all included or excluded junctions involved in calculating the splicing events are supported by at least 10 or more reads. ```NA``` in the PSI matrix returned are splicing events whose sample that did not have sufficient reads, i.e. lower number of reads than that specified by the user. The coverage threshold of 10 has been used multiple times in previous single-cell studies for selecting splicing events for downstream analysis (*Song et al., 2017*; *Buen Abad Najar et al, 2020*).  

```{r message=FALSE, echo=FALSE}
# Check plot
output <- system.file("extdata", "Vignette_Fig_1_PSI_Formula-min.png", package="MARVEL")
knitr::include_graphics(output)
```

**Figure 1**: Alternative splicing event types and their respective PSI formula. PSI: Percent splice-in

## Running the code
The function to compute PSI is ```ComputePSI```. Specify ```"SE"```, ```"MXE"```, ```"RI"```, ```"A5SS"```, or ```"A3SS"``` in the ```EventType``` argument to compute the PSI for a specific event type. The options for this function are:  
(1) ```MarvelObject``` Marvel object generated from ```CreateMarvelObject```.  
(2) ```CoverageThreshold``` Coverage threshold below which the PSI of the splicing event will be censored, i.e. annotated as missing (NA). Coverage defined as the total number of reads supporting both included and excluded isoforms.  

```{r size="small"}
# Validate and filter splicing events, compute PSI
marvel <- ComputePSI.SE(MarvelObject=marvel, CoverageThreshold=10)

# Check validated splicing events
marvel$SpliceFeatureValidated$SE[1:5,]

# Check computed PSI values
marvel$PSI$SE[1:5,1:5]
```

## Understanding the output
Two data frames are returned and saved into the following slots of the Marvel object.  
(1) ```SpliceFeatureValidated$SE``` Validated splicing events.  
(2) ```PSI$SE``` PSI matrix where rows represent splicing events and columns represent samples.  

# 4. Pre-computed PSI
Users may also use their own splicing events and PSI values. For example, using pre-validated splicing events and pre-computed PSI values from external softwares or that of previously generated using MARVEL.  

```{r size="small"}
# Read sample metadata
path_to_file <- system.file("extdata", "SE_phenoData.txt", package="MARVEL")
df.pheno <- read.table(path_to_file, sep="\t", header=TRUE, stringsAsFactors=FALSE, na.strings="NA")

# Subset samples that passed sequencing QC
df.pheno <- df.pheno[which(df.pheno$qc.seq=="pass"), ]
df.pheno[1:5,]
  
# Read pre-validated splicing event metadata
df.feature.list <- list()
path_to_file <- system.file("extdata", "SE_featureDataValidated.txt", package="MARVEL")
df.feature.list[[1]] <- read.table(path_to_file, sep="\t", header=TRUE, stringsAsFactors=FALSE, na.strings="NA")
names(df.feature.list) <- "SE"
df.feature.list[["SE"]][1:5, ]

# Read PSI file (pre-computed)
df.list <- list()
path_to_file <- system.file("extdata", "SE.txt", package="MARVEL")
df.list[[1]] <- read.table(path_to_file, sep="\t", header=TRUE, stringsAsFactors=FALSE, na.strings="NA")
names(df.list) <- "SE"
df.list[["SE"]][1:5,1:5]

# Create Marvel object
marvel.temp <- CreateMarvelObject(
            SplicePheno=df.pheno,                    # Sample metadata
            SpliceFeatureValidated=df.feature.list,  # Validated splicing event metadata
            PSI=df.list,                             # Pre-computed PSI matrices
            GenePheno=df.tpm.pheno,                  # Sample metadata
            GeneFeature=df.tpm.feature,              # Gene metadata
            Exp=df.tpm                               # Gene expression matrix
            )
```

# 5. Assign modalities

## Principles
Percent spliced-in (PSI) for a splicing event take values between 0 and 1. An average PSI of 1 means that majority of the cells express the included isoform. Conversely, an average PSI of 0 means that majority of the cells express the excluded isoform. Finally, an average PSI of 0.5 means that cells overall express both included and excluded isoforms in roughly equal proportions (Figure 2).  

Based on the PSI distribution, each splicing event can be categorized into 5 modalities: Included, excluded, bimodal, middle, multimodal. To this end, MARVEL uses ```fitdistr``` function from ```fitdistrplus``` R package to determine the modality of each splicing event. Specifically, MARVEL models each splicing event as a beta distribution and estimates the alpha and beta parameters using the maximum likelihood approach. Based on the parameters' values, each splicing event can be categorized sequentially into their respective modality as follows:  
(1) Bimodal (PSI $\approx$ 0, 1): $\alpha$ < 0.5 | $\beta$ < 0.5  
(2) Included (PSI $\approx$ 1): $\alpha$ > $\beta$  
(3) Excluded (PSI $\approx$ 0): $\alpha$ < $\beta$  
(4) Middle (PSI $\approx$ 0.5): $\alpha$ > 1 & $\beta$ > 1 & $\alpha$ = $\beta$  
(5) Multimodal (uniform): $\alpha$ = $\beta$ = 1  

```{r message=FALSE, echo=FALSE}
# Check plot
output <- system.file("extdata", "Vignette_Fig_2_PSI_Modalities-min.png", package="MARVEL")
knitr::include_graphics(output)
```

**Figure 2**: (Left) Representative PSI distribution of each modality. Each red diamond respresents the average PSI value across all single cells and the expected value for the corresponding bulk-level sample (Right) Illustration of each modality definition

Herein lies the **advantage** of studying  splicing at single-cell resolution compared to that of bulk-level. Because bulk represents the average PSI across all cells, it would be virtually impossible to distinguish splicing events with bimodal, middle, and multimodal patterns.

## Sub-modalities
To provide finer resolution to the included and excluded modalities, we further stratify these two modalities into primary and dispersed, depending on their on how spread out the PSI values are (variance; see Figure 2 above). The default variance value to distinguish between these two sub-modalities is 0.001. This threshold may be customized by the user.  

```{r message=FALSE, echo=FALSE}
# Check plot
output <- system.file("extdata", "Vignette_Fig_3_PSI_Modalities_Included-min.png", package="MARVEL")
knitr::include_graphics(output)
```

**Figure 3**: To demonstrate the various variance values of included modality

```{r message=FALSE, echo=FALSE}
# Check plot
output <- system.file("extdata", "Vignette_Fig_4_PSI_Modalities_Excluded-min.png", package="MARVEL")
knitr::include_graphics(output)
```

**Figure 4**: To demonstrate the various variance values of excluded

## Identifying false bimodals
False bimodal splicing patterns are characterized by an imbalance in the proportion of cells at the ends of the PSI range, i.e. at 0 and 1. This may be due, but not limited, to amplification bias, rare sub-population of cells within an expected homogenous cell population, and artifacts arising from library preparation and next-generation sequencing (*Buen Abad Najar et al, 2020*).  

To formally distinguish true from false bimodal distributions, we compared PSI distribution of splicing events obtained from RNA-sequencing and compared them to that of from qPCR and smFISH (small molecule fluorescent *in situ* hybridization) experimental validations. We further included true bimodal distributions identified from genes with at least 10 cDNA molecules captured as recommended previously (*Buen Abad Najar et al, 2020*). We included 3 independent studies for this exercise (*Song et al., 2017*, *Trapnell et al., 2014*, *Linker et al., 2019*).  

```{r message=FALSE, echo=FALSE}
# Check plot
output <- system.file("extdata", "Vignette_Fig_5_PSI_True_vs_False_Bimodal-min.png", package="MARVEL")
knitr::include_graphics(output)
```

**Figure 5**: Representative examples of false and true bimodal patterns. First row indicates the gene name. Second row indicates the splicing event type. Third row represents the cell type. Notice that false bimodals tend to be funnel-shaped where there is a disproportionate amount of cells at one end compared to the other end. On the other hand, true bimodals tend to be hour-shaped with roughly equal proportion of cells at each end. iPSC: Induced pluripotent stem cells. MN: Motor neurons. NPC: Neural progenitor cells.

## Running the code
The function to assign modality to each splicing event across a group of cells is ```AssignModality```.    
```{r size="small"}
# Assign modality
marvel <- AssignModality(
            MarvelObject=marvel, # Marvel object
            cell.type="iPSC",    # Cell type to include for analysis
            n.cells=25,          # Min. no. of cells PSI != NA required 
            sigma.sq=0.001,      # Variance value below which sub-modality is primary,
                                                # above which sub-modality is dispersed
            bimodal.adjust=TRUE, # Detect and rectify false bimodals
            seed=1               # Ensures MLE model returns reproducible parameter values
            )

marvel$Modality$Results[1:5,]
```

## Understanding the output
The results are returned to the ```$Modality$Results``` slot.
(1) ```n.cells``` column indicates the no. of cells expressing the splicing event.  
(2) ```modality``` column indicates the 5 basic modalities.  
(3) ```modality.var``` column indicates the extended modalities as proposed by us.  
(4) ```modality.bimodal.adj``` column indicates the extended modalities whose false bimodals have been identified and reclassified into either included or excluded modalities.  

# 6. Differential splicing analysis

## Principles
The aim of single-cell differential gene analysis is to determine if there is a statistically significant difference between the **means** of different group of cells. The non-parametric wilcoxon test and parametric t-test are suitable for comparing two groups of cells. The non-parametric Kruskal-Wallis test and parametric analysis of variance (ANOVA) are suitable for comparing two or more groups of cells. In the case of gene expression values, variance is a measure of **uncertainty**. The larger the variance, the lower the statistical power to detect a significant difference in means between groups of cells.  

In contrast, comparing the means of the PSI distributions between groups of cells alone is insufficient. For example, bimodal, middle, and multimodal all have PSI $\approx$ 0.5 but have clearly different PSI distributions and therefore are categorized as different modalities. Moreover, the middle modality has different variance compared to both bimodal and multimodal modalities. Hence, in splicing analysis, variance is **informative**.  

Therefore, the aim of single-cell differential splicing analysis is to determine if the **distribution** of PSI values of one group of cells is significantly different from another group of cells. A suitable statistical method for comparing the PSI distribution between groups of single cell is the Kolmogorov-Smirnov test. Other statistical test options available in MARVEL are wilcoxon and t-tests.  

## Running the code
The function to perform differential splicing and gene expression analysis is ```CompareValues```. Set the ```level``` argument to ```"splicing"``` or ```"gene"``` for differential splicing or gene expression analysis, respectively.  

```{r size="small", warning=FALSE, message=FALSE, echo=FALSE, eval=FALSE}
# Differential splicing analysis
marvel <- CompareValues(
            MarvelObject=marvel,               # Marvel object
            cell.types=c("iPSC", "Endoderm"),  # Cell types to analyse
            n.cells=25,                        # Min. no. of cells PSI != NA required 
            method="ks",                       # "ks"/"wilcox"/"t.test"
            method.adj="fdr",                  # Adjust for multiple testing as per p.adjust
            level="splicing"                   # "gene"/"splicing" data to analyse
            )

marvel$DE$PSI[1:5,]

# Differential gene expression analysis
marvel <- CompareValues(
            MarvelObject=marvel,              # Marvel object
            cell.types=c("iPSC", "Endoderm"), # Cell types to include for analysis
            n.cells=3,                        # Min. no. of cells expression value > 1 required 
            method="wilcox",                  # "wilcox"/"t.test" 
            method.adj="fdr",                 # Adjust for multiple testing as per p.adjust
            level="gene"                      # "gene"/"splicing" data for DE analysis
            )

marvel$DE$Gene[1:5,]
```

## Understanding the output
The results for differential splicing or gene expression analysis are returned to ```$DE$PSI``` or ```DE$Gene``` splot, respectively.  
(1) ```n.cells``` column indicates no. of cells expressing the splicing event (PSI != NA) or gene (value != 0).  
(2) ```mean.g1``` and ```mean.g2``` columsn indicate mean values for first and second cell type, respectively, as specified in the ```cell.type``` argument.  
(3) ```mean.diff``` / ```mean.fd``` = ```mean.g2``` - ```mean.g1```.  
(4) ```p.val``` column indicates unadjusted p-values from statistical test as specified in ```method``` argument.  
(5) ```p.val.adj``` column indicates adjusted p-values as specified in ```method.adj``` argument.  

# 7. Final remarks
## How MARVEL was benchmarked 

We selected three datasets to orthogonally validate MARVEL. The results of our benchmarking exercise are described under Appendix.  
(1) The first dataset consists of single and matched-bulk induced pluripotent stem cells, motor neurons, and neural progenitor cells. Single cells were prepared using SMARTer Ultra Low RNA cDNA Synthesis Kit and sequenced on Illumina HiSeq2000 in 100bp PE mode (*Song et al., 2017*).  
(2) The second dataset consists of single and matched-bulk cultured myoblast harvested at 0-, 24-, 48, and 72-hrs. Single cells were prepared using SMARTer Ultra Low RNA cDNA Synthesis Kit and sequenced on Illumina HiSeq2500 in 100bp PE mode (*Trapnell et al., 2014*).  
(3) The third dataset consists of single induced pluripotent stem cells and endoderm cells. Single cells were prepared using Smart-seq2 and sequenced on Illumina HiSeq2000 in 125bp PE mode (*Linker et al., 2019*).  

We compared MARVEL against two other peer-reviewed single-cell splicing softwares, namely BRIE and Expedition. Notably, MARVEL includes features that were not available or limited in either BRIE or Expedition (Please refer to comparison table under **Introduction**). Therefore, MARVEL is a more comprehensive package compared to either BRIE or Expedition alone for single-cell splicing analysis.  

It is noteworthy that although MARVEL was benchmarked using single-cell data prepared using Smart-seq2 and SMARTer protocols, MARVEL can in principle be used to quantify and perform differential splicing analysis from any Illumina short-read RNA-sequencing experiments. These include 5'- and 3'-biased datasets such as those generated using 10x Genomics.  

## Companion of MARVEL

Previously, we developed VALERIE for *in silico* validation of splicing events at single-cell resolution. VALERIE is an acronym for **V**isualizing **AL**ternative splicing **E**vents from single-cell **RI**bonucleic-acid (RNA)-sequencing **E**xperiments. Users may use MARVEL to quantify splicing events and identify splicing events that are differentially expressed between groups of single cells. These significant splicing events can be visually validated using VALERIE to further narrow down true positives for downstream analysis and functional validation. Our paper on VALERIE has been published in PLoS Computational Biology (*Wen et al., 2020b*).  

# 8. Appendix

## Compute percent spliced-in (PSI)

```{r message=FALSE, echo=FALSE}
# Check plot
output <- system.file("extdata", "Vignette_Fig_6_Appendix_Single-Cell_Correlation-min.png", package="MARVEL")
knitr::include_graphics(output)
```

**Figure 6**: Correlation of single-cell PSI values for skipped-exon (SE) between BRIE, Expedition, and MARVEL. Overall correlation between the softwares are high (median > 0.80 for all comparisons across all three datasets). BRIE combines sequence features predictive of PSI together with sequencing reads information to **infer** PSI values. This is known as the Bayesian approach. On the other hand, Expedition and MARVEL use sequencing reads to **directly** compute PSI values. This is known as the frequentist approach. Therefore, correlaton between frequentist approaches (e.g. MARVEL vs Expedition) is higher compared to that of frequentist vs Bayesian approach (e.g. Expedition vs BRIE, MARVEL vs BRIE). PSI: Percent spliced-in.

```{r message=FALSE, echo=FALSE}
# Check plot
output <- system.file("extdata", "Vignette_Fig_7_Appendix_Single-Cell-Bulk_Correlation-min.png", package="MARVEL")
knitr::include_graphics(output)
```

**Figure 7**: Correlation of single-cell average PSI values vs matched-bulk PSI values for skipped-exon (SE) for BRIE, Expedition, and MARVEL. Overall correlation for all softwares are high (median > 0.75 all two datasets). Notably, MARVEL has the highest single-cell vs matched-bulk correlation, followed by BRIE and Expedition. PSI: Percent spliced-in.

## Correcting false bimodals

```{r message=FALSE, echo=FALSE}
# Check plot
output <- system.file("extdata", "Vignette_Fig_8_Appendix_Genes_vs_mRNA_Counts-min.png", package="MARVEL")
knitr::include_graphics(output)
```

**Figure 8**: A previous study recommended including only genes with at least 10 mRNA molecule counts for modality analysis in order to eliminate false bimodal patterns (*Buen Abad Najar et al, 2020*). However, using this stringent threshold removes a significant proportion of genes for modality analysis. More genes become ineligible when a higher no. of cells are specified for modality analysis. iPSC: Induced pluripotent stem cells. MN: Motor neurons. NPC: Neural progenitor cells.

```{r message=FALSE, echo=FALSE}
# Check plot
output <- system.file("extdata", "Vignette_Fig_9_Appendix_Bimodal_Features-min.png", package="MARVEL")
knitr::include_graphics(output)
```

**Figure 9**: To distinguish between true and false bimodal patterns, we first identified a set of true and false bimodal patterns as controls (please see *Assign modalities: Identifying false bimodals*). We found the (Left) ratio of 3 and (Right) differences of 45% between the proportion of cells at one end over the other end are able to make this distinction.

```{r message=FALSE, echo=FALSE}
# Check plot
output <- system.file("extdata", "Vignette_Fig_10_Appendix_Modality_Proportions-min.png", package="MARVEL")
knitr::include_graphics(output)
```

**Figure 10**: Comparison of modality proportion across different strategies of modality assignment. Expedition, which does not identify and adjust for false bimodal patterns gives the highest proportion of bimodal modality. On the other hand, requiring genes to have at least 10 mRNA counts for modality assignment removes false bimodals but at the expense of decreasing the no. of splicing events eligible for modality assignment. MARVEL, which identifies and adjusts for false bimodal patterns, yields comparable bimodal proportion without lost of splicing events available for modality analysis.

# 9. References

Buen Abad Najar, C. F., Yosef, N., & Lareau, L. F. (2020). Coverage-dependent bias creates the appearance of binary splicing in single cells. Elife, 9. doi:10.7554/eLife.54603  

Dobin, A., Davis, C. A., Schlesinger, F., Drenkow, J., Zaleski, C., Jha, S., . . . Gingeras, T. R. (2013). STAR: ultrafast universal RNA-seq aligner. Bioinformatics, 29(1), 15-21. doi:10.1093/bioinformatics/bts635  

Huang, Y., & Sanguinetti, G. (2017). BRIE: transcriptome-wide splicing quantification in single cells. Genome Biol, 18(1), 123. doi:10.1186/s13059-017-1248-5  

Kovaka, S., Zimin, A. V., Pertea, G. M., Razaghi, R., Salzberg, S. L., & Pertea, M. (2019). Transcriptome assembly from long-read RNA-seq alignments with StringTie2. Genome Biol, 20(1), 278. doi:10.1186/s13059-019-1910-1  

Li, B., & Dewey, C. N. (2011). RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome. BMC |     Bioinformatics, 12, 323. doi:10.1186/1471-2105-12-323  

Linker, S. M., Urban, L., Clark, S. J., Chhatriwala, M., Amatya, S., McCarthy, D. J., . . . Bonder, M. J. (2019). Combined single-cell profiling of expression and DNA methylation reveals splicing regulation and heterogeneity. Genome Biol, 20(1), 30. doi:10.1186/s13059-019-1644-0  

Shen, S., Park, J. W., Lu, Z. X., Lin, L., Henry, M. D., Wu, Y. N., . . . Xing, Y. (2014). rMATS: robust and flexible detection of differential alternative splicing from replicate RNA-Seq data. Proc Natl Acad Sci U S A, 111(51), E5593-5601. doi:10.1073/pnas.1419161111  

Smart, A. C., Margolis, C. A., Pimentel, H., He, M. X., Miao, D., Adeegbe, D., . . . Van Allen, E. M. (2018). Intron retention is a source of neoepitopes in cancer. Nat Biotechnol, 36(11), 1056-1058. doi:10.1038/nbt.4239  

Smith, M. A., Choudhary, G. S., Pellagatti, A., Choi, K., Bolanos, L. C., Bhagat, T. D., . . . Starczynowski, D. T. (2019). U2AF1 mutations induce oncogenic IRAK4 isoforms and activate innate immune pathways in myeloid malignancies. Nat Cell Biol, 21(5), 640-650. doi:10.1038/s41556-019-0314-5  

Song, Y., Botvinnik, O. B., Lovci, M. T., Kakaradov, B., Liu, P., Xu, J. L., & Yeo, G. W. (2017). Single-Cell Alternative Splicing Analysis with Expedition Reveals Splicing Dynamics during Neuron Differentiation. Mol Cell, 67(1), 148-161 e145. doi:10.1016/j.molcel.2017.06.003  

Trapnell, C., Cacchiarelli, D., Grimsby, J., Pokharel, P., Li, S., Morse, M., . . . Rinn, J. L. (2014). The dynamics and regulators of cell fate decisions are revealed by pseudotemporal ordering of single cells. Nat Biotechnol, 32(4), 381-386. doi:10.1038/nbt.2859  

Wen, W. X., Mead, A. J., & Thongjuea, S. (2020a). Technological advances and computational approaches for alternative splicing analysis in single cells. Comput Struct Biotechnol J, 18, 332-343. doi:10.1016/j.csbj.2020.01.009  

Wen, W. X., Mead, A. J., & Thongjuea, S. (2020b). VALERIE: Visual-based inspection of alternative splicing events at single-cell resolution. PLoS Comput Biol, 16(9), e1008195. doi:10.1371/journal.pcbi.1008195  
