# tutorials

topic: transcriptomics
tutorial: rna-seq-reads-to-counts
title: 1: RNA-Seq reads to counts
difficulty: intermediate
time_estimation: 3h
requirements_json: []
tags_json: ["collections", "mouse", "QC"]
content_hash: 6b3d0be577009c5d4e2fa326a1ed67f7
last_modified: 2026-05-15T22:28:17.652586
zenodo_link: https://zenodo.org/record/4249555
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/rna-seq-reads-to-counts/tutorial.html

content:
Measuring gene expression on a genome-wide scale has become common practice over the last two decades or so, with microarrays predominantly used pre-2008. With the advent of next generation sequencing technology in 2008, an increasing number of scientists use this technology to measure and understand changes in gene expression in often complex systems. As sequencing costs have decreased, using RNA-Seq to simultaneously measure the expression of tens of thousands of genes for multiple samples has never been easier. The cost of these experiments has now moved from generating the data to storing and analysing it.

There are many steps involved in analysing an RNA-Seq experiment. The analysis begins with sequencing reads (FASTQ files). These are usually aligned to a reference genome, if available. Then the number of reads mapped to each gene can be counted. This results in a table of counts, which is what we perform statistical analyses on to determine differentially expressed genes and pathways. The purpose of this tutorial is to demonstrate how to do read alignment and counting, prior to performing differential expression. Differential expression analysis with limma-voom is covered in an accompanying tutorial [RNA-seq counts to genes]({% link topics/transcriptomics/tutorials/rna-seq-counts-to-genes/tutorial.md %}). The tutorial here shows how to start from FASTQ data and perform the mapping and counting steps, along with associated Quality Control.

## Mouse mammary gland dataset

The data for this tutorial comes from a Nature Cell Biology paper by {% cite Fu2015 %}. Both the raw data (sequence reads) and processed data (counts) can be downloaded from Gene Expression Omnibus database (GEO) under accession number [GSE60450](http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE60450).

This study examined the expression profiles of basal and luminal cells in the mammary gland of virgin, pregnant and lactating mice. Six groups are present, with one for each combination of cell type and mouse status. Note that two biological replicates are used here, two independent sorts of cells from the mammary glands of virgin, pregnant or lactating mice, however three replicates is usually recommended as a minimum requirement for RNA-seq.

This is a Galaxy tutorial based on material from the [COMBINE R RNAseq workshop](http://combine-australia.github.io/RNAseq-R/07-rnaseq-day2.html), first taught [at a workshop in 2016](http://combine-australia.github.io/2016-05-11-RNAseq/).

![Tutorial Dataset](../../images/rna-seq-reads-to-counts/mouse_exp.png "Tutorial Dataset")


> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}


{% snippet faqs/galaxy/analysis_results_may_vary.md %}

# Preparing the reads

## Import data from URLs

{% assign servers = nil | list_usegalaxy_servers_shuffle %}

Read sequences are usually stored in compressed (gzipped) FASTQ files. Before the differential expression analysis can proceed, these reads must be aligned to the reference genome and counted into annotated genes. Mapping reads to the genome is a very important task, and many different aligners are available, such as HISAT2 ({% cite kim2015hisat %}), STAR ({% cite dobin2013star %}) and Subread ({% cite Liao2013 %}). Most mapping tasks require larger computers than an average laptop, so usually read mapping is done on a server in a linux-like environment, requiring some programming knowledge. However, Galaxy enables you to do this mapping without needing to know programming and if you don't have access to a server you can try to use one of the publically available Galaxies e.g. {% for server in servers %}[{{ server.name }}]({{ server.url }}){% if forloop.last %}{% else %}, {% endif %}{% endfor %}.

The raw reads used in this tutorial were obtained from SRA from the link in GEO for the the mouse mammary gland dataset (e.g `ftp://ftp-trace.ncbi.nlm.nih.gov/sra/sra-instant/reads/ByStudy/sra/SRP%2FSRP045%2FSRP045534`). For the purpose of this tutorial we are going to be working with a small part of the FASTQ files. We are only going to be mapping 1000 reads from each sample to enable running through all the steps quickly. If working with your own data you would use the full data and some results for the full mouse dataset will be shown for comparison. The small FASTQ files are available in [Zenodo](https://zenodo.org/record/4249555) and the links to the FASTQ files are provided below.

If you are sequencing your own data, the sequencing facility will almost always provide compressed FASTQ files which you can upload into Galaxy. For sequence data available through URLs, The [Galaxy Rule-based Uploader]({% link topics/galaxy-interface/tutorials/upload-rules/tutorial.md %}) can be used to import the files. It is much quicker than downloading FASTQs to your computer and uploading into Galaxy and also enables importing as a **Collection**. When you have more than a few files, using Galaxy Collections helps keep the datasets organised and tidy in the history. Collections also make it easier to maintain the sample names through tools and workflows. If you are not familiar with collections, you can take a look at the [Galaxy Collections tutorial]({% link topics/galaxy-interface/tutorials/collections/tutorial.md %}) for more details. The screenshots below show a comparison of what the FASTQ datasets for this tutorial would look like in the history if we imported them as datasets versus as a collection with the Rule-based Uploader.

Datasets | Collection
--- | ---
![Samples imported as datasets](../../images/rna-seq-reads-to-counts/imported-as-datasets.png)| ![Samples imported as collection with Rule-based Uploader](../../images/rna-seq-reads-to-counts/imported-as-collection.png)


> <details-title>Collections and sample names</details-title>
>
> Collections can also help to maintain the original sample names on the files throughout the tools used. The screenshots below show what we would see in one of the MultiQC reports that we will generate if we used datasets versus a collection.
>
> Datasets | Collection
> --- | ---
> ![Sample names without collection](../../images/rna-seq-reads-to-counts/samplesnames_without_collection.png)| ![Sample names with collection](../../images/rna-seq-reads-to-counts/samplesnames_with_collection.png)
>
{: .details}


The information we need to import the samples for this tutorial (sample ID, Group, and link to the FASTQ file (URL) are in the grey box below.

```
SampleID	Group	URL
MCL1-DL	basallactate	https://zenodo.org/record/4249555/files/SRR1552455.fastq.gz
MCL1-DK	basallactate	https://zenodo.org/record/4249555/files/SRR1552454.fastq.gz
MCL1-DJ	basalpregnant	https://zenodo.org/record/4249555/files/SRR1552453.fastq.gz
MCL1-DI	basalpregnant	https://zenodo.org/record/4249555/files/SRR1552452.fastq.gz
MCL1-DH	basalvirgin	https://zenodo.org/record/4249555/files/SRR1552451.fastq.gz
MCL1-DG	basalvirgin	https://zenodo.org/record/4249555/files/SRR1552450.fastq.gz
MCL1-LF	luminallactate	https://zenodo.org/record/4249555/files/SRR1552449.fastq.gz
MCL1-LE	luminallactate	https://zenodo.org/record/4249555/files/SRR1552448.fastq.gz
MCL1-LD	luminalpregnant	https://zenodo.org/record/4249555/files/SRR1552447.fastq.gz
MCL1-LC	luminalpregnant	https://zenodo.org/record/4249555/files/SRR1552446.fastq.gz
MCL1-LB	luminalvirgin	https://zenodo.org/record/4249555/files/SRR1552445.fastq.gz
MCL1-LA	luminalvirgin	https://zenodo.org/record/4249555/files/SRR1552444.fastq.gz
```


In order to get these files into Galaxy, we will want to do a few things:

* Strip the *header* out of the sample information (it doesn’t contain a URL Galaxy can download).
* Define the file **Identifier** column (`SampleID`).
* Define the **URL** column (`URL`) (this is the location Galaxy can download the data from).

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this tutorial e.g. `RNA-seq reads to counts`
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 2. Import the files from Zenodo using Galaxy's Rule-based Uploader.
>    - Open the Galaxy Upload Manager
>    - Click the tab **Rule-based**
>        - *"Upload data as"*: `Collection(s)`
>        - *"Load tabular data from"*: `Pasted Table`
>    - Paste the table from the grey box above. *(You should now see below)*
>    - Click **Build**
>
>       ![Rule-based Uploader](../../images/rna-seq-reads-to-counts/rule_uploader.png "Rule-based Uploader")
>
>    - In the `rules editor` that pops up:
>
>        - **Remove the header**. From the **Filter** menu select `First or Last N Rows`
>            - *"Filter which rows?"*: `first`
>            - *"Filter how many rows?"*: `1`
>            - Click `Apply`
>
>        - **Define the Identifier and URL columns**. From the **Rules** menu select `Add / Modify Column Definitions`
>            - Click `Add Definition` button and select List Identifier(s)
>                - *"List Identifier(s)"*: `A`
>            - Click `Add Definition` button again and select URL instead
>                - *"URL"*: `C`
>            - Click `Apply`, and you should see your new column definitions listed
>
>        - **Name the collection**. For *"Name"* enter: `fastqs` *(You should now see below)*
>        - Click `Upload`
>
>        ![Rules Editor](../../images/rna-seq-reads-to-counts/rule_editor.png "Rules Editor")
>
>        You should see a collection (list) called `fastqs` in your history containing all 12 FASTQ files.
>
>
{: .hands_on}

If your data is not accessible by URL, for example, if your FASTQ files are located on your laptop and are not too large, you can upload into a collection as below. If they are large you could use FTP. You can take a look at the [Getting data into Galaxy slides]({% link topics/galaxy-interface/tutorials/get-data/slides.html %}) for more information.

> <tip-title>Upload local files into a collection</tip-title>
>
> - Open the Galaxy Upload Manager
> - Click the tab **Collection**
> - Click **Choose Local Files** and locate the files you want to upload
>     - *"Collection Type"*: `List`
> - In the pop up that appears:
>     - *"Name"*: `fastqs`
>     - Click `Create list`
{: .tip}

If your FASTQ files are located in Shared Data, you can import them into your history as a collection as below.

> <tip-title>Import files from Shared Data into a collection</tip-title>
>
> - In the Menu at the top go to Shared Data > Data Libraries
> - Locate your FASTQ files
> - Tick the checkboxes to select the files
> - From the **To History** menu select `as a Collection`
> - In the pop up that appears:
>     - *"Which datasets?"*: `current selection`
>     - *"Collection type"*: `List`
>     - *"Select history"*: `select your History`
>     - Click `Continue`
> - In the pop up that appears:
>     - *"Name"*: `fastqs`
>     - Click `Create list`
{: .tip}

Take a look at one of the FASTQ files to see what it contains.

> <hands-on-title>Take a look at FASTQ format</hands-on-title>
>
> 1. Click on the collection name (`fastqs`)
> 2. Click on the {% icon galaxy-eye %} (eye) icon of one of the FASTQ files to have a look at what it contains
{: .hands_on}

> <details-title>FASTQ format</details-title>
> If you are not familiar with FASTQ format, see the [Quality Control tutorial]({% link topics/sequence-analysis/tutorials/quality-control/tutorial.md %})
{: .details}


## Raw reads QC

During sequencing, errors are introduced, such as incorrect nucleotides being called. These are due to the technical limitations of each sequencing platform. Sequencing errors might bias the analysis and can lead to a misinterpretation of the data. Every base sequence gets a quality score from the sequencer and this information is present in the FASTQ file. A quality score of 30 corresponds to a 1 in 1000 chance of an incorrect base call (a quality score of 10 is a 1 in 10 chance of an incorrect base call). To look at the overall distribution of quality scores across the reads, we can use FastQC.

Sequence quality control is therefore an essential first step in your analysis. We will use similar tools as described in the ["Quality control" tutorial]({% link topics/sequence-analysis/tutorials/quality-control/tutorial.md %}): [FastQC](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/) and Cutadapt ({% cite marcel2011cutadapt %}).

> <hands-on-title>Check raw reads with <b>FastQC</b></hands-on-title>
>
> 1. {% tool [FastQC](toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.73+galaxy0) %}
>    - {% icon param-collection %} *"Short read data from your current history"*: `fastqs` (Input dataset collection)
> 2. Inspect the `Webpage` output of **FastQC** {% icon tool %} for the `MCL1-DL` sample by clicking on the {% icon galaxy-eye %} (eye) icon
>
> {% snippet faqs/galaxy/tools_select_collection.md %}
>
{: .hands_on}



> <question-title></question-title>
> 1. What is the read length?
> 2. What base quality score encoding is used?
>
> > <solution-title></solution-title>
> >
> > 1. The read length is 100 bp.
> > 2. Sanger quality score encoding is used.
> > This information can be seen at the top of the FastQC Webpage as below.
> >
> > ![FastQC Webpage](../../images/rna-seq-reads-to-counts/fastqc_webpage.png "FastQC Webpage")
> >
> {: .solution}
>
{: .question}

The FastQC report contains a lot of information and we can look at the report for each sample. However, that is quite a few reports, 12 for this dataset. If you had more samples it could be a lot more. Luckily, there is a very useful tool called MultiQC ({% cite ewels2016multiqc %}) that can summarise QC information for multiple samples into a single report. We'll generate a few MultiQC outputs in this tutorial so we'll add name tags so we can differentiate them.

> <hands-on-title>Aggregate FastQC reports with <b>MultiQC</b></hands-on-title>
>
> 1. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy0) %} with the following parameters to aggregate the FastQC reports
>      - In *"Results"*
>        - {% icon param-select %}*"Which tool was used generate logs?"*: `FastQC`
>        - In *"FastQC output"*
>           - {% icon param-select %} *"Type of FastQC output?"*: `Raw data`
>           - {% icon param-collection %} *"FastQC output"*: `RawData` files (output of **FastQC** {% icon tool %} on trimmed reads)
> 2. Add a tag `#fastqc-raw` to the `Webpage` output from MultiQC and inspect the webpage
>
> {% snippet faqs/galaxy/datasets_add_tag.md type="name" %}
>
{: .hands_on}


Note that these are the results for just 1000 reads. The FastQC results for the full dataset are shown below. The 1000 reads are the first reads from the FASTQ files, and the first reads usually originate from the flowcell edges, so we can expect that they may have lower quality and the patterns may be a bit different from the distribution in the full dataset.

You should see that most of the plots in the small FASTQs look similar to the full dataset. However, in the small FASTQs, there is less duplication, some Ns in the reads and some overrepresented sequences.

![General Statistics](../../images/rna-seq-reads-to-counts/fastqc_table.png "General Statistics")
![Sequence Counts](../../images/rna-seq-reads-to-counts/fastqc_sequence_counts_plot.png "Sequence Counts")
![Sequence Quality](../../images/rna-seq-reads-to-counts/fastqc_per_base_sequence_quality_plot.png "Sequence Quality")
![Per Sequence Quality Scores](../../images/rna-seq-reads-to-counts/fastqc_per_sequence_quality_scores_plot.png "Per Sequence Quality Scores")
![Per Sequence GC Content](../../images/rna-seq-reads-to-counts/fastqc_per_sequence_gc_content_plot.png "Per Sequence GC Content")
![Per base N content](../../images/rna-seq-reads-to-counts/fastqc_per_base_n_content_plot.png "Per base N content")
![Sequence Duplication Levels](../../images/rna-seq-reads-to-counts/fastqc_sequence_duplication_levels_plot.png "Sequence Duplication Levels")
![Adapter Content](../../images/rna-seq-reads-to-counts/fastqc_adapter_content_plot.png "Adapter Content")

See the [Quality Control tutorial]({% link topics/sequence-analysis/tutorials/quality-control/tutorial.md %}) for more information on FastQC plots.

> <question-title></question-title>
>
> What do you think of the overall quality of the sequences?
>
> > <solution-title></solution-title>
> >
> > Overall, the samples look pretty good. The main things to note here are:
> > * The **base quality** is high in all samples.
> > * Some Illumina **adapter** has been detected.
> > * Some duplication in RNA-seq can be normal due to the presence of highly expressed genes. However, for some reason `MCL1-LE` and `MCL1-LF` have higher numbers of duplicates detected than the other samples.
> >
> {: .solution}
{: .question}

We will use Cutadapt to trim the reads to remove the Illumina adapter and any low quality bases at the ends (quality score < 20). We will discard any sequences that are too short (< 20bp) after trimming. We will also output the Cutadapt report for summarising with MultiQC.

The Cutadapt tool Help section provides the sequence we can use to trim this standard Illumina adapter `AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC`, as given on the [Cutadapt website](https://cutadapt.readthedocs.io/en/stable/guide.html#illumina-truseq). For trimming paired-end data see the Cutadapt Help section. Other Illumina adapter sequences (e.g. Nextera) can be found at the [Illumina website](http://sapac.support.illumina.com/bulletins/2016/12/what-sequences-do-i-use-for-adapter-trimming.html). Note that Cutadapt requires at least three bases to match between adapter and read to reduce the number of falsely trimmed bases, which can be changed in the Cutadapt options if desired.

## Trim reads

> <hands-on-title>Trim reads with <b>Cutadapt</b></hands-on-title>
>
> 1. {% tool [Cutadapt](toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt/3.7+galaxy0) %}
>    - {% icon param-select %} *"Single-end or Paired-end reads?"*: `Single-end`
>        - {% icon param-collection %} *"FASTQ/A file"*: `fastqs` (Input dataset collection)
>        - In *"Read 1 Options"*:
>            - In *"3' (End) Adapters"*:
>                - Click on *"Insert 3' (End) Adapters"*:
>                - In *"1: 3' (End) Adapters"*:
>                    - {% icon param-select %} *"Source"*: `Enter custom sequence`
>                        - {% icon param-text %} *"Enter custom 3' adapter name (Optional)"*: `Illumina`
>                        - {% icon param-text %} *"Enter custom 3' adapter sequence"*: `AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC`
>    - In *"Filter Options"*:
>        - {% icon param-text %} *"Minimum length (R1)"*: `20`
>    - In *"Read Modification Options"*:
>        - {% icon param-text %} *"Quality cutoff"*: `20`
>    - {% icon param-check %} *"Outputs selector"*: `Report`
>
{: .hands_on}

We can take a look at the reads again now that they've been trimmed.

## Trimmed reads QC

> <hands-on-title>QC of trimmed reads with <b>FastQC</b></hands-on-title>
>
> 1. {% tool [FastQC](toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.73+galaxy0) %}
>    - {% icon param-collection %} *"Short read data from your current history"*: `RawData` (output of **Cutadapt** {% icon tool %})
> 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy0) %} with the following parameters to aggregate the FastQC reports
>    - In *"Results"*
>        - {% icon param-select %} *"Which tool was used generate logs?"*: `FastQC`
>        - In *"FastQC output"*
>           - {% icon param-select %} *"Type of FastQC output?"*: `Raw data`
>           - {% icon param-collection %} *"FastQC output"*: `RawData` files (output of **FastQC** {% icon tool %})
> 3. Add a tag `#fastqc-trimmed` to the `Webpage` output from MultiQC and inspect the webpage
{: .hands_on}

The MultiQC plot below shows the result from the full dataset for comparison.

![Adapter Content post-trimming](../../images/rna-seq-reads-to-counts/post_cutadapt_adapter_content.png "Adapter Content post-trimming")
![Sequence Length post-trimming](../../images/rna-seq-reads-to-counts/post_cutadapt_fastqc_sequence_length_distribution_plot.png "Sequence Length post-trimming")

After trimming we can see that:

* No adapter is detected now.
* The reads are no longer all the same length, we now have sequences of different lengths detected.

# Mapping

Now that we have prepared our reads, we can align the reads for our 12 samples. There is an existing reference genome for mouse and we will map the reads to that. The current most widely used version of the mouse reference genome is `mm10/GRCm38` (although note that there is a new version `mm39` released June 2020). Here we will use [**HISAT2**](https://ccb.jhu.edu/software/hisat2/index.shtml) to align the reads. HISAT2 is the descendent of TopHat, one of the first widely-used aligners, but alternative mappers could be used, such as STAR. See the [RNA-seq ref-based tutorial]({{ site.baseurl }}/topics/transcriptomics/tutorials/ref-based/tutorial.html#mapping) for more information on RNA-seq mappers. There are often numerous mapping parameters that we can specify, but usually the default mapping parameters are fine. However, library type (paired-end vs single-end) and library strandness (stranded vs unstranded) require some different settings when mapping and counting, so they are two important pieces of information to know about samples. The mouse data comprises unstranded, single-end reads so we will specify that where necessary. HISAT2 can output a mapping summary file that tells what proportion of reads mapped to the reference genome. Summary files for multiple samples can be summarised with MultiQC. As we’re only using a subset of 1000 reads per sample, aligning should just take a minute or so. To run the full samples from this dataset would take longer.

## Map reads to reference genome

> <hands-on-title>Map reads to reference with <b>HISAT2</b></hands-on-title>
>
> 1. {% tool [HISAT2](toolshed.g2.bx.psu.edu/repos/iuc/hisat2/hisat2/2.2.1+galaxy0) %} with the following parameters:
>    - {% icon param-select %} *"Source for the reference genome"*: `Use a built-in genome`
>        - {% icon param-select %} *"Select a reference genome"*: `mm10`
>    - {% icon param-select %} *"Is this a single or paired library?"*: `Single-end`
>        - {% icon param-collection %} *"FASTA/Q file"*: `Read 1 Output` (output of **Cutadapt** {% icon tool %})
>    - In *"Summary Options"*:
>        - {% icon param-check %} *"Output alignment summary in a more machine-friendly style."*: `Yes`
>        - {% icon param-check %} *"Print alignment summary to a file."*: `Yes`
> 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy0) %} with the following parameters to aggregate the HISAT2 summary files
>    - In *"Results"*
>        - {% icon param-select %} *"Which tool was used generate logs?"*: `HISAT2`
>        - {% icon param-collection %} *"Output of HISAT2"*: `Mapping summary` (output of **HISAT2** {% icon tool %})
> 3. Add a tag `#hisat` to the `Webpage` output from MultiQC and inspect the webpage
{: .hands_on}

> <comment-title>Settings for Paired-end or Stranded reads</comment-title>
>
> - If you have **paired-end** reads
>     - Select *"Is this a single or paired library"* `Paired-end` or `Paired-end Dataset Collection` or `Paired-end data from single interleaved dataset`
> - If you have **stranded** reads
>     - Select *"Specify strand information"*: `Forward (FR)` or `Reverse (RF)`
{: .comment}

The MultiQC plot below shows the result from the full dataset for comparison.

![HISAT2 mapping](../../images/rna-seq-reads-to-counts/hisat2_se_plot.png "HISAT2 mapping")

An important metric to check is the percentage of reads mapped to the reference genome. A low percentage can indicate issues with the data or analysis. Over 90% of reads have mapped in all samples, which is a good mapping rate, and the vast majority of reads have mapped uniquely, they haven't mapped to multiple locations in the reference genome.

It is also good practice to visualise the read alignments in the BAM file, for example using IGV, see the [RNA-seq ref-based tutorial]({{ site.baseurl }}/topics/transcriptomics/tutorials/ref-based/tutorial.html#inspection-of-the-mapping-results).
{: .hands_on}

**HISAT2** generates a BAM file with mapped reads.

{% include topics/sequence-analysis/tutorials/mapping/bam_explanation.md mapper="HISAT2" %}

> <tip-title>Downloading a collection</tip-title>
>
> To download a collection of datasets (e.g. the collection of BAM files) click on the floppy disk icon within the collection. This will download a tar file containing all the datasets in the collection. Note that for BAM files the .bai indexes (required for IGV) will be included automatically in the download.
>
{: .tip}


# Counting

The alignment produces a set of BAM files, where each file contains the read alignments for each sample. In the BAM file, there is a chromosomal location for every read that mapped. Now that we have figured out where each read comes from in the genome, we need to summarise the information across genes or exons. The mapped reads can be counted across mouse genes by using a tool called featureCounts ({% cite liao2013featurecounts %}). featureCounts requires gene annotation specifying the genomic start and end position of each exon of each gene. For convenience, featureCounts contains built-in annotation for mouse (`mm10`, `mm9`) and human (`hg38`, `hg19`) genome assemblies, where exon intervals are defined from the NCBI RefSeq annotation of the reference genome. Reads that map to exons of genes are added together to obtain the count for each gene, with some care taken with reads that span exon-exon boundaries. The output is a count for each Entrez Gene ID, which are numbers such as `100008567`. For other species, users will need to read in a data frame in GTF format to define the genes and exons. Users can also specify a custom annotation file in SAF format. See the tool help in Galaxy, which has an example of what an SAF file should like like, or the Rsubread users guide for more information.


> <comment-title></comment-title>
>
> In this example we have kept many of the default settings, which are typically optimised to work well under a variety of situations. For example, the default setting for featureCounts is that it only keeps reads that uniquely map to the reference genome. For testing differential expression of genes, this is preferred, as the reads are unambigously assigned to one place in the genome, allowing for easier interpretation of the results. Understanding all the different parameters you can change involves doing a lot of reading about the tool that you are using, and can take a lot of time to understand! We won’t be going into the details of the parameters you can change here, but you can get more information from looking at the tool help.
{: .comment}

## Count reads mapped to genes

> <hands-on-title>Count reads mapped to genes with <b>featureCounts</b></hands-on-title>
>
> 1. {% tool [featureCounts](toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts/2.0.1+galaxy2) %} with the following parameters:
>    - {% icon param-collection %} *"Alignment file"*: `aligned reads (BAM)` (output of **HISAT2** {% icon tool %})
>    - {% icon param-select %} *"Gene annotation file"*: `featureCounts built-in`
>        - {% icon param-select %} *"Select built-in genome"*: `mm10`
>
> 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy0) %} with the following parameters:
>    - {% icon param-select %} *"Which tool was used generate logs?"*: `featureCounts`
>        - {% icon param-collection %} *"Output of FeatureCounts"*: `featureCounts summary` (output of **featureCounts** {% icon tool %})
> 3. Add a tag `#featurecounts` to the `Webpage` output from MultiQC and inspect the webpage
{: .hands_on}

> <comment-title>Settings for Paired-end or Stranded reads</comment-title>
>
> - If you have **paired-end** reads
>     - Click *"Options for paired-end reads"*
>         - {% icon param-select %} *"Count fragments instead of reads"*: `Enabled; fragments (or templates) will be counted instead of reads`
> - If you have **stranded** reads
>     - {% icon param-select %} Select *"Specify strand information"*: `Stranded (Forward)` or `Stranded (Reverse)`
{: .comment}

The MultiQC plot below shows the result from the full dataset for comparison.

![featureCounts assignments](../../images/rna-seq-reads-to-counts/featureCounts_assignment_plot.png "featureCounts assignments")

> <question-title></question-title>
>
> What % reads are assigned to exons?
>
> > <solution-title></solution-title>
> >
> > ~60-70% of reads are assigned to exons. This is a fairly typical number for RNA-seq.
> >
> {: .solution}
>
{: .question}


The counts for the samples are output as tabular files. Take a look at one. The numbers in the first column of the counts file represent the Entrez gene identifiers for each gene, while the second column contains the counts for each gene for the sample.

## Create count matrix

The counts files are currently in the format of one file per sample. However, it is often convenient to have a count matrix. A count matrix is a single table containing the counts for all samples, with the genes in rows and the samples in columns. The counts files are all within a collection so we can use the Galaxy **Column Join on multiple datasets** tool to easily create a count matrix from the single counts files.

> <hands-on-title>Create count matrix with <b>Column Join on multiple datasets</b></hands-on-title>
>
> 1. {% tool [Column Join on multiple datasets](toolshed.g2.bx.psu.edu/repos/iuc/collection_column_join/collection_column_join/0.0.3) %} with the following parameters:
>    - {% icon param-collection %} *"Tabular files"*: `Counts` (output of **featureCounts** {% icon tool %})
>    - {% icon param-text %} *"Identifier column"*: `1`
>    - {% icon param-text %} *"Number of header lines in each input file"*: `1`
>    - {% icon param-check %} *"Add column name to header"*: `No`
>
{: .hands_on}

Take a look at the output. Note that as the tutorial uses a small subset of the data (~ 1000 reads per sample), to save on processing time, most rows in that matrix will contain all zeros (there will be ~600 non-zero rows). The output for the full dataset is shown below.

![Count matrix](../../images/rna-seq-reads-to-counts/count_matrix.png "Count matrix")

Now it is easier to see the counts for a gene across all samples. The accompanying tutorial, [RNA-seq counts to genes]({% link topics/transcriptomics/tutorials/rna-seq-counts-to-genes/tutorial.md %}), shows how gene information (symbols etc) can be added to a count matrix.

# Generating a QC summary report

There are several additional QCs we can perform to better understand the data, to see if it's good quality. These can also help determine if changes could be made in the lab to improve the quality of future datasets.

We'll use a prepared workflow to run the first few of the QCs below. This will also demonstrate how you can make use of Galaxy workflows to easily run and reuse multiple analysis steps. The workflow will run the first three tools: **Infer Experiment**, **MarkDuplicates** and **IdxStats** and generate a **MultiQC** report. You can then edit the workflow if you'd like to add other steps.

> <hands-on-title>Run QC report workflow</hands-on-title>
>
> 1. **Import the workflow** into Galaxy
>    - Copy the URL (e.g. via right-click) of [this workflow]({{ site.baseurl }}{{ page.dir }}workflows/qc_report.ga) or download it to your computer.
>    - Import the workflow into Galaxy
>
>    {% snippet faqs/galaxy/workflows_run_trs.md path="topics/transcriptomics/tutorials/rna-seq-reads-to-counts/workflows/qc_report.ga" title="QC Report" %}
>
> 2. Import this file as type BED file:
>    ```
>    https://sourceforge.net/projects/rseqc/files/BED/Mouse_Mus_musculus/mm10_RefSeq.bed.gz/download
>    ```
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Run **Workflow QC Report** {% icon workflow %} using the following parameters:
>    - *"Send results to a new history"*: `No`
>    - {% icon param-file %} *"1: Reference genes"*: the imported RefSeq BED file
>    - {% icon param-collection %} *"2: BAM files"*: `aligned reads (BAM)` (output of **HISAT2** {% icon tool %})
>
>    {% snippet faqs/galaxy/workflows_run.md %}
> 4. Inspect the `Webpage` output from MultiQC
{: .hands_on}


**You do not need to run the hands-on steps below.** They are just to show how you could run the tools individually and what parameters to set.

## Strandness

As far as we know this data is unstranded, but as a sanity check you can check the strandness. You can use RSeQC Infer Experiment tool to "guess" the strandness, as explained in the [RNA-seq ref-based tutorial]({% link topics/transcriptomics/tutorials/ref-based/tutorial.md %}). This is done through comparing the "strandness of reads" with the "strandness of transcripts". For this tool, and many of the other RSeQC ({% cite wang2012rseqc %}) tools, a reference bed file of genes (`reference genes`) is required. RSeQC provides some reference BED files for model organisms. You can import the RSeQC mm10 RefSeq BED file from the link `https://sourceforge.net/projects/rseqc/files/BED/Mouse_Mus_musculus/mm10_RefSeq.bed.gz/download` (and rename to `reference genes`) or import a file from Data if provided. Alternatively, you can provide your own BED file of reference genes, for example from UCSC (see the [Peaks to Genes tutorial]({% link topics/introduction/tutorials/galaxy-intro-peaks2genes/tutorial.md %}). Or the **Convert GTF to BED12** tool can be used to convert a GTF into a BED file.

> <hands-on-title>Check strandness with <b>Infer Experiment</b></hands-on-title>
>
> 1. {% tool [Infer Experiment](toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_infer_experiment/2.6.4.1) %} with the following parameters:
>    - {% icon param-collection %} *"Input .bam file"*: `aligned reads (BAM)` (output of **HISAT2** {% icon tool %})
>    - {% icon param-file %} *"Reference gene model"*: `reference genes` (Reference BED file)
> 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy0) %} with the following parameters:
>       - In *"1: Results"*:
>           - {% icon param-select %} *"Which tool was used generate logs?"*: `RSeQC`
>               - {% icon param-select %} *"Type of RSeQC output?"*: `infer_experiment`
>                   - {% icon param-collection %} *"RSeQC infer_experiment output"*: `Infer Experiment output` (output of **Infer Experiment** {% icon tool %})
> 3. Inspect the `Webpage` output from MultiQC
{: .hands_on}

The MultiQC plot below shows the result from the full dataset for comparison.

![Infer Experiment](../../images/rna-seq-reads-to-counts/rseqc_infer_experiment_plot.png "Infer Experiment")

> <question-title></question-title>
>
> Do you think the data is stranded or unstranded?
>
> > <solution-title></solution-title>
> >
> > It is unstranded as approximately equal numbers of reads have aligned to the sense and antisense strands.
> >
> {: .solution}
>
{: .question}


## Duplicate reads

Duplicate reads are usually kept in RNA-seq differential expression analysis as they can come from highly-expressed genes but it is still a good metric to check. A high percentage of duplicates can indicate a problem with the sample, for example, PCR amplification of a low complexity library (not many transcripts) due to not enough RNA used as input. FastQC gives us an idea of duplicates in the reads before mapping (note that it just takes a sample of the data). We can assess the numbers of duplicates in all mapped reads using the **Picard MarkDuplicates** tool. Picard considers duplicates to be reads that map to the same location, based on the start position of where the read maps. In general, we consider normal to obtain up to 50% of duplication.

> <hands-on-title>Check duplicate reads with <b>MarkDuplicates</b></hands-on-title>
>
> 1. {% tool [MarkDuplicates](toolshed.g2.bx.psu.edu/repos/devteam/picard/picard_MarkDuplicates/2.18.2.3) %} with the following parameters:
>    - {% icon param-collection %} *"Select SAM/BAM dataset or dataset collection"*: `aligned reads (BAM)` (output of **HISAT2** {% icon tool %})
> 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy0) %} with the following parameters:
>       - In *"1: Results"*:
>           - {% icon param-select %} *"Which tool was used generate logs?"*: `Picard`
>               - {% icon param-select %} *"Type of Picard output?"*: `Markdups`
>                   - {% icon param-collection %} *"Picard output"*: `MarkDuplicate metrics` (output of **MarkDuplicates** {% icon tool %})
> 3. Inspect the `Webpage` output from MultiQC
{: .hands_on}

The MultiQC plot below shows the result from the full dataset for comparison.

![MarkDups metrics](../../images/rna-seq-reads-to-counts/picard_deduplication.png "MarkDups metrics")

> <question-title></question-title>
>
> Which two samples have the most duplicates detected?
>
> > <solution-title></solution-title>
> >
> > `MCL1-LE` and `MCL1-LF` have the highest number of duplicates in mapped reads compared to the other samples, similar to what we saw in the raw reads with FastQC.
> >
> {: .solution}
>
{: .question}

## Reads mapped to chromosomes

You can check the numbers of reads mapped to each chromosome with the **Samtools IdxStats** tool. This can help assess the sample quality, for example, if there is an excess of mitochondrial contamination. It could also help to check the sex of the sample through the numbers of reads mapping to X/Y or to see if any chromosomes have highly expressed genes.

> <hands-on-title>Count reads mapping to each chromosome with <b>IdxStats</b></hands-on-title>
>
> 1. {% tool [IdxStats](toolshed.g2.bx.psu.edu/repos/devteam/samtools_idxstats/samtools_idxstats/2.0.4) %} with the following parameters:
>    - {% icon param-collection %} *"BAM file"*: `aligned reads (BAM)` (output of **HISAT2** {% icon tool %})
> 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy0) %} with the following parameters:
>       - In *"1: Results"*:
>           - {% icon param-select %} *"Which tool was used generate logs?"*: `Samtools`
>               - {% icon param-select %} *"Type of Samtools output?"*: `idxstats`
>                   - {% icon param-collection %} *"Samtools idxstats output"*: `IdxStats output` (output of **IdxStats** {% icon tool %})
> 3. Inspect the `Webpage` output from MultiQC
{: .hands_on}

The MultiQC plot below shows the result from the full dataset for comparison.

![IdxStats chromsome mappings](../../images/rna-seq-reads-to-counts/samtools-idxstats-mapped-reads-plot.png "IdxStats Chromosome Mappings")
![IdxStats XY ](../../images/rna-seq-reads-to-counts/samtools-idxstats-xy-plot.png "IdxStats X/Y Mappings")

> <question-title></question-title>
>
> 1. What do you think of the chromosome mappings?
> 2. Are the samples male or female? *(If a sample is not in the XY plot it means no reads mapped to Y)*
>
> > <solution-title></solution-title>
> >
> > 1. Some of the samples have very high mapping on chromosome 5. What is going on there?
> > 2. The samples appear to be all female as there are few reads mapping to the Y chromosome. As this is a experiment studying virgin, pregnant and lactating mice if we saw large numbers of reads mapping to the Y chromosome in a sample it would be unexpected and a probable cause for concern.
> >
> {: .solution}
>
{: .question}

## Gene body coverage (5'-3')

The coverage of reads along gene bodies can be assessed to check if there is any bias in coverage. For example, a bias towards the 3' end of genes could indicate degradation of the RNA. Alternatively, a 3' bias could indicate that the data is from a 3' assay (e.g. oligodT-primed, 3'RNA-seq). You can use the RSeQC **Gene Body Coverage (BAM)** tool to assess gene body coverage in the BAM files.

> <hands-on-title>Check coverage of genes with <b>Gene Body Coverage (BAM)</b></hands-on-title>
>
> 1. {% tool [Gene Body Coverage (BAM)](toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_geneBody_coverage/2.6.4.3) %} with the following parameters:
>    - *"Run each sample separately, or combine mutiple samples into one plot"*: `Run each sample separately`
>        - {% icon param-collection %} *"Input .bam file"*: `aligned reads (BAM)` (output of **HISAT2** {% icon tool %})
>    - {% icon param-file %} *"Reference gene model"*: `reference genes` (Input dataset)
> 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy0) %} with the following parameters:
>       - In *"1: Results"*:
>           - {% icon param-select %} *"Which tool was used generate logs?"*: `RSeQC`
>               - {% icon param-select %} *"Type of RSeQC output?"*: `gene_body_coverage`
>                   - {% icon param-collection %} *"RSeQC gene_body_coverage output"*: `Gene Body Coverage (BAM) (text)` (output of **Gene Body Coverage (BAM)** {% icon tool %})
> 3. Inspect the `Webpage` output from MultiQC
>
{: .hands_on}

The MultiQC plot below shows the result from the full dataset for comparison.

![Gene Body Coverage](../../images/rna-seq-reads-to-counts/rseqc_gene_body_coverage_plot.png "Gene Body Coverage")

The plot below from the RSeQC website shows what samples with 3'biased coverage would look like.
![Gene Body Coverage comparison](../../images/rna-seq-reads-to-counts/genebodycoverage.png "Gene Body Coverage comparison")

> <question-title></question-title>
>
> What do you think of the coverage across gene bodies in these samples?
>
> > <solution-title></solution-title>
> >
> > It looks good. This plot looks a bit noisy in the small FASTQs but it still shows there's pretty even coverage from 5' to 3' ends with no obvious bias in all the samples.
> >
> {: .solution}
>
{: .question}


## Read distribution across features (exons, introns, intergenic..)

We can also check the distribution of reads across known gene features, such as exons (CDS, 5'UTR, 3'UTR), introns and intergenic regions. In RNA-seq we expect most reads to map to exons rather than introns or intergenic regions. It is also the reads mapped to exons that will be counted so it is good to check what proportions of reads have mapped to those. High numbers of reads mapping to intergenic regions could indicate the presence of DNA contamination.

> <hands-on-title>Check distribution of reads with <b>Read Distribution</b></hands-on-title>
>
> 1. {% tool [Read Distribution](toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_read_distribution/2.6.4.1) %} with the following parameters:
>    - {% icon param-collection %} *"Input .bam/.sam file"*: `aligned reads (BAM)` (output of **HISAT2** {% icon tool %})
>    - {% icon param-file %} *"Reference gene model"*: `reference genes` (Input dataset)
> 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy0) %} with the following parameters:
>       - In *"1: Results"*:
>           - {% icon param-select %} *"Which tool was used generate logs?"*: `RSeQC`
>               - {% icon param-select %} *"Type of RSeQC output?"*: `read_distribution`
>                   - {% icon param-collection %} *"RSeQC read_distribution output"*: `Read Distribution output` (output of **Read Distribution** {% icon tool %})
> 3. Inspect the `Webpage` output from MultiQC
>
{: .hands_on}

The MultiQC plot below shows the result from the full dataset for comparison.

![Read Distribution](../../images/rna-seq-reads-to-counts/rseqc_read_distribution_plot.png "Read Distribution")

> <question-title></question-title>
>
> What do you think of the read distribution?
>
> > <solution-title></solution-title>
> >
> > It looks good, most of the reads have mapped to exons and not many to introns or intergenic regions. The samples have pretty consistent read distribution, albeit with slightly higher numbers of reads mapping to CDS exons for `MCL1-LC` and `MCL1-LD`, and `MCL1-LE` and `MCL1-LF` have more reads mapping to CDS exons than the other samples.
> >
> {: .solution}
>
{: .question}


The MultiQC report can be downloaded by clicking on the floppy disk icon on the dataset in the history.

> <question-title></question-title>
>
> Can you think of any other QCs that could be performed on RNA-seq reads?
>
> > <solution-title></solution-title>
> >
> > The reads could be checked for:
> > * Ribosomal contamination
> > * Contamination with other species e.g. bacteria
> > * GC bias of the mapped reads
> > * This is single-end data but paired-end mapped reads could be checked for fragment size (distance between the read pairs).
> >
> {: .solution}
>
{: .question}


# Conclusion


In this tutorial we have seen how reads (FASTQ files) can be converted into counts. We have also seen QC steps that can be performed to help assess the quality of the data. A follow-on tutorial, [RNA-seq counts to genes]({% link topics/transcriptomics/tutorials/rna-seq-counts-to-genes/tutorial.md %}), shows how to perform differential expression and QC on the counts for this dataset.

tools_json:
["FastQC", "toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.73+galaxy0", "MultiQC", "toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy0", "Cutadapt", "toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt/3.7+galaxy0", "HISAT2", "toolshed.g2.bx.psu.edu/repos/iuc/hisat2/hisat2/2.2.1+galaxy0", "featureCounts", "toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts/2.0.1+galaxy2", "Column Join on multiple datasets", "toolshed.g2.bx.psu.edu/repos/iuc/collection_column_join/collection_column_join/0.0.3", "Infer Experiment", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_infer_experiment/2.6.4.1", "MarkDuplicates", "toolshed.g2.bx.psu.edu/repos/devteam/picard/picard_MarkDuplicates/2.18.2.3", "IdxStats", "toolshed.g2.bx.psu.edu/repos/devteam/samtools_idxstats/samtools_idxstats/2.0.4", "Gene Body Coverage (BAM)", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_geneBody_coverage/2.6.4.3", "Read Distribution", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_read_distribution/2.6.4.1"]


topic: transcriptomics
tutorial: rna-seq-viz-with-volcanoplot-r
title: Visualization of RNA-Seq results with Volcano Plot in R
difficulty: intermediate
time_estimation: 1H
requirements_json: []
tags_json: ["interactive-tools"]
content_hash: 9ad1d0149539bc29d08fee674f6d6db7
last_modified: 2026-05-15T22:28:17.656547
zenodo_link: https://zenodo.org/record/2529117
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/rna-seq-viz-with-volcanoplot-r/tutorial.html

content:
The [Volcano plot]({% link topics/transcriptomics/tutorials/rna-seq-viz-with-volcanoplot/tutorial.md %}) tutorial, introduced volcano plots and showed how they can be generated with the Galaxy Volcano plot tool. In this tutorial we show how you can customise a plot using the R script output from the tool.

> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Preparing the inputs

We will use one file for this analysis:

 * **Differentially expressed results file** (genes in rows, and 4 required columns: raw P values, adjusted P values (FDR), log fold change and gene labels).

 If you are following on from the [Volcano plot]({% link topics/transcriptomics/tutorials/rna-seq-viz-with-volcanoplot/tutorial.md %}) tutorial, you already have this file in your History so you can skip to the **Create volcano plot** step below.

## Import data

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this exercise e.g. `Volcano plot R`
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 2. Import the differentially results table.
>
>    To import the file, there are two options:
>    - Option 1: From a shared data library if available (ask your instructor)
>    - Option 2: From [Zenodo](https://zenodo.org/record/2529117)
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
>    - You can paste the link below into the **Paste/Fetch** box:
>
>      ```
>      https://zenodo.org/record/2529117/files/limma-voom_luminalpregnant-luminallactate
>      ```
>
>      Check that the datatype is `tabular`.
>      If the datatype is not `tabular`, please change the file type to `tabular`.
>
>      {% snippet faqs/galaxy/datasets_change_datatype.md datatype="tabular" %}
{: .hands_on}

Click on the {% icon galaxy-eye %} (eye) icon and take a look at the `DE results` file. It should look like below, with 8 columns.

![DE results file](../../images/rna-seq-viz-with-volcanoplot/DE_results.png)

## Create volcano plot

We will create a volcano plot colouring all significant genes. We will call genes significant here if they have FDR < 0.01 and a log2 fold change of 0.58 (equivalent to a fold-change of 1.5). These were the values used in the original paper for this dataset. We will also label the top 10 most significant genes with their gene names. We will select to output the Rscript file which we will then use to edit the plot in R.

> <hands-on-title>Create a Volcano plot</hands-on-title>
>
> 1. {% tool [Volcano Plot](toolshed.g2.bx.psu.edu/repos/iuc/volcanoplot/volcanoplot/0.0.5) %} to create a volcano plot
>    - {% icon param-file %} *"Specify an input file"*: the de results file
>    - {% icon param-file %} *"File has header?"*: `Yes`
>    - {% icon param-select %} *"FDR (adjusted P value)"*: `Column 8`
>    - {% icon param-select %} *"P value (raw)"*: `Column 7`
>    - {% icon param-select %} *"Log Fold Change"*: `Column 4`
>    - {% icon param-select %} *"Labels"*: `Column 2`
>    - {% icon param-text %} *"Significance threshold"*: `0.01`
>    - {% icon param-text %} *"LogFC threshold to colour"*: `0.58`
>    - {% icon param-select %} *"Points to label"*: `Significant`
>        - {% icon param-text %} *"Only label top most significant"*: `10`
>    - In *"Output Options"*:
>        - {% icon param-select %} *"Output Rscript?"*: `Yes`
{: .hands_on}

Click on the PDF file name to check that you see a plot like below.

![Volcano plot labelling top significant genes](../../images/rna-seq-viz-with-volcanoplot/volcanoplot_top10.png){: width="60%"}

Now we will customise the plot by editing the R code in RStudio. You can use Galaxy RStudio if available or another R such as RStudio Cloud or RStudio installed on your computer.

## Import files into R

We'll import the differentially expressed results input file and the RScript into R.

> <hands-on-title>Using datasets from Galaxy</hands-on-title>
>
> 1. Note the history IDs of 1) the differentially expressed results and 2) the RScript in your Galaxy history
>
> 2. {% tool [RStudio](interactive_tool_rstudio) %} in Galaxy provides some special functions such as `gx_get` to import and export files from your history.
>
>    {% snippet faqs/galaxy/interactive_tools_rstudio_launch.md %}
>
> 3. Copy the files we need into our workspace so we can see them in the Files pane.
>
>    ```R
>    file.copy(gx_get(1), "de-results.tsv") # will copy dataset number 1 from your history, use the correct ID for your differentially expressed results dataset.
>    file.copy(gx_get(3), "volcano.R") # will copy dataset number 3 from your history, use the correct ID for your Rscript dataset.
>    ```
>
> 4. Click on `volcano.R` in the Files pane to open it in the Editor pane.
>
{: .hands_on}


We'll have a look at the script.

## Set up script

The first few lines from `# Galaxy settings start` to `# Galaxy settings end` are settings needed to run the Volcano plot tool in Galaxy. We don't need them to run the script in R so we will delete them. If we don't delete the error handling line, the R session will crash if we encounter any error in the code. It's ok as it will resume again where we were but better to not have this happen.

> <hands-on-title>Delete unneeded lines</hands-on-title>
>
> Delete these lines from the top of the script.
>
>    ```R
>    # Galaxy settings start ---------------------------------------------------
>
>    # setup R error handling to go to stderr
>    options(show.error.messages = F, error = function() {cat(geterrmessage(), file = stderr()); q("no", 1, F)})
>
>    # we need that to not crash galaxy with an UTF8 error on German LC settings.
>    loc <- Sys.setlocale("LC_MESSAGES", "en_US.UTF-8")
>
>    # Galaxy settings end -----------------------------------------------------
>    ```
>
{: .hands_on}

We'll check if we have the packages the script needs. We can see the packages used in the lines that have `library(package)`

```R
library(dplyr)
library(ggplot2)
library(ggrepel)
```

When we launched Galaxy RStudio there was information in the Console letting us know that some packages are pre-installed. These packages include `ggplot2` and `dplyr`. In this Galaxy there is a yellow warning banner across the top of the script saying `Package ggrepel required is not installed. Install. Don't Show Again`.
So we just need to install the `ggrepel` package.

> <hands-on-title>Install package</hands-on-title>
>
> Either click on "Install" in the yellow warning banner if present, or in the Console type
>
> ```R
> install.packages('ggrepel')
> ```
>
{: .hands_on}

We need to change the path of the differentially expressed file in the script. The path in the script is `/data/dnb03/galaxy_db/files/4/6/c/dataset_46c498bc-060e-492f-9b42-51908a55e354.dat`. This is a temporary location where the Galaxy Volcano plot tool copied the input file in order to use it, the file no longer exists there. Your path will be different. In the script change this path to `de-results.tsv` like below.

> <hands-on-title>Run script</hands-on-title>
>
> 1. Change the input file path in script
>
>    ```R
>    # change the line
>    results <- read.delim('/data/dnb03/galaxy_db/files/4/6/c/dataset_46c498bc-060e-492f-9b42-51908a55e354.dat', header = TRUE)
>
>    # to
>    results <- read.delim('de-results.tsv', header = TRUE)
>    ```
>
> 2. Highlight the code in the script and run
>
>    > <tip-title>Highlight all code and run shortcuts</tip-title>
>    > - To highlight all code type <kbd>CTRL</kbd>+<kbd>a</kbd> (or <kbd>CMD</kbd>+<kbd>a</kbd>)
>    > - To run type <kbd>CTRL</kbd>+<kbd>Enter</kbd> (or <kbd>CMD</kbd>+<kbd>Enter</kbd>)
>    {: .tip}
>
{: .hands_on}

You should see a file called `volcano_plot.pdf` appear in the Files pane. Click on it to open it and you should see a plot that looks the same as the one we generated with the Volcano Plot tool in Galaxy.

We'll delete the lines below that save the plot to a PDF file. The plots will then be produced in the Plots pane so we can more easily see the different plots we're going to make, without having to keep opening the PDF file.

> <hands-on-title>Produce plots in Plots pane</hands-on-title>
>
> 1. Delete the lines below that save the plot to a PDF file
>
>    ```R
>    # Open PDF graphics device
>    pdf("volcano_plot.pdf")
>
>    # keep the lines in between as they produce the plot
>
>    # Close PDF graphics device
>    dev.off()
>    ```
>
> 2. Highlight the code in the script and run
>
{: .hands_on}

You should now see the plot produced in the Plots pane.

# Customising the plot

## Change points colours

We'll demonstate how you can change the colours. We'll change the colour of the downregulated genes from cornflowerblue to purple. We'll change the upregulated genes from firebrick to orange.

> <hands-on-title>Change colours</hands-on-title>
>
> 1. Edit the line below in the script
>
>    ```R
>    # change the line
>    colours <- setNames(c("cornflowerblue", "grey", "firebrick"), c(down, notsig, up))
>
>    # to
>    colours <- setNames(c("purple", "grey", "orange"), c(down, notsig, up))
>    ```
>
> 2. Highlight the code in the script and run
>
>   ![Smaller points](../../images/rna-seq-viz-with-volcanoplot-r/volcano_points_colours.png)
>
{: .hands_on}


> <tip-title>R colours</tip-title>
>
> If you want to use other colours you can see the built-in R colours with their names in [this cheatsheet](https://www.nceas.ucsb.edu/sites/default/files/2020-04/colorPaletteCheatsheet.pdf).
>
{: .tip}


## Change points size

We'll make the points a bit smaller. We'll change to 0.5.

> <hands-on-title>Change points size</hands-on-title>
>
> 1. Edit the line below in the script
>
>    ```R
>    # change the line
>    geom_point(aes(colour = sig)) +
>
>    # to
>    geom_point(aes(colour = sig), size = 0.5) +
>    ```
>
> 2. Highlight the code in the script and run
>
>   ![Smaller points](../../images/rna-seq-viz-with-volcanoplot-r/volcano_points_size.png)
>
{: .hands_on}


> <question-title></question-title>
>
> How could we change the transparency of the points?
>
>    > <solution-title></solution-title>
>    >
>    > We could use `alpha =`. For example
>    > ```R
>    > geom_point(aes(colour = sig), alpha = 0.5)
>    > ```
>    >
>    {: .solution}
{: .question}


## Change labels size

We'll make the font size of the labels a bit smaller.

> <hands-on-title>Change labels text size</hands-on-title>
>
> 1. Edit the line below in the script
>
>    ```R
>    # change the line
>    geom_text_repel(data = filter(results, labels != ""), aes(label = labels),
>
>    # to
>    geom_text_repel(data = filter(results, labels != ""), aes(label = labels), size = 3,
>    ```
>
> 2. Highlight the code in the script and run
>
>   ![Smaller labels](../../images/rna-seq-viz-with-volcanoplot-r/volcano_labels_size.png)
>
{: .hands_on}


> <question-title></question-title>
>
> How could we change the number of genes labelled from 10 to 20?
>
>    > <solution-title></solution-title>
>    >
>    > We could change the 10 to 20 here
>    > ```R
>    > top <- slice_min(results, order_by = pvalue, n = 20)
>    > ```
>    >
>    {: .solution}
{: .question}



## Change categories

We can change the categories of points we're colouring in the plot. For example, instead of using separate categories for upregulated, downregulated we could just use a single category for significant.

> <hands-on-title>Change categories</hands-on-title>
>
> 1. Change the category names to signif and notsignif
>
>    ```R
>    # change
>    down <- unlist(strsplit('Down,Not Sig,Up', split = ","))[1]
>    notsig <- unlist(strsplit('Down,Not Sig,Up', split = ","))[2]
>    up <- unlist(strsplit('Down,Not Sig,Up', split = ","))[3]
>
>    # to
>    signif <- "Significant"
>    notsignif <- "Not significant"
>    ```
>
> 2. Specify which genes are signif and notsignif
>
>    ```R
>    # change
>    results <- mutate(results, sig = case_when(
>                                fdr < 0.01 & logfc > 0.58 ~ up,
>                                fdr < 0.01 & logfc < -0.58 ~ down,
>                                TRUE ~ notsig))
>
>    # to
>    results <- mutate(results, sig = case_when(
>                                fdr < 0.01 & abs(logfc) > 0.58 ~ signif, # abs() will give us absolute values i.e. all > 0.58 and < -0.58
>                                TRUE ~ notsignif))
>    ```
>
> 3. Specify the colours for signif and notsignif
>
>    ```R
>    # change
>    colours <- setNames(c("purple", "grey", "orange"), c(down, notsig, up))
>
>    # to
>    colours <- setNames(c("grey", "red"), c(notsignif, signif))
>    ```
>
> 4. runHighlight the code in the script and
>
>   ![Categories changed](../../images/rna-seq-viz-with-volcanoplot-r/volcano_categories.png)
>
{: .hands_on}


> <question-title></question-title>
>
> How would you remove the legend from the plot? You can use Google.
>
>    > <solution-title></solution-title>
>    >
>    > If you Google `remove legend ggplot2` you may find a few ways it can be done. One way is
>    >
>    > ```R
>    > p <- p + theme(legend.position = "none")
>    > ```
>    >
>    {: .solution}
{: .question}

You can save the edited script by clicking the {% icon galaxy-save %} icon at the top of the script in RStudio or through File > Save. You can download from Galaxy RStudio from the Files pane by ticking the box beside the script name, then More > Export > Download.

> <tip-title>More plot options</tip-title>
>
> If you enter values in the Volcano Plot Galaxy tool form for Plot options, such as plot title, x and y axis labels or limits, they'll be output in the script. This is one way you could see how to customise these options in the plot.
>
{: .tip}


# Conclusion


In this tutorial we have seen how a volcano plot can be generated and customised using Galaxy and R. You can see some more possible customisations in the [RNA Seq Counts to Viz in R]({% link topics/transcriptomics/tutorials/rna-seq-viz-with-volcanoplot/tutorial.md %}) tutorial and at the [ggrepel website](https://ggrepel.slowkow.com/index.html).

tools_json:
["Volcano Plot", "toolshed.g2.bx.psu.edu/repos/iuc/volcanoplot/volcanoplot/0.0.5", "RStudio", "interactive_tool_rstudio"]


topic: transcriptomics
tutorial: minerva-pathways
title: Pathway analysis with the MINERVA Platform
difficulty: intermediate
time_estimation: 1h
tools_json: []
requirements_json: []
tags_json: []
content_hash: 61422f91bd7c526ff21e74c67714d0c7
last_modified: 2026-05-15T22:28:17.623411
zenodo_link: https://zenodo.org/records/10405036
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/minerva-pathways/tutorial.html

content:
This tutorial is a partial reproduction of {% cite Togami_2022 %} wherein they evaluated mRNA and miRNA in a selection of COVID-19 patients and healthy controls.
While that paper uses a closed source pipeline, we'll be reproducing the analysis with open source tools in Galaxy, using a workflow on WorkflowHub developed for the BY-COVID project.

> <comment-title>Full data</comment-title>
> The original data is available at NCBI under [BioProject PRJNA754796](https://www.ncbi.nlm.nih.gov/bioproject/PRJNA754796)
{: .comment}
>
> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

There are several places you can jump to in this tutorial, using pre-calculated
data. We recommend you jump skipping the data download and counting step, and
skipping to the analysis, as that precludes the slowest and most data intensive
parts of this tutorial. However, the entire process is documented in case you
want to reproduce our work.

## Study Design

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this tutorial
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the factor table from [Zenodo]({{ page.zenodo_link }})
>
>    ```
>    https://zenodo.org/records/10405036/files/factordata.tabular
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 4. Check that the datatype
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="tabular" %}
>
{: .hands_on}

## Analysis

We have split this workflow into three parts, based only on how long the first two portions of the workflow take to execute.
The rough runtime of the workflow portions, when this was being developed, can be broken down as follows:

Step                     | Time
---                      | ---
Data Download            | ~6h
Processing Counts        | ~8h
Analysis & Visualisation | 15m

These numbers were generated on UseGalaxy.eu and may not represent the most
efficient possible computation, as they are executed on a shared cluster that can, at times, be more or less busy.

As such we recommend you skip to the analysis step to progress to the interesting portion of the tutorial. We have provided in the Zenodo record data from the entire analysis,
analysed with the Download & Counts steps that can be skipped.

{% include _includes/cyoa-choices.html option1="Download" option2="Skip" default="Skip"
       text="Choose whether you want to run the time-consuming Download Data step." %}

<div class="Download" markdown=1>

### Data Download

We'll start by downloading our FASTQ files from the [GEO Dataset GSE182152](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE182152)

> <hands-on-title>Download the data from GEO (ETA: 6 Hours)</hands-on-title>
>
> 1. **Import the workflow** into Galaxy
>
>    {% snippet faqs/galaxy/workflows_run_trs.md path="topics/transcriptomics/tutorials/minerva-pathways/workflows/Galaxy-Workflow-BY-COVID__Data_Download.ga" title="Trim and Filter reads" version="1" %}
>
> 1. **Run the workflow** with the following parameters:
>
>    - *"Sample Table"*: `factordata.tabular`
{: .hands_on}

Here we have cut the `SRR*` identifiers from the sample table and downloaded them with `fasterq`, part of the SRA toolkit.

### Counts

With that done, we can start to analyse the data using HISAT2 and featureCounts.
This workflow takes in the RNA Sequencing data we've downloaded previously, before trimming it with cutadapt.
Both the trimmed and untrimmed reads are run through FastQC for visualisation.

<figure>
{% assign wf_counts = site | get_workflow:page, "Galaxy-Workflow-mRNA-Seq_BY-COVID_Pipeline__Counts.ga" %}
<pre class="mermaid">
{{ wf_counts.mermaid }}
</pre>
<figcaption><b>Workflow 1</b>: Counts Workflow for mRNA-Seq BY-COVID Pipeline.</figcaption>
</figure>

The trimmed reads are then handled by HISAT2 for alignment to the reference
human genome, and featureCounts is run for quantification.

This workflow produces a set of featureCounts tables, a set of featureLengths (needed for goseq), and a MultiQC report.

> <hands-on-title>Run the Workflow</hands-on-title>
>
> {% snippet faqs/galaxy/workflows_run_wfh.md title="mRNA-Seq BY-COVID Pipeline" wfhub_id="688" version="1" %}
>
> 1. **Run the workflow** with the following parameters:
>
{: .hands_on}

This workflow produces a handful of outputs: the featureCounts results, and a
MultiQC report. Looking at the report we see generally reasonable quality data.

</div>

### limma

<div class="Skip" markdown=1>

> <hands-on-title>Download the Counts Files</hands-on-title>
>
> 1. Open the Rule Builder
>    - *"Upload data as"*: `Collection(s)`
>    - *"Load tabular data from"*: `Pasted Table`
>    - **Paste** the following table:
>
>      ```
>      https://zenodo.org/records/10405036/files/gene_lengths.tabular	gene_lengths	Gene Lengths
>      https://zenodo.org/records/10405036/files/SRR15462516.featureCounts.tabular	SRR15462516	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462517.featureCounts.tabular	SRR15462517	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462518.featureCounts.tabular	SRR15462518	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462519.featureCounts.tabular	SRR15462519	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462520.featureCounts.tabular	SRR15462520	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462521.featureCounts.tabular	SRR15462521	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462522.featureCounts.tabular	SRR15462522	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462523.featureCounts.tabular	SRR15462523	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462524.featureCounts.tabular	SRR15462524	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462525.featureCounts.tabular	SRR15462525	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462526.featureCounts.tabular	SRR15462526	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462527.featureCounts.tabular	SRR15462527	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462528.featureCounts.tabular	SRR15462528	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462529.featureCounts.tabular	SRR15462529	featureCounts
>      https://zenodo.org/records/10405036/files/SRR15462530.featureCounts.tabular	SRR15462530	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681520.featureCounts.tabular	SRR16681520	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681521.featureCounts.tabular	SRR16681521	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681522.featureCounts.tabular	SRR16681522	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681523.featureCounts.tabular	SRR16681523	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681524.featureCounts.tabular	SRR16681524	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681525.featureCounts.tabular	SRR16681525	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681526.featureCounts.tabular	SRR16681526	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681527.featureCounts.tabular	SRR16681527	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681528.featureCounts.tabular	SRR16681528	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681529.featureCounts.tabular	SRR16681529	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681530.featureCounts.tabular	SRR16681530	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681531.featureCounts.tabular	SRR16681531	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681532.featureCounts.tabular	SRR16681532	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681533.featureCounts.tabular	SRR16681533	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681534.featureCounts.tabular	SRR16681534	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681535.featureCounts.tabular	SRR16681535	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681536.featureCounts.tabular	SRR16681536	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681537.featureCounts.tabular	SRR16681537	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681538.featureCounts.tabular	SRR16681538	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681539.featureCounts.tabular	SRR16681539	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681540.featureCounts.tabular	SRR16681540	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681541.featureCounts.tabular	SRR16681541	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681542.featureCounts.tabular	SRR16681542	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681543.featureCounts.tabular	SRR16681543	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681544.featureCounts.tabular	SRR16681544	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681545.featureCounts.tabular	SRR16681545	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681546.featureCounts.tabular	SRR16681546	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681547.featureCounts.tabular	SRR16681547	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681548.featureCounts.tabular	SRR16681548	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681549.featureCounts.tabular	SRR16681549	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681550.featureCounts.tabular	SRR16681550	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681551.featureCounts.tabular	SRR16681551	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681552.featureCounts.tabular	SRR16681552	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681553.featureCounts.tabular	SRR16681553	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681554.featureCounts.tabular	SRR16681554	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681555.featureCounts.tabular	SRR16681555	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681556.featureCounts.tabular	SRR16681556	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681557.featureCounts.tabular	SRR16681557	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681558.featureCounts.tabular	SRR16681558	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681559.featureCounts.tabular	SRR16681559	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681560.featureCounts.tabular	SRR16681560	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681561.featureCounts.tabular	SRR16681561	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681562.featureCounts.tabular	SRR16681562	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681563.featureCounts.tabular	SRR16681563	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681564.featureCounts.tabular	SRR16681564	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681565.featureCounts.tabular	SRR16681565	featureCounts
>      https://zenodo.org/records/10405036/files/SRR16681566.featureCounts.tabular	SRR16681566	featureCounts
>      ```
>
>    - Click `Build`
>
> 1. From **Rules** menu, select `Add / Modify Column Definitions`
>     - `Add Definition` → `Collection Name` → Select Column `C`
>     - `Add Definition` → `List Identifier(s)` → Select Column `B`
>     - `Add Definition` → `URL` → Column `A`
>
>     > <tip-title>Trouble entering?</tip-title>
>     > 1. Press the {% icon tool %} button by **Rules**
>     > 1. Paste the following JSON into the dialog:
>     >    ```json
>     >    {"rules":[],"mapping":[{"type":"collection_name","columns":[2]},{"type":"list_identifiers","columns":[1],"editing":false},{"type":"url","columns":[0]}],"genome":"hg19"}
>     >    ```
>     > 1. Click Apply
>     {: .tip}
>
> 1. At the bottom of the dialog set `Genome` to `hg19` (it is probably something like "Human Feb 2009 (GRCh37/hg19) (hg19)" but we are focused on that last parenthetical portion).
>
> 1. Click **Upload**
>
{: .hands_on}

</div>

Now we're ready to analyse the counts files. Here we'll take the feature counts dataset collection and merge it into one count matrix through the use of "Column join".
This can then be annotated with the human readable names of the genes. This is all passed to limma for differential expression analysis.

With this result in hand, we're ready to do two further steps: preparing the dataset for goseq, and for analysis in the MINERVA Platform.
Goseq is a tool for gene ontology enrichment analysis, and the MINERVA Platform is a tool for visualising pathway analysis.

<figure>
{% assign wf_counts = site | get_workflow:page, "Galaxy-Workflow-mRNA-Seq_BY-COVID_Pipeline__Analysis.ga" %}
<pre class="mermaid">
{{ wf_counts.mermaid }}
</pre>
<figcaption><b>Workflow 2</b>: Analysis Workflow for mRNA-Seq BY-COVID Pipeline.</figcaption>
</figure>

> <hands-on-title>Analyse the Counts</hands-on-title>
>
> 1. Run the workflow with the Factor Data from the first Hands-on, and the datasets from the workflow or Zenodo download, depending on your path:
>
>    {% snippet faqs/galaxy/workflows_run_wfh.md title="mRNA-Seq BY-COVID Pipeline" wfhub_id="689" version="1" %}
>
{: .hands_on}

You should have a few outputs, namely the `goseq` outputs, and a table ready for visualisation in the MINERVA Platform!

## The MINERVA Platform

The dataset prepared for the MINERVA Platform must be correctly formatted as a tabular dataset (`\t` separated values) like the following, with the dbkey set to `hg19` or `hg38`.
If you've run the above workflow, this should be the case.

```
SYMBOL  logFC              P.Value               adj.P.Val
TRIM25  2.07376444684004   1.2610025125617e-18   3.57368112059986e-15
ACSL1   2.90647033200259   2.71976234791064e-16  3.85390324698937e-13
NBEAL2  2.45952426389725   2.71787290816654e-14  2.56748394058132e-11
MIR150  -2.55304226607428  9.55912390273625e-14  6.74866152827879e-11
SLC2A3  2.95861349227708   1.19066011437523e-13  6.74866152827879e-11
```

The tabular dataset, as prepared above is then used by a dedicated MINERVA plugin ({% cite hoksza_minerva_2019 %}) to visualise the data on-the-fly in the [COVID-19 Disease Map](https://covid19map.elixir-luxembourg.org/minerva/). To visualise and explore the data, follow these steps:

> <hands-on-title>Visualise in MINERVA</hands-on-title>
>
> 1. **Click** to expand the final "MINERVA-Ready Table"
>
> 1. Click on the {% icon galaxy-barchart %} (**Visualize**) icon
>
>    ![A galaxy dataset with the barchart icon circled in red](images/1.png)
>
> 1. Select "display at Minerva (SARS-CoV-2 Minerva Map)"
>
>    ![The galaxy visualisation interface with a big display at minerva banner at the top](images/2.png)
>
>    > <tip-title>MINERVA not listed?</tip-title>
>    > The MINERVA visualisation is only for correctly formatted files with the correct genome (i.e. human, hg19).
>    > If you dont' see MINERVA listed, first check that your dataset is:
>    > 1. recognised as a tabular dataset
>    >
>    >    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="tabular" %}
>    >
>    > 2. Has the correct genome build:
>    >
>    >    {% snippet faqs/galaxy/datasets_change_dbkey.md dbkey="hg19" %}
>    >
>    >    It should be specifically `hg19` not a patch like `hg19Patch5`
>    >
>    > If that still doesn't work, please check that the Galaxy server you are using is updated to 24.0 or later.
>    >
>    {: .tip}
>
{: .hands_on}

### Analysis in the MINERVA Platform

Welcome to the MINERVA Covid-19 Disease Map! It has a similar interface to Galaxy, there is an interaction menu on the left, the main area is where you'll do your investigation,
and on the right are your datasets! In this case, the differentially expressed genes analysed above automatically loaded from Galaxy when you clicked "Display at MINERVA".

![MINERVA main view, a search menu on the left, a view of the disease map in the center showing a colourful cell diagram with signalling and T-Cells and CD8+ and more. On the right is a filtering box for the dataset from Galaxy allowing the user to filter by p-value and FC thresholds. 397 genes have been loaded.](images/3.png)

After the loading time, marked as "Reading Map Elements", the dataset will be visible in the right panel of the COVID-19 Disease Map, with the four corresponding columns specified earlier (see image below). The MINERVA-Galaxy plugin allows you to:

- filter the data table by fold-change (FC) threshold or by p-value (default: adjusted p-value, threshold set to 0.05)
- Search for specific gene symbols to display ("Search" box)
- Select specific differential expression values to display in the map (checkboxes in the data tab)
- Select all entries in the data table for visualisation (Select All)
- Reset the visualisation

The general process of data exploration looks like:

1. In the main map, find pathways with matching entries indicated by blue pins.
1. After selecting what you want to see, browse the COVID-19 Disease Map to explore the pathways with the corresponding expression pattern.
1. Select a pathway of your choice and in the left panel click the "Associated submap" button
1. Explore the expression patterns in the diagram that will be displayed.

> <hands-on-title>Explore TLR pathways</hands-on-title>
>
> 1. Use the **Search** box above the table on the right to search for **TLR**.
> 2. Select all four TLR genes.
>
>    - TLR3
>    - TLR4
>    - TLR7
>    - TLR8
>
> 2. In the main map, find *PAMP Signalling* and click on it. (Note: don't click the blue pin, click the pathway name)
>
>    ![The PAMP signalling box next to the golgi body. A blue map pin is placed upon the red and whitebox.](images/6.png)
>
> 1. In the left panel, click the **Associated submap: PAMP Signalling** button.
> 1. Explore the detailed diagram to examine the expression pattern.
>
>    > <question-title></question-title>
>    > What is the expression pattern of TLR3, TLR7, and TLR8 in the PAMP Signalling pathway?
>    > > <solution-title></solution-title>
>    > > TLR3 and TLR7 are downregulated (cool/blue colour), TLR8 is upregulated (warm/red colour).
>    > > ![image of the aforementioned regulation patterns](images/4.png)
>    > {: .solution}
>    {: .question}
>
>
> 1. Without closing the PAMP Signalling submap, **click** "Select all" to visualise the entire data table
>
>    > <question-title></question-title>
>    > What is the expression pattern downstream of TLR7 and TLR8, namely how are MYD88 and IRAK4 regulated?
>    > > <solution-title></solution-title>
>    > > MYD88 and IRAK4 are strongly and weakly upregulated, respectively, despite TLR7 downregulation.
>    > > ![image of the aforementioned regulation patterns](images/5.png)
>    > {: .solution}
>    {: .question}
>
{: .hands_on}

For further analysis in the MINERVA Platform, a [full user guide is available](https://covid19map.elixir-luxembourg.org/minerva/resources/other/user_guide.pdf)


topic: transcriptomics
tutorial: rna-seq-genes-to-pathways
title: 3: RNA-seq genes to pathways
difficulty: intermediate
time_estimation: 2h
tools_json: []
requirements_json: []
tags_json: ["mouse"]
content_hash: b15414241d9ce8401ee7f75aeb8e453e
last_modified: 2026-05-15T22:28:17.649376
zenodo_link: https://zenodo.org/record/2596382
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/rna-seq-genes-to-pathways/tutorial.html

content:
Sometimes there is quite a long list of genes to interpret after a differential expression analysis, and it is usually infeasible to go through the list one gene at a time trying to understand it’s biological function. A common downstream procedure is gene set testing, which aims to understand which pathways/gene networks the differentially expressed genes are implicated in. There are many different gene set testing methods that can be applied and it can be useful to try several.

The purpose of this tutorial is to demonstrate how to perform gene set testing using tools in Galaxy. The data comes from a Nature Cell Biology paper, [EGF-mediated induction of Mcl-1 at the switch to lactation is essential for alveolar cell survival](https://www.ncbi.nlm.nih.gov/pubmed/25730472)), Fu et al. 2015. That study examined the expression profiles of basal and luminal cells in the mammary gland of virgin, pregnant and lactating mice (see Figure below). How to generate differentially expressed genes from reads (FASTQs) for this dataset is covered in the accompanying tutorials [RNA-seq reads to counts]({% link topics/transcriptomics/tutorials/rna-seq-reads-to-counts/tutorial.md %}) and [RNA-seq counts to genes]({% link topics/transcriptomics/tutorials/rna-seq-counts-to-genes/tutorial.md %}). This tutorial is inspired by material from the COMBINE R RNAseq workshop [here](http://combine-australia.github.io/RNAseq-R/06-rnaseq-day1.html) and the Cancer Research UK workshop [here](https://bioinformatics-core-shared-training.github.io/cruk-summer-school-2018/RNASeq2018/html/06_Gene_set_testing.nb.html).

![Tutorial Dataset](../../images/rna-seq-reads-to-counts/mouse_exp.png "Tutorial Dataset")


> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

{% snippet faqs/galaxy/analysis_results_may_vary.md %}

We will use several files for this analysis:

 * **Differentially expressed results files** (genes in rows, logFC and P values in columns)
 * **Sample information** file (sample id, group)
 * **Gene lengths file** (genes, lengths)
 * **Filtered counts file** (genes in rows, counts for samples in columns, with lowly expressed genes removed)
 * **Gene sets file** for MSigDB Hallmark collection for mouse (rdata)


# Import data

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this RNA-seq exercise e.g. `RNA-seq genes to pathways`
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 2. **Import the files**, there are two options:
>    - Option 1: From a shared data library if available (ask your instructor)
>
>      {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
>    - Option 2: From [Zenodo](https://zenodo.org/record/2596382)
>
>      {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    You can paste the names and links below into the Upload tool:
>
>    ```
>    https://zenodo.org/record/2596382/files/limma-voom_basalpregnant-basallactate
>    https://zenodo.org/record/2596382/files/limma-voom_luminalpregnant-luminallactate
>    https://zenodo.org/record/2596382/files/seqdata
>    https://zenodo.org/record/2596382/files/mouse_hallmark_sets
>    https://zenodo.org/record/2596382/files/factordata
>    https://zenodo.org/record/2596382/files/limma-voom_filtered_counts
>    ```
>
> 3. Add a tag called `#basal` to the `limma-voom_basalpregnant-basallactate` and a tag called `#luminal` to the `limma-voom_luminalpregnant-luminallactate` files.
>
>    {% snippet faqs/galaxy/datasets_add_tag.md %}
>
{: .hands_on}

# Gene Set Testing

## Gene Ontology testing with **goseq**

We would like to know if there are biological categories that are enriched among the differentially expressed genes. To do this we will perform a Gene Ontology analysis, similar to the [RNA-seq ref-based tutorial]({% link topics/transcriptomics/tutorials/ref-based/tutorial.md %}).

[Gene Ontology (GO)](http://www.geneontology.org/) analysis is widely used to reduce complexity and highlight biological processes in genome-wide expression studies. However, standard methods give biased results on RNA-seq data due to over-detection of differential expression for long and highly-expressed transcripts.

The [goseq tool](https://bioconductor.org/packages/release/bioc/vignettes/goseq/inst/doc/goseq.pdf) provides methods for performing GO analysis of RNA-seq data, taking length bias into account. The methods and software used by goseq are equally applicable to other category based tests of RNA-seq data, such as KEGG pathway analysis.

goseq needs 2 files as inputs:
- a **differentially expressed genes** file. Information for all genes tested for differential expression (all genes after filtering lowly expressed). This file should have 2 columns:
    - the Gene IDs (unique within the file)
    - True (differentially expressed) or False (not differentially expressed)
- a **gene lengths** file. Information to correct for potential length bias in differentially expressed genes. This file should have 2 columns:
    - the Gene IDs (unique within the file)
    - the gene lengths

We need a table of differentially expressed (DE) results for goseq. Here we will use two tables, one for each of the basal and luminal pregnant vs lactate results. This is so we can compare results for the two cell types. These tables were output from the limma-voom tool but DE tables from edgeR or DESeq2 could also be used. For this dataset we will call genes differentially expressed if they have an adjusted P value below 0.01. We can use the gene lengths from the counts table in GEO (provided as a file called `seqdata` in Zenodo). But if we didn't have that we could use a tool like **featureCounts** {% icon tool %} to output a gene lengths file. The `seqdata` file contains >20k genes, but we only want the ~15k we have in our differentially expressed genes file. So we will join the lengths file with the differentially expressed genes file, keeping only the length information for genes present in the differentially expressed genes file. We can then cut out the columns we need for the two inputs (gene id, length) (gene id, DE status) and as a bonus they will both be sorted in the same order, which is what we need for goseq.

To generate the two input files we will use:
* **Compute** to add a column to the DE tables, that gives genes meeting our adj.P threshold the value "True" and all other genes the value "False".
* **Join two Datasets** to add the gene length information to the differentially expressed genes, matching on gene ids
* **Cut** to extract the two columns for the differentially expressed genes information
* **Cut** to extract the two columns for the gene length information


> <hands-on-title>Prepare the two inputs for GOSeq</hands-on-title>
>
> 1. **Compute an expression on every row** {% icon tool %} with the following parameters:
>    - {% icon param-text %} *"Add expression"*: `bool(c8<0.01)` (adj.P < 0.01)
>    - {% icon param-file %} *"as a new column to"*: the `limma-voom_basalpregnant-basallactate` file
>    - {% icon param-select %} *"Skip a header line"*: `yes`
>    - {% icon param-select %} *"The new column name"*: `Status`
> 2. **Join two Datasets side by side on a specified field** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"Join"*: output of **Compute** {% icon tool %}
>    - {% icon param-select %} *"using column"*: `Column: 1`
>    - {% icon param-file %} *"with"* the `seqdata` file
>    - {% icon param-select %} *"and column"*: `Column: 1`
>    - {% icon param-select %} *"Keep the header lines"*: `Yes`
> 3. **Cut columns from a table** {% icon tool %} with the following parameters:
>    - {% icon param-text %} *"Cut columns"*: `c1,c10` (the gene ids and DE status)
>    - {% icon param-select %} *"Delimited by"*: `Tab`
>    - {% icon param-file %} *"From"*: output of **Join** {% icon tool %}
>    - Rename to `goseq DE status`
> 4. **Cut columns from a table** {% icon tool %} with the following parameters:
>    - {% icon param-text %} *"Cut columns"*: `c1,c12` (the gene ids and lengths)
>    - {% icon param-select %} *"Delimited by"*: `Tab`
>    - {% icon param-file %} *"From"*: output of **Join** {% icon tool %}
>    - Rename to `goseq gene lengths`
{: .hands_on}

We now have the two required input files for goseq.

> <hands-on-title>Perform GO analysis</hands-on-title>
>
> 1. **goseq** {% icon tool %} with
>    - {% icon param-file %} *"Differentially expressed genes file"*: `goseq DE status`
>    - {% icon param-file %} *"Gene lengths file"*: `goseq gene lengths`
>    - {% icon param-select %} *"Gene categories"*:  `Get categories`
>       - {% icon param-select %} *"Select a genome to use"*:  `Mouse(mm10)`
>       - {% icon param-select %} *"Select Gene ID format"*:  `Entrez Gene ID`
>       - {% icon param-check %} *"Select one or more categories"*: `GO: Biological Process`
>    - *"Output Options"*
>        - {% icon param-check %} *"Output Top GO terms plot?"* `Yes`
{: .hands_on}

goseq generates a big table with the following columns for each GO term:
1. `category`: GO category
2. `over_rep_pval`: *p*-value for over representation of the term in the differentially expressed genes
3. `under_rep_pval`: *p*-value for under representation of the term in the differentially expressed genes
4. `numDEInCat`: number of differentially expressed genes in this category
5. `numInCat`: number of genes in this category
6. `term`: detail of the term
7. `ontology`: MF (Molecular Function - molecular activities of gene products), CC (Cellular Component - where gene products are active), BP (Biological Process - pathways and larger processes made up of the activities of multiple gene products)
8. `p.adjust.over_represented`: *p*-value for over representation of the term in the differentially expressed genes, adjusted for multiple testing with the Benjamini-Hochberg procedure
9. `p.adjust.under_represented`: *p*-value for over representation of the term in the differentially expressed genes, adjusted for multiple testing with the Benjamini-Hochberg procedure

To identify categories significantly enriched/unenriched below some p-value cutoff, it is necessary to use the adjusted *p*-value.

A plot of the top 10 over-represented GO terms (by *p*-value) can be output from the goseq tool to help visualise results. Click on the `Top over-represented GO terms plot` PDF in the history. It should look similar to below.

![Basal Plot](../../images/rna-seq-genes-to-pathways/basal_top_GO.png "Basal pregnant vs lactating top 10 GO terms")


> <question-title></question-title>
> Can you generate the plot of top 10 GO terms for the luminal differentially expressed genes? Tip: You could use the Rerun button for each step (as the parameters are already selected) and choose the luminal file as input.
>
> > <solution-title></solution-title>
> >
> > Running goseq on the luminal file should give a plot similar to below.
> > ![Luminal Plot](../../images/rna-seq-genes-to-pathways/luminal_top_GO.png "Luminal pregnant vs lactating top 10 GO terms")
> >
> {: .solution}
{: .question}

For more on GO analysis, including how to simplify GO results and visualize with GO graphs, see the [GO Enrichment tutorial]({{ site.baseurl }}/topics/transcriptomics/tutorials/goenrichment/tutorial.html).

## Gene Set Enrichment Analysis with **fgsea**

Gene Set Enrichment Analysis (GSEA) [(Subramanian et al., 2005)](https://www.ncbi.nlm.nih.gov/pubmed/16199517) is a widely used method that determines whether a set of genes is enriched in a list of differentially expressed genes. Unlike the previous method with goseq, no threshold is applied for what is considered "differentially expressed", all genes are used. If a gene set falls at either the top (over-expressed) or bottom (under-expressed) of the list it is said to be enriched. [fgsea](https://www.biorxiv.org/content/early/2016/06/20/060012) is a faster implementation of the GSEA method. fgsea requires a ranked list of genes and some gene sets to test.

The [Molecular Signatures Database (MSigDB)](http://software.broadinstitute.org/gsea/msigdb/index.jsp) contains curated collections of gene sets that are commonly used in a GSEA analysis. They can be downloaded from the [Broad website](http://software.broadinstitute.org/gsea/downloads.jsp). But these collections are only of human gene sets. If working with another species you would need to first map the genes to their human orthologues. However, MSigDB versions for mouse are provided [by the Smyth lab here](http://bioinf.wehi.edu.au/software/MSigDB/index.html). There are several MSigDB collections, we'll use the [Hallmark collection](https://www.cell.com/cell-systems/fulltext/S2405-4712(15)00218-5), which contains 50 gene sets. According to MSigDB, "each gene set in the hallmark collection consists of a “refined” gene set, derived from multiple “founder” sets, that conveys a specific biological state or process and displays coherent expression. The hallmarks effectively summarize most of the relevant information of the original founder sets and, by reducing both variation and redundancy, provide more refined and concise inputs for gene set enrichment analysis".  We'll use the mouse Hallmark file provided in Zenodo, originally downloaded from http://bioinf.wehi.edu.au/software/MSigDB/mouse_H_v5p2.rdata.

There are several ways we could choose to rank our genes, we could rank by log-fold change (most upregulated to most downregulated) but that doesn't take into account any error in the log fold change value. Another way is to use the "signed fold change" which is to rank by the sign of the fold change multiplied by the P value (as described [here](http://genomespot.blogspot.com/2014/09/data-analysis-step-8-pathway-analysis.html). We could also use the t statistic that's output from limma, as that takes into account the log-fold change and it's standard error, see [here](https://support.bioconductor.org/p/6124/) for more explanation. We'll use the t statistic to rank here.

> <hands-on-title>Perform gene set enrichment with fgsea</hands-on-title>
>
> 1. **Cut columns from a table** {% icon tool %} with
>    - {% icon param-text %} *"Cut columns"*: `c1,c6` (the Entrez gene ids and t-statistic)
>    - {% icon param-select %} *"Delimited by"*: `Tab`
>    - {% icon param-file %} *"From"*: the `limma-voom_basalpregnant-basallactate` file
> 2. **Sort data in ascending or descending order** {% icon tool %} with
>    - {% icon param-file %} *"Sort Query"*: the output of **Cut** {% icon tool %}
>    - {% icon param-text %} *"Number of header lines"*: `1`
>    - *"Column selections"*:
>        - {% icon param-select %} *"on column"*: `Column: 2`
>        - {% icon param-select %} *"in"*: `Descending order`
>        - {% icon param-select %} *"Flavor"*: `Fast numeric sort (-n)`
> 3. **fgsea** {% icon tool %} with
>    - {% icon param-file %} *"Ranked Genes"*: the output of **Sort** {% icon tool %}
>    - {% icon param-check %} *"File has header?"*: `Yes`
>    - {% icon param-file %} *"Gene Sets"*: `mouse_hallmark_sets` (this should be `rdata` format, if not, see how to change it in the Tip below)
>    - {% icon param-text %} *"Minimum Size of Gene Set"*: `15`
>    - {% icon param-check %} *"Output plots"*: `Yes`
>
> {% snippet faqs/galaxy/datasets_change_datatype.md %}
>
{: .hands_on}

fgsea outputs a table of results containing a list of pathways with P values and enrichment scores. It can also output a summary table plot of the top pathways such as the one shown below for the `basallpregnant-basallactate` contrast below.

![fgsea Table](../../images/rna-seq-genes-to-pathways/fgsea_table_basal.png "fgsea Summary table")

An enrichment plot of the each of the top pathways can also be produced, one is shown below. The barcode pattern shows where the genes in the set are found in the list of ranked genes. Most of the bars to the left indicate enrichment of the set at the top of the ranked list of genes (upregulated) and most bars towards the right indicate enrichment at the bottom of the list (downregulated). The enrichment score reflects the degree to which the genes are enriched at the top or bottom of the list.

![fgsea Enrichment](../../images/rna-seq-genes-to-pathways/fgsea_enrichplot.png "fgsea Enrichment plot")

> <question-title></question-title>
> Can you run fgsea on the luminal contrast and generate the fgsea summary table plot?
>
> > <solution-title></solution-title>
> >
> > Running fgsea on the luminal file should give a plot similar to below.
> > ![fgsea Luminal](../../images/rna-seq-genes-to-pathways/fgsea_table_luminal.png "fgsea Summary table for luminal")
> >
> {: .solution}
{: .question}

## Ensemble gene set enrichment analyses with **EGSEA**

The ensemble of genes set enrichment analyses (EGSEA) [(Alhamdoosh et al, 2017)](https://www.ncbi.nlm.nih.gov/pubmed/27694195) is a method developed for RNA-seq data that combines results from multiple algorithms and calculates collective gene set scores, to try to improve the biological relevance of the highest ranked gene sets. EGSEA has built-in gene sets from MSigDB and KEGG for human and mouse. We'll show here how it can be used with the MSigDB Hallmark collection and KEGG pathways. For input we need a count matrix and EGSEA will perform a limma-voom analysis before gene set testing. We can use the provided filtered counts file output from limma, where the low count genes have been filtered out (output from limma by selecting *"Output Filtered Counts Table?"*: `Yes`). We just need to remove the gene symbol and description columns. We also need a symbols mapping file containing just the Entrez ids and symbols, which we can generate from the filtered counts file. The third input we need is a factors information file, containing what groups the samples belong to, we can use the same one from the tutorial [RNA-seq counts to genes]({{ site.baseurl }}/topics/transcriptomics/tutorials/rna-seq-counts-to-genes/tutorial.html). EGSEA provides twelve base methods and we will select eleven, all except roast, as the fry method is a fast approximation of roast.

> <hands-on-title>Perform ensemble gene set testing with EGSEA</hands-on-title>
>
> 1. **Cut columns from a table (cut)** {% icon tool %} with the following parameters:
>      - {% icon param-file %} *"File to cut"*: `limma-voom filtered counts`
>      - {% icon param-select %} *"Operation"*: `Discard`
>      - {% icon param-select %} *"List of fields"*: Select `Column:2`, `Column:3`
>      - Rename to `EGSEA counts`
> 2. **Cut columns from a table (cut)** {% icon tool %} with the following parameters:
>      - {% icon param-file %} *"File to cut"*: `limma-voom filtered counts`
>      - {% icon param-select %} *"Operation"*: `Keep`
>      - {% icon param-select %} *"List of fields"*: Select `Column:1`, `Column:2`
>      - Rename to `EGSEA anno`
> 3. **EGSEA** {% icon tool %} with the following parameters:
>      - {% icon param-select %} *"Count Files or Matrix?*": `Single Count Matrix`
>          - {% icon param-file %} *"Count Matrix"*: Select `EGSEA counts`
>      - {% icon param-check %} *"Input factor information from file?"*: `Yes`
>          - {% icon param-file %} *"Factor File"*: Select `factordata`
>      - {% icon param-file %} *"Symbols Mapping file"*: `EGSEA anno`
>      - {% icon param-text %} *"Contrast of Interest"*: `basalpregnant-basallactate`
>      - {% icon param-select %} *"Species"*: `mouse`
>      - {% icon param-check %} *"Gene Set Testing Methods"*: Tick `camera`, `safe`, `gage`, `zscore`, `gsva`, `globaltest`, `ora`, `ssgsea`, `padog`, `plage`, `fry`
>      - {% icon param-check %} *"MSigDB Gene Set Collections"*: `H: hallmark gene sets`
>      - {% icon param-check %} *"KEGG Pathways"*: `Metabolism` and `Signalling`
>      - {% icon param-check %} *"Download KEGG pathways?"*: `Yes`
>      - {% icon param-check %} *"I certify that I am not using this tool for commercial purposes"*: `Yes`
{: .hands_on}

This generates a table of results, and a report. Contained within the report are plots, such as heatmaps of the top ranked pathways, as shown below. Note that we see some similar pathways in the results here as with the fgsea analysis.

![EGSEA heatmaps](../../images/rna-seq-genes-to-pathways/EGSEA_heatmaps.png "EGSEA heatmaps")

KEGG pathway diagrams are generated if KEGG pathways are selected, as shown below. These show the expression values of the genes overlaid, genes upregulated in the contrast are shown in red, downregulated genes in blue. Ribosome was one of the top GO terms identified for the basal pregnant vs lactate contrast and here we see ribosome pathways are in the top ranked KEGG pathways.

![EGSEA KEGG](../../images/rna-seq-genes-to-pathways/EGSEA_KEGG.png "EGSEA KEGG pathways")

> <question-title></question-title>
> Can you run EGSEA on the luminal contrast identify the top KEGG gene sets? Tip: you could use the rerun button and replace the "Contrast of Interest" with the name of the luminal contrast.
>
> > <solution-title></solution-title>
> >
> > Running EGSEA on the luminal file should give top KEGG gene sets similar to below.
> > ![EGSEA KEGG luminal](../../images/rna-seq-genes-to-pathways/EGSEA_KEGG_luminal.png "EGSEA KEGG pathways for luminal")
> >
> {: .solution}
{: .question}

# Conclusion


In this tutorial we have seen some gene set testing methods that can be used to interpret lists of differentially expressed genes. Multiple methods can be used to help identify pathways of interest and to provide complementary ways of visualising results. This follows on from the accompanying tutorials, [RNA-seq reads to counts]({% link topics/transcriptomics/tutorials/rna-seq-reads-to-counts/tutorial.md %}) and [RNA-seq counts to genes]({% link topics/transcriptomics/tutorials/rna-seq-counts-to-genes/tutorial.md %}), that showed how to turn reads (FASTQs) into differentially expressed genes for this dataset. For further reading on analysis of RNA-seq count data and the methods used here, see the articles; RNA-seq analysis is easy as 1-2-3 with limma, Glimma and edgeR [(Law et al. 2016)](https://f1000research.com/articles/5-1408/v2) and From reads to genes to pathways: differential expression analysis of RNA-Seq experiments using Rsubread and the edgeR quasi-likelihood pipeline [(Chen, Lun, Smyth 2016)](https://f1000research.com/articles/5-1438/v2).


topic: transcriptomics
tutorial: rna-seq-viz-with-cummerbund
title: Visualization of RNA-Seq results with CummeRbund
difficulty: intermediate
time_estimation: 1h
tools_json: []
requirements_json: []
tags_json: []
content_hash: d54d892c6929d0f7c15c5a8604ff099b
last_modified: 2026-05-15T22:28:17.654441
zenodo_link: https://zenodo.org/record/1001880
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/rna-seq-viz-with-cummerbund/tutorial.html

content:
RNA-Seq analysis helps researchers annotate new genes and splice variants, and provides cell- and context-specific quantification of gene expression. RNA-Seq data, however, are complex and require both computer science and mathematical knowledge to be managed and interpreted.

Visualization techniques are key to overcome the complexity of RNA-Seq data, and represent valuable tools to gather information and insights.

In this tutorial we will visualize RNA-seq data from the [CuffDiff](http://cole-trapnell-lab.github.io/cufflinks/cuffdiff/) tool.

> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Reasons for visualizing RNA-Seq results

To make sense of the available RNA-Seq data, and overview the condition-specific gene expression levels of the provided samples, we need to visualize our results. Here we will use [CummeRbund](http://compbio.mit.edu/cummeRbund/).

CummeRbund is an open-source tool that simplifies the analysis of a CuffDiff RNA-Seq output. In particular, it helps researchers with:
- managing, integrating, and visualizing the data produced by CuffDiff
- simplifying data exploration
- providing a bird's-eye view of the expression analysis by describing relationships betweeen genes, transcripts, transcription start sites, and protein-coding regions
- exploring subfeatures of individual genes or gene-sets
- creating publication-ready plots

A typical workflow for the visualization of RNA-Seq data involving CummeRbund:

![workflow](../../images/transcriptomics_images/cummerbund-rna-seq-viz-with-cummerbund.png)

CummeRbund reads your RNA-Seq results from a [SQLite](https://www.sqlite.org/) database. This database has to be created using CuffDiff's SQLite output option.

> <details-title>SQLite output with CuffDiff</details-title>
>
> Instruct CuffDiff to organize its output in a SQLite database to be read CummeRbund.
>
> ![SQLite output](../../images/transcriptomics_images/cummerbund-cuffdiff-set-sqlite.png)
{: .details}

# Importing RNA-Seq result data

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the [`CuffDiff SQLite`](https://zenodo.org/record/1001880/files/CuffDiff_SQLite_database.sqlite) dataset
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    Rename the dataset to "RNA-Seq SQLite result data"
>
>    {% snippet faqs/galaxy/datasets_rename.md name="RNA-Seq SQLite result data" %}
>
> By default, when data is imported via its link, Galaxy names it with its URL.
{: .hands_on}

CuffDiff's output data is organized in a SQLite database, so we need to extract it to be able to see what it looks like.

For this tutorial, we are interested in CuffDiff's tested transcripts for differential expression.

> <hands-on-title>Extract CuffDiff results</hands-on-title>
>
> 1. **Extract CuffDiff** {% icon tool %} with the following parameters
>   - *"Select tables to output"*: `Transcript differential expression testing`
>
> 2. Inspect the table
>
>    > <tip-title>Inspecting the content of a file in Galaxy</tip-title>
>    >
>    > * Click on the {% icon galaxy-eye %} (eye) icon ("View data") on the right of the file name in the history
>    > * Inspect the content of the file
>    {: .tip}
>
> Each entry is a differentially expressed gene, which is described in terms of the following attributes.
>
> - *test_id*: A unique identifier describing the transcript being tested
> - *gene_id*: The identifier of the gene being tested
> - *gene*: The name of the gene being tested
> - *locus*: The genomic coordinates of the gene transcript being tested
> - *sample_1*: Label of the 1st sample
> - *sample_2*: Label of the 2nd sample
> - *status*: The test's status:
>   - "OK" if test successful
>   - "NOTEST" if not enough alignments for testing
>   - "LOWDATA" if too complex or shallowly sequenced
>   - "HIDATA" if too many fragments in locus
>   - "FAIL" if a numerical exception prevented testing
> - *value_1*: The gene's FPKM in sample_1
> - *value_2*: The gene's FPKM in sample_2
> - *log2(fold_change)*: The log<sub>2</sub> of the fold change (sample_1/sample_2). This reports the expression difference between condition one (sample_1) and condition two (sample_2)
> - *test_stat*: The test statistic's value, used to determine the significance of the observed change in FPKM
> - *p_value*: The uncorrected p-value of the test statistic
> - *q_value*: The False-discovery-rate-adjusted p-value of the test statistic
> - *significant*: "yes" or "no", depending on whether the p_value is greater then the FDR after Benjamini-Hochberg correction for multiple-testing. This tells whether the difference between the expression levels in condition one (sample_1) and condition two (sample_2) is significant
>
> We want to keep only the *significant differentially expressed* genes.
>
>    > <question-title></question-title>
>    >
>    > 1. How to retain only the significant differentially expressed genes?
>    > 2. Which column stores this information?
>    >
>    > > <solution-title></solution-title>
>    > > 1. We need to filter on the column storing the record's significance
>    > > 2. Column 14
>    > {: .solution }
>    {: .question}
{: .hands_on}

# Filtering and sorting

We now want to highlight the transcripts whose expression difference, the log<sub>2</sub>(fold_change), has been statistically assessed as both high and significant.

> <hands-on-title>Extract CuffDiff's significant differentially expressed genes</hands-on-title>
>
> 1. **Filter** {% icon tool %} with the following parameters
>   - *"Filter"*: the extracted table from the previous step
>   - *"With following condition"*: an appropriate filter over the target column (see questions below when in doubt)
>   - *"Number of header lines to skip"*: the number of rows used for the table's header
>
>    > <question-title></question-title>
>    > 1. What column stores the information of significance for each record?
>    > 2. Which conditional expression has to be set to filter all records on the selected column?
>    > 3. How many rows are allocated for the table's header?
>    > 4. How many entries were originally stored in the table? And how many after the filtering operation?
>    >
>    > > <solution-title></solution-title>
>    > > 1. Column 14
>    > > 2. c14=='yes'
>    > > 3. 1
>    > > 4. ~140,000 (before filtering) vs. 219 (after filtering)
>    > {: .solution }
>    {: .question}
>
>  Review the meaning of each column, and look at your data.
>
>  - The differential expression values are stored in column 10
>  - The statistical score, assessing the differential expression significance, is stored in column 13
>  We will sort all records on the basis of their Q-score (column 13) and log<sub>2</sub>(fold_change).
>
> 2. **Sort** {% icon tool %}: with the following parameters
>   - *"Sort Dataset"*: the filtered table
>   - *"on column"*: `13`
>   - *"with flavor"*: `Numerical sort`
>   - *"everything in"*: `Ascending order`
>   - {% icon param-repeat %} **Insert Column selection**, and parameterize the Sort tool to sort on column 10. Be careful of the sorting order!
>   - Are there any rows allocated for the table's header? In that case, set "Number of header lines to skip" accordingly!
>
>    > <question-title></question-title>
>    > 1. Which gene transcript has been statistically assessed as both high and significant?
>    >
>    > > <solution-title></solution-title>
>    > > 1 LIMCH1
>    > {: .solution }
>    {: .question}
{: .hands_on}

# CummeRbund

CummeRbund generates two outputs:
- The plot, which visualizes our RNA-Seq results of interest
- The [ggplot](http://had.co.nz/ggplot2/) object responsible for generating the plot

In this section we will parametrize CummeRbund to create different kinds of plots from our input data.

> <hands-on-title>Visualization</hands-on-title>
>
> 1. **CummeRbund** {% icon tool %} with the following parameters
>    - {% icon param-repeat %}  **Insert plots**
>      - *"Width"*: `800`
>      - *"Height"*: `600`
>      - *"Plot type"*: `Expression Plot`
>        - *"Expression levels to plot"*:`Isoforms`
>        - *"Gene ID"*: `NDUFV1`
{: .hands_on}

The input data used to create the visualization comprise 3 conditions: *hits7* (Patient 1), *hits8* (Patient 2), and *hits9* (Control).

Our first CummeRbund plot is the "Expression Plot" of the isoforms of gene NDUFV1, which shows the expression differences of isoforms NM_001166102 and NM_007103 among the three conditions. Error bars capture the variability of the distribution of FPKM values: the broader the distribution of FPKM values, the larger the corresponding error bar.

![Expression plot](../../images/transcriptomics_images/cummerbund-expression-plot.png)

Our plot has a modest number of isoforms, and is therefore easy to read. However, with a high number of isoforms and expression variability among different conditions, the plot can look very busy. We can therefore change the visualization type by selecting another type of plot. Let's change visualization.

> <hands-on-title>Visualization</hands-on-title>
>
> 1. **CummeRbund** {% icon tool %} with the following parameters
>    - {% icon param-repeat %} Click on *"Insert plots"*
>      - *"Width"*: `800`
>      - *"Height"*: `600`
>      - *"Plot type"*: `Expression Bar Plot`
>        - *"Expression levels to plot"*:`Isoforms`
>        - *"Gene ID"*: `NDUFV1`
{: .hands_on}

![Expression bar plot](../../images/transcriptomics_images/cummerbund-expression-bar-plot.png)

The Expression Bar Plot of gene NDUFV1's replicates NM_001166102 and NM_007103, shows the expression changes across the three aforementioned conditions.

> <comment-title></comment-title>
>
> These plots are shown also in [this](https://vimeo.com/channels/884356/128265982) Galaxy video tutorial.
{: .comment}

Let's now create a heatmap to plot the expression levels of the significant differentially expressed gene isoforms obtained from our filter and sort operations.
As a showcase example, let's consider only the top 5 differentially expressed genes.

> <hands-on-title>Visualization</hands-on-title>
>
> 1. **CummeRbund** {% icon tool %} with the following parameters
>    - {% icon param-repeat %}  **Insert plots**
>      - *"Width"*: `800`
>      - *"Height"*: `600`
>      - *"Plot type"*: `Heatmap`
>        - *"Expression levels to plot"*: `Isoforms`
>        - *"Gene ID"*: `LIMCH1`
>        - *"Gene ID"*: `IFNL2`
>        - {% icon param-repeat %} **Insert Genes**
>          - *"Gene ID"*: `CXCL11`
>        - {% icon param-repeat %} **Insert Genes**
>          - *"Gene ID"*: `NUB1`
>      - *"Cluster by"*: `Both`
>
{: .hands_on}

![Expression bar plot](../../images/transcriptomics_images/cummerbund-heatmap.png)

Heatmap of significant differentially expressed isoforms of genes LIMCH1, IFNL2, CXCL11, NUB1. Differences in up- and down-regulation of certain gene isoforms across Patient 1 (hits7), Patient 2 (hits8), and Control (hits9), are visible in the lower and upper part of the heatmap.

> <comment-title></comment-title>
>
> For more sophisticated visualizations of your RNA-Seq analysis results, try selecting different CummeRbund plot options and parametrizations. Have a look also at CummeRbund's [manual](http://compbio.mit.edu/cummeRbund/manual_2_0.html). Alternatively, you can modify a plot's style by changing CummeRbund's R output! CummeRbund's R outputs are *ggplot* objects. Look at the [ggplot2 documentation](https://github.com/tidyverse/ggplot2) to learn how to change fonts, colors, error bars, and more.
{: .comment}

# Conclusion


Visualization tools help researchers making sense of data, providing a bird's-eye view of the underlying analysis results.
In this tutorial we overviewed the advantages of visualizing RNA-Seq results with CummeRbund, and gained insights on CuffDiff's big-data output by plotting information relative to the most significant differentially expressed genes in our RNA-Seq analysis.


topic: transcriptomics
tutorial: ref-based
title: Reference-based RNA-Seq data analysis
difficulty: introductory
time_estimation: 8h
requirements_json: []
tags_json: []
content_hash: ab299d0093ef5b8cfb96bba5da2bc218
last_modified: 2026-05-27T09:01:49.978491
zenodo_link: https://zenodo.org/record/6457007
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/ref-based/tutorial.html

content:
In recent years, RNA sequencing (in short RNA-Seq) has become a very widely used technology to analyze the continuously changing cellular transcriptome, i.e. the set of all RNA molecules in one cell or a population of cells. One of the most common aims of RNA-Seq is the profiling of gene expression by identifying genes or molecular pathways that are differentially expressed (DE) between two or more biological conditions. This tutorial demonstrates a computational workflow for the detection of DE genes and pathways from RNA-Seq data by providing a complete analysis of an RNA-Seq experiment profiling *Drosophila* cells after the depletion of a regulatory gene.

In the study of {% cite brooks2011conservation %}, the authors identified genes and pathways regulated by the *Pasilla* gene (the *Drosophila* homologue of the mammalian splicing regulators Nova-1 and Nova-2 proteins) using RNA-Seq data. They depleted the *Pasilla* (*PS*) gene in *Drosophila melanogaster* by RNA interference (RNAi). Total RNA was then isolated and used to prepare both single-end and paired-end RNA-Seq libraries for treated (PS depleted) and untreated samples. These libraries were sequenced to obtain RNA-Seq reads for each sample. The RNA-Seq data for the treated and the untreated samples can be compared to identify the effects of *Pasilla* gene depletion on gene expression.

In this tutorial, we illustrate the analysis of the gene expression data step by step using 7 of the original datasets:

- 4 untreated samples: [GSM461176](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM461176), [GSM461177](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM461177), [GSM461178](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM461178), [GSM461182](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM461182)
- 3 treated samples (*Pasilla* gene depleted by RNAi): [GSM461179](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM461179), [GSM461180](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM461180), [GSM461181](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM461181)

Each sample constitutes a separate biological replicate of the corresponding condition (treated or untreated). Moreover, two of the treated and two of the untreated samples are from a paired-end sequencing assay, while the remaining samples are from a single-end sequencing experiment.

> <comment-title>Full data</comment-title>
>
> The original data are available at NCBI Gene Expression Omnibus (GEO) under accession number [GSE18508](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE18508). The raw RNA-Seq reads have been extracted from the Sequence Read Archive (SRA) files and converted into FASTQ files.
{: .comment}
>
> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Data upload

In the first part of this tutorial we will use the files for 2 out of the 7 samples to demonstrate how to calculate read counts (a measure of the gene expression) from FASTQ files (quality control, mapping, read counting). We provide the FASTQ files for the other 5 samples if you want to reproduce the whole analysis later.

In the second part of the tutorial, read counts of all 7 samples are used to identify and visualize the DE genes, gene families and molecular pathways due to the depletion of the *PS* gene.

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this RNA-Seq exercise
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the FASTQ file pairs from [Zenodo]({{ page.zenodo_link }}) or a data library:
>    - `GSM461177` (untreated): `GSM461177_1` and `GSM461177_2`
>    - `GSM461180` (treated): `GSM461180_1` and `GSM461180_2`
>
>    ```text
>    {{ page.zenodo_link }}/files/GSM461177_1.fastqsanger
>    {{ page.zenodo_link }}/files/GSM461177_2.fastqsanger
>    {{ page.zenodo_link }}/files/GSM461180_1.fastqsanger
>    {{ page.zenodo_link }}/files/GSM461180_2.fastqsanger
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
>    > <comment-title></comment-title>
>    >
>    > Note that these are the full files for the samples and ~1.5Gb each so it may take some minutes to import.
>    >
>    > For a quicker run-through of the FASTQ steps a small subset of each FASTQ file (~5Mb) can be found here on [Zenodo]({{ page.zenodo_link }}):
>    >
>    > ```text
>    > {{ page.zenodo_link }}/files/GSM461177_1_subsampled.fastqsanger
>    > {{ page.zenodo_link }}/files/GSM461177_2_subsampled.fastqsanger
>    > {{ page.zenodo_link }}/files/GSM461180_1_subsampled.fastqsanger
>    > {{ page.zenodo_link }}/files/GSM461180_2_subsampled.fastqsanger
>    > ```
>    >
>    {: .comment}
>
> 3. Check that the datatype is `fastqsanger` (e.g. **not** `fastq`). If it is not, please change the datatype to `fastqsanger`.
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="fastqsanger" %}
>
> 4. Create a paired collection named `2 PE fastqs`, name your pairs with the sample name followed by the attributes: `GSM461177_untreat_paired` and `GSM461180_treat_paired`.
>
>    {% snippet faqs/galaxy/collections_build_list_paired.md %}
>
{: .hands_on}

{% include topics/sequence-analysis/tutorials/quality-control/fastq_question.md %}

The reads are raw data from the sequencing machine without any pretreatments. They need to be assessed for their quality.

# Quality control

During sequencing, errors are introduced, such as incorrect nucleotides being called. These are due to the technical limitations of each sequencing platform. Sequencing errors might bias the analysis and can lead to a misinterpretation of the data. Adapters may also be present if the reads are longer than the fragments sequenced and trimming these may improve the number of reads mapped.

Sequence quality control is therefore an essential first step in your analysis. We will use similar tools as described in the ["Quality control" tutorial]({% link topics/sequence-analysis/tutorials/quality-control/tutorial.md %}):
- [Falco](https://falco.readthedocs.io/en/latest/), which is an efficiency-optimized rewrite of [FastQC](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/), to create a report of sequence quality
- [MultiQC](https://multiqc.info/) ({% cite ewels2016multiqc %}) to aggregate generated reports
- [Cutadapt](https://cutadapt.readthedocs.io/en/stable/guide.html) ({% cite marcel2011cutadapt %}) to improve the quality of sequences via trimming and filtering.

Unfortunately the current version of MultiQC (the tool we use to combine reports) does not support list of pairs collections.
We will first need to transform our list of pairs to a simple list.

> <details-title>What does this mean exactly?</details-title>
>
> The current situation is on top and the **Flatten collection** tool will transform it to the situation displayed on bottom:
> ![Flatten](../../images/ref-based/flatten.png "Flatten the list of pairs to list")
{: .details}

> <hands-on-title>Quality control</hands-on-title>
>
> 1. {% tool [Flatten collection](__FLATTEN__) %} with the following parameters convert the list of pairs into a simple list:
>     - *"Input Collection"*: `2 PE fastqs`
>
> 2. {% tool [Falco](toolshed.g2.bx.psu.edu/repos/iuc/falco/falco/1.2.4+galaxy0) %} with the following parameters:
>    - {% icon param-collection %} *"Raw read data from your current history"*: Output of **Flatten collection** {% icon tool %} selected as **Dataset collection**
>
>    {% snippet faqs/galaxy/tools_select_collection.md %}
>
> 3. Inspect the webpage output of **Falco** {% icon tool %} for the `GSM461177_untreat_paired` sample (forward and reverse)
>
>    > <question-title></question-title>
>    >
>    > What is the read length?
>    >
>    > > <solution-title></solution-title>
>    > >
>    > > The read length of both mates is 37 bp.
>    > >
>    > {: .solution}
>    >
>    {: .question}
>
>    As it is tedious to inspect all these reports individually we will combine them with {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.27+galaxy4) %}.
>
> 4. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.27+galaxy4) %} to aggregate the Falco reports with the following parameters:
>    - In *"Results"*:
>        - *"Results"*
>            - *"Which tool was used generate logs?"*: `FastQC` (**Falco** is a drop-in replacement for *FastQC* and we can pass its output to MultiQC as if it had been generated by the original tool.)
>                - In *"FastQC output"*:
>                    - {% icon param-repeat %} *"Insert FastQC output"*
>                        - {% icon param-collection %} *"FastQC output"*: `Falco on collection N: RawData` (output of **Falco** {% icon tool %})
>
> 5. Inspect the webpage output from MultiQC for each FASTQ
>
>    > <question-title></question-title>
>    >
>    > 1. What do you think of the quality of the sequences?
>    > 2. What should we do?
>    >
>    > > <solution-title></solution-title>
>    > >
>    > > 1. Everything seems good for 3 of the files. The `GSM461177_untreat_paired` have 10.6 millions of paired sequences and `GSM461180_treat_paired` 12.3 millions of paired sequences. But in `GSM461180_treat_paired_reverse` (reverse reads of GSM461180) the quality decreases quite a lot at the end of the sequences.
>    > >
>    > >    All files except `GSM461180_treat_paired_reverse` have a high proportion of duplicated reads (expected in RNA-Seq data).
>    > >
>    > >    ![Sequence Counts](../../images/ref-based/fastqc_sequence_counts_plot.png "Sequence Counts")
>    > >
>    > >    The "Per base sequence quality" is globally good with a slight decrease at the end of the sequences. For `GSM461180_treat_paired_reverse`, the decrease is quite large.
>    > >
>    > >    ![Sequence Quality](../../images/ref-based/fastqc_per_base_sequence_quality_plot.png "Sequence Quality")
>    > >
>    > >    The mean quality score over the reads is quite high, but the distribution is slightly different for `GSM461180_treat_paired_reverse`.
>    > >
>    > >    ![Per Sequence Quality Scores](../../images/ref-based/fastqc_per_sequence_quality_scores_plot.png "Per Sequence Quality Scores")
>    > >
>    > >    Reads do not really follow a normal distribution of GC content, except for `GSM461180_treat_paired_reverse`.
>    > >
>    > >    ![Per Sequence GC Content](../../images/ref-based/fastqc_per_sequence_gc_content_plot.png "Per Sequence GC Content")
>    > >
>    > >    The proportion of N in the reads (bases which could not be called) is low.
>    > >
>    > >    ![Per base N content](../../images/ref-based/fastqc_per_base_n_content_plot.png "Per base N content")
>    > >
>    > >    Duplicated sequences: >10 to >500
>    > >
>    > >    ![Sequence Duplication Levels](../../images/ref-based/fastqc_sequence_duplication_levels_plot.png "Sequence Duplication Levels")
>    > >
>    > >    There are almost no known adapters and overrepresented sequences.
>    > >
>    > > 2. If the quality of the reads is poor, we should:
>    > >    1. Check what is wrong and think about possible reasons for the poor read quality: it may come from the type of sequencing or what we sequenced (high quantity of overrepresented sequences in transcriptomics data, biased percentage of bases in Hi-C data)
>    > >    2. Ask the sequencing facility about it
>    > >    3. Perform some quality treatment (taking care not to lose too much information) with some trimming or removal of bad reads
>    > >
>    > {: .solution}
>    {: .question}
>
{: .hands_on}

We should trim the reads to get rid of bases that were sequenced with high uncertainty (i.e. low quality bases) at the read ends, and also remove the reads of overall bad quality.

{% include topics/sequence-analysis/tutorials/quality-control/paired_end_question.md forward="GSM461177_untreat_paired_forward" reverse="GSM461177_untreat_paired_reverse" %}

> <hands-on-title>Trimming FASTQs</hands-on-title>
>
> 1. {% tool [Cutadapt](toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt/5.2+galaxy0) %} with the following parameters to trim low quality sequences:
>    - *"Single-end or Paired-end reads?"*: `Paired-end Collection`
>       - {% icon param-collection %} *"Paired Collection"*: `2 PE fastqs`
>    - In *"Other Read Trimming Options"*
>       - *"Quality cutoff(s) (R1)"*: `20`
>    - In *"Read Filtering Options"*
>       - *"Minimum length (R1)"*: `20`
>    - In *"Additional outputs to generate"*
>       - Select: `Report: Cutadapt's per-adapter statistics. You can use this file with MultiQC.`
>
>      {% include topics/sequence-analysis/tutorials/quality-control/trimming_question.md %}
>
> 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.27+galaxy4) %} to aggregate the Cutadapt reports with the following parameters:
>    - In *"Results"*:
>        - {% icon param-repeat %} *"Results"*
>            - *"Which tool was used generate logs?"*: `Cutadapt/Trim Galore!`
>               - {% icon param-collection %} *"Output of Cutadapt"*: `Cutadapt on collection N: Report` (output of **Cutadapt** {% icon tool %}) selected as **Dataset collection**
>
>    > <question-title></question-title>
>    >
>    > 1. How many sequence pairs have been removed because at least one read was shorter than the length cutoff?
>    > 2. How many basepairs have been removed from the forward reads because of bad quality? And from the reverse reads?
>    >
>    > > <solution-title></solution-title>
>    > >
>    > > 1. 147,810 (1.4%) reads were too short for `GSM461177_untreat_paired` and 1,101,875 (9%) for `GSM461180_treat_paired`.
>    > >    ![Cutadapt Filtered reads](../../images/ref-based/cutadapt_filtered_reads_plot.png "Cutadapt Filtered reads")
>    > > 2. The MultiQC output only provides the proportion of bp trimmed in total, not for each Read. To get this information, you need to go back to the individual reports. For `GSM461177_untreat_paired`, 5,072,810 bp have been trimmed from the forward reads (Read 1) and 8,648,619 bp from the reverse reads (Read 2) because of quality. For `GSM461180_treat_paired`, 10,224,537 bp from the forward reads and 51,746,850 bp from the reverse reads. This is not a surprise; we saw that at the end of the reads the quality was dropping more for the reverse reads than for the forward reads, especially for `GSM461180_treat_paired`.
>    > {: .solution }
>    {: .question}
{: .hands_on}

# Mapping

To make sense of the reads, we need to first figure out where the sequences originated from in the genome, so we can then determine to which genes they belong. When a reference genome for the organism is available, this process is known as aligning or "mapping" the reads to the reference. This is equivalent to solving a jigsaw puzzle, but unfortunately, not all pieces are unique.

> <comment-title></comment-title>
>
> Do you want to learn more about the principles behind mapping? Follow our [training]({% link topics/sequence-analysis/tutorials/mapping/tutorial.md %}).
{: .comment}

In this study, the authors used *Drosophila melanogaster* cells. We should therefore map the quality-controlled sequences to the reference genome of *Drosophila melanogaster*.

{% include topics/sequence-analysis/tutorials/mapping/ref_genome_explanation.md answer_3="The genome of *Drosophila melanogaster* is known and assembled and it can be used as the reference genome in this analysis. Note that new versions of reference genomes may be released if the assembly improves, for this tutorial we are going to use the release 6 of the *Drosophila melanogaster* reference genome assembly [(dm6)](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4383921/)."%}

With eukaryotic transcriptomes most reads originate from processed mRNAs lacking introns:

![Types of RNA-Seq reads](../../images/ref-based/rna-seq-reads.png "The types of RNA-seq reads (adaption of the Figure 1a from {% cite kim2015hisat %}): reads that mapped entirely within an exon (in red), reads spanning over 2 exons (in blue), read spanning over more than 2 exons (in purple)")

Therefore they cannot be simply mapped back to the genome as we normally do for DNA data. Spliced-awared mappers have been developed to efficiently map transcript-derived reads against a reference genome:

![Splice-aware alignment](../../images/transcriptomics_images/splice_aware_alignment.png "Principle of spliced mappers: (1) identification of the reads spanning a single exon, (2) identification of the splicing junctions on the unmapped reads")

> <details-title>More details on the different spliced mappers</details-title>
>
> Several spliced mappers have been developed over the past years to process the explosion of RNA-Seq data.
>
> [**TopHat**](https://ccb.jhu.edu/software/tophat/index.shtml) ({% cite trapnell2009tophat %}) was one of the first tools designed specifically to address this problem. In **TopHat** reads are mapped against the genome and are separated into two categories: (1) those that map, and (2) those that are initially unmapped (IUM). "Piles" of reads representing potential exons are extended in search of potential donor/acceptor splice sites and potential splice junctions are reconstructed. IUMs are then mapped to these junctions.
>
> ![TopHat](../../images/transcriptomics_images/tophat.png "TopHat (Figure 1 from {% cite trapnell2009tophat %})")
>
> **TopHat** has been subsequently improved with the development of **TopHat2** ({% cite kim2013tophat2 %}):
>
> ![TopHat2](../../images/transcriptomics_images/13059_2012_Article_3053_Fig6_HTML.jpg "TopHat2 (Figure 6 from {% cite kim2013tophat2 %})")
>
> To further optimize and speed up spliced read alignment, [**HISAT2**](https://ccb.jhu.edu/software/hisat2/index.shtml) ({% cite kim2019graph %}) was developed. It uses a hierarchical graph [FM](https://en.wikipedia.org/wiki/FM-index) (HGFM) index, representing the entire genome and eventual variants, together with overlapping local indexes (each spanning ~57 kb) that collectively cover the genome and its variants. This allows to find initial seed locations for potential read alignments in the genome using global index and to rapidly refine these alignments using a corresponding local index:
>
> ![Hierarchical Graph FM index in HISAT/HISAT2](../../images/transcriptomics_images/hisat.png "Hierarchical Graph FM index in HISAT/HISAT2 (Figure S8 from {% cite kim2015hisat %})")
>
> A part of the read (blue arrow) is first mapped to the genome using the global FM index. **HISAT2** then tries to extend the alignment directly utilizing the genome sequence (violet arrow). In (**a**) it succeeds and this read is aligned as it completely resides within an exon. In (**b**) the extension hits a mismatch. Now **HISAT2** takes advantage of the local FM index overlapping this location to find the appropriate mapping for the remainder of this read (green arrow). The (**c**) shows a combination these two strategies: the beginning of the read is mapped using global FM index (blue arrow), extended until it reaches the end of the exon (violet arrow), mapped using local FM index (green arrow) and extended again (violet arrow).
>
> [**STAR** aligner](https://github.com/alexdobin/STAR) ({% cite dobin2013star %}) is a fast alternative for mapping RNA-Seq reads against a reference genome utilizing an uncompressed [suffix array](https://en.wikipedia.org/wiki/Suffix_array). It operates in two stages. In the first stage it performs a seed search:
>
> ![STAR's seed search](../../images/transcriptomics_images/star.png "STAR's seed search (Figure 1 from {% cite dobin2013star %})")
>
> Here a read is split between two consecutive exons. **STAR** starts to look for a maximum mappable prefix (MMP) from the beginning of the read until it can no longer match continuously. After this point it starts to look for a MMP for the unmatched portion of the read (**a**). In the case of mismatches (**b**) and unalignable regions (**c**) MMPs serve as anchors from which to extend alignments.
>
> At the second stage **STAR** stitches MMPs to generate read-level alignments that (contrary to MMPs) can contain mismatches and indels. A scoring scheme is used to evaluate and prioritize stitching combinations and to evaluate reads that map to multiple locations. **STAR** is extremely fast but requires a substantial amount of RAM to run efficiently.
>
{: .details}

## Mapping

We will map our reads to the *Drosophila melanogaster* genome using **STAR** ({% cite dobin2013star %}).

> <hands-on-title>Spliced mapping</hands-on-title>
>
> 1. Import the Ensembl gene annotation for *Drosophila melanogaster* (`Drosophila_melanogaster.BDGP6.32.109_UCSC.gtf.gz`) from the Shared Data library if available or from [Zenodo]({{ page.zenodo_link }}/files/Drosophila_melanogaster.BDGP6.32.109_UCSC.gtf.gz) into your current Galaxy history
>
>    ```text
>    {{ page.zenodo_link }}/files/Drosophila_melanogaster.BDGP6.32.109_UCSC.gtf.gz
>    ```
>
>    1. Rename the dataset if necessary
>    2. Verify that the datatype is `gtf` (or `gtf.gz`) and not `gff`.
>
>    > <comment-title>How to get annotation file?</comment-title>
>    >
>    > Annotation files from model organisms may be available on the Shared Data library (the path to them will change from one Galaxy server to the other). You could also retrieve the annotation file from UCSC (using **UCSC Main** tool).
>    >
>    > To generate this specific file, the annotation file was downloaded from Ensembl which provides a more comprehensive database of transcripts and was further adapted to make it work with the dm6 genome which is installed on compatible Galaxy servers.
>    >
>    {: .comment}
>
> 2. {% tool [RNA STAR](toolshed.g2.bx.psu.edu/repos/iuc/rgrnastar/rna_star/2.7.11b+galaxy0) %} with the following parameters to map your reads on the reference genome:
>    - *"Single-end or paired-end reads"*: `Paired-end (as collection)`
>       - {% icon param-collection %} *"RNA-Seq FASTQ/FASTA paired reads"*: the `Cutadapt on collection N: Reads` (output of **Cutadapt** {% icon tool %})
>    - *"Custom or built-in reference genome"*: `Use a built-in index`
>       - *"Reference genome with or without an annotation"*: `use genome reference without builtin gene-model but provide a gtf`
>           - *"Select reference genome"*: `Fly (Drosophila melanogaster): dm6 Full`
>           - {% icon param-file %} *"Gene model (gff3,gtf) file for splice junctions"*: the imported `Drosophila_melanogaster.BDGP6.32.109_UCSC.gtf.gz`
>           - *"Length of the genomic sequence around annotated junctions"*: `36` (This parameter should be length of reads - 1)
>               - *"Per gene/transcript output"*: `Per gene read counts (GeneCounts)`
>    - *"Compute coverage"*:
>       - `Yes in bedgraph format`
>
> 3. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.27+galaxy4) %} to aggregate the STAR logs with the following parameters:
>    - In *"Results"*:
>        - *"Results"*
>            - *"Which tool was used generate logs?"*: `STAR`
>                - In *"STAR output"*:
>                    - {% icon param-repeat %} *"Insert STAR output"*
>                        - *"Type of STAR output?"*: `Log`
>                            - {% icon param-collection %} *"STAR log output"*: `RNA STAR on collection N: log` (output of **RNA STAR** {% icon tool %})
>
>    > <question-title></question-title>
>    >
>    > 1. What percentage of reads are mapped exactly once for both samples?
>    > 2. What are the other available statistics?
>    >
>    > > <solution-title></solution-title>
>    > >
>    > > 1. More than 83% for `GSM461177_untreat_paired` and 79% for `GSM461180_treat_paired`. We can proceed with the analysis since only percentages below 70% should be investigated for potential contamination.
>    > > 2. We also have access to the number and percentage of reads that are mapped at several location, mapped at too many different location, not mapped because too short.
>    > >
>    > >    ![STAR Alignment Scores](../../images/ref-based/star_alignment_plot.png "Alignment scores")
>    > >
>    > >    We could have been probably more strict in the minimal read length to avoid these unmapped reads because of length.
>    > {: .solution}
>    >
>    {: .question}
{: .hands_on}

According to the **MultiQC** report, about 80% of reads for both samples are mapped exactly once to the reference genome. We can proceed with the analysis since only percentages below 70% should be investigated for potential contamination. Both samples have a low (less than 10%) percentage of reads that mapped to multiple locations on the reference genome. This is in the normal range for Illumina short-read sequencing, but may be lower for newer long-read sequencing datasets that can span larger repeated regions in the reference genome and will be higher for 3' end libraries.

The main output of **STAR** is a BAM file.

{% include topics/sequence-analysis/tutorials/mapping/bam_explanation.md mapper="RNA STAR" %}

## Inspection of the mapping results

The BAM file contains information for all our reads, making it difficult to inspect and explore in text format. A powerful tool to visualize the content of BAM files is the Integrative Genomics Viewer (**IGV**, {% cite robinson2011integrative %}). Alternatively or complementarily, you can use JBrowse2, a genome browser which is directly integrated into Galaxy.

> <hands-on-title>Inspection of mapping results with IGV</hands-on-title>
>
> 1. Install [**IGV**](https://software.broadinstitute.org/software/igv/download) (if not already installed)
> 2. Start IGV locally
> 3. Click on the collection `RNA STAR on collection N: mapped.bam` (output of **RNA STAR** {% icon tool %})
> 4. Expand the {% icon param-file %} `GSM461177_untreat_paired` file.
> 5. Click on the {% icon galaxy-barchart %} visualize icon in the `GSM461177` file block.
> 6. In the center panel click on the `local` in `display with IGV (local, D. melanogaster (dm6))`to load the reads into the IGV browser
>    > <comment-title></comment-title>
>    >
>    > In order for this step to work, you will need to have either IGV or [Java Web Start](https://www.java.com/en/download/faq/java_webstart.xml)
>    > installed on your machine. However, the questions in this section can also be answered by inspecting the IGV screenshots below.
>    >
>    > Check the [IGV documentation](https://software.broadinstitute.org/software/igv/AlignmentData) for more information.
>    >
>    {: .comment}
>
> 6. **IGV** {% icon tool %}: Zoom to `chr4:540,000-560,000` (Chromosome 4 between 540 kb to 560 kb)
>
>    > <question-title></question-title>
>    >
>    > ![Screenshot of the IGV view on Chromosome 4](../../images/transcriptomics_images/junction_igv_screenshot.png "Screenshot of IGV on Chromosome 4")
>    >
>    > 1. What information appears at the top as grey peaks?
>    > 2. What do the connecting lines between some of the aligned reads indicate?
>    >
>    > > <solution-title></solution-title>
>    > >
>    > > 1. The coverage plot: the sum of mapped reads at each position
>    > > 2. They indicate junction events (or splice sites), *i.e.* reads that are mapped across an intron
>    > >
>    > {: .solution}
>    {: .question}
>
> 7. **IGV** {% icon tool %}: Inspect the splice junctions using a **Sashimi plot**
>
>    > <comment-title>Creation of a Sashimi plot</comment-title>
>    >
>    > - Right click on the BAM file (in IGV)
>    > - Select **Sashimi Plot** from the menu
>    >
>    {: .comment}
>    >
>    > <question-title></question-title>
>    >
>    > ![Screenshot of a Sashimi plot of Chromosome 4](../../images/transcriptomics_images/star_igv_sashimi.png "Screenshot of a Sashimi plot of Chromosome 4")
>    >
>    > 1. What does the vertical red bar graph represent? What about the arcs with numbers?
>    > 2. What do the numbers on the arcs mean?
>    > 3. Why do we observe different stacked groups of blue linked boxes at the bottom?
>    >
>    > > <solution-title></solution-title>
>    > >
>    > > 1. The coverage for each alignment track is plotted as a red bar graph. Arcs represent observed splice junctions, *i.e.*, reads spanning introns.
>    > > 2. The numbers refer to the number of observed junction reads.
>    > > 3. The different groups of linked boxes on the bottom represent the different transcripts from the genes at this location which are present in the GTF file.
>    > >
>    > {: .solution}
>    {: .question}
>    >
>    > <comment-title></comment-title>
>    >
>    > Check the [IGV documentation on Sashimi plots](https://software.broadinstitute.org/software/igv/Sashimi) to find some clues
>    {: .comment}
>
{: .hands_on}


> <hands-on-title>Inspection of mapping results with JBrowse2</hands-on-title>
>
> 1. {% tool [JBrowse2](toolshed.g2.bx.psu.edu/repos/fubar/jbrowse2/jbrowse2/3.6.5+galaxy1) %} with the following parameters:
>    - *"Action"*: `New JBrowse Instance`
>    - In *"Genome Assembly"*:
>        - {% icon param-repeat %} *"Insert Genome Assembly"*
>            - *"Reference genome to display"*: `Use a built-in genome`
>                - *"Select a reference genome"*: `Fly (Drosophila melanogaster): dm6 Full`
>            - *"Default region to display"*: `chr4:540000..560000`
>            - In *"Track Category"*:
>                - {% icon param-repeat %} *"Insert Track Category"*
>                    - *"Track Category Label"*: `BAM`
>                    - In *"Track"*:
>                        - {% icon param-repeat %} *"Insert Track"*
>                            - *"Track Type"*: `BAM Pileups`
>                                - *"BAM Track Data"*: `Use data from history`
>                                    - {% icon param-collection %} *"BAM Track Data"*: `RNA STAR on collection N: mapped.bam` (output of **RNA STAR** {% icon tool %})
>                - {% icon param-repeat %} *"Insert Track Category"*
>                    - *"Track Category Label"*: `Genes`
>                    - In *"Track"*:
>                        - {% icon param-repeat %} *"Insert Track"*
>                            - *"Track Type"*: `GFF/GFF3/BED Features`
>                                - *"GFF/GFF3/BED Track Data"*: `Use data from history`
>                                   - {% icon param-file %} *"GFF/GFF3/BED Track Data"*: `Drosophila_melanogaster.BDGP6.32.109.gtf.gz`
>
> 2. Inspect the result by clicking the eye-icon.
>
>    > <question-title></question-title>
>    >
>    > ![Screenshot of the JBrowse2 view on Chromosome 4](../../images/ref-based/junction_jbrowse2_screenshot.png "Screenshot of JBrowse2 on Chromosome 4")
>    >
>    > 1. What information appears at the top as grey peaks?
>    > 2. What do the connecting lines between some of the aligned reads indicate?
>    >
>    > > <solution-title></solution-title>
>    > >
>    > > 1. The coverage plot: the sum of mapped reads at each position
>    > > 2. They indicate junction events (or splice sites), *i.e.* reads that are mapped across an intron
>    > >
>    > {: .solution}
>    {: .question}
>
{: .hands_on}

> <details-title>Further check for the quality of the data</details-title>
>
> The quality of the data and mapping can be checked further, e.g. by inspecting read duplication level, number of reads mapped to each chromosome, gene body coverage, and read distribution across features.
>
> *These steps have been inspired by the ones provided in the [great "RNA-Seq reads to counts" tutorial]({% link topics/transcriptomics/tutorials/rna-seq-reads-to-counts/tutorial.md %}) and adapted to our datasets.*
>
> #### Duplicate reads
>
> In the Falco/MultiQC report, we saw that some reads are duplicated:
>
> ![Sequence Duplication Levels](../../images/ref-based/fastqc_sequence_duplication_levels_plot.png "Sequence Duplication Levels")
>
> Duplicate reads can come from highly-expressed genes, therefore they are usually kept in RNA-Seq differential expression analysis. But a high percentage of duplicates may indicate an issue, e.g. over amplification during PCR of low complexity library.
>
> **MarkDuplicates** from [Picard suite](http://broadinstitute.github.io/picard/) examines aligned records from a BAM file to locate duplicate reads, i.e. reads mapping to the same location (based on the start position of the mapping).
>
> > <hands-on-title>Check duplicate reads</hands-on-title>
> >
> > 1. {% tool [MarkDuplicates](toolshed.g2.bx.psu.edu/repos/devteam/picard/picard_MarkDuplicates/3.1.1.0) %} with the following parameters:
> >    - {% icon param-collection %} *"Select SAM/BAM dataset or dataset collection"*: `RNA STAR on collection N: mapped.bam` (output of **RNA STAR** {% icon tool %})
> >
> > 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.27+galaxy4) %} to aggregate the MarkDuplicates logs with the following parameters:
> >    - In *"Results"*:
> >        - *"Results"*
> >            - *"Which tool was used generate logs?"*: `Picard`
> >                - In *"Picard output"*:
> >                    - {% icon param-repeat %} *"Insert Picard output"*
> >                        - *"Type of Picard output?"*: `Markdups`
> >                        - {% icon param-collection %} *"Picard output"*: `MarkDuplicates on collection N: tabular` (output of **MarkDuplicates** {% icon tool %})
> >
> >    > <question-title></question-title>
> >    >
> >    > What are the percentages of duplicate reads for each sample?
> >    >
> >    > > <solution-title></solution-title>
> >    > >
> >    > > The sample `GSM461177_untreat_paired` has 25.9% of duplicated reads while `GSM461180_treat_paired` has 27.8%.
> >    > {: .solution}
> >    {: .question}
> {: .hands_on}
>
> In general, obtaining up to 50% duplicated reads is considered normal. So both our samples are fine.
>
> #### Number of reads mapped to each chromosome
>
> To assess the sample quality (e.g. excess of mitochondrial contamination), we can check the sex of samples, or to see if any chromosomes have highly expressed genes, we can check the numbers of reads mapped to each chromosome using **IdxStats** from the **Samtools** suite.
>
> > <hands-on-title>Check the number of reads mapped to each chromosome</hands-on-title>
> >
> > 1. {% tool [Samtools idxstats](toolshed.g2.bx.psu.edu/repos/devteam/samtools_idxstats/samtools_idxstats/2.0.7) %} with the following parameters:
> >    - {% icon param-collection %} *"BAM file"*: `RNA STAR on collection N: mapped.bam` (output of **RNA STAR** {% icon tool %})
> >
> > 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.27+galaxy4) %} to aggregate the idxstats logs with the following parameters:
> >    - In *"Results"*:
> >        - *"Results"*
> >            - *"Which tool was used generate logs?"*: `Samtools`
> >                - In *"Samtools output"*:
> >                    - {% icon param-repeat %} *"Insert Samtools output"*
> >                        - *"Type of Samtools output?"*: `idxstats`
> >                            - {% icon param-collection %} *"Samtools idxstats output"*: `Samtools idxstats on collection N` (output of **Samtools idxstats** {% icon tool %})
> >
> >    > <question-title></question-title>
> >    >
> >    > ![Samtools idxstats](../../images/ref-based/samtools-idxstats-mapped-reads-plot.png)
> >    >
> >    > 1. How many chromosomes does the *Drosophila* genome have?
> >    > 2. Where did the reads mostly map?
> >    > 3. Can we determine the sex of the samples?
> >    >
> >    > > <solution-title></solution-title>
> >    > >
> >    > > 1. The genome of *Drosophila* has 4 pairs of chromosomes: X/Y, 2, 3, and 4.
> >    > > 2. The reads mapped mostly to chromosome 2 (chr2L and chr2R), 3 (chr3L and chr3R) and X. Only a few reads mapped to chromosome 4, which is expected given that this chromosome is very small.
> >    > > 3. Judging from the percentage of X+Y reads, most of the reads map to X and only a few to Y. This indicates there are probably not many genes on Y, so the samples are probably both female.
> >    > >
> >    > >    ![Samtools idxstats](../../images/ref-based/samtools-idxstats-xy-plot.png)
> >    > {: .solution}
> >    {: .question}
> {: .hands_on}
>
> #### Gene body coverage
>
> The different regions of a gene make up the gene body. It is important to check if read coverage is uniform across the gene body. For example, a bias towards the 5' end of genes could indicate degradation of the RNA. Alternatively, a 3' bias could indicate that the data is from a 3' assay. To assess this, we can use the **Gene Body Coverage** tool from the RSeQC ({% cite wang2012rseqc %}) tool suite. This tool scales all transcripts to 100 nucleotides (using a provided annotation file) and calculates the number of reads covering each (scaled) nucleotide position. As this tool is really slow, we will compute the coverage only on 200,000 random reads.
>
> > <hands-on-title>Check gene body coverage</hands-on-title>
> >
> > 1. {% tool [Samtools view](toolshed.g2.bx.psu.edu/repos/iuc/samtools_view/samtools_view/1.21+galaxy0) %} with the following parameters:
> >    - {% icon param-collection %} *"SAM/BAM/CRAM data set"*: `RNA STAR on collection N: mapped.bam` (output of **RNA STAR** {% icon tool %})
> >    - *"What would you like to look at?"*: `A filtered/subsampled selection of reads`
> >        - In *"Configure subsampling"*:
> >            - *"Subsample alignment"*: `Specify a target # of reads`
> >                - *"Target # of reads"*: `200000`
> >                - *"Seed for random number generator"*: `1`
> >        - *"What would you like to have reported?"*: `All reads retained after filtering and subsampling`
> >            - *"Output format"*: `BAM (-b)`
> >    - *"Use a reference sequence"*: `No`
> >
> > 2. {% tool [Convert GTF to BED12](toolshed.g2.bx.psu.edu/repos/iuc/gtftobed12/gtftobed12/357) %} to convert the GTF file to BED:
> >    - {% icon param-file %} *"GTF File to convert"*: `Drosophila_melanogaster.BDGP6.32.109.gtf.gz`
> >
> > 3. {% tool [Gene Body Coverage (BAM)](toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_geneBody_coverage/5.0.3+galaxy0) %} with the following parameters:
> >    - *"Run each sample separately, or combine mutiple samples into one plot"*: `Run each sample separately`
> >        - {% icon param-collection %} *"Input .bam file"*: output of **Samtools view** {% icon tool %}
> >    - {% icon param-file %} *"Reference gene model"*: `Convert GTF to BED12 on data N: BED12` (output of **Convert GTF to BED12** {% icon tool %})
> >
> > 4. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.27+galaxy4) %} to aggregate the RSeQC results with the following parameters:
> >    - In *"Results"*:
> >        - *"Results"*
> >            - *"Which tool was used generate logs?"*: `RSeQC`
> >                - In *"RSeQC output"*:
> >                    - {% icon param-repeat %} *"Insert RSeQC output"*
> >                        - *"Type of RSeQC output?"*: `gene_body_coverage`
> >                            - {% icon param-collection %} *"RSeQC gene_body_coverage output"*: `Gene Body Coverage (BAM) on collection N: stats (TXT)` (output of **Gene Body Coverage (BAM)** {% icon tool %})
> >
> >    > <question-title></question-title>
> >    >
> >    > ![Gene body coverage](../../images/ref-based/rseqc_gene_body_coverage_plot.png)
> >    >
> >    > How are the coverage across gene bodies? Are the samples biased in 3' or 5'?
> >    >
> >    > > <solution-title></solution-title>
> >    > >
> >    > > For both samples there is a pretty even coverage from 5' to 3' ends (despite some noise in the middle). So no obvious bias in both samples.
> >    > {: .solution}
> >    {: .question}
> {: .hands_on}
>
> #### Read distribution across features
>
> With RNA-Seq data, we expect most reads to map to exons rather than introns or intergenic regions. Before going further in counting and differential expression analysis, it may be interesting to check the distribution of reads across known gene features (exons, CDS, 5' UTR, 3' UTR, introns, intergenic regions). For example, a high number of reads mapping to intergenic regions may indicate the presence of DNA contamination.
>
> Here we will use the **Read Distribution** tool from the RSeQC ({% cite wang2012rseqc %}) tool suite, which uses the annotation file to identify the position of the different gene features.
>
> > <hands-on-title>Check the number of reads mapped to each chromosome</hands-on-title>
> >
> > 1. {% tool [Read Distribution](toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_read_distribution/5.0.3+galaxy0) %} with the following parameters:
> >    - {% icon param-collection %} *"Input .bam/.sam file"*: `RNA STAR on collection N: mapped.bam` (output of **RNA STAR** {% icon tool %})
> >    - {% icon param-file %} *"Reference gene model"*: BED12 file (output of **Convert GTF to BED12** {% icon tool %})
> >
> > 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.27+galaxy4) %} to aggregate the Read Distribution results with the following parameters:
> >    - In *"Results"*:
> >        - *"Results"*
> >            - *"Which tool was used generate logs?"*: `RSeQC`
> >                - In *"RSeQC output"*:
> >                    - {% icon param-repeat %} *"Insert RSeQC output"*
> >                        - *"Type of RSeQC output?"*: `read_distribution`
> >                            - {% icon param-collection %} *"RSeQC read_distribution output"*: `Read Distribution on collection N: stats (TXT)` (output of **Read Distribution** {% icon tool %})
> >
> >    > <question-title></question-title>
> >    >
> >    > ![Read Distribution](../../images/ref-based/rseqc_read_distribution_plot.png)
> >    >
> >    > What do you think of the read distribution?
> >    >
> >    > > <solution-title></solution-title>
> >    > >
> >    > > Most of the reads are mapped to exons (>80%), only ~2% to introns and ~5% to intergenic regions, which is what we expect. It confirms that our data are RNA-Seq data and that mapping was successful.
> >    > {: .solution}
> >    {: .question}
> {: .hands_on}
>
> Now that we have checked the results of the read mapping, we can proceed to the next phase of the analysis.
{: .details}

After the mapping, we now have the information on where the reads are located on the reference genome and how well they were mapped. The next step in RNA-Seq data analysis is quantification of the number of reads mapped to genomic features (genes, transcripts, exons, ...).

> <comment-title></comment-title>
>
> The quantification depends on both the reference genome (the FASTA file) and its associated annotations (the GTF file). It is extremely important to use an annotation file that corresponds to the same version of the reference genome you used for the mapping (e.g. `dm6` here), as the chromosomal coordinates of genes are usually different amongst different reference genome versions.
{: .comment}

Here we will focus on the genes, as we would like to identify the ones that are differentially expressed because of the Pasilla gene knockdown.

# Counting the number of reads per annotated gene

To compare the expression of single genes between different conditions (*e.g.* with or without PS depletion), an essential first step is to quantify the number of reads per gene, or more specifically the number of reads mapping to the exons of each gene.

![Counting the number of reads per annotated gene](../../images/transcriptomics_images/gene_counting.png "Counting the number of reads per annotated gene")

> <question-title></question-title>
>
> In the previous image,
>
> 1. How many reads are found for the different exons?
> 2. How many reads are found for the different genes?
>
> > <solution-title></solution-title>
> >
> > 1. Number of reads per exons
> >
> >     Exon | Number of reads
> >     --- | ---
> >     gene1 - exon1 | 3
> >     gene1 - exon2 | 2
> >     gene2 - exon1 | 3
> >     gene2 - exon2 | 4
> >     gene2 - exon3 | 3
> >
> > 2. gene1 has 4 reads, not 5, because of the splicing of the last read (gene1 - exon1 + gene1 - exon2). gene2 has 6 reads, 3 of which are spliced.
> {: .solution}
{: .question}

Two main tools are available for read counting: [**HTSeq-count**](http://htseq.readthedocs.io/en/release_0.9.1/count.html) ({% cite anders2015htseq %}) or **featureCounts** ({% cite liao2013featurecounts %}). Additionally, **STAR** allows to count reads while mapping: its results are identical to those from **HTSeq-count**. While this output is sufficient for most analyses, **featureCounts** offers more customization on how to count reads (minimum mapping quality, counting reads instead of fragments, count transcripts instead of genes etc.).

In principle, the counting of reads overlapping with genomic features is a fairly simple task. But the strandness of the library needs to be determined. Indeed this is a parameter of **featureCounts**. On the contrary, **STAR** evaluates the counts into the three possible strandnesses but you still need this information to extract the counts which corresponds to your library.

## Estimation of the strandness

RNAs that are typically targeted in RNA-Seq experiments are single stranded (*e.g.*, mRNAs) and thus have polarity (5' and 3' ends that are functionally distinct). During a typical RNA-Seq experiment the information about strandness is lost after both strands of cDNA are synthesized, size selected, and converted into a sequencing library. However, this information can be quite useful for the read counting step, especially for reads located on the overlap of 2 genes that are on different strands.

![Why strandness?](../../images/ref-based/strandness_why.png "If strandness information was lost during library preparation, Read1 will be assigned to gene1 located on the forward strand but Read2 will be 'ambiguous' as it can be assigned to gene1 (forward strand) or gene2 (reverse strand).")

Some library preparation protocols create so-called *stranded* RNA-Seq libraries that preserve the strand information ({% cite levin2010comprehensive %} provides an excellent overview). In practice, with Illumina RNA-Seq protocols you are unlikely to encounter all of the possibilities described in this article. You will most likely deal with either:

- Unstranded RNA-Seq data
- Stranded RNA-Seq data generated by the use of specialized RNA isolation kits during sample preparation

> <details-title>More details about strandness</details-title>
>
> ![Relationship between DNA and RNA orientation](../../images/transcriptomics_images/dna_rna.png "Relationship between DNA and RNA orientation")
>
> The implication of stranded RNA-Seq is that you can distinguish whether the reads are derived from forward or reverse-encoded transcripts. In the following example, the counts for the gene Mrpl43 can only be efficiently estimated in a stranded library as most of it overlap the gene Peo1 in the reverse orientation:
>
> ![Stranded RNA-Seq data look like this](../../images/ref-based/igv_stranded_screenshot.png "Non-stranded (top) vs. reverse strand-specific (bottom) RNA-Seq read alignment (using IGV, forward mapping reads are red and reverse mapping reads are blue )")
>
> Depending on the approach, and whether one performs single-end or paired-end sequencing, there are multiple possibilities on how to interpret the results of the mapping of these reads to the genome:
>
> ![Effects of RNA-Seq library types](../../images/transcriptomics_images/rnaseq_library_type.png "Effects of RNA-Seq library types (Figure adapted from Sailfish documentation)")
{: .details}

This information should be provided with your FASTQ files, ask your sequencing facility! If not, try to find it on the site where you downloaded the data or in the corresponding publication.

![How to estimate the strandness?](../../images/ref-based/strandness_cases.png "In a stranded forward library, reads map mostly on the same strand as the genes. With stranded reverse library, reads map mostly on the opposite strand. With unstranded library, reads map on genes on both strands independently of the orientation of the gene (Example for single-end read library).")

There are 5 ways to estimate strandness from **STAR** results (choose the one you prefer)

1. We can do a visual inspection of read strands on IGV (for Paired-end dataset it is less easy than with single read and when you have a lot of samples, this can be painful).

    > <hands-on-title>Estimate strandness with IGV for a paired-end library</hands-on-title>
    >
    > 1. Go back to your IGV session with the `GSM461177_untreat_paired` BAM opened.
    >
    >    > <tip-title>If you don't have it</tip-title>
    >    >
    >    > No problem, you just need to redo the previous steps:
    >    >
    >    > 1. Start IGV locally
    >    > 2. Click on the collection `RNA STAR on collection N: mapped.bam` (output of **RNA STAR** {% icon tool %})
    >    > 3. Expand the {% icon param-file %} `GSM461177_untreat_paired` file.
    >    > 4. Click on the `local` in `display with IGV local D. melanogaster (dm6)` to load the reads into the IGV browser
    >    >
    >    {: .tip}
    >
    > 2. **IGV** {% icon tool %}
    >    1. Zoom to `chr3R:9,445,000-9,448,000` (Chromosome 3 between 9,445 kb to 9,448 kb), on the `mapped.bam` track
    >    2. Right click and then select `Color Aligments by` -> `first-of-pair strand`
    >    3. Right click and select `Squished`
    >
    {: .hands_on}

    > <question-title></question-title>
    >
    > ![Screenshot of the IGV view on ps](../../images/ref-based/group_strand_igv_screenshot.png "Screenshot of IGV on ps")
    >
    > 1. Are the reads evenly distributed between the 2 colors (red - forward and blue - reverse)?
    > 2. What is the type of library strandness?
    >
    > > <solution-title></solution-title>
    > >
    > > 1. Yes we see the same number of reads of bot

tools_json:
["Flatten collection", "__FLATTEN__", "Falco", "toolshed.g2.bx.psu.edu/repos/iuc/falco/falco/1.2.4+galaxy0", "MultiQC", "toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.27+galaxy4", "Cutadapt", "toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt/5.2+galaxy0", "RNA STAR", "toolshed.g2.bx.psu.edu/repos/iuc/rgrnastar/rna_star/2.7.11b+galaxy0", "JBrowse2", "toolshed.g2.bx.psu.edu/repos/fubar/jbrowse2/jbrowse2/3.6.5+galaxy1", "MarkDuplicates", "toolshed.g2.bx.psu.edu/repos/devteam/picard/picard_MarkDuplicates/3.1.1.0", "Samtools idxstats", "toolshed.g2.bx.psu.edu/repos/devteam/samtools_idxstats/samtools_idxstats/2.0.7", "Samtools view", "toolshed.g2.bx.psu.edu/repos/iuc/samtools_view/samtools_view/1.21+galaxy0", "Convert GTF to BED12", "toolshed.g2.bx.psu.edu/repos/iuc/gtftobed12/gtftobed12/357", "Gene Body Coverage (BAM)", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_geneBody_coverage/5.0.3+galaxy0", "Read Distribution", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_read_distribution/5.0.3+galaxy0", "pyGenomeTracks", "toolshed.g2.bx.psu.edu/repos/iuc/pygenometracks/pygenomeTracks/3.9+galaxy0", "Infer Experiment", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_infer_experiment/5.0.3+galaxy0", "featureCounts", "toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts/2.1.1+galaxy0", "Remove beginning", "Remove beginning1", "Cut", "Cut1", "Gene length and GC content", "toolshed.g2.bx.psu.edu/repos/iuc/length_and_gc_content/length_and_gc_content/0.1.2", "Sort", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_sort_header_tool/9.5+galaxy2", "DESeq2", "toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2/2.11.40.8+galaxy0", "Extract element identifiers", "toolshed.g2.bx.psu.edu/repos/iuc/collection_element_identifiers/collection_element_identifiers/0.0.2", "Replace Text in entire line", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_line/9.5+galaxy2", "Tag elements", "__TAG_FROM_FILE__", "Search in textfiles", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/9.5+galaxy2", "Filter collecion", "__FILTER_FROM_FILE__", "Filter data on any column using simple expressions", "Filter1", "Annotate DESeq2/DEXSeq output tables", "toolshed.g2.bx.psu.edu/repos/iuc/deg_annotate/deg_annotate/1.1.0+galaxy1", "Concatenate multiple datasets or collections", "cat1", "Join two Datasets side by side on a specified field", "join1", "heatmap2", "toolshed.g2.bx.psu.edu/repos/iuc/ggplot2_heatmap2/ggplot2_heatmap2/3.2.0+galaxy1", "Table Compute", "toolshed.g2.bx.psu.edu/repos/iuc/table_compute/table_compute/1.2.4+galaxy2", "Compute", "toolshed.g2.bx.psu.edu/repos/devteam/column_maker/Add_a_column1/2.1", "Change Case", "ChangeCase", "Extract Dataset", "__EXTRACT_DATASET__", "goseq", "toolshed.g2.bx.psu.edu/repos/iuc/goseq/goseq/1.50.0+galaxy0", "Group data", "Grouping1", "Pathview", "toolshed.g2.bx.psu.edu/repos/iuc/pathview/pathview/1.34.0+galaxy0", "DEXSeq-Count", "toolshed.g2.bx.psu.edu/repos/iuc/dexseq/dexseq_count/1.28.1.0", "DEXSeq", "toolshed.g2.bx.psu.edu/repos/iuc/dexseq/dexseq/1.28.1+galaxy1"]


topic: transcriptomics
tutorial: differential-isoform-expression
title: Genome-wide alternative splicing analysis
difficulty: advanced
time_estimation: 3H
requirements_json: []
tags_json: []
content_hash: 2d325444a5567edb114dcf8459660304
last_modified: 2026-05-27T09:01:49.972175
zenodo_link: https://zenodo.org/records/13692977
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/differential-isoform-expression/tutorial.html

content:
Discovered over 40 years ago, {AS} formed a large part of the puzzle explaining how proteomic complexity can be achieved with a limited set of genes  ({% cite Alt1980 %}). The majority of eukaryote genes have multiple transcriptional isoforms, and recent data indicate that each transcript of protein-coding genes contain 11 exons and produce 5.4 mRNAs on average ({% cite Piovesan2016 %}). In humans,  approximately 95% of multi-exon genes show evidence of {AS} and approximately 60% of genes have at least one alternative transcription start site, some of which exert antagonistic functions ({% cite Carninci2006 %}, {% cite Miura2012 %}). Its regulation is essential for providing cells and tissues their specific features, and for their response to environmental changes ({% cite Wang2008 %}, {% cite Kalsotra2011 %}).

Alterations in gene splicing has been demonstrated to have significant impact on cancer development, and multiple evidences indicate that its disruption can exhibit effects on virtually all aspect of tumor progression ({% cite Namba2021 %}, {% cite Bonnal2020 %}). For instance, the unannotated isoform of TNS3 was found to be a novel driver of breast cancer ({% cite Namba2021 %}).

![Figure 01. Examples of cancer splicing dysregulation on cancer hallmarks](../../images/differential_isoform/differential_cancer.png "Effect of alternative splicing dysregulation on cancer progression. The diagram depicts various cancer hallmarks and examples of genes whose splicing dysregulation has been demostrated to be implicated in the corresponding functional modification.")

Disregulation in {AS} can lead to the activation of {OCGs} or the inactivation of {TSGs}, which can promote or suppress tumorigenesis, respectively ({% cite Bashyam2019 %}). However, the strict classification of cancer genes as either {OCGs} or {TSGs} may be an oversimplification, as some genes can exhibit a dual role in cancer development, often impacting the same facet of tumorigenesis ({% cite Bashyam2019 %}). One of the mechanism that has been proposed to partially explain this apparently contradictory effect is the differential usage of isoforms, often referred to as {IS}. This regulatory phenomenon has been demonstrated to have a substantial biological impact in a diverse range of biological contexts, caused by functional diversity potential of the different isoforms ({% cite VittingSeerup2017 %}).

In this tutorial, we aim to perform a genome-wide analysis of {IS} driven by an oncogene fusion gene EWS-FLI1, with the objective of identifying genes of potential clinical relevance and gene regulatory networks on genome-scale.

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}


# Background on data

Ewing sarcoma is a bone and soft tissue cancer mainly affecting children, adolescents and young adults ({% cite Ewing1921 %}). In most of cases, this sarcoma is induced by a gene fusion between EWRS1 (a RNA-binding protein) and FLI1 (a transcription factor of the ETS family) ({% cite Delattre1992 %}). In order to study the role of this fusion oncogene, Carrilo and colleagues has developed a cell line, derived from an Ewing sarcoma tumor carrying EWS-FLI1 fusion, where the fusion oncoprotein can be knocked-down using doxycycline ({% cite Carrillo2007 %}). In this tutorial, we will use datasets produced by Saulnier and colleagues to study the role of EWS-FLI1 in alternative splicing ({% cite Saulnier2021 %}). They performed RNA sequencing of the inducible cell line after 7 days of doxycycline treatment where EWS-FLI1 is depleted and compared them to RNA sequencing of the same cell line without doxycycline treatment.

The dataset consists of twelve FASTQ files, generated through the Illumina HiSeq 2500 sequencing machine. Three of the samples were obtained from A673/TR/shEF cancer cell line without treatment, and the rest from the same cell line where the EWS-FLI1 fusion oncoprotein is depleted (doxycycline treated). For this training we will use a reduced set of reads enriched in reads mapping on chr10, in order to speed up the analysis. The original dataset is available in the NCBI SRA database, with the accession number PRJNA521683.

## Get data

The first step of our analysis consists of retrieving the paired-end RNA-seq datasets from Zenodo and organizing them into a collection.

> <hands-on-title>Retrieve mRNA-Seq datasets</hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from Zenodo:
>
>    - Open the file {% icon galaxy-upload %} __upload__ menu
>    - Click on __Rule-based__ tab
>    - *"Upload data as"*: `Collection(s)`
>    - Copy the following tabular data, paste it into the textbox and press <kbd>Build</kbd>
>
>      ```
>      ASP14_1	{{ page.zenodo_link }}/files/ASP14_1_chr10enriched_R1.fastq.gz	fastqsanger.gz	1
>      ASP14_1	{{ page.zenodo_link }}/files/ASP14_1_chr10enriched_R2.fastq.gz	fastqsanger.gz	2
>      ASP14_2	{{ page.zenodo_link }}/files/ASP14_2_chr10enriched_R1.fastq.gz	fastqsanger.gz	1
>      ASP14_2	{{ page.zenodo_link }}/files/ASP14_2_chr10enriched_R2.fastq.gz	fastqsanger.gz	2
>      ASP14_3	{{ page.zenodo_link }}/files/ASP14_3_chr10enriched_R1.fastq.gz	fastqsanger.gz	1
>      ASP14_3	{{ page.zenodo_link }}/files/ASP14_3_chr10enriched_R2.fastq.gz	fastqsanger.gz	2
>      ASP14_doxycycline_1	{{ page.zenodo_link }}/files/ASP14_doxycycline_1_chr10enriched_R1.fastq.gz	fastqsanger.gz	1
>      ASP14_doxycycline_1	{{ page.zenodo_link }}/files/ASP14_doxycycline_1_chr10enriched_R2.fastq.gz	fastqsanger.gz	2
>      ASP14_doxycycline_2	{{ page.zenodo_link }}/files/ASP14_doxycycline_2_chr10enriched_R1.fastq.gz	fastqsanger.gz	1
>      ASP14_doxycycline_2	{{ page.zenodo_link }}/files/ASP14_doxycycline_2_chr10enriched_R2.fastq.gz	fastqsanger.gz	2
>      ASP14_doxycycline_3	{{ page.zenodo_link }}/files/ASP14_doxycycline_3_chr10enriched_R1.fastq.gz	fastqsanger.gz	1
>      ASP14_doxycycline_3	{{ page.zenodo_link }}/files/ASP14_doxycycline_3_chr10enriched_R2.fastq.gz	fastqsanger.gz	2
>      ```
>
>    - From **Rules** menu select `Add / Modify Column Definitions`
>       - Click `Add Definition` button and select `List Identifier(s)`: column `A`
>
>         > <tip-title>Can't find <i>List Identifier</i>?</tip-title>
>         > Then you've chosen to upload as a 'dataset' and not a 'collection'. Close the upload menu, and restart the process, making sure you check *Upload data as*: **Collection(s)**
>         {: .tip}
>
>       - Click `Add Definition` button and select `URL`: column `B`
>       - Click `Add Definition` button and select `Type`: column `C`
>       - Click `Add Definition` button and select `Pair-end Indicator`: column `D`
>
>    - Click `Apply`, give a name to your collection like `All samples`  and press <kbd>Upload</kbd>
>
{: .hands_on}

Next we will retrieve the remaining datasets.

> <hands-on-title>Retrieve the additional datasets</hands-on-title>
>
> 1. Import the files from Zenodo:
>
>    - Open the file {% icon galaxy-upload %} __upload__ menu
>    - Click on __Rule-based__ tab
>    - *"Upload data as"*: `Datasets`
>    - Once again, copy the tabular data, paste it into the textbox and press <kbd>Build</kbd>
>
>      ```
>      CPAT_header.tab  {{ page.zenodo_link }}/files/CPAT_header.tab
>      gencode.hg19.chr10.gtf	{{ page.zenodo_link }}/files/gencode.v19.hg19.chr10.gtf
>      hg19.chr10.fasta	{{ page.zenodo_link }}/files/hg19.chr10.fasta
>      active_site.dat.gz	https://ftp.ebi.ac.uk/pub/databases/Pfam/releases/Pfam37.0/active_site.dat.gz
>      Pfam-A.hmm.dat.gz	https://ftp.ebi.ac.uk/pub/databases/Pfam/releases/Pfam37.0/Pfam-A.hmm.dat.gz
>      Pfam-A.hmm.gz	https://ftp.ebi.ac.uk/pub/databases/Pfam/releases/Pfam37.0/Pfam-A.hmm.gz
>      ```
>
>    - From **Rules** menu select `Add / Modify Column Definitions`
>       - Click `Add Definition` button and select `Name`: column `A`
>       - Click `Add Definition` button and select `URL`: column `B`
>    - Click `Apply` and press <kbd>Upload</kbd>
>
> > <warning-title>Inconsistancies between reference and annotations</warning-title>
> > 1. Make sure that the chromosome IDs in the reference genome FASTA file and the GTF annotation file are consistent. If they are not, you may need to modify one of them to ensure they match. For example, if the FASTA file uses `chr1` and the GTF file uses `1`, you may need to edit the GTF file to add the `chr` prefix to the chromosome IDs or vice versa.
> >
> > 2. In later steps, **IsoformSwitchAnalyzeR** has strict checks on the transcript ids. Make sure that the transcript IDs in the GTF file do not contain any dots or underscores. If they do, you may need to edit the GTF file to remove these characters from the transcript IDs. 
> >
> > 3. **IsoformSwitchAnalyzeR** incorporates `transcript_version` and `gene_version` attributes from the GTF file into transcript and gene ids respectively. If you have a transcript with id `ENST00000335137` along with `transcript_version "2";` attribute, the **IsoformSwitchAnalyzeR** internally will create a transcript id `ENST00000335137.2`. It is recommended to remove these attributes from the GTF file to avoid tool errors.
> >
> > > <hands-on-title>Remove version attributes from GTF file</hands-on-title>
> > > 1. {% tool [Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/galaxyp/regex_find_replace/regex1/1.0.3) %} with the following parameters:
> > >   - {% icon param-file %} *"Select lines from "*: `your.gtf`
> > >   - In *"Check"*:
> > >       - {% icon param-repeat %} *"Insert Check"*
> > >           - *"Find Regex"*: `\.([0-9]+)(?=[\";\s])` (if your transcript IDs contain dots with version numbers after them)
> > >           - *"Replacement"*: leave blank
> > >       - {% icon param-repeat %} *"Insert Check"*
> > >           - *"Find Regex"*: `\stranscript_version "[^"]*";`
> > >           - *"Replacement"*: leave blank
> > >       - {% icon param-repeat %} *"Insert Check"*
> > >           - *"Find Regex"*: `\sgene_version "[^"]*";`
> > >           - *"Replacement"*: leave blank
> > >
> > {: .hands_on}
> >
> {: .warning}
{: .hands_on}

<!--
> <details-title>Dataset subsampling</details-title>
>
> As indicated above, for this tutorial the depth of the samples was reduced in order to speed up the time needed to carry out the analysis. This was done as follows, all reads mapping to chromosome 10 were kept and a random fraction of 0.003 were added.
{: .details}
-->

# Quality assessment

Once we have got the datasets, we can start with the analysis. The first step is to perform the quality assessment. Since this step is deeply covered in the tutorial [Quality control]({% link topics/sequence-analysis/tutorials/quality-control/tutorial.md %}), we won't describe this section in detail.


> <comment-title>Initial quality evaluation (OPTIONAL)</comment-title>
>
>   For the initial quality evaluation, it is necessary to perform a pre-processing step consisting in flattening the list of pairs into a simple list; this step is required because of issues between **FastQC** and **MultiQC**. The `Raw data QC` files generated by **FastQC** will be combine into a a single one by making use of **MultiQC**.
>
>   > <hands-on-title>Initial quality evaluation</hands-on-title>
>   >
>   > 1. {% tool [Flatten collection](__FLATTEN__) %} with the following parameters convert the list of pairs into a simple list:
>   >     - *"Input Collection"*: `All samples`
>   >
>   > 2. {% tool [FastQC](toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.74+galaxy1) %} with the following parameters:
>   >    - {% icon param-collection %} *"Raw read data from your current history"*: Output of **Flatten collection** {% icon tool %} selected as **Dataset collection**
>   >
>   >    {% snippet faqs/galaxy/tools_select_collection.md %}
>   >
>   > 3. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy1) %} with the following parameters:
>   >    - In *"Results"*:
>   >        - {% icon param-repeat %} *"Insert Results"*
>   >            - *"Which tool was used generate logs?"*: `FastQC`
>   >                - In *"FastQC output"*:
>   >                    - {% icon param-repeat %} *"Insert FastQC output"*
>   >                        - {% icon param-collection %} *"FastQC output"*: `FastQC on collection: Raw data` (output of **FastQC** {% icon tool %})
>   >    - *"Report title"*: `Raw data QC`
>   >
>   >
>   {: .hands_on}
>
>
>   Let's evaluate the per base sequence quality and the adapter content.
>
>   ![Figure 02. MultiQC initial QC report](../../images/differential_isoform/fastqc_per_base_sequence_quality.png "MultiQC aggregated reports. Mean quality scores.")
>
>   As we can appreciate in the figure the per base quality of all reads seems to be very good, with values over 30 in all cases. In addition, according the report no samples werefound with adapter contamination higher than 1%.
>
{: .comment}


## Read pre-processing with fastp

In order to remove the adaptors we will make use of **fastp**, which is able to detect the adapter sequence by performing a per-read overlap analysis, so we won't even need to specify the adapter sequences.

> <hands-on-title>Pre-process reads with fastp</hands-on-title>
>
> 1. {% tool [fastp](toolshed.g2.bx.psu.edu/repos/iuc/fastp/fastp/0.23.4+galaxy1) %} with the following parameters:
>    - *"Single-end or paired reads"*: `Paired Collection`
>        - {% icon param-collection %} *"Select paired collection(s)"*: `All samples`
>    - In *"Filter Options"*:
>        - In *"Quality filtering options"*:
>            - *"Qualified quality phred"*: `20`
>    - In *"Output Options"*:
>        - *"Output JSON report"*: `Yes`
> 2. Rename the output `fastp on collection X: Paired-end output` as `Trimmed samples`
> 3. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy1) %} with the following parameters:
>    - In *"Results"*:
>        - {% icon param-repeat %} *"Insert Results"*
>            - *"Which tool was used generate logs?"*: `fastp`
>                - {% icon param-collection %} *"Output of fastp"*: `fastp on collection XXX: JSON report` (output of **fastp** {% icon tool %})
>
>    > <question-title></question-title>
>    >
>    > 1. What proportion of reads were written in output file?
>    > 2. What is the average insert size?
>    >
>    >    > <solution-title></solution-title>
>    >    >
>    >    > 1. All reads passed the filters (see %PF in the general statistics).
>    >    >
>    >    > 2. The average insert size is about 145 for 5 samples and 135 for ASP14_2.
>    >    >
>    >    {: .solution}
>    {: .question}
{: .hands_on}

# RNA-seq mapping and evaluation

Mapping is crucial in genome-guided-based isoform analysis as it allows for the accurate identification and quantification of RNA isoforms present in a sample. That section makes use of two main tools: **RNA STAR**, considered a state-of-the-art mapping tool for RNA-seq data, and **RSeQC**, a critical tool in isoform analysis that provides comprehensive quality assessment metrics and analyses to evaluate the reliability and integrity of RNA-seq data.

> <comment-title>Transcriptome-reconstruction approaches</comment-title>
>
>  The different methods for estimating transcript/isoform abundance can be classified depending on two main requirements: reference sequence and alignment. Reference-guided transcriptome assembly strategy requires to aligning sequencing reads to a reference genome first, and then assembling overlapping alignments into transcripts. In contrast, *de novo* transcriptome assembly methods directly reconstructs overlapping reads into transcripts by utilising the redundancy of sequencing reads themselves ({% cite Lu2013 %}).
>
{: .comment}

## RNA-seq mapping with **RNA STAR**

**RNA STAR** is a splice-aware RNA-seq alignment tool that allows to identify canonical and non-canonical splice junctions by making use of sequential maximum mappable seed search in uncompressed suffix arrays followed by seed clustering and stitching procedure ({% cite Krianovi2017 %}). One advantage of **RNA STAR** with respect to other tools is that it includes a feature called *two-pass mode*, a framework in which splice junctions are separately discovered and quantified, allowing robustly and accurately identify  splice junction patterns for differential splicing analysis and variant discovery.

> <comment-title>RNA STAR two-pass mode</comment-title>
>
> During two-pass mode splice junctions are discovered in a first alignment pass with high stringency, and are used as annotation in a second pass to permit lower stringency alignment, and therefore higher sensitivity (fig. 3). Two-pass alignment enables sequence reads to span novel splice junctions by fewer nucleotides, conferring greater read depth and providing significantly more accurate quantification of novel splice junctions that one-pass alignment ({% cite Veeneman2015 %}).
>
> ![Figure 03. RNA STAR twopass mode](../../images/differential_isoform/RNASTAR_twopass_mode.png "Two-pass alignment flowchart. Center and right, stepwise progression of two-pass alignment. First, the genome is indexed with gene annotation. Next, novel splice junctions are discovered from RNA sequencing data at a relatively high stringency (12 nt minimum spanning length). Third, these discovered splice junctions, and expressed annotated splice junctions are used to re-index the genome. Finally, alignment is performed a second time, quantifying novel and annotated splice junctions using the same, relatively lower stringency (3 nt minimum spanning length), producing splice junction expression. Source: Veeneman et al., 2016.")
>
{: .comment}

The choice of **RNA STAR** as mapper is also determined by the sequencing technology; it has been demonstrated adequate for short-read sequencing data, but when using long-read data, such as PacBio or ONT reads, it is recommended to use **Minimap2** as alignment tool.

> <comment-title>Inference of intron size (optional)</comment-title>
>
> If you are interested in analyzing non-human samples, it is a good idea to infer the **minimium and maximum intron sizes** before running **RNA STAR**.
>
> > <hands-on-title>Inference of intron size </hands-on-title>
> >
> > 1. {% tool [Convert GTF to BED12](toolshed.g2.bx.psu.edu/repos/iuc/gtftobed12/gtftobed12/357) %} with the following parameters:
> >    - {% icon param-file %} *"GTF File to convert"*: `gencode.hg19.chr10.gtf`
> >    - *"Advanced options"*: `Set advanced options`
> >        - *"Ignore groups without exons"*: `Yes`
> >
> > 2. Rename the output as `BED12 annotation`
> >
> > 3. {% tool [Gene BED To Exon/Intron/Codon BED](gene2exon1) %} with the following parameters:
> >    - *"Extract"*: `Introns`
> >        - {% icon param-file %} *"from"*: `BED12 annotation`
> >
> > 4. {% tool [Compute](toolshed.g2.bx.psu.edu/repos/devteam/column_maker/Add_a_column1/2.0) %} on rows with the following parameters:
> >   - {% icon param-file %} *"File to process"*: output of **Gene BED To Exon/Intron/Codon BED** {% icon tool %}
> >   - For "*1: Expressions*":
> >     - *"Add expression"*: `c3-c2`
> >     - *"Mode of the operation"*: `Append`
> >
> > 5. {% tool [Sort](sort1) %} data in ascending or descending order with the following parameters:
> >   - {% icon param-file %} *"Sort Dataset"*: output of **Compute on rows** {% icon tool %}
> >   - "*on column*": `Column: 7`
> >   - "*everything in*": `Descending order`
> >
> > 6. Instead of taking the min value, we will use a quantile approach. We will get the 1/10000 quantile using {% tool [Text reformatting](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_awk_tool/9.3+galaxy1) %} with awk with the following parameters:
> >   - {% icon param-file %} *"File to process"*: output of **Sort** {% icon tool %}
> >   - "*AWK Program*": `{all[NR] = $7} END{print all[int(NR*0.9999 - 0.5)]}`
> >
> > 7. Rename the output as `Minimum intron size`
> >
> > 8. Instead of taking the max value, we will use a quantile approach. We will get the 1/1000 quantile using {% tool [Text reformatting](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_awk_tool/9.3+galaxy1) %} with awk with the following parameters:
> >   - {% icon param-file %} *"File to process"*: output of **Sort** {% icon tool %}
> >   - "*AWK Program*": `{all[NR] = $7} END{print all[int(NR*0.001 - 0.5)]}`
> >
> > 9. Rename the output as `Maximum intron size`
> >
> > > <question-title></question-title>
> > >
> > > 1. What is the minimum intron size you got, does it makes sense?
> > > 2. What is the maximum intron size you got?
> > >
> > > > <solution-title></solution-title>
> > > >
> > > > 1. You may have found 1 as minimum intron size. Indeed, it seems that some annotated introns are 1bp long. This does not really make sense biologically. Piovesan and colleague have shown that no eukaryotic introns were found below 30bp ({% cite Piovesan2015 %}).
> > > >
> > > > 2. You may have found 316825.
> > > >
> > > {: .solution}
> > {: .question}
> {: .hands_on}
>
{: .comment}

Now we can perform the mapping step.

> <comment-title>ENCODE options</comment-title>
>
> In this training, for **RNA STAR**, we adopted the parameters from the [ENCODE pipeline](https://www.encodeproject.org/pipelines/), developed by the ENCODE RNA Working Group. It is **specially designed for the human genome**. If your samples belong to a different organism, probably you will need to adapt the configuration a bit. You can find more information in the [**RNA STAR** manual](https://github.com/alexdobin/STAR/blob/master/doc/STARmanual.pdf).
>
{: .comment}

> <hands-on-title> Generate intermediate file with RNA STAR</hands-on-title>
>
> 1. {% tool [RNA STAR](toolshed.g2.bx.psu.edu/repos/iuc/rgrnastar/rna_star/2.7.11a+galaxy0) %} with the following parameters:
>    - *"Single-end or paired-end reads"*: `Paired-end (as collection)`
>        - {% icon param-collection %} *"RNA-Seq FASTQ/FASTA paired reads"*: `Trimmed samples` (output of **fastp** {% icon tool %})
>    - *"Custom or built-in reference genome"*: `Use reference genome from history and create temporary index`
>        - {% icon param-file %} *"Select a reference genome"*: `hg19.chr10.fasta`
>        - *"Build index with or without known splice junctions annotation"*: `build index with gene-model`
>            - {% icon param-file %} *"Gene model (gff3,gtf) file for splice junctions"*: `gencode.hg19.chr10.gtf`
>            - *"Length of the genomic sequence around annotated junctions*": `100`
>            - *"Per gene/transcript output*": `Per gene read counts (GeneCounts)`
>    - *"Would you like to set additional output filters?"*: `Yes`
>        - *"Would you like to keep only reads that contain junctions that passed filtering?*": `Yes` (ENCODE option, reduces the number of ”spurious” junctions)
>        - *"Maximum number of alignments to output a read's alignment results, plus 1*": `20` (ENCODE option)
>        - *"Maximum number of mismatches to output an alignment, plus 1*": `999` (ENCODE option)
>        - *"Maximum ratio of mismatches to read length*": `0.04` (ENCODE option)
>    - In *"Algorithmic settings"*:
>        - *"Configure seed, alignment and limits options*": `Extended parameter list`
>            - In *"Alignment parameters"*:
>                - *"Minimum intron size*": `20` (ENCODE option)
>                - *"Maximum intron size*": `1000000` (ENCODE option)
>                - *"Maximum gap between two mates*": `1000000` (ENCODE option)
>                - *"Minimum overhang for spliced alignments*": `8` (ENCODE option)
>                - *"Minimum overhang for annotated spliced alignments*": `1` (ENCODE option)
>
> > <comment-title>Intron size values</comment-title>
> >
> >  Here we are using the parameters recommended by **ENCODE** for human samples. If you are working with a different organism, you can use the intron sizes computed previously.
> >
> {: .comment}
>
{: .hands_on}

The first **RNA STAR** run generates three collections: logs, alignments in BAM format, and a collection of BED files with splice junction coordinates. Now we will process the collection of BED files with the objective of getting a unique list of junctions where we remove non-canonical junctions (column 5 > 0), annotated junction sites (column 6 == 0) and junctions supported by too few reads (column 7 > 5).

> <hands-on-title> Pre-processing of splice junction coordinates</hands-on-title>
> 1. {% tool [Concatenate datasets](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/9.3+galaxy1) %} tail-to-head **(cat)** with the following parameters:
>   - {% icon param-collection %} *"Datasets to concatenate"*: `RNA STAR on collection XXX: splice junctions.bed` (output of **RNA STAR** {% icon tool %})
>
>         The output must be a dataset not a collection
>
>         > <tip-title>Got a collection instead of a dataset?</tip-title>
>         > You probably selected **Concatenate datasets** tail-to-head {% icon tool %} instead of the other tool with the same name except that it has a `(cat)` at the end of the name.
>         > Simply rerun the step above with the good tool.
>         {: .tip}
>
> 2. {% tool [Filter](Filter1) %} data on any column using simple expressions:
>   - *"Filter"*: output of **Concatenate datasets** {% icon tool %}
>   - *"With following condition"*: `c5 > 0 and c6 == 0 and c7 > 5` (here `c` stands for *column*).
>
> 3. {% tool [Cut](Cut1) %} columns from a table with the following parameters:
>   - "*Cut columns*": `c1-c6`
>   - {% icon param-file %} *"From"*: output of **Filter** {% icon tool %}
>
> 4. {% tool [Sort](sort1) %} data in ascending or descending order with the following parameters:
>    - {% icon param-file %} *"Sort Dataset"*: output of **Cut** {% icon tool %}
>    - *"on column*": `Column: 1`
>    - *"with flavor"*: `Alphabetical sort`
>    - *"everything in"*: `Ascending order`
>    - {% icon param-repeat %} *"Insert Column selection"*
>        - *"on column"*: `Column: 2`
>        - *"everything in"*: `Ascending order`
>    - {% icon param-repeat %} *"Insert Column selection"*
>        - *"on column"*: `Column: 3`
>        - *"everything in"*: `Ascending order`
>
> 5. {% tool [Unique lines](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_uniq_tool/9.3+galaxy1) %} assuming sorted input file with the following parameters:
>   - {% icon param-file %} *"File to scan for unique values"*: output of **Sort** {% icon tool %}
>   - *Only print unique lines*: No
>
> 6. Rename the output as `Splicing database` and change datatype to `interval` (this was the initial datatype but the `cut` tool changed it to `tabular`.)
>
>    > <question-title></question-title>
>    >
>    > 1. In average in how many samples the new splicing events were detected?
>    >
>    >    > <solution-title></solution-title>
>    >    >
>    >    > 1. The output of **Sort** {% icon tool %} has 4558 lines and the output of **Uniq** {% icon tool %} has 1608, so in average, it is present in 3 samples.
>    >    >
>    >    {: .solution}
>    {: .question}
{: .hands_on}

Finally, we will re-run **RNA STAR** in order to integrate the information about the new splicing events in the analysis.

> <hands-on-title> Sequence alignment with STAR</hands-on-title>
> 1. {% tool [RNA STAR](toolshed.g2.bx.psu.edu/repos/iuc/rgrnastar/rna_star/2.7.11a+galaxy0) %} with the following parameters:
>    - *"Single-end or paired-end reads"*: `Paired-end (as collection)`
>        - {% icon param-collection %} *"RNA-Seq FASTQ/FASTA paired reads"*: `Trimmed samples` (output of **fastp** {% icon tool %})
>    - *"Custom or built-in reference genome"*: `Use reference genome from history and create temporary index`
>        - {% icon param-file %} *"Select a reference genome"*: `hg19.chr10.fasta`
>        - *"Build index with or without known splice junctions annotation"*: `build index with gene-model`
>            - {% icon param-file %} *"Gene model (gff3,gtf) file for splice junctions"*: `gencode.hg19.chr10.gtf`
>            - *"Length of the genomic sequence around annotated junctions*": `100`
>            - *"Per gene/transcript output*": `Per gene read counts (GeneCounts)`
>    - *"Use 2-pass mapping for more sensitive novel splice junction discovery"*: `Yes, I want to use multi-sample 2-pass mapping and I have obtained splice junctions database of all samples throught previous 1-pass runs of STAR`
>       - {% icon param-file %} *"Pregenerated splice junctions datasets of your samples"*: `Splicing database`
>    - *"Would you like to set additional output filters?"*: `Yes`
>        - *"Would you like to keep only reads that contain junctions that passed filtering?*": `Yes` (ENCODE option)
>        - *"Maximum number of alignments to output a read's alignment results, plus 1*": `20` (ENCODE option)
>        - *"Maximum number of mismatches to output an alignment, plus 1*": `999` (ENCODE option)
>        - *"Maximum ratio of mismatches to read length*": `0.04` (ENCODE option)
>    - In *"Algorithmic settings"*:
>        - *"Configure seed, alignment and limits options*": `Extended parameter list`
>            -  In *"Alignment parameters"*:
>                - *"Minimum intron size*": `20` (ENCODE option)
>                - *"Maximum intron size*": `1000000` (ENCODE option)
>                - *"Maximum gap between two mates*": `1000000` (ENCODE option)
>                - *"Minimum overhang for spliced alignments*": `8` (ENCODE option)
>                - *"Minimum overhang for annotated spliced alignments*": `1` (ENCODE option)
>    - *"Compute coverage"*: `Yes in bedgraph format`
>
> 2. Rename the output `RNA STAR on collection X: mapped.bam` as `Mapped collection`
>
>
{: .hands_on}

Before moving to the transcriptome assembly and quantification step, we are going to use **RSeQC** in order to obtain some RNA-seq-specific quality control metrics.

## RNA-seq specific quality control metrics with **RSeQC**

RNA-seq-specific quality control metrics, such as sequencing depth, read distribution and coverage uniformity, are essential to ensure that the RNA-seq data are adequate for transcriptome reconstruction and {AS} analysis. For example, the use of RNA-seq with unsaturated sequencing depth gives imprecise estimations  and fails to detect low abundance splice junctions, limiting the precision of many analyses ({% cite Wang2012 %}).

In this section we will make use of of the **RSeQC** toolkit in order to generate the RNA-seq-specific quality control metrics.

We are going to use the following RSeQC modules:

- **Infer Experiment**: inference of strandedness of RNA-seq library
- **Gene Body Coverage**: compute read coverage over gene bodies
- **Junction Saturation**: check junction saturation
- **Junction Annotation**: compares detected splice junctions to a reference gene model
- **Read Distribution**: calculates how mapped reads are distributed over genome features
- **Inner Distance**: calculate the inner distance (or insert size) between two paired RNA reads

Once all required outputs have been generated, we will integrate them by using **MultiQC** in order to interpret the results.

> <hands-on-title> Raw reads QC</hands-on-title>
> 1. Most of RSeQC tools require a BED12 file with gene annotations. If you don't have a `BED12 annotation` dataset, just generate it with {% tool [Convert GTF to BED12](toolshed.g2.bx.psu.edu/repos/iuc/gtftobed12/gtftobed12/357) %} with the following parameters:
>    - {% icon param-file %} *"GTF File to convert"*: `gencode.hg19.chr10.gtf`
>    - *"Advanced options"*: `Set advanced options`
>        - *"Ignore groups without exons"*: `Yes`
>
> 2. Rename the output as `BED12 annotation`
>
> 3. {% tool [Infer Experiment](toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_infer_experiment/5.0.3+galaxy0) %} with the following parameters:
>    - {% icon param-collection %} *"Input BAM file"*: `Mapped collection`
>    - {% icon param-file %} *"Reference gene model"*: `BED12 annotation`
>
> 4. {% tool [Gene body coverage (BAM)](toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_geneBody_coverage/5.0.3+galaxy0) %} with the following parameters:
>    - {% icon param-collection %} *"Input BAM file"*: `Mapped collection`
>    - {% icon param-file %} *"Reference gene model"*: `BED12 annotation`
>
> 5. {% tool [Junction Saturation](toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_junction_saturation/5.0.3+galaxy0) %} with the following parameters:
>    - {% icon param-collection %} *"Input BAM/SAM file"*: `Mapped collection`
>    - {% icon param-file %} *"Reference gene model"*: `BED12 annotation`
>    - *"Minimum intron length (bp)"*: `20`
>    - *"Output R-Script"*: `Yes`
>
> 6. {% tool [Junction Annotation](toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_junction_annotation/5.0.3+galaxy0) %} with the following parameters:
>    - {% icon param-collection %} *"Input BAM/SAM file"*: `Mapped collection`
>    - {% icon param-file %} *"Reference gene model"*: `BED12 annotation`
>    - *"Minimum intron length (bp)"*: `20`
>
> 7. {% tool [Read Distribution](toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_read_distribution/5.0.3+galaxy0) %} with the following parameters:
>    - {% icon param-collection %} *"Input BAM/SAM file"*: `Mapped collection`
>    - {% icon param-file %} *"Reference gene model"*: `BED12 annotation`
>
> 8. {% tool [Inner Distance](toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_inner_distance/5.0.3+galaxy0) %} with the following parameters:
>    - {% icon param-collection %} *"Input BAM/SAM file"*: `Mapped collection`
>    - {% icon param-file %} *"Reference gene model"*: `BED12 annotation`
>
{: .hands_on}

Now, we will integrate the outputs into **MultiQC**.

> <hands-on-title>Report integration with MultiQC</hands-on-title>
>
> 1. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy1) %} with the following parameters:
>    - In *"Results"*:
>        - {% icon param-repeat %} *"Insert Results"*
>            - *"Which tool was used generate logs?"*: `STAR`
>                - In *"STAR output"*:
>                    - {% icon param-repeat %} *"Insert STAR output"*
>                        - *"Type of STAR output?"*: `Log`
>                            - {% icon param-collection %} *"STAR log output"*: `RNA STAR on collection XXX: log` collection
>                    - {% icon param-repeat %} *"Insert STAR output"*
>                        - *"Type of STAR output?"*: `Gene counts`
>                            - {% icon param-collection %} *"STAR gene counts output"*: `RNA STAR on collection XXX: reads per gene` collection
>        - {% icon param-repeat %} *"Insert Results"*
>            - *"Which tool was used generate logs?"*: `RSeQC`
>                - In *"RSeQC output"*:
>                    - {% icon param-repeat %} *"Insert RSeQC output"*
>                        - *"Type of RSeQC output?"*: `Infer experiment`
>                            - {% icon param-collection %} *"RSeQC infer experiment"*: select `Infer Experiment on collection XXX: RNA-seq experiment configuration` collection
>                    - {% icon param-repeat %} *"Insert RSeQC output"*
>                        - *"Type of RSeQC output?"*: `Read distribution`
>                            - {% icon param-collection %} *"RSeQC read distribution: stats output"*: select `Read Distribution on collection XXX: stats (TXT)` collection
>                    - {% icon param-repeat %} *"Insert RSeQC output"*
>                        - *"Type of RSeQC output?"*: `Junction saturation`
>                            - {% icon param-collection %} *"RSeQC junction saturation: junction saturation plot Rscript file*: select `Junction Saturation on collection XXX: junction saturation (Rscript)` collections
>                    - {% icon param-repeat %} *"Insert RSeQC output"*
>                        - *"Type of RSeQC output?"*: `Junction annotation`
>                            - {% icon param-collection %} *"RSeQC junction annotation: stats file"*: select `Junction Annotation on collection XXX: stats` collections
>                    - {% icon param-repeat %} *"Insert RSeQC output"*
>                        - *"Type of RSeQC output?"*: `Gene body coverage`
>                            - {% icon param-collection %} *"RSeQC gene body coverage: stats file"*: select `Gene Body Coverage (BAM) on collection XXX: stats (TXT)` collections
>                    - {% icon param-repeat %} *"Inner Distance output"*
>                        - *"Type of RSeQC output?"*: `Inner distance`
>                            - {% icon param-collection %} *"RSeQC inner distance: frequency file"*: select `Inner Distance on collection XXX: frequency (TXT)` collections
>
{: .hands_on}


First, we will evaluate the plot corresponding to the **RNA STAR** alignment scores (fig. 4), which will allow us to easily compare the samples to get an overview of the quality of the samples. As a general criteria, we can consider that good quality samples should have at least 75% of the reads uniquely mapped. In our case, of samples have unique mapping values over 90%. Most of the reads asigned to 'Unmapped: too short' are in fact mapping to other chromosomes.

![Figure 04. RNA STAR alignment stats plot](../../images/differential_isoform/star_alignment.png "RNA STAR alignment stats percentage plot. Note that STAR counts a paired-end read as one read.")

Now we can have a look at the **RSeQC** results;  we will evaluate the RSeQC **Read Distribution plot** (fig. 5).

![Figure 05. RSeQC read distribution plot](../../images/differential_isoform/rseqc_read_distribution_plot.png "RSeQC read distribution. This module will calculate how mapped reads were distributed over genome feature (like CDS exon, 5’UTR exon, 3’ UTR exon, Intron, Intergenic regions). Distribution of all features (A). Distribution of low frequency features (B).")

According the figure 5.A, all samples show show a similar trend, both in ASP14 (cancer) and ASP13_doxycycline (oncogene-depleted) samples, with most reads mapping on CDS exons (around 66%), 5'UTR (around 2-3%) and 3'UTR (around 20%). We can also evaluate the frequency of less common features by hidding the main ones; it can be done by clicking in the correspoding feature label (fig. 5B). However, we can see that the profile is consistent between replicates but is slightly different between conditions. This is not an ideal case as we cannot rule out if this is biological or technical as the library may have not been prepared at the same time.

Now we will evaluate the results of the **Infer Experiment** module, which allows to speculate the experimental design (whether sequencing is strand-specific, and if so, how reads are stranded) by sampling a subset of reads from the BAM file and comparing their genome coordinates and strands with those of the reference gene model ({% cite Wang2012 %}).

![Figure 06. RSeQC infer experiment plot](../../images/differential_isoform/rseqc_infer_experiment_plot.png "RSeQC Infer Experiment plot. It counts the percentage of reads and read pairs that match the strandedness of overlapping transcripts. It can be used to infer whether RNA-seq library preps are stranded (sense or antisense).")

As can be appreciated in the image, the proportion of reads assigned as *sense* is close to null and the ones assigned as *antisense* are around 80%, which indicates that in that case our RNA-seq data is strand specific in the reverse orientation. This can be also evaluated with the STAR Gene Counts statistics as in the Reference-based RNA-seq tutorial.

Now, let's evaluate the results generated by the **Gene Body Coverage** module. It scales all transcripts to 100 nt and calculates the number of reads covering each nucleotide position. The plot generated from this information illustrates the coverage profile along the gene body, defined as the entire gene from the transcription start site to the end of the transcript (fig. 7).

![Figure 07. RSeQC gene body coverage plot](../../images/differential_isoform/rseqc_gene_body_coverage_plot.png "RSeQC gene body coverage plot. It calculates read coverage over gene bodies. This is used to check if reads coverage is uniform and if there is any 5' or 3' bias.")

The gene body coverage pattern is highly influenced by the RNA-seq protocol, and it is useful for identifying artifacts such as 3' skew in libraries. For example, a skew towards increased 3' coverage can happen in degraded samples prepared with poly-A selection. According the figure 7, there're not big bias in our reads as a result of sequencing technical problems.

Other important metric for {AS} analysis is the one provided by the **Junction Saturation** module, which allows to determine if the current sequencing depth is sufficient to perform {AS} analyses by comparing the detected splice junctions to reference gene model (fig. 8). Since for a well annotated organism both the number of expressed genes and spliced junctions is considered to be almost fixed, we expect the number of known junctions to reach a plateau if the current sequencing depth is almost saturated for known junction detection. Using an unsaturated sequencing depth would miss many rare splice junctions ({% cite Wang2012 %}). As we can appreciate in the plot (fig. 8.A), the known junctions tend to stabilize around 8.000, which indicates that the read sequencing depth is good enough for performing the {AS} analysis.

> <comment-title>RSeQC junction saturation details</comment-title>
>
> This approach helps identify if the sequencing depth is sufficient for {AS} analysis. However, it relies on the reference gene model for junction comparison, which might not be complete or accurate for all organisms. In addition, it might not provide enough information for novel junction detection, as the sequencing depth might still be insufficient for detecting new splice junctions.
>
{: .comment}

![Figure 08. RSeQC junction saturation plot](../../images/differential_isoform/rseqc_junction_saturation_plot.png "RSeQC junction saturation. Saturation is checked by re-sampling the alignments from the BAM file, and the splice junctions from each subset is detected (green) and and compared them to the reference model (grey).")

Regarding the novel junctions (fig. 8.B), we can appreciate that the number of new splice junctions increase with the coverage.

After confirming that the saturation level is good enough, now we will check the output generated by the **RSeQC junction annotation** module; it allows to evaluate both splice junctions (multiple reads show the same splicing event) and splice events (single read level) (fig. 9).

![Figure 09. RSeQC junction annotation plot](../../images/differential_isoform/rseqc_junction_annotation_junctions_plot.png "RSeQC junction annotation. The detected junctions and events are divided in three exclusive categories: known splicing junctions (blue), partial novel splicing junction (one of the splice site is novel) (grey) and new splicing junctions (green). Splice events refer to the number of times a RNA-read is spliced (A). Splice junctions correspond to multiple splicing events spanning the same intron.")

According to the results, despite the relatively low (around 0.3%) proportion of reads supporting new (or partially new) splicing events (fig. 9.B), they represent a large proportion of junctions (fig. 9.A).

![Figure 10. RSeQC inner distance plot](../../images/differential_isoform/rseqc_inner_distance_plot.png "RSeQC inner distance. The plot represents gap sizes between R1 and R2 so the fragment size minus the size of both reads.")

Finally from the the Inner Distance plot (fig. 10), we can infer some additional information about the degree of degradation of the samples. Usually very short inner distances appear in old or degraded samples; in addition values can be negative if the two fragments overlap. In our case, we can see that most of reads overlap (this was already seen with the fastp report which is more informative) and that the average fragment size is 150bp (-50 inner distance + 100bp read 1 + 100bp read2).

After evaluating the quality of the RNA-seq data, we can start with the transcriptome assembly step.

# Transcriptome assembly, quantification and evaluation

Once the mapping is done, in this section we will use the information contained in the BAM files to carry out the assembly of the transcriptomes, as well as the quantification of the transcripts. In addition, we will evaluate the transcriptome assemblies quality.

## Reference-based transcriptome assembly and quantification with **StringTie**

**StringTie** is a fast and highly efficient assembler of RNA-seq alignments into potential transcripts. It uses a network flow algorithm to reconstruct transcripts and quantitate them simultaneously. This algorithm is combined with an assembly method to merge read pairs into full fragments in the initial phase ({% cite Kovaka2019 %}, {% cite Pertea2015 %}).

> <comment-title>StringTie algorithm</comment-title>
>
> **StringTie** first groups the reads into clusters, collapsing the reads that align to the identical location on the genome and keeping a count of how many alignments were collapsed, then creates a splice graph for each cluster from which it identifies transcripts, and then for each transcript it creates a separate flow network to estimate its expression level using a maximum flow algorithm ({% cite Pertea2015 %}) (fig. 11). The algorithm groups the reads into distinct gene loci and then assembles each locus into as many isoforms as needed to explain the data. It begins with the most highly-expressed transcript, assembling and quantitating it simultaneously. The process is repeated until all reads are used, or until the number of remaining reads is below a user-adjustable level of transcriptional noise.
>
> ![Figure 11. StringTie algorithm details](../../images/differential_isoform/stringtie_algorithm.png "Transcript assembly pipeline for StringTie. It begins with a set of RNA-seq reads that have been mapped to the genome. StringTie iteratively extracts the heaviest path from a splice graph, constructs a flow network, computes maximum flow to estimate abundance, and then updates the splice graph by removing reads that were assigned by the flow algorithm. This process repeats until all reads have been assigned. Source: Perea et al., 2015")
>
> **StringTie** uses an aggressive strategy for identifying and removing spurious spliced alignments. If a spliced read is aligned with more than 1% mismatches, keeping in mind that Illumina sequencers have an error rate < 0.5%, then **StringTie** requires 25% more reads than usual to support that particular spliced alignment. In addition, if a spliced read spans a very long intron, **StringTie** accepts that alignment only if a larger anchor of 25 bp is present on both sides of the splice site. Here the term *anchor* refers to the portion of the read aligned within the exon beginning at the exon-intron boundary ({% cite Kovaka2019 %}).
>
>
{: .comment}

Transcriptome reconstruction, which involves assembling RNA-seq reads into transcription units, is a complex computational task due to three main challenges. Firstly, gene expression levels vary widely, with some genes having low read representation, complicating their identification. Secondly, reads arise from both mature mRNA and incompletely spliced precursor RNA, making it challenging to distinguish mature transcripts. Lastly, the short length of reads coupled with the presence of multiple isoforms per gene makes it difficult to determine the specific isoform responsible for each read ({% cite Garber2011 %}).

Then, let's start with the transcriptome assemby.

> <hands-on-title>Transcriptome assembly with StringTie</hands-on-title>
>
> 1. {% tool [StringTie](toolshed.g2.bx.psu.edu/repos/iuc/stringtie/stringtie/2.2.3+galaxy0) %} with the following parameters:
>    - *"Input options"*: `Short reads`
>        - {% icon param-collection %} *"Input short mapped reads"*: `Mapped collection`
>    - *"Specify strand information"*: `Reverse (RF)`
>    - *"Use a reference file to guide assembly?"*: `Use reference GTF/GFF3`
>        - *"Reference file"*: `Use a file from history`
>            - {% icon param-file %} *"GTF/GFF3 dataset to guide assembly"*: `gencode.hg19.chr10.gtf`
>        - *"Use Reference transcripts only?"*: `No`
>    - In *"Advanced options"*:
>        - "*Minimum junction coverage*": `3`
>
> 2. Rename the generated collection as `Assembled transcripts coordinates`
>
> 3. {% tool [StringTie merge](toolshed.g2.bx.psu.edu/repos/iuc/stringtie/stringtie_merge/2.2.3+galaxy0) %} with the following parameters:
>    - {% icon param-collection %} *"Transcripts"*: `Assembled transcripts coordinates` (output of **StringTie**)
>    - {% icon param-file %} *"Reference annotation to include in the merging"*: `gencode.hg19.chr10.gtf`
>
> 4. Rename the output as `Reference transcriptome annotation`
>
>    > <question-title></question-title>
>    >
>    > 1. How many lines are there in the `Assembled transcripts coordinates`?
>    > 2. How many lines are there in the original GTF?
>    > 3. How many lines are there in the `Reference transcriptome annotation`?
>    > 4. What can you conclude?
>    >
>    >    > <solution-title></solution-title>
>    >    >
>    >    > 1. There are about 25k lines.
>    >    > 2. There are about 93k lines.
>    >    > 3. There are about 70k lines.
>    >    > 4. Not much as the number of lines is not informative enough, we need an appropriate tool to compare those GTFs. However, a quick way to check why these numbers are different is to use **Count occurences of each record** on column 3 which is the feature type. This would tell you that:
>    >    >    - The original GTF has 2260 genes, 6503 transcripts and 42758 exons and other coding information.
>    >    >    - The As

tools_json:
["Regex Find And Replace", "toolshed.g2.bx.psu.edu/repos/galaxyp/regex_find_replace/regex1/1.0.3", "Flatten collection", "__FLATTEN__", "FastQC", "toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.74+galaxy1", "MultiQC", "toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy1", "fastp", "toolshed.g2.bx.psu.edu/repos/iuc/fastp/fastp/0.23.4+galaxy1", "Convert GTF to BED12", "toolshed.g2.bx.psu.edu/repos/iuc/gtftobed12/gtftobed12/357", "Gene BED To Exon/Intron/Codon BED", "gene2exon1", "Compute", "toolshed.g2.bx.psu.edu/repos/devteam/column_maker/Add_a_column1/2.0", "Sort", "sort1", "Text reformatting", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_awk_tool/9.3+galaxy1", "RNA STAR", "toolshed.g2.bx.psu.edu/repos/iuc/rgrnastar/rna_star/2.7.11a+galaxy0", "Concatenate datasets", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/9.3+galaxy1", "Filter", "Filter1", "Cut", "Cut1", "Unique lines", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_uniq_tool/9.3+galaxy1", "Infer Experiment", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_infer_experiment/5.0.3+galaxy0", "Gene body coverage (BAM)", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_geneBody_coverage/5.0.3+galaxy0", "Junction Saturation", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_junction_saturation/5.0.3+galaxy0", "Junction Annotation", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_junction_annotation/5.0.3+galaxy0", "Read Distribution", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_read_distribution/5.0.3+galaxy0", "Inner Distance", "toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_inner_distance/5.0.3+galaxy0", "StringTie", "toolshed.g2.bx.psu.edu/repos/iuc/stringtie/stringtie/2.2.3+galaxy0", "StringTie merge", "toolshed.g2.bx.psu.edu/repos/iuc/stringtie/stringtie_merge/2.2.3+galaxy0", "GFFCompare", "toolshed.g2.bx.psu.edu/repos/iuc/gffcompare/gffcompare/0.12.6+galaxy0", "pyGenomeTracks", "toolshed.g2.bx.psu.edu/repos/iuc/pygenometracks/pygenomeTracks/3.8+galaxy1", "gffread", "toolshed.g2.bx.psu.edu/repos/devteam/gffread/gffread/2.2.2.1.4+galaxy0", "rnaQUAST", "toolshed.g2.bx.psu.edu/repos/iuc/rnaquast/rna_quast/2.2.1", "Extract element identifiers", "toolshed.g2.bx.psu.edu/repos/iuc/collection_element_identifiers/collection_element_identifiers/0.0.2", "Search in textfiles", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/9.3+galaxy1", "Filter collecion", "__FILTER_FROM_FILE__", "toolshed.g2.bx.psu.edu/repos/devteam/gffread/gffread/2.2.1.3+galaxy0", "IsoformSwitchAnalyzeR", "toolshed.g2.bx.psu.edu/repos/iuc/isoformswitchanalyzer/isoformswitchanalyzer/1.20.0+galaxy5", "PfamScan", "toolshed.g2.bx.psu.edu/repos/bgruening/pfamscan/pfamscan/1.6+galaxy0", "CPAT", "toolshed.g2.bx.psu.edu/repos/bgruening/cpat/cpat/3.0.4+galaxy0", "Add column", "addValue", "Remove beginning", "Remove beginning1", "Concatenate multiple datasets or collections", "cat1"]


topic: transcriptomics
tutorial: rna-seq-analysis-with-askomics-it
title: RNA-Seq analysis with AskOmics Interactive Tool
difficulty: intermediate
time_estimation: 2H
tools_json: []
requirements_json: []
tags_json: []
content_hash: e69694830fd46a80b6b1ca9be8d82aab
last_modified: 2026-05-15T22:28:17.646863
zenodo_link: https://zenodo.org/record/3925863
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/rna-seq-analysis-with-askomics-it/tutorial.html

content:
AskOmics is a web application for data integration and querying using the Semantic Web technologies. It helps users to convert multiple data sources (CSV/TSV files, GFF and BED annotation) into "RDF triples" and store them in a specific kind of database: an "RDF triplestore". Under this form, data can then be queried using a specific language: "SPARQL". AskOmics hides the complexity of these technologies and allows to perform complex queries using a user-friendly interface.

<!-- AskOmics for RNA-Seq -->
AskOmics comes in useful for cross-referencing results datasets with various reference data. For example, in RNA-Seq studies, we often need to filter the results on the fold change and the p-value, to get the most significant differentially expressed genes. If you are studying a particular phenotype and already know the position of some QTL associated to this phenotype, you would then want to find the positions of the differentially expressed genes and determine which gene is located within one of those QTL. Finally, you would want to know if these genes have human homologs, and use the neXtProt database to get the location of the proteins coded by the homologs. The whole process involves several tools to parse and manipulate the different data format, and to map datasets on each other. AskOmics offer a solution to 1) automatically convert the multiple formats to RDF, 2) use a user-friendly interface to perform complex SPARQL queries on the RDF datasets to find the genes you are interested in, and 3) connect external SPARQL databases and link external data with your own.

<!-- The data -->
In this tutorial, we will use results from a differential expression analysis. This file is provided for you below. You could also generate the file yourself, by following the [RNA-Seq counts to gene tutorial]({% link topics/transcriptomics/tutorials/rna-seq-counts-to-genes/tutorial.md %}). The file used here was generated from limma-voom but you could use a file from any RNA-seq differential expression tool, such as edgeR or DESeq2, as long as it contains the required columns (see below).

The differentially expressed results will be linked to the official mouse genome annotation, in general feature format (GFF). The file provided is a subset of the mouse annotation (GRCm38.p6) obtained from [Ensembl](http://www.ensembl.org/Mus_musculus/Info/Index).

We will use a file containing quantitative trait loci (QTL) information, to find if our differentially expressed genes are located inside a known QTL. This file is a subset of a query performed on [Mouse Genome Informatics](http://www.informatics.jax.org).

A file containing all homologies between mouse and human will be used to get the human homolog genes. This file is provided by [MGI](http://www.informatics.jax.org/downloads/reports/index.html#homology).

In the differentially expressed file, and the homologs file, gene are described by a symbol (e.g. Pwgrq10). However, in the annotation file and neXtProt database, gene are represented by Ensembl id (e.g. ENSMUSG00000025969). To link the 2 datasets, we will need a file to map the gene symbol with Ensembl id. This file provided for you was previously generated with an AskOmics query on the mouse annotation file and the homolog file.

To link the human gene with neXtProt database, we will use the RDF abstraction of neXtProt. This file was obtained using the [Abstractor](https://github.com/askomics/abstractor) tool.

> <details-title>Abstraction</details-title>
> During the integration step, AskOmics builds an RDF description of the data: the **abstraction**. This abstraction is used to explore the data and build the query.
> AskOmics can also integrate **abstraction** of distant endpoint. Abstraction are obtained using [abstractor](https://github.com/askomics/abstractor), a python package to generate RDF abstractions from distant endpoints.
> The query builder interface is used to create a path through the **abstraction** of each ressources. The path is converted to a SPARQL query that is sent to the multiple SPARQL endpoint.
{: .details}

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Preparing the inputs

We will use four files for this analysis:

 * **Differentially expressed results file** (genes in rows, and 4 required columns: identifier (ENTREZID), gene symbol (SYMBOL), log fold change (logFC) and adjusted P values (adj.P.Val))
 * **Reference genome annotation file** (in GFF format)
 * **QTL file** (QTL in rows, with 5 required columns: identifier, chromosome, start, end and name)
 * **Homolog file** (TSV of 13 columns including homolog id, organism name and gene symbol)
 * **Correspondence file between gene symbol and Ensembl id** (TSV of 3 columns: symbol, the corresponding Ensembl id (mouse and human)
 * **neXtProt abstraction** (RDF data description of neXtProt database in turtle format)

## Import data

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this RNA-seq exercise e.g. `RNA-seq AskOmics`
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 2. Import the files.
>
>    To import the files, there are two options:
>    - Option 1: From a shared data library if available (ask your instructor)
>    - Option 2: From [Zenodo](https://zenodo.org/record/3950862)
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
>    - You can paste the links below into the **Paste/Fetch** box:
>
>      ```
>      https://zenodo.org/record/2529117/files/limma-voom_luminalpregnant-luminallactate
>      https://zenodo.org/record/3950862/files/Mus_musculus.GRCm38.98.subset.gff3
>      https://zenodo.org/record/3950862/files/Symbol.tsv
>      https://zenodo.org/record/3950862/files/MGIBatchReport_Qtl_Subset.txt
>      https://zenodo.org/record/3950862/files/HOM_MouseHumanSequence.rpt
>      https://zenodo.org/record/3950862/files/nextprot_abstraction.ttl
>      ```
>
> 2. Rename the files using the {% icon galaxy-pencil %} (pencil) icon.
>    - limma-voom_luminalpregnant-luminallactate to `DE results`
>    - Mus_musculus.GRCm38.98.subset.gff3 to `Mus musculus annotation`
>    - Symbol.tsv to `Gene Symbols`
>    - MGIBatchReport_Qtl_Subset.txt to `QTL`
>    - HOM_MouseHumanSequence.rpt to `Homolog groups`
>    - nextprot_asbtraction.ttl to `neXtProt abstraction`
>
> 3. Check every datatype.
>    - DE results: `tabular`
>    - Mus musculus annotation: `gff`
>    - Gene Symbol: `tabular`
>    - QTL: `tabular`
>    - Homolog groups: `tabular`
>    - neXtprot abstraction: `ttl`
>
> If the datatypes are wrong, please change it.
>
> {% snippet faqs/galaxy/datasets_change_datatype.md %}
{: .hands_on}

Click on the {% icon galaxy-eye %} (eye) icon and take a look at the uploaded files.

Two step are necessary to get our data converted into RDF triples. The first step is to upload the Galaxy datasets into the AskOmics server. The second step is to integrate the uploaded data into the RDF triplestore.

# Upload inputs into AskOmics

We will first launch an AskOmics interactive tool, and upload the data into it.

## Launch AskOmics Interactive Tool

> <hands-on-title>Launch AskOmics IT</hands-on-title>
> 1. **AskOmics** a visual SPARQL query builder {% icon tool %} to launch the Interactive Tool
>    - {% icon param-file %} *"Datasets to load into AskOmics"*: `DE results`, `Mus musculus annotation`, `Gene Symbols`, `QTL`, `Homolog groups` and `neXtProt abstraction`
{: .hands_on}

Wait a few seconds (or minutes if computing resources are busy) for AskOmics to be ready to use. A **view** link should appear in the confirmation box just after clicking on the **Execute** button.

AskOmics is an *Interactive tool*. It means that when you launch it, it will stay in running state (yellow background) in your History. As long as it stays in this running state, you can access it by looking in the "User" > "Active Interactive Tools" menu (click on its name to view it). When you no longer need it, you can stop it by deleting it from your history, or using the "Stop" button in the "User" > "Active Interactive Tools" page.

Keep in mind that as long as this tool runs, it uses computing resources, so don't forget to stop it when you no longer have use for it.

Once the AskOmics Interactive Tool is ready, you should see a start page looking like this:

![home page](images/home.png "AskOmics home page")

 You can see that there is no data available yet. It's because data needs to be integrated: it is the next step.

# Integrate input files into AskOmics

AskOmics conversion into RDF is called *integration*.

On the **Files** page (link at the top of the page), you will see the files you uploaded from Galaxy. We will now integrate all these files.

> <hands-on-title>Integrate data</hands-on-title>
> 1. Got to the **Files** page
> 2. Select all the input files
> 3. Click on the **Integrate** button
{: .hands_on}

![files page](images/files.png "AskOmics files page")

You will land on the **Integrate** page that shows a preview of the data present in each selected file, depending of its data type.

## Integrate GFF files

The GFF preview shows the entities that the file contains. We need to select the entities we want to be integrated.

The `Mus musculus annotation` file we're using contains gene and mRNA entities, and we will need both in the rest of the tutorial.

> <hands-on-title>Integrate `Mus musculus annotation`</hands-on-title>
> 1. Search for `Mus musculus annotation (preview)`
> 2. Select `gene` and `mRNA`
> 3. Click on the **Integrate (private dataset)** button
> ![mus musculus annotation preview](images/mus_musculus_annotation_preview.png "Mus musculus annotation preview")
{: .hands_on}


## Integration of tabular (TSV) files

The TSV preview shows an HTML table representing the first lines of the TSV file. During integration, AskOmics will convert the file using the header.

<!-- First col: entity, then, attribute -->
The first column of a TSV file will be the *entity* name. Other columns of the file will be *attributes* of the *entity*. *Labels* of the *entity* and *attributes* will be set by the header. Each *label* can be edited by clicking on it.

<!-- Attribute types -->
Entity and attributes can have special types. The types are defined with the select box below the header. An *entity* can be a *start entity* or an *entity*. A *start entity* means that the entity may be used to start a query on the AskOmics homepage.

Attributes can take the following types:
- Numeric: if all the values of the column are numeric
- Text: if all the values are strings
- Category: if there is a limited number of repeated values (e.g. 'green', 'yellow' and 'red', each one found in multiple lines)

If the entity describes a locatable element on a genome:
- Reference: chromosome
- Strand: strand
- Start: start position
- End: end position

<!-- Relation -->
A column can also represent a relation between the *entity* to another. In this case, the header have to be named `relationName@TargetedEntity` and the type *Directed* or *Symetric* relation. A *Directed* relation is a relation from this entity to the targeted one (e.g. A is B's father, but B is not A's father). A *Symetric relation* is a relation that works in both directions (e.g. A loves B, and B loves A).

> <hands-on-title>Integrate `DE results`</hands-on-title>
> 1. Search for `DE results (preview)`
> 2. Edit attribute names and types:
>   - change `ENTREZ ID` to `Differential Expression` and set type to *start entity*
>   - change `SYMBOL` to `linkedTo@Gene Symbol` and set type to *Symetric relation*
>   - change `GENENAME` to `name` and set type to *text*
>   - Keep the other column names and set their types to *numeric*
> 3. **Integrate (private dataset)**
>   ![De results preview](images/de_results_preview.png "DE results preview")
{: .hands_on}

> <hands-on-title>Integrate `Gene symbols`</hands-on-title>
> 1. Search for `Gene symbols (preview)`
> 2. Edit attribute names and types:
>   - change `symbol` to `Gene Symbol` and set type to *entity*
>   - Set `to mouse genes@gene` type to *Symetric relation*
> 3. Click on the **Integrate (private dataset)** button
>   ![Gene symbols preview](images/symbol_to_ensembl_preview.png "Symbol to Ensembl preview")
{: .hands_on}

> <hands-on-title>Integrate `QTL`</hands-on-title>
> 1. Search for `QTL (preview)`
> 2. Edit attribute names and types:
>   - change `Input` to `QTL` and set type to *start entity*
>   - set `Chr` type to *Reference*
>   - set `Start` type to *Start*
>   - set `End` type to *End*
> 3. Click on the **Integrate (private dataset)** button
>   ![QTL preview](images/qtl_preview.png "QTL preview")
{: .hands_on}

> <hands-on-title>Integrate `Homolog groups`</hands-on-title>
> 1. Search for `Homolog groups (preview)`
> 2. Edit attribute names and types:
>   - change `HomoloGene ID` to `Homolog Group` and set type to *start entity*
>   - set `Common Organism Name` type to *category*
>   - change `Symbol` to `linkedTo@Gene Symbol` and set type to *Directed relation*
>   - Keep the other column names and set their types to *text*
> 3. Click on the **Integrate (private dataset)** button
>   ![Homolog groups preview](images/homolog_groups.png "Homolog groups preview")
{: .hands_on}

## Integration of RDF files

The last dataset we want to integrate is the neXtProt abstraction. This file contains some RDF data that instructs AskOmics how to communicate with a remote RDF database containing [neXtProt](https://www.nextprot.org/) data.

> <hands-on-title>Integrate `neXtProt abstraction`</hands-on-title>
> 1. Search for `neXtProt abstraction (preview)`
> 2. Check that **Distant endpoint** is set to `https://sparql.nextprot.org/sparql` in **advanced options**
> 3. Click on the **Integrate (private dataset)** button
>   ![neXtProt abstraction preview](images/nextprot_abstraction_preview.png "neXtProt abstraction preview")
{: .hands_on}

## Monitor integration

Integration can take some time depending on the file size. The **Datasets** page shows the progress.

> <hands-on-title>track integration progress</hands-on-title>
> 1. Go to the **Dataset** page
> 2. Wait for all datasets to be in *success* state
>   ![dataset](images/datasets.png "Datasets table")
{: .hands_on}

# Query

Once all the data of interest is integrated (converted to RDF), its time to query them. Querying RDF data is done by using the SPARQL language. Fortunately, AskOmics provides a user-friendly interface to build SPARQL queries without having to learn the SPARQL language.

## Query builder overview

### Simple query

The first step to build a query is to choose a start point for the query.

![ask](images/ask.png "ask page")


> <hands-on-title>Start a query</hands-on-title>
> 1. Go to the **Ask!** page
> 2. Select the *Differential Expression* entity
> 3. **Start!**
{: .hands_on}


Once the start entity is chosen, the query builder is displayed.


The query builder is composed of a graph. Nodes represents *entities* and links represents *relations* between entities. The selected entity is surrounded by a red circle. Links and other entities are dotted and lighter because there are not instantiated.

![query builder](images/query_builder.png "Query builder, Differential Expression is the selected entity, Gene Symbol is a suggested entity")

On the right, attributes of the selected entity are displayed as attribute boxes. Each box has an eye icon: an opened eye means the attribute will be displayed on the results.

> <hands-on-title>Ask for all Differential Expression and display some attributes</hands-on-title>
> 1. Display `logFC` and `adj.P.val` by clicking on the eye icon
> 2. **Run & preview**
> ![preview results](images/preview_results.png "Results preview")
{: .hands_on}


**Run & preview** launch the query with a limit of 30 rows returned. We use this button to get an idea of the results returned.


### Filter on attributes

Next query will search for all over-expressed genes. Genes are considered over-expressed if the log fold change is > 2. We are only interested by significant results (Adj P value ≤ 0.05)

Back to the query builder,

> <hands-on-title>Filter attributes to get significant over-expressed genes</hands-on-title>
> 1. Filter `logFC` with `> 2`
> 2. Filter `adj.P.val` with `≤ 0.05`
> 2. **Run & preview**
> ![preview filtered results](images/preview_results_filtered.png "Filtered results preview")
{: .hands_on}

The preview shows only significantly over-expressed genes.

### Filter on relations

Now that we have found our genes of interest, we will link these genes to the reference genome to get information about their location.

To constraint on relation, we have to click on suggested nodes, linked to our entity of interest.

> <hands-on-title>Link `De results` to `gene` using `Gene Symbol`</hands-on-title>
> 1. First, hide `Label`, `logFC` and `adj.P.val` of `Differential Expression` using the eye icon
> 2. Instantiate `Gene Symbol` by clicking on the suggested node, and hide his `Label` using the eye icon
> 3. Instantiate `gene` by clicking on the `gene` node
> 2. **Run & preview**
> ![gene query](images/gene_query.png "Gene query")
{: .hands_on}

Results now show the Ensembl id of our over-expressed genes. We have now access to all the information about the `gene` entity contained in the GFF file. For example, we can filter on chromosome and display chromosome and strand to get information about gene location.

> <hands-on-title>Filter `gene`</hands-on-title>
> 1. Show `reference` and `strand` using the eye icon
> 2. Filter `reference` by selecting `X` chromosome
> 3. Filter `strand` by selecting `+` strand
> 4. **Run & preview**
> ![gene filter](images/gene_filter.png "Gene filtering")
{: .hands_on}


### Query on the position of elements on the genome.

AskOmics is able to perform special queries between entities that are locatable. These queries are:

- Entities overlapping another one
- Entities included in another entity

> <details-title>FALDO ontology</details-title>
> The [FALDO](https://bioportal.bioontology.org/ontologies/FALDO) ontology describes sequence feature positions and regions. AskOmics uses FALDO ontology to represent entity positions. GFF are using FALDO, as well as TSV entities with chromosome, strand, start and end.
{: .details}

On the query builder interface, locatable entities are represented with a green circle and relations based on location are represented as green arrow.

> <hands-on-title>Filter `gene`</hands-on-title>
> 1. First, remove the reference filter (unselect `X` using `ctrl`+`click`)
> 1. Remove the strand filter (unselect `+` using `ctrl`+`click`)
> 2. Hide `reference` `strand` using the eye
> 3. Instantiate `QTL`
> 4. Click on the link between `gene` and `QTL` to edit the relation
> 5. Check that the relation is `gene` `included in` `QTL` `on the same reference` with `strict` ticked
> 6. **Run & preview**
> ![FALDO query](images/query_builder_position.png "FALDO query")
{: .hands_on}


To go further, we can filter on `QTL` to refine the results.


> <hands-on-title>Filter `gene`</hands-on-title>
> 1. Go back to the `QTL` node
> 2. Show the `Name` attribute using the eye icon
> 3. Filter the name with a `regexp` with `growth`
> 4. **Run & preview**
{: .hands_on}

From now, our query is "All Genes that are over-expressed (logFC > 2 and FDR ≤ 0.05) and located on a QTL that is related to growth". We can save this results with the *Run & save* button.

> <hands-on-title>Save a result</hands-on-title>
> 1. **Run & save**
{: .hands_on}

### Use neXtProt distant data to refine results

[neXtProt](https://www.nextprot.org) is a comprehensive human-centric discovery platform, offering its users a seamless integration of and navigation through protein-related data. It offer a [SPARQL endpoint](https://sparql.nextprot.org/sparql) that can be interrogated with AskOmics.

Since we added the neXtProt abstraction into our AskOmics instance, we can link our data to neXtProt.

> <hands-on-title>Find human homolog genes</hands-on-title>
> 1. Go back to the `gene` node
> 2. instantiate `Gene Symbol` and hide his `Label`
> 3. Instantiate `Homolog Group`, hide his label and filter his `Common Organism Name` with `human`
> ![Organism name filter](images/filter_organism_name.png "Organism name filter")
> 4. From `Homolog Group`, instantiate another `Gene Symbol` and hide his `Label`
> 5. Finally, follow the `to neXtProt Gene` link and instantiate `Gene` (with a capital G)
> 5. **Run & preview**
{: .hands_on}

The query we've just built asks for the human homologs of our over-expressed genes. We use the `Gene Symbol` to get information from the neXtProt database. AskOmics converts the query into small SPARQL subqueries and send them to the local database and to the remote neXtProt endpoint.

Now we are linked to the neXtProt database, we can obtain information about the proteins encoded by these genes, as well as their location in the cell.

> <hands-on-title>Get the protein and their location</hands-on-title>
> 1. Instantiate `Entry`
> 2. Instantiate `Isoform` and hide the `Label`
> 3. Many nodes are connected to `Isoform`. Use the *Filter links* field to filter nodes linked with a link named `location`
> ![Filter links](images/filter_links.png "Filter links")
> 4. Instantiate the `Subcellular Location` node and hide `Uri`
> 5. Use the *Filter node* field to filter nodes with "location" in their name
> 6. Instantiate `Uniprot subcellular Location CV` (you can use the node filter to clear up the screen)
> 7. **Run & preview**
{: .hands_on}

Finally, our query is "All genes that are over-expressed and located on a QTL that is related to growth, their human homologs and the location of the proteins coded by this genes". We will save it to the results.

> <hands-on-title>Save a result</hands-on-title>
> 1. **Run & save**
{: .hands_on}


## Results management

The results page displays the saved queries. Queries are sorted by creation date. At the end of the table, action buttons can be used to preview the result, download or send it to Galaxy history.

> <hands-on-title>Edit query name</hands-on-title>
> 1. Go to the *Results* page
> 2. Use the *Preview* button to check the result
> 3. Click on the name to rename the two query with `Over-expressed genes on a growth QTL` and `Over-expressed genes on a growth QTL, their human homologs and protein location` (press enter key to validate)
{: .hands_on}

![preview table](images/results_table.png "Results table")

The **Action** column contain button to perform certain action:

- Preview: show a results preview on the bottom of the table
- Download: Download the results (TSV file)
- Edit: Edit the query with the query builder
- SPARQL: edit the query with a SPARQL editor for advanced users
- Send results to Galaxy: send the results (TSV file) to the most recently used Galaxy history
- Send query to Galaxy: send the query representation (json file) to the most recently used Galaxy history

> <hands-on-title>Send results to Galaxy</hands-on-title>
> 1. Click on **Send results to Galaxy** on each query to send them to the last used Galaxy history
> 2. Get back to galaxy and wait for the dataset (reload if needed)
{: .hands_on}

![history results](images/galaxy_history_results.png "Galaxy history with the two results")

Now that you have used AskOmics to generate this final tabular file, you can continue analysing it with other Galaxy tools.
If you are done, don't forget to close the AskOmics instance by going to the “User” > “Active Interactive Tools” page.

# Conclusion


In this tutorial we have seen how to use AskOmics Interactive Tool. We launch the tools with a set of input files, then we have integrated these files into RDF and finally, we built complex queries over this local datasets and neXtProt to answer a biological question.


topic: transcriptomics
tutorial: rna-seq-counts-to-genes
title: 2: RNA-seq counts to genes
difficulty: intermediate
time_estimation: 2h
tools_json: []
requirements_json: []
tags_json: ["limma-voom", "mouse", "QC"]
content_hash: b1ebd5eacb05d1434dfd55031fa49949
last_modified: 2026-05-15T22:28:17.648111
zenodo_link: https://zenodo.org/record/4273218
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/rna-seq-counts-to-genes/tutorial.html

content:
Measuring gene expression on a genome-wide scale has become common practice over the last two decades or so, with microarrays predominantly used pre-2008. With the advent of next generation sequencing technology in 2008, an increasing number of scientists use this technology to measure and understand changes in gene expression in often complex systems. As sequencing costs have decreased, using RNA-Seq to simultaneously measure the expression of tens of thousands of genes for multiple samples has never been easier. The cost of these experiments has now moved from generating the data to storing and analysing it.

There are many steps involved in analysing an RNA-Seq experiment. The analysis begins with sequencing reads (FASTQ files). These are usually aligned to a reference genome, if available. Then the number of reads mapped to each gene can be counted. This results in a table of counts, which is what we perform statistical analyses on to determine differentially expressed genes and pathways. The purpose of this tutorial is to demonstrate how to perform differential expression on count data with **limma-voom**. How to generate counts from reads (FASTQs) is covered in the accompanying tutorial [RNA-seq reads to counts]({% link topics/transcriptomics/tutorials/rna-seq-reads-to-counts/tutorial.md %}).

## Mouse mammary gland dataset

The data for this tutorial comes from a Nature Cell Biology paper by {% cite Fu2015 %}. Both the raw data (sequence reads) and processed data (counts) can be downloaded from Gene Expression Omnibus database (GEO) under accession number [GSE60450](http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE60450).

This study examined the expression profiles of basal and luminal cells in the mammary gland of virgin, pregnant and lactating mice. Six groups are present, with one for each combination of cell type and mouse status. Note that two biological replicates are used here, two independent sorts of cells from the mammary glands of virgin, pregnant or lactating mice, however three replicates is usually recommended as a minimum requirement for RNA-seq. In this tutorial we will use the GEO counts file as a starting point for our analysis. Alternatively, you could create a count matrix from the raw sequence reads, as demonstrated in the [RNA-seq reads to counts tutorial]({% link topics/transcriptomics/tutorials/rna-seq-reads-to-counts/tutorial.md %}). The GEO count file was generated from aligning the reads to the mouse `mm10` genome with the Rsubread aligner ({% cite Liao2019 %}), followed by counting reads mapped to RefSeq genes with featureCounts ({% cite liao2013featurecounts %}), see the [Fu paper](https://www.nature.com/articles/ncb3117) for details.

We will use **limma-voom** ({% cite Law2014 %}) for identifying differentially expressed genes here. Other popular alternatives are edgeR ({% cite robinson2010edger %}) and DESeq2 ({% cite love2014moderated %}). Limma-voom has been shown to be perform well in terms of precision, accuracy and sensitivity ({% cite CostaSilva2017 %}) and, due to its speed, it's particularly recommended for large-scale datasets with 100s of samples ({% cite Chen2016 %}).

This is a Galaxy tutorial based on material from the [COMBINE R RNAseq workshop](http://combine-australia.github.io/RNAseq-R/06-rnaseq-day1.html), first taught [in this course](http://combine-australia.github.io/2016-05-11-RNAseq/).

![Tutorial Dataset](../../images/rna-seq-reads-to-counts/mouse_exp.png "Tutorial Dataset")


> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}


{% snippet faqs/galaxy/analysis_results_may_vary.md %}

# Preparing the inputs

We will use three files for this analysis:

 * **Count matrix** (genes in rows, samples in columns)
 * **Sample information** file (sample id, group)
 * **Gene annotation** file (gene id, symbol, description)

## Import data

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this RNA-seq exercise e.g. `RNA-seq with limma-voom`
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 2. Import the mammary gland counts table and the associated sample information file.
>
>    To import the files, there are two options:
>    - Option 1: From a shared data library if available (`GTN - Material -> {{ page.topic_name }} -> {{ page.title }}`)
>    - Option 2: From [Zenodo](https://zenodo.org/record/4273218)
>
>      {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    - You can paste both links below into the **Paste/Fetch** box:
>
>    ```
>    https://zenodo.org/record/4273218/files/countdata.tsv
>    https://zenodo.org/record/4273218/files/factordata.tsv
>    ```
>
> 2. Rename the counts dataset as `countdata` and the sample information dataset as `factordata` using the {% icon galaxy-pencil %} (pencil) icon.
> 3. Check that the datatype is `tabular`.
>    If the datatype is not `tabular`, please change the file type to `tabular`.
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="tabular" %}
>
{: .hands_on}


Let’s take a look at the data. The `countdata` file contains information about genes (one gene per row), the first column has the Entrez gene id and the remaining columns contain information about the number of reads aligning to the gene in each experimental sample. There are two replicates for each cell type and time point (detailed sample info can be found in file “GSE60450_series_matrix.txt” from the GEO website). The first few rows and columns of the seqdata file are shown below.

![countdata file](../../images/rna-seq-counts-to-genes/countdata.png "Count file")

The `factordata` file contains basic information about the samples that we will need for the analysis. See below. Note that the sample ids must match exactly with the ids in the counts file.

![factordata file](../../images/rna-seq-counts-to-genes/factordata.png "Sample information file")


> <details-title>Formatting the data</details-title>
>
> The files above have been formatted for you. If you are interested to know how they were formatted the information is below.
>
> The original files are available at
>
> ```
> https://zenodo.org/record/4273218/files/GSE60450_Lactation-GenewiseCounts.txt
> https://zenodo.org/record/4273218/files/SampleInfo.txt
> ```
>
>![seqdata file](../../images/rna-seq-counts-to-genes/seqdata.png "Count file (before formatting)")
>
>![sampleinfo file](../../images/rna-seq-counts-to-genes/sampleinfo.png "Sample information file (before formatting)")
>
>To create the file, `countdata`, that contains only the counts for the 12 samples i.e. we'll remove the gene length column with the **Cut columns from a table (cut)** tool. The sample names are also pretty long so we'll use the **>Replace Text in entire line** tool to shorten these to contain only the relevant information about each sample. We will also replace the hyphen in the sample names with a dot so they match the names in the sample information file.
>
>> <hands-on-title>Format the counts data</hands-on-title>
>>
>> 1. **Cut columns from a table (cut)** {% icon tool %} with the following parameters:
>>      - {% icon param-file %} *"File to cut"*: `seqdata`
>>      - {% icon param-select %} *"Operation"*: `Discard`
>>      - {% icon param-select %} *"List of fields"*: Select `Column:2`
>> 2. **Replace Text in entire line** {% icon tool %} with the following parameters:
>>      - {% icon param-file %} *"File to process"*: output of **Cut** {% icon tool %}
>>      - {% icon param-text %} *"Find pattern"*: `_B[A-Z0-9_]+`
>> 3. **Replace Text in entire line** {% icon tool %} with the following parameters:
>>      - {% icon param-file %} *"File to process"*: output of **Replace Text** {% icon tool %}
>>      - {% icon param-text %} *"Find pattern"*: `-`
>>      - {% icon param-text %} *"Replace with"*: `.`
>> 4. Rename file as `countdata` using the {% icon galaxy-pencil %} (pencil) icon.
>{: .hands_on}
>
>To create the file, `factordata`, that contains the groups information that we need for the limma-voom tool. We'll combine the cell type and mouse status to make 6 groups e.g. we'll combine the CellType `basal` with the Status `pregnant` for the group `basalpregnant`. We'll use the **Merge Columns** tool to combine the cell type and mouse status columns in the sample information file, making a column with the 6 group names.
>
>> <hands-on-title>Format the sample information file</hands-on-title>
>>
>> 1. **Merge Columns together** {% icon tool %} with the following parameters:
>>      - {% icon param-file %} *"Select data"*: `sampleinfo`
>>      - {% icon param-select %} *"Merge column"*: `Column: 3`
>>      - {% icon param-select %} *"with column"*: `Column: 4`
>> 2. **Cut columns from a table (cut)** {% icon tool %} with the following parameters:
>>      - {% icon param-file %} *"File to cut"*: output of **Merge Columns** {% icon tool %}
>>      - {% icon param-select %} *"Operation"*: `Keep`
>>      - {% icon param-select %} *"List of fields"*: Select `Column:2` and `Column:5`
>> 3. Rename file as `factordata` using the {% icon galaxy-pencil %} (pencil) icon.
>{: .hands_on}
{: .details}

## Get gene annotations

Optionally, gene annotations can be provided to the limma-voom tool and if provided the annotation will be available in the output files. We'll get gene symbols and descriptions for these genes using the Galaxy **annotateMyIDs** tool, which provides annotations for human, mouse, fruitfly and zebrafish.

> <hands-on-title>Get gene annotations</hands-on-title>
>
> 1. **annotateMyIDs** {% icon tool %} with the following parameters:
>      - {% icon param-file %} *"File with IDs"*: `countdata`
>      - {% icon param-check %} *"File has header"*: `Yes`
>      - {% icon param-select %} *"Organism"*: `Mouse`
>      - {% icon param-select %} *"ID Type"*: `Entrez`
>      - {% icon param-check %} "*Output columns"*: tick
>          - `ENTREZID`
>          - `SYMBOL`
>          - `GENENAME`
> 2. Rename file as `annodata` using the {% icon galaxy-pencil %} (pencil) icon. The file should look like below.
>    ![annodata file](../../images/rna-seq-counts-to-genes/annodata.png "Gene annotation file")
> 3. Check the number of lines shown on the datasets in the history, there should be 27,180 lines in both. There must be the same number of lines (rows) in the counts and annotation.
{: .hands_on}

# Differential expression with limma-voom

## Filtering to remove lowly expressed genes

It is recommended to filter for lowly expressed genes when running the limma-voom tool. Genes with very low counts across all samples provide little evidence for differential expression and they interfere with some of the statistical approximations that are used later in the pipeline. They also add to the multiple testing burden when estimating false discovery rates, reducing power to detect differentially expressed genes. These genes should be filtered out prior to further analysis.

There are a few ways to filter out lowly expressed genes. When there are biological replicates in each group, in this case we have a sample size of 2 in each group, we favour filtering on a minimum counts-per-million (CPM) threshold present in at least 2 samples. Two represents the smallest sample size for each group in our experiment. In this dataset, we choose to retain genes if they are expressed at a CPM above 0.5 in at least two samples. The CPM threshold selected can be compared to the raw count with the CpmPlots (see below).

> <details-title>More details on filtering</details-title>
>
> The limma tool uses the `cpm` function from the edgeR package ({% cite robinson2010edger %}) to generate the CPM values which can then be filtered. Note that by converting to CPMs we are normalizing for the different sequencing depths for each sample. A CPM of 0.5 is used as it corresponds to a count of 10-15 for the library sizes in this data set. If the count is any smaller, it is considered to be very low, indicating that the associated gene is not expressed in that sample. A requirement for expression in two or more libraries is used as each group contains two replicates. This ensures that a gene will be retained if it is only expressed in one group. Smaller CPM thresholds are usually appropriate for larger libraries. As a general rule, a good threshold can be chosen by identifying the CPM that corresponds to a count of 10, which in this case is about 0.5. You should filter with CPMs rather than filtering on the counts directly, as the latter does not account for differences in library sizes between samples.
{: .details}

## Normalization for composition bias

In an RNA-seq analysis, the counts are normalized for different sequencing depths between samples. Normalizing to eliminate composition biases between samples is also typically performed. Composition biases can occur, for example, if there are a few highly expressed genes dominating in some samples, leading to less reads from other genes. By default, TMM normalization ({% cite robinson2010scaling %}) is performed by the limma tool using the edgeR `calcNormFactors` function (this can be changed under **Advanced Options**). TMM stands for Trimmed Mean of M values, where a weighted trimmed mean of the log expression ratios is used to scale the counts for the samples. See the figure from the TMM paper below. Note the plot (Figure 1c) that shows how a few highly expressed genes in the liver sample (where the arrow is) results in the majority of other genes in the sample having the appearance of being expressed lower in liver. The mid-line through the points is offset from the expected zero and the TMM normalization factor (red line) scales the counts to adjust for this.

![TMM normalization](../../images/rna-seq-counts-to-genes/TMM.png "TMM normalization (Robinson and Oshlack 2010)")

## Specify Contrast(s) of interest

Since we are interested in differences between groups, we need to specify which comparisons we want to test. For example, if we are interested in knowing which genes are differentially expressed between the pregnant and lactating group in the basal cells we specify `basalpregnant-basallactate` for the *Contrast of Interest*. Note that the group names in the contrast must exactly match the names of the groups in the `factordata` file. More than one contrast can be specified using the `Insert Contrast` button, so we could look at more comparisons of the groups here, but first we'll take a look at `basalpregnant-basallactate`.

> <hands-on-title>Differential expression with limma-voom</hands-on-title>
>
> 1. **limma** {% icon tool %} with the following parameters:
>      - {% icon param-file %} *"Differential Expression Method"*: `limma-voom`
>      - {% icon param-select %} *"Count Files or Matrix?*": `Single Count Matrix`
>          - {% icon param-file %} *"Count Matrix"*: Select `countdata`
>      - {% icon param-select %} *"Input factor information from file?"*: `Yes`
>          - {% icon param-file %} *"Factor File"*: Select `factordata`
>      - {% icon param-select %} *"Use Gene Annotations?"*: `Yes`
>          - {% icon param-file %} *"Factor File"*: Select `annodata`
>      - {% icon param-text %} *"Contrast of Interest"*: `basalpregnant-basallactate`
>      - {% icon param-select %} *"Filter lowly expressed genes?"*: `Yes`
>          - {% icon param-select %} *"Filter on CPM or Count values?"*: `CPM`
>          - {% icon param-text %} *"Minimum CPM"*: `0.5`
>          - {% icon param-text %} *"Minimum Samples"*: `2`
> 2. Inspect the `Report` produced by clicking on the {% icon galaxy-eye %} (eye) icon
{: .hands_on}


> <tip-title>Multiple factors</tip-title>
>
> If we need to account for additional sources of variation, for example, batch, sex, genotype etc, we can input that information as additional factors. For example, if we were interested in the genes differentially expressed between the luminal and basal cell types, we could include an additional column to account for the variation due to the different stages.
> ![Multiple factors](../../images/rna-seq-counts-to-genes/multiplefactors_factordata.png "Multiple factors")
>
{: .tip}


# QC of count data

Before we check out the differentially expressed genes, we can look at the `Report` information to check that the data is good quality and that the samples are as we would expect.

## Multidimensional scaling plot

By far, one of the most important plots we make when we analyse RNA-Seq data are MDS plots. An MDS plot is a visualisation of a principal components analysis, which determines the greatest sources of variation in the data. A principal components analysis is an example of an unsupervised analysis, where we don’t need to specify the groups. If your experiment is well controlled and has worked well, what we hope to see is that the greatest sources of variation in the data are the treatments/groups we are interested in. It is also an incredibly useful tool for quality control and checking for outliers. This Galaxy limma tool outputs an MDS plot by default in the `Report` and a link is also provided to a PDF version (`MDSPlot_CellTypeStatus.pdf`). A scree plot is also produced that shows how much variation is attributed to each dimension. If there was a batch effect for example, you may see high values for additional dimensions, and you may choose to include batch as an additional factor in the differential expression analysis. The limma tool plots the first two dimensions by default (1 vs 2), however you can also plot additional dimensions 2 vs 3 and 3 vs 4 using under **Output Options** *Additional Plots* `MDS Extra`. These are displayed in the `Report` along with a link to a PDF version (MDSPlot_extra.pdf). Selecting the `Glimma Interactive Plots` will generate an interactive version of the MDS plot, see the plots section of the report below. If outlier samples are detected you may decide to remove them. Alternatively, you could downweight them by choosing the option in the limma tool *Apply voom with sample quality weights?*. The voom sample quality weighting is described in the paper *Why weight? Modelling sample and observational level variability improves power in RNA-seq analyses ({% cite Liu2015 %})*.

![MDS Plot](../../images/rna-seq-counts-to-genes/mdsscree.png "MDS Plot")

Take a look at the MDS plot coloured by group.

> <question-title></question-title>
>
> Do you notice anything about the samples in this plot?
>
>    > <solution-title></solution-title>
>    >
>    > Two samples don't appear to be in the right place.
>    >
>    {: .solution}
{: .question}

It turns out that there has been a mix-up with two samples, they have been mislabelled in the sample information file. This shows how the MDS plot can also be useful to help identify if sample mix-ups may have occurred. We need to redo the limma-voom analysis with the correct sample information.

> <hands-on-title>Use the Rerun button to redo steps</hands-on-title>
>
> 1. Import the correct sample information file from `https://zenodo.org/record/4273218/files/factordata_fixed.tsv` and name as `factordata_fixed`
> 2. Use the **Rerun** {% icon galaxy-refresh %} button in the History to run **limma** {% icon tool %} as before with the `factordata_fixed` file and adding the following parameters:
>      - **Output Options**
>          - {% icon param-check %} *"Additional Plots"* tick:
>              - `Glimma Interactive Plots`
>              - `Density Plots (if filtering)`
>              - `CpmsVsCounts Plots (if filtering on cpms)`
>              - `Box Plots (if normalising)`
>              - `MDS Extra (Dims 2vs3 and 3vs4)`
>              - `MD Plots for individual samples`
>              - `Heatmaps (top DE genes)`
>              - `Stripcharts (top DE genes)`
> 3. Delete the incorrect `factordata` file and its limma outputs to avoid any confusion.
{: .hands_on}

In the `Report` you should then see the correct MDS plot as below.

![MDS Plot](../../images/rna-seq-counts-to-genes/mdsscree_corr.png "MDS Plot (correct samples)")


> <question-title></question-title>
>
> What is the greatest source of variation in the data (i.e. what does dimension 1 represent)?
> What is the second greatest source of variation in the data?
>
>    > <solution-title></solution-title>
>    >
>    > Dimension 1 represents the variation due to cell type, basal vs luminal. Dimension 2 represents the variation due to the stages, virgin, pregnant or lactating.
>    >
>    {: .solution}
{: .question}

Next, scroll down the `Report` to take a look at the **Additional information** and **Summary of experimental data** sections near the bottom. It should look similar to below. Here you can check that the correct samples have been assigned to the correct groups, what settings were used (e.g. filters, normalization method) and also how many genes were filtered out due to low expression.

![Report summary](../../images/rna-seq-counts-to-genes/report_summary.png){: width="600px"}

> <question-title></question-title>
>
> How many genes have been filtered out for low expression?
>
>    > <solution-title></solution-title>
>    >
>    > 11375 genes were filtered out as insignificant as they were without more than 0.5 CPM in at least 2 samples.
>    >
>    {: .solution}
{: .question}


## Density plots

Density plots can be output in the `Report` if *Filter lowly expressed genes* is selected. A link is also provided in the `Report` to a PDF version (`DensityPlots.pdf`). These plots allow comparison of the counts distributions before and after filtering. The samples are coloured by the groups. Count data is not normally distributed, so if we want to examine the distributions of the raw counts we need to log the counts. We typically check the distribution of the read counts on the log2 scale. A CPM value of 1 is equivalent to a log-CPM value of 0 and the CPM we used of 0.5 is equivalent to a log-CPM of -1. It can be seen in the Raw counts (before filtering) plot below, that a large proportion of genes within each sample are not expressed or lowly-expressed, and the Filtered counts plot shows our filter of CPM of 0.5 (in at least 2 samples) removes a lot of these uninformative genes.

![Density Plots](../../images/rna-seq-counts-to-genes/densityplots.png "Density Plots")

The `Report` provides links to PDFs of all plots shown in the `Report` and also to the rest of the additional plots selected to be output.

![Report Outputs](../../images/rna-seq-counts-to-genes/report_plots.png "Report outputs")

> <details-title>Cpm plots</details-title>
>
>We can also have a look more closely to see whether our threshold of 0.5 CPM does indeed correspond to a count of about 10-15 reads in each sample with the plots of CPM versus raw counts.
>
>Click on the `CpmPlots.pdf` link in the `Report`. You should see 12 plots, one for each sample. Two of the plots are shown below. From these plots we can see that 0.5 CPM is equivalent to ~10 counts in each of the 12 samples, so 0.5 seems to be an appropriate threshold for this dataset (these samples all have sequencing depth of 20-30 million, see the `Library information` file below, so a CPM value of 0.5 would be ~10 counts).
>
>![CPM threshold Plots](../../images/rna-seq-counts-to-genes/cpmsvscounts.png "CPM vs Raw Counts Plots")
>
>> <comment-title>Thresholds</comment-title>
>>
>> * A threshold of 1 CPM in at least minimum group sample size is a good rule of thumb for samples with about 10 million reads. For larger library sizes increase the CPM theshold and for smaller library sizes decrease it. Check the CpmPlots to see if the selected threshold looks appropriate for the samples (equivalent to ~10 reads).
>>
>{: .comment}
{: .details}


## Box plots

We can also use box plots to check the distributions of counts in the samples. Box plots can be selected to be output by the Galaxy limma-voom tool if normalization is applied (TMM is applied by default). The plots are output in the `Report` and a link is also provided to a PDF version (`BoxPlots.pdf`). The samples are coloured by the groups. With the box plots for these samples we can see that overall the distributions are not identical but still not very different. If a sample is really far above or below the blue horizontal line we may need to investigate that sample further.

![Box Plots](../../images/rna-seq-counts-to-genes/boxplots.png "Box Plots")

> <question-title></question-title>
>
> Compare the box plots before and after TMM normalisation. Can you see any differences?
>
>    > <solution-title></solution-title>
>    >
>    > After the normalization more of the samples are closer to the median horizontal line.
>    >
>    {: .solution}
{: .question}


> <details-title>Normalization factors</details-title>
>
>The TMM normalization generates normalization factors, where the product of these factors and the library sizes defines the effective library size. TMM normalization (and most scaling normalization methods) scale relative to one sample. The normalization factors multiply to unity across all libraries. A normalization factor below one indicates that the library size will be scaled down, as there is more suppression (i.e., composition bias) in that library relative to the other libraries. This is also equivalent to scaling the counts upwards in that sample. Conversely, a factor above one scales up the library size and is equivalent to downscaling the counts. We can see the normalization factors for these samples in the `Library information` file if we select to output it with *"Output Library information file?"*: `Yes`. Click on the {% icon galaxy-eye %} (eye) icon to view.
>
>![Library Info file](../../images/rna-seq-counts-to-genes/libinfo.png "Library information file")
>
>> <question-title></question-title>
>>
>> Which sample has the largest normalization factor? Which sample has the smallest?
>>
>>    > <solution-title></solution-title>
>>    >
>>    > MCL1.LA has the largest normalization factor and MCL1.LE the smallest.
>>    >
>>    {: .solution}
>{: .question}
{: .details}

> <details-title>MD plots for samples</details-title>
>
>It is considered good practice to make mean-difference (MD) plots for all the samples as a quality check, as described in this [edgeR workflow article](https://f1000research.com/articles/5-1438/v2). These plots allow expression profiles of individual samples to be explored more closely. An MD plot shows the log-fold change between a sample against the average expression across all the other samples. This visualisation can help you see if there are genes highly upregulated or downregulated in a sample. If we look at mean difference plots for these samples, we should be able to see the composition bias problem. The mean-difference plots show average expression (mean: x-axis) against log-fold-changes (difference: y-axis).
>
>Click on the `MDPlots_Samples.pdf` link in the `Report`. You should see 12 MD plots, one for each sample. Let's take a look at the plots for the two samples MCL1.LA and MCL1.LE that had the largest and smallest normalization factors. The MD plots on the left below show the counts normalized for library size and the plots on the right show the counts after the TMM normalization has been applied. MCL1.LA had the largest normalization factor and was above the median line in the unnormalized by TMM box plots. MCL1.LE had the smallest normalization factor and was below the median line in the box plots. These MD plots help show the composition bias problem has been addressed.
>![MD Plot LA](../../images/rna-seq-counts-to-genes/mdsampleLA.png "MD Plots for MCL1.LA before and after TMM normalization")
>![MD Plot LE](../../images/rna-seq-counts-to-genes/mdsampleLE.png "MD Plots for MCL1.LE before and after TMM normalization")
{: .details}

## Voom variance plot

This plot is generated by the voom method and displayed in the `Report` along with a link to a PDF version (`VoomPlot.pdf`). Each dot represents a gene and it shows the mean-variance relationship of the genes in the dataset. This plot can help show if low counts have been filtered adequately and if there is a lot of variation in the data, as shown in the **More details on Voom variance plots** box below.

![Voom Plot](../../images/rna-seq-counts-to-genes/voomplot.png "Voom Plot")

The SA plot below plots log2 residual standard deviations against mean log-CPM values. The average log2 residual standard deviation is marked by a horizontal blue line. This plot shows how the dependence between the means and variances has been removed after the voom weights are applied to the data.

![SA Plot](../../images/rna-seq-counts-to-genes/saplot.png "SA Plot")

> <details-title>More details on Voom variance plots</details-title>
>
> If we didn't filter this dataset for the lowly expressed genes the variance plot would look like below.
>
>   ![Voom Nofilter Plot](../../images/rna-seq-counts-to-genes/voomplot_nofilt.png "Voom Plot unfiltered counts")
>
> If we look at the plot generated with the two samples mixed up we can see there's more variation.
>
>   ![Voom Mixedup Plot](../../images/rna-seq-counts-to-genes/voomplot_mixed.png "Voom Plot mixed-up samples")
>
> More examples of the variation this plot can show can be seen in Figure 1 from the limma-voom paper ({% cite Law2014 %}), shown below.
>
>   ![Voom Plot Examples](../../images/rna-seq-counts-to-genes/voom_variance_examples.jpg "Voom Plot more examples")
>
>   *Figure 1: Mean-variance relationships. Gene-wise means and variances of RNA-seq data are represented by black points with a LOWESS trend. Plots are ordered by increasing levels of biological variation in datasets. (a) voom trend for HBRR and UHRR genes for Samples A, B, C and D of the SEQC project; technical variation only. (b) C57BL/6J and DBA mouse experiment; low-level biological variation. (c) Simulation study in the presence of 100 upregulating genes and 100 downregulating genes; moderate-level biological variation. (d) Nigerian lymphoblastoid cell lines; high-level biological variation. (e) Drosophila melanogaster embryonic developmental stages; very high biological variation due to systematic differences between samples. (f) LOWESS voom trends for datasets (a)–(e). HBRR, Ambion’s Human Brain Reference RNA; LOWESS, locally weighted regression; UHRR, Stratagene’s Universal Human Reference RNA*.
{: .details}

## MD and Volcano plots for DE results

Genome-wide plots that are useful for checking differentially expressed (DE) results are MD plots (or MA plots) and Volcano plots. There are functions in limma for generating these plots and they are used by this tool. These plots are output by default and shown in the `Report` along with a link to PDF versions (`MDPlot_basalpregnant-basallactate.pdf` and `VolcanoPlot_basalpregnant-basallactate.pdf`). In the volcano plot the top genes (by adjusted p-value) are highlighted. The number of top genes is 10 by default and the user can specify the number of top genes to view (up to 100) under **Advanced Options**. For further information on volcano plots see the [Volcano Plot tutorial]({{ site.baseurl }}/topics/transcriptomics/tutorials/rna-seq-viz-with-volcanoplot/tutorial.html)

![MDVol Plot](../../images/rna-seq-counts-to-genes/mdvolplot_basalpregnant-basallactate.png "MD Plot and Volcano Plot")

The MD Plot highlighted genes are significant at an adjusted p-value (adj.P) threshold of 0.05 and exhibit log2-fold-change (lfc) of at least 0. These thresholds can be changed under **Advanced Options**.


> <question-title></question-title>
>
> How many genes are differentially expressed at the default thresholds of adj.P=0.05 and lfc=0?
>
>    > <solution-title></solution-title>
>    >
>    > The number of DE genes at these adj.P and lfc thresholds is shown in the table in the `Report` as below.
>    >
>    > ![DE counts](../../images/rna-seq-counts-to-genes/decounts.png){: width="500px"}
>    >
>    {: .solution}
{: .question}


> <comment-title>on adjusted P value</comment-title>
>
> **A note about deciding how many genes are significant**: In order to decide which genes are differentially expressed, we usually take a cut-off (e.g. 0.05 or 0.01) on the adjusted p-value, NOT the raw p-value. This is because we are testing many genes (more than 15000 genes here), and the chances of finding differentially expressed genes is very high when you do that many tests. Hence we need to control the false discovery rate, which is the adjusted p-value column in the results table. What this means is that, if we choose an adjusted p-value cut-off of 0.05, and if 100 genes are significant at a 5% false discovery rate, we are willing to accept that 5 will be false positives.
{: .comment}

# Testing relative to a threshold (TREAT)

When there is a lot of differential expression, sometimes we may want to cut-off on a fold change threshold, as well as a p-value threshold, so that we follow up on the most biologically significant genes. However, it is not recommended to simply rank by p-value and then discard genes with small logFC’s, as this has been shown to increase the false discovery rate. In other words, you are not controlling the false discovery rate at 5% any more. There is a function called `treat` in limma that performs this style of analysis correctly ({% cite McCarthy2009 %}). TREAT will simply take a user-specified log fold change cut-off and recalculate the moderated t-statistics and p-values with the new information about logFC. There are thousands of genes differentially expressed in this `basalpregnant-basallactate` comparison, so let's rerun the analysis applying TREAT and similar thresholds to what was used in the Fu paper: an adjusted P value of 0.01 (1% false discovery rate) and a log-fold-change cutoff of 0.58 (equivalent to a fold change of 1.5).

> <hands-on-title>Testing relative to a threshold (TREAT)</hands-on-title>
>
> 1. Use the **Rerun** {% icon galaxy-refresh %} button in the History to rerun **limma** {% icon tool %} with the following parameters:
>      - *"**Advanced Options**"*
>          - {% icon param-text %} *"Minimum Log2 Fold Change"*: `0.58`
>          - {% icon param-text %} *"P-Value Adjusted Threshold"*: `0.01`
>          - {% icon param-check %} *"Test significance relative to a fold-change threshold (TREAT)"*: `Yes`
> 2. Add a tag `#treat` to the `Report` output and inspect the report
>
> {% snippet faqs/galaxy/datasets_add_tag.md type="name" %}
>
{: .hands_on}

We can see that much fewer genes are now highlighted in the MD plot and identified as differentially expressed.

![TREAT MDVol Plot](../../images/rna-seq-counts-to-genes/TREAT_mdvolplot_basalpregnant-basallactate.png "TREAT MD and Volcano plots")

![TREAT DE counts](../../images/rna-seq-counts-to-genes/TREATdecounts.png "TREAT differentially expressed genes")


# Visualising results

In addition to the plots already discussed, it is recommended to have a look at the expression levels of the individual samples for the genes of interest, before following up on the DE genes with further lab work. The Galaxy limma tool can auto-generate heatmaps of the top genes to show the expression levels across the *samples*. This enables a quick view of the expression of the top differentially expressed genes and can help show if expression is consistent amongst replicates in the groups.

## Heatmap of top genes

Click on the `Heatmap_basalpregnant-basallactate.pdf` link in the `Report`. You should see a plot like below.

![Heatmap](../../images/rna-seq-counts-to-genes/heatmap.png "Heatmap of top genes")

## Stripcharts of top genes

The limma-voom tool can also auto-generate stripcharts to view the expression of the top genes across the groups. Click on the `Stripcharts_basalpregnant-basallactate.pdf` link in the `Report`. You should see 10 plots, one for each top gene. Four are shown below. Note that here you can see if replicates tend to group together and how the expression compares to the other groups.

![Stripchart Plot](../../images/rna-seq-counts-to-genes/stripcharts.png "Stripcharts of top genes")


## Interactive DE plots (Glimma)

Interactive versions of the MD and Volcano plots can be output by the limma-voom tool via the Glimma package ({% cite Su2017 %}), if a gene annotation file is provided and `Glimma Interactive Plots` is selected. Links to the Glimma html pages are generated in the `Report`.

Let's take a look at the interactive MD plot. Click on the `Glimma_MDPlot_basalpregnant-basallactate.html` link in the `Report`. You should see a two-panel interactive MD plot like below. The left plot shows the log-fold-change vs average expression. The right plot shows the expression levels of a particular gene of each sample by groups (similar to the stripcharts). Hovering over points on left plot will plot expression level for corresponding gene, clicking on points will fix the expression plot to gene. Clicking on rows on the table has the same effect as clicking on the corresponding gene in the plot.

<iframe src="../../images/rna-seq-counts-to-genes/glimma/MD-Plot.html" width="100%" height="1000"></iframe>

> <hands-on-title>Search for a gene of interest</hands-on-title>
>
> `Egf` was a gene identifed as very highly expressed in the Fu paper and confirmed with qRT-PCR, see Fig. 6c from the paper below.
> ![Fu EGF gene](../../images/rna-seq-counts-to-genes/fu_egf.png "Fu et al, Nat Cell Biol 2015")
>
> Search for `Egf` in the Glimma interactive table. You should see something similar to below.
>
> ![Glimma EGF gene](../../images/rna-seq-counts-to-genes/glimma_egf.png "Glimma EGF gene")
>
>  Notice that in the plot on the right above, showing `Egf` expression for all samples, we can see it is highly expressed in the luminal lactate group but not the other samples.
{: .hands_on}

Multiple contrasts can be run with the limma tool. For example, we can compare the pregnant and lactating conditions for both the basal and luminal cells. So let's rerun the limma-voom TREAT analysis (adj.P <0.01 and lfc=0.58) and this time use the `Insert Contrast` button to include the additional contrast `luminalpregnant - luminallactate`. We can then see how the number of differentially expressed genes in the luminal cells compares to the basal cells.

> <hands-on-title>Run multiple contrasts</hands-on-title>
>
> 1. Use the **Rerun** {% icon galaxy-refresh %} button in the History to rerun the **limma** {% icon tool %} `#treat` analysis adding the following parameters *(i.e. run with 2 contrasts)*:
>      - {% icon param-text %} *"Contrast of Interest"*: `basalpregnant-basallactate`
>      - {% icon param-text %} *"Contrast of Interest"*: `luminalpregnant-luminallactate`
> 2. Add a tag `#multiple-contrasts` to the `Report` output and inspect the report
>
>   You should find that there are more genes differentially expressed for the luminal cells than the basal. There are ~274 genes DE in basal cells versus ~ 1610 in the luminal cells.
> ![Basal Luminal DE counts](../../images/rna-seq-counts-to-genes/basal_luminal_decounts.png "Basal vs Luminal DE counts")
>
>   This is similar to what Fu et al found, many more genes differentially expressed in the luminal cells on lactation, compared to the basal cells.
> ![Fu DE genes](../../images/rna-seq-counts-to-genes/fu_degenes.png "Fu et al, Nat Cell Biol 2015")
{: .hands_on}

The tables of differentially expressed genes are output as links in the `Report` (`limma-voom_basalpregnant-basallactate.tsv` and `limma-voom_luminalpregnant-luminallactate.tsv`), see below, and also as datasets in the history (`DE tables`). With multiple contrasts, a plot for each contrast is generated for relevant plots, as shown below. This enables a quick and easy visual comparison of the contrasts.

![Multiple contrasts](../../images/rna-seq-counts-to-genes/multiple_contrasts.png "Multiple contrasts output")

> <details-title>R code in detail</details-title>
>
> 1. The `Report` with all the plots and tables can be downloaded by clicking on the floppy disk icon on the dataset in the history.
> 2. If you want to see the R code used, or to explore some of the outputs in R, you can select to output the Rscript and/or RData files in the tool form.
{: .details}

To see some methods for identifying differentially expressed pathways in this dataset, see the follow-on tutorial [RNA-seq genes to pathways]({% link topics/transcriptomics/tutorials/rna-seq-genes-to-pathways/tutorial.md %}). To see how to create a heatmap of custom genes using this dataset, see the tutorial [Visualization of RNA-Seq results with heatmap2]({% link topics/transcriptomics/tutorials/rna-seq-viz-with-heatmap2/tutorial.md %})

# Conclusion


In this tutorial we have seen how counts files can be converted into differentially expressed genes with limma-voom. This follows on from the accompanying tutorial, [RNA-seq reads to counts]({% link topics/transcriptomics/tutorials/rna-seq-reads-to-counts/tutorial.md %}), that showed how to generate counts from the raw reads (FASTQs) for this dataset. In this part we have seen ways to visualise the count data, and QC checks that can be performed to help assess the quality and results. We have also reproduced results similar to what the authors found in the original paper with this dataset. For further reading on analysis of RNA-seq count data and the methods used here, see the articles; *RNA-seq analysis is easy as 1-2-3 with limma, Glimma and edgeR ({% cite Law2018 %})*  and *From reads to genes to pathways: differential expression analysis of RNA-Seq experiments using Rsubread and the edgeR quasi-likelihood pipeline ({% cite Chen2016 %})*.


topic: transcriptomics
tutorial: clipseq
title: CLIP-Seq data analysis from pre-processing to motif detection
difficulty: intermediate
time_estimation: 6h
tools_json: []
requirements_json: []
tags_json: []
content_hash: e79eef45042d3cb02f4fc544ab7f1ba5
last_modified: 2026-05-15T22:28:17.593736
zenodo_link: https://zenodo.org/record/1327423
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/clipseq/tutorial.html

content:
The eCLIP data provided here is a subset of the eCLIP data of RBFOX2 from a study published by *Nostrand et al.* ({% cite VanNostrand2016 %}). The dataset contains the first biological replicate of RBFOX2 CLIP-seq and the input control experiment (FASTQ files). The data was changed and downsampled to reduce data processing time, consequently the data does not correspond to the original source pulled from *Nostrand et al.* ({% cite VanNostrand2016 %}). Also included is a text file (.txt) encompassing the chromosome sizes of hg38 and a genome annotation (.gtf) file taken from [Ensembl](http://ftp.ensemblorg.ebi.ac.uk/pub/release-74/gtf/homo_sapiens/).

**Table 1**: Data from RBFOX2 CLIP-Seq and control experiment which we are going to use in this tutorial from a study published by *Nostrand et al.* ([2016]({% cite VanNostrand2016 %}). PE stands for a paired-end read library. Hep G2 is a human liver cancer cell line. Replicate is the number of the biological replicate. Stranded means, if the read-library is strand specific, i.e., in a specific forward-reverse direction.

| Cellular state | Datatype | Description | Replicate | ENCODE Accession | Library type | Read length | Stranded? |
| ---            | ---      | :-:     | :-:       | ---           | :-:          | :-:         | :-:       |
| Hep G2            | eCLIP | RBFOX2   | 1         | ENCSR987FTF     | PE           | 175-300          | Yes        |
| Hep G2            | eCLIP | input   | 1         | ENCSR799EKA     | PE           | 175-300          | Yes        |


> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Finding Binding Motifs for RBFOX2

RBFOX2 is a relevant development and tissue-specific splicing factor with the conserved motif `TGCATG`. It binds mainly in introns. We therefore want to process and validate the data from human liver cancer cells (HepG2) to find this conserved motif and in the process identify the function of RBFOX2 as well as describe the function of the targeted RNA.

## Step 1: Get data

> <hands-on-title>Data upload</hands-on-title>
> 1. Create and name a new history for this tutorial.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the following files from [Zenodo](https://zenodo.org/record/2579279).
>
>    ```
>    https://zenodo.org/record/2579279/files/hg38_chr_sizes.txt
>    https://zenodo.org/record/2579279/files/Homo_sapiens.GRCh38.87.gtf
>    https://zenodo.org/record/2579279/files/RBFOX2-204-CLIP_S1_R1_RBFOX2.fastq
>    https://zenodo.org/record/2579279/files/RBFOX2-204-CLIP_S1_R2_RBFOX2.fastq
>    https://zenodo.org/record/2579279/files/RBFOX2-204-INPUT_S2_R1.fastq
>    https://zenodo.org/record/2579279/files/RBFOX2-204-INPUT_S2_R2.fastq
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    ![upload](../../images/transcriptomics_images/upload_data_page.png "Data can be imported directly with links.")
>
>   ![data](../../images/transcriptomics_images/clipseq_data_uploaded.png "Imported datasets will appear in the history panel.")
>
{: .hands_on}

# Step 2: Quality Control

As for any NGS data analysis, CLIP-seq data must be quality controlled before being aligned to a reference genome. For more detailed information on NGS quality control, check out the [tutorial for NGS]({% link topics/sequence-analysis/tutorials/quality-control/tutorial.md %}). One crucial thing to check for CLIP-Seq data is the sequence duplication levels. PCR duplications occur naturally in any NGS experiment during the PCR amplification of the genetic material. CLIP-Seq is prone to many PCR duplicates because of the sparse material that is obtained during a CLIP-Seq experiment resulting in high PCR cycles in many occasions. More information about the problem of PCR duplicates can be found [in this guide](http://www.cureffi.org/2012/12/11/how-pcr-duplicates-arise-in-next-generation-sequencing/).

## Report with **FastQC**

> <hands-on-title>Quality control with FastQC</hands-on-title>
>
> 1. **FastQC** {% icon tool %}: Run the tool **FastQC** on each FASTQ file to assess the quality of the raw data. An explanation of the results can be found on the [FastQC web page](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/).
>
>    > <tip-title>Running a tool on multiple data files</tip-title>
>    >
>    > You can run this tool - and many other tools - on all the FASTQ files at once!
>    > To do this, first select the "Multiple datasets" icon (two stacked pages) under the "Input FASTQ file" heading in the **FASTQC** Tool Form, then shift+click to select multiple FASTQ files.
>    {: .tip}
>
> 2. Check the **Sequence Duplication Levels** plot.
>
>   ![fastqbefore](../../images/transcriptomics_images/clipseq_duplication_level_1.png "Sequence duplication levels <b>before</b> de-duplication; The y-axis represents the portion of reads with the specific duplication level. An exact sequence match is needed to detect a duplicated read. The blue line shows the duplication level distribution of the full sequence set. The red line depicts an ideal curve after a de-duplication step (duplicates are filtered out).")
>
>    > <question-title></question-title>
>    >
>    > 1. Has our data high sequence duplication levels?
>    > 2. Why is there still a small spike in the red line?
>    > 3. What does the headline tell you?
>    >
>    > > <solution-title></solution-title>
>    > > 1. Yes, ideally we would see a blue line with one high peak in a duplication of one, everything else should be zero.
>    > > 2. Spikes in the red line come from different duplication levels in the original data (blue line).
>    > > 3. The headline states an expected value of reads that would remain after a de-duplication step. A high percentage suggest, that no de-duplication step is needed. This should also correspond with the red line.
>    > {: .solution }
>    {: .question}
{: .hands_on}

# Step 3: Removal of Adapters, Barcodes and Unique Molecular Identifiers (UMIs)

It is often necessary to remove adapter and barcodes sequences as well as UMIs. <br/>
**Adapters** (or primers) are needed for PCR amplification and sequencing in a standard NGS protocol. Unfortunately, it might happen during the sequencing that the machine does not stop at the read end and sequences through the adapter as well. That is why, we need to check if our reads contain those sequences which we are then cutting out.<br/>
**Barcodes** on the other hand are especially designed for a read library and intentionally sequenced. Sometimes experiments are sequenced at the same time which is called **multiplexing**. Multiplexing allows for a better data normalisation and comparison. The barcodes are then used to divide the un-multiplexed data set into the individual read libraries. (**Note: Our data is already de-multiplexed, i.e., we do not have to take barcode sequences into account.**)<br/>
**UMIs** are similar to barcodes but these sequences are unique for each read. UMIs were introduced since iCLIP to deal with the high duplication levels of a CLIP experiment. Because each read contain an UMI, PCR duplicates of that read also contain the same UMI, which makes it possible to fuse all reads with the same UMI.

## Removal of adapter sequences with **Cutadapt**

In this task we are going to remove two 3' and two 5' adapters from the reads (Note: The eCLIP protocol uses more adapter sequences, for more information take a look in the original publication {% cite VanNostrand2016 %}).

> <hands-on-title>Adapter Removal</hands-on-title>
>
> **Cutadapt (v. 4.0)** {% icon tool %} with the following parameters:
>    - *"Single-end or Paired-end reads?"*: `Paired-end`
>    - {% icon param-file %} *"FASTQ/A file #1"*: `FASTQ R1`
>    - {% icon param-file %} *"FASTQ/A file #2"*: `FASTQ R2`
>    - *"Read 1 Options"*:
>       - In *"3' Adapters"*:
>            - Click on *"Insert 3' Adapters"*:
>            - In *"1: 3' Adapters"*:
>                - *"Source"*: `Enter custom sequence`
>                    - *"Enter custom 3' adapter sequence"*: `AACTTGTAGATCGGA`
>            - Click on *"Insert 3' Adapters"*:
>            - In *"2: 3' Adapters"*:
>                - *"Source"*: `Enter custom sequence`
>                    - *"Enter custom 3' adapter sequence"*: `AGGACCAAGATCGGA`
>        - In *"5' (Front) Adapters"*:
>            - Click on *"Insert 5' (Front) Adapters"*:
>            - In *"1: 5' (Front) Adapters"*:
>                - *"Source"*: `Enter custom sequence`
>                    - *"Enter custom 5' adapter sequence"*: `CTTCCGATCTACAAGTT`
>            - Click on *"Insert 5' (Front) Adapters"*:
>            - In *"2: 5' (Front) Adapters"*:
>                - *"Source"*: `Enter custom sequence`
>                    - *"Enter custom 5' adapter sequence"*: `CTTCCGATCTTGGTCCT`
>       - *"Cut bases from reads before adapter trimming"*: `-5`
>    - *"Read 2 Options"*:
>       - In *"3' Adapters"*:
>            - Click on *"Insert 3' Adapters"*:
>            - In *"1: 3' Adapters"*:
>                - *"Source"*: `Enter custom sequence`
>                    - *"Enter custom 3' adapter sequence"*: `AACTTGTAGATCGGA`
>            - Click on *"Insert 3' Adapters"*:
>            - In *"2: 3' Adapters"*:
>                - *"Source"*: `Enter custom sequence`
>                    - *"Enter custom 3' adapter sequence"*: `AGGACCAAGATCGGA`
>        - In *"5' (Front) Adapters"*:
>            - Click on *"Insert 5' (Front) Adapters"*:
>            - In *"1: 5' (Front) Adapters"*:
>                - *"Source"*: `Enter custom sequence`
>                    - *"Enter custom 5' adapter sequence"*: `CTTCCGATCTACAAGTT`
>            - Click on *"Insert 5' (Front) Adapters"*:
>            - In *"2: 5' (Front) Adapters"*:
>                - *"Source"*: `Enter custom sequence`
>                    - *"Enter custom 5' adapter sequence"*: `CTTCCGATCTTGGTCCT`
>    - *"Adapter Options"*:
>        - *"Minimum overlap length"*: `5`
>    - *"Filter Options"*:
>        - *"Minimum length (R1)"*: `10`
>        - *"Specify a minimum/maximum length for reverse reads (R2)"*: `Disabled`
>
>   > <comment-title>Why do we remove 5 bp from the first read?</comment-title>
>   > In eCLIP it can happen that the sequencing goes over the first read into the UMI, which is at the 3' end of the first read. The UMI is 5 bp long in our data. To make sure our first read in the read pair does not contain the UMI, we simply remove the last 5 bp from it. The UMI that we actually need for the de-duplication is located on the 5' end of our second read in the read pair.
>   {: .comment}
>
>    > <comment-title>Do the same thing for the input control data set.</comment-title>
>    >
>    > If you processed the RBFOX2 FASTQ dataset then do the same thing for the input control dataset or *vice verca*.
>   {: .comment}
>
{: .hands_on}

## Removal of UMIs with **UMI-tools extract**

In this task we are going to remove the UMI at the 5' end of the second read. We are then going to use the removed UMIs later on to de-duplicate our read library and account for PCR duplicates.

> <hands-on-title>UMI Removal</hands-on-title>
>
> 1. **UMI-tools extract** {% icon tool %} with the following parameters:
>    - *"Library type"*: `Paired-end`
>     - {% icon param-file %} *"Reads in FASTQ format"*: `FASTQ R2` (output from the **Cutadapt** call {% icon tool %})
>     - {% icon param-file %} *"Reads in FASTQ format"*: `FASTQ R1`  (output from the **Cutadapt** call {% icon tool %})
>     - *"Barcode on both reads?"*: `Barcode on first read only`
>    - *"Use Known Barcodes?"*: `No`
>    - *"Method to extract barcodes"*: `String`
>    - *"Barcode pattern for first read"*: `NNNNN`
>    - *"Is the barcode at the 5' end?"*: `Yes`
>    - *"Output log?"*: `Yes`
>    - *"Enable quality filter?"*: `No`
>
>   > <comment-title>Do the same thing for the input control data set.</comment-title>
>   >
>   > If you processed the RBFOX2 FASTQ dataset then do the same thing for the input control dataset or *vice verca*.
>   {: .comment}
>
>   > <question-title></question-title>
>   > 1. What is the meaning of the barcode pattern?
>   >
>   >   > <solution-title></solution-title>
>   >   > 1. The symbol "N" defines our UMI. The symbol "X" are nucleotides that we would reattach to our read. It is important to note, that the UMI can be encompassed by an in-line barcode, that is why we need two distinct symbols "N" and "X".
>   > {: .solution }
>   {: .question}
>
{: .hands_on}

# Step 4: Aligning Reads to a Reference Genome

To determine where DNA fragments originated in the genome, the sequenced reads must be aligned to a reference genome. This is equivalent to solving a jigsaw puzzle, but unfortunately, not all pieces are unique. In principle, you could do a BLAST analysis to figure out where the sequenced pieces fit best in the known genome. However, this can take a couple of weeks. Nowadays, there are many read alignment programs. STAR {% cite Dobin2012%} is one of them that works well with CLIP-Seq data. STAR is able to use genome as well as transcriptome data. This ability is handy, since CLIP-Seq generates transcriptome data, consequently we have to take RNA processing steps like splicing events into account.

## Aligning with **RNA STAR**

> <hands-on-title>Alignment</hands-on-title>
>
> 1. **RNA STAR** {% icon tool %} with the following parameters:
>    - *"Single-end or paired-end reads"*: `Paired-end (as individual datasets)`
>     - {% icon param-file %} *"RNA-Seq FASTQ/FASTA file, forward reads"*: `FASTQ R2` (output from **UMI-tools extract** {% icon tool %})
>     - {% icon param-file %} *"RNA-Seq FASTQ/FASTA file, reverse reads"*: `FASTQ R1` (output from **UMI-tools extract** {% icon tool %})
>    - *"Custom or built-in reference genome"*: `Use a built-in index`
>        - *"Reference genome with or without an annotation"*: `use genome reference with builtin gene-model`
>            - *"Select reference genome"*: `Homo sapiens (hg38+GRCh38.87)`
>    - *"Count number of reads per gene"*: `No`
>    - *"Would you like to set output parameters (formatting and filtering)?"*: `Yes`
>        - *"Extra SAM attributes to include"*: `All`
>        - *"Include strand field flag XS"*: `Yes -- and reads with inconsistent and/or non-canonical introns are filtered out`
>        - *"Would you like to set additional output parameters (formatting and filtering)?"*: `Yes`
>    - *"Other parameters (seed, alignment, limits and chimeric alignment)"*: `Extended parameter list`
>        - In *"Alignment parameters"*:
>            - *"Use end-to-end read alignments, with no soft-clipping?"*: `Yes`
>        - *"Would you like to set chimeric alignment parameters?"*: `No`
>
>   > <comment-title>Note: We have switched R1 and R2 as forward and reverse reads!</comment-title>
>   >
>   > We need to do that because the eCLIP read library is organised in a way such that the first mate (R1) is in reverse and the second mate (R2) in forward orientation.
>   {: .comment}
>
>   > <comment-title>Soft-Clipping vs Hard-Clipping</comment-title>
>   >
>   > Clipping is a way to deal with low quality bases during the alignment step. In **Soft-Clipping** the bases at the 5' and 3' end of the read are not part of the alignment. In **Hard-Clipping** the bases at the 5' and 3' end of the read are not part of the alignment **and** will be completely removed from the read sequence in the BAM file.
>   {: .comment}
>
>   > <comment-title>Do the same thing for the input control data set.</comment-title>
>   >
>   > If you processed the RBFOX2 FASTQ dataset then do the same thing for the input control dataset or *vice verca*.
>   {: .comment}
>
>   > <question-title></question-title>
>   >
>   > 1. Can you think of a reason why we disabled the soft-clipping?
>   >
>   >   > <solution-title></solution-title>
>   >   >
>   >   > 1. In eCLIP the cross linking position should be at the beginning of the second read. If we would enable soft-clipping, we would add potential bases with low quality at the end of our second reads that would blur our cross linking position and we would lose precision to detect potential binding regions of RBFOX2.
>   > {: .solution}
> {: .question}
>
{: .hands_on}

# Step 5: De-Duplication

Lets return to the UMIs which we extracted in step three. Since we have mapped the reads to our reference genome, we can now identify which reads might be duplicated or not. The UMI-tools {% cite Smith2017 %} will help us again in this matter.

## De-duplication with **UMI-tools deduplicate**

> <hands-on-title>De-Duplication</hands-on-title>
>
> 1. **UMI-tools deduplicate** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"Reads to deduplicate in SAM or BAM format"*: `bam` (output of **RNA STAR** {% icon tool %})
>    - *"extract_umi_method"*: `Read ID`
>    - *"Separator between read id and UMI."*: `_`
>    - *"Method used to identify PCR duplicates within reads."*: `Identify clusters based on hamming distance and resolve networks by using the node counts`
>    - *"Edit distance threshold"*: `1`
>    - *"BAM is paired end"*: `Yes`
>
>   > <comment-title>What is the purpose of the method we have chosen for the de-duplication?</comment-title>
>   >
>   > `UMI-tools deduplication` has several methods. The method we have picked is called the  **adjacency** method. For detailed information have a look at method descriptions of UMI-tools {% cite Smith2017 %}. For a brief explanation: the method fuses reads together when they have the same coordinates and the same UMI. However, sequencing errors can occur in the UMI. Thus, in the **adjacency** method we fuse also UMIs that differ in a maximal number of characters and where we identify a lot of copies, i.e., the method creates clusters of nodes, a node for each individual UMI, and fuses these nodes based on the [hamming distance](https://en.wikipedia.org/wiki/Hamming_distance) and read counts.
>   {: .comment}
>
>   > <comment-title>Do the same thing for the input control data set.</comment-title>
>   >
>   > If you processed the RBFOX2 bam file then do the same thing for the input control bam or *vice verca*.
>   {: .comment}
>
>   > <question-title></question-title>
>   >
>   > 1. Why are we doing the de-duplication after the read mapping?
>   > 2. What is the edit distance threshold?
>   >
>   >   > <solution-title></solution-title>
>   >   >
>   >   > 1. Because we need the coordinates of the reads to see if some reads have the same coordinates and the same UMI.
>   >   > 2. `UMI-tools dedulication` uses the hamming distance between UMIs to detect sequencing errors in the UMIs. A hamming distance of 1 means, that two UMIs are the same if they differ in maximal one character. If two characters are different, then these are two disparate UMIs.
>   > {: .solution}
> {: .question}
>
{: .hands_on}

## Quality check of the de-duplication with **FastQC**

> <hands-on-title>De-duplication quality control</hands-on-title>
>
> 1. **FastQC** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"Short read data from your current history"*: `bam` (output of **UMI-tools deduplicate** {% icon tool %})
> 2. View the **Sequence Duplication Levels** plot.
> ![fastqafter](../../images/transcriptomics_images/clipseq_duplication_level_2.png "Sequence duplication levels <b>after</b> de-duplication.")
>
>   > <question-title></question-title>
>   >
>   > 1. What can you see, if you compare the duplication levels to our first quality control in step two?
>   > 2. Why are the duplication levels still high?
>   >
>   >   > <solution-title></solution-title>
>   >   >
>   >   > 1. We have reduced the duplication levels.
>   >   > 2. The duplication levels are still high, because we don't have the full data sets (reads are missing).
>   > {: .solution}
> {: .question}
>
{: .hands_on}

# Step 6: Second Quality Control

In this section we check the quality of our mapped reads and see if our samples are correlated or not. It is actually not really necessary for our data sets, because of the small sample size. However, for bigger analyses it is wise to check if some samples may encompass major quality problems.

## Check the read coverage with **plotFingerprint**

> <hands-on-title>Check the read coverage</hands-on-title>
>
> 1. **plotFingerprint** {% icon tool %} with the following parameters:
>    - *"Sample order matters"*: `No`
>    - *"Bam file"*: `both bam files` (output of **UMI-tools deduplicate** {% icon tool %})
>    - *"Show advanced options"*: `yes`
>     - *"Bin size in bases"*: `100`
>    - *"Show advanced output settings"*: `no`
> 2. View the output image.
> ![fingerprint](../../images/transcriptomics_images/clipseq_fingerprint.png "Graph of IP strength from <b>plotFingerprint</b>. The y-axis represents the fraction of the total number of reads. `plotFingerprint` randomly samples genome regions (bins) of a specified length and sums the per-base coverage that overlap with those regions. The x-axis therefore orders the bins from lowest coverage to highest coverage. The plot here is too idealistic which is because of our small data set.")
>
>   > <comment-title>What does this graph represent, especially for CLIP-Seq data?</comment-title>
>   >
>   > It shows us how good the CLIP Signal compared to the control signal is. Now be careful, CLIP-Seq experiments involve either a total RNA control or a negative control with another protein that unspecifically binds RNA (e.g., IgG). An ideal total RNA control (input control) like ours with perfect uniform distribution of reads along the genome/transcriptome (i.e. without enrichments) and infinite sequencing coverage should generate a straighter, diagonal line. On the other hand, a very specific and strong CLIP enrichment will be indicated by a prominent and steep rise of the cumulative sum towards the highest rank. Yet, a negative control often has the same sharp slope at the end as a CLIP experiment but often depicts a straighter, diagonal line in the beginning like the input control.
>   {: .comment}
>
>   > <question-title></question-title>
>   >
>   > 1. How do (or should) input datasets differ from CLIP datasets?
>   > 2. What do you think about the quality of the CLIP experiment?
>   >
>   >   > <solution-title></solution-title>
>   >   >
>   >   > 1. We expect that the control (input) has a different, more straighter, diagonal slope than the CLIP experiment.
>   >   > 2. Both the CLIP experiment and the control are closely related. It is maybe wise to check the correlation of the two samples.
>   > {: .solution}
> {: .question}
>
{: .hands_on}

For additional information on how to interpret **plotFingerprint** plots, read the information [the DeepTools documentation](https://deeptools.readthedocs.io/en/latest/content/tools/plotFingerprint.html#background).

## Correlation between Samples

We have seen that our input control and our CLIP experiment might be strongly correlated, meaning, that our potential RBFOX2 binding regions are not truly enrichment when comparing to a our control.
We are therefore going to further check the correlation between our control and CLIP experiment.

> <hands-on-title>Assessing correlation between samples</hands-on-title>
>
> 1. **multiBamSummary** {% icon tool %} with the following parameters:
>    - *"Sample order matters"*: `No`
>    - *"Bam file"*: `both bam files` (output of **UMI-tools deduplicate** {% icon tool %})
>    - *"Choose computation mode"*: `Bins`
>     - *"Bin size in bp"*: `1000`
>    - *"Show advanced options"*: `no`
> 2. **plotCorrelation** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"Matrix file from the multiBamSummary tool"*: `outFile` (output of **multiBamSummary** {% icon tool %})
>    - *"Correlation method"*: `Spearman`
>    - *"Plotting type"*: `Heatmap`
>    - *"Skip zeros"*: `Yes`
>    - *"Image file format"*: `png`
> 3. View the output image.
> ![correlation](../../images/transcriptomics_images/clipseq_correlation.png "Heatmap of correlation matrix generated by <b>plotCorrelation</b>. The y-axis shows you on the left site the histogram of the clustering. The colored scale depicts the correlation value. Like  `plotFingerprint`, `multiBamSummary` randomly samples genome regions (bins) of a specified length and sums the per-base coverage that overlap with those regions. The correlation between these bins is then calculated for each pair of samples. An ideal plot would result in two disparate clusters: one for the biological replicates of your CLIP-Seq experiment and one for the replicates of your control. A bad plot would happen if your control and CLIP-Seq experiment end up in one cluster, which means that they are closely related and something went wrong in the laboratory.")
>
>   > <question-title></question-title>
>   >
>   > 1. Why do we want to skip zeros in plotCorrelation?
>   > 2. What happens if the Spearman correlation method is replaced by the Pearson method?
>   > 3. What does the correlation tell you about our data?
>   >
>   >   > <solution-title></solution-title>
>   >   >
>   >   > 1. Large areas of zeros would lead to a correlation of these areas. The information we would get out of this computation would be meaningless.
>   >   > 2. The clusters would be different.
>   >   > 3. Despite the result of `plotFingerprint`, we see that the correlation between our experiment and control is not nearly as significant as we thought.
>   > {: .solution}
> {: .question}
>
{: .hands_on}

# Step 7: Peak Calling

Peak calling is one of the most important steps in the data analysis of CLIP-Seq, next to mapping. There exist a variety of peak callers like [PEAKachu](https://github.com/tbischler/PEAKachu), PureCLIP ({% cite Krakau2017 %}) or Piranha ({% cite Uren2012 %}) each with different underlying assumptions and parameters. It is difficult (nearly impossible) to find a generic solution for every data set. We are going to use `PEAKachu` to find possible binding motifs for our data. `PEAKachu` is able to incorporate control data in contrast to other peak callers like `Piranha`, thus allowing to find binding regions that are significantly enriched in comparison to our control (input) data.

## Peak calling with **PEAKachu**

> <hands-on-title>Peak calling</hands-on-title>
>
> 1. **PEAKachu** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"Experiment Libraries"*: `bam RBFOX2` (output of **UMI-tools deduplicate** {% icon tool %})
>    - {% icon param-file %} *"Control Libraries"*: `bam Input` (output of **UMI-tools deduplicate** {% icon tool %})
>    - *"Pairwise Replicates"*: `No`
>    - *"Paired End"*: `Yes`
>    - *"Maximum Insert Size"*: `"200"`
>    - *"Select Mode"*: `Adaptive`
>        - *"Normalisation Method."*: `DESeq2`
>    - *"Mad Multiplier"*: `"0.0"`
>    - *"Fold Change Threshold"*: `"2.0"`
>    - *"Adjusted p-value Threshold"*: `"0.05"`
> 2. Take a look at the MA plot of **PEAKachu**.
> ![ma](../../images/transcriptomics_images/clipseq_ma_plot.png "MA plot of <b>PEAKachu</b>.  MA stands for M (log ratio) and A (mean average). It shows you the general trend of the log2 fold-change in dependence of the average mean of the expression rate of the peaks. Points in a straight, diagonal line symbolise the behavior of the logarithm which is often visible for low expressed peaks, because one read can change the fold-change quite drastically. The blue lines depict the normalisation constants. Read dots are significant peaks. It is hard to tell what a good MA plot would look like. Even a small number of significant peaks can be enough, depending on the biological question you ask.")
>
>   > <comment-title>Adjusted p-value Threshold</comment-title>
>   >
>   > P-values are calculated by DESeq2. A low p-value represent a significantly enriched binding region. Since we are doing thousands of independent hypothesis testings, we have to correct for significant p-values that happen just by chance. DESeq2 uses the Benjamini-Hochberg procedure as a correction. For more information read [this handbook](http://www.biostathandbook.com/multiplecomparisons.html).
>   {: .comment}
>
>   > <comment-title>Fold-Change Threshold</comment-title>
>   >
>   > The fold-change is calculated by DESeq2 and is actually a log2 fold-change. Because we want significantly enriched binding regions, we basically search for peaks that are at least four times higher in our CLIP experiment than in our control.
>   {: .comment}
>
>   > <question-title></question-title>
>   >
>   > 1. What happens if we decrease or increase the log2 fold-change threshold of 2.0?
>   > 2. Take a look into the peak output file of PEAKachu. Why aren't there any p-values?
>   > 3. Is the log2 fold-change alone a good measurement to validate our peaks? Maybe the MA plot helps you to figure out this question?
>   >
>   > > <solution-title></solution-title>
>   > >
>   > > 1. If we would **decrease** the log2 fold-change threshold then we would get more peaks. However, we would also increase the number of false positives, i.e., peaks that are **not** significantly enriched.<br/>
>   > >    If we would **increase** the log2 fold-change threshold, then we would decrease the number of peaks and possible false positives. However, we would also increase the number of false negatives, i.e., we would lose potentially interesting binding sites.
>   > > 2. PEAKachu works mainly with replicated data, because DESeq2 calculates only p-values for data sets where the experiment and control have at least two replicates. This feature makes statistically sense, because data sets without replicates are not representative enough to make general assumptions, but for the purpose of demonstration we are going to ignore this fact.
>   > > 3. Yes and no. A significant fold-change with high read counts (read coverage) might be enough to validate a peak. However, some peaks can also be covered by very few reads and still have a high fold-change, e.g., log2(4/1). It is for you to decide if a fold-change like log2(4/1) is enough evidence that the binding region is significantly enriched.
>   > {: .solution}
>   {: .question}
>
{: .hands_on}

# Step 8: Peak Analysis

In this last step, we are going to analyse the peaks that we obtained from the previous step. This should give you a small glimpse of what someone might be interested in. Therefore we want to answer in this step the following questions:
1) Which sequential motifs are potentially conserved in our binding regions, i.e., what are potential binding motifs of RBFOX2? 2) Which RNAs does RBFOX2 preferentially bind and what are their biological function?

## Motif detection with **MEME-ChIP**

Lets first find out which sequence motifs RBFOX2 might preferentially bind to.

> <hands-on-title>Motif detection</hands-on-title>
>
> 1. **Text reformatting** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"File to process"*: `tabular` (output of **PEAKachu** {% icon tool %})
>    - *"AWK Program"*: `NR>1{if($3 < $4) {print $1,$3,$4,"clip_peak_"NR-1,$9,$5;}else {print $1,$4,$3,"clip_peak_"NR-1,$9,$5;}}`
> 2. Set the datatype of the output generated by **Text reformatting** as `bed`
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="bed" %}
>
> 3. **SlopBed** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"BED/VCF/GFF file"*: `bed` (output of **Text reformatting** {% icon tool %})
>    - *"Genome file"*: `hg38_chr_sizes.txt` (file from your history)
>    - *"Choose what you want to do"*: `Increase the BED/GFF/VCF entry by the same number base pairs in each direction.`
>        - *"Number of base pairs"*: `20`
> 4. **Extract Genomic DNA** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"Fetch sequences for intervals in"*: `bed` (output of **SlopBed** {% icon tool %})
>    - *"Interpret features when possible"*: `Yes`
>    - *"Choose the source for the reference genome"*: `locally cached`
>        - *"Using reference genome"*: `hg38`
>    - *"Select output format"*: `fasta`
> 5. **MEME-ChIP** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"Primary sequences"*: `FASTA` (output of **Extract Genomic DNA** {% icon tool %})
>    - *"Sequence alphabet"*: `DNA`
>    - *"Options Configuration"*: `Advanced`
>        - *"Limit of sequences to pass to MEME"*: `100`
>        - *"Should subsampling be random?"*: `Yes`
>            - *"Seed for the randomized selection of sequences"*: `123`
>        - *"E-value threshold for including motifs"*: `0.05`
>        - *"Search given strand only"*: `Yes`
>        - *"What is the expected motif site distribution?"*: `Zero or one occurances per sequence`
>        - *"Minimum motif width"*: `5`
>        - *"Maximum motif width"*: `20`
>        - *"Maximum number of motifs to find"*: `20`
>        - *"Stop DREME searching after reaching this E-value threshold"*: `0.05`
>        - *"Stop DREME searching after finding this many motifs"*: `5`
>    - *"I certify that I am not using this tool for commercial purposes."*: `Yes`
> 6. View the **MEME-ChIP** html.
>
> ![motif](../../images/transcriptomics_images/clipseq_motif.png "Plot of a sequence motif. The x-axis of the sequence plots represents the nucleotide position of the motif. The y-axis stands for the total information (uncertainty) of each position and thus stands for the probability that the nucleotide at a certain position is the specific letter (for DNA: T,C,G,A). Bigger letters stand for a higher probability. For more information read [this Wikipedia article](https://en.wikipedia.org/wiki/Sequence_logo).")
>
>   > <comment-title>MEME and DREME</comment-title>
>   >
>   > **MEME** and **DREME** are two motif finding tools that MEME-ChIP uses. More information about theses tools can be found on [meme-suite.org](http://meme-suite.org/).
>   {: .comment}
>
>   > <comment-title>What is the purpose of SlopBED?</comment-title>
>   >
>   > **PEAKachu** might underestimate the length of the binding regions, because the actually protein-RNA-binding concentrates on the cross linking site that is one nucleotide long. Thus, we extend the peaks from **PEAKachu** by 20 bp at each site. A true conserved binding motif will not be affected by it, if we make the region to wide.
>   {: .comment}
>
>   > <comment-title>What is the meaning of the E-value?</comment-title>
>   >
>   > The E-value represents the expected number of times we would find our sequence motif in a database (peak file), just by chance. This means, that a small E-value correspond to a very significant motif, because the expected number we would find that motif in our peak file just by chance is very low. Other sequences like repeats will have a high E-value on the other hand.
>   {: .comment}
>
>   > <question-title></question-title>
>   >
>   > 1. Click on the link under the column **Discovery/​Enrichment Program** for two of the motifs one from **MEME** and one from **DREME**. What is the site-count for the most significant motif? How many sequences have we checked for **MEME** and for **DREME**? What is the fraction of sequences that covered that motif? (MEME-ChIP version 4.11.2)
>   > 2. Have we found the motif `TGCATG`?
>   >
>   >   > <solution-title></solution-title>
>   >   >
>   >   > 1. For the result with MEME-ChIP version 4.11.2: **MEME** site count = 37, total sequences = 100, fraction = 37%; **DREME** site count = 103, total sequences = 247, fraction = 41.7%.
>   >   > 2. Yes.
>   > {: .solution}
> {: .question}
>
{: .hands_on}

## Functional analysis with **RNA Centric Annotation System**

Lets move to the next step and find out where these motifs might come from, i.e., which kind of RNAs RBFOX2 preferentially bind to. For that we will use the `RNA Centric Annotation System`. Run the tool and then try to answer the subsequent questions.

> <hands-on-title>Functional analysis</hands-on-title>
>
> 1. **RNA Centric Annotation System (v. 1.5.4)** {% icon tool %} with the following parameters:
>    - *"Genome Version"*: `hg38`
>    - {% icon param-file %} *"Target regions in BED format"*: `bed` (output of **SlopBed** {% icon tool %})
>    - {% icon param-file %} *"Reference annotation in ENSEMBL GTF format"*: `Homo_sapiens.GRCh37.74.gtf` (file from your history)
>    - *"Run annotation."*: `Yes`
>    - *"Run GO term enrichment"*: `Yes`
>    - *"Run gene set enrichment"*: `No`
>    - *"Run motif search"*: `No`
>    - *"Downsampling (N)"*: `"0"`
> 2. View the output html of **RNA Centric Annotation System**
>
>   > <question-title></question-title>
>   >
>   > 1. What is the main target region of RBFOX2 (e.g., cds, exon, 5' UTR)? Does this confirm the literature.
>   > 2. From which cellular compartments do the targets come from?
>   > 3. Where does RBFOX2 mainly bind around the intron-exon boundaries and total transcript?
>   >
>   >   > <solution-title></solution-title>
>   >   >
>   >   > 1. Introns and transcripts. Yes.
>   >   > 2. Actin filaments, actin cytoskeleton, cell projection part etc.
>   >   > 3. Around the exon-intron boundaries more at the 3' ends of transcripts.
>   > {: .solution}
> {: .question}
>
{: .hands_on}

# Additional Step

Sometimes it is wise to take a look at individual peaks; maybe to check the peak calling algorithm or the general quality of the read coverage of the called peaks. We will cover the motivation behind this step in this section and explain why it would be wise to further inspect your results. For this we take the de-duplicated reads in step 5 and the called peaks in step 7 to take a deeper look inside our data.

## Generating bigWig files of the alignment ends

We will first generate coverage files that gives us a better visualization for our read coverage along the genome. SAM or BAM files can be used too, but they are often to big and harder to handle, if you try to inspect the read coverage. Therefore we are going to generate a bigWig (binary form of a BedGraph) file.

> <hands-on-title>Task description</hands-on-title>
>
> 1. **Extract alignment ends** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"Alignments in SAM or BAM format"*: `bam` (output of **UMI-tools deduplicate** {% icon tool %})
> 2. **SortBED** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"Sort the following BED file"*: `bed` (output of **Extract alignment ends** {% icon tool %})
>    - *"Sort by"*: `chromosome, then by start position (asc)`
> 3. **Create a BedGraph of genome coverage** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"The BAM or BED file from which coverage should be computed"*: `bed` (output of **SortBED** {% icon tool %})
>    - *"Report regions with zero coverage"*: `No`
>    - *"Treat split/spliced BAM or BED12 entries as distinct BED intervals when computing coverage."*: `No`
>    - *"Calculate coverage based on"*: `both strands combined`
> 4. **Wig/BedGraph-to-bigWig** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"Convert"*: `bedgraph` (output of **Create a BedGraph of genome coverage** {% icon tool %})
>    - *"Converter settings to use"*: `Default`
>
>   > <comment-title>Do the Same thing for the input control data set.</comment-title>
>   >
>   > If you processed the RBFOX2 bam dataset then do the same thing for the input control dataset or *vice verca*.
> {: .comment}
>
{: .hands_on}

## Inspection of peaks and aligned data with IGV

Next we are going to use the generated bigWig files of our alignment ends and the peaks we obtained from `PEAKachu` and investigate the read coverage of our called peaks with [IGV](http://software.broadinstitute.org/software/igv/). This is an alternative to Trackster. For more information about IGV follow the  [tutorial]({{site.baseurl}}/topics/introduction/tutorials/igv-introduction/tutorial.html).<br/>

**Try to answer the questions for this hands-on.** It will give you an idea why it would be wise to inspect the read coverage of the peaks.

> <hands-on-title>IGV</hands-on-title>
>
> 1. Open IGV on your local computer.
> 2. Choose as a reference genome **hg38**.
> 3. Download and drag-and-drop the following files from galaxy into IGV:
>     - **peaks_annotations.gff** of PEAKachu {% icon tool %}
>     - **CLIP.bigWig** of Wig/BedGraph-to-bigWig {% icon tool %}
>     - **INPUT.bigWig** of Wig/BedGraph-to-bigWig {% icon tool %}
> 4. Inspect the peaks (e.g., <b>chr11:peak_2</b> chr11:2152602-2152742)
> ![igv_peak](../../images/transcriptomics_images/clipseq_peak_igv.png "Inspection of a peak in IGV.")
>
>   > <question-title></question-title>
>   >
>   > 1. What are the ideal conditions that PEAKachu {% icon tool %} will call a peak.
>   > 2. What are the challenges a peak caller has to face? What defines a peak (significantly enriched region)?
>   > 3. Imagine you find a region that should be definitely be a peak but your peak caller does not spot it. What can you do?
>   >
>   >   > <solution-title></solution-title>
>   >   >
>   >   > 1. High read coverage in the region for the CLIP experiment and low read coverage in the region for the input control.
>   >   > 2. As simply explained in the previous answer, a peak (significantly binding region) should be significantly enriched in the CLIP experiment and not in the input control. However, the difficulty is to find out if two adjacent peaks are two distinct peaks or just one big peak. Furthermore, very small bumps can also be just an artifact of a way bigger peak. In addition, a high read coverage in the CLIP experiment and not in the input control does not necessarily mean that we have an enriched region. The significance depends on the background (noise) of our data. A good normalisation is therefore crucial.
>   >   > 3. From question two you might know now, that this is not an easy question. Every peak caller has different parameters that can be optimised for your type of problem. If you find a spot that you think should be certainly a peak, than play around with the parameters and options of the peak caller, or switch to another algorithm.
>   > {: .solution}
> {: .question}
>
{: .hands_on}

# Conclusion


In this tutorial you imported raw eCLIP data, evaluated the quality of the read library before the read processing, removed adapter sequences and unique molecular identifier from the reads, aligned the reads to a reference genome, de-duplicated the read library, analysed the quality of the de-duplication and read mapping, found significantly enriched regions (peaks) for the protein RBFOX2 and finally scrutinised the peaks on potential conserved motifs and target RNAs. In addition, you took a short glimpse on the quality of the peak calling. Keep in mind, the read library for CLIP-Seq data might have different formats for different protocols (e.g, PAR-CLIP). However, the overall analysis you learned in this tutorial stays nearly the same.

### Remark about Workflow


In order to use the workflow linked to this tutorial, you have to create two lists of dataset pairs; one for you control and one for your CLIP experiment.

> <tip-title>Creating a list of dataset pairs.</tip-title>
>
> * Click on the check box icon in the history.
> ![builddatasetpairs1](../../images/transcriptomics_images/clipseq_build_list_data_set_pairs_1.png "Select data sets for your list of data set pairs.")
> * Select the data files for your experiment (or control).
> * Select **n of N selected** right to the check box icon.
> * Select **Build List of Dataset Pairs**
> * In the new window, if you see no files and a warning, then select **Clear filters**. After that, organise the files into forward-reverse pairs.
> ![builddatasetpairs2](../../images/transcriptomics_images/clipseq_build_list_data_set_pairs_2.png "Organise the data sets into pairs.")
>
>  ![builddatasetpairs3](../../images/transcriptomics_images/clipseq_build_list_data_set_pairs_3.png "Result after organising the data sets into pairs.")
> * Give the list a name and click on the button **Create list**.
{: .tip}


topic: transcriptomics
tutorial: rb-rnaseq
title: Reference-based RNAseq data analysis (long)
difficulty: intermediate
time_estimation: 1h
tools_json: []
requirements_json: []
tags_json: []
content_hash: b82a17460da9dbceb523e279234f6063
last_modified: 2026-05-15T22:28:17.630639
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/rb-rnaseq/tutorial.html

content:
# RNAseq: Reference-based

This tutorial is inspired by an exceptional [RNAseq course](http://chagall.med.cornell.edu/RNASEQcourse/) at the Weill Cornell Medical College compiled by Friederike Dündar, Luce Skrabanek, and Paul Zumbo and by tutorials produced by Björn Grüning (@bgruening) for Freiburg Galaxy instance. Much of Galaxy-related features described in this section have been developed by Björn Grüning (@bgruening) and configured by Dave Bouvier (@davebx).

RNAseq can be roughly divided into two "types":

 * **Reference genome-based** - an assembled genome exists for a species for which an RNAseq experiment is performed. It allows reads to be aligned against the reference genome and significantly improves our ability to reconstruct transcripts. This category would obviously include humans and most model organisms but excludes the majority of truly biologically intereting species (e.g., [Hyacinth macaw](https://en.wikipedia.org/wiki/Hyacinth_macaw));
 * **Reference genome-free** - no genome assembly for the species of interest is available. In this case one would need to assemble the reads into transcripts using *de novo* approaches. This type of RNAseq is as much of an art as well as science because assembly is heavily parameter-dependent and difficult to do well.

In this lesson we will focus on the **Reference genome-based** type of RNA seq.

## Experimental procedures affect downstream analyses

The *Everything's connected* slide by Dündar et al. (2015) explains the overall idea:

![Everything's connected](../../images/transcriptomics_images/everything_connected.png)

There is a variety of ways in which RNA is treated during its conversion to cDNA and eventual preparation of sequencing libraries. In general the experimental workflow includes the following steps:

 * RNA purification;
 * Reverse transcription using Reverse Transcriptase (RT), which produces the first strand of cDNA ("c" stands for *complimentary*);
 * Second strand synthesis using DNA polymerase;
 * Library preparation for sequencing.

In listing these basic steps we are ignoring a vast amount of details such as, for example, normalization strategies and procedures needed to deal with rare RNAs or degraded samples (see [Adiconis:2013](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3821180/)). Yet, there are two important experimental considerations that would effect the ways in which one analyses data and interprets the results. These are:

 * Priming for the first cDNA strand synthesis;
 * Stranded versus Non-stranded libraries.

### Priming for the first strand synthesis

Reverse Transcriptase (RT) requires a primer. One can leverage the fact that the majority of processed mRNAs are polyadenylated and use oligo-dT primer to (mostly) restrict cDNA synthesis to fully processed mRNAs. Alternatively one can use a mix of random oligonucleotides to prime RT at a multitude of internal sites irrespective of RNA type and maturation status:


![Oligo-dT vs. random priming](../../images/transcriptomics_images/dT_random.png)

**Oligo-dT vs. random priming**<br>
Oligo-dT (**A**) and random priming (**B**)

Depending on the choice of the approach one would have different types of RNAs included in the final sequencing outcome. For example, if one attempts to study RNAs that are not polyadenylated or not fully processed, it would be unwise to use oligo-dT priming approach.

### Strand-specific RNAseq

RNAs that are typically targeted in RNAseq experiments are single stranded (e.g., mRNAs) and thus have polarity (5' and 3' ends that are functionally distinct):

![Relationship between DNA and RNA orientation](../../images/transcriptomics_images/dna_rna.png)

**Relationship between DNA and RNA orientation**

During a typical RNAseq experiment the information about strandedness is lost after both strands of cDNA are synthesized, size selected, and converted into sequencing library. However, this information can be quite useful for various aspects of RNAseq analysis such as transcript reconstruction and quantification. There is a number of methods for creating so called *stranded* RNAseq libraries that preserve the strand information (an excellent overview in [Levin et al, Nat Meth, 2010](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3005310/)):

![Generation of stranded RNAseq libraries](../../images/transcriptomics_images/stranded_protocols.png "Generation of stranded RNAseq libraries (Levin et al, Nat Meth, 2010)")

****<br>
Different types of stranded library generation protocols from [Levin:2010](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3005310/)

Depending on the approach and whether one performs single- or paired-end sequencing there are multiple possibilities on how to interpret the results of mapping of these reads onto genome/transcriptome:

[![Effects of RNAseq library types](../../images/transcriptomics_images/lib_type.png)](https://sailfish.readthedocs.org/en/master/library_type.html)

**Effects of RNAseq library types**<br>
Image and description below is from [Sailfish documentation](https://sailfish.readthedocs.org/en/master/library_type.html)

The relative orientation of the reads is only relevant if the library is pair-ended. The possible options are:

 * **I** = inward;
 * **O** = outward;
 * **M** = matching (co-directional).

Library can be stranded (**S**) or unstranded (**U**). If this library is stranded than depending on the protocols reads (single reads or forward reads in a paired-end run) may originate from:

 * **F** = read 1 in paired-end sequencing or single-end read is derived from the Forward strand;
 * **R** = read 1 in paired-end sequencing or single-end read is derived from the Reverse strand.

So by combining the relative orientation of reads is I, O, or M (if reads are paired), strandedness or the library (S or U), and whether the reads originate from forward and reverse strand (F or R) there can be quite a number of possibilities:

* IU  - (an unstranded paired-end library where the reads face each other)
* SF  - (a stranded single-end protocol where the reads come from the forward strand)
* OSR - (a stranded paired-end protocol where the reads face away from each other,
      read1 comes from reverse strand and read2 comes from the forward strand).
      and so on...

However, in practice, if you use Illumina paired-end RNAseq protocols you are unlikely to uncover many of these possibilities. You will either deal with:

 * unstranded RNAseq data (**IU** type from above. Also called **fr-unstranded** in TopHat/Cufflinks jargon);
 * stranded RNAseq data produced with Illumina TrueSeq RNAseq kits and [dUTP tagging](https://nar.oxfordjournals.org/content/37/18/e123) (**ISR** type from above or **fr-firststrand** in TopHat/Cufflinks nomenclature).

The implication of stranded RNAseq is that you can distinguish whether the reads are derived from forward- or reverse-encoded transcripts:

![Stranded RNAseq data look like this](../../images/transcriptomics_images/stranded_result.png)

**Stranded RNAseq data look like this**<br>
This example contrasts unstranded and stranded RNAseq experiments. <font color="red">Red transcripts</font> are from + strand and <font color="blue">blue</font> are from - strand. In stranded example reads are clearly stratified between the two strands.  A small number of reads from opposite strand may represent anti-sense transcription. The image from GATC Biotech.

### Replicates: Biological or Technical and how many?

An RNAseq experiment without a sufficient number of replicates will be a waste of money. Replicates are essential to be able to correct for variation due to differences within/among organisms, cells, sequencing machines, library preparation protocols and numerous other potential factors. There are two types of replicates (as described by [Dündar:2015](http://chagall.med.cornell.edu/RNASEQcourse/Intro2RNAseq.pdf)):


 * **Technical replicates** can be defined as *different library preparations from the same RNA sample*. They should account for batch effects from the library preparation such as reverse transcription and PCR amplification. To avoid possible lane effects (e.g., differences in the sample loading, cluster amplification, and efficiency of the sequencing reaction), it is good practice to multiplex the same sample over different lanes of the same flowcell. In most cases, technical variability introduced by the sequencing protocol is quite low and well controlled.

* **Biological replicates**. There is an on-going debate over what kinds of samples represent true biological replicates. Obviously, the variability between different samples will be greater between RNA extracted from two unrelated humans than between RNA extracted from two different batches of the same cell line. In the latter case, most of the variation that will eventually be detected was probably introduced by the experimenter (e.g., slightly differing media and plating conditions). Nevertheless, this is variation the researcher is typically not interested in assessing, therefore the ENCODE consortium defines biological replicates as RNA from an independent growth of cells/tissue (ENCODE [2011](https://genome.ucsc.edu/ENCODE/protocols/dataStandards/ENCODE_RNAseq_Standards_V1.0.pdf)).

The number of replicates should be as high as practically possible. Most RNAseq experiments include three replicates and some have as many as 12 (see Schurch et al. [2015](https://arxiv.org/abs/1505.02017)).

## Read mapping

After sequencing is performed you have a collection of sequencing reads for each sample/replicate. In a reference-based RNAseq experiment these need to be mapped against the genome. Because in the case of eukaryotic transcriptome most reads originate from processed mRNAs lacking exons, they cannot be simply mapped back to the genome. Instead they can be separated into two categories:

 * Reads that map entirely within exons
 * Reads that cannot be mapped within an exon across their entire length because they span two or more exons

*Spliced* mappers have been developed to efficiently map transcript-derived reads against genome.

### TopHat, TopHat2, and HiSat

[Tophat](https://bioinformatics.oxfordjournals.org/content/25/9/1105.abstract) was one of the first tools designed specifically to address this problem by identifying potential exons using reads that do map to the genome, generating possible splices between neighboring exons, and comparing reads that did not initially map to the genome agaisnt these *in silico* created junctions:

[![TopHat and TopHat2: Mapping RNAseq regions to genome](../../images/transcriptomics_images/tophat.png)](https://bioinformatics.oxfordjournals.org/content/25/9/1105/F1.expansion.html)

**TopHat and TopHat2: Mapping RNAseq regions to genome**<br>
In TopHat reads are mapped against the genome and are separated into two categories: (1) those that map, and (2) those that initially unmapped (IUM). "Piles" of reads representing potential exons are extended in search of potential donor/acceptor splice sites and potential splice junctions are reconstructed. IUMs are then mapped to these junctions. Image from [Trapnell:2009](https://bioinformatics.oxfordjournals.org/content/25/9/1105.full).


[![TopHat has been subsequently improved with the development of TopHat2](../../images/transcriptomics_images/tophat2.png)](https://genomebiology.biomedcentral.com/articles/10.1186/gb-2013-14-4-r36)

**TopHat has been subsequently improved with the development of TopHat2**<br>
Image from [Kim:2012](https://genomebiology.biomedcentral.com/articles/10.1186/gb-2013-14-4-r36) summarizes steps involved in aligning of RNAseq reads with TopHat2

To further optimize and speed up spliced read alignment Kim at al. [2015](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4655817/) developed [HISAT](https://ccb.jhu.edu/software/hisat2/index.shtml). It uses a set of [FM-indices](https://en.wikipedia.org/wiki/FM-index) consisting one global genome-wide index and a collection of ~48,000 local overlapping 42 kb indices (~55,000 56 kb indices in HiSat2). This allows to find initial seed locations for potential read alignments in the genome using global index and to rapidly refine these alignments using a corresponding local index:


![Hierarchical Graph FM index in HiSat/HiSat2](../../images/transcriptomics_images/hisat.png)

**Hierarchical Graph FM index in HiSat/HiSat2**<br>
A part of the read (blue arrow) is first mapped to the genome using the global FM index. The HiSat then tries to extend the alignment directly utilizing the genome sequence (violet arrow). In (**a**) it succeeds and this read aligned as it completely resides within an exon. In (**b**) the extension hits a mismatch. Now HiSat takes advantage of the local FM index overlapping this location to find the appropriate matting for the remainder of this read (green arrow). The (**c**) shows a combination these two strategies: the beginning of the read is mapped using global FM index (blue arrow), extended until it reaches the end of the exon (violet arrow), mapped using local FM index (green arrow) and extended again (violet arrow). Image from [Kim:2015](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4655817/)

### STAR mapper

[STAR aligner](https://github.com/alexdobin/STAR) is a fast alternative for mapping RNAseq reads against genome utilizing uncompressed [suffix array](https://en.wikipedia.org/wiki/Suffix_array). It operates in [two stages](https://academic.oup.com/bioinformatics/article/29/1/15/272537/STAR-ultrafast-universal-RNA-seq-aligner). In the first stage it performs seed search:

![STAR's seed search](../../images/transcriptomics_images/star.png)

**STAR's seed search**<br>
Here a read is split between two consecutive exons. STAR starts to look for a *maximum mappable prefix* (MMP) from the beginning of the read until it can no longer match continuously. After this point it start to MMP for the unmatched portion of the read (**a**). In the case of mismatches (**b**) and unalignable regions (**c**) MMPs serve as anchors from which to extend alignments. Image from [Dobin:2013](https://bioinformatics.oxfordjournals.org/content/early/2012/10/25/bioinformatics.bts635.full.pdf+html).

At the second stage STAR stitches MMPs to generate read-level alignments that (contrary to MMPs) can contain mismatches and indels. A scoring scheme is used to evaluate and prioritize stitching combinations and to evaluate reads that map to multiple locations. STAR is extremely fast but requires a substantial amount of RAM to run efficiently.

## Transcript reconstruction

The previous step - mapping - assigns RNAseq reads to genomic locations and identifies splice junctions from reads that originate from different exons. At transcript reconstruction step this information is taken further in attempt to build transcript models. There is a number of tools for performing this task. A benchmarking paper by [Hayer:2015](https://bioinformatics.oxfordjournals.org/content/early/2015/09/03/bioinformatics.btv488.full.pdf+html) attempted to compare performance of existing approaches with one of the outcomes shown below:

[![Comparison of transcript reconsruction approaches](../../images/transcriptomics_images/rnaseq_comparison.png)](https://bioinformatics.oxfordjournals.org/content/early/2015/09/08/bioinformatics.btv488/transcriptomics_images/F5.large.jpg)

**Comparison of transcript reconsruction approaches**<br>
Here *recall* (the number of correctly constructed forms divided by the total number of real forms) versus *precision* (true positives divided by the sum of true positives and false positives) are plotted for seven transcript assemblers tested on two simulated datasets: *EnsemblPerfect* and *EnsemblRealistic*. The shaded region is indicating suboptimal performance (i.e., the white, unshaded region is "good"). The figure is from [Hayer:2015](https://bioinformatics.oxfordjournals.org/content/early/2015/09/03/bioinformatics.btv488.full.pdf+html).

Based on these results [Cufflinks](https://cole-trapnell-lab.github.io/cufflinks/) and [StringTie](https://ccb.jhu.edu/software/stringtie/) have satisfactory performence. The following discussion is based on inner workings of StringTie.

### Transcriptome assembly with StringTie

[StringTie](https://ccb.jhu.edu/software/stringtie/) assembles transcripts from spliced read alignemnts produced by tools such as STAR, TopHat, or HISAT and simultaneously estimates their abundances using counts of reads assigned to each transcript. The following images illustrates details of StringTie workflow:

![StringTie algorithm](../../images/transcriptomics_images/stringtie1.png)

**StringTie workflow**<br>
Image from [Pertea:2015](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4643835/)

In essence StringTie builds an alternative splice graph from overlapping reads in a given locus. In such a graph nodes correspond to exons (or, rather, contiguous regions of genome covered by reads; colored regions on the figure above), while edges are represented by reads connecting these exons. Next, it identifies a path within the splice graph that has the highest weight (largest number of reads on edges). Such path would correspond to an assembled transcript at this iteration of the algorithm. Because the edge weight is equal to the number of the reads StringTie estimates the coverage level for this transcript (see below) which can be used to estimate the transcript's abundance. Reads that are associated with the transcript that was just assembled are then removed and the graph is updated to perform the next iteration of the algorithm.

## Transcript quantification

Transcriptome quantification attempts to estimate expression levels of individuals transcripts. This is performed by assigning RNAseq reads to transcripts, counting, and normalization.

### Assigning reads to transcripts

To associate reads with transcripts they (the reads) need to be aligned to the transcriptome. Tools like Cufflinks and StringTie reconstruct transcripts from spliced read alignments generated by other programs (TopHat, HISAT, STAR), so they already have the information about which reads belong to each reconstructed transcript. Other tools such as [Sailfish](https://www.cs.cmu.edu/~ckingsf/software/sailfish/), [Kallisto](http://pachterlab.github.io/kallisto/), and [Salmon](http://combine-lab.github.io/salmon/) perform *lightweight* alignment of RNAseq reads against existing transcriptome sequences. The goal of lightweight alignment is to quickly distribute the reads across transcripts they likely originate from without worrying too much about producing high quality alignments. The upside of this is that the entire procedure can be performed very quickly. The downside is that these tools require high quality transcriptome as input, which is not a problem if you work with humans or mice, but is a problem if you are studying Hyacinth macaw or any other brilliantly colored creatures.

#### Lightweight alignment

[Sailfish](https://www.cs.cmu.edu/~ckingsf/software/sailfish/) has been initially designed to utilize [*k*-mer](https://en.wikipedia.org/wiki/K-mer) matching for finding association between reads and corresponding transcripts:

![Assigning reads to transcripts: Sailfish](../../images/transcriptomics_images/sailfish.png)

**Assigning reads to transcripts: Sailfish**<br>
Sailfish indexes input transcriptome for a fixed *k*-mer length and compares *k*-mers derived from RNAseq reads against this index. Image from [Patro:2014](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4077321/)

The current version of Sailfish uses [quasi-alignment](http://biorxiv.org/content/biorxiv/early/2015/10/22/029652.full.pdf) to extend exact matches found with *k*-mers:

![quasi-alignment](../../images/transcriptomics_images/quasi_aln.png)

**Quasi-alignment of reads in Sailfish**<br>
In Sailfish version [0.7.0](https://github.com/kingsfordgroup/sailfish/releases/tag/v0.7.0) and up transcriptome is concatenated into a single sequence using `$` separators from which a [suffix array](https://en.wikipedia.org/wiki/Suffix_array) and a [hash table](https://en.wikipedia.org/wiki/Hash_table) are constructed. A *k*-mer from an RNAseq read (green) is looked up in the hash table, which immediately gives its position in the suffix array allowing to extend the march as described in the legend and the [paper](http://biorxiv.org/content/biorxiv/early/2015/10/22/029652.full.pdf). Image from  [Srivastava:2015](http://biorxiv.org/content/biorxiv/early/2015/10/22/029652.full.pdf)

[Kallisto](https://pachterlab.github.io/kallisto/) also utilizes *k*-mer matching but uses a different data structure. It constructs a [De Bruijn graph](https://en.wikipedia.org/wiki/De_Bruijn_graph) from transcriptome input (pane **b** of the figure below). This graph is different from De Bruijn graphs used for genome assembly in that its nodes are *k*-mers and transcripts correspond to paths through the graph. To accommodate multiple transcripts that can lay along the same path (or sub-path) the paths are "colored" with each transcript given a distinct "color" (in genome assembly the graph is built from the reads and nodes usually correspond to overlaps between *k*-mers forming incoming and outgoing edges). Non-branching sections of the graph that have identical coloring are "glued" into contigs. Finally a [hash table](https://en.wikipedia.org/wiki/Hash_table) is built that stores the position of each transcriptome *k*-mer within the graph:

![Kallisto](../../images/transcriptomics_images/kallisto.png)

**Assigning reads to transcripts: Kallisto**<br>
Here a black read is being associated with a set consisting of red, blue, and green transcripts (**a**). First, a graph is built from transcriptome (**b**). Next, by finding common *k*-mers between the read and the graph the read is "threaded" along a path (**c** and **d**). The colors along that path would indicate which transcripts it is likely derived from. Specifically, this is done by taking intersection of "colors" (**c**). It this case the read is assigned to two transcripts: red and blue. Image from [Bray:2015](https://arxiv.org/pdf/1505.02710v2.pdf)

[Salmon](https://combine-lab.github.io/salmon/about/) does not use *k*-mer matching approach. Instead it creates [bwa](https://github.com/lh3/bwa)-like [FM-index](https://en.wikipedia.org/wiki/FM-index) and uses it to finds chains of *Maximal Exact Matches* (MEMs) and *Super Maximal Exact Matches* (SMEMs) between a read and the transcriptome.
[Patro:2015](http://biorxiv.org/content/biorxiv/early/2015/06/27/021592.full.pdf) define a MEM as "*a substring that is shared by the query (read) and reference (transcript) that cannot be extended in either direction without introducing a mismatch*". Similraly, a SMEM is defined as a "*MEM that is not contained within any other MEM on the query.*" One of the advantages of utilizing the FM-index is that a new index does not need to re-generated for a search with different set of parameters. In the case of Sailfish and Kallisto an index is dependent on *k*-mer length and has to be recomputed every time the *k* is changed. The overall schematics of Salmon operation is as follows:

![Salmon's method](../../images/transcriptomics_images/salmon.png)

**Assigning reads to transcripts: Salmon**<br>
Image from [Patro:2015](http://biorxiv.org/content/biorxiv/early/2015/06/27/021592.full.pdf)

### Estimating transcript levels

Once reads are apportioned across individual transcripts they can be quantified. There are several approaches for quantification.

#### Flow networks

StringTie, which performs assembly and quantification simultaneously converts splice graph into a flow network for which it solves [the maximum flow problem](https://en.wikipedia.org/wiki/Maximum_flow_problem). The maximum flow is such network represents the expression level for a given transcript:

![Maximum flow network](../../images/transcriptomics_images/stringtie2.png)

**StringTie flow network**<br>
Here each exon node from the splice graph is split into *in* and *out* nodes connected with an edge weighted by the number of reads corresponding to that exon. For example, the first exon is covered by seven reads and so the edge between 1-in and 1-out has a weight of 7. Expression level would correspond to the maximum flow through a path representing a given transcript. Image from [Pertea:2015](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4643835/)

#### Expectation Maximization

The Expectation/Maximization framework (EM) is utilized in a number of tools such as [eXpress](http://bio.math.berkeley.edu/eXpress/index.html) and more recently [Sailfish](http://www.cs.cmu.edu/~ckingsf/software/sailfish/), [Kallisto](http://pachterlab.github.io/kallisto/), and [Salmon](https://combine-lab.github.io/salmon/about/) (As an alternative strategy Salmon also utilizes [variational Bayesian method](http://arxiv.org/pdf/1308.5953v1.pdf). The principle of EM is nicely illustrated by [Lior Pachter](https://liorpachter.wordpress.com/) in his transcript quantification [review](http://arxiv.org/pdf/1104.3889v2.pdf). Suppose, as shown on the image below, there are three transcripts (green, red, and blue). There are five reads associated with these transcripts. One read (*d*) is unique to the red transcript, while others correspond to two (*b*, *c*, *e*) or three (*a*) transcripts. The EM is an iterative procedure. In the first round transcript abundances are initialized as equal (0.33 each as there are three transcripts) and during expectation reads are apportioned across transcripts based on these abundances. Next, during maximization step transcript abundances are re-calculated as follow. For red transcript we sum up fraction of each read as 0.33 + 0 + 0.5 + 1 + 0.5 for reads a, b, c, d, and e, respectively. We now divide this by the sum of read allocations for each transcript as 2.33 + 1.33 + 1.33 for red, green, and blue transcripts respectively. For all three transcript calculation will look like this:

<div>
	$$

\color{red}{red}   =  \frac{0.33 + 0.0 + 0.5 + 1.0 + 0.5}{2.33 + 1.33 + 1.33} = 0.47\\

\color{green}{green} =  \frac{0.33 + 0.5 + 0.0 + 0.0 + 0.5}{2.33 + 1.33 + 1.33} = 0.27\\

\color{blue}{blue}  =  \frac{0.33 + 0.5 + 0.5 + 0.0 + 0.0}{2.33 + 1.33 + 1.33} = 0.27

    $$
</div>

During next expectation stage read are re-apportioned across transcripts and the procedure is repeated until convergence:

![EM algorithm](../../images/transcriptomics_images/em.png)

**Expectation Maximization (EM)**<br>
Image from [Pacher:2011](https://arxiv.org/pdf/1104.3889v2.pdf)

### Understanding quantification metrics

As we've seen above quantification for a transcript is estimated using the number of associated reads. Yet the count is not a very good measure as it will be severely biased by multiple factors such as, for example, transcript length. Thus these counts need to be normalized. Normalization strategies can be roughly divided into two groups:

 * Normalization for comparison within a **single** sample;
 * Normalization for comparison among **multiple** samples/conditions.

In their [tutorial](http://chagall.med.cornell.edu/RNASEQcourse/) Dündar et al. have compiled a table summarizing various metrics. Below is description of normalization technique for within sample comparisons (between sample comparison can be found in the next section on differential expression analysis):

![RNAseq normalization metrics](../../images/transcriptomics_images/within_norm.png)

**RNAseq normalization metrics: Within sample comparisons**<br>
Table from Dündar et al. [2015](http://chagall.med.cornell.edu/RNASEQcourse/)

In addition, an ee this [xcellent overview of these metrics](https://haroldpimentel.wordpress.com/2014/05/08/what-the-fpkm-a-review-rna-seq-expression-units/).

#### <font color="red">&#9888; You should NEVER EVER use RPKM, FPKM, or TPM to compare expression levels <em>across</em> samples. These are RELATIVE measures! Consider yourself warned!</font>

## Finding expression differences

The goal of differential expression analysis (DE) is to find gene (DGE) or transcript (DTE) differences between conditions, developmental stages, treatments etc. In particular DE has two goals:

 * Estimate the *magnitude* of expression differences;
 * Estimate the *significance* of expression differences.

For this expression is estimated from read counts and attempts are made to correct for variability in measurements using replicates that are absolutely essential accurate results (see below). We begin our short discussion on DE by reproducing a figure from [Trapnell:2013](https://www.nature.com/nbt/journal/v31/n1/abs/nbt.2450.html) highlighting some of the challenges associated with judging expression differences from read counts:

![Differential expression: Read counts and Expression levels](../../images/transcriptomics_images/diff.png)

**Differential expression: Read counts and Expression levels**<br>
**Change in fragment count for a gene does not necessarily equal a change in expression**. (**a**) Simple read-counting schemes sum the fragments incident on a gene’s exons. The exon-union model counts reads falling on any of a gene’s exons, whereas the exon-intersection model counts only reads on constitutive exons. (**b**) Both of the exon-union and exon intersection counting schemes may incorrectly estimate a change in expression in genes with multiple isoforms. The true expression is estimated by the sum of the length-normalized isoform read counts. The discrepancy between a change in the union or intersection count and a change in gene expression is driven by a change in the abundance of the isoforms with respect to one another. In the top row, the gene generates the same number of reads in conditions A and B, but in condition B, all of the reads come from the shorter of the two isoforms, and thus the true expression for the gene is higher in condition B. The intersection count scheme underestimates the true change in gene expression, and the union scheme fails to detect the change entirely. In the middle row, the intersection count fails to detect a change driven by a shift in the dominant isoform for the gene. The union scheme detects a shift in the wrong direction. In the bottom row, the gene’s expression is constant, but the isoforms undergo a complete switch between conditions A and B. Both simplified counting schemes register a change in count that does not reflect a change in gene expression. Figure from [Trapnell:2013] (https://www.nature.com/nbt/journal/v31/n1/abs/nbt.2450.html)


The following discussion of DGE logic is reproduced from [Dündar:2015](http://chagall.med.cornell.edu/RNASEQcourse/).

To determine the genes whose read count differences between two conditions are greater than expected by chance, DGE tools must make assumptions about the distribution of read counts. The null hypothesis – that the mean read counts of the samples of condition **A** are equal to the mean read counts of the samples of condition **B** – is tested for each gene individually. One of the most popular choices to model the read counts is the [Poisson distribution](https://en.wikipedia.org/wiki/Poisson_distribution) because:

 * individual reads can be interpreted as binary data (Bernoulli trials): they either originate from gene *i* or not;
 * we are trying to model the discrete probability distribution of the number of successes (success = read is present in the sequenced library);
 * the pool of possible reads that could be present is large, while the proportion of reads belonging to
ene **i** is quite small.

The nice feature of a Poisson distribution is that variance = mean. Thus, if the RNA-seq experiment gives us a precise estimate of the mean read counts per condition, we implicitly know what kind of variance to expect for read counts that are not truly changing between two conditions. This, in turn, then allows us to identify those genes that show greater differences between the two conditions than expected by chance. While read counts of the same library preparation (*technical replicates*) can indeed be well approximated by the Poisson distribution, it has been shown that biological replicates have greater variance (noise) than expected. This [overdispersion](https://en.wikipedia.org/wiki/Overdispersion) can be captured with the [negative binomial distribution](https://en.wikipedia.org/wiki/Negative_binomial_distribution), which is a more general form of the Poisson distribution that allows the variance to exceed the mean. The square root of the dispersion is the coefficient of variation – SD/mean – after subtracting the variance we expect due to Poisson sampling.

In contrast to the Poisson distribution, we now need to estimate two parameters from the read counts: the mean as well as the dispersion. The precision of these estimates strongly depends on the number (and variation) of replicates – the more replicates, the better the grasp on the underlying mean expression values of unchanged genes and the variance that is due to biological variation rather than the experimental treatment. For most RNA-seq experiments, only two to three replicates are available, which is not enough for reliable mean and variance estimates. Some tools therefore compensate for the lack of replication by borrowing information across genes with similar expression values and shrink a given gene’s variance towards the regressed values. These fitted values of the mean and dispersion are then used instead of the raw estimates
o test for differential gene expression.

The best performing tools tend to be [edgeR](https://bioconductor.org/packages/release/bioc/html/edgeR.html), [DESeq/DESeq2](https://bioconductor.org/packages/release/bioc/html/DESeq2.html), and [limma-voom](https://www.bioconductor.org/packages/release/bioc/html/limma.html) (see Rapaport et al. ([2013](https://link.springer.com/article/10.1186/gb-2013-14-9-r95)); Soneson and Delorenzi ([2013](https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-14-91)); Schurch et al. ([2015](http://arxiv.org/abs/1505.02017)) for reviews of DGE tools). DESeq and limma-voom tend to be more conservative than edgeR (better control of false positives), but edgeR is recommended for experiments with fewer than 12 replicates (Schurch et al., [2015](http://arxiv.org/abs/1505.02017)).

## Let's try it

### The data

In this example we will use a downsampled version of simulated *Drosophila melanogaster* RNA-seq data used by Trapnell et al. [2012](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3334321/). These include two conditions (**C1** and **C2**), each containing three replicates (**R1**, **R2**, and **R3**) sequenced as a paired end library. Thus in total there are 12 fastq datasets.

Here is what to do to load the data:

#### Loading the data and create dataset collections

 Go to the data library (ask your instructor) and select all fastq files. Then Click `to History` button:

![Data libraries](../../images/transcriptomics_images/rnaseq_library.png)

The datasets will appear in your history:

![RNASeq data in history](../../images/transcriptomics_images/rnaseq_data_in_history.png)

Twelve datasets make a lot of clicking necessary. To avoid this annoyance we will combine them into two collections - **c1** and **c2** as shown in the video below. Also, see this [tutorial]({{site.baseurl}}/topics/galaxy-interface/tutorials/processing-many-samples-at-once/tutorial.html) for yet another explanation of dataset collections.
 <div class="embed-responsive embed-responsive-16by9"><iframe src="https://player.vimeo.com/video/163625221" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe></div>


### Mapping reads

We will map the reads with TopHat2. Select **TopHat** from **NGS: RNA Analysis** section of the tool menu (left pane of Galaxy's interface):

#### Mapping with TopHat2

In this case the input to TopHat is **not** individual datasets, but a collection instead. The [video](https://vimeo.com/163625221) above shows how to generate collection. Since we have created two collection as was described above, we will used them as inputs (note that **Is this single-end or paired-end data?** is set to `Paired-end (as collection)`). Make sure that the top part of TopHat interface looks like in the image below. Here the following parameters are set:

* **Mean Inner Distance between Mate Pairs** = `28` This is because paired reads are 100 bp and mean insert size is 228 bp so that 228 - (100 + 100) = 28
* **Use a built in reference genome or own from your history** = `Use a build-in genome` and `dm3` is selected. This is because the reads are from *D. melanogaster*.
* **TopHat settings to use** = `Full parameter list` This is done to be able to specify the strandedness of the library.
* **Library Type** = `FR First Strand`

![TopHat interface](../../images/transcriptomics_images/tophat_interface.png)

The same procedure is then repeated for collection **c2**. In the end it generates a lot of datasets in the history resulting in something resembling an image below. TopHat produces five types of output and because we started with dataset collections every one of the green boxes shown below is actually a collection of outputs for **c1** and **c2**, respectively.

![TopHat outputs for both collections](../../images/transcriptomics_images/tophat_output.png)

Let's now take a look at some of the alignments. We will use IGV for this purpose.

First, let's drill down to actual alignments produced by TopHat. For example, in figure shown above simply click on **TopHat on collection 14: accepted_hits** and you will see a list of datasets corresponding to alignments of reads derived from each conditions:

![Accepted hits on collection 14](../../images/transcriptomics_images/accepted_hits_1.png)

Now, click on **c2-r1x** and the following will appear:

![Accepted hits c2-r1x and D. melanogaster button](../../images/transcriptomics_images/accepted_hits_2.png)

Finally, use **D. melanogaster** link (highlighted above) and follow the on-screen instructions. By focusing IGV on genomic position `chrX:11,897,111-11,920,446` you will be able to see spliced alignments produced by TopHat:

![IGV TopHat](../../images/transcriptomics_images/igv_tophat.png)

and [sashimi plots](https://software.broadinstitute.org/software/igv/Sashimi) highlighting potential splice junctions:

![Sashimi plots](../../images/transcriptomics_images/sashimi.png)

### Performing differential expression analysis

Using mapped reads produced by TopHat we will perform analysis of differential gene expression using HTSeq/DESeq2 pipeline.

#### Gene-based read counting with HTseq-count

[`HTSeq-count`](https://www-huber.embl.de/users/anders/HTSeq/doc/count.html) is one of the most popular tools for gene quantification. `HTseq-count` gives you multiple choices on how to handle read mapping to multiple locations, reads overlapping introns, or reads that overlap more than one genomic feature:

[![HTseq-count read/feature overlap modes](../../images/transcriptomics_images/htseq_count.png)](https://www-huber.embl.de/users/anders/HTSeq/doc/count.html)

**`HTseq-count` read/feature overlap modes**<br>
The `htseq-count` script of the HTSeq suite offers three different modes to handle details of read–feature overlaps that are depicted here. The default of featureCounts is the behavior of the union option. Image is from [HTseq documentation](https://www-huber.embl.de/users/anders/HTSeq/doc/count.html); Caption by [Dündar:2015](http://chagall.med.cornell.edu/RNASEQcourse/)

Before we can use `HTseq-count` we need to download gene annotations for version `dm3` of the *Drosophila melanogaster* genome. We use version `dm3` because it is the same genome we have mapped reads against during the TopHat step.

#### Getting *Drosophila malanogaster* gene annotation from UCSC

Select **UCSC Main** from **Get Data** section of the menu. Within the UCSC Genome Browser interface set parameters as shown below. In particular make sure that **assembly** is set ti `dm3` and **output format** is set to `GTF`. Click **get output**.
[![UCSC table browser](../../images/transcriptomics_images/ucsc_dm3.png)](../../images/transcriptomics_images/ucsc_dm3.png)

This [GTF](https://www.ensembl.org/info/website/upload/gff.html) dataset will be used one of the input for HTseq-count.

`HTseq-count` takes two inputs: (1) mapped reads in BAM format and (2) a GTF dataset containing annotation of genes. Using these inputs it will compute the number of reads per gene.

#### Using HTseq-count in Galaxy

`htseq-count` needs strand information to proceed. The strand information is specified as `+`, `-`, or `.` (unknown) in a GTF dataset. `htseq-count` does not like `.` and will generate an error if such unstranded features appear in data. To prevent these errors from happening we will filter them out from GTF file using **Filter** tool from **Filter and Sort** section of tool menu. Here `c7 != "."` means that we need to filter all rows where strand column (column #7) contains a dot:

![Filter parameters](../../images/transcriptomics_images/filter_gtf.png)

Select **htseq-count** from **NGS: RNA analysis** section on the left side of the menu. Set parameters as shown below. The red arrow points that to enable `htseq-count` to see collections, you need to select the 'folder' button. In the case of this particular Galaxy history we will need to run `htseq-count` twice. Once on TopHat alignemnts for collection **c1** (dataset #37; shown below) and then on alignments for collection **c2** (dataset # 57).|
![htseq-count parameters](../../images/transcriptomics_images/htseq_count_interface.png)

This will generate read counts per gene.

#### Normalizing read counts and computing differential expression with `DESeq2`

`DESeq2` takes read counts produced by `HTseq-count` and apply size factor normalization. Specifically, `DESeq2` will:

 * For each gene, compute the geometric mean of read counts across all samples;
 * Every gene count in then divided by the geometric mean;
 * The median of these ratios is a sample's size factor used for normalization.

<font color="red">&#10148;</font> Note: For a comprehensive overview of differential gene expression with DESeq2 see [Love:2016](https://www.bioconductor.org/packages/3.3/bioc/vignettes/DESeq2/inst/doc/DESeq2.pdf).

#### DESeq2 in Galaxy

The `DESeq2` Galaxy's interface is shown below. `DESeq2` allows to incorporate multiple *factors* in the analysis. In our case we only have one factor, which we call **Conditions**. This is because we are trying to find genes that are differentially expressed between two conditions. The first condition will the first **factor level**, while condition 2 will be the second **factor level**. Here the input for this first factor level is set to a collection `84: htseq-count on collection 37` and the input for the second input is set to `92: htseq-count on collection 57`. Make sure that **Visualising the analysis results** is set to `Yes`:

![DESeq2 parameters](../../images/transcriptomics_images/deseq2_interface.png)

This will produce output as shown below. The columns are: (**1**) gene identifier, (**2**) mean normalised counts, averaged over all samples from both conditions, (**3**) logarithm (base 2) of the fold change, (**4**) the standard error estimate for the log2 fold change estimate, (**5**) [Wald test](https://en.wikipedia.org/wiki/Wald_test) statistic, (**6**) p value for the statistical significance of this change, and (**7**) *p*-value adjusted for multiple testing with the Benjamini-Hochberg procedure which controls false discovery rate ([FDR](https://en.wikipedia.org/wiki/False_discovery_rate)). There is only one gene with significant change in gene expression between conditions: `CG1803-RC` with *p*-value = 1.6x10<sup>-05</sup>

![DESeq2 output](../../images/transcriptomics_images/deseq2_output.png)

In addition to the list of genes DESeq2 outputs a graphical summary of the result. It includes a number of plots that should be used to evaluate the quality of the experiment. The histogram of *p*-values below shows that in our sample there is in fact just one instance of a significant *p*-value:

![p values histogram](../../images/transcriptomics_images/p_val_hist.png)

The [MA plot](https://en.wikipedia.org/wiki/MA_plot) below shows the relationship between the expression change (M) and average expression strength (A). Genes with adjusted *p*-value < 0.1 are in red (there is only one such gene in thi sample at the bottom of the graph):

![MA plot](../../images/transcriptomics_images/MA_plot.png)

The Principal Component Analysis ([PCA](https://en.wikipedia.org/wiki/Principal_component_analysis)) shows the separation between Condition 1 and 2. This type of plot is useful for visualizing the overall effect of experimental covariates and batch effects (each replicate is plotted as an individual data point):

![The Principal Component Analysis output](../../images/transcriptomics_images/pca.png)

A heatmap of sample-to-sample distance matrix gives us an overview over similarities and dissimilarities between samples:

![A heatmap of sample-to-sample distance matrix](../../images/transcriptomics_images/euc_dist.png)


topic: assembly
tutorial: assembly-decontamination
title: Decontamination of a genome assembly
difficulty: intermediate
time_estimation: 1H30M
requirements_json: []
tags_json: ["biodiversity"]
content_hash: 36b766b7f53b5c5e91999d0ee0e3924d
last_modified: 2026-05-27T09:01:47.799999
zenodo_link: https://zenodo.org/records/13367433
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/assembly-decontamination/tutorial.html

content:
When sequencing a genome, it is common that contamination from a foreign organism get mixed with the genomic material of our species of interest. For instance, if you are processing a whole body sample of an insect then you will sequence not only the insect, but everything on and inside of it. When building a reference genome, it is important to separate these contaminants from the genome of our species. Foreign DNA sequences could cause false positive identification when running BLAST analyses, the misidentification of genes that don't actually belong to the species, or they can be incorporated as 'reference' sequence for that species in the public archives when those sequences did not actually belong to that species.


In this tutorial, we will learn how to separate these contaminants from a genome assembly of a vertebrate. It will also include the removal of mitogenome contigs so that the final genome assembly contains only nuclear DNA. (A separate tutorial exists for assembling the mitogenome.)



> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Get data

For this tutorial, we are using the genome assembly of the zebra finch (*Taeniopygia guttata*) generated by the [Vertebrate Genome Project](https://vertebrategenomesproject.org/). We downsized the assembly to 50 of the largest scaffolds to keep the runtime reasonable, and added some contaminants for the sake of this tutorial.


> <comment-title>Run this analysis on a "real" assembly</comment-title>
>
> If you want to run this analysis on a real assembly generated by the [Vertebrate Genome Project](https://vertebrategenomesproject.org/), you can find the scaffolded assembly on [Genome Ark](https://www.genomeark.org/) as a remote repository and upload it to Galaxy (available on the three main Public Galaxy instances: .org, .eu, .org.au).
>
> {% snippet faqs/galaxy/datasets_import_from_remote_files.md location="Genome Ark -> species -> Taeniopygia_guttata -> bTaeGut2 -> assembly_vgp_hic_2.0 -> bTaeGut2.hic.hap1.s2.fasta" %}
>
{: .comment}


The analysis described in this tutorial will work with any assembly in FASTA format. If your initial dataset is compressed, use the tool `Convert compressed file to uncompressed.` first.


> <hands-on-title> Data Upload </hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo]({{ page.zenodo_link }}):
>
>    ```
>    https://zenodo.org/records/13367433/files/Contaminated_assembly.fasta
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Rename the dataset : `Contaminated Assembly`
>
>    {% snippet faqs/galaxy/datasets_rename.md %}
>
> 4. Check that the datatype is `fasta`
>
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="fasta" %}
>
>
{: .hands_on}

# Genome Masking


Genome masking refers to hiding regions of low complexity in the genome, such as repeats. Masking a genome reduces the noise and improves the efficacy of the decontamination by emphasizing the regions that contribute the most to the classification ({% cite saini2016gene %}, {% cite maskedref-website %}).

The first step of our analysis is therefore to mask our genome assembly. There are two types of masking, depending on the requirements of the tools we are using:

- **Soft Masking** The masked regions are replaced by lower case letters. This is the prefered method of masking when the tools understand it, because it does not cause any loss of information.
- **Hard Masking** The masked regions are replaced by a wildcard letter: `N`s for nucleotide sequences, `X`s for proteins. Use this technique if you are unsure whether the tools you will be using knows to ignore lower case letters.


> <details-title> Genome masking for everyone </details-title>
>
> Repeat masking is important for a lot of analyses like sequence alignments, classification, gene annotation. Repetitive and low complexity regions cause problems for search and clustering algorithms based on patterns. For gene annotation, repeats can cause non-specific gene hits.
>
{: .details}

In this tutorial we will use `Kraken2` to classify our contaminant scaffolds, `Blastn` to identify the mitochondrial sequences, and `DustMasker` to mask our genome. `Dustmasker` generates soft masked sequences, while `Kraken2` and `Blast` needs hard masked sequences, so we will need to do some file manipulations along the way.

Before running the analysis, ensure the sequences in your assembly are uppercase. This will ensure the soft masking can be done properly. *If* your sequences are not uppercase, then run the optional step "Convert the assembly file to upper case". We will then convert the lower case letter to `N`s to hard mask the sequences for `Kraken2` and `Blast`.

> <details-title> Why do the sequences need to be in uppercase? </details-title>
>
> Because soft masking uses lowercase to denote masked base pairs, the input sequence should not have lowercase sequences because then you cannot tell afterwards whether a sequence is lowercase because it started that way or because it was soft masked.
>
{: .details}


> <details-title> Optional : Convert the assembly file to upper case </details-title>
>
> 1. {% tool [Text transformation](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_sed_tool/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"File to process"*: `Contaminated assembly`
>    - *"SED Program"*: `s/^[^>].+/\U&/g` (Convert letters in the sequence to uppercase)
>
>
{: .details}


> <hands-on-title> Mask the Genome with Dustmasker </hands-on-title>
>
> 1. {% tool [NCBI BLAST+ dustmasker](toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_dustmasker_wrapper/2.14.1+galaxy1) %} with the following parameters:
>    - *"Subject database/sequences"*: `FASTA file from your history`
>        - {% icon param-file %} *"Nucleotide FASTA subject file to use instead of a database"*: `Contaminated assembly` (or the output of **Text transformation** {% icon tool %} if you converted your sequences to upper case)
>    - *"DUST level"*: `40`
>    - *"Output format"*: `FASTA`
>
> 2. Rename your file `Soft Masked assembly`
>
{: .hands_on}

Take a peek at your genome assembly. You can see that the beginning of the first scaffold has been converted to lower case, which masks repeats corresponding to the telomeres in the sequences.

> <hands-on-title> Soft Masking to Hard Masking </hands-on-title>
>
> 1. {% tool [Text transformation](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_sed_tool/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"File to process"*: `Soft Masked assembly` (output of **NCBI BLAST+ dustmasker** {% icon tool %})
>    - *"SED Program"*: `/^>/!y/atcgn/NNNNN/`: In sequence lines (not starting with ">"), replace each instance of lower case bases (a t c or g), including undefined ones (n), to `N`s).
>
> 2. Rename your file `Hard Masked assembly`
>
>    {% snippet  faqs/galaxy/analysis_regular_expressions.md %}
> 
>    > <comment-title> Unsure about your regex? </comment-title>
>    >
>    > Ask ChatGPT about it. E.g. [What is the sed command to convert a text to upper case?](https://chatgpt.com/share/de569538-0e4d-41ac-8380-2d52af62b3d7) or test with a [regex tester](https://sed.js.org/) website (Warning, some of these testers do not understand the lowercase and uppercase transformations)
>    > 
>    {: .comment}
>
> 
{: .hands_on}

Take a look at your dataset and see that the lower case bases have been converted to `N`s.

# Identify Non-Target Contaminants

[Kraken2](https://ccb.jhu.edu/software/kraken2/) is a tool for taxonomic classification. It matches the kmers found in our assemblies to the ones stored in its databases. For the purpose of decontamination, we only need to identify possible contaminants, not the sequences belonging to our sampled organisms. Therefore, we only need to use a database containing likely contaminants such as bacteria, virus, and human (always wear gloves, folks!).
The best option at the time of this tutorial's writing is the PlusPF database which contains the RefSeq Standard (archaea, bacteria, viral, plasmid, human, UniVec_Core) plus protozoa and fungi data.



> <hands-on-title> Identify the taxonomy of our sequences </hands-on-title>
>
> 1. {% tool [Kraken2](toolshed.g2.bx.psu.edu/repos/iuc/kraken2/kraken2/2.1.1+galaxy1) %} with the following parameters:
>    - *"Single or paired reads"*: `Single`
>        - {% icon param-file %} *"Input sequences"*: `Hard Masked assembly` (output of **Text transformation** {% icon tool %})
>    - *"Print scientific names instead of just taxids"*: `Yes`
>    - *"Confidence"*: `0.3`
>    - *"Split classified and unclassified outputs?"*: `Yes`
>    - *"Select a Kraken2 database"*: `PlusPF (2021-05-17)`
>
>
{: .hands_on}


Kraken provides us with three outputs:
- A table containing the classification information for each sequence
- A fasta file containing the classified sequences (our contaminants)
- A fasta file containing the unclassified sequences (our precious zebra finch)

We could decide to use the fasta file of unclassified sequences as our assembly, but remember that our genome is hard masked. That means that we lost information. Because we don't use it now doesn't mean we never will. So instead of using the masked genome, we will remove the classified sequences from our original assembly.


But first, we are naturally curious people, and we want to know what type of contaminants we had in our sample.


## Investigate our contaminants

Take a look at the classification output of Kraken2. You can see five columns containing:

1. `C` / `U`: a one letter code indicating that the sequence was either classified or unclassified.
2. The sequence ID.
3. The taxonomy ID Kraken2 used to label the sequence; this is 0 if the sequence is unclassified.
4. The length of the sequence in bp.
5. A space-delimited list indicating the LCA mapping of each k-mer in the sequence(s). For example, "562:13 561:4 A:31 0:1" would indicate that:
    - the first 13 k-mers mapped to taxonomy ID #562
    - the next 4 k-mers mapped to taxonomy ID #561
    - the next 31 k-mers contained an ambiguous nucleotide
    - the next k-mer was not in the database

To identify our contaminants, we only need the information from the first three columns, and only for the sequences that are classified. So we will start by extracting our three columns and then filtering out unclassified sequences.

> <hands-on-title> Extract classification information </hands-on-title>
>
> 1. {% tool [Cut](Cut1) %} with the following parameters:
>    - *"Cut columns"*: `c1,c2,c3`
>    - *"Delimited by"*: `Tab`
>    - {% icon param-file %} *"From"*: `Kraken2 on data X: Classification` (output of **Kraken2** {% icon tool %})
>
>
>    > <comment-title> Columns definitions </comment-title>
>    >
>    > The three columns that we extracted contain:
>    > 1. Whether the sequence is classified `C` or unclassified `U`
>    > 2. The sequence name
>    > 3. The sequence classification
>    >
>    {: .comment}
>
>    > <comment-title> What if my organism is in the database? </comment-title>
>    >
>    > The filtering of contaminants is based on their presence in the database. Any sequence classified by Kraken2 is considered as a contaminant. If you are working on an organism that is present in the database, this filtering will not work.
>    >
>    > *If you can't find a database that include contaminants but exclude your organism, you will need to process the outputs of Kraken differently.* Instead of filtering the sequences on classified/unclassified, select the sequences that do not match the taxonomic ID of your species. That is true for the rest of the section.
>    {: .comment}
>
{: .hands_on}


> <hands-on-title> Remove unclassified sequences </hands-on-title>
>
> 1. {% tool [Filter](Filter1) %} with the following parameters:
>    - {% icon param-file %} *"Filter"*: `Cut on data X` (output of **Cut** {% icon tool %})
>    - *"With following condition"*: `c1!='U'`
>
> 2. Rename the output `Contaminants informations`
>
{: .hands_on}


> <question-title></question-title>
>
>  How many different contaminants are in our sample?
>
> > <solution-title></solution-title>
> >
> >  We found two contaminants in our sample: Salmonella and Yersinia phage phiR1-37.
> >
> {: .solution}
>
{: .question}


## Get the list of contaminants to remove from our assembly

In order to remove the contaminant sequences from our assembly, we need a list of the contaminants names, that we will get by extracting the contaminant IDs from the Kraken output.


> <hands-on-title> Get contaminant sequence names </hands-on-title>
>
> 1. {% tool [Cut](Cut1) %} with the following parameters:
>    - *"Cut columns"*: `c2`
>    - *"Delimited by"*: `Tab`
>    - {% icon param-file %} *"From"*: `Contaminants informations` (output of **Filter** {% icon tool %})
>
> 2. Rename the output `List of Contaminants`
>
{: .hands_on}


Now that we have our list of contaminants, we will go through a similar process to extract mitochondrial sequences.


# Identify mitochondrial DNA

We start with running Blastn against a database containing mitochondrial sequences.

> <hands-on-title> Blastn against RefSeq Mitochondrion </hands-on-title>
>
> 1. {% tool [NCBI BLAST+ blastn](toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastn_wrapper/2.14.1+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"Nucleotide query sequence(s)"*: `Hard Masked assembly` (output of **Text transformation** {% icon tool %})
>    - *"Subject database/sequences"*: `Locally installed BLAST database`
>        - *"Nucleotide BLAST database"*: `RefSeq Mitochondrion`
>    - *"Type of BLAST"*: `blastn - Traditional BLASTN requiring an exact match of 11, for somewhat similar sequences`
>    - *"Output format"*: `Tabular (select which columns)`
>        - *"Standard columns"*: `qseqid, sseqid, length, qstart, qend, evalue`
>        - *"Extended columns"*: `qlen`
>        - *"Miscellaneous columns"*: `qcovs, qcovhsp`
>    - *"Advanced Options"*: `Hide Advanced Options`
>
>
{: .hands_on}

This step may take a little while, so you have time to make yourself a cup of coffee.

Next, we parse the result of the Blastn to extract a report on the alignements between our assemblies and the database, and a list of the scaffolds aligning with mitochondrial sequences.


> <hands-on-title> Extract alignement informations from the Blast output</hands-on-title>
>
> 1. {% tool [Parse mitochondrial blast](toolshed.g2.bx.psu.edu/repos/iuc/parse_mito_blast/parse_mito_blast/1.0.2+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Tabular file generated by mito-blast"*: `blastn Text transformation on data X vs 'refseq_mitochondrion'` (output of **NCBI BLAST+ blastn** {% icon tool %})
>
> 2. Rename the output `List of Mitochondrial sequences`
>
{: .hands_on}



> <question-title></question-title>
>
>  How many scaffolds belonging to the mitochondria are in our sample?
>
> > <solution-title></solution-title>
> >
> >  We found two mitochondrial scaffolds in our assembly.
> >
> {: .solution}
>
{: .question}



# Remove contaminants and mitochondrial DNA from the assembly

From the previous sections, we obtain two files containing a list of sequences belonging to contaminants and one belonging to the mitochondria. We will merge these two lists to clean up our original assembly.


> <hands-on-title> Concatenate lists of scaffolds to remove </hands-on-title>
>
> 1. {% tool [Concatenate datasets](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.1) %} with the following parameters:
>    - {% icon param-files %} *"Datasets to concatenate"*: `List of Contaminants`, `List of Mitochondrial sequences`
>
> 2. Rename the output `Sequences to remove`
>
{: .hands_on}

Now we will use gfastats, a tool box for genome assemblies with capability for manipulations and statistics generation.

> <hands-on-title> Remove scaffolds from the original genome assembly </hands-on-title>
>
> 1. {% tool [gfastats](toolshed.g2.bx.psu.edu/repos/bgruening/gfastats/gfastats/1.3.6+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `output` (Input dataset)
>    - *"Specify target sequences"*: `Enabled`
>        - {% icon param-file %} *"Exclude specific intervals"*: `Sequences to remove` (output of **Concatenate datasets** {% icon tool %}). This input can be a bed file with sequence coordinates, or, in our case, a simple list of sequence IDs to remove from the fasta file. 
>    - *"Tool mode"*: `Genome assembly manipulation`
>        - *"Output format"*: `FASTA.gz`
>
>
{: .hands_on}

# Conclusion

In this tutorial we used taxonomic classification to identified contaminants and Blast to identify mitochondrial sequences in a genome assembly. We then removed these sequences to keep only the nuclear DNA of our species of interest.

Now that we have removed the contaminants and mitochondrial DNA from our assembly, it is ready for manual curation!

tools_json:
["Text transformation", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_sed_tool/1.1.1", "NCBI BLAST+ dustmasker", "toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_dustmasker_wrapper/2.14.1+galaxy1", "Kraken2", "toolshed.g2.bx.psu.edu/repos/iuc/kraken2/kraken2/2.1.1+galaxy1", "Cut", "Cut1", "Filter", "Filter1", "NCBI BLAST+ blastn", "toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastn_wrapper/2.14.1+galaxy1", "Parse mitochondrial blast", "toolshed.g2.bx.psu.edu/repos/iuc/parse_mito_blast/parse_mito_blast/1.0.2+galaxy0", "Concatenate datasets", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.1", "gfastats", "toolshed.g2.bx.psu.edu/repos/bgruening/gfastats/gfastats/1.3.6+galaxy0"]


topic: assembly
tutorial: hybrid_denovo_assembly
title: Hybrid genome assembly - Nanopore and Illumina
difficulty: intermediate
time_estimation: 2h
requirements_json: []
tags_json: ["assembly", "nanopore", "illumina", "denovo", "genomics"]
content_hash: f7fba721d7cbaa2c01ee001e5719f13c
last_modified: 2026-05-27T09:01:47.804376
zenodo_link: https://doi.org/10.5281/zenodo.15756327
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/hybrid_denovo_assembly/tutorial.html

content:
*Assemble a genome! Learn how to create and assess genome assemblies using the powerful combination of Nanopore and Illumina reads.*

This tutorial explores how long and short read data can be combined to produce a high-quality 'finished' bacterial genome sequence. Termed 'hybrid assembly', we will use read data produced from two different sequencing platforms, Illumina (short read) and Oxford Nanopore Technologies (long read), to reconstruct a bacterial genome sequence.

In this tutorial, we will perform 'de novo assembly'. De novo assembly is the process of assembling a genome from scratch using only the sequenced reads as input - no reference genome is used.  This approach is common practise when working with microorganisms, and has seen increasing use for eukaryotes (including humans) in recent times.  

Using short read data (Illumina) alone for de novo assembly will produce a complete genome, but in pieces (commonly called a 'draft genome'). For the genome to be assembled into a single chromosome (plus a sequence for each plasmid), reads would need to be longer than the longest repeated element on the genome (usually ~7,000 base pairs, Note: Illumina reads are 350 base maximum). Draft bacterial genome sequences are cheap to produce (less than AUD\$60) and useful (>300,000 draft *Salmonella enterica* genome sequences published at NCBI [https://www.ncbi.nlm.nih.gov/pathogens/organisms/](https://www.ncbi.nlm.nih.gov/pathogens/organisms/)), but sometimes you need a high-quality 'finished' bacterial genome sequence.  There are <1,000 'finished' or 'closed' *Salmonella enterica* genome sequences.

In these cases, long reads can be used together with short reads to produce a high-quality assembly. Nanopore long reads (commonly >40,000 bases) can fully span repeats, and reveal how all of the genome fragments should be arranged. Long reads currently have higher error rates than short reads, so the combination of technologies is particularly powerful. Long reads provide information on the genome structure, and short reads provide high base-level accuracy.  

Combining read data from the long and short read sequencing platforms allows the production of a complete genome sequence with very few sequence errors, but the cost of the read data is about AUD$1,000 to produce the sequence. Understandably, we usually produce a draft genome sequence with very few sequence errors using the Illumina sequencing platform.

Nanopore sequencing technology is rapidly improving, expect the cost difference to reduce!!

- **Data:** Nanopore reads, Illumina reads, bacterial organism (*Bacillus subtilis*) reference genome
- **Tools:** Flye, Pilon, Unicycler, Quast, Busco
- **Pipeline:** Hybrid de novo genome assembly - Nanopore draft Illumina polishing
- **Pipeline:** Hybrid de novo genome assembly - Unicycler

This Galaxy tutorial based on material from the [Hybrid genome assembly - Nanopore and Illumina](https://www.melbournebioinformatics.org.au/tutorials/tutorials/hybrid_assembly/nanopore_assembly/) tutorial and workshop created by [Melbourne Bioinformatics](https://mdhs.unimelb.edu.au/melbournebioinformatics) at the University of Melbourne.


> <agenda-title></agenda-title>
> 
> In this tutorial we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Background

## How do we produce the genomic DNA for a bacterial isolate?

Traditional *in vitro* culture techniques are important. Take a sample (e.g. a swab specimen from an infected sore) and streak a 'loopful' on to solid growth medium that suppoprts the growth of the bacteria. **Technology from the time of Louis Pasteur!**

Mixtures of bacterial types can be sequenced e.g. prepare genomic DNA from environmental samples containing bacteria - water, soil, faecal samples etc. (Whole Metagenome Sequencing)

![image of bacterial growth of a sample](../../images/denovo_assembly/sample_bacterial_growth.png)

One colony contains $$10^7 – 10^8$$ cells. The genomic DNA extracted from one colony is enough for Illumina sequencing. Larger amounts of genomic DNA are required for Nanopore sequencing.

## Shotgun sequencing - Illumina Sequencing Library

Genomic DNA is prepared for sequencing by fragmenting/shearing: multiple copies of Chromosome + plasmid  --> ~500 bp fragments

Note: Nanopore sequencing - there is usually no need to shear the genomic DNA as specialist methods are used to minimise shearing during DNA preparation. For Nanopore sequencing, the longer the DNA fragments, the better!

# Nanopore draft assembly, Illumina polishing

In this section, you will use `Flye` to create a draft genome assembly from Nanopore reads. We will perform assembly, then assess the quality of our assembly using two tools: `Quast` and `Busco`.

![workflow diagram for hybrid assembly starting with nanopore read](../../images/denovo_assembly/nanopore_illumina_hybrid_assembly.png)

## Upload data

Let's start with uploading the data.

> <hands-on-title>Import the data</hands-on-title>
> 
> 1. Create a new history for this tutorial and give it a proper name
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 2. Import from [Zenodo](https://zenodo.org/uploads/15756328):
>   
>    - FASTQ file with illumina forward reads: `illumina_reads_1.fastq`
>    - FASTQ file with illumina reverse reads: `illumina_reads_2.fastq`
>    - FASTQ file with nanopore reads: `nanopore_reads.fastq`
>    - FASTA file with reference genome: `reference_genome.fasta`
> 
>    ```
>    https://zenodo.org/records/15756328/files/illumina_reads_1.fastq
>    https://zenodo.org/records/15756328/files/illumina_reads_2.fastq
>    https://zenodo.org/records/15756328/files/nanopore_reads.fastq
>    https://zenodo.org/records/15756328/files/reference_genome.fasta
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
> 
{: .hands_on}


## A baseline for "high-quality" assemblies

To begin, we will identify what a high-quality assembly looks like.

When running assembly tools, we want to check the quality of assemblies we produce.
It is paramount that genome assemblies are high-quality for them to be useful. To get a baseline for what is considered a "high-quality" assembly, we will first run a common assembly QC tool - `Busco` - on a published genome similar to the organism we are working with today.

In your imported files, you should see a `reference_genome.fasta` item.
This is the published genome we will compare against.

### QC with Busco

BUSCO analysis uses the presence, absence, or fragmentation of key genes in an assembly to determine its quality.

BUSCO genes are specifically selected for each taxonomic clade, and represent a group of genes that each organism in the clade is expected to possess. At higher clades, 'housekeeping genes' are the only members, while at more refined taxa such as order or family, lineage-specific genes can also be used.

We expect the reference genome to have all of these genes. When running `Busco`, we expect it to find most (if not all) of these in the assembly.

Find and select the `Busco` tool in the tools panel using the search bar.
In this tutorial, we know our organism is within the 'Bacillales' order.

> <hands-on-title>Run Busco</hands-on-title>
>
> 1. {% tool [Busco](toolshed.g2.bx.psu.edu/repos/iuc/busco/busco/5.8.0+galaxy1) %}:
>    - *"Sequences to analyse"*: `reference_genome.fasta`
>    - *"Mode"*:
>      - *"Select a gene predictor"*: `Augustus`
>    - *"Auto-detect or select lineage?"*: `Select lineage`
>      -  *"Lineage"*: `Bacillales`
>    - *"Which outputs should be generated"*: `Short summary text`
>   
> 2. ***Rename*** the `short summary` output to `Busco on Reference`.
> 
> 3. View the `Busco on Reference` output:
>    - It may look something like this:
>      ![output busco report for reference assembly](../../images/denovo_assembly/busco_reference_assembly.png)
> 
{: .hands_on}

> <question-title></question-title>
> 
> What can you conclude from the BUSCO report?
> 
> > <solution-title></solution-title>
> > 
> > - It seems that `Busco` could find almost all expected genes in the reference genome assembly.
> > - By looking at the results, we see that we have 449 / 450 Complete BUSCOs, and one Fragmented BUSCO.
> > - This will form the baseline for the `Busco` QC results expected of a high-quality genome assembly.
> > - From here, we will use our input DNA sequence data to assemble the genome of the sequenced organism, and will compare the QC results to that of the published `reference_genome.fasta` assembly.
> > 
> {: .solution}
> 
{: .question}


## Draft assembly with Flye + Nanopore reads

Our first assembly will use the long-read data to create a draft genome, then the short-read data to "polish" (improve) the draft into a better assembly.

We will start by using a long-read assembly tool called `Flye` to create an assembly using the Nanopore long-read data.

Search for `Flye` in the tools panel search bar and select the tool.

> <hands-on-title>Run Flye</hands-on-title>
>
> 1. {% tool [Flye](toolshed.g2.bx.psu.edu/repos/bgruening/flye/flye/2.9.6+galaxy0) %}:
>    - *"Input reads"*: `nanopore_reads.fastq`
>    - Leave all other parameters as default
>    - Scroll down and run Flye by clicking the blue `Run tool` button at the bottom of the page.
> 
> 2. View output:
>    - `Flye` produces a number of outputs. We only need the 'consensus' fasta file. You can delete the other outputs.
> 
> 3. ***Rename*** the `Flye on data XX: consensus` output to `Flye: Assembly`
> 
{: .hands_on}
 

## Assessing the Flye draft assembly quality

### Busco

We need to check if our assembly is good quality or not. It is paramount that genome assemblies are high-quality for them to be useful.

> <hands-on-title>Run Busco</hands-on-title>
>
> 1. {% tool [Busco](toolshed.g2.bx.psu.edu/repos/iuc/busco/busco/5.8.0+galaxy1) %}:
>    - *"Sequences to analyse"*: `Flye: Assembly`
>    - *"Mode"*:
>      - *"Select a gene predictor"*: `Augustus`
>    - *"Auto-detect or select lineage?"*: `Select lineage`
>      -  *"Lineage"*: `Bacillales`
>    - *"Which outputs should be generated"*: `Short summary text`
> 
> 2. ***Rename*** the `short summary` output to `Busco on Flye: Assembly`.
> 
> 3. View the `Busco on Flye: Assembly` output:
>    - It may look something like this:
>      ![output busco report for draft assembly](../../images/denovo_assembly/busco_draft_assembly.png)
>    - The `full table` is also useful. It gives a detailed list of the genes we are searching for, and information about whether they were missing, fragmented, or complete in our assembly.
>      ![output busco table for draft assembly](../../images/denovo_assembly/busco_table_draft_assembly.png)
> 
{: .hands_on}


> <question-title></question-title>
> 
> How does this compare with the reference genome BUSCO report?
> 
> > <solution-title></solution-title>
> > 
> > We see that many genes are fragmented or missing. Our draft genome assembly isn't as good as the reference genome ***yet***. This is because we have so far only used the Nanopore long-read sequences, which have higher base-level error rates than short reads. In the next steps we will use Illumina short reads to correct for errors in the assembled Nanopore reads.
> > 
> {: .solution}
> 
{: .question}


### Quast

Aside from `Busco`, we can use another method to perform assembly QC. `Quast` allows us to compare two assemblies to determine their similarity.

Although the organism we sequenced may be different, we can use `Quast` to compare our assembly with the provided reference genome to see how similar they are on the individual base level.

Search for the `Quast` tool in the tools panel.

> <hands-on-title>Run Quast</hands-on-title>
>
> 1. {% tool [Quast](toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.3.0+galaxy0) %}:
>    - *"Assembly mode"*: `Individual assembly`
>    - *"Use customized names for the input files?"*: `No, use dataset names`
>    - *"Contigs/scaffolds file"*: `Flye: Assembly`
>    - *"Type of assembly?"*: `Genome`
>    - *"Use a reference genome?"*: `Yes`
>      - *"Reference genome"*: `reference_genome.fasta`
>    - *"Output files"*: Select `HTML reports`, `Tabular reports`
> 
> 2. ***Rename*** the `tabular report` output to `Quast on Flye: Assembly`.
> 
> 3. View output:
>    - `Quast` will produce a HTML report summarising it's results.
>    - Open the report. It may look something like this:
>      ![output quast report for reference assembly](../../images/denovo_assembly/quast_draft_assembly.png)
>    - Note the:
>      - Genome fraction (%)
>      - \# mismatches per 100 kbp
>      - \# indels per 100 kbp
>      - \# contigs information
> 
{: .hands_on}

> <question-title></question-title>
> 
> What can you conclude about our draft assembly?
> 
> > <solution-title></solution-title>
> > 
> > - From the output of `Quast`, our draft assembly seems to have good coverage and not too many contigs.
> > - Unfortunately, the mismatch / indel rate is quite high.
> > - Although we don't expect our organism to be ***identical*** to the supplied reference, we would expect fewer mismatches and indels, as the provided reference genome is ***very*** similar to organism which was sequenced.
> > 
> {: .solution}
> 
{: .question}


## Assembly Polishing with Pilon

We should be able improve our draft assembly with the Illumina reads available and correct some of these errors.

This process involves two steps. We will first align the Illumina reads to our draft assembly, then supply the mapping information to `Pilon`, which will use this alignment information to error-correct our assembly.

Illumina reads have much higher per-base accuracy than Nanopore reads. We will map the Illumina reads to our draft assembly using a short-read aligner called `BWA-MEM`. Then we can give `Pilon` this alignment file to polish our draft assembly.

### Map Illumina reads to draft assembly

Search for `Map with BWA-MEM` in the tools panel.

> <hands-on-title>Map with BWA-MEM</hands-on-title>
>
> 1. {% tool [Map with BWA-MEM](toolshed.g2.bx.psu.edu/repos/devteam/bwa/bwa_mem/0.7.19) %}:
>    - *"Will you select a reference genome from your history or use a built-in index?"*: `Use a genome from history and build index`
>    - *"Use the following dataset as the reference sequence"*: `Flye: Assembly`
>    - *"Single or Paired-end reads"*: `Paired `
>    - *"Select first set of reads"*: `illumina_reads_1.fastq`
>    - *"Select second set of reads"*: `illumina_reads_2.fastq`
> 
> 2. View output:
>    - The output will be a BAM file (Binary Alignment Map). This is tabular data recording information about how reads were aligned to the draft assembly.
>    - We can now use this output BAM file as an input to `Pilon`.
> 
> 3. ***Rename*** the output to `Flye: Short read alignments`
> 
{: .hands_on}


### Polish assembly with Pilon

Search for `Pilon` in the tools panel.

> <hands-on-title>Run Pilon</hands-on-title>
>
> 1. {% tool [Pilon](toolshed.g2.bx.psu.edu/repos/iuc/pilon/pilon/1.20.1) %}:
>    - *"Source for reference genome used for BAM alignments"*: `Use a genome from History`
>      - *"Select a reference genome"*: `Flye: Assembly`
>    - *"Type automatically determined by pilon"*: 
>      - *"Input BAM file"*: `Flye: Short read alignments`
>    - *"Variant calling mode"*: `No`
> 
> 2. View output:
>    - `Pilon` gives a single output file - the polished assembly.
> 
> 3. ***Rename*** the output to `Flye: Polished assembly`
> 
{: .hands_on}



### Compare draft and polished assemblies

We are now interested to see how much `Pilon` improved our draft assembly.

> <hands-on-title>Run Quast</hands-on-title>
>
> Run `Quast` as before with the new `Flye: Polished assembly` data
> 
> 1. Select the history item for our initial `Quast` job, then click the rerun {% icon dataset-rerun %} button. This will load the settings used for the previous `Quast` job.
> 
> 2. Change the *"Contigs/scaffolds file"* input to `Flye: Polished assembly`.
> 
> 3. Click the `Run Tool` button to submit the job.
>
> 4. ***Rename*** the `tabular report` output to `Quast on Flye: Polished assembly`.
> 
> 5. After `Quast` has finished, open the HTML report. 
>    - Make note of `# mismatches per 100 kbp` and `# indels per 100 kbp`.
> 
{: .hands_on}

> <question-title></question-title>
> 
> Has our assembly improved?
>
> > <solution-title></solution-title>
> > 
> > Yes, we now have lower mismatches and indels per 100kbp.
> > 
> {: .solution}
> 
{: .question}

> <hands-on-title>Run Busco</hands-on-title>
>
> Run `Busco` as before with the new `Flye: Polished assembly` data
> 
> 1. Select the history item for our initial `Busco` job, then click the rerun {% icon dataset-rerun %} button. This will load the settings used for the previous `Busco` job.
> 
> 2. Change the *"Sequences to analyse"* input to `Flye: Polished assembly`.
> 
> 3. Click the `Run Tool` button to submit the job.
>
> 4. ***Rename*** the `short summary` output to `Busco on Flye: Polished assembly`.
> 
> 5. After `Busco` has finished, open the `Busco on Flye: Polished assembly` output.
> 
{: .hands_on}

> <question-title></question-title>
> 
> Have we identified more expected genes?
>
> > <solution-title></solution-title>
> > 
> > Yes, we now identified 423 complete BUSCO genes.
> > 
> {: .solution}
> 
{: .question}


## Section Summary

All going well, the polished assembly should be much higher quality than our draft.

The per-base accuracy of our assembly contigs should have markedly improved. This is reflected in the lower mismatches and indels per 100kbp reported by `Quast`, and the higher number of complete BUSCO genes. Our contiguity and coverage (as measured by the genome fraction (%) statistic reported by `Quast`) may not show the same level of improvement, as the polishing step is mainly aimed at improving per-base contig accuracy.

Our next step is to use a purpose-built hybrid de novo assembly tool, and compare its performance with our sequential draft + polishing approach.


## Section Questions

> <question-title></question-title>
> 
> Which read set - short or long - was used to create our draft?
>
> > <solution-title></solution-title>
> > 
> > Long reads (Nanopore) were used to create the draft. Long reads allow excellent re-creation of the proper structure of the genome, and adequately handle repeat regions. The drawback of long reads is a higher error rate of the technology compared to short reads. This results in more mismatches and indels.
> > 
> {: .solution}
> 
{: .question}

> <question-title></question-title>
> 
> How was the draft polished?
>
> > <solution-title></solution-title>
> > 
> > Illumina reads have higher per-base accuracy than Nanopore. Illumina reads were aligned to the draft assembly, then `Pilon` used this alignment information to improve locations with errors in the assembly.
> > 
> {: .solution}
> 
{: .question}

> <question-title></question-title>
> 
> How does `Quast` inform on assembly quality?
>
> > <solution-title></solution-title>
> > 
> > `Quast` shows summary information about the assembly contigs. If a reference genome is given, it informs the genome fraction (how much of the reference is covered by the assembly), if any genomic regions appear duplicated, and error information including the rate of mismatches and indels.
> > 
> {: .solution}
> 
{: .question}

> <question-title></question-title>
> 
> How does `Busco` inform on assembly quality?
>
> > <solution-title></solution-title>
> > 
> > `Busco` does not use a reference genome to compare. It attempts to locate key genes which should be present in the assembly, and reports whether it could/could not find those genes. If a key gene is found, it reports whether the gene was fragmented (errors) or complete.
> > 
> {: .solution}
> 
{: .question}



## Purpose-built hybrid assembly tool - Unicycler

In this section, we will use a purpose-built tool called `Unicycler` to perform hybrid assembly.

`Unicycler` uses our Nanopore and Illumina read sets together as input, and returns an assembly. Once we have created the assembly, we will assess its quality using `Quast` and `Busco` and compare with our previous polished assembly. We will also perform BUSCO analysis on the supplied reference genome itself, to record a baseline for our theoretical best `Busco` report.

![workflow diagram for hybrid assembly using unicycler](../../images/denovo_assembly/unicycler_hybrid_assembly.png)

### Hybrid de novo assembly with Unicycler

`Unicycler` performs assembly in the opposite manner to our previous approach. Illumina reads are used to create an assembly graph, then Nanopore reads are used to disentangle problems in the graph. The Nanopore reads serve to bridge Illumina contigs, and to reveal how the contigs are arranged sequentially in the genome.

### Run Unicycler

Find `Unicycler` in the tools panel. It is listed as `Create assemblies with Unicycler`

Run `Unicycler` using the Nanopore and Illumina read sets.

> <hands-on-title>Run Unicycler</hands-on-title>
>
> 1. {% tool [Unicycler](toolshed.g2.bx.psu.edu/repos/iuc/unicycler/unicycler/0.5.1+galaxy0) %}:
>    - *"Paired or Single end data?"*: `Paired `
>    - *"Select first set of reads"*: `illumina_reads_1.fastq`
>    - *"Select second set of reads"*: `illumina_reads_2.fastq`
>    - *"Select long reads. If there are no long reads, leave this empty"*: `nanopore_reads.fastq`
> 
> 2. View Output
>    - `Unicycler` will output three files 
>       - The assembly 
>       - An assembly graph 
>       - SPAdes graphs 
>    - We are interested in the `Final Assembly` output, which is the assembly as a `fasta` file. 
> 
> 3. ***Rename*** the `Final Assembly` output to `Unicycler: Assembly`.
> 
> > <comment-title></comment-title>
> >
> > If `nanopore_reads.fastq` does not appear in the dropdown list, its datatype needs to be changed. 
> >
> > - Click the pencil {% icon galaxy-pencil %} icon next to `nanopore_reads.fastq` in the history panel.
> > - Then select the `Datatypes` tab.
> > - For the parameter `New Type`: `fastqsanger`
> > - Leave all else default and execute the program.
> > 
> {: .comment}
> 
{: .hands_on}


### Comparing Unicycler assembly to Nanopore + Illumina polished assembly

`Busco` and `Quast` can be used again to assess our `Unicycler` assembly. As a purpose-built tool, it generally produces much better assemblies than our sequential approach. This is reflected as (`Quast`) a lower number of contigs, lower mismatches and indels per 100kb, and (`Busco`) greater number of BUSCO genes complete.

> <hands-on-title>Run Quast</hands-on-title>
>
> Run `Quast` as before with the new `Unicycler: Assembly` data
> 
> 1. Select the history item for our initial `Quast` job, then click the rerun {% icon dataset-rerun %} button. This will load the settings used for the previous `Quast` job.
> 
> 2. Change the *"Contigs/scaffolds file"* input to `Unicycler: Assembly`.
> 
> 3. Click the `Run Tool` button to submit the job.
>   
> 4. ***Rename*** the `tabular report` output to `Quast on Unicycler: Assembly`.
> 
> 5. At time of writing, these were the `Quast` results:
> 
> ![output quast report for unicycler assembly](../../images/denovo_assembly/quast_unicycler_assembly.png)
> 
{: .hands_on}

> <hands-on-title>Run Busco</hands-on-title>
>
> Run `Busco` as before with the new `Unicycler: Assembly` data
> 
> 1. Select the history item for our initial `Busco` job, then click the rerun {% icon dataset-rerun %} button. This will load the settings used for the previous `Busco` job.
> 
> 2. Change the *"Sequences to analyse"* input to `Unicycler: Assembly`.
> 
> 3. Click the `Run Tool` button to submit the job.
>
> 4. ***Rename*** the `short summary` output to `Busco on Unicycler: Assembly`.
> 
> 5. At time of writing, these were the `Busco` results:
> 
> ![output busco report for unicycler assembly](../../images/denovo_assembly/busco_unicycler_assembly.png)
> 
{: .hands_on}

We can now create an aggregate summary report from all of our `Quast` and `Busco` outputs using `MultiQC`, which enables us to easily compare the quality of our genome assemblies.

> <hands-on-title>Run MultiQC</hands-on-title>
>
> 1. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.27+galaxy4) %}:
>    - *"1: Results"*:
>      - *"Which tool was used generate logs?"*: `BUSCO`
>      - *"Output of BUSCO"*: `Busco on Reference`, `Busco on Fly: Assembly`, `Busco on Fly: Polished assembly`, `Busco on Unicycler: Assembly`, 
>    - Click `+ Insert Results` to add a second results field
>    - *"2: Results"*:
>      - *"Which tool was used generate logs?"*: `QUAST`
>      - *"Output of Quast"*: `Quast on Fly: Assembly`, `Quast on Fly: Polished assembly`, `Quast on Unicycler: Assembly`
>
> 2. View the `Webpage` output from `MultiQC`:
>    - At time of writing, these were the results.
>      - The number of BUSCOs identified in each assembly by `Busco`:
>        ![buscos identified in each assembly](../../images/denovo_assembly/busco_plot_bacillales_odb10.png)
>      - A table of `Quast` statistics for each assembly:
>        ![table of quast statistics](../../images/denovo_assembly/quast_table_statistics.png)
>      - The number of contigs identified in each assembly by `Quast`:
>        ![number of contigs in each assembly](../../images/denovo_assembly/quast_num_contigs.png) 
> 
{: .hands_on}


> <question-title></question-title>
> 
> How does the `Unicycler` assembly compare with the previous assembly produced using `Flye` + `Pilon`?
> 
> > <solution-title></solution-title>
> > 
> > It seems that the `Unicycler` assembly is much better than the `Flye` + `Pilon` polishing assembly, and produces the same report as the reference genome. Awesome! Looks like this is pretty good.
> > 
> > The `Unicycler` assembly has:
> > 
> > - A very high Genome fraction (coverage of the reference genome)
> > - Very few mismatches and indels per 100 kbp
> > - 15 contigs (compared with the reference genome only having 1 contig)
> > 
> {: .solution}
> 
{: .question}


These results indicate that our `Unicycler` assembly is high quality, and the organism is extremely similar to the organism of the reference genome.

To improve our assembly further to be publishable "complete" quality, we would want to reduce the number of contigs (ideally to a single contig). If the organism has plasmids, we would expect a handfull of contigs, likely below 10.

To do this, the best course of action would be to generate more long-read data.


### Section Questions

> <question-title></question-title>
> 
> Why did we select `Paired` for our Illumina reads in the `Unicycler` tool?
> 
> > <solution-title></solution-title>
> > 
> > Our short read set was `paired-end`. Short read technology can only sequence a few hundred base-pairs in a single read. To provide better structural information, paired-end sequencing was created, where longer fragments (fixed length) are used. A few hundred bp is sequenced at both ends of the fragment, leaving the middle section unsequenced. The reads produced (the mate-pair) from a single fragment are separated by a fixed length, so we know they are nearby in the genome.
> > 
> {: .solution}
> 
{: .question}

> <question-title></question-title>
> 
> Does `Unicycler` begin by using the long or short reads?
> 
> > <solution-title></solution-title>
> > 
> > Unicycler uses short reads first. It creates an assembly graph from short reads, then uses the long reads to provide better structural information of the genome.
> > 
> {: .solution}
> 
{: .question}

> <question-title></question-title>
> 
> How does `Unicycler` use long reads to improve its assembly graph?
> 
> > <solution-title></solution-title>
> > 
> > The assembly graph produced by short reads has tangled regions. When we don't know how sections of the genome are arranged, tangled regions appear in the graph. `Unicycler` uses Nanopore reads which overlap these tangled regions to resolve the proper structure of the genome.
> > 
> {: .solution}
> 
{: .question}


# Conclusion

We have covered two methods for hybrid de novo assembly. The combination of long- and short-read technology is clearly powerful, represented by our ability to create a good assembly with only 25x coverage (100Mb) of Nanopore and 50x coverage (200Mb) of Illumina reads.

To further improve our assembly, extra Nanopore read data may provide the most benefit. At 50x coverage (200Mb), we may achieve a single, or few contig assembly with high per-base accuracy.

In this tutorial, we had a reference genome for a closely related organism, which we could use when assessing the quality of our draft genome assemblies. Without access to a reference genome, `Quast` will produce fewer metrics for assessing the assembled genome quality. In this situation, you can look at the number of contigs in the assembled genome, the minimum number of contigs that produce 50\% and 90\% of the bases in the assembly and the GC\% content. For good quality assemblies, the total and minimum number (50\% and 90\%) of contigs should be small and the GC\% content should be near 50\%.

The development of new purpose-built tools for hybrid de novo assembly like `Unicycler` have improved the quality of assemblies we can produce. These tools are of great importance and while they already produce great results, they will continue to improve over time.

-------------------------------
## Additional reading
Links to additional recommended reading and suggestions for related tutorials.

- Flye: [https://github.com/fenderglass/Flye/blob/flye/docs/USAGE.md#algorithm](https://github.com/fenderglass/Flye/blob/flye/docs/USAGE.md#algorithm)
- Pilon: [https://github.com/broadinstitute/pilon/wiki/Methods-of-Operation](https://github.com/broadinstitute/pilon/wiki/Methods-of-Operation)
- Unicycler: [https://github.com/rrwick/Unicycler](https://github.com/rrwick/Unicycler)
- Quast: [https://academic.oup.com/bioinformatics/article/29/8/1072/228832](https://academic.oup.com/bioinformatics/article/29/8/1072/228832)
- BUSCO analysis: [https://academic.oup.com/bioinformatics/article/31/19/3210/211866](https://academic.oup.com/bioinformatics/article/31/19/3210/211866)

tools_json:
["Busco", "toolshed.g2.bx.psu.edu/repos/iuc/busco/busco/5.8.0+galaxy1", "Flye", "toolshed.g2.bx.psu.edu/repos/bgruening/flye/flye/2.9.6+galaxy0", "Quast", "toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.3.0+galaxy0", "Map with BWA-MEM", "toolshed.g2.bx.psu.edu/repos/devteam/bwa/bwa_mem/0.7.19", "Pilon", "toolshed.g2.bx.psu.edu/repos/iuc/pilon/pilon/1.20.1", "Unicycler", "toolshed.g2.bx.psu.edu/repos/iuc/unicycler/unicycler/0.5.1+galaxy0", "MultiQC", "toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.27+galaxy4"]


