# tutorials

topic: assembly
tutorial: mrsa-nanopore
title: Genome Assembly of MRSA from Oxford Nanopore MinION data (and optionally Illumina data)
difficulty: introductory
time_estimation: 2h
requirements_json: []
tags_json: ["nanopore", "assembly", "amr", "microgalaxy"]
content_hash: 1dbab14f5188d339ab6e979fc19960ed
last_modified: 2026-05-27T09:01:47.817190
zenodo_link: https://zenodo.org/record/10669812
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/mrsa-nanopore/tutorial.html

content:
Sequencing (determining of DNA/RNA nucleotide sequence) is used all over
the world for all kinds of analysis. The product of these sequencers are
reads, which are sequences of detected nucleotides. Depending on the
technique these have specific lengths (30-500bp) or using Oxford
Nanopore Technologies sequencing have much longer variable lengths.

{% snippet faqs/galaxy/sequencing_nanopore.md %}

In this training we will build an assembly of a bacterial genome, from data produced in
"Complete Genome Sequences of Eight Methicillin-Resistant
*Staphylococcus aureus* Strains Isolated from Patients in
Japan" {% cite Hikichi_2019 %}:

> Methicillin-resistant *Staphylococcus aureus* (MRSA) is a major pathogen
> causing nosocomial infections, and the clinical manifestations of MRSA
> range from asymptomatic colonization of the nasal mucosa to soft tissue
> infection to fulminant invasive disease. Here, we report the complete
> genome sequences of eight MRSA strains isolated from patients in Japan.
{: .quote cite="{% cite_url Hikichi_2019 %}"}

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Galaxy and data preparation

Any analysis should get its own Galaxy history. So let's start by creating a new one and get the data into it.

> <hands-on-title>History creation</hands-on-title>
>
> 1. Create a new history for this analysis
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Rename the history
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
{: .hands_on}

Now, we need to import the data: 1 FASTQ file containing the reads from the sequencer.

> <hands-on-title>Data upload</hands-on-title>
> 1. {% tool [Import](upload1) %} the files from [Zenodo]({{ page.zenodo_link }}) or from the shared data library
>
>    ```
>    {{ page.zenodo_link }}/files/DRR187567.fastq.bz2
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
> 2. Rename the dataset to keep only the sequence run ID (`DRR187567`)
>
>    {% snippet faqs/galaxy/datasets_rename.md name="DRR187567" %}
>
> 3. Tag the dataset `#unfiltered`
>
>    {% snippet faqs/galaxy/datasets_add_tag.md %}
>
> 4. **View** {% icon galaxy-eye %} the renamed file
>
{: .hands_on}

The dataset is a FASTQ file.

> <question-title></question-title>
>
> 1. What are the 4 main features of each read in a fastq file.
> 2. What is the name of your first read?
>
> > <solution-title></solution-title>
> > 1. The following:
> >
> >    -   A `@` followed by a name and sometimes information of the read
> >    -   A nucleotide sequence
> >    -   A `+` (optional followed by the name)
> >    -   The quality score per base of nucleotide sequence (Each symbol
> >        represents a quality score, which will be explained later)
> >
> > 2. `DRR187567.1`
> {: .solution}
{: .question}

{% include _includes/cyoa-choices.html option1="Without Illumina MiSeq data" option2="With Illumina MiSeq data" default="Without Illumina MiSeq data" text="Do you have associated Illumina MiSeq data?" disambiguation="miseq"%}

<span class="Without-Illumina-MiSeq-data"></span>

<div class="With-Illumina-MiSeq-data" markdown="1">

> <hands-on-title>Illumina Data upload</hands-on-title>
> 1. {% tool [Import](upload1) %} the files from [Zenodo]({{ page.zenodo_link }}) or from the shared data library
>
>    ```
>    {{ page.zenodo_link }}/files/DRR187559_1.fastqsanger.bz2
>    {{ page.zenodo_link }}/files/DRR187559_2.fastqsanger.bz2
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
> 2. Rename the datasets to remove `.fastqsanger.bz2` and keep only the sequence run ID (`DRR187559_1` and `DRR187559_2`)
>
>    {% snippet faqs/galaxy/datasets_rename.md name="DRR187567" %}
>
> 3. Create a paired collection named `Paired Reads`
>
>    {% snippet faqs/galaxy/collections_build_list_paired.md %}
>
{: .hands_on}

</div>

# Quality Control

During sequencing, errors are introduced, such as incorrect nucleotides being called. These are due to the technical limitations of each sequencing platform. Sequencing errors might bias the analysis and can lead to a misinterpretation of the data. **Sequence quality control is therefore an essential first step in any analysis.**

When assessing the fastq files all bases had their own quality (or Phred score) represented by symbols. You can read more in our dedicated [Quality Control Tutorial]({% link topics/sequence-analysis/tutorials/quality-control/tutorial.md %}).

To assess the quality by hand would be too much work. That's why tools like [NanoPlot](https://github.com/wdecoster/NanoPlot) or
[FastQC](http://www.bioinformatics.babraham.ac.uk/projects/fastqc/) are made, which will generate a summary and plots of the data statistics. NanoPlot is mainly used for long-read data, like ONT and PACBIO and FastQC for any read.

> <hands-on-title>Quality Control</hands-on-title>
>
> 1. {% tool [FastQC](toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.74+galaxy0) %} with the following parameters:
>    - {% icon param-files %} *"Short read data from your current history"*: `DRR187567`
>
> 2. Inspect the webpage output
>
{: .hands_on}

FastQC combines quality statistics from all separate reads and combines them in plots. An important plot is the Per base sequence quality. 

![FastQC plot showing reads that mostly stay in the read](./images/fastqc.png)

<div class="With-Illumina-MiSeq-data" markdown="1">

Here, we are going to trim the Illumina data using **fastp** ({% cite Chen2018 %}):

- Trim the start and end of the reads if those fall below a quality score of 20
- Filter for reads to keep only reads with at least 30 bases: Anything shorter will complicate the assembly

> <hands-on-title>Quality improvement of the Illumina reads</hands-on-title>
>
> 1. {% tool [fastp](toolshed.g2.bx.psu.edu/repos/iuc/fastp/fastp/1.0.1+galaxy3) %} with the following parameters:
>    - *"Single-end or paired reads"*: `Paired Collection`
>        - *"Select paired collection(s)"*: `Paired Reads`
>    - In *"Filter Options"*:
>        - In *"Length filtering Options"*:
>            - *Length required*: `30`
>    - In *"Read Modification Options"*:
>        - In *"Per read cuitting by quality options"*:
>            - *Cut by quality in front (5')*: `Yes`
>            - *Cut by quality in tail (3')*: `Yes`
>            - *Cutting window size*: `4`
>            - *Cutting mean quality*: `20`
>    - In *"Output Options"*:
>        - *"Output JSON report"*: `Yes`
>     
> We next need to extract the **fastp** filtered forward and reverse reads from the paired collection as the next tool we are going to run, **filtlong**, requires the forward and reverse short reads to be provided as separate datasets.
>
> 2. {% tool [Flatten collection](__FLATTEN__) %} with the following parameters:
>    - *"Input Collection"*: **fastp** `Paired-end output`
>      
> 3. **Rename** {% icon galaxy-pencil %} the output to `Flattened fastp output`
>      
> 4. {% tool [Extract dataset](__EXTRACT_DATASET__) %} with the following parameters:
>    - *"Input List"*: `Flattened fastp output`
>    - *"How should a dataset be selected?"*: `Select by element identifier`
>    - *"Element identifier"*: `DRR187559_forward` (filtered forward reads)
> 
> 5. **Rename** {% icon galaxy-pencil %} `DRR187559_forward` to `Trimmed DRR187559_forward`
>   
> 6. {% tool [Extract dataset](__EXTRACT_DATASET__) %} with the following parameters:
>    - *"Input List"*: `Flattened fastp output`
>    - *"How should a dataset be selected?"*: `Select by element identifier`
>    - *"Element identifier"*: `DRR187559_reverse` (filtered forward reads)
>   
> 7. **Rename** {% icon galaxy-pencil %} `DRR187559_reverse` to `Trimmed DRR187559_reverse`
> 
{: .hands_on}

</div>

Depending on the analysis it could be possible that a certain quality or length is needed. The reads can be filtered using the [Filtlong](https://github.com/rrwick/Filtlong) tool. In this training all reads below 1000bp will be filtered.

<div class="With-Illumina-MiSeq-data" markdown="1">

When Illumina reads are available, we can use them **if they are good Illumina reads (high depth and complete coverage)**  as external reference. In this case, Filtlong ignores the Phred quality scores and instead judges read quality using k-mer matches to the reference (a more accurate gauge of quality).

</div>

> <hands-on-title>Filtering</hands-on-title>
> 1. {% tool [filtlong](toolshed.g2.bx.psu.edu/repos/iuc/filtlong/filtlong/0.2.1+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Input FASTQ"*: `DRR187567`
>    - In *"Output thresholds"*:
>        - *"Min. length"*: `1000`
>
>    <div class="With-Illumina-MiSeq-data" markdown="1">
>    - In *"External references"*:
>        - {% icon param-file %} *"Reference Illumina read"*: `Trimmed DRR187559_forward`
>        - {% icon param-file %} *"Reference Illumina read"*: `Trimmed DRR187559_reverse`
>    </div>
>
> 2. Rename the dataset to `DRR187567-filtered`
>
>    {% snippet faqs/galaxy/datasets_rename.md name="DRR187567-filtered" %}
>
{: .hands_on}

The output can be evaluated using **NanoPlot** for plotting long read sequencing data and alignments

> <hands-on-title>Filtering</hands-on-title>
> 1. Convert the datatype of `DRR187567` to uncompress it
>
>    {% snippet faqs/galaxy/datasets_convert_datatype.md conversion="Convert compressed to uncompressed" %}
>
> 2. {% tool [NanoPlot](toolshed.g2.bx.psu.edu/repos/iuc/nanoplot/nanoplot/1.41.0+galaxy0) %} with the following parameters:
>    - *"Select multifile mode"*: `batch`
>        - *"Type of the file(s) to work on"*: `fastq`
>            - {% icon param-files %} *"files"*: **both** `DRR187567 uncompressed` and `DRR187567-filtered`
>    - In *"Options for customizing the plots created"*:
>        - *"Show the N50 mark in the read length histogram."*: `Yes`
>
{: .hands_on}

We ran the **NanoPlot** two times: one for the raw reads (`DRR187567`) and one for the reads after filtering (`DRR187567-filtered`). 

For each run, **NanoPlot** generates 5 outputs:

- 2 plots: 
   - Histogram Read Length
   - Histogram Read Length after log transformation

      ![before filtering, showing a large mostly gaussian curve](./images/nanoplot_before.png "Before filtering")
      
      ![after filtering, same curve as before but everything below 1000bp is gone](./images/nanoplot_after.png "After filtering")


- 2 tabular files with statistics: one general and one after filtering.

   The second one is empty because we did not used **NanoPlot** filtering options.

- A HTML report summarizing above information

   We can compare the two generated reports. Galaxy allows to view several datasets side-by-side using the Window Manager function

   > <hands-on-title>Inspect NanoPlot reports</hands-on-title>
   >
   > 1. Enable Window Manager
   >
   >    {% snippet faqs/galaxy/features_scratchbook.md %}
   >
   > 2. Open both NanoPlot HTML Reports
   > 3. Check the **Summary statistics** section of each to compare the results
   >
   {: .hands_on}

   Summary statistics  | Not Filtered   | Filtered (Filtlong) | Change (%)
   ------------------- | -------------- | ------------------- | ----------
   Number of reads     | 91,288         | 69,906              | -23.4%
   Number of bases     | 621,945,741.0  | 609,657,642.0       | -2.0%
   Median read length  | 3,400.0        | 5,451.0             | 60.3%
   Mean read length    | 6,813.0        | 8,721.1             | 28.0%
   Read length N50     | 14,810.0       | 15,102.0            | 2.0%
   Mean read quality   | 9              | 9                   | 0.0%
   Median read quality | 8.9            | 9.0                 | 1.1%

   > <question-title></question-title>
   >
   > 1. What is the increase of your median read length?
   > 2. What is the decrease in the number of bases?
   > 3. What is coverage?
   > 4. What would be the coverage before and after trimming, based on a genome size of 2.9 Mbp?
   >
   > > <solution-title></solution-title>
   > > 1. 3,400 bp to 5,451 bp, a 60.3% increase
   > > 2. A -2.0% decrease is not a very significant decrease. Our data was quite good to start with and didn't have many short reads which were removed (23.4%)
   > > 3. The coverage is a measure of how many reads 'cover' on average a single base in the genome. If you divide the total reads by the genome size, you will get a number how many times your genomes could theoretically be "covered" by reads.
   > > 4. Before $$ \frac{621,945,741}{2,900,000} = 214.4$$ and after $$ \frac{609,657,642}{2,900,000} = 210.2$$. This is *not* a very big decrease in coverage, so no cause for concern. Generally in sequencing experiments you have some minimum coverage you expect to see based on how much of your sample you sequenced. If it falls below that threshold it might be cause for concern.
   > >
   > {: .solution}
   {: .question}

While there is currently no community consensus over the best trimming or filtering practices with long read data, there are still some steps that can be beneficial to do for the assembly process. {% tool [Porechop](toolshed.g2.bx.psu.edu/repos/iuc/porechop/porechop/0.2.3) %} is a commonly used tool for removing adapter sequences, and we used {% tool [filtlong](toolshed.g2.bx.psu.edu/repos/iuc/filtlong/filtlong/0.2.0) %} for removing shorter reads which might make the assembly process more difficult.

Many people do not do any trimming of their NanoPore data based on the quality as it is expected that the quality is low, and often the focus is on assembling large {SVs} rather than having high quality reads and base-level variation analyses.

# Assembly

When the quality of the reads is determined and the data is filtered (like we
did with filtlong) and/or trimmed (like is more often done with short read data)
an assembly can be made.

There are many tools that create assembly for long-read data, e.g. Canu ({% cite koren2017canu %}), Raven ({% cite vaser2021time %}), Miniasm  ({% cite Li_2016 %}). In this tutorial, we use [**Flye**](https://github.com/fenderglass/Flye) ({% cite lin2016assembly %}). **Flye** is a de
novo assembler for single molecule sequencing reads. It can be used from bacteria to human assemblies. The Flye assembly is based on finding overlapping reads with variable length with high error tolerance. 

{% snippet faqs/galaxy/analysis_results_may_vary.md %}

> <hands-on-title>Assembly using Flye</hands-on-title>
>
> 1. {% tool [Flye assembly](toolshed.g2.bx.psu.edu/repos/bgruening/flye/flye/2.9.1+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Input reads"*: `DRR187567-filtered` (output of **filtlong** {% icon tool %})
>    - *"Mode"*: `Nanopore corrected (--nano-corr)`
>    - *"Reduced contig assembly coverage"*: `Disable reduced coverage for initial disjointing assembly`
>
{: .hands_on}

**Flye** generates 4 outputs:

- A FASTA file called `consensus` with contigs, i.e. the contiguous sequences made by combining separate reads in the assembly, and possibly scaffolds built by **Flye**

   > <question-title></question-title>
   > 
   > How many contigs are there?
   > > <solution-title></solution-title>
   > >
   > > There are 2 sequences in the `consensus` dataset
   > {: .solution}
   {: .question}

- 2 assembly graph files: `assembly graph` and `graphical fragment assembly`

   We can visualize the assembly graph using the `graphical fragment assembly` with **Bandage** ({% cite wick2015bandage %}), a package for exploring assembly graphs through summary reports and visualizations of their contents.

   > <hands-on-title>Assembly inspection</hands-on-title>
   > 1. {% tool [Bandage Image](toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/2022.09+galaxy4) %} with the following parameters:
   >    - {% icon param-file %} *"Graphical Fragment Assembly"*: `graphical fragment assembly` (output of **Flye**)
   >
   {: .hands_on}

   ![Bandage output showing 2 contigs](./images/bandage_before_polishing.jpg)

- A table `assembly info` with assembly information

   It should be something similar but probably sligtly different because Flye can differ a bit per assembly:

   #seq_name | length  | cov. | circ. | repeat | mult. | alt_group | graph_path
   --------- | ------  | ---- | ----- | ------ | ----- | --------- | ----------
   contig_1  | 2927029 | 183  | Y     | N      | 2     | *         | 1
   contig_2  | 60115   | 379  | Y     | Y      | 2     | *         | 2

   > <question-title></question-title>
   > 1. What is the coverage of your longest contig?
   > 2. What is the length of your longest contig?
   > 3. Could this contig potentially be a MRSA genome?
   >
   > > <solution-title></solution-title>
   > > While results may vary due to randomness in the assembly process, in our case we had:
   > > 1. 183
   > > 2. 2.9Mb
   > > 3. 2.9Mb is approximately the size of a MRSA genome. So contig 1 could be the genome. Contig 2 could be one small potential plasmid genome.
   > {: .solution}
   {: .question}

## Assembly Evaluation

To evaluate the assembly, we use also [Quast](http://quast.bioinf.spbau.ru/) ({% cite Gurevich2013 %}) (QUality ASsessment Tool), a tool providing quality metrics for assemblies. This tool can be used to compare multiple assemblies, can take an optional reference file as input to provide complementary metrics, etc

> <hands-on-title>Quality Control of assembly using Quast</hands-on-title>
>
> 1. {% tool [Quast](toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1) %} with the following parameters:
>    - *Assembly mode?*: `Co-assembly`
>       - *"Use customized names for the input files?"*: `No, use dataset names`
>         - {% icon param-file %} *"Contigs/scaffolds file"*: `consensus` output of **Flye**
>
{: .hands_on}

QUAST outputs assembly metrics as an HTML file with metrics and graphs. 

> <question-title></question-title>
>
> 1. How many contigs is there?
> 2. How long is the largest contig?
> 3. What is the total length of all contigs?
> 4. What is you GC content?
> 5. How does it compare to results for KUN1163 in [Table 1](https://journals.asm.org/doi/10.1128/mra.01212-19#tab1) in {% cite Hikichi_2019 %}?
>
> > <solution-title></solution-title>
> >
> > 1. 2 contigs
> > 2. The largest contig is 2,907,099 bp.
> > 3. 2,967,214 (*Total length (>= 0 bp)*). 
> > 4. The GC content for our assembly is 32.91%. 
> > 5. KUN1163 has a genome size of 2,914,567 bp (not far from the length of the largest contig), with a GC content of 32.91% (as in our assembly)
> >
> {: .solution}
{: .question}

<div class="With-Illumina-MiSeq-data" markdown="1">

## Assembly Polishing

We can now polish the assembly using both the short reads and/or long reads. This process aligns the reads to the assembly contigs, and makes corrections to the contigs where warranted. 

Several tools exist for polishing, e.g. Racon ({% cite Vaser_2017 %}). Here we will use **Polypolish** ({% cite Wick_2022 %}), a tool for polishing genome assemblies with short reads.

**Polypolish** needs as input the assembly but also SAM/BAM files where each read has been aligned to *all* possible locations (not just a single best location). Errors in repeats will be covered by short-read alignments, and **Polypolish** can therefore fix those errors. 

To get the SAM/BAM files, we need to map the short reads on the assembly. We will use **BWA-MEM** ({% cite Li_2009 %}, {% cite Li_2010 %}, {% cite Li2013Aligning %}).

We need to set up **BWA-MEM** so it aligns each read to all possible locations, not just the best location. This option does not work with paired-end alignment. We will then need to align forward and reverse read files separately, instead of aligning both read files with a single BWA-MEM run as usually recommended.

> <hands-on-title> Align short-reads on assembly </hands-on-title>
> 1. Change the datatype of both FASTQ outputs of **fastp** to `fastqsanger.gz`
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="fastqsanger.gz" %}
>
> 2. {% tool [BWA-MEM2](toolshed.g2.bx.psu.edu/repos/iuc/bwa_mem2/bwa_mem2/2.2.1+galaxy1) %} with the following parameters:
>    - *"Will you select a reference genome from your history or use a built-in index?"*: `Use a genome from history and build index`
>        - {% icon param-file %} *"Use the following dataset as the reference sequence"*: `consensus` output of **Flye**
>    - *"Single or Paired-end reads"*: `Single`
>        - {% icon param-files %} *"Select fastq dataset"*: `DRR187559_forward`, `DRR187559_reverse`
>    - *"Set read groups information?"*: `Do not set`
>    - *"Select analysis mode"*: `5.Full list of options`
>        - *"Set algorithmic options?"*: `Do not set`
>        - *"Set scoring options?"*: `Do not set`
>        - *"Set input/output options"*: `Set`
>            - *"Output all alignments for single-ends or unpaired paired-ends"*: `Yes`
>    - *"BAM sorting mode"*: `Not sorted (sorted as input)`
>
{: .hands_on}

We can now run **Polypolish**.

> <hands-on-title> Polish assembly </hands-on-title>
>
> 1. {% tool [Polypolish](toolshed.g2.bx.psu.edu/repos/iuc/polypolish/polypolish/0.5.0+galaxy2) %} with the following parameters:
>    - In *"Input sequences"*:
>        - {% icon param-file %} *"Select a draft genome for polishing"*: `consensus` output of **Flye**
>        - *"Select aligned data to polish"*: `Paired SAM/BAM files`
>            - {% icon param-file %} *"Select forward SAM/BAM file"*: output of **BWA-MEM2** on the `DRR187559_forward` output of **fastp**
>            - {% icon param-file %} *"Select reverse SAM/BAM file"*: output of **BWA-MEM2** on the `DRR187559_reverse` output of **fastp**
>
{: .hands_on}

To check the impact of the polishing, let's run **QUAST** on both **Flye** and **Polypolish** outputs.

> <hands-on-title>Quality Control of polished assembly</hands-on-title>
>
> 1. {% tool [Quast](toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1) %} with the following parameters:
>    - *Assembly mode?*: `Co-assembly`
>       - *"Use customized names for the input files?"*: `No, use dataset names`
>         - {% icon param-files %} *"Contigs/scaffolds file"*: `consensus` output of **Flye** and output of **Polypolish**
>
{: .hands_on}

The HTML report generated by **QUAST** gives metrics for both assembly side-by-side

Statistics | **Flye** output | **Polypolish** output
--- | --- | ---
Number of contigs | 2 | 2
Largest contig | 2,907,099 | 2,915,230
Total length (>= 0 bp) | 2,967,214 | 2,975,666
GC (%) | 32.91 | 32.84

> <question-title></question-title>
>
> Is the assembly after polishing better than before given the results for KUN1163 in [Table 1](https://journals.asm.org/doi/10.1128/mra.01212-19#tab1) in {% cite Hikichi_2019 %}?
>
> > <solution-title></solution-title>
> >
> > The largest contig after polishing has a length of 2,915,230 bp, which closr to the expected 2,914,567 bp. But the GC content (32.84% after polishing) is slightly worst given the  expected 32.91% also found in the assembly before polishing.
> >
> {: .solution}
{: .question}

</div>

# Conclusion

In this tutorial, we prepared long reads (using short reads if we had some) assembled them, inspect the produced assembly for its quality, and polished it (if short reads where provided). The assembly, even if uncomplete, is reasonable good to be used in downstream analysis, like [AMR gene detection]({% link topics/genome-annotation/tutorials/amr-gene-detection/tutorial.md %})

tools_json:
["Import", "upload1", "FastQC", "toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.74+galaxy0", "fastp", "toolshed.g2.bx.psu.edu/repos/iuc/fastp/fastp/1.0.1+galaxy3", "Flatten collection", "__FLATTEN__", "Extract dataset", "__EXTRACT_DATASET__", "filtlong", "toolshed.g2.bx.psu.edu/repos/iuc/filtlong/filtlong/0.2.1+galaxy0", "NanoPlot", "toolshed.g2.bx.psu.edu/repos/iuc/nanoplot/nanoplot/1.41.0+galaxy0", "Porechop", "toolshed.g2.bx.psu.edu/repos/iuc/porechop/porechop/0.2.3", "toolshed.g2.bx.psu.edu/repos/iuc/filtlong/filtlong/0.2.0", "Flye assembly", "toolshed.g2.bx.psu.edu/repos/bgruening/flye/flye/2.9.1+galaxy0", "Bandage Image", "toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/2022.09+galaxy4", "Quast", "toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1", "BWA-MEM2", "toolshed.g2.bx.psu.edu/repos/iuc/bwa_mem2/bwa_mem2/2.2.1+galaxy1", "Polypolish", "toolshed.g2.bx.psu.edu/repos/iuc/polypolish/polypolish/0.5.0+galaxy2"]


topic: assembly
tutorial: ERGA-post-assembly-QC
title: ERGA post-assembly QC
difficulty: intermediate
time_estimation: 3H
requirements_json: []
tags_json: ["plants", "animals", "genome", "assembly", "QC", "biodiversity"]
content_hash: 5d09a80c34c1aa429fd4823bc8641491
last_modified: 2026-05-27T09:01:47.799732
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/ERGA-post-assembly-QC/tutorial.html

content:
![ERGA logo, 4 colorful letters on white background, with the European star circle](../../images/post-assembly-QC/ERGA.logo.jpg){: width="20px"}

The European Reference Genome Atlas ([ERGA](https://www.erga-biodiversity.eu/), {% cite Mazzoni2023 %}) is a large-scale network of researchers aiming to generate high-quality reference genomes for all eukaryotic life in Europe and build capacity to allow researchers anywhere to generate reference genomes and use them to answer questions regarding species conservation and biodiversity. ERGA uses state-of-the-art sequencing technologies and advanced bioinformatics tools to produce high-quality genome assemblies.

Reference genomes provide a baseline for understanding genetic diversity within and among populations, and can be used to identify populations at risk of genetic erosion. This information is crucial for developing effective conservation strategies and management plans for threatened and endangered species ({% cite Shafer2015 %}). Additionally, by better understanding the genetic basis of important traits, such as disease resistance and adaptation to changing environments, researchers can develop targeted interventions to mitigate the effects of environmental change and prevent the loss of genetic diversity ({% cite Frankham2011 %}). The ERGA project has the potential to greatly benefit biodiversity conservation efforts and advance our understanding of the genetic basis of biodiversity.

Genome post-assembly quality control is a crucial step for evaluating the accuracy and completeness of newly assembled genomes. This involves assessing the contiguity, completeness, accuracy, and consistency of the genome assembly using various bioinformatic tools and methods ({% cite Koren2017 %}, {% cite Hunt2015 %}, {% cite Mikheenko2015 %}, {% cite Vaser2017 %}, {% cite Zimin2017 %}). The QC process aims to ensure that genomic data is reliable and useful for downstream analyses such as annotation, comparative genomics, and functional studies.

In this tutorial you will learn how to implement the ERGA Genome Assembly QC pipeline, and how to interpretate the potential outcomes.

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Study cases

In this tutorial we will evaluate three genome assemblies, belonging to three different taxonomic groups, in order to illustrate the different scenarios that can be identified. The characteristics of each of them are described briefly below.

<strong>Case 1: <em>Chondrosia reniformis</em></strong>

*Chondrosia reniformis* is a slow-growing marine sponge cosmopolitan species which can be found in the Mediterranean Sea and the eastern Atlantic Ocean in shallow waters; it is considered to playing an important ecological role in the marine ecosystem by filtering large volumes of water and providing habitat for other species ({% cite Voultsiadou2005 %}). The members of this species are gonochoristic and oviparous, whose physiology and behavious seems to highly influence the presence of endosymbiosis heterotrophic bacteria ({% cite sara1997 %}). 

![Figure 1: Chrondosia reniformis](../../images/post-assembly-QC/chondrosia.jpg "Example <em>Chrondosia reniformis</em> sponge.")

The assembly is based on 70x PacBio HiFi and Arima2 Hi-C data generated by the [Aquatic Symbiosis Genomics Project](https://www.aquaticsymbiosisgenomics.org/). The assembly process included the following sequence of steps: initial PacBio assembly generation with Hifiasm, retained haplotig separation with purge_dups, and Hi-C based scaffolding with YaHS. The mitochondrial genome was assembled using MitoHiFi. Finally, the primary assembly was analysed and manually improved using gEVAL.

<strong>Case 2: <em>Erythrolamprus reginae</em></strong>

*Erythrolamprus reginae* is a species of colubrid snake found in South America. This species has been reported to include triploid individuals with parthenogenic reproduction; this type of seems to be associated with higher mutation rates and more tandem duplications in the genome ({% cite Bogart1980 %}). 

![Figure 2: Erythrolamprus reginae](../../images/post-assembly-QC/erytrocampus.jpg "Example <em>Erythrolamprus reginae</em> snake.")

The assembly used in this tutorial correspond to the curated primary assembly generated by the [VGP project](https://vertebrategenomesproject.org/), based on 34x Pacbio HiFi and Arima2 Hi-C data, by using the [VGP assembly pipeline]({% link topics/assembly/tutorials/vgp_genome_assembly/tutorial.md %}).

<strong>Case 3: <em>Eschrichtius robustus</em></strong>

*Eschrichtius robustus*, commonly known as the gray whale, is a species of whale found primarily in the North Pacific Ocean. Adult gray whales can reach lengths of up to 14.9 meters and weights of up to 36,000 kilograms. It is a diploid species, genetically characterized by high homozygosity ({% cite BrnicheOlsen2018 %}).

![Figure 3: Eschrichtius robustus](../../images/post-assembly-QC/eschrichtius.jpg "Example <em>Eschrichtius robustus</em> whale.")

The assembly used in this tutorial correspond to the curated primary assembly generated by the [VGP project](https://vertebrategenomesproject.org/), based on 29x Pacbio HiFi data, by using the [VGP assembly pipeline]({% link topics/assembly/tutorials/vgp_genome_assembly/tutorial.md %}).


# Get data

As a first step we will get the data from Zenodo.

> <hands-on-title>Upload data</hands-on-title>
>
> 1. Create a new history for allocating temporally all the datasets. This history can be named as `Datasets history`
>
>
> 2. Import the Pacbio HiFi files from Zenodo:
>
>    - Open the file {% icon galaxy-upload %} __upload__ menu
>    - Click on **Rule-based** tab
>    - *"Upload type"*: `Collections`
>    - Copy the tabular data, paste it into the textbox and press <kbd>Build</kbd>
>
>       ```
>   dataset_01   https://zenodo.org/record/7786773/files/hifi.fastq.gz   fastqsanger.gz    HiFi    CReniformis_Pacbio
>   dataset_01   https://zenodo.org/record/7784764/files/m64055_210602_211608.hifi_reads.fastq.gz   fastqsanger.gz    HiFi    EReginae_PacBio
>   dataset_02   https://zenodo.org/record/7784764/files/m54306Ue_211020_191957.hifi_reads.fastq.gz   fastqsanger.gz    HiFi    EReginae_PacBio
>   dataset_03   https://zenodo.org/record/7784764/files/m54306Ue_211016_070813.hifi_reads.fastq.gz   fastqsanger.gz    HiFi    EReginae_PacBio
>   dataset_04   https://zenodo.org/record/7786773/files/m54306U_210529_225553.hifi_reads.fastq.gz   fastqsanger.gz    HiFi    EReginae_PacBio
>   dataset_01   https://zenodo.org/record/7781236/files/m54306Ue_220411_220734.demultiplex.bc1010--bc1010.hifi_reads.fastq.gz   fastqsanger.gz    HiFi    ERobustus_PacBio
>   dataset_02   https://zenodo.org/record/7781236/files/m54306Ue_220520_051743.demultiplex.bc1010--bc1010.hifi_reads.fastq.gz   fastqsanger.gz    HiFi    ERobustus_PacBio
>   dataset_03   https://zenodo.org/record/7781236/files/m64055e_220615_033108.demultiplex.bc1010--bc1010.hifi_reads.fastq.gz   fastqsanger.gz    HiFi    ERobustus_PacBio
>   dataset_04   https://zenodo.org/record/7786773/files/m64055e_220603_182128.demultiplex.bc1010--bc1010.hifi_reads.fastq.gz   fastqsanger.gz    HiFi    ERobustus_PacBio
>       ```
>    - Click on `Build`
>    - From **Rules** menu select `Add / Modify Column Definitions`
>       - Click `Add Definition` button and select `List Identifier(s)`: column `A`
>       - Click `Add Definition` button and select `URL`: column `B`
>       - Click `Add Definition` button and select `Type`: column `C`
>       - Click `Add Definition` button and select `Group Tag`: column `D`
>       - Click `Add Definition` button and select `Collection Name`: column `E`
>    - Click `Apply` and press <kbd>Upload</kbd>
>
> 3. Import the remainig datasets from Zenodo:
>
>    - Open the file {% icon galaxy-upload %} __upload__ menu
>    - Click on **Rule-based** tab
>    - *"Upload type"*: `Datasets`
>    - Copy the tabular data, paste it into the textbox and press <kbd>Build</kbd>
>
>       ```
>   CReformitis_assembly    https://zenodo.org/record/7831298/files/assembly_sponge.fasta.gz    fasta.gz    assembly
>   CReformitis_metadata    https://zenodo.org/record/7781236/files/metadata_chon.yaml  yaml    metadata
>   CReniformis_Hi-C_F   https://zenodo.org/record/7786773/files/hiC_1.fastq.gz   fastqsanger.gz    Hi-C
>   CReniformis_Hi-C_R   https://zenodo.org/record/7786773/files/hiC_2.fastq.gz   fastqsanger.gz    Hi-C
>   EReginae_Hi-C_F_01   https://zenodo.org/record/7831762/files/rEryReg1_Royal_Ground_Snake_R1.fastq.gz.gz.001   fastqsanger.gz    Hi-C
>   EReginae_Hi-C_F_02   https://zenodo.org/record/7831762/files/rEryReg1_Royal_Ground_Snake_R1.fastq.gz.gz.002   fastqsanger.gz    Hi-C
>   EReginae_Hi-C_F_03   https://zenodo.org/record/7831763/files/rEryReg1_Royal_Ground_Snake_R1.fastq.gz.gz.003   fastqsanger.gz    Hi-C
>   EReginae_Hi-C_R_01   https://zenodo.org/record/7833514/files/rEryReg1_Royal_Ground_Snake_R2.fastq.gz.gz.001   fastqsanger.gz    Hi-C
>   EReginae_Hi-C_R_02   https://zenodo.org/record/7833514/files/rEryReg1_Royal_Ground_Snake_R2.fastq.gz.gz.002   fastqsanger.gz    Hi-C
>   EReginae_Hi-C_R_03   https://zenodo.org/record/7831763/files/rEryReg1_Royal_Ground_Snake_R2.fastq.gz.gz.003   fastqsanger.gz    Hi-C
>   EReginae_assembly    https://zenodo.org/record/7788734/files/rEryReg1.pri.cur.20230105.fasta.gz    fasta.gz    assembly
>   EReginae_metadata    https://zenodo.org/record/7781236/files/metadata_eryth.yaml  yaml    metadata
>   ERobustus_assembly  https://zenodo.org/record/7781236/files/mEscRob2.pri.cur.20221201.fasta.gz  fasta.gz    assembly
>   ERobustus_metadata  https://zenodo.org/record/7781236/files/metadata_esch.yaml  yaml    metadata
>   Taxonomy_data   https://zenodo.org/record/7781236/files/new_taxdump.tar.gz  gz  taxonomy
>   Diamond_db   https://zenodo.org/record/7788734/files/Diamond_makedb.dmnd  dmnd  database
>       ```
>
>    - From **Rules** menu select `Add / Modify Column Definitions`
>       - Click `Add Definition` button and select `Name`: column `A`
>       - Click `Add Definition` button and select `URL`: column `B`
>       - Click `Add Definition` button and select `Type`: column `C`
>       - Click `Add Definition` button and select `Name Tag`: column `D`
>    - Click `Apply` and press <kbd>Upload</kbd>
>
{: .hands_on}

# What data have we imported?

For each species, we have uploaded the fasta file corresponding to the assembly, the raw sequencing data used to generate each assembly - in these cases PacBio HiFi longreads and Illumina Hi-C reads - and databases necessary to run the QC tools included in this tutorial. We will use the taxonomy database `new_taxdump.tar.gz` and the diamond database of protein sequences from the ncbi nt database to identify any sequence from contaminants or symbiont which are in our assembly using `BlobToolKit`.

Once we have imported all the datasets, we will move each one to its correspondent history.

> <hands-on-title>Upload data</hands-on-title>
>
> 1. Create three new empty histories, one for each specie.
> 2. Rename the histories as  `Case 1: *Chondrosia reniformis*`, `Case 2: *Erythrolamprus reginae*` and  `Case 3: *Eschrichtius robustus*`.
> 3. Click in **History options** and select `Show Histories Side-by-Side`
> 4. Click in `Select histories`, and include the histories corresponding to the three species.
> 5. Drag and drop to move the datasets to their correspondent histories.
>
>    > <comment-title>Non-unique datasets</comment-title>
>    > Both the **Taxonomic_data** and the **Diamond_database** should be included in all of them.
>    {: .comment}
>
> {% snippet faqs/galaxy/histories_rename.md %}
>
{: .hands_on}

Once all the datasets have been copied to their correspondent history, we should obtain something similar to this:

![Figure 4: Distribution of data](../../images/post-assembly-QC/histories_side_by_side.png "Histories corresponding to the three cases of study.")

# Genome assembly overview with BlobToolKit

**BlobToolKit** is a tool designed to assist researchers in analyzing and visualizing genome assembly data. The tool uses information from multiple data sources such as read coverage, sequence length, and taxonomic annotations to generate a comprehensive overview of genome assembly data ({% cite Challis2020 %}). One of the key characteristics of BlobToolKit is its ability to provide with a user-friendly interactive interface for analyzing complex genome assembly data.

In this tutorial, we will use BlobToolKit in order to integrate the following data:

- **Read coverage data**: BlobToolKit can use read coverage information to identify potential errors or gaps in the genome assembly. By comparing the depth of coverage across the genome, it can highlight regions that may be overrepresented or underrepresented, which can indicate potential issues with the assembly.
- **Taxonomic annotations**: BlobToolKit can use taxonomic information to identify potential contaminants or foreign DNA in the genome assembly. It does this by comparing the taxonomic profile of the genome assembly to a reference database of known organisms.
- **Sequence similarity data**: Sequence similarity data can be used to identify potential misassemblies or contaminants in the genome assembly. BlobToolKit can use BLAST/DIAMOND searches to compare the genome assembly to reference databases and identify regions that may be problematic.
- **BUSCO reports**: BlobToolKit can use BUSCO data to provide additional information about the quality of a genome assembly. It can generate plots of the number of complete and partial BUSCO genes in the genome assembly, as well as the number of missing and fragmented genes.

> <comment-title>Why should we evaluate contaminants?</comment-title>
>
> A significant proportion of the genome sequences in both GenBank and RefSeq (0.54% and 0.34% of entries, respectively) include sequences from contaminants; the contamination primarily exists in the form of short contigs, flanking regions on longer contigs, or areas of larger scaffolds that are flanked by Ns, although a few longer sequences with contamination were also detected ({% cite Steinegger2020 %}).
>
{: .comment}

In the next steps, we will generate the data required for generating the visualization plots with BlobToolKit.

## Generate read coverage data with **Minimap2**

Read coverage is an essential metric for evaluating the quality of genome assemblies, and it provides valuable information for identifying regions of high and low quality, detecting misassemblies, and identifying potential contaminants. Thus, for example, unexpected regions of low coverage suggests potential errors, such as misassemblies, gaps, or low complexity regions ({% cite Koren2017 %}).

In this tutorial we will use **Minimap2** for generation the coverage data. Minimap2 is a multi-purpose aligner which is particularly efficient at aligning long-reads produced by sequencing machines from PacBio or Oxford Nanopore ({% cite Li2018 %}).

> <comment-title>How is coverage information encoded in the BAM file?</comment-title>
>
> Coverage information is encoded in the BAM file format through the number of reads that align to each position in the reference genome. Specifically, the BAM file stores each read's start and end position in the genome, as well as the length and orientation of the read, among other information. To calculate coverage, the number of reads that overlap each position in the reference genome is counted.
>
{: .comment}

> <hands-on-title>Generate BAM file with Minimap2</hands-on-title>
>
> 1. {% tool [Collapse Collection](toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/5.1.0) %} with the following parameters:
>    - {% icon param-collection %} *"Collection of files to collapse into single dataset"*: `CReniformis_Pacbio`
>
>
> 2. {% tool [Map with minimap2](toolshed.g2.bx.psu.edu/repos/iuc/minimap2/minimap2/2.28+galaxy0) %} with the following parameters:
>    - *"Will you select a reference genome from your history or use a built-in index?"*: `Use a genome from history and build index`
>        - {% icon param-file %} *"Select the reference genome"*: `CReformitis_assembly`
>    - *"Single or Paired-end reads"*: `Single`
>        - {% icon param-file %} *"FASTA/Q file"*: output of **Collapse Collection** {% icon tool %}
>    - *"Select a profile of preset options"*: `PacBio HiFi reads vs reference mapping (-k19 -w19 -U50,500 -g10k -A1 -B4 -O6,26 -E2,1 -s200 ) (map-hifi)`
>
> 3. Repeat the prevous steps with the datasets from the two remanining species.
>
{: .hands_on}


## Generate sequence similarity data with **DIAMOND**

**DIAMOND** is a sequence alignment tool that utilizes a more efficient algorithm compared to BLAST, allowing for much faster searches of large sequence databases. Specifically, DIAMOND uses a sensitive seed-extension approach that compares a set of small segments (seeds) from the query sequence to a database, and then extends the alignments based on the highest-scoring hits. This approach allows DIAMOND to perform up to 20,000 times faster than BLAST, with comparable or improved sensitivity and accuracy ({% cite Buchfink2014 %}).

> <hands-on-title> Task description </hands-on-title>
>
> 1. {% tool [Diamond](toolshed.g2.bx.psu.edu/repos/bgruening/diamond/bg_diamond/2.0.15+galaxy0) %} with the following parameters:
>    - *"Alignment mode"*: `DNA query sequences (blastx)`
>        - *"Allow for frameshifts?"*: `yes`
>            - *"restrict hit culling to overlapping query ranges"*: `Yes`
>            - *"frame shift penalty"*: `15`
>    - {% icon param-file %} *"Input query file in FASTA or FASTQ format"*: `CReformitis_assembly`
>    - *"Will you select a reference database from your history or use a built-in index?"*: `Use one from the history`
>        - {% icon param-file %} *"Select the reference database"*: `Diamond_db`
>    - *"Restrict search taxonomically?"*: `No`
>    - *"Sensitivity Mode"*: `Fast (--fast)`
>        - *"Block size in billions of sequence letters to be processed at a time"*: `10.0`
>    - *"Method to filter?"*: `Maximum e-value to report alignments`
>    - *"Method to restrict the number of hits?"*: `Maximum number of target sequences`
>    - In *"Output options"*:
>        - *"Format of output file"*: `BLAST tabular`
>            - *"Tabular fields"*: `Query Seq-id`,`Subject Seq -id`, `Start of alignment in query`, `End of alignment in query`, `Expected value`, `Bit score` and `Unique Subject Taxonomy ID(s), separated by a ',' (in numerical order)`
>
> 2. Repeat the prevous steps with the datasets from the two remanining species.
>
{: .hands_on}


## Generate **BUSCO** report

**BUSCO** (Benchmarking Universal Single-Copy Orthologs) is a tool used for quantitative assessment of genome assembly, gene set, and transcriptome completeness. It is based on evolutionarily informed expectations of gene content derived from near-universal single-copy orthologs ({% cite Simo2015 %})

> <comment-title> Orthologs </comment-title>
>
> Orthologs are genes in different species which have usually the same function and have evolved from a common ancestral gene. They are important for new genome assemblies in order to predict gene functions and help with gene annotation ({% cite Gennarelli2010 %}).
>
{: .comment}

> <hands-on-title> Estimate single copy gene representation completeness </hands-on-title>
>
> 1. {% tool [Busco](toolshed.g2.bx.psu.edu/repos/iuc/busco/busco/5.7.1+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Sequences to analyse"*: `CReformitis_assembly`
>    - *"Lineage data source"*: `Use cached lineage Data`
>    - *"Cached database with lineage"*: `Busco v5 Lineage Datasets`
>    - *"Mode"*: `Genome assemblies (DNA)`
>        - *"Select a gene predictor"*: `Metaeuk`
>    - *"Auto-detect or select lineage?"*: `Auto-detect` (For *Erythrolamprus reginae*, select `Select lineage` then `Vertebrata` in *"Lineage"*)
>    - *"auto-lineage group"*: `All taxonomic groups (--auto-lineage)`
>    - *"Which outputs should be generated"*: `Short summary text`
>
>
>    > <details-title> Additional information </details-title>
>    >
>    > BUSCO sets represent 3,023 genes for vertebrates, 2,675 for arthropods, 843 for metazoans, 1,438 for fungi and 429 for eukaryotes. An intuitive metric is provided in BUSCO notation - C:complete[S:single, D:duplicated], F:fragmented, M:missing, n:number of genes used.
>    {: .details}
>
> 2. Repeat the prevous steps with the datasets from the two remanining species.
>
{: .hands_on}

> <question-title></question-title>
>
> What are the values of complete BUSCO genes for each species?
>
> > <solution-title></solution-title>
> >
> > 1. Eschrichtius robustus: 96.0%
> > 2. Erythrolamprus reginae: 91.3%
> > 3. Chondrosia reniformis: 82.8%
> >
> {: .solution}
>
{: .question}


## Generate interactive plots with **BlobToolKit**

**BlobToolKit** is a tool designed to assist researchers in analyzing and visualizing genome assembly data. The tool uses information from multiple data sources such as read coverage, gene expression, and taxonomic annotations to generate a comprehensive overview of genome assembly data ({% cite Challis2020 %}). One of the key characteristics of BlobToolKit is its ability to provide with a user-friendly interactive interface for analyzing complex genome assembly data.

To work with BlobToolKit we need to create a new dataset structure called **BlobDir**. Therefore the minimum requirement is a fasta file which contains the sequence of our assembly. A list of sequence identifiers and some statistics like length, GC proportion and undefined bases will then be generated.

To get a more meaningful analysis and therefore more useful information about our assembly, it is better to provide as much data as we can get. In our case we will also provide a metadata file if possible, NCBI taxonomy ID and the NCBI taxdump directory ({% cite Challis2020 %}).

> <hands-on-title> Creating the BlobDir dataset </hands-on-title>
>
> 1. {% tool [BlobToolKit](toolshed.g2.bx.psu.edu/repos/bgruening/blobtoolkit/blobtoolkit/3.4.0+galaxy0) %} with the following parameters:
>    - *"Select mode"*: `Create a BlobToolKit dataset`
>        - {% icon param-file %} *"Genome assembly file"*: `CReformitis_assembly`
>        - {% icon param-file %} *"Metadata file"*: `CReformitis_metadata`
>        - *"NCBI taxonomy ID"*: `121349`
>        - {% icon param-file %} *"NCBI taxdump directory"*: `Taxonomy_Data`
>
>
> 2. {% tool [BlobToolKit](toolshed.g2.bx.psu.edu/repos/bgruening/blobtoolkit/blobtoolkit/3.4.0+galaxy0) %} with the following parameters:
>    - *"Select mode"*: `Add data to a BlobToolKit dataset`
>        - {% icon param-file %} *"Blobdir.tgz file"*: output of **BlobToolKit** {% icon tool %}
>        - {% icon param-file %} *"BUSCO full table file"*: `Full table` (output of **Busco** {% icon tool %})
>        - *"BLAST/Diamond hits"*: `Enabled`
>           - *"BLAST/Diamond hits dataset"*: output of **Diamond** {% icon tool %}
>           - *"BLAST/Diamond file column order"*: `1=qseqid,7=staxids,6=bitscore,2=sseqid,3=qstart,4=qend,5=evalue` 
>        - {% icon param-file %} *"BAM/SAM/CRAM read alignment file"*: output of **Minimap2** {% icon tool %}
>
> 3. {% tool [Interactive BlobToolKit](interactive_tool_blobtoolkit) %} with the following parameters:
>    - {% icon param-file %} *"Blobdir file"*: output of **BlobToolKit** {% icon tool %}
>
> 4. Click on the {% icon galaxy-eye %} (eye) icon and inspect the output of **Interactive BlobToolKit**.
>
> 5. Repeat the prevous steps with the datasets from the two remanining species.
>
{: .hands_on}

Now, we will evaluate three of plots that we can find in the Blobtoolkit interactve interface. First, we will start with the **snailplot**, which provides a holistic view of the assembly.

![Figure 5: BlobToolKit snail plot](../../images/post-assembly-QC/Blobdir.snail.choRen.png  "Chondrosia reniformis snail plot summary. The grey line represents the assembly scaffolds, where the distance to the center of the circle indicates the length of the scaffolds. A scale line in the center of the circle helps to gauge the length. Longest contig: The red line in the plot indicates the longest contig in the assembly. N50 and N90: The dark orange and light orange lines represent the scaffolds contained in the N50 and N90 metrics, respectively. GC/AT content: The outer light and dark blue lines represent the GC and AT content, respectively. In an ideal assembly, the line between the two colors should be consistent, without much fluctuation. Gaps: Gaps in the assembly, if present, are denoted by the percent N in the bottom right of the image.")

The main plot is divided into 1,000 size-ordered bins around the circumference with each bin representing 0.1% of the 117,390,217 bp assembly. The **distribution of sequence lengths is shown in dark grey** with the plot radius scaled to the longest sequence present in the assembly (10,413,042 bp, shown in red).

**Orange and pale-orange arcs show the N50 and N90 sequence lengths** (8,459,200 and 6,903,244 bp), respectively. The **pale grey spiral shows the cumulative sequence count on a log scale** with white scale lines showing successive orders of magnitude. The **blue and pale-blue area around the outside of the plot shows the distribution of GC, AT and N percentages** in the same bins as the inner plot. A summary of complete, fragmented, duplicated and missing BUSCO genes in the eukaryota_odb10 set is shown in the top right.

> <question-title>Snailplot question</question-title>
>
> Which differences do you appreciate between the snailplots corresponding to *Erythrolamprus reginae* and *Eschrichtius robustus*?
>
> > <solution-title></solution-title>
> >
> > ![Figure 6: BlobToolKit snailplots comparison](../../images/post-assembly-QC/Blobdir.snail.png "Comparison between snailplots from E. reginae (A) and E. robustus (B).")
> >
> > - (A) The snake (*Erythrolamprus reginae*) does have a relatively large proportion of the genome assembled into some few and long scaffolds (reflected in the high "longest scaffold" value and the high "Log10 scaffold count" value).
> > - (B) The whale (*Eschrichtius robustus*) does have a relatively large proportion of the genome assembled into fragmented scaffolds (reflected in the low N90 value compared to the N50 value and the high "Log10 scaffold count" value)
> >
> {: .solution}
>
{: .question}

Now, we are going to analyze the blob plot corresponding to *Erythrolamprus reginae*, which is especially useful for gaining insights into the composition of your genomic data and identify potential contaminants or endosymbionts. 

![Figure 7: BlobToolKit circle plot](../../images/post-assembly-QC/Blobdir.blob.circle.EryReg.png "Blob plot of base coverage in input against GC proportion for sequences in *E. reginate assembly*. Sequences are coloured by phylum. Circles are sized in proportion to sequence length . Histograms show the distribution of sequence length sum along each axis")

The blob plot is a two-dimensional scatter plot that helps in visualizing and analyzing genomic data for quality control, contaminant detection, and filtering. In the circle plot, **each sequence is represented by a circle**, with its diameter proportional to the sequence length. Circles are colored based on their taxonomic affiliation, and their positions on the X and Y axes are determined by their GC content and coverage, respectively. GC content is the proportion of G and C bases in the sequence, which can differ substantially between genomes. Coverage, on the other hand, is a measure of the number of times a particular sequence has been read during the sequencing process. The plot also includes coverage and GC histograms for each taxonomic group, weighted by the total span (cumulative length) of sequences occupying each bin.

> <comment-title>Pre-curated assembly evaluation</comment-title>
>
> Usually, the evaluation of the pre-curated assembly yield quite different results when compared with the post-curation evaluation. A frequent difference is the abudance of contigs that belong to different organisms, as a result of contamination in the reads. It can be clearly appreciated in the next figure, which correspond to the *E. reginae* pre-curated assembly. A striking element is the presence of 38 contigs corresponding to the phylum Artverviricota (dark green), a group of viruses that encode a reverse transcriptase.
>
> ![Figure 8: BlobToolKit circle plot](../../images/post-assembly-QC/Blobdir.blob.circle_precuration.png "Blob plot of base coverage in input against GC proportion for sequences in pre-curated E. reginate assembly. Sequences are coloured by phylum. Circles are sized in proportion to sequence length . Histograms show the distribution of sequence length sum along each axis")
>
> In addition, as we can appreciate, the samples included contamination from fungi, arthropoda and bacteria.
>
{: .comment}

> <question-title>Blob plot question</question-title>
>
> Is there significant contamination in the *Eschrichtius robustus* and *Chondrosia reniformis* assemblies?
>
> > <solution-title></solution-title>
> >
> > ![Figure 9: BlobToolKit snailplots comparison](../../images/post-assembly-QC/Blobdir.blob.circle.EscRob_ChoRen.png "Comparison between blob plots from E. robustus (A) and C. reniformis (B).")
> >
> > - (A) The whale (*Eschrichtius robustus*) does have significant contamination. Approximately half of the assembled sequences of the whale does contain contaminants or cosymbionts (total count: 703, Chordata count: 316), however as the foreign sequences are smaller, they make up a small percentage of the total assembly.
> > - (B) The sponge (*Condrosia reniformis*) doesn't contain any foreign sequences or contaminatio (total count: 15, Chordata count: 15).
> >
> {: .solution}
>
{: .question}

Finally, let's have a look at the cumulative sequence length plot, which shows the curves for subsets of scaffolds assigned to each phylum relative to the overall assembly. It is useful for evaluating the **contribution of the contigs from contaminated reads to the final assembly**. In that case, first we will evaluate all sequences together, and then we will remove those correspoding to chordata or not classified (not-hit) in order to be able to evaluate in detail each of the contaminants. So, let's start with the accumulative plot corresponding to all sequences (fig. 10).

In this kind of plot, **the x-axis represent the number of contigs** (sorted by species and length), and **the y-axis correspond to the cumulative length in nucleotides**. The gray line shows the cumulative length of all sequences. As we can appreciate, most sequences corespond to the taxa chordata (1.9Gb, distributed along 151 contigs). In addition, we can see that the final assembly includes 174 contigs corresponding to different taxa, or not classified at all. In order to be able to analyze the contribution of those sequences, we will filter the contigs corresponding to the chordata.

![Figure 10: BlobToolKit cumulative plot](../../images/post-assembly-QC/Blobdir.cumulative.png "Cumulative sequence length for assembly Blobdir. The grey line shows cumulative length for all sequences. Coloured lines show cumulative lengths of sequences assigned to each phylum using the bestsum taxrule. Plot axes are scaled to the filtered assembly.")

> <comment-title> How do I apply filters to a dataset? </comment-title>
>
> Datasets can be filtered based on any category or variable using the "Filters" menu. To filter based on a category, click the tab above any bar of the preview histogram to hide/show records assigned to this category. To filter variables, use the sliders at either end of the preview histogram to change the maximum and minimum values or enter the numbers directly into the text boxes above the histogram.
>
{: .comment}

By hidding the contings corresponding to chordata, we can have a detailed view of the contributions of each contaminant (fig. 11).

![Figure 11: BlobToolKit cumulative plot](../../images/post-assembly-QC/Blobdir.cumulative_filtered.png "The grey line shows cumulative length for all sequences. Coloured lines show cumulative lengths of sequences assigned to each phylum using the bestsum taxrule and are stacked by cumulative value on the x-axis to show the proportion of each phylum in the overall assembly. The assembly has been filtered to exclude sequences with phylum matches Chordata. Plot axes are scaled to the filtered assembly.")

In figure 11 we can appreciate clearly the relative contribution of each contaminant with respect to the total contamination. 

# K-mer based genome profiling and evaluation

k-mers can be utilized to infer genome characteristics such as length and heterozygosity by analyzing their frequency distribution in raw sequencing reads ({% cite Manekar2018 %}). In this tutorial we will make use of three different tools in order to perform the k-mer based genome profiling and assembly evaluation.

> <comment-title> Genome profiling details </comment-title>
>
> You can find more details about genome profiling in the [VGP training]({% link topics/assembly/tutorials/vgp_genome_assembly/tutorial.md %}#genome-profile-analysis).
>
{: .comment}


## Generating k-mer profile with **Meryl**

DNA is double stranded and normally only one strand is sequenced. For our assembly we want to consider the other strand as well. Therefore canonical k-mers are used in most counting tools, exactly like in **Meryl**. A full k-mer pair is a sequence and the reverse complement of the sequence (e.g. ATG/CAT). The canonical sequence of a k-mer pair is the lexicographically smaller of the two reverse complementary sequences. So if CAT appears it will be counted as ATG. ({% cite Clavijo %})

Meryl is a powerful tool for counting k-mers in large-scale genomic datasets. It uses a sorting-based approach that sorts the k-mers in lexicographical order.

> <hands-on-title> Generating k-mer profile </hands-on-title>
>
> 1. {% tool [Meryl](toolshed.g2.bx.psu.edu/repos/iuc/meryl/meryl/1.3+galaxy6) %} with the following parameters:
>    - *"Operation type selector"*: `Count operations`
>        - {% icon param-collection %} *"Input sequences"*: `CReniformis_Pacbio` 
>        - *"K-mer size selector"*: `Set a k-mer size`
>            - *"K-mer size"*: `21`
>
>    > <comment-title> compute k </comment-title>
>    >
>    > In general k can be computed as k=log4 (G(1-p)/p), with G as genome size and p as tolerable collision rate.
>    > In our case we set the k-mer size to 21.
>    {: .comment}
>
>
> 2. Run {% tool [Meryl](toolshed.g2.bx.psu.edu/repos/iuc/meryl/meryl/1.3+galaxy6) %} again with the following parameters:
>    - *"Operation type selector"*: `Operations on sets of k-mers`
>        - *"Operations on sets of k-mers"*: `Union-sum: return k-mers that occur in any input, set the count to the sum of the counts`
>        - {% icon param-file %} *"Input meryldb"*: previous output of **Meryl** {% icon tool %}
>
> 3. Run {% tool [Meryl](toolshed.g2.bx.psu.edu/repos/iuc/meryl/meryl/1.3+galaxy6) %} a third time with the following parameters:
>    - *"Operation type selector"*: `Generate histogram dataset`
>        - {% icon param-file %} *"Input meryldb"*: previous output of **Meryl** {% icon tool %}
>
> 4. Rename the output as `Histogram dataset`
>
{: .hands_on}


## K-mer profile analysis with **GenomeScope**

**Genomescope** is used for analysing genomes with the help of k-mer profile analysis. It estimates the overall genome characteristics and the overall read characteristics. The tool will use a given k-mer profile which is calculated only from raw reads sequencing data. It then generates a plot with the calculated data giving us information about the completeness and quality of the to be assembled data. ({% cite Vurture2017 %})

> <hands-on-title> Generate plots for analysis </hands-on-title>
>
> 1. {% tool [GenomeScope](toolshed.g2.bx.psu.edu/repos/iuc/genomescope/genomescope/2.0+galaxy2) %} with the following parameters:
>    - {% icon param-file %} *"Input histogram file"*: `Histogram dataset`
>    - *"Ploidy for model to use"*: `2` for *C. reniformis* and *E. robustus*, `3` for *E. reginae*.
>
>
>    > <comment-title> Plots </comment-title>
>    >
>    > The generated plots will have the coverage on the x-axis and the frequency on the y-axis. It shows the fitted model in the observed k-mer data as well as the k-mer peaks and other information like some estimated parameters of the assembly.
>    {: .comment}
>
{: .hands_on}

![Figure 12: Genomescope plot](../../images/post-assembly-QC/Chondrosia-reniformis-Linear_plot.png "Genomescope 21-mer profile (k:21) of Chondrosia reniformis, diploid (p:2). The plot includes estimations about the genome length (len:124,167,659bp), unique sequences (uniq:61.5%), homozygous portion (aa: 98.7%), heterozygous portions (ab:1.32%), mean k-mer coverage for heterozygous bases (kcov:35.4), read error rate (err:0.752%) and average rate of read duplications (dup:0.378).")

Figure 12 corresponds to the k-mer profile of the sponge (*Chondrosia reniformis*). Presumably, the large gene number in the sponge genome is due to regional gene duplication; so far evidence for a transposition in sponges has been presented. Data indicate that only 0.25 % of the total sponge genome comprises CA/TG microsatellites, and until now also no SINEs/transposable elements have been identified. The estimated genome size around 124,Mbp is relatively close to *Chondrosia reniformis* genome size (117,39Mbp).

> <question-title>Genome profile question</question-title>
>
> Which differences can you appreciate between the genome profile of *Erythrolamprus reginae* and *Eschrichtius robustus*?
>
> > <solution-title></solution-title>
> >
> > ![Figure 13: Genomescope plot ](../../images/post-assembly-QC/genome_profiling_comparison.png "Genomescope genome profile. Erythrolamprus reginae (A). Eschrichtius robustus (B). ")
> >
> > - (A) The genome profile of the snake (*Erythrolamprus reginae*) suggests a high level of heterozygosity (reflected by the first peak at 11.1x coverage). 51.6% unique sequences have been estimated. In this case the low value is attributed to the parthenogenesis and the polyploidy of this snake which causes tandem duplications and higher DNA replication as a result of the triploid-diploid cycles.
> > - (B) The genome profile of the whale (*Eschrichtius robustus*) suggests a high level of homozygosity (reflected by the second peak at 29x coverage). The homozygous k-mer distribution can be expected from the high grade of inbreeding.
> >
> {: .solution}
>
{: .question}



## K-mer based assembly evaluation with **Merqury**

Merqury is designed for evaluating the completeness and accuracy of long-read genome assemblies using short-read sequencing data. Thus the quality of assemblies which are generated by using third-generation sequencing technologies can be reviewed and assessed by the tool.
Merqury works by comparing k-mers of an assembly to those from unassembled high-accuracy reads of the raw sequencing data. K-mer-based methods are also used to identify errors and missing sequences. ({% cite Rhie2020 %})

> <hands-on-title> Generating stats and plots </hands-on-title>
>
> 1. {% tool [Merqury](toolshed.g2.bx.psu.edu/repos/iuc/merqury/merqury/1.3+galaxy2) %} with the following parameters:
>    - *"Evaluation mode"*: `Default mode`
>        - {% icon param-file %} *"K-mer counts database"*: `read_db` (output of **Meryl** {% icon tool %})
>        - *"Number of assemblies"*: `One assembly (pseudo-haplotype or mixed-haplotype)`
>            - {% icon param-file %} *"Genome assembly"*: `CReformitis_assembly` 
>
>    > <comment-title> Output </comment-title>
>    >
>    > Merqury will now generate following outputs:
>    >
>    >  1. stats with completeness statistics
>    >  2. QV stats with quality value statistics
>    >  3. plots
>    >
>    {: .comment}
>
{: .hands_on}

Let's have a closer look at the copy number plots for each of the three species. Merqury will generate k-mers from the raw sequencing data (in the following called the 'read set') and will compare them to the assembly, in contrast to genomescope. Despite being diploid or triploid the species only got plotted with the primary assembly. This is why only one-copy k-mers get evaluated.

![Figure 14: Merqury plot](../../images/post-assembly-QC/Eschrichtius-robustus.spectra-cn.fl.png "Merqury copy numbers (CN) plot of the whale (*Eschrichtius robustus*). The red area displays the k-mers of the assembly. The black area displays the k-mers only found in the read set. The black area can be indicative for sequencing error in the read set or missing sequences in the assembly.")

The small black area indicates that most of the k-mers found in the read set are also found in the assembly (but not all since only the primary assembly got plotted). It reflects a highly homozygous assembly. This plot indicates a sequencing coverage at ~30x as seen before with genomescope.

> <question-title>Genome profile question</question-title>
>
> Which differences can you appreciate between the genome profile of *Chondrosia reniformis* and *Erythrolamprus reginae*?
>
> > <solution-title></solution-title>
> >
> > ![Figure 15: Merqury plot](../../images/post-assembly-QC/merqury_comparison_plot.png "Merqury copy numbers (CN) plot of Chondrosia reniformis (A) and Erythrolamprus reginae (B). The red area displays the k-mers of the assembly. The black area displays the k-mers only found in the read set. The black area can be indicative for sequencing error in the read set or missing sequences in the assembly.")
> >
> > - (A) The sponge (*Chondrosia reniformis*) does have two peaks as seen before with genomescope which is typical for a diploid species. The large black area indicates that there is a high amount of k-mers in the read set which is not used in the assembly since only the primary assembly got plotted. The secondary assembly is missing.
> > - (B) The snake (*Erythrolamprus reginae*) does have three peaks as seen before with genomescope which is typical for a triploid species. The black area is even larger compared to the sponge since only the primary assembly got plotted. The secondary and third assembly are missing.
> >
> {: .solution}
>
{: .question}


## Providing assembly statistics with **gfastats**

gfastats is a tool for providing summary statistics and genome file manipulation. In our case it will generate genome assembly statistics in a tabular-format output. Metrics like N50/L50, GC-content and lengths of contigs, scaffolds and gaps as well as other statistical information are provided for assessing the contiguity of the assembly.

> <hands-on-title> Generate summary statistics </hands-on-title>
>
> 1. {% tool [gfastats](toolshed.g2.bx.psu.edu/repos/bgruening/gfastats/gfastats/1.2.0+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `CReformitis_assembly` 
>    - *"Specify target sequences"*: `Disabled`
>    - *"Tool mode"*: `Summary statistics generation`
>        - *"Report mode"*: `Genome assembly statistics (--nstar-report)`
>
>
>    > <comment-title> N50/NG50 </comment-title>
>    >
>    > Consider taking all contigs and sorting them by size. Starting with the largest and ending with the smallest. Now add up the length of each contig beginning with the largest, then the second largest and so on. When reaching 50% of the total length of all contigs it's done. The length of the contig you stopped is the N50 value. ({% cite Videvall201703 %})
>    >
>    > NG50 or more general NG(X) is based on the same idea as N50. The difference is that in this case the whole genome size or estimated genome size is taken into account. Through this comparisons can be made over different assemblies and genome sizes.
>    >
>    {: .comment}
>
>    > <comment-title> L50 </comment-title>
>    >
>    > Remember adding up the length of each contig until reaching the 50%. The L50 value is the number of the contig you have stopped.
>    >
>    > Example: The sum of all contigs together is 2000 kbp. The contig at 50% has length 300 kbp and is the third one and thus the third largest.
>    > Then N50 = 300 kbp and L50 = 3.
>    >
>    {: .comment}
>
> In the best case a high quality assembly should consist of just a few and large contigs to represent the genome as a whole. Therefore a good assembly should lead to a high N50 value and in contrast a low quality assembly with tiny, fragmented contigs would lead to a low N50 value. ({% cite Videvall201704 %})
>
> The metric doesn't only rely on measuring the 50% mark. The general case is N(X) where X ranges from 0 - 100 mostly in ten steps. However the NX metric is not suitable for comparing different species with different genome lengths.
>
>    > <details-title> GC-content </details-title>
>    >
>    > The GC-content or guanine-cytosine ratio tells one about the occurrence of guanine and cytosine in a genome. It is stated in percent. The two nucleobases are held together by three hydrogen bonds. A high GC-content makes DNA more stable than a low GC-content. Because the ratio of most species and organisms has been found out by now, it is also a good metric to gauge completeness.({% cite Wikipedia %})
>    {: .details}
>
>    > <details-title> CC(contig/chromosome) ratio </details-title>
>    >
>    > Another good metric for gauging contiguity is the CC ratio. The value is calculated by dividing contig counts by the chromosome pair number. A perfect score would be 1. Therefore the lower the value the better the contiguity of the assembly.
>    {: .details}
{: .hands_on}


# Hi-C scaffolding

To understand how chromosomes are arranged in their three-dimensional structure in the nucleus, high resolution and high throughput imaging techniques have been developed. Hi-C is a high throughput method to measure pairwise contacts between pairs of genomic loci. 
It is based on cross-linking the DNA in the nucleus and its histones (proteins around which the DNA is wrapped). However the DNA gets cut and marked and afterwards the cut parts get ligated together. Then the DNA gets fragmented and sequenced forwards and backwards generating paired reads. The resulting information can be used to assemble the reads to the corresponding chromosome.
({% cite Pal2018 %})

## Pre-processing Hi-C data

The following analysis use Hi-C reads in single datasets: one for the forward reads and one for the reverse reads. If you have more than one fastq datasets, as we do for *Erythrolamprus reginae* you will need to merge them into single datasets. 
If you have only one pair of datasets, skip this step. 

> <hands-on-title> Merge  <em>Erythrolamprus reginae</em> Hi-C data into single datasets </hands-on-title>
> 1. Create a collection with the Hi-C forward reads name `EReginae Hi-C forward reads`.
>
> 2. Do the same with the reverse reads (R) and name it `EReginae Hi-C reverse reads`.
>
> 3. For species with more than one set of Hi-C reads, verify that the datasets are ordered the same way in both collections. If not, sort both collections with the tool  {% tool Sort collection %} 
>
> 4. {% tool [Collapse Collection](toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/5.1.0) %} with the following parameters:
>    - {% icon param-collection %} *"Collection of files to collapse into single dataset"*: `EReginae Hi-C forward Reads`
>
> 5. Rename the dataset `EReginae Hi-C forward Reads`
> 
> 6. {% tool [Collapse Collection](toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/5.1.0) %} with the following parameters:
>    - {% icon param-collection %} *"Collection of files to collapse into single dataset"*: `EReginae Hi-C reverse reads`
>
> 7. Rename the dataset `EReginae Hi-C reverse Reads`
>
> {% snippet faqs/galaxy/collections_build_list.md name="EReginae Hi-C forward Reads" datasets_description="the fastq.gz containing the Hi-C forward reads (F)" n="1" %}
>
> {% snippet faqs/galaxy/datasets_rename.md name="Hi-C forward Reads" %}
> 
{: .hands_on}

> <hands-on-title> Mapping Hi-C reads against a reference genome </hands-on-title>
> 
> 1. {% tool [BWA-MEM2](toolshed.g2.bx.psu.edu/repos/iuc/bwa_mem2/bwa_mem2/2.2.1+galaxy0) %} with the following parameters:
>    - *"Will you select a reference genome from your history or use a built-in index?"*: `Use a genome from history and build index`
>        - {% icon param-file %} *"Use the following dataset as the reference sequence"*: `CReformitis_assembly` 
> 
>    - *"Single or Paired-end reads"*: `Single`
>        - {% icon param-file %} *"Select fastq dataset"*: `Hi-C forward Reads` (output of **Collapse Collection** {% icon tool %})
> 
>    - *"Set read groups information?"*: `Do not set`
>    - *"Select analysis mode"*: `1.Simple Illumina mode`
>    - *"BAM sorting mode"*: `Sort by read names  (i.e., the QNAME field) `
>
> 2. Rename the bam dataset `Mapped Hi-C forward Reads`
>
>
> 3. {% tool [BWA-MEM2](toolshed.g2.bx.psu.edu/repos/iuc/bwa_mem2/bwa_mem2/2.2.1+galaxy0) %} with the following parameters:
>    - *"Will you select a reference genome from your history or use a built-in index?"*: `Use a genome from history and build index`
>        - {% icon param-file %} *"Use the following dataset as the reference sequence"*: `CReformitis_assembly`
>    - *"Single or Paired-end reads"*: `Single`
>        - {% icon param-file %} *"Select fastq dataset"*: `Hi-C reverse Reads` (output of **Collapse Collection** {% icon tool %})
>    - *"Set read groups information?"*: `Do not set`
>    - *"Select analysis mode"*: `1.Simple Illumina mode`
>    - *"BAM sorting mode"*: `Sort by read names  (i.e., the QNAME field) `
>
> 4. Rename the bam dataset `Mapp

tools_json:
["Collapse Collection", "toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/5.1.0", "Map with minimap2", "toolshed.g2.bx.psu.edu/repos/iuc/minimap2/minimap2/2.28+galaxy0", "Diamond", "toolshed.g2.bx.psu.edu/repos/bgruening/diamond/bg_diamond/2.0.15+galaxy0", "Busco", "toolshed.g2.bx.psu.edu/repos/iuc/busco/busco/5.7.1+galaxy0", "BlobToolKit", "toolshed.g2.bx.psu.edu/repos/bgruening/blobtoolkit/blobtoolkit/3.4.0+galaxy0", "Interactive BlobToolKit", "interactive_tool_blobtoolkit", "Meryl", "toolshed.g2.bx.psu.edu/repos/iuc/meryl/meryl/1.3+galaxy6", "GenomeScope", "toolshed.g2.bx.psu.edu/repos/iuc/genomescope/genomescope/2.0+galaxy2", "Merqury", "toolshed.g2.bx.psu.edu/repos/iuc/merqury/merqury/1.3+galaxy2", "gfastats", "toolshed.g2.bx.psu.edu/repos/bgruening/gfastats/gfastats/1.2.0+galaxy0", "BWA-MEM2", "toolshed.g2.bx.psu.edu/repos/iuc/bwa_mem2/bwa_mem2/2.2.1+galaxy0", "Filter and merge", "toolshed.g2.bx.psu.edu/repos/iuc/bellerophon/bellerophon/1.0+galaxy0", "PretextMap", "toolshed.g2.bx.psu.edu/repos/iuc/pretext_map/pretext_map/0.1.9+galaxy1", "Pretext Snapshot", "toolshed.g2.bx.psu.edu/repos/iuc/pretext_snapshot/pretext_snapshot/0.0.3+galaxy1", "toolshed.g2.bx.psu.edu/repos/bgruening/gfastats/gfastats/1.3.6+galaxy0", "Bandage Image", "toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/2022.09+galaxy4"]


topic: assembly
tutorial: hifi-assembly
title: Genome assembly using PacBio data
difficulty: intermediate
time_estimation: 3h
requirements_json: []
tags_json: ["assembly", "pacbio", "biodiversity"]
content_hash: 0734d0671af244911fc6a855bd14c2db
last_modified: 2026-05-27T09:01:47.803648
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/hifi-assembly/tutorial.html

content:
In this tutorial, we will assemble a genome of a species of fungi in the family Aspergillus, *Aspergillus niger*, from PacBio sequencing data. These data will be downloaded from ENA. The quality of the assembly obtained will be analyzed, in particular by comparing it to a reference assembly, available on ENA.


> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Get data

We will use long reads sequencing data: HiFi (High Fidelity long reads) from PacBio sequencing of *Aspergillus niger* genome. This data is available on ENA. We will also use later a reference genome assembly downloaded from ENA.

## Get data from ENA

> <hands-on-title>Data upload from ENA</hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [ENA](https://www.ebi.ac.uk/ena/browser/)
>
>    ```
>    ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR317/012/SRR31719412/SRR31719412_subreads.fastq.gz
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>
> 3. Rename the datasets
> 4. Check that the datatype is `fastq.gz`
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
>
>
{: .hands_on}

## Get reference genome from ENA

> <hands-on-title>Data upload from ENA</hands-on-title>
>
> 1. Reference genome is available here: [ASM4765177v1 assembly for Aspergillus niger](https://www.ebi.ac.uk/ena/browser/view/GCA_047651775.1)
> 2. Download the `WGS Set FASTA (JBKZXA01.fasta.gz)` on your computer
> Make sure you download `WGS Set FASTA (JBKZXA01.fasta.gz)` and *NOT*  `ALL Set FASTA`
> 4. Upload this file on Galaxy (Upload --> Choose local file --> Select the file --> Start --> Close)
> 5. Check that the datatype is `fasta.gz`
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
>
{: .hands_on}

# Genome Assembly

{% include _includes/cyoa-choices.html option1="Hifiasm" option2="Flye" default="Hifiasm"
       text="[Hifiasm](https://github.com/chhylp123/hifiasm) and [Flye](https://github.com/mikolmogorov/Flye) are two well known assembler." %}
<div class="Hifiasm" markdown="1">

Hifiasm is a fast haplotype-resolved de novo assembler initially designed for PacBio HiFi reads. In general, hifiasm generates the assembly graphs in the GFA format, so a step of conversion to fasta is necessary. The GFA 1.0 format is a tab-delimited text format for describing a set of sequences and their overlap.
<br>
Hifiasm produces arguably the best single-sample telomere-to-telomere assemblies combing HiFi, ultralong and Hi-C reads, and it is one of the best haplotype-resolved assemblers for the trio-binning assembly given parental short reads. For a human genome, hifiasm can produce the telomere-to-telomere assembly in one day.

> <hands-on-title>Assembly with Hifiasm</hands-on-title>
>
> 1. {% tool [Hifiasm](toolshed.g2.bx.psu.edu/repos/bgruening/hifiasm/hifiasm/0.25.0+galaxy0) %} with the following parameters:
>    - *"Mode"*: `Standard`
>    - {% icon param-file %} *"Input reads"*: the raw data (fastq.gz)
>    - *"Output log file"*: Set to yes
>
>     The tool produces five datasets: Haplotype-resolved raw unitig graph, Haplotype-resolved processed unitig graph without small bubbles, Primary assembly contig graph, Alternate assembly contig graph, [hap1]/[hap2] contig graph.
>
> 2. {% tool [GFA to FASTA](toolshed.g2.bx.psu.edu/repos/iuc/gfa_to_fa/gfa_to_fa/0.1.2) %} with the following parameters:
>    - {% icon param-file %} *"Input GFA file"*: primary assembly contig graph
{: .hands_on}

> <question-title></question-title>
>
> What are the different output datasets from Hifiasm?
>
> > <solution-title></solution-title>
> >
> > - Haplotype-resolved raw unitig graph: This graph keeps all haplotype information, including somatic mutations and recurrent sequencing errors.
> > - Haplotype-resolved processed unitig graph without small bubbles: This graph 'pops' small bubbles in the raw unitig graph; small bubbles might be caused by somatic mutations or noise in data, which are not the real haplotype information.
> > - Primary assembly contig graph: This graph includes a complete assembly with long stretches of phased blocks, though there may be some haplotype collapse.
> > - Alternate assembly contig graph: This graph consists of all contigs that are discarded from the primary contig graph.
> > - [hap1]/[hap2] contig graph: Each graph consists of phased contigs (output only with Hi-C phasing enabled).
> >
> {: .solution}
>
{: .question}

</div>

<div class="Flye" markdown="1">

We will use *Flye*, a de novo assembler for single molecule sequencing reads, such as those produced by PacBio and Oxford Nanopore Technologies. It is designed for a wide range of datasets, from small bacterial projects to large mammalian-scale assemblies. The package represents a complete pipeline: it takes raw PacBio / ONT reads as input and outputs polished contigs. Flye also has a special mode for metagenome assembly. All informations about Flye assembler are here: [Flye](https://github.com/fenderglass/Flye/).

> <hands-on-title>Assembly with Flye</hands-on-title>
>
> 1. {% tool [Flye](toolshed.g2.bx.psu.edu/repos/bgruening/flye/flye/2.9.5+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"Input reads"*: the raw data (fastq.gz)
>    - *"Mode"*: `PacBio HiFi`
>    - *"Number of polishing iterations"*: `1`
>    - *"Reduced contig assembly coverage"*: `Disable reduced coverage for initial disjointing assembly`
>    - *"Generate a log file"*: Set to yes
>
>     The tool produces four datasets: consensus, assembly graph, graphical fragment assembly and assembly info
{: .hands_on}

> <question-title></question-title>
>
> What are the different output datasets?
>
> > <solution-title></solution-title>
> >
> > - The first dataset (consensus) is a fasta file containing the final assembly (1461 contigs). You may notice that the result (contigs number) you obtained is sligthy different from the one presented here. This is due to the Flye assembly algorithm which doesn't always give the eact same results.
> > - The second and third dataset are assembly graph files. These graphs are used to represent the final assembly of a genome, they are based on reads and their overlap information. Some tools such as [Bandage](http://rrwick.github.io/Bandage/) allow to visualize the assembly graph.
> > - The fourth dataset is a tabular file (assembly_info) containing extra information about contigs/scaffolds.
> >
> {: .solution}
>
{: .question}
  
</div>

# Quality assessment

## Genome assemblies comparison with **Quast**

A way to calculate metrics assembly is to use ***QUAST = QUality ASsessment Tool***. Quast is a tool to evaluate genome assemblies by computing various metrics. The manual of Quast is here: [Quast](http://quast.sourceforge.net/docs/manual.html#sec3)
Quast allows to compare the assembly carried out with the reference genome and produces statistics such as the genome fraction.
However, when comparing to a reference genome, Quast results do not display the assembly N50, therefore, in this tutorial, you will generate two quast reports : One specifying a reference genome and one not specifying a reference genome.

> <hands-on-title>Quast</hands-on-title>
>
> 1. {% tool [Quast](toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.0.2+galaxy3) %} without specifying a reference genome with the following parameters:
>    - *"Assembly mode?"*: `Individual assembly (1 contig file per sample)`
>    - *"Use customized names for the input files?"*: `No, use dataset names`
>        - {% icon param-collection %} *"Contigs/scaffolds file"*: `JBKZXA01.fasta.gz` (reference assembly), `fasta file` (output of **GFA to FASTA** {% icon tool %}) and/or `consensus` (output of **Flye** {% icon tool %})
>    - *"Type of assembly"*: `Genome`
>        - *"Use a reference genome?"*: `No`
>        - *"Type of organism"*: `Fungus: use of GeneMark-ES for gene finding, ...`
>
> 2. {% tool [Quast](toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.0.2+galaxy3) %} specifying a reference genome with the following parameters:
>    - *"Assembly mode?"*: `Individual assembly (1 contig file per sample)`
>    - *"Use customized names for the input files?"*: `No, use dataset names`
>        - {% icon param-collection %} *"Contigs/scaffolds file"*: `fasta file` (output of **GFA to FASTA** {% icon tool %}) and/or `consensus` (output of **Flye** {% icon tool %})
>    - *"Type of assembly"*: `Genome`
>        - *"Use a reference genome?"*: `Yes`
>          - *"Reference genome"*: `JBKZXA01.fasta.gz` (reference assembly)
>        - *"Type of organism"*: `Fungus: use of GeneMark-ES for gene finding, ...`
>
> 
{: .hands_on}

> <question-title></question-title>
>
> Compare the different metrics obtained for the assembly you generated and the reference genome.
>
> > <solution-title></solution-title>
> >
> > We compare the metrics of the three assemblies:
> > - The reference genome (JBKZXA01.fasta.gz): 8 contigs, Contig N50 = 3.5Mb, Contig length max = 6.2 Mb, size = 35.4Mb, 49.51% GC
> > - The Hifiasm assembly: 105 contigs, N50 = 4.9 Mb, length max = 7.1Mb, assembly size = 42.1 Mb, 47.91% GC
> > - The Flye assembly: 13 contigs, N50 = 4.9Mb, length max = 6.8Mb, size = 38.7 Mb, 49.31% GC
> >
> {: .solution}
>
{: .question}

## Genome assembly assessment with **BUSCO**

***BUSCO (Benchmarking Universal Single-Copy Orthologs)*** allows a measure for quantitative assessment of genome assembly based on evolutionarily informed expectations of gene content. Details for this tool are here: [Busco website](https://busco.ezlab.org/)

> <hands-on-title>BUSCO on assembly</hands-on-title>
>
> 1. {% tool [Busco](toolshed.g2.bx.psu.edu/repos/iuc/busco/busco/5.8.0+galaxy0) %} with the following parameters:
>    - *"Tool version"*: `Galaxy Version 5.8.0+galaxy0`
>    - {% icon param-file %} *"Sequences to analyse"*: Multiple datasets
>    - {% icon param-collection %} *"Sequences to analyse"*: `JBKZXA01.fasta.gz` (reference assembly), `fasta file` (output of **GFA to FASTA** {% icon tool %}) and/or `consensus` (output of **Flye** {% icon tool %})
>    - *"Lineage data source"*: `Use Cached Lineage Data`
>    - *"Auto-detect or select lineage"*: `Select lineage`
>        - *"Lineage"*: `Ascomycota`
>        - *"Which outputs should be generated"*: `short summary text; summary image`
{: .hands_on}

> <question-title></question-title>
>
> Compare the number of BUSCO genes identified in the generated assembly and the reference genome. What do you observe ?
>
> > <solution-title></solution-title>
> >
> > Short summary generated by BUSCO indicates that reference genome (JBKZXA01.fasta.gz) contains:
> > 1. 1678 Complete BUSCOs (of which 1675 are single-copy and 3 are duplicated),
> > 2. 7 fragmented BUSCOs,
> > 3. 21 missing BUSCOs.
> >
> > Short summary generated by BUSCO indicates that Hifiasm assembly contains:
> > 1. 1685 complete BUSCOs (1679 single-copy and 6 duplicated),
> > 2. 6 fragmented BUSCOs
> > 3. 15 missing BUSCOs.
> >
> > Short summary generated by BUSCO indicates that Flye assembly contains:
> > 1. 1685 complete BUSCOs (1679 single-copy and 6 duplicated),
> > 2. 6 fragmented BUSCOs
> > 3. 15 missing BUSCOs.
> >
> > BUSCO analysis confirms that these two assemblies are of similar quality, with similar number of complete, fragmented and missing BUSCOs genes.
> >
> {: .solution}
>
{: .question}



# Conclusion


This pipeline shows how to generate and evaluate a genome assembly from long reads PacBio data. Once you are satisfied with your genome sequence, you might want to annotate it: have a look at the [RepeatMasker]({% link topics/genome-annotation/tutorials/repeatmasker/tutorial.md %}) and [Funannoate]({% link topics/genome-annotation/tutorials/funannotate/tutorial.md %}) tutorials to learn how to do it!

tools_json:
["Hifiasm", "toolshed.g2.bx.psu.edu/repos/bgruening/hifiasm/hifiasm/0.25.0+galaxy0", "GFA to FASTA", "toolshed.g2.bx.psu.edu/repos/iuc/gfa_to_fa/gfa_to_fa/0.1.2", "Flye", "toolshed.g2.bx.psu.edu/repos/bgruening/flye/flye/2.9.5+galaxy1", "Quast", "toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.0.2+galaxy3", "Busco", "toolshed.g2.bx.psu.edu/repos/iuc/busco/busco/5.8.0+galaxy0"]


topic: assembly
tutorial: mrsa-illumina
title: Genome Assembly of a bacterial genome (MRSA) sequenced using Illumina MiSeq
difficulty: intermediate
time_estimation: 2h
requirements_json: []
tags_json: ["illumina", "assembly", "microgalaxy"]
content_hash: 820f016a959bb0d8966f353d15f95dad
last_modified: 2026-05-27T09:01:47.816122
zenodo_link: https://zenodo.org/record/10669812
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/mrsa-illumina/tutorial.html

content:
Sequencing (determining of DNA/RNA nucleotide sequence) is used all over
the world for all kinds of analysis. The product of these sequencers are
reads, which are sequences of detected nucleotides. Depending on the
technique these have specific lengths (30-500bp) or using Oxford
Nanopore Technologies sequencing have much longer variable lengths.

{% snippet faqs/galaxy/sequencing_illumina_miseq.md %}

In this training we will build an assembly of a bacterial genome, from data produced in
"Complete Genome Sequences of Eight Methicillin-Resistant
*Staphylococcus aureus* Strains Isolated from Patients in
Japan" {% cite Hikichi_2019 %}:

> Methicillin-resistant *Staphylococcus aureus* (MRSA) is a major pathogen
> causing nosocomial infections, and the clinical manifestations of MRSA
> range from asymptomatic colonization of the nasal mucosa to soft tissue
> infection to fulminant invasive disease. Here, we report the complete
> genome sequences of eight MRSA strains isolated from patients in Japan.
{: .quote cite="{% cite_url Hikichi_2019 %}"}

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Galaxy and data preparation

Any analysis should get its own Galaxy history. So let's start by creating a new one and get the data into it.

> <hands-on-title>History creation</hands-on-title>
>
> 1. Create a new history for this analysis
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Rename the history
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
{: .hands_on}

Now, we need to import the data: 2 FASTQ files containing the reads from the sequencer.

> <hands-on-title>Import datasets</hands-on-title>
> 1. {% tool [Import](upload1) %} the files from [Zenodo]({{ page.zenodo_link }}) or from Galaxy shared data libraries:
>
>    ```
>    {{ page.zenodo_link }}/files/DRR187559_1.fastqsanger.bz2
>    {{ page.zenodo_link }}/files/DRR187559_2.fastqsanger.bz2
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
>
> 2. Rename the datasets to remove `.fastqsanger.bz2` and keep only the sequence run ID (`DRR187559_1` and `DRR187559_2`)
>
>    {% snippet faqs/galaxy/datasets_rename.md name="DRR187559_1" %}
>
> 3. Create a paired collection named `Paired Reads`
>
>    {% snippet faqs/galaxy/collections_build_list_paired.md %}
>
> 4. Tag the collection `#unfiltered`
>
>    {% snippet faqs/galaxy/collections_add_tag.md %}
>
> 5. **View** {% icon galaxy-eye %} the renamed files in the collection
>
{: .hands_on}

The datasets are both FASTQ files.

> <question-title></question-title>
>
> 1. What are the 4 main features of each read in a FASTQ file.
> 2. What does the `_1` and `_2` mean in your filenames?
>
> > <solution-title></solution-title>
> > 1. The following:
> >
> >    -   A `@` followed by a name and sometimes information of the read
> >    -   A nucleotide sequence
> >    -   A `+` (optional followed by the name)
> >    -   The quality score per base of nucleotide sequence (Each symbol
> >        represents a quality score, which will be explained later)
> >
> > 2. Forward and reverse reads, by convention, are labelled `_1` and `_2`, but they might also be `_f`/`_r` or `_r1`/`_r2`.
> {: .solution}
{: .question}

# Quality Control

During sequencing, errors are introduced, such as incorrect nucleotides being called. These are due to the technical limitations of each sequencing platform. Sequencing errors might bias the analysis and can lead to a misinterpretation of the data. Adapters may also be present if the reads are longer than the fragments sequenced and trimming these may improve the number of reads mapped. **Sequence quality control is therefore an essential first step in any analysis.**

Assessing the quality by hand would be too much work. That's why tools like
[NanoPlot](https://github.com/wdecoster/NanoPlot) or
[Falco](https://falco.readthedocs.io/en/latest/) are made, as they  generate a summary and plots of the data statistics. NanoPlot is
mainly used for long-read data, like ONT and PACBIO and Falco for short read,
like Illumina and Sanger. Falco is an efficiency-optimized rewrite of [FastQC](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/). You can read more in our dedicated [Quality Control
Tutorial]({% link topics/sequence-analysis/tutorials/quality-control/tutorial.md %}).

Before doing any assembly, the first questions we should ask about the input
reads include:

- What is the coverage of my genome?
- How good are my reads?
- Do I need to ask/perform for a new sequencing run?
- Is it suitable for the analysis I need to do?

> <hands-on-title>Quality Control</hands-on-title>
>
> 1. {% tool [Falco](toolshed.g2.bx.psu.edu/repos/iuc/falco/falco/1.2.4+galaxy0) %} with the following parameters:
>    - {% icon param-collection %} *"Raw read data from your current history"*: `Paired Reads`
>
> 2. Inspect the webpage outputs
>
{: .hands_on}

Falco combines quality statistics from all separate reads and combines them in plots. An important plot is the Per base sequence quality. 

DRR187559_1 | DRR187559_2
----------- | -----------
![Falco plot showing reads that are green](../../images/mrsa/falco-1.png) | ![Same as previous plot, but the beginning of the reads are slightly better quality](../../images/mrsa/falco-2.png)

Here you have the reads sequence length on the x-axes against the quality score (Phred-score) on the y-axis. The colour of the beams indicates three quality levels: 
- Green = good quality, 
- Yellow = mediocre quality, and 
- Red = bad quality.

For each position, a boxplot is drawn with:

- the median value, represented by the central intense coloured line
- the inter-quartile range (25-75%), represented by the geen, yellow or red box
- the 10% and 90% values in the upper and lower whiskers

For Illumina data it is normal that the first few bases are of some lower quality and how longer the reads get the worse the quality becomes. This is often due to signal decay or phasing during the sequencing run.


Before doing any assembly, the first questions we should ask about the input
reads include:

- What is the coverage of my genome?
- How good are my reads?
- Do I need to ask/perform for a new sequencing run?
- Is it suitable for the analysis I need to do?

Depending on the analysis it could be possible that a certain quality or length
is needed. In this case we are going to trim the data using **fastp** ({% cite Chen2018 %}):

- Trim the start and end of the reads if those fall below a quality score of 20

  Different trimming tools have different algorithms for deciding when to cut but trimmomatic will cut based on the quality score of one base alone. Trimmomatic starts from each end, and as long as the base is below 20, it will be cut until it reaches one greater than 20. A sliding window trimming will be performed where if the average quality of 4 bases drops below 20, the read will be truncated there. 
  
- Filter for reads to keep only reads with at least 30 bases: Anything shorter will complicate the assembly

> <hands-on-title>Quality improvement</hands-on-title>
>
> 1. {% tool [fastp](toolshed.g2.bx.psu.edu/repos/iuc/fastp/fastp/1.0.1+galaxy3) %} with the following parameters:
>    - *"Single-end or paired reads"*: `Paired Collection`
>        - *"Select paired collection(s)"*: `Paired Reads`
>    - In *"Filter Options"*:
>        - In *"Length filtering Options"*:
>            - *Length required*: `30`
>    - In *"Read Modification Options"*:
>        - In *"Per read cuitting by quality options"*:
>            - *Cut by quality in front (5')*: `Yes`
>            - *Cut by quality in tail (3')*: `Yes`
>            - *Cutting window size*: `4`
>            - *Cutting mean quality*: `20`
>    - In *"Output Options"*:
>        - *"Output JSON report"*: `Yes`
>
> 2. Edit the tags of the **fastp** FASTQ outputs to 
>    1. Remove the `#unfiltered` tag
>    2. Add a new tag `#filtered`
{: .hands_on}

**fastp** generates also a report, similar to Falco, useful to compare the impact of the trimming and filtering.

> <question-title></question-title>
>
> Looking at **fastp** HTML report
>
> 1. How did the average read length change before and after filtering?
> 3. Did trimming improve the mean quality scores?
> 4. Did trimming affect the GC content?
> 5. Is this data ok to assemble? Do we need to re-sequence it?
>
> > <solution-title></solution-title>
> >
> > 1. Read lengths went down more significantly:
> >    - Before filtering: 190bp, 221bp
> >    - After filtering: 189bp, 219bp
> > 3. It increased the percentage of Q20 and Q30 bases (bases with quality score above 20 and 30 respectively)
> > 4. No, it did not. If it had, that would be unexpected.
> > 5. This data looks OK. The number of short reads in R1 is not optimal but assembly should partially work but not the entire, closed genome.
> >
> {: .solution}
{: .question}


# Assembly

Now that the quality of the reads is determined and the data is filtered and/or trimmed, we can try to assemble the reads together to build longer sequences.

There are many tools that create assembly for short-read data, e.g. SPAdes ({% cite prjibelski2020using %}), Abyss ({% cite simpson2009abyss %}). In this tutorial, we use **Shovill**. Shovill is a SPAdes-based genome assembler, improved to work faster and only for smaller (bacterial) genomes.

## Assembly

{% snippet faqs/galaxy/analysis_results_may_vary.md %}

> <hands-on-title>Assembly using Shovill</hands-on-title>
>
> 1. {% tool [Shovill](toolshed.g2.bx.psu.edu/repos/iuc/shovill/shovill/1.1.0+galaxy1) %} with the following parameters:
>    - *"Input reads type, collection or single library"*: `Paired Collection`
>        - *"Paired collection"*: **fastp** `Paired-end output`
>    - In *"Advanced options"*:
>        - *"Estimated genome size"*: `2914567`
><!--        - *"Minimum contig length"*: `500`
>-->
{: .hands_on}

**Shovill* generates 3 outputs:
- A log file
- A FASTA file with contigs, i.e. contiguous length of genomic sequences in which bases are known to a high degree of certainty.

  > <question-title></question-title>
  > Inspect the `Contigs` file
  >
  > 1. How big is the first contig?
  > 2. What is the coverage of your biggest (first) contig?
  >
  > > <solution-title></solution-title>
  > > Both of these can be found in the header line of the Contigs produced by Shovill
  > >
  > > *The results can differ from this example, because Shovill can differ a bit per assembly*
  > >
  > > 1. 589,438 bp, almost 6 times lower than the estimated genome size
  > > 2. 17.7
  > {: .solution}
  {: .question}

- A file with the contig graph or assembly graph

  The assembly graph is more information-rich because it not only contains the sequences of all assembled fragments (including the ones shorter than the threshold length defined for inclusion of the fragments into the FASTA output), but also indicates the relative average coverage of the fragments by sequenced reads and how some of the fragments could potentially be bridged after resolving ambiguities manually.

## Assembly inspection

Th assembly graph format takes some getting used to before you can make sense out of the information it provides. This issue can be alleviated through the use of **Bandage** ({% cite wick2015bandage %}), a package for exploring assembly graphs through summary reports and visualizations of their contents.

> <hands-on-title>Assembly inspection</hands-on-title>
> 1. {% tool [Bandage Info](toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_info/2022.09+galaxy2) %} with the following parameters:
>    - {% icon param-file %} *"Graphical Fragment Assembly"*: **Shovill** `Contig Graph`
>    - *"Produce jpg, png or svg file?"*: `.jpg`
>
{: .hands_on}

Let's look at the assembly statistics

> <question-title></question-title>
>
> 1. What is the number of dead end?
> 2. What is the number of nodes?
>
> > <solution-title></solution-title>
> >
> > 1. Only 2
> > 2. 131: a bit disapointing as it should be encoded in one contig
> {: .solution}
{: .question}

Let's now visualize the graph.

> <hands-on-title>Assembly inspection</hands-on-title>
> 1. {% tool [Bandage Image](toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/2022.09+galaxy4) %} with the following parameters:
>    - {% icon param-file %} *"Graphical Fragment Assembly"*: **Shovill** `Contig Graph`
>    
{: .hands_on}

![Bandage output showing a mess of a genome graph](./images/bandage.jpg)


> <question-title></question-title>
> First read [this page in the Bandage wiki](https://github.com/rrwick/Bandage/wiki/Simple-example) to help understand what the graph means.
>
> What do you think of this assembly? Is it useful? Is it good enough?
>
> > <solution-title></solution-title>
> >
> > This is a very messy assembly, with a lot of potential paths through the sequence. We cannot feel confident in the output FASTA file (as it is much smaller than the expected 2.9Mbp). In real life we might consider doing a hybrid assembly with Nanopore or other long read data to help resolve these issues.
> >
> {: .solution}
{: .question}


## Assembly Evaluation

To evaluate the assembly, we use also [Quast](http://quast.bioinf.spbau.ru/) ({% cite Gurevich2013 %}) (QUality ASsessment Tool), a tool providing quality metrics for assemblies. This tool can be used to compare multiple assemblies, can take an optional reference file as input to provide complementary metrics, etc

> <hands-on-title>Quality Control of assembly using Quast</hands-on-title>
>
> 1. {% tool [Quast](toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1) %} with the following parameters:
>    - *Assembly mode?*: `Co-assembly`
>       - *"Use customized names for the input files?"*: `No, use dataset names`
>          - {% icon param-file %} *"Contigs/scaffolds file"*: **Shovill** `Contigs` output
>
{: .hands_on}

QUAST outputs assembly metrics as an HTML file with metrics and graphs. 

This fasta file contains 

> <question-title></question-title>
>
> 1. How many contigs is there?
> 2. What is the total length of all contigs?
> 1. What is you GC content?
>
> > <solution-title></solution-title>
> >
> > 1. 30 contigs, meaning the chromosome is separated over multiple contigs. These contigs can also contain (parts of) plasmids.
> > 2. 2,911,349 (*Total length (>= 0 bp)*). Not far from the estimated genome size: 2,914,567 
> > 1. The GC content for our assembly was 32.77%. For comparison [MRSA Isolate HC1335 Strain](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5174738/) GC% is 32.89%.
> >
> {: .solution}
{: .question}

The total length and the GC content of the assembly are coherent with expectations. 

# Conclusion

In this tutorial, we prepared short reads, assembled them, and inspect the produced assembly for its quality. The assembly, even if uncomplete, is reasonable good to be used in downstream analysis, like [AMR gene detection]({% link topics/genome-annotation/tutorials/amr-gene-detection/tutorial.md %})

tools_json:
["Import", "upload1", "Falco", "toolshed.g2.bx.psu.edu/repos/iuc/falco/falco/1.2.4+galaxy0", "fastp", "toolshed.g2.bx.psu.edu/repos/iuc/fastp/fastp/1.0.1+galaxy3", "Shovill", "toolshed.g2.bx.psu.edu/repos/iuc/shovill/shovill/1.1.0+galaxy1", "Bandage Info", "toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_info/2022.09+galaxy2", "Bandage Image", "toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/2022.09+galaxy4", "Quast", "toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1"]


topic: assembly
tutorial: debruijn-graph-assembly
title: De Bruijn Graph Assembly
difficulty: introductory
time_estimation: 2h
requirements_json: []
tags_json: []
content_hash: 25035b6a9f664cfb83dbaedf3cd6a39d
last_modified: 2026-05-27T09:01:47.800963
zenodo_link: https://doi.org/10.5281/zenodo.582600
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/debruijn-graph-assembly/tutorial.html

content:
# Optimised de Bruijn Graph assemblies using the Velvet Optimiser and SPAdes

In this activity, we will perform *de novo* assemblies of a short read set using the Velvet Optimiser and the SPAdes assemblers. We are using the Velvet Optimiser for illustrative purposes. For real assembly work, a more suitable assembler should be chosen - such as SPAdes.

The Velvet Optimiser is a script written by @gtn:slugger70 to optimise the k-mer size and coverage cutoff parameters for Velvet. More information can be found [in its repository](https://github.com/slugger70/VelvetOptimiser).

SPAdes is a de novo genome assembler written by Pavel Pevzner's group in St. Petersburg. More details on it can be found [on Spades' website](http://cab.spbu.ru/software/spades/)>



> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. [Get the data](#get-the-data)
> 2. [Assemble with the Velvet Optimiser](#assembly-with-the-velvet-optimiser)
> 3. [Assemble with SPAdes](#assemble-with-spades)
{: .agenda}

# Get the data

We will be using the same data that we used in the introductory tutorial, so if you have already completed that and have the data, skip this section.

> <hands-on-title>Getting the data</hands-on-title>
>
> 1. Create and name a new history for this tutorial.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the sequence read raw data (\*.fastq) from [Zenodo](https://zenodo.org/record/582600)
>
>    ```
>    https://zenodo.org/record/582600/files/mutant_R1.fastq
>    https://zenodo.org/record/582600/files/mutant_R2.fastq
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Rename the files {% icon galaxy-pencil %}
>    - The name of the files are the full URL, let's make the names a little clearer
>    - Change the names to just the last part, `Mutant_R1.fastq`, `Mutant_R2.fastq`, respectively
>
>    {% snippet faqs/galaxy/datasets_rename.md %}
>
>    > <question-title></question-title>
>    >
>    > 1. What are four key features of a FASTQ file?
>    > 2. What is the main difference between a FASTQ and a FASTA file?
>    {: .question}
>
> 4. Create a paired collection named `Paired Reads`
>
>    {% snippet faqs/galaxy/collections_build_list_paired.md %}
>
>    - We will need to use both the individual datasets (`Mutant_R1.fastq` and `Mutant_R2.fastq`) and the paired end collection (`Paired Reads`), so toggle off the `Hide original elements` option when creating the collection.
>    - Alternatively, you can un-hide the datasets by selecting the {% icon galaxy-show-hidden %} icon on the hidden dataset in your history.
>
>    {% snippet faqs/galaxy/datasets_unhidden.md %}
>
{: .hands_on}

# Assembly with the Velvet Optimiser

We will perform an assembly with the Velvet Optimiser, which automatically runs and optimises the output of the Velvet assembler ({% cite Velvet2008 %}). It will automatically choose a suitable value for the k-mer size (**k**). It will then go on to optimise the coverage cutoff (**cov_cutoff**) which corrects for read errors. It will use the "*n50*" metric for optimising the k-mer size and the "*total number of bases in contigs*" for optimising the coverage cutoff.

> <hands-on-title>Assemble with the Velvet Optimiser</hands-on-title>
>
>  1. {% tool [Velvet Optimiser](toolshed.g2.bx.psu.edu/repos/simon-gladman/velvetoptimiser/velvetoptimiser/2.2.6) %}: Optimise your assembly with the following parameters:
>    - *"Start k-mer size"*: `45`
>    - *"End k-mer size"*: `73`
>    - *"Input Files"*:
>        - `1: Input Files`
>            - *"Input file type"*: `Fastq`
>            - *"Single or paired end reads"*: `Paired`
>            - {% icon param-file %} *"Select first set of reads"*: `mutant_R1.fastq`
>            - {% icon param-file %} *"Select second set of reads"*: `mutant_R2.fastq`
>
{: .hands_on}

Your history will now contain a number of new files:

* Velvet optimiser contigs
  * A fasta file of the final assembled contigs
* Velvet optimiser contig stats
  * A table of the lengths (in k-mer length) and coverages (k-mer coverages) for the final contigs.

Have a look at each file.


> <hands-on-title>Get contig statistics for Velvet Optimiser contigs</hands-on-title>
>
> 1. {% tool [Fasta Statistics](toolshed.g2.bx.psu.edu/repos/iuc/fasta_stats/fasta-stats/1.0.1) %}: Produce a summary of the velvet optimiser contigs:
>    - {% icon param-file %} *"fasta or multifasta file"*: Select your velvet optimiser contigs file
>
> 2. View the output
>
>    > <question-title></question-title>
>    >
>    > Compare the output we got here with the output of the simple assemblies obtained in the introductory tutorial.
>    > 1. What are the main differences between them?
>    > 2. Which has a higher "n50"? What does this mean?
>    {: .question}
>
{: .hands_on}

Tables of results from **(a)** Simple assembly and **(b)** optimised assembly.

**(a)** ![The results of the contigs from Simple assembly.](../../images/image12.png)

**(b)** ![The results of the contigs from Optimised assembly. In contrast to simple assembly produced much higher n_50, while num_seq is lower.](../../images/optstats.png)

> <details-title>Further reading on assembly with Velvet</details-title>
> - Heuristic Resolution of Repeats and Scaffolding in the Velvet Short-Read de Novo Assembler ({% cite Zerbino2009 %})
>
{: .details}

## Visualisation of the Assembly

Now that we've assembled the genomes, let's visualise this assembly using [Bandage](https://rrwick.github.io/Bandage/) ({% cite Wick2015 %}). This tool will let us better understand how the assembly graph really looks, and can give us a feeling for if the genome was well assembled or not.

Currently VelvetOptimiser does not include the LastGraph output, so we will manually run `velveth` and `velvetg` with the optimised parameters.

> <hands-on-title>Manually running velvetg/h</hands-on-title>
>
> 1. Locate the output called "VelvetOptimiser: Contigs" in your history
>
> 2. Click the {% icon dataset-info %} information icon
>
> 3. Check the tool `stderr` in the information page for the optimised k-mer value
{: .hands_on}

> <question-title></question-title>
> What was the optimal k-mer value? (referred to as *"hash"* in the stderr log)
> > <solution-title></solution-title>
> > 55
> {: .solution}
{: .question}

With this information in hand, let's run velvet:

> <hands-on-title>Manually running velvetg/h</hands-on-title>
>
> 1. {% tool [velveth](toolshed.g2.bx.psu.edu/repos/devteam/velvet/velveth/1.2.10.3) %}: Prepare a dataset for the Velvet `velvetg` Assembler
>    - *"Hash length"*: `55`
>    - *"Input Files"*:
>      - `+ Insert Input Files`
>      - `1: Input Files`
>        - *"Choose the input type"*: `separate paired reads`
>        - *"read type"*: `shortPaired reads`
>        - *"Dataset"*: `mutant_R1.fastq` (forward reads)
>        - *"Dataset"*: `mutant_R2.fastq` (reverse reads)
>
> 2. {% tool [velvetg](toolshed.g2.bx.psu.edu/repos/devteam/velvet/velvetg/1.2.10.2) %}: Velvet sequence assembler for very short reads
>    - *"Velvet dataset"*: output from **velveth** {% icon tool %}
>    - *"Coverage cutoff"*: `Specify Cutoff Value`
>      - *"Remove nodes with coverage below"*: `1.44`
>    - *"Additional outputs"*: `Generate velvet LastGraph file`
>    - *"Using Paired Reads"*: `Yes`
>
{: .hands_on}

The LastGraph contains a detailed representation of the De Bruijn graph, which can give us an idea how velvet has assembled the genome and potentially resolved any conflicts.

> <hands-on-title>Bandage</hands-on-title>
>
> 1. {% tool [Bandage Image](toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/2022.09+galaxy4) %}: visualize de novo assembly graphs
>    - *"Graphical Fragment Assembly"*: The "LastGraph" output of **velvetg** {% icon tool %}
>    - *"Produce jpg, png or svg file?"*: `.png`
>
> 2. Execute
> 3. View the output file
{: .hands_on}

And now you should be able to see the graph that velvet produced:

![velvet graph](../../images/bandage-velvet.svg)

## Interpreting Bandage Graphs

k-mer size has a [significant effect](https://github.com/rrwick/Bandage/wiki/Effect-of-kmer-size) on the assembly. You can play around with various k-mers to see this effect in practice.

k-mer | graph
----- | -----
21    | [![21](../../images/bandage-velvet-21.svg)](../../images/bandage-velvet-21.svg)
33    | [![33](../../images/bandage-velvet-33.svg)](../../images/bandage-velvet-33.svg)
53    | [![53](../../images/bandage-velvet-53.svg)](../../images/bandage-velvet-53.svg)
77    | [![77](../../images/bandage-velvet-77.svg)](../../images/bandage-velvet-77.svg)

The next thing to be aware of is that there can be multiple valid interpretations of a graph, all equally valid in absence of other data. The following is taken verbatim [from Bandage's wiki](https://github.com/rrwick/Bandage/wiki/Simple-example):

> For a simple case, imagine a bacterial genome that contains a single repeated element in two separate places in the chromosome:
>
> ![Simple example 1]({% link topics/assembly/images/debruijn_assembly/bandage1.png %})
>
> A researcher (who does not yet know the structure of the genome) sequences it, and the resulting 100 bp reads are assembled with a de novo assembler:
>
> ![Simple example 2]({% link topics/assembly/images/debruijn_assembly/bandage2.png %})
>
> Because the repeated element is longer than the sequencing reads, the assembler was not able to reproduce the original genome as a single contig. Rather, three contigs are produced: one for the repeated sequence (even though it occurs twice) and one for each sequence between the repeated elements.
>
> Given only the contigs, the relationship between these sequences is not clear. However, the assembly graph contains additional information which is made apparent in Bandage:
>
> ![Simple example 3]({% link topics/assembly/images/debruijn_assembly/bandage3.png %})
>
> There are two principal underlying sequences compatible with this graph: two separate circular sequences that share a region in common, or a single larger circular sequence with an element that occurs twice:
>
> ![Simple example 4]({% link topics/assembly/images/debruijn_assembly/bandage4.png %})
>
> Additional knowledge, such as information on the approximate size of the bacterial chromosome, can help the researcher to rule out the first alternative. In this way, Bandage has assisted in turning a fragmented assembly of three contigs into a completed genome of one sequence.
{: .quote cite="https://github.com/rrwick/Bandage/wiki/Simple-example"}

# Assemble with SPAdes

We will now perform an assembly with the much more modern SPAdes assembler ({% cite Bankevich2012 %}). It goes through a similar process to Velvet in the fact that it uses and simplifies de Bruijn graphs but it uses multiple values for k-mer size and combines the resultant graphs. This combination produces very good assemblies. When using SPAdes it is typical to choose at least 3 k-mer sizes. One low, one medium and one high. We will use 33, 55 and 91.

> <hands-on-title>Assemble with SPAdes</hands-on-title>
>
> 1. {% tool [SPAdes](toolshed.g2.bx.psu.edu/repos/nml/spades/spades/4.2.0+galaxy0) %}: Assemble the reads:
>    - *"Operation mode"*: `Only assembler (--only_assembler)`
>    - *"Single-end or paired-end short-reads"*: `Paired-end: list of dataset pairs`
>      - *"FASTA/FASTQ file(s): collection"*: `Paired Reads`
>    - *"Set coverage cutoff option"*: `auto`
>    - *"Select k-mer detection option"*: `User specific`
>      - *"K-mer size values"*: `33,55,91` [note: no spaces!]
>    - *"Select optional output file(s)"*: `Assembly graph`, `Assembly graph with scaffold`, `Contigs`, `Scaffolds`, `Log`
>
{: .hands_on}

You will now have 5 new files in your history:

* two Fasta files, one for contigs and one for scaffolds
* two statistics files, one for contigs and one for scaffolds
* the SPAdes log file.

Examine each file, especially the stats files.

![Contig stats file with NODE_5 being the shortest contig with the highest coverage and NODE_1 being the opposite.](../../images/contig_stats.png)

> <question-title></question-title>
>
> 1. Why would one of the contigs have much higher coverage than the others?
> 2. What could this represent?
>
{: .question}


> <hands-on-title>Visualize assembly with Bandage</hands-on-title>
>
> 1. {% tool [Bandage Image](toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/2022.09+galaxy4) %} with the following parameters:
>    - *"Graphical Fragment Assembly"*: `assembly graph with scaffolds` output from **SPAdes** {% icon tool %}
>    - *"Produce jpg, png or svg file?"*: `.png`
>
> 2. Examine the output image {% icon galaxy-eye %}
>
{: .hands_on}

The visualized assembly should look something like this:

![bandage spades](../../images/bandage_spades.svg)


> <question-title></question-title>
>
> Which assembly looks better to you? Why?
>
{: .question}



> <hands-on-title>Get contig statistics for SPAdes contigs</hands-on-title>
>
> 1. {% tool [Fasta Statistics](toolshed.g2.bx.psu.edu/repos/iuc/fasta_stats/fasta-stats/1.0.1) %}: Produce a summary of the SPAdes contigs:
>    - {% icon param-file %} *"fasta or multifasta file"*: Select your velvet optimiser contigs file
>
> 2. Look at the output file.
>
>    > <question-title></question-title>
>    >
>    > Compare the output we got here with the output of the simple assemblies obtained in the introductory tutorial.
>    > 1. What are the main differences between them?
>    > 2. Did SPAdes produce a better assembly than the Velvet Optimiser?
>    {: .question}
>
{: .hands_on}

tools_json:
["Velvet Optimiser", "toolshed.g2.bx.psu.edu/repos/simon-gladman/velvetoptimiser/velvetoptimiser/2.2.6", "Fasta Statistics", "toolshed.g2.bx.psu.edu/repos/iuc/fasta_stats/fasta-stats/1.0.1", "velveth", "toolshed.g2.bx.psu.edu/repos/devteam/velvet/velveth/1.2.10.3", "velvetg", "toolshed.g2.bx.psu.edu/repos/devteam/velvet/velvetg/1.2.10.2", "Bandage Image", "toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/2022.09+galaxy4", "SPAdes", "toolshed.g2.bx.psu.edu/repos/nml/spades/spades/4.2.0+galaxy0"]


topic: assembly
tutorial: chloroplast-assembly
title: Chloroplast genome assembly
difficulty: intermediate
time_estimation: 2h
requirements_json: []
tags_json: ["plants", "nanopore", "jbrowse1", "biodiversity"]
content_hash: 52db5a66dadcb9a04cc298c5673df94b
last_modified: 2026-05-27T09:01:47.800812
zenodo_link: https://doi.org/10.5281/zenodo.3567224
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/chloroplast-assembly/tutorial.html

content:
## What is genome assembly?

Genome assembly is the process of joining together DNA sequencing fragments into longer pieces, ideally up to chromosome lengths.The DNA fragments are produced by DNA sequencing machines, and are called "reads". These are in lengths of about 150 nucleotides (base pairs), to up to a million+ nucleotides, depending on the sequencing technology used. Currently, most reads are from Illumina (short), PacBio (long) or Oxford Nanopore (long and extra-long).

It is difficult to assemble plant genomes as they are often large (for example, 3,000,000,000 base pairs), have many repeat regions (such as transposons), and may be polyploid. This tutorial shows genome assembly for a smaller data set - the plant chloroplast genome - a single circular chromosome which is typically about 160,000 base pairs. It is thought that the the chloroplast evolved from a cyanobacteria that was living in plant cells.

In this tutorial, we will use a subset of a real data set from sweet potato, from the paper {% cite Zhou2018 %}. To find out more about each of the tools used here, see the tool panel page for a summary and links to more information.

><agenda-title></agenda-title>
> In this tutorial we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Upload data

Let's start with uploading the data.

> <hands-on-title>Import the data</hands-on-title>
> 1. Create a new history for this tutorial and give it a proper name
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 2. Import from [Zenodo](https://zenodo.org/record/3567224) or a data library (ask your instructor):
>   - FASTQ file with illumina reads: `sweet-potato-chloroplast-illumina-reduced.fastq`
>   - FASTQ file with nanopore reads: `sweet-potato-chloroplast-nanopore-reduced.fastq`
>   - Note: make sure to import the files with "reduced" in the names, not the ones with "tiny" in the names.
>    ```
>    https://zenodo.org/record/3567224/files/sweet-potato-chloroplast-illumina-reduced.fastq
>    https://zenodo.org/record/3567224/files/sweet-potato-chloroplast-nanopore-reduced.fastq
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
{: .hands_on}

# Check read quality

We will look at the quality of the nanopore reads.

> <hands-on-title>Check read quality</hands-on-title>
>
> 1. {% tool [Nanoplot](toolshed.g2.bx.psu.edu/repos/iuc/nanoplot/nanoplot/1.28.2+galaxy1) %}:
>    - *"Select multifile mode"*: `batch`
>    - *"Type of file to work on"*: `fastq`
>    - *"files"*: select the `nanopore FASTQ file`
> 2. **View output**:
>    * There are five output files.
>    *  Look at the `HTML report` to learn about the read quality.
{: .hands_on}

> <question-title></question-title>
> What summary statistics would be useful to look at?
>
> > <solution-title></solution-title>
> > This will depend on the aim of your analysis, but usually:
> > * **Sequencing depth** (the number of reads covering each base position; also called "coverage"). Higher depth is usually better, but at very high depths it may be better to subsample the reads, as errors can swamp the assembly graph.
> > * **Sequencing quality** (the quality score indicates probability of base call being correct). You may trim or filter reads on quality. Phred quality scores are logarithmic: phred quality 10 = 90% chance of base call being correct; phred quality 20 = 99% chance of base call being correct. More detail [on Wikipedia](https://en.wikipedia.org/wiki/Phred_quality_score).
> > * **Read lengths** (read lengths histogram, and reads lengths vs. quality plots). Your analysis or assembly may need reads of a certain length.
> {: .solution}
{: .question}

*Optional further steps:*
* Find out the quality of your reads using other tools such as fastp or FastQC.
* To visualize base quality using emoji you can also use FASTQE.
* Run FASTQE for the illumina reads. In the output, look at the mean values (the middle row)
* Repeat FASTQE for the nanopore reads. In the tool settings, increase the maximum read length to 30000.
* To learn more, see the [Quality Control tutorial]({{site.baseurl}}/topics/sequence-analysis/tutorials/quality-control/tutorial.html)

# Assemble reads

We will assemble the long nanopore reads.

> <hands-on-title>Assemble reads</hands-on-title>
> 1. {% tool [Flye](toolshed.g2.bx.psu.edu/repos/bgruening/flye/flye/2.6+galaxy0) %}:
>    - *"Input reads"*: `sweet-potato-chloroplast-nanopore-reduced.fastq`
>    - *"Estimated genome size"*: `160000`
>    - *Leave other settings as default*
>
> 2. Re-name the `consensus` output file to `flye-assembly.fasta`
>
>    {% snippet faqs/galaxy/datasets_rename.md name="flye-assembly.fasta" %}
>
> 3. **View output**:
>    * There are five output files.
>    * *Note: this tool is heuristic; your results may differ slightly from the results here, and if repeated.*
>    * View the `log` file and scroll to the end to see how many contigs (fragments) were assembled and the length of the assembly.
>    * View the `assembly_info` file to see contig names and lengths.
{: .hands_on}


> <hands-on-title>View the assembly</hands-on-title>
> 1. {% tool [Bandage Info](toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_info/0.8.1+galaxy1) %}
>    - *"Graphical Fragment Assembly"*: the Flye output file `Graphical Fragment Assembly` (not the "assembly_graph" file)
>    - *Leave other settings as default*
> 2. {% tool [Bandage Image](toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/0.8.1+galaxy2) %}
>    - *"Graphical Fragment Assembly"*: the Flye output file `Graphical Fragment Assembly` (not the "assembly_graph" file)
>    - *"Node length labels"*: `Yes`
>    - *Leave other settings as default*
{: .hands_on}

Your assembly graph may look like this:

![assembly graph](../../images/sweet-potato-assembly-graph.png "Assembly graph of the nanopore reads, for the sweet potato chloroplast genome. In your image, some of the labels may be truncated; this is a known bug under investigation.")

*Note: a newer version of the Flye assembly tool now resolves this assembly into a single circle.*

> <question-title></question-title>
> What is your interpretation of this assembly graph?
> > <solution-title></solution-title>
> > One interpretation is that this represents the typical circular chloroplast structure: There is a long single-copy region (the node of around 78,000 bp), connected to the inverted repeat (a node of around 28,000 bp), connected to the short single-copy region (of around 11,000 bp). In the graph, each end loop is a single-copy region (either long or short) and the centre bar is the collapsed inverted repeat which should have about twice the sequencing depth.
> {: .solution}
{: .question}

> <comment-title>Further Learning</comment-title>
> * Repeat the Flye assembly with different parameters, and/or a filtered read set.
> * You can also try repeating the Flye assembly with an earlier version of the tool, to see the difference it makes. In the tool panel for Flye, click on the 'Versions' button at the top to change.
> * Try an alternative assembly tool, such as Canu or Unicycler.
{: .comment}

# Polish assembly

Short illumina reads are more accurate the nanopore reads. We will use them to correct errors in the nanopore assembly.

First, we will map the short reads to the assembly and create an alignment file.

> <hands-on-title>Map reads</hands-on-title>
> 1. {% tool [Map with BWA-MEM](toolshed.g2.bx.psu.edu/repos/devteam/bwa/bwa_mem/0.7.17.1) %}:
>    - *"Will you select a reference genome from your history"*: `Use a genome from history`
>    - *"Use the following dataset as the reference sequence"*: `flye-assembly.fasta`
>    - *"Algorithm for constructing the BWT index"*: `Auto. Let BWA decide`
>    - *"Single or Paired-end reads"*: `Single`
>    - *"Select fastq dataset"*: `sweet-potato-illumina-reduced.fastq`
>    - *"Set read groups information?"*: `Do not set`
>    - *"Select analysis mode"*: `Simple Illumina mode`
>2. **Re-name output file**:
>   - Re-name this file `illumina.bam`
{: .hands_on}

Next, we will compare the short reads to the assembly, and create a polished (corrected) assembly file.

> <hands-on-title>Polish</hands-on-title>
> 1. {% tool [pilon](toolshed.g2.bx.psu.edu/repos/iuc/pilon/pilon/1.20.1) %}:
>    - *"Source for reference genome used for BAM alignments"*: `Use a genome from history`
>    - *"Select a reference genome"*: `flye-assembly.fasta`
>    - *"Type automatically determined by pilon"*: `Yes`
>    - *"Input BAM file"*: `illumina.bam`
>    - *"Variant calling mode"*: `No`
>    - *"Create changes file"*: `Yes`
>
> 2. **View output**:
>   * What is in the `changes` file?
>   * Rename the fasta output to `polished-assembly.fasta`
>
>    {% snippet faqs/galaxy/datasets_rename.md name="polished-assembly.fasta" %}
>
> 3. {% tool [Fasta Statistics](toolshed.g2.bx.psu.edu/repos/iuc/fasta_stats/fasta-stats/1.0.1) %}
>    - Find and run the tool called "Fasta statistics" on both the original flye assembly and the polished version.
{: .hands_on}

> <question-title></question-title>
> How does the polished assembly compare to the unpolished assembly?
> > <solution-title></solution-title>
> > This will depend on the settings, but as an example: your polished assembly might be about 10-15 Kbp longer. Nanopore reads can have homopolymer deletions - a run of AAAA may be interpreted as AAA - so the more accurate illumina reads may correct these parts of the long-read assembly. In the changes file, there may be a lot of cases showing a supposed deletion (represented by a dot) being corrected to a base.
> {: .solution}
{: .question}

*Optional further steps:*
* Run a second round (or more) of Pilon polishing. Keep track of file naming; you will need to generate a new bam file first before each round of Pilon.
* Run an alternative polishing tool, such as Racon. This uses the long reads themselves to correct the long-read (Flye) assembly. It would be better to run this tool on the Flye assembly before running Pilon, rather than after Pilon.

# Annotate the assembly

We can now annotate our assembled genome with information about genomic features.

* A chloroplast genome annotation tool is not yet available in Galaxy; for an approximation, here we can use the tool for bacterial genome annotation, Prokka.

> <hands-on-title>Annotate with Prokka</hands-on-title>
>
> 1. {% tool [Prokka](toolshed.g2.bx.psu.edu/repos/crs4/prokka/prokka/1.14.5+galaxy0) %} with the following parameters (leave everything else unchanged)
>    - {% icon param-file %} *"contigs to annotate"*: `polished-assembly.fasta`
> 2. **View output**:
>    * The GFF and GBK files contain all of the information about the features annotated (in different formats.)
>    * The .txt file contains a summary of the number of features annotated.
>    * The .faa file contains the protein sequences of the genes annotated.
>    * The .ffn file contains the nucleotide sequences of the genes annotated.
{: .hands_on}


Alternatively, you might want to use a web-based tool designed for chloroplast genomes.

* One option is the GeSeq tool, described here. Skip this step if you have already used Prokka above.

> <hands-on-title>Annotate with GeSeq</hands-on-title>
> * Download `polished-assembly.fasta` to your computer (click on the file in your history; then click on the disk icon).
> * In a new browser tab, go to [Chlorobox](https://chlorobox.mpimp-golm.mpg.de/geseq.html) where we will use the GeSeq tool ({% cite Tillich_2017 %}) to annotate our sequence.
> * Upload the `fasta` file there. Information about how to use the tool is available on the page.
> * Once the annotation is completed, download the required files.
> * In Galaxy, import the annotation `GFF3` file.
{: .hands_on}

Now make a JBrowse file to view the annotations (the GFF3 file - produced from either Prokka or GeSeq) under the assembly (the polished-assembly.fasta file).

> <hands-on-title>View annotations</hands-on-title>
> 1. {% tool [JBrowse genome browser](toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/jbrowse/1.16.4+galaxy3) %}:
>    - *"Reference genome to display"*: `Use a genome from history`
>        - *"Select a reference genome"*: `polished-assembly.fasta`
>    - *"Output JBrowse"*: `Minimal for viewing (Documentation removed)`
>    - *"Genetic Code"*: `11. The Bacterial, Archaeal and Plant Plastid Code`
>    - *"JBrowse-in-Galaxy Action"*: `New JBrowse instance`
>    - *"Insert Track Group"*
>        - *"Insert Annotation Track"*
>            - *"Track Type"*: `GFF/GFF3/BED Features`
>            - *"GFF/GFF3/BED Track Data"*: the `GFF3` file
>            - *Leave the other track features as default*
> 2. **Re-name output file**:
>    - JBrowse may take a few minutes to run. There is one output file: re-name it `view-annotations`
> 3. **View output**:
>    * Click on the eye icon to view the annotations file.
>    * Select the right contig to view, in the drop down box.
>    * Zoom out (with the minus button) until annotations are visible.
{: .hands_on}

Here is an embedded snippet showing JBrowse and the annotations:

{% snippet topics/visualisation/faqs/visualizations_jbrowse.html datadir="data2" %}

# View reads

We will look at the original sequencing reads mapped to the genome assembly. In this tutorial, we will import very cut-down read sets so that they are easier to view.

> <hands-on-title>Import cut-down read sets</hands-on-title>
> 1. {% tool [Import](upload1) %} from [Zenodo](https://zenodo.org/record/3567224) or a data library (ask your instructor):
>   - FASTQ file with illumina reads: `sweet-potato-chloroplast-illumina-tiny.fastq`
>   - FASTQ file with nanopore reads: `sweet-potato-chloroplast-nanopore-tiny.fastq`
>   - Note: these are the "tiny" files, not the "reduced" files we imported earlier.
>    ```
>    https://zenodo.org/record/3567224/files/sweet-potato-chloroplast-illumina-tiny.fastq
>    https://zenodo.org/record/3567224/files/sweet-potato-chloroplast-nanopore-tiny.fastq
>    ```
{: .hands_on}

> <hands-on-title>Map the reads to the assembly</hands-on-title>
> * Map the Illumina reads (the new "tiny" dataset) to the `polished-assembly.fasta`, the same way we did before, using bwa mem.
> * This creates one output file: re-name it `illumina-tiny.bam`
> * Map the Nanopore reads (the new "tiny" dataset) to the `polished-assembly.fasta`. The settings will be the same, except `Select analysis mode` should be `Nanopore`
> * This creates one output file: re-name it `nanopore-tiny.bam`
{: .hands_on}

> <hands-on-title>Visualise mapped reads</hands-on-title>
> 1. {% tool [JBrowse genome browser](toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/jbrowse/1.16.4+galaxy3) %}:
>    - *"Reference genome to display"*: `Use a genome from history`
>        - *"Select a reference genome"*: `polished-assembly.fasta`
>    - *"Output JBrowse"*: `Minimal for viewing (Documentation removed)`
>    - *"Genetic Code"*: `11. The Bacterial, Archaeal and Plant Plastid Code`
>    - *"JBrowse-in-Galaxy Action"*: `New JBrowse instance`
>    - *"Insert Track Group"*
>        - *"Insert Annotation Track"*
>            - *"Track Type"*: `BAM pileups`
>            - *"BAM track data"*: `nanopore-tiny.bam`
>            - *"Autogenerate SNP track"*: `No`
>            - *Leave the other track features as default*
>        - *"Insert Annotation Track"*.
>            - *"Track Type"*: `BAM pileups`
>            - *"BAM track data"*: `illumina-tiny.bam`
>            - *"Autogenerate SNP track"*: `No`
>            - *Leave the other track features as default*
> 2. **Re-name output file**:
>    - JBrowse may take a few minutes to run. There is one output file: re-name it `assembly-and-reads`
> 3. **View output**:
>    * Click on the eye icon to view. (For more room, collapse Galaxy side menus with corner < > signs).
>    *  Make sure the bam files are ticked in the left hand panel.
>    * Choose a contig in the drop down menu. Zoom in and out with + and - buttons.
{: .hands_on}

Here is an embedded snippet showing JBrowse and the mapped reads:

{% snippet topics/visualisation/faqs/visualizations_jbrowse.html datadir="data" %}


> <question-title></question-title>
>
> 1. What are the differences between the nanopore and the illumina reads?
> 2. What are some reasons that the read coverage may vary across the reference genome?
>
> > <solution-title></solution-title>
> > 1. Nanopore reads are longer and have a higher error rate.
> > 2. There may be lots of reasons for varying read coverage. Some possibilities: In areas of high read coverage: this region may be a collapsed repeat. In areas of low or no coverage: this region may be difficult to sequence; or, this region may be a misassembly.
> {: .solution}
{: .question}

* To learn more about JBrowse and its features, see the [Genomic Data Visualisation with JBrowse tutorial]({{site.baseurl}}/topics/visualisation/tutorials/jbrowse/tutorial.html)


# Repeat with new data

*Optional extension exercise*

We can assemble another chloroplast genome using sequence data from a different plant species: the snow gum, *Eucalyptus pauciflora*. This data is from {% cite Wang_etal_2018 %}. It is a subset of the original FASTQ read files (Illumina - SRR7153063, Nanopore - SRR7153095).

> <hands-on-title>Assembly and annotation</hands-on-title>
> * Get data: at this [Zenodo link](https://doi.org/10.5281/zenodo.3600662), then upload to Galaxy.
> * Check reads: Run Nanoplot on the nanopore reads.
> * Assemble: Use Flye to assemble the nanopore reads, then get Fasta statistics *Note: this may take several hours.*
> * Polish assembly: Use Pilon to polish the assembly with short Illumina reads. *Note: Don't forget to map these Illumina reads to the assembly first using bwa-mem, then use the resulting `bam` file as input to Pilon.*
> * Annotate: Use the GeSeq tool at [Chlorobox](https://chlorobox.mpimp-golm.mpg.de/geseq.html) or the Prokka tool within Galaxy.
> * View annotations:Use JBrowse to view the assembled, annotated genome.
{: .hands_on}

# Conclusion

tools_json:
["Nanoplot", "toolshed.g2.bx.psu.edu/repos/iuc/nanoplot/nanoplot/1.28.2+galaxy1", "Flye", "toolshed.g2.bx.psu.edu/repos/bgruening/flye/flye/2.6+galaxy0", "Bandage Info", "toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_info/0.8.1+galaxy1", "Bandage Image", "toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/0.8.1+galaxy2", "Map with BWA-MEM", "toolshed.g2.bx.psu.edu/repos/devteam/bwa/bwa_mem/0.7.17.1", "pilon", "toolshed.g2.bx.psu.edu/repos/iuc/pilon/pilon/1.20.1", "Fasta Statistics", "toolshed.g2.bx.psu.edu/repos/iuc/fasta_stats/fasta-stats/1.0.1", "Prokka", "toolshed.g2.bx.psu.edu/repos/crs4/prokka/prokka/1.14.5+galaxy0", "JBrowse genome browser", "toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/jbrowse/1.16.4+galaxy3", "Import", "upload1"]


topic: assembly
tutorial: metagenomics-assembly
title: Assembly of metagenomic sequencing data
difficulty: introductory
time_estimation: 2H
requirements_json: []
tags_json: ["assembly", "metagenomics", "microgalaxy"]
content_hash: 0f2c5783ec6a46920b4227c792889ec4
last_modified: 2026-05-27T09:01:49.354276
zenodo_link: https://zenodo.org/record/7818827
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/metagenomics-assembly/tutorial.html

content:
Metagenomics involves the extraction, sequencing and analysis of combined genomic DNA from **entire microbiome** samples. It includes then DNA from **many different organisms**, with different taxonomic background.

Reconstructing the genomes of microorganisms in the sampled communities is critical step in analyzing metagenomic data. To do that, we can use **assembly** and assemblers, *i.e.* computational programs that stich together the small fragments of sequenced DNA produced by sequencing instruments.

Assembling seems intuitively similar to putting together a jigsaw puzzle. Essentially, it looks for reads "that work together" or more precisely, reads that overlap. Tasks like this are **not straightforward**, but rather complex because of the complexity of the genomics (specially the repeats), the missing pieces and the errors introduced during sequencing.

> <comment-title></comment-title>
>
> Do you want to learn more about the principles behind single genome assembly? Follow our [tutorials]({% link topics/assembly/index.md %}).
{: .comment}

**Metagenomic assembly** is further complicated by
- the large volume of data produced
- the quality of the sequence
- the unequal representation of members of the microbial community
- the presence of closely related microorganisms with similar genomes
- the presence of several strains of the same microorganism
- an insufficient amount of data for minor community members

> <details-title>Strategies for assembly</details-title>
> For assembly, there are 3 main strategies:
> 1. Greedy extension
> 2. Overlap Layout Consensus
> 3. De Bruijn graphs. The following figure illustrates these strategies in brief.
>
> ![Image shows greedy extention, overlap layout consensus, and de Brujin graphs assembly algorithms.](./images/assembly-algorithms.png "Assembly algorithms. Image from {% cite carpentries %}."){:width="70%"}
>
> The nice paper {% cite miller2010 %} on assemblers based on these algorithms will help you to better understand how they work.
{: .details}

For metagenomic assembly, several tools exist: metaSPAdes ({% cite nurk2017 %}), MEGAHIT ({% cite li2015 %}), etc. The different assemblers have different computational characteristics and their performance varies according to the microbiome as shown in by both rounds of  Critical Assessment of Metagenome Interpretation initiative ({% cite sczyrba2017 %}, {% cite meyer2022 %}, {% cite meyer2021 %}). The preference of one assembler over another depends on the purpose at hand.

In this tutorial, we will learn how to run metagenomic assembly tool and evaluate the quality of the generated assemblies. To do that, we will use data from the study: [Temporal shotgun metagenomic dissection of the coffee fermentation ecosystem](https://www.ebi.ac.uk/metagenomics/studies/MGYS00005630#overview). For an in-depth analysis of the structure and functions of the coffee microbiome, a temporal shotgun metagenomic study (six time points) was performed. The six samples have been sequenced with Illumina MiSeq utilizing whole genome sequencing.

Based on the 6 original dataset of the coffee fermentation system, we generated mock datasets for this tutorial.

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Prepare analysis history and data

To run assembly, we first need to get the data into Galaxy. Any analysis should get its own Galaxy history. So let's start by creating a new one:

> <hands-on-title>Prepare the Galaxy history</hands-on-title>
>
> 1. Create a new history for this analysis
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Rename the history
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
{: .hands_on}

We need to get the data into our history.

In case of a not very large dataset it's more convenient to upload data directly from your computer to Galaxy.

> <hands-on-title>Upload data into Galaxy</hands-on-title>
>
> 1. Import the sequence read raw data (\*.fastqsanger.gz) from [Zenodo]({{ page.zenodo_link }}) or a data library:
>
>    ```text
>    {{ page.zenodo_link }}/files/ERR2231567_1.fastqsanger.gz
>    {{ page.zenodo_link }}/files/ERR2231567_2.fastqsanger.gz
>    {{ page.zenodo_link }}/files/ERR2231568_1.fastqsanger.gz
>    {{ page.zenodo_link }}/files/ERR2231568_2.fastqsanger.gz
>    {{ page.zenodo_link }}/files/ERR2231569_1.fastqsanger.gz
>    {{ page.zenodo_link }}/files/ERR2231569_2.fastqsanger.gz
>    {{ page.zenodo_link }}/files/ERR2231570_1.fastqsanger.gz
>    {{ page.zenodo_link }}/files/ERR2231570_2.fastqsanger.gz
>    {{ page.zenodo_link }}/files/ERR2231571_1.fastqsanger.gz
>    {{ page.zenodo_link }}/files/ERR2231571_2.fastqsanger.gz
>    {{ page.zenodo_link }}/files/ERR2231572_1.fastqsanger.gz
>    {{ page.zenodo_link }}/files/ERR2231572_2.fastqsanger.gz
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
>    > <comment-title></comment-title>
>    > In case of large dataset, we can use FTP server or the [Galaxy Rule-based Uploader]({% link topics/galaxy-interface/tutorials/upload-rules/tutorial.md %}).
>    {: .comment}
>
> 2. Create a paired collection named `Raw reads`, rename your pairs with the sample name
>
>    {% snippet faqs/galaxy/collections_build_list_paired.md %}
>
{: .hands_on}

# Assembly

As explained before, there are many challenges to metagenomics assembly, including:

1. Differences in coverage between samples, resulting from differences in abundance;
2. The fact that different species often share conserved regions ({%cite kececioglu2001%}), and
3. The presence of multiple strains of a single species ({%cite miller2010%}).

To reduce the differences in coverage between samples, we can use a **co-assembly** approach, where reads from all samples are aligned together:

![Image show one pile of sample1 reads and another pile of sample2 reads, then, green arrow leads to assembled reads from both piles](./images/co-assembly.png "Co-assembly"){:width="60%"}

Pros of co-assembly | Cons of co-assembly
--- | ---
More data | Higher computational overhead
Better/longer assemblies | Risk of shattering your assembly
Access to lower abundant organisms | Risk of increased contamination

Co-assembly is then not always beneficial:
- Changes in strain can cause the assembly graph to collapse
- Binned contigs are likely to be misclassified: MAGs must be treated as a population genome.

![Image shows the process of assembled reads from 2 samples followed by binning and there is detailed information about broken contigs, dominant coverage, chimeric contigs found during assembly process](./images/co-assembly-process.png "Co-assembly process"){:width="80%"}

In these cases, co-assembly is reasonable if:
- Same samples
- Same sampling event
- Longitudinal sampling of the same site
- Related samples

{% snippet faqs/galaxy/fastq_groupmerge.md %}

Examples where co-assembly would be reasonable:
- Repeated sampling of the **same patient** along a particular amount of time.
- Multiple samples taken from the **same site** and **similar environmental conditions**, eg. a patch of soil during the same sampling season.

Examples where co-assembly would NOT be recommended:
- Samples from different patients.
If samples differ as described, **individual assembly** is preferred. In the case of individual assembly, **contigs should be binned** per sample and an extra step of **de-replication** should be used as binning:

If samples differ as described, **individual assembly** is preferred. In the case of individual assembly, **contigs should be binned** per sample and an extra step of **de-replication** should be used after binning:


> <comment-title></comment-title>
> For more information on dereplication, check out the [metagenomic binning tutorial]({% link topics/microbiome/tutorials/metagenomics-binning/tutorial.md %}).
{: .comment}

In this tutorial, to show all steps, we will run an **individual assembly**.

> <comment-title>Why not both?</comment-title>
> It is also possible to run both individual assembly and co-assembly, and this approach can recover MAGs effectively. In this case: individual assembly can recover MAGs with a low amount of contamination, while co-assembly also allows for the recovery of low-abundance MAGs, with the downside of potentially more contamination. Although this approach can be effective, it also requires high computational resources and should be considered carefully.
>
> {% snippet faqs/galaxy/fastq_groupmerge.md %}
>
{: .comment}

As mentioned in the introduction, several tools are available for metagenomic assembly. But 2 are the most used ones:

- **MetaSPAdes** ({%cite nurk2017%}): an short-read assembler designed specifically for large and complex metagenomics datasets.

  MetaSPAdes is part of the SPAdes toolkit, which has several assembly pipelines. Since SPAdes handles non-uniform coverage, it is useful for assembling simple communities, but metaSPAdes also handles other problems, allowing it to assemble complex communities' metagenomes.

  As input for metaSPAdes it can accept short reads. However, there is an option to use additionally long reads besides short reads to produce hybrid input.

- **MEGAHIT** ({% cite li2015 %}): a single node assembler for large and complex metagenomics NGS reads, such as soil

  It makes use of the Succinct de Bruijn Graph (SdBG) approach to achieve low memory assembly.

Both tools are available in Galaxy. But currently, only **MEGAHIT** can be used in individual mode for several samples, as a option in the wrapper.
**MetaSPAdes** performs co-assembly by default if multiple samples are supplied. However, individually assembly can be enforced:

{% snippet faqs/galaxy/collections_force_mapping_over.md %}

# Assembly with MEGAHIT

> <hands-on-title>Individual assembly of short-reads with MEGAHIT</hands-on-title>
> 1.  {% tool [MEGAHIT](toolshed.g2.bx.psu.edu/repos/iuc/megahit/megahit/1.2.9+galaxy2) %} with parameters:
>     - *"Select your input option"*: `Paired-end collection`
>       - *"Run in batch mode?"*: `Run individually`
>
>          > <comment-title></comment-title>
>          > To run as co-assembly, select `Merge all fastq pair-end`, instead of `Run individually`
>          {: .comment}
>
>       - {% icon param-collection %} *"Select a paired collection"*: `Raw reads`
>     - In *Basic assembly options*
>       - *"K-mer specification method"*: `Specify min, max, and step values`
>         - *"Minimum kmer size"*: 21
>         - *"Maximum kmer size"*: 91
>         - *"Increment of kmer size of each iteration"*: 12
>
{: .hands_on}

**MEGAHIT** produced a collection of output assemblies - one per sample - that can be used for the subsequent step of **metagenomic binning**. The output contains **contigs**, contiguous lengths of genomic sequences in which bases are known to a high degree of certainty.

> <comment-title>Scaffolds</comment-title>
> Contrary to **MetaSPAdes**, **MEGAHIT** does not output **scaffolds**. **Scaffolds** are segments of genome sequence reconstructed fron contigs and gaps. The gaps occur when reads from the two sequenced ends of at least one fragment overlap with other reads from two different contigs (as long as the arrangement is otherwise consistent with the contigs being adjacent). It is possible to estimate the number of bases between contigs based on fragment lengths.
{: .comment}

> <comment-title></comment-title>
>
> Since the assembly process would take ~1h we are just going to import the results of the assembly previously run.
>
> > <hands-on-title>Import generated assembly files</hands-on-title>
> >
> > 1. Import the six contig files from [Zenodo]({{ page.zenodo_link }}) or the Shared Data library:
> >
> >    ```text
> >    {{ page.zenodo_link }}/files/contigs_ERR2231567.fasta
> >    {{ page.zenodo_link }}/files/contigs_ERR2231568.fasta
> >    {{ page.zenodo_link }}/files/contigs_ERR2231569.fasta
> >    {{ page.zenodo_link }}/files/contigs_ERR2231570.fasta
> >    {{ page.zenodo_link }}/files/contigs_ERR2231571.fasta
> >    {{ page.zenodo_link }}/files/contigs_ERR2231572.fasta
> >    ```
> >
> >
> > 2. Create a collection named `MEGAHIT Contigs`, rename your pairs with the sample name
> >
> {: .hands_on}
{: .comment}

> <question-title>Contig metrics</question-title>
>
> 1. How many contigs have been assembled for sample ERR2231568 ?
> 2. And how many have been assembled for ERR2231572?
> 3. What is the minimum length of the contigs?
>
> > <solution-title></solution-title>
> >
> > 1. There are 228,719 sequences in the file so 228,719 contigs
> > 2. 122,526 contigs
> > 3. Sequences seems bigger than 200 bp (`len` attribute of the sequence information in Fasta files). It is the default value set up in MEGAHIT.
> >
> {: .solution}
>
{: .question}

> <details-title>Co-assembly with MetaSPAdes</details-title>
> MetaSPAdes supports co-assembly by passing a list of paired-end read files.
>
> > <hands-on-title>Assembly with MetaSPAdes</hands-on-title>
> > 1. {% tool [MetaSPAdes](toolshed.g2.bx.psu.edu/repos/nml/metaspades/metaspades/4.2.0+galaxy0) %} with the following parameters:
>     - *"Pair-end reads input format"*: `Paired-end: list of dataset pairs`
> >      - {% icon param-collection %} *"FASTQ file(s): collection"*: `Raw reads`
> >
> >    > <comment-title>Tip</comment-title>
> >    > To run **individual assembly** instead, follow the previous FAQ to create a **nested list of samples**.
> >     {: .comment}
> >
> >   - *"Select k-mer detection option"*: `User specific`
> >    - *"K-mer size values"*: `21,33,55,77`
> {: .hands_on}
{: .details}

# Quality control of assembly

Once assembly is done, it is important to check its quality.

Assemblies can be evaluated with **metaQUAST** ({%cite mikheenko2016%}), the metagenomics mode of **QUAST** ({%cite gurevich2013%}).

> <hands-on-title>Evaluation of assembly quality with metaQUAST</hands-on-title>
>
> 1. {% tool [QUAST](toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1) %} with parameters:
>     - *"Assembly mode?"*: `Individual assembly (1 contig file per samples)`
>       - *"Use customized names for the input files?"*: `No, use dataset names`
>         - {% icon param-collection %} *"Contigs/scaffolds file"*: output **MEGAHIT**
>       - *"Reads options"*: `Illumina paired-end reads in paired collection`
>
>       > <comment-title>Tip</comment-title>
>       > To speed up the processing, you may choose `Disabled` here. In this case, raw reads will **not** be mapped back to the assembly to compute metrics such as read coverage.
>       {: .comment}
>
>       - {% icon param-collection %} *"FASTQ/FASTA files"*: `Raw reads`
>     - *"Type of assembly"*: `Metagenome`
>     - *"Output files"*: `HTML report`, `PDF report`, `Tabular reports`, `Log file`, `Key metric summary (metagenome mode)`, `Krona charts (metagenome mode without reference genomes)`
>
> 2. Inspect the HTML reports:
> ![Screenshot of QUAST HTML report](./images/quast_html_report.png)
{: .hands_on}


> <tip-title>Preview MetaQUAST results</tip-title>
>
> 1. Since the QUAST process takes some time, we also provide an example history here:
>
>    - [Galaxy EU](https://usegalaxy.eu/histories/view?id=11ac94870d0bb33aeff2ed936aab8446)
>
{: .tip}

QUAST main output are HTML reports which aggregate different metrics.

## Assembly statistics

On the top of each report is a table with in rows statistics for contigs larger than 500 bp for the different sample assemblies (columns). Let's now look at the table and go from top to bottom:

1. **Genome statistics**

    - **Genome fraction (%)**: percentage of aligned bases in the reference genome

      A base in the reference genome is counted as aligned if at least one contig has at least one alignment to this base.

      We did not provide any reference genome, but QUAST tries to identify the genome content of the metagenome by aligning contigs to [SILVA](https://www.arb-silva.de/) 16S rRNA database. For each assembly, 50 reference genomes with top scores are chosen. The full reference genomes of the identified organisms are downloaded from NCBI to map the assemblies on them and compute the genome fractions.

      > <comment-title>Metagenome reference</comment-title>
      > The alignment to automatically downloaded genomes for metagenomes is rather ambiguous and time-consuming. Most large-scale pipelines skip this step and set the **Maximum number of reference genomes (per each assembly) to download after searching in the SILVA database*** option to `0`.
      {: .comment}

      For each identified genomes, the genome fraction is given when clicking on **Genome fraction (%)**

      > <question-title></question-title>
      >
      > 1. What is the genome fraction for ERR2231568? And for ERR2231572?
      > 2. Which reference genome has the highest genome fraction for ERR2231568? And for ERR2231572?
      >
      > > <solution-title></solution-title>
      > >
      > > 1. The genome fraction is 29.429% for ERR2231568 and 60.463% for ERR2231572
      > > 2. The highest genome fraction was found for *Leuconostoc pseudomesenteroides* for ERR2231568 (84.2%) and for *Lactobacillus vaccinostercus* for ERR2231572 (92.2%). The genomes of *Leuconostoc pseudomesenteroides* and *Lactobacillus vaccinostercus* could be then almost completely recovered from the assemblies of ERR2231568 and ERR2231572 respectively.
      > >
      > {: .solution}
      >
      {: .question}

    - **Duplication ratio**: total number of aligned bases / genome fraction * reference length

      If an assembly contains many contigs that cover the same regions of the reference, the duplication ratio may be much larger than 1.

      > <question-title></question-title>
      >
      > 1. What is the duplication ratio for ERR2231568? And for ERR2231572?
      > 2. Which reference genome has the highest duplication ratio for ERR2231568? And for ERR2231572?
      >
      > > <solution-title></solution-title>
      > >
      > > 1. The duplication ratio is 1.062 for ERR2231568 and 1.112 for ERR2231572 (column ERR2231572 in ERR2231572 report)
      > > 2. The highest duplication ratio was found for *Gluconobacter oxydans* for ERR2231568 (1.156) and for *Lactobacillus pseudomesenteroides* for ERR2231572 (1.204).
      > >
      > {: .solution}
      >
      {: .question}

2. **Read mapping**: results of the mapping of the raw reads on the different assemblies (only if the *"Reads options"* is not disabled)

    > <question-title></question-title>
    >
    > 1. What is the % of read mapped for ERR2231568 assembly to ERR2231568 raw reads? And for ERR2231572 assembly to ERR2231572 raw reads?
    > 2. What is the percentage of reads used to build the assemblies for ERR2231568? and ERR2231572?
    >
    > > <solution-title></solution-title>
    > >
    > > 1. 79.47% of ERR2231568 raw reads were mapped to ERR2231568 assembly and 86.98% of ERR2231572 raw reads to ERR2231572 assembly.
    > > 2. 79.47% of reads were used to the assemblies for ERR2231568 and  86.98% for ERR2231572.
    > {: .solution}
    >
    {: .question}

    > <details-title>Alternative ways to compute coverage</details-title>
    >
    > There are 2 alternative ways to compute coverage:
    > 1. Use [CoverM](https://github.com/wwood/CoverM), which is available in Galaxy
    >
    >    > <hands-on-title>Calculate coverage using CoverM</hands-on-title>
    >    >
    >    > 1. {% tool [CoverM contig](toolshed.g2.bx.psu.edu/repos/iuc/coverm_contig/coverm_contig/0.7.0+galaxy0) %} with parameters:
    >    >    - *"Have the reads already been mapped to contigs?"*: `No`
    >    >       - *"Assembly mode?"*: `Individual assembly (1 contig file per sample)`
    >    >          - *"Read type"*: `Paired collection`
    >    >             - {% icon param-collection %} *"Collection of paired-end FASTA/Q files(s) for mapping"*: `Raw reads`
    >    >    - {% icon param-collection %} *"Contigs"*: output of MEGAGIT
    >    >
    >    > 2. Open the {% icon dataset-info %} "Dataset details" for ERR2231568
    >    > 3. Expand the `Tool Standard Error`
    >    {: .hands_on}
    >
    >    > <question-title></question-title>
    >    >
    >    > 1. What is the overall alignment rate for ERR2231568
    >    > 2. What is the percentage of reads used in assemblies for ERR2231568?
    >    > 3. Are the results similar to the ones in QUAST report?
    >    >
    >    > > <solution-title></solution-title>
    >    > >
    >    > > 1. The overall alignment rate for ERR2231567 is 75.48%
    >    > > 2. 75.48% of the reads were used in assemblies for ERR2231567 and 73.67% for ERR2231571.
    >    > > 3. Slighlty lower results that the ones found by QUAST (79.47% for ERR2231568)
    >    > >
    >    > {: .solution}
    >    >
    >    {: .question}
    >
    > 2. Map the original reads onto contigs and extract the percentage of mapped reads:
    >
    >    > <hands-on-title>Computation of the % reads used in assemblies</hands-on-title>
    >    >
    >    > 1. {% tool [Bowtie2](toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.5.3+galaxy1) %} with the following parameters:
    >    >    - *"Is this single or paired library"*: `Paired-end Dataset Collection`
    >    >      - {% icon param-collection %} *"FASTQ Paired Dataset"*: `Raw reads`
    >    >    - *"Will you select a reference genome from your history or use a built-in index?"*: `Use a genome from the history and build index`
    >    >      - {% icon param-collection %} *"Select reference genome"*: MEGAHIT output
    >    >    - *"Save the Bowtie2 mapping statistics to the history"*: `Yes`
    >    > 3. Inspect the mapping statistics output
    >    {: .hands_on}
    >
    >    > <question-title></question-title>
    >    >
    >    > 1. What is the overall alignment rate for ERR2231568? and ERR2231572?
    >    > 2. What is the percentage of reads used in assemblies for ERR2231568? and ERR2231571?
    >    > 3. Are the results similar to the ones in QUAST report?
    >    >
    >    > > <solution-title></solution-title>
    >    > >
    >    > > 1. The overall alignment rate for ERR2231568 is 65.67% and 78.42% for ERR2231572.
    >    > > 2. 65.97% of the reads were used in assemblies for ERR2231568 and 78.42% for ERR2231572.
    >    > > 3. Lower results that the ones found by QUAST (79.47% for ERR2231568, 86.98%  for ERR2231572)
    >    > >
    >    > {: .solution}
    >    >
    >    {: .question}
    {: .details}

3. **Misassemblies**: joining sequences that should not be adjacent.

    QUAST identifies missassemblies by mapping the contigs to the reference genomes of the identified organisms. 3 types of misassemblies can be identified:

    ![Image shows on the top a contig with a blue and a gren parts with white arrows (pointing on the right) on them and below a reference with 2 chromosomes. The 3 types of misassemblies are after schematized. Relocation: the blue and gren parts of the contig are on chr 1 but separated. Inversion: the blue and gren parts of the contig are on chr 1 but separated and with the arrows facing each other. Translocation: the blue part is on chr 1 and gren part on chr 2.](./images/quast_misassemblies.png "Source: <a href="https://quast.sourceforge.net/docs/manual.html#sec3.1.2">QUAST manual</a>"){:width="60%"}

    1. **Relocation** occur based on signal from two mappings of the same contig against the same chromosome, with 2 cases:
        1. the 2 mappings are separated by an unmapped region of at least 1 kbp
        2. they map on the same chromosome with a shared mapping area of at least 1 kbp

        ![Image shows on the top a contig with a blue and a gren parts with white arrows (pointing on the right) on them and below a reference with 2 chromosomes. The 3 types of misassemblies are after schematized. Relocation: the blue and gren parts of the contig are on chr 1 but separated. Inversion: the blue and gren parts of the contig are on chr 1 but separated and with the arrows facing each other. Translocation: the blue part is on chr 1 and gren part on chr 2.](./images/relocation.svg "Source: <a href="https:///blog.pierre.marijon.fr/misassemblies-in-noisy-assemblies/">Yet Another Bioinformatic blog by Pierre Marijon</a>"){:width="80%"}

        > <question-title></question-title>
        >
        > 1. How many relocations has been found for ERR2231568? And for ERR2231572?
        > 2. For which reference genomes are there the most relocation found for ERR2231568? And for ERR2231572?
        >
        > > <solution-title></solution-title>
        > >
        > > 1. 83 for ERR2231568 and 159 for ERR2231572
        > > 2. *Leuconostoc pseudomesenteroides* and *Tatumella morbirosei* for ERR2231568 and *Lactobacillus hordei* for ERR2231572
        > {: .solution}
        >
        {: .question}

    2. **Translocation** occur when a contig has mapped on more than one reference chromosomes

        > <question-title></question-title>
        >
        > 1. How many translocations has been found for ERR2231568? And for ERR2231572?
        > 2. For which reference genomes are there the most translocations found for ERR2231568? And for ERR2231572?
        > 3. What are the interspecies translocations?
        > 4. How many interspecies translocations has been found for ERR2231568? And for ERR2231572?
        >
        > > <solution-title></solution-title>
        > >
        > > 1. 30 for ERR2231568 and 90 for ERR2231572.
        > > 2. *Leuconostoc pseudomesenteroides* for ERR2231568 and *Lactobacillus vaccinostercus* for ERR2231572.
        > > 3. Interspecies translocations are translocations where a contig has mapped on different reference genomes.
        > > 4. 114 for ERR2231568 and 85 for ERR2231572.
        > {: .solution}
        >
        {: .question}

    3. **Inversion** occurs when a contig has two consecutive mappings on the same chromosome but in different strands

        > <question-title></question-title>
        >
        > 1. How many inversion has been found for ERR2231568? And for ERR2231572?
        > 2. For which reference genomes are there the most inversions found for ERR2231568? And for ERR2231572?
        >
        > > <solution-title></solution-title>
        > >
        > > 1. 4 for ERR2231568 and 9 for ERR2231572.
        > > 2. *Tatumella morbirosei* for ERR2231568 and *Lactobacillus plantarum* for ERR2231572.
        > {: .solution}
        >
        {: .question}

4. **Mismatches** or mismatched bases in the contig-reference alignment

    > <question-title></question-title>
    >
    > 1. How many mismatches have been identified for ERR2231568? And for ERR2231572?
    > 2. For which reference genomes are there the most mismatches for ERR2231568? And for ERR2231572?
    >
    > > <solution-title></solution-title>
    > >
    > > 1. 469,292 for ERR2231568 and 279,694 for ERR2231572.
    > > 2. *Pantoea rwandensis* for ERR2231568 and *Leuconostoc pseudomesenteroides* for ERR2231572.
    > {: .solution}
    >
    {: .question}

5. **Statistics without reference**

    - **# contigs**: total number of contigs

        > <question-title></question-title>
        >
        > 1. How many contigs are for ERR2231568? And for ERR2231572?
        > 2. How many sequences are in the output of MEGAHIT for ERR2231568? And for ERR2231572?
        > 3. Why are these numbers different from the number of sequences in the output of MEGAHIT?
        > 4. Which statistics in the metaQUAST report corresponds to number of sequences in the output of MEGAHIT?
        > 5. Which reference genomes have the most contigs ($$\geq$$ 500 bp) in ERR2231568? And in ERR2231572?
        >
        > > <solution-title></solution-title>
        > >
        > > 1. 66,434 contigs for ERR2231568 and 36,112 for ERR2231572.
        > > 2. In the outputs of MEGAHIT, there are 228,719 contigs for ERR2231568 and 122,526 contigs.
        > > 3. The numbers are lower in the metaQUAST results because metaQUAST reports there only the contigs longer than 500bp.
        > > 4. The **# contigs (>= 0 bp)**
        > > 5. Except the non aligned contigs, *Tatumella morbirosei* for ERR2231568 and *Leuconostoc brevis* for ERR2231572.
        > {: .solution}
        >
        {: .question}

    - **Largest contig**: length of the longest contig in the assembly

        > <question-title></question-title>
        >
        > 1. What is the length of the longest contig in ERR2231568? And in ERR2231572?
        > 2. Is the longest contig assigned to a reference genome in ERR2231568? And in ERR2231572?
        >
        > > <solution-title></solution-title>
        > >
        > > 1. 63,871 bp in ERR2231568 and 65,608 for ERR2231572.
        > > 2. It is assigned to *Leuconostoc pseudomesenteroides KCTC 3652* in ERR2231568 and not assigned (`not_aligned`) in ERR2231572.
        > {: .solution}
        >
        {: .question}

    - **N50**: length for which the collection of all contigs of that length or longer covers at least half an assembly

        N50 statistic defines assembly quality in terms of contiguity. If all contigs in an assembly are ordered by length, the N50 is the minimum length of contigs that contains 50% of the assembled bases. For example, an N50 of 10,000 bp means that 50% of the assembled bases are contained in contigs of at least 10,000 bp.

        Another example. Let's consider 9 contigs with the lengths 2, 3, 4, 5, 6, 7, 8, 9, and 10:
        - The sum of the length is 54
        - Half of the sum is 27
        - 10 + 9 + 8 = 27 (half the length of the sequence)
        - N50 = 8, i.e. the size of the contig which, along with the larger contigs, contain half of sequence of a particular genome

        > <question-title></question-title>
        >
        > 1. What is N50 for ERR2231568? And for ERR2231572?
        > 2. What is N90?
        >
        > > <solution-title></solution-title>
        > >
        > > 1. 921 for ERR2231568 and 1,233 for ERR2231572.
        > > 2. N90 is similar to the N50 metric but with 90% of of the sum of the lengths of all contigs
        > {: .solution}
        >
        {: .question}

        When comparing N50 values from different assemblies, the assembly sizes must be the same size in order for N50 to be meaningful.

        Also the N50 alone is not a useful measure to assess the quality of an assembly. For example, the assemblies with the following contig lengths:
        - 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 25, 25, 150, 1500
        - 50, 500, 530, 650

        Both assemblies have the same N50 although one is more contiguous than the other.

    - **L50**: number of contigs equal to or longer than N50

        In other words, L50, for example, is the minimal number of contigs that cover half the assembly.

        If we take the previous example in N50, L50 = 3.

        > <question-title></question-title>
        >
        > 1. What is the L50 for ERR2231568? And for ERR2231572?
        >
        > > <solution-title></solution-title>
        > >
        > > 1. 17,280 for ERR2231568 and 7,496 for ERR2231572.
        > {: .solution}
        >
        {: .question}

## Icarus contig browser

Icarus generates contig size viewer and one or more contig alignment viewers (if reference genome/genomes are provided) that are accessible from the HTML report, by clicking on `View on Icarus contig browser`.

### Contig size viewer

This viewer draws contigs ordered from longest to shortest. Let's inspect this viewer for ERR2231568.

> <question-title></question-title>
>
> Open the Contig size viewer for ERR2231568 and define start as `0` and end as `500000`
>
> ![Image shows on the Icarus Contig Size Viewer for ERR2231568, with a zoom between 0 500000. Below the menu, the contigs are drawn from the longest on the left to the shortest on the right. Each contig is filled with a different color: green for correct, red for misassembled, etc. Below the contigs is dispayed a bar to navigate through the contigs](./images/ERR2231568-contig-size-viewer.png)
>
> 1. What is the color of the first contig? Why?
> 2. What is the red contig?
>
> > <solution-title></solution-title>
> >
> > 1. The first contig is white because >50% of the contigs is unaligned. By clicking on the contig, we see that only a small block is aligned: 223.41 – 223.65 kbp to Leuconostoc_pseudomesenteroides_KCTC_3652_NZ_BMBP01000002.1.
> > 2. The red contig is a missamblied contig: it contains 2 blocks, with a translocation between them.
> {: .solution}
>
{: .question}

Click `Main menu` on the top left to go back to the main Icarus page.

### Contig alignment viewer

If a reference genome is provided, there should be a table on the main Icarus page that looks like:

Genome | # fragments | Length, bp | Mean genome fraction, % | # misassembled blocks
--- | --- | --- | --- | ---
Gluconobacter oxydans H24 | 2 | 3 816 232 | 10.989 | 38
Kosakonia cowanii | 5 | 4 806 998 | 28.224 | 60
Lactococcus lactis subsp. lactis CV56 | 6 | 2 518 737 | 18.940 	23

When clicking on the genome name, the contigs are displayed according to their mapping to the reference genome. The viewer can additionally visualize genes, operons, and read coverage distribution along the genome, if any of those were fed to QUAST.

> <question-title></question-title>
>
> Open the Contig alignment viewer for ERR2231568 and *Leuconostoc pseudomesenteroides* KCTC 3652, the most covered by contigs
>
> ![Image shows on the Icarus alignment size viewer of ERR2231568 contigs for Leuconostoc pseudomesenteroides KCTC 3652. On the top are displayed the contigs aligend on the reference genome, with a zoom. Each contig is filled with a different color: green for correct, red for misassembled, etc. Below the contigs is dispayed a bar to navigate through the contigs. Below is a graph representing the GC percentage and coverage along the reference genome](./images/ERR2231568-contig-alignment-viewer-leuconostoc.png)
>
> 1. How are the organized the contigs on the top?
> 2. What the different colors for the contigs?
> 3. Why is there a big red block on the right?
> 4. What is the graph on the bottom?
>
> > <solution-title></solution-title>
> >
> > 1. The contigs are displayed based on their mapping on the reference genome of *Leuconostoc pseudomesenteroides* KCTC 3652
> > 2. The different colors represent the different status for the contig: correct contigs, correct contigs but with >50% of the contig unaligned, misassembled blocks. unchecked misassembled blocks, ambiguously mapped contigs, alternative blocks of misassembled contigs, etc.
> > 3. The big red block on the right is contig k91_183138 with a misassembly on the left side, and overlap with 2 other contigs
> > 4. The graph on the bottom represents the GC percentage and coverage by contigs along the reference genome
> {: .solution}
>
{: .question}

> <details-title>Visualization of the *de novo* assembly graph</details-title>
>
> Current metagenome assemblers like MEGAHIT and MetaSPAdes use **graphs**, most typically a de Bruijn graph to stich reads together. In an ideal case, the graph would contain one distinct path for each genome of each micro-organisms, but complexities such as repeated sequences usually prevent this.
>
> Assembly graphs contain then **branching structures**: one node may lead into multiple others. **Contigs** correspond to the longest sequences in the graph that can be determined unambiguously. They are the final results of most assembler. But the assembly graph contains more information. It can be useful for finding sections of the graph, such as rRNA, or to try to find parts of a genome.
>
> **Bandage** ({% cite wick2015bandage %}) is a tool creating interactive visualisations of assembly graphs.
>
> > <hands-on-title>Visualization the assembly graph</hands-on-title>
> >
> > 1. {% tool [megahit contig2fastg](toolshed.g2.bx.psu.edu/repos/iuc/megahit_contig2fastg/megahit_contig2fastg/1.1.3+galaxy10) %} with parameters:
> >    - {% icon param-collection %} *"Contig file"*: Output of **MEGAHIT**
> >    - *"K-mer length"*: `91`
> >
> >      > <comment-title></comment-title>
> >      > To get the value, you need to
> >      > 1. Go into the **MEGAHIT** output collection
> >      > 2. Expand one of the contig file by clicking on it in the history
> >      > 3. Check in the dataset peek the name of the contig
> >      > 4. Extract the value after the first `k` in the contig names
> >      {: .comment}
> >
> > 2. {% tool [Bandage Image](toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/0.8.1+galaxy4) %} with parameters:
> >    - {% icon param-collection %} *"Graphical Fragment Assembly"*: Output of **megahit contig2fastg**
> >
> > 3. Inspect the generated image for ERR2231571
> {: .hands_on}
>
> ![Image shows the assembly graphs, with longer stretch on the top and many small contigs on the bottom](./images/ERR2231571_graph.jpg "Assembly graph for ERR2231571 sample")
>
> The graph is quite disconnected. On the top, we can see the longer stretches, that includes multiples contigs (each contig having a different color). On the bottom are the shortest stretches or single contigs.
>
> But it is really hard to read or extract any information from the graph. Let's inspect the information about the assembly graph
>
> > <hands-on-title>Visualization the *de novo* assembly graph</hands-on-title>
> >
> > 1. {% tool [Bandage Info](toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_info/0.8.1+galaxy2) %} with parameters:
> >    - {% icon param-collection %} *"Graphical Fragment Assembly"*: Output of **megahit contig2fastg**
> >
> > 2. {% tool [Column join](toolshed.g2.bx.psu.edu/repos/iuc/collection_column_join/collection_column_join/0.0.3) %} with parameters:
> >    - {% icon param-collection %} *"Tabular files"*: Output of **Bandage Info**
> >
> > 3. Inspect the generated output
> {: .hands_on}
>
> > <question-title></question-title>
> >
> > 1. How many nodes are in the graph for ERR2231568? And for ERR2231572? What does they correspond to?
> > 2. How many edges are in the graph for ERR2231568? And for ERR2231572? What is the impact of these numbers in relation to the number of nodes on the graph?
> > 3. How many connected components are there for ERR2231568? And for ERR2231572? What does they correspond to?
> > 4. What is the percentage of dead ends are there for ERR2231568? And for ERR2231572?
> > 5. What are the smallest and larges edge overlaps?
> > 6. What is the largest component? For which sample?
> > 7. What is the shortest node? What does they correspond to?
> >
> > > <solution-title></solution-title>
> > >
> > > 1. There are 228,719 nodes for ERR2231568 and 122,526 for ERR2231572. They correspond to the number of contigs
> > > 2. There are 16,580 edges for ERR2231568 and 13,993 for ERR2231572. There are less edges than nodes in the graph. It means that many nodes/contigs are disconnected
> > > 3. There are 212,598 connected components, i.e. number of regions of the graph which are disconnected from each other, for ERR2231568 and 109,044 for ERR2231572
> > > 4. There are 94.0702% dead ends, i.e. the end of a node not connected to any other nodes, for ERR2231568 and 90.7032% for ERR2231572. It confirms the previous observation
> > > 5. The smallest and larges edge overlaps are 91bp, i.e. the k-mer length
> > > 6. The largest component is 340,003 bp for ERR2231567
> > > 7. The shortest node is 200 bp, i.e. the minimal size for a contig
> > >
> > {: .solution}
> >
> {: .question}
{: .details}

# Conclusion

Metagenomic data can be assembled to, ideally, obtain the genomes of the species that are represented within the input data. But metagenomic assembly is **complex** and there are
- **different approaches** like de Bruijn graphs methods
- **different strategies**, such as co-assembly, when we assembly all samples together, and individual assembly, when we assembly samples one by one
- **different tools** like MetaSPAdes and MEGAHIT

Once the choices made, metagenomic assembly can start:
1. Input data are assembled to obtain contigs and sometimes scaffolds.
2. Assembly quality is evaluated with various metrics.
3. The assembly graph can be visualized.

Once all these steps done, we can move to the next phase to build Metagenomics Assembled Genomes (MAGs): [metagenomic binning]({% link topics/microbiome/tutorials/metagenomics-binning/tutorial.md %}).

tools_json:
["MEGAHIT", "toolshed.g2.bx.psu.edu/repos/iuc/megahit/megahit/1.2.9+galaxy2", "MetaSPAdes", "toolshed.g2.bx.psu.edu/repos/nml/metaspades/metaspades/4.2.0+galaxy0", "QUAST", "toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1", "CoverM contig", "toolshed.g2.bx.psu.edu/repos/iuc/coverm_contig/coverm_contig/0.7.0+galaxy0", "Bowtie2", "toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.5.3+galaxy1", "megahit contig2fastg", "toolshed.g2.bx.psu.edu/repos/iuc/megahit_contig2fastg/megahit_contig2fastg/1.1.3+galaxy10", "Bandage Image", "toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_image/0.8.1+galaxy4", "Bandage Info", "toolshed.g2.bx.psu.edu/repos/iuc/bandage/bandage_info/0.8.1+galaxy2", "Column join", "toolshed.g2.bx.psu.edu/repos/iuc/collection_column_join/collection_column_join/0.0.3"]


topic: assembly
tutorial: unicycler-assembly
title: Unicycler Assembly
difficulty: introductory
time_estimation: 4h
tools_json: []
requirements_json: []
tags_json: ["prokaryote", "microgalaxy", "biodiversity"]
content_hash: 1b4a9aeab44fce4f0c0dc42e70fea01d
last_modified: 2026-05-27T09:01:47.817613
zenodo_link: https://doi.org/10.5281/zenodo.940733
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/unicycler-assembly/tutorial.html

content:
# The goal: *E. coli* C-1 assembly

In this tutorial we assemble and annotate the genome of *E. coli* strain [C-1](http://cgsc2.biology.yale.edu/Strain.php?ID=8232). This strain is routinely used in experimental evolution studies involving bacteriophages. For instance, now classic works by Holly Wichman and Jim Bull ({% cite Bull1997 %}, {% cite Bull1998 %}, {% cite Wichman1999 %}) have been performed using this strain and bacteriophage phiX174.

To sequence the genome we have obtained the strain from the [Yale E. coli Stock Center](http://cgsc2.biology.yale.edu/). The stock center sent us a filter paper disk infused with cells. The disk was placed in the center of an LB-agar plate. A single colony was picked and resuspended in a liquid LB medium, grown overnight, and genomic DNA was isolated. The DNA was then sequenced using two methods. To obtain high coverage, high accuracy data we used Illumina miSEQ to generated 250-bp paired end reads. To generate high length reads we used the Oxford Nanopore MinION machine.

Our goal is to reconstruct and annotate the full genome of *E. coli* C-1. As you will see in this tutorial a combination of many short, high accuracy reads with long, error-prone reads helps us produce an almost perfect assembly.

> ### Outline step-by-step
>
> In this tutorial, we will deal with:
>
>
> 1. TOC
> {:toc}
>
{: .agenda}


# Background on data and tools


## The data

In this tutorial we will assemble a genome using two types of input data: (1) Illumina 250 bp paired-end reads and (2) Oxford Nanopore reads.

### Illumina data

We generated 9,345,897 250 bp read pairs (library preparation performed on genomic DNA fragmented to mean size of 600 bp). However, to make sure that you can complete this tutorial in a finite amount of time we have downsampled (reduced in size) to 1,000,000 paired end reads - just enough to produce an accurate assembly.

### Oxford Nanopore Data

There are 12,738 [2d-reads](http://www.nature.com/nmeth/journal/v12/n4/fig_tab/nmeth.3290_SF13.html). Maximum read length is 27,518 bp. The distribution of reads lengths looks like this:

![Nanopore read length distribution](../../images/ont_length.png "Distribution of nanopore read lengths.")

You can see that there many reads under the second peak with median of approximately 7.5 kb.

> <warning-title>Oxford Nanopore Data Format</warning-title>
> Oxford Nanopore machines output data in [fast5](http://bioinformatics.cvr.ac.uk/blog/exploring-the-fast5-format/) format that contains additional information besides sequence data. In this tutorial we assume that this data is *already* converted into [fastq](https://en.wikipedia.org/wiki/FASTQ_format). An additional tutorial dedicated to handling fast5 datasets will be developed shortly.
{: .warning}


## The tools

In this analysis we will perform two tasks: (1) assembly and (2) annotation. Below we will briefly outline the main ideas behind these two procedures and will describe the tools we will be using.

### Assembly

> <comment-title>Knowing your assembly</comment-title>
>
> Here we assume that you know a thing or two about assembly process. If you don't: look at the slides accompanying this tutorial as well as other tutorials is this section.
{: .comment}

![Logo unicycler](https://github.com/rrwick/Unicycler/raw/master/misc/logo.png)

For assembly we will be using [Unicycler](https://github.com/rrwick/Unicycler) (also see publication {% cite Wick2017 %}). Unicycler is designed specifically for *hybrid assembly* (that is, using both short- and long-read sequencing data) of small (e.g., bacterial, viral, organellar) genomes. In our hands it has produced complete high quality assemblies. Unicycler employs a multi-step process that utilizes a number of software tools:

![Unicycler process](../../images/unicycler.png "Simplified view of the Unicycler assembly process (From {% cite Wick2017 %}) In short, Unicycler uses SPAdes (see below) to produce an assembly graph, which is then bridged (simplified) using long reads to produce the longest possible set of contigs. These are then polished by aligning the original short reads against contigs and feeding these alignments to Pilon - an assembly improvement tool. {% cite Wick2017 %}")



As you can see Unicycler relies heavily on SPAdes ({% cite Bankevich2012 %}) and [Pilon](https://github.com/broadinstitute/pilon/wiki). We will briefly describe these two tools.

#### Spades

##### Multisized deBruijn graph

Assemblers usually construct graphs for *k*-mers of a fixed size. We have noted that when *k* is small it is difficult to resolve the repeats. If *k* is too large a corresponding graph may be fragmented (especially if read coverage is low). SPAdes uses several values for *k* (that are either manually set or inferred automatically) to create a *multisized* graph that minimized tangledness and fragmentation by combining various *k*-mers ({% cite Bankevich2012 %})):

![Multigraph approach implemented in SPAdes](../../images/multiGraph.jpg "Multisized de Bruijn graph. A circular Genome CATCAGATAGGA is covered by a set of Reads consisting of nine 4-mers, {ACAT, CATC, ATCA, TCAG, CAGA, AGAT, GATA, TAGG, GGAC}. Three out of 12 possible 4-mers from Genome are missing from Reads (namely {ATAG,AGGA,GACA}), but all 3-mers from the Genome are present in the Reads. (A) The outside circle shows a separate black edge for each 3-mer from Reads. Dotted red lines indicate vertices that will be glued. The inner circle shows the result of applying some of the glues. (B) The graph DB(Reads, 3) resulting from all the glues is tangled. The three h-paths of length 2 in this graph (shown in blue) correspond to h-reads ATAG, AGGA, and GACA. Thus Reads<sub>3,4</sub> contains all 4-mers from Genome. (C) The outside circle shows a separate edge for each of the nine 4-mer reads. The next inner circle shows the graph DB(Reads, 4), and the innermost circle represents the Genome. The graph DB(Reads, 4) is fragmented into 3 connected components. (D) The multisized de Bruijn graph DB (Reads, 3, 4). Figure and text from {% cite Bankevich2012 %}.")

##### Read pair utilization

While the use of paired reads and mate pairs is key to genome assembly, and not new, SPAdes utilizes so called paired DeBruin graphs to take the advantage of the paired end data. One of the key issues with paired DeBruin graphs is that the resulting genome assemblies do not tolerate variability in insert sizes: The initial formulation of paired DeBruijn graphs assumed constant distance between pairs of reads. In practice this distance is always variable. SPAdes performs *k*-bimer (these are *k*-mers derived from *paired* reads) adjustment to identify exact or nearly-exact distances for each *k*-bimer pair.

##### Error correction

Sequencing data contains a substantial number of sequencing errors that manifest themselves as deviations (bulges and non-connected components) within the assembly graph. One way to improve the graph before assembly it is to minimize the number of sequencing errors by performing error correction. SPAdes uses BayesHammer ({% cite Nikolenko2013 %}) to correct the reads. Here is a brief summary of what it does:

1. SPAdes (or rather BayesHammer) counts *k*-mers in reads and computes *k*-mer statistics that take into account base quality values.
2. A [Hamming graph](https://en.wikipedia.org/wiki/Hamming_graph) is constructed in which *k*-mers are nodes. In this graph edges connect nodes (*k*-mers) if they differ from each other by a number of nucleotides up to a certain threshold (the [Hamming distance](https://en.wikipedia.org/wiki/Hamming_distance)). The graph is central to the error correction algorithm.
3. Then Bayesian subclustering is done on the graph from the previous step. For each *k*-mer we now know the center of its subcluster.
4. **Solid** *k*-mers are derived from cluster centers and are assumed to be *error free*.
5. Solid *k*-mers are mapped back to the reads.
6. Reads are corrected using solid *k*-mers:

![Read correction with BayesHammer](../../images/readCorrection.jpg "Read correction. Black <em>k</em>-mers are solid. Grey <em>k</em>-mers are non-solid. Red <em>k</em>-mers are the centers of the corresponding clusters (two grey <em>k</em>-mers striked through on the right are non-solid singletons). As a result, one nucleotide is changed based on majority rule. (From {% cite Nikolenko2013 %})")

In the case of the full dataset, SPAdes error correction changed 14,013,757 bases in 3,382,337 reads - a substantial fraction of the full ~18 million read dataset.

#### Pilon

Pilon improves draft assemblies by using the information from the original reads aligned to the draft assembly. The following image from a publication by {% cite Walker2014 %} highlights the steps of this process:

![Pilon workflow](../../images/pilon.png "The left column depicts the conceptual steps of the Pilon process, and the center and right columns describe what Pilon does at each step while in assembly improvement and variant detection modes, respectively. During the first step (top row), Pilon scans the read alignments for evidence where the sequencing data disagree with the input genome and makes corrections to small errors and detects small variants. During the second step (second row), Pilon looks for coverage and alignment discrepancies to identify potential mis-assemblies and larger variants. Finally (bottom row), Pilon uses reads and mate pairs which are anchored to the flanks of discrepant regions and gaps in the input genome to reassemble the area, attempting to fill in the true sequence including large insertions. The resulting output is an improved assembly and/or a VCF file of variants. (From {% cite Walker2014 %})")

### Annotation

For annotation we are using [Prokka](https://vicbioinformatics.com/software.prokka.shtml) (also see {% cite Seemann2014 %}). It scans the assembly generated with Unicycler with a set of feature prediction tools and compiles a list of genome annotation. It predicts the following features (Table from {% cite Seemann2014 %}):

| Feature | Tool used by Prokka |
|---------|----------------------|
| Protein-coding sequences (CDS) | [Prodigal](https://github.com/hyattpd/Prodigal) |
| Ribosomal RNA genes | [RNAmmer](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1888812) |
| Transfer RNA genes | [Aragorn](https://www.ncbi.nlm.nih.gov/pubmed/14704338) |
| Signal leader peptides | [SignalP](https://www.ncbi.nlm.nih.gov/pubmed/21959131) |
| Non-coding RNA genes | [Infernal](http://eddylab.org/infernal/) |

Prokka predicts protein-coding regions using a two step process. It first identifies coordinates of putative genes using [Prodigal](https://github.com/hyattpd/Prodigal) and then compares the gene sequence against databases of known sequences at protein level using [Blast+](https://www.ncbi.nlm.nih.gov/books/NBK279690/) and [HMMer](http://hmmer.org/).

# Let's try it


## Load data and assess quality

In this example we will use a downsampled version of *E. coli* C-1 Illumina and ONT sequencing data. These include 3 files: forward and reverse reads for Illumina, and Long read file produced by ONT. All data are in [fastq](https://en.wikipedia.org/wiki/FASTQ_format) format.

> <hands-on-title>Obtaining our data</hands-on-title>
>
> 1. Make sure you have an empty analysis history. Give it a name.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the following file from [Zenodo](https://zenodo.org/record/940733)
>
>    ```
>    https://zenodo.org/record/940733/files/illumina_f.fq
>    https://zenodo.org/record/940733/files/illumina_r.fq
>    https://zenodo.org/record/940733/files/minion_2d.fq
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
{: .hands_on}

If all goes well you will see datasets uploading and changing states from gray to green as shown below. The figure below also shows how datasets can be tagged.

![Datasets in History](../../images/starting_data.png  "Sequencing data loaded into Galaxy history. The full progression from gray (scheduling) to green (all OK) state is shown. To make it easier to identify datasets as we progress through the analysis we use so-called <em>Hashtags</em>. To tag a dataset: click on dataset to expand it (as shown in panel four); click the tag icon (<i class='fa fa-tags' aria-hidden='true'></i>) and a text field will appear. Add a tag (in this case <b>F</b>) pre-pended with hash (#). Hit enter. Do this for all three datasets and it will appear as in panel five.")

### Assess Read Quality

To assess quality we will use two tools: FastQC ({% cite FastQC %}) to generate quality statistics and multiQC ({% cite Ewels2016 %}) to summarize these statistics.

> <hands-on-title>Quality Control</hands-on-title>
>
> 1. **FastQC** {% icon tool %}:
>
>    - {% icon param-files %} *"Short read data from your current history"*: Select all three FastQ datasets simultaneously
>
> 2. **MultiQC** {% icon tool %}: to generate a summary of the FastQC reports with
>   - *"Which tool was used generate logs?"*: `FastQC`
>   - *"Type of FastQC output"*: `Raw data`
>   - {% icon param-files %} *"FastQC Output"*: `RawData` outputs of FastQC
>
{: .hands_on}

A quick look at quality score distribution will show a confusing picture:

![QC reported zoomed out](../../images/multiqc1.png "Because Illumina reads (green) are <b>much</b> shorter that ONT reads (red) the plot looks strange. ONT reads generally have low quality scores and so they are not really meaningful in the context of this technology. However, in the case of Illumina data they mean a lot...")

So let's zoom in into Illumina data:

![QC reported zoomed in](../../images/multiqc2.png "Zooming in shows quality distribution for Illumina reads. This is excellent data with mean base qualities above 30 across all reads.")

## Assembly with Unicycler

Now it is time to perform assembly.

> <hands-on-title>Unicycler Assembly</hands-on-title>
>
> 1. **Unicycler** {% icon tool %} with the following parameters :
>   - "Paired or Single end data?" to `Paired`
>   - "First Set of reads" to the forward reads file `f`
>   - "Second Set of reads" to the reverse reads file `r`
>   - "Long reads" to the MinION file
>   - Use default parameters
>
{: .hands_on}


> ### <i class="fa fa-cutlery" aria-hidden="true"></i> <i class="fa fa-coffee" aria-hidden="true"></i> Assembly takes time!
>
> There is no such thing as Assembly in real time. It takes time so it is a good time to have lunch or at least coffee. This Unicycler run will take anywhere between 90 minutes and two hours.
{: .warning}

## Assess Assembly quality with Quast

Quast ({% cite Gurevich2013 %}) is a tool providing quality metrics for assemblies, and can also be used to compare multiple assemblies. The tool can also take an optional reference file as input, and will provide complementary metrics.

> <hands-on-title>Assembly Quality</hands-on-title>
>
> 1. **Quast** {% icon tool %}: with the following parameters
>   - *"Contigs/scaffolds output file"*: Select the fasta file resulting from the Unicycler assembly.
>
{: .hands_on}

The Quast tool outputs assembly metrics as an html file with metrics and graphs. The image below looks exceptionally boring. This is a **good** thing:

![Quast Interface](../../images/quast_output.png  "Quast Output: Quast provides different statistics such as the number of contigs or scaffolds, the N50 and N75, and the total length of the assembly. You can also access 3 plots, the cumulative length of the contigs, the Nx, or the GC content.")

One can see that there are two (!) contigs. The largest contig is 4,576,290 bp (for comparison *E. coli* K12 MG1655 strain genome length is [4,656,144 bp](https://www.ncbi.nlm.nih.gov/nuccore/NZ_APIN00000000.1)) and the smallest is 4,581,676 (total length) - 4,576,290 (length of the largest) = 5,386 bp. When we analyzed this dataset for the first time we were initially puzzled by this second contig. But we quickly realized that this is simply the genome of bacteriophage [phiX174](https://www.ncbi.nlm.nih.gov/nuccore/NC_001422.1) which is routinely used as a spike-in in Illumina sequencing. Thus we have two genomes: the one of *E.coli* C-1 and phiX174! We can now use Prokka to annotate our two genomes.

## Annotation with Prokka


> <hands-on-title>Annotation</hands-on-title>
>
> 1. **Prokka** {% icon tool %}:
>   - *"Contigs to annotate"*: Select the assembly ouput of `Unicycler`
>   - *"Genus name"*: `Escherichia`
>   - *"Species name"*: `coli`
>   - *"Strain name"*: `C-1`
>   - *"Use genus-specific BLAST database"*: `yes`
>
{: .hands_on}

Prokka outputs 10 datasets (including two log files). These are in various formats:

* **txt** : Provides Statistics on the annotation : number of CDS predicted, number of rRNA etc.
* **tbl** : Provides a tabulated list of annotated features.
* **fsa** : Nucleotide fasta file of the input contig sequence.
* **sqn** : [ASN.1](https://www.ncbi.nlm.nih.gov/Structure/asn1.html) format file for submission to GenBank.
* **ffn** : Nucleotide FASTA file of all the prediction transcripts.
* **faa** : Protein FASTA file of the translated CDS sequences.
* **fna** : Nucleotide fasta file of the input contig sequence.
* **gbk** : [GenBank](https://www.ncbi.nlm.nih.gov/Sitemap/samplerecord.html) file.
* **gff** : [gff3](http://gmod.org/wiki/GFF3) file.

## Visualize the results in IGV

Let's look at the entire assembly and its annotation in the genome browser. We can do this using Integrated Genome Browser ([IGV](http://software.broadinstitute.org/software/igv/)).

Visualization requires a local installation of IGV. If you have IGV installed - just start it. If you don't - read on.

### Starting IGV

Go to IGV [download page](http://software.broadinstitute.org/software/igv/download) and select one of the options. The one I would try first would be **Java Web Start**. Simply click the **Launch** button for 10 GB distribution.


> <hands-on-title>Visualization in IGV</hands-on-title>
>
>
> 1. Start IGV. It will look something like this:
>
>      ![IGV just started](../../images/igv1.png "IGV started by using Java Web Start. Note that it is currently showing Human Genome (hg38) build. This is obviously <b>not</b> what we want.")
>
> 2. Locate the output of Unicycler and expand it :
>
>      ![Unicycler result Visualization](../../images/unicycler_igv.png  "Expanding Unicycler assembly reveals IGV link.")
>
> 3. Click on the *local* link highlighted with orange outline. The browser will change:
>
>      ![IGV with Unicycler assembly](../../images/igv2.png "Now you can see the major contig shown in the browser window.")
>
> 4. Let's add Prokka annotations to the browser image. For this simply expand Prokka's GFF3 dataset and click on the *local* link:
>
>      ![Expanded GFF3 dataset representing Prokka annotations](../../images/prokka_item.png "Expanded GFF dataset generated with Prokka. Click on the <em>local</em> link (highlighted with orange outline) to display this dataset within IGV.")
>
{: .hands_on}

You will now see the annotations within the browser window:


![Prokka result Visualization](../../images/igv3.png  "Viewing annotated assembly withing IGV.")


topic: assembly
tutorial: general-introduction
title: An Introduction to Genome Assembly
difficulty: introductory
time_estimation: 30m
requirements_json: []
tags_json: []
content_hash: 00ecc8da31dabd327e2164f1d40198fd
last_modified: 2026-05-27T09:01:47.802416
zenodo_link: https://doi.org/10.5281/zenodo.582600
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/general-introduction/tutorial.html

content:
# Genome assembly with Velvet: Background


Velvet is one of a number of *de novo* assemblers that use short read sets as input (*e.g.* Illumina Reads). The assembly method is based on the manipulation of de Bruijn graphs, via the removal of errors and the simplification of repeated regions.

> <comment-title></comment-title>
>
> For information about Velvet, you can check its (nice) [Wikipedia page](https://en.wikipedia.org/wiki/Velvet_assembler).
{: .comment}

For this tutorial, we have a set of reads from an imaginary *Staphylococcus aureus* bacterium with a miniature genome (197,394 bp). Our mutant strain read set was sequenced with the whole genome shotgun method, using an Illumina DNA sequencing instrument. From these reads, we would like to rebuild our imaginary *Staphylococcus aureus* bacterium via a *de novo* assembly of a short read set using the Velvet assembler.

We also have a sequence for a reference genome that we will use later in the tutorial.

> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Get the data

> <hands-on-title>Getting the data</hands-on-title>
>
> 1. Create and name a new history for this tutorial.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import from [Zenodo](https://doi.org/10.5281/zenodo.582600) or from the data library the files:
>
>    ```
>    https://zenodo.org/record/582600/files/mutant_R1.fastq
>    https://zenodo.org/record/582600/files/mutant_R2.fastq
>    https://zenodo.org/record/582600/files/wildtype.fna
>    ```
>
>    > <tip-title>Importing data via links</tip-title>
>    >
>    > * Copy the link location (Right-click on the filename then "Copy Link Address")
>    > * Open the Galaxy Upload Manager
>    > * Select **Paste/Fetch Data**
>    > * Paste the link into the text field
>    > * For the read files, change the data-type to **fastqsanger**
>    > * Press **Start**
>    {: .tip}
>
> 3. Change the name of the files to `mutant_R1`, `mutant_R2` and `wildtype.fna`.
>
>    As a default, Galaxy uses the link as the name of the new dataset. It also does not link the dataset to a database or a reference genome.
>
>    {% snippet faqs/galaxy/datasets_rename.md %}
>
> 4. Inspect the content of a file.
>
>    > <tip-title>Inspecting the content of a dataset</tip-title>
>    >
>    > * Click on the {% icon galaxy-eye %} (eye) icon next to the relevant history entry
>    > * View the content of the file in the central panel
>    {: .tip}
>
>    > <question-title></question-title>
>    >
>    > 1. What are four key features of a FASTQ file?
>    > 2. What is the main difference between a FASTQ and a FASTA file?
>    >
>    > > <solution-title></solution-title>
>    > > 1. Each sequence in a FASTQ file is represented by 4 lines: 1st line is the id, 2nd line is the sequence, 3rd line is not used, and 4th line is the quality of sequencing per nucleotide
>    > > 2. In a FASTQ file, not only are the sequences present, but information about the quality of sequencing is also included.
>    > {: .solution }
>    {: .question}
>
{: .hands_on}

The reads have been sequenced from an imaginary *Staphylococcus aureus* bacterium using an Illumina DNA sequencing instrument. We obtained the 2 files we imported (`mutant_R1` and `mutant_R2`)

> <question-title></question-title>
>
> Why do we have 2 files here if we only sequenced the bacteria once?
>
> > <solution-title></solution-title>
> > 1. The bacteria has been sequenced using paired-end sequencing. The first file corresponds to forward reads and the second file to reverse reads.
> {: .solution }
{: .question}

# Evaluate the input reads

Before doing any assembly, the first questions you should ask about your input reads include:

- What is the coverage of my genome?
- How good is my read set?
- Do I need to ask for a new sequencing run?
- Is it suitable for the analysis I need to do?

We will evaluate the input reads using the FastQC tool. This tool runs a standard series of tests on your read set and returns a relatively easy-to-interpret report. We will use it to evaluate the quality of our FASTQ files and combine the results with MultiQC.

> <hands-on-title>FastQC on a fastq file</hands-on-title>
>
> 1. {% tool [FastQC](toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.73+galaxy0) %} with the following parameters:
>    - {% icon param-files %} *"Raw read data from your current history"*:  `mutant_R1.fastq` and `mutant_R2.fastq`
>
>    {% snippet faqs/galaxy/tools_select_multiple_datasets.md %}
>  
> 2. {% tool [MultiQC](toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy1) %} with the following parameters:
>    - *"Results: Which tool was used to generate logs?"*: `FastQC`
>    - *Click "Insert FastQC output"*
>    - *"Type of FastQC output?"*: `multiple datasets, select the raw data files from FastQC`
{: .hands_on}

MultiQC generates a webpage combining reports for FastQC on both datasets. It includes these graphs and tables:

- General statistics

    We need to know about the data for our analysis. In particular, we need to know the read lengths as it is important in setting the maximum k-mer size for an assembly.

    > <comment-title>Getting the length of sequences</comment-title>
    >
    > * Find the MultiQC output that is a webpage and click to view
    > * The first table shows General Statistics for the input read files
    > * At the top of this table, click on **Configure Columns**
    > * Make sure the box next to **Length** is ticked
    > * Close the window
    > * This table should now show a column for read lengths
    {: .comment}

    > <question-title></question-title>
    >
    > 1. How long are the sequences?
    > 2. What is the average coverage of the genome, given our imaginary *Staphylococcus aureus* bacterium has a genome of 197,394 bp?
    >
    > > <solution-title></solution-title>
    > > 1. The sequences are 150 bp long
    > > 2. We have 2 x 12,480 sequences of 150 bp, so the average genome coverage is: 2 * 12480 * 150 / 197394, or approximately 19 X coverage.
    > {: .solution }
    {: .question}

- Sequence Quality Histograms

    Dips in quality near the beginning, middle or end of the reads may determine the trimming/cleanup methods and parameters to be used, or may indicate technical problems with the sequencing process/machine run.

    ![Sequence Quality Histograms with the mean quality value across each base position in the read](../../images/fastqc_per_base_sequence_quality_plot.png "The mean quality value across each base position in the read")

    > <question-title></question-title>
    >
    > 1. What does the y-axis represent?
    > 2. Why is the quality score decreasing across the length of the reads?
    >
    > > <solution-title></solution-title>
    > > 1. The y-axis represents the quality score for each base (an estimate of the error during sequencing).
    > > 2. The quality score is decreasing accross the length of the reads because the sequencing become less and less reliable at the end of the reads.
    > {: .solution }
    {: .question}

- Per Sequence GC Content

    High GC organisms tend not to assemble well and may have an uneven read coverage distribution.

- Per Base N Content

    The presence of large numbers of Ns in reads may point to a poor quality sequencing run. You will need to trim these reads to remove Ns.

- k-mer content

    The presence of highly recurring k-mers may point to contamination of reads with barcodes or adapter sequences.


> <comment-title></comment-title>
>
> For a fuller discussion of FastQC outputs and warnings, see the [FastQC website link](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/), including the section on each of the output [reports](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/3%20Analysis%20Modules/), and examples of ["good"](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/good_sequence_short_fastqc.html) and ["bad"](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/bad_sequence_fastqc.html) Illumina data.
{: .comment}

We won't be doing anything to these data to clean it up as there isn't much need. Therefore we will get on with the assembly!


# Assemble reads with Velvet

Now, we want to assemble our reads to find the sequence of our imaginary *Staphylococcus aureus* bacterium. We will perform a *de novo* assembly of the reads into long contiguous sequences using the Velvet short read assembler.

The first step of the assembler is to build a de Bruijn graph. For that, it will break our reads into k-mers, *i.e.* fragments of length *k*. Velvet requires the user to input a value of *k* (k-mer size) for the assembly process. Small k-mers will give greater connectivity, but large k-mers will give better specificity.

> <hands-on-title>Assemble the reads</hands-on-title>
>
> 1. {% tool [FASTQ interlacer](toolshed.g2.bx.psu.edu/repos/devteam/fastq_paired_end_interlacer/fastq_paired_end_interlacer/1.2.0.1+galaxy0) %} with the following parameters:
>    - *"Type of paired-end datasets"*: `2 separate datasets`
>    - *"Left-hand mates"*: `mutant_R1.fastq`
>    - *"Right-hand mates"*: `mutant_R2.fastq`
>
>    Currently our paired-end reads are in 2 files (one with the forward reads and one with the reverse reads), but Velvet requires only one file, where each read is next to its mate read. In other words, if the reads are indexed from 0, then reads 0 and 1 are paired, 2 and 3, 4 and 5, etc. Before doing the assembly *per se*, we need to prepare the files by combining them.
>
> 2. {% tool [velveth](toolshed.g2.bx.psu.edu/repos/devteam/velvet/velveth/1.2.10.3) %} with the following parameters:
>    - *"Hash Length"*: `29`
>    - *"Input Files"*
>       - Click on {% icon param-repeat %} *"Input Files"*
>       - In "1: Input Files"
>          - *"Choose the input type"*: `interleaved paired end`
>          - *"read type"*: `shortPaired reads`
>          - {% icon param-files %} *"Dataset"*:  pairs output of **FASTQ interlacer** 
>
>    The tool takes our reads and break them into k-mers.
>
> 3. {% tool [velvetg](toolshed.g2.bx.psu.edu/repos/devteam/velvet/velvetg/1.2.10.2) %} with the following parameters:
>    - {% icon param-files %} *"Velvet Dataset"*:  outputs of **velveth**
>    - *"Using Paired Reads"*: `Yes`
>
>    This last tool actually does the assembly.
{: .hands_on}

Five files are generated. We will look at the contigs file and the stats file:

- The **Contigs** file

    This file contains the sequences of the contigs. In the header of each contig, a bit of information is added:
    - the k-mer length (called "length"): For the value of k chosen in the assembly, a measure of how many k-mers overlap (by 1 bp each overlap) to give this length
    - the k-mer coverage (called "coverage"): For the value of k chosen in the assembly, a measure of how many k-mers overlap each base position (in the assembly).
    - Note that your results may look different to the example in the image below. 

    ![Contigs output](../../images/image10.png)

- The **Stats** file

    This is a tabular file giving for each contig the k-mer lengths, k-mer coverages and other measures.  Note that your results may look different to the example in the image below. 

    ![Contigs stats output](../../images/image11.png)

# Collect some statistics on the contigs

> <question-title></question-title>
>
> 1. How many contigs have been built?
> 2. What is the mean, min and max length of the contigs?
>
> > <solution-title></solution-title>
> > 1. 306
> > 2. To compute this information, we can use the Datamash tool on the 2nd columns (length). Be careful with the first line, the header. As a result, we obtain approximately: 516 as mean, 1 as min and 8836 as max. It would mean that the smallest contig has a length of 1 bp, even smaller than k. The length on the 2nd column corresponds to length of the contig in k-mers. This means that the smallest contig has a length of 1k = 29. So to obtain the real length, we need to add k-1 to the length. We then obtain a mean contig length of 544 bp, a min contig of 29 bp and a max contig of 8864 bp.
> {: .solution }
{: .question}

This table is limitted, but we will now collect more basic statistics on our assembly.

> <hands-on-title>Collect fasta statistics on our contigs</hands-on-title>
>
> 1. {% tool [Quast](toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1) %} with the following parameters:
>    - *"Assembly mode"*: `Individual assembly (1 contig file per sample)`
>    - *"Use customized names?"*: `No`
>    - *"Contigs/scaffolds file"*: contigs output of **velvetg**
>    - *"Type of assembly"*: `Genome`
>    - *"Use a reference genome?"*: `Yes`
>    - *"Reference genome"*: `wildtype.fna`
>    - *"Type of organism"*: `Prokaryotes`
>    - *"Lower Threshold"*: `500`
>    - *"Advanced options: Comma-separated list of contig length thresholds"*: `0,1000`
{: .hands_on}


> <question-title></question-title>
>
> 1. What is represented in the Icarus viewer?
>
> > <solution-title></solution-title>
> > 1. Icarus is a novel genome visualizer for accurate assessment and analysis of genomic draft assemblies. It draws contigs ordered from longest to shortest, highlights N50, N75 (NG50, NG75) and long contigs larger than a user-specified threshold
> {: .solution }
{: .question}

The HTML report reports many statistics computed by QUAST to assess the quality of the assembly:

- Statistics about the quality of the assembly when compared to the reference (fraction of the genome, duplication ratio, etc)
- Misassembly statistics, including the number of misassemblies

    A misassembly is a position in the contigs (breakpoints) that satisfy one of the following criteria:
    - the left flanking sequence aligns over 1 kbp away from the right flanking sequence on the reference;
    - flanking sequences overlap on more than 1 kbp
    - flanking sequences align to different strands or different chromosomes

- Unaligned regions in the assembly
- Mismatches compared to the reference genomes
- Statistics about the assembly *per se*, such as the number of contigs and the length of the largest contig

> <question-title></question-title>
>
> 1. How many contigs have been constructed?
> 2. Which proportion of the reference genome do they represent?
> 3. How many misassemblies have been found?
> 4. Has the assembly introduced mismatches and indels?
> 5. What are N50 and L50?
> 6. Is there a bias in GC percentage induced by the assembly?
>
> > <solution-title></solution-title>
> > 1. 190 contigs have been constructed, but only 47 have a length > 500 bp.
> > 2. The contigs represents 87.965% of the reference genome.
> > 3. 1 misassembly has been found: it corresponds to a relocation, *i.e.* a misassembly event (breakpoint) where the left flanking sequence aligns over 1 kbp away from the right flanking sequence on the reference genome.
> > 4. 8.06 mismatches per 100 kbp and 4.03 indels per 100 kbp are found.
> > 5. N50 is the length for which the collection of all contigs of that length or longer covers at least half an assembly. In other words, if contigs were ordered from small to large, half of all the nucleotides will be in contigs this size or larger. And L50 is the number of contigs equal to or longer than N50: L50 is the minimal number of contigs that cover half the assembly.
> > 6. The GC % in the assembly is 33.64%, really similar to the one of the reference genome (33.43%).
> {: .solution }
{: .question}

# Discussion

> <hands-on-title>(Optional) Rerun for values <i>k</i> ranging from 31 to 101</hands-on-title>
>
> 1. {% tool [velveth](toolshed.g2.bx.psu.edu/repos/devteam/velvet/velveth/1.2.10.3) %} with the same parameters as before except
>    - *"Hash Length"*: a value between 31 and 101
> 2. {% tool [velvetg](toolshed.g2.bx.psu.edu/repos/devteam/velvet/velvetg/1.2.10.2) %} with the same parameters as before
> 3. {% tool [Quast](toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1) %} with the same parameters as before
{: .hands_on}

We have completed an assembly on this data set for a number of k values ranging from 29 to 101. A few of the assembly metrics appear below.

![contigs](../../images/number_of_contigs.png "Number of contigs in the assembly for various k-mer sizes")

![largest_contig](../../images/largest_contig.png "Largest contig in each of the assemblies by k-mer size")

![total_bp](../../images/total_bp.png "Total number of base pairs in all the contigs for each assembly by k-mer size")

![n50](../../images/n50.png "N50 metric for each of the assemblies by k-mer size")

> <question-title></question-title>
>
> 1. Are there any distinct features in the charts?
> 2. Does it look like one assembly might be better than some of the others?
>
{: .question}

The reasons for these patterns will be discussed in detail in the De Bruijn graph assembly slides and tutorial.

tools_json:
["FastQC", "toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.73+galaxy0", "MultiQC", "toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.11+galaxy1", "FASTQ interlacer", "toolshed.g2.bx.psu.edu/repos/devteam/fastq_paired_end_interlacer/fastq_paired_end_interlacer/1.2.0.1+galaxy0", "velveth", "toolshed.g2.bx.psu.edu/repos/devteam/velvet/velveth/1.2.10.3", "velvetg", "toolshed.g2.bx.psu.edu/repos/devteam/velvet/velvetg/1.2.10.2", "Quast", "toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1"]


topic: assembly
tutorial: ecoli_comparison
title: Making sense of a newly assembled genome
difficulty: intermediate
time_estimation: 4h
requirements_json: []
tags_json: ["gmod", "prokaryote", "microgalaxy", "jbrowse1"]
content_hash: b8794e62f5e6f75bd163cb4c5a778e9d
last_modified: 2026-05-27T09:01:47.802312
zenodo_link: https://zenodo.org/record/3382053
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/ecoli_comparison/tutorial.html

content:
In this tutorial we begin with a new genome assembly just produced in the [Unicycler tutorial]({{site.baseurl}}/topics/assembly/tutorials/unicycler-assembly/tutorial.html). This is an assembly of *E. coli* C, which we will be comparing to assemblies of all other complete genes of this species.

> <agenda-title></agenda-title>
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Finding closely related genomes

[*E. coli*](https://en.wikipedia.org/wiki/Escherichia_coli) is one of the most studied organisms. There are thousands of complete genomes (in fact, the total number of *E. coli* assemblies in Genbank is over 10,500). Here we will shows how to uploaded all (!) complete *E. coli* genomes at once.

> <comment-title>Slow steps ahead</comment-title>
> The first part of this tutorial can take a significant amount of time to find the most related genomes. If you want, you can upload this (outdated) copy of the NCBI E. Coli Genomes table to your history:
>
> 1. Import the following URL:
>
>    ```
>    https://zenodo.org/record/3382053/files/genomes_proks.txt
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 2. And skip ahead to [comparing the most related genomes](#comparing-genome-architectures).
{: .comment}


## Getting complete *E. coli* genomes into Galaxy

Our initial objective is to compare our assembly against all complete *E. coli* genomes to identify the most related ones and to find any interesting genome alterations. In order to do this we need to align our assembly against all other genomes. And in order to do that we need to first obtain all these other genomes.

[NCBI](https://www.ncbi.nlm.nih.gov/) is the resource that would store [all complete *E. coli* genomes](https://www.ncbi.nlm.nih.gov/genome/genomes/167). This list contains over 500 genomes and so uploading them by hand will likely result in carpal tunnel syndrome, which we want to prevent. Galaxy has several features that are specifically designed for uploading and managing large sets of similar types of data. The following two **Hands-on** sections show how they can be used to import all completed *E. coli* genomes into Galaxy.

> <hands-on-title>Preparing a list of all complete <i>E. coli</i> genomes</hands-on-title>
>
> 1. Import the genome list from Zenodo:
>
>    ```
>    https://zenodo.org/record/3382053/files/genomes_proks.txt
>    ```
>
{: .hands_on}


> <details-title>Getting the data directly from NCBI</details-title>
>
> For this tutorial we made this dataset available from Zenodo, but it is of course also possible to obtain the data directly from NCBI.
> Note that the format of the files on NCBI may change, which means some of the parameter settings of tools in this tutorial will need
> to be altered (e.g. column numbers) when using data directly from NCBI.
>
> Below we describe how you could obtain this data from NCBI.
>
> 1. Open [the NCBI list of of *E. coli* genomes](https://www.ncbi.nlm.nih.gov/genome/genomes/167) in a new window
>
> 2. Click on "Filters" at the top right:
>
>    ![Filter menu button](../../images/ecoli-list-filter.png)
>
> 3. Select only the "Complete" genomes with the filter at the top
>
>    ![Filter settings, only "complete" is checked](../../images/ecoli-list.png)
>
> 3. At the top right, click "Download"
>
> 4. Upload this table to Galaxy
>
> 5. As this file is a CSV file, we need to convert it to TSV before Galaxy can use it.
>
>    {% snippet faqs/galaxy/datasets_convert_datatype.md conversion="Convert CSV to Tabular" %}
>
> 6. Rename this file to `genomes.tsv`
>
>    {% snippet faqs/galaxy/datasets_convert_datatype.md conversion="Convert CSV to Tabular" %}
>
{: .details}


> <hands-on-title>Preparing a list of all complete <i>E. coli</i> genomes</hands-on-title>
>
> 1. {% tool [Cut](Cut1) %}   columns from a table:
>
>    - *"Cut columns"*: `c8,c20`
>    - *"From"*: `genome_proks.txt`
>
> > <question-title></question-title>
> >
> > How does your data look?
> >
> > > <solution-title></solution-title>
> > >
> > > It should look like this:
> > > 1               | 2
> > > --              | --
> > > GCA_000005845.2 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/005/845/GCA_000005845.2_ASM584v2
> > > GCA_000008865.2 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/008/865/GCA_000008865.2_ASM886v2
> > > GCA_003697165.2 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/003/697/165/GCA_003697165.2_ASM369716v2
> > > GCA_003018455.1 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/003/018/455/GCA_003018455.1_ASM301845v1
> > > GCA_001650295.1 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/001/650/295/GCA_001650295.1_ASM165029v1
> > > GCA_003018035.1 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/003/018/035/GCA_003018035.1_ASM301803v1
> > > GCA_003112225.1 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/003/112/225/GCA_003112225.1_ASM311222v1
> > > GCA_001695515.1 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/001/695/515/GCA_001695515.1_ASM169551v1
> > > GCA_001721125.1 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/001/721/125/GCA_001721125.1_ASM172112v1
> > > GCA_000091005.1 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/091/005/GCA_000091005.1_ASM9100v1
> > > GCA_005037725.2 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/005/037/725/GCA_005037725.2_ASM503772v2
> > > GCA_005037815.2 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/005/037/815/GCA_005037815.2_ASM503781v2
> > > GCA_004358405.1 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/004/358/405/GCA_004358405.1_ASM435840v1
> > > GCA_003018575.1 | ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/003/018/575/GCA_003018575.1_ASM301857v1
> > >
> > {: .solution}
> {: .question}
>
>
{: .hands_on}

Now that the list is formatted as a table in a spreadsheet, it is time to upload it into Galaxy. There is a problem though: the URLs (web addresses) in the list do not actually point to sequence files that we would need to perform alignments. Instead they point to directories. For example, this URL: [GCA_000008865.1_ASM886v1](ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/008/865/GCA_000008865.1_ASM886v1) points to a directory (rather than a file) containing many files, most of which we do not need.

![GenBank assembly files for an E. coli strain](../../images/genbank_dir.png "A list of files for an <i>E. coli</i> assembly. For further analyses we only need the dataset ending with <code>_genomic.fna.gz</code>.")

So to download sequence files we need to edit URLs by adding filenames to them. For example, in the case of the URL shown above we need to add `/GCA_000008865.1_ASM886v1` and `_genomic.fna.gz` to the end to get this:

```
ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/008/865/GCA_000008865.1_ASM886v1/GCA_000008865.1_ASM886v1_genomic.fna.gz
```

This can be done as a two step process where we first copy the end part of the existing URL (`/GCA_000008865.1_ASM886v1`) and then add a fixed string `_genomic.fna.gz` to the end of it. Doing this by hand is crazy and trying to do it in a spreadsheet is complicated. Fortunately, Galaxy's new rule-based uploader can help, as shown in the next **Hands-on** section:

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Again **Upload** {% icon tool %} data
>
> 2. Switch to the `Rule-based` tab on the right
>
>    > <tip-title>Using the Rule-based Uploader</tip-title>
>    > There is a detailed tutorial on using the [Rule based Uploader]({% link topics/galaxy-interface/tutorials/upload-rules/tutorial.md %}) if you want to learn about the more advanced features available.
>    {: .tip}
>
>    - *"Upload data as"*: `Collection(s)`
>    - *"Load tabular data from"*: `History Dataset`
>    - *"Select dataset to load"*: output of the cut tool
>
>    > <tip-title>dataset not there?</tip-title>
>    > If the dataset doesn't appear in the select list, refresh your page.
>    {: .tip}
>
> 3. From **Column**, select `Using a Regular Expression`
>    - *"From Column"*: `B`
>    - Select `Create columns matching expression groups`
>    - *"Regular Expression"*: `.*(\/GCA.*$)`
>    - *"Number of Groups"*: `1`
>    - Click `Apply`
> 4. From **Column**, select `Concatenate Columns`
>    - *"From Column"*: `B`
>    - *"From Column"*: `C`
>    - Click `Apply`
> 6. From **Column**, select `Fixed Value`
>    - *"Value"*: `_genomic.fna.gz`
>    - Click `Apply`
> 7. From **Column**, select `Concatenate Columns`
>    - *"From Column"*: `D`
>    - *"From Column"*: `E`
>    - Click `Apply`
> 9. From **Rules** menu, select `Add / Modify Column Definitions`
>    - `Add Definition`, `List Identifier(s)`, Select Column `A`
>    - `Add Definition`, `URL`, Column `F`
>    - Click `Apply`
> 10. Set the **Type** in the bottom left to `fasta.gz`
> 11. Give the upload a name like `Complete genomes`
> 12. **Upload**
>
>
{: .hands_on}

Now we have all complete *E. coli* genomes in Galaxy's history. It is time to do a few things to our assembly.

## Preparing assembly


Before starting any analyses we need to upload the assembly produced in [Unicycler tutorial]({{site.baseurl}}/topics/assembly/tutorials/unicycler-assembly/tutorial.html) from Zenodo:

> <hands-on-title>Uploading <i>E. coli</i> assembly into Galaxy</hands-on-title>
>
> 1. {% tool [Upload](upload1) %}  :
>   - Click **Paste/Fetch data** button (Bottom of the interface box)
>   - **Paste** `https://zenodo.org/record/1306128/files/Ecoli_C_assembly.fna` into the box.
>   - *"Type"*: `fasta`
>   - Click **Start**
{: .hands_on}

> <tip-title>Finding tools mentioned in this tutorial</tip-title>
>Galaxy instances contain hundreds of tools. As a result, it can be hard to find tools mentioned in tutorials such as this one. To help with this challenge, Galaxy has a search box at the top of the left panel. Use this box to find the tools mentioned here.
>![Tool search](../../images/tool_search.png "Use search box to find tools!")
{: .tip}

The assembly we just uploaded has two issues that need to be addressed before proceeding with our analysis:

1. It contains two sequences: the one of *E. coli* C genome (the one we really need) and another representing phage phiX174 (a by product of Illumina sequencing where it is used as a spike-in DNA).
2. Sequences have unwieldy names like `>1 length=4576293 depth=1.00x circular=true`. We need to rename it to something more meaningful.

Let's fix these two problems.

Because phiX173 is around 5,000bp, we can remove those sequences by setting a minimum length of 10,000:

> <hands-on-title>Fixing assembly</hands-on-title>
>
> 1. {% tool [Filter sequences by length](toolshed.g2.bx.psu.edu/repos/devteam/fasta_filter_by_length/fasta_filter_by_length/1.2) %} with the following parameters:
>   - *"Fasta file"*: the dataset you've just uploaded. (`https://zenodo.org/record/1306128/files/Ecoli_C_assembly.fna`).
>   - *"Minimal length"*: `10000`
>
>
> 2. {% tool [Replace Text](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_line/1.1.2) %} in entire line:
>    - *"File to process"*: the output of the Filter sequences by length {% icon tool %}
>    - *"1: Replacement"*
>      - *"Find Pattern"*: `^>1.*`
>      - *"Replace with"*: `>Ecoli_C`
>
>    {% snippet faqs/galaxy/datasets_rename.md name="E. coli C" %}
>
>    > <tip-title>Regular Expressions</tip-title>
>    >
>    > The program we just entered is a so-called [Regular Expression](https://en.wikipedia.org/wiki/Regular_expression)
>    >
>    > The expression `^>1.*` contains several pieces that you need to understand. Let's write it top-to-bottom and explain:
>    >
>    > - `^` - says start looking at *the beginning* of each line
>    > - `>` - is the first character we want to match. Remember that name of the sequence in FASTA files starts with `>`
>    > - `1` - is the number present is our old name (`>1 length=4576293 depth=1.00x circular=true` to `>Ecoli_C`)
>    > - `.` - dot has a special meaning. It signifies *any* character
>    > - `*` - is a *quantifier*. From [Wikipedia](https://en.wikipedia.org/wiki/Regular_expression): "The asterisk indicates zero or more occurrences of the preceding element. For example, ab*c matches `ac`, `abc`, `abbc`, `abbbc`, and so on."
>    >
>    >So in short we are replacing `>1 length=4576293 depth=1.00x circular=true` with `>Ecoli_C`. The *Regular expression* `^>1.*` is used here to represent `>1 length=4576293 depth=1.00x circular=true`.<br>
>    >Detailed description of regular expressions is outside of the scope of this tutorial, but there are other great resources. Start with [Software Carpentry Regular Expressions tutorial](http://v4.software-carpentry.org/regexp/index.html).
>    {: .tip}
>
> > <question-title></question-title>
> >
> > 1. What is the meaning of `^` character is SED expression?
> >
> > > <solution-title></solution-title>
> > >
> > > 1. It tells SED to start matching from the beginning of the string.
> > >
> > {: .solution}
> {: .question}
>
{: .hands_on}

## Generating alignments


Now everything is loaded and ready to go. We will now align our assembly against each of the *E. coli* genomes we have uploaded into the collection. To do this we will use [LASTZ](https://lastz.github.io/lastz/)&mdash;an aligner designed for long sequences.

> <hands-on-title>Running LASTZ</hands-on-title>
> 1. {% tool [LASTZ](toolshed.g2.bx.psu.edu/repos/devteam/lastz/lastz_wrapper_2/1.3.2) %} with the following parameters:
>   - *"Select TARGET sequence(s) to align against"*: `from your history`
>   - {% icon param-collection %} *"Select a reference dataset"*: the "Complete genomes" collection we uploaded earlier
>   - {% icon param-file %} *"Select QUERY sequence(s)"*: our *E. coli* assembly which was prepared in the previous step.
>   - **Chaining**
>        - *"Perform chaining of HSPs with no penalties"*: `Yes`
>   - **Output**
>        - *"Specify the output format"*: `blastn`
>
{: .hands_on}

Note that because we started LASTZ on *a collection* of *E. coli* genomes, it will output alignment information as *a collection* as well. A collection is simply a way to represent large sets of similar data in a compact way within Galaxy's interface.

> <warning-title>It will take a while!</warning-title>
> Please understand that alignment is not an instantaneous process: allow several hours for these jobs to clear.
{: .warning}

## Finding closely related assemblies


### Understanding LASTZ output

LASTZ produced data in so-called `blastn` format (because we explicitly told LASTZ to output in this format, see previous step), which looks like this:

```
      1          2     3   4  5 6       7       8    9   10      11    12
-------------------------------------------------------------------------
Ecoli_C BA000007.2 66.81 232 51 6 3668174 3668397 5936 6149 3.2e-40 162.7
Ecoli_C BA000007.2 57.77 206 38 8  643802  643962 5945 6146 1.6e-18  90.6
Ecoli_C BA000007.2 67.03 185 32 6 4849373 4849528 5965 6149 2.9e-28 122.9
Ecoli_C BA000007.2 63.06 157 33 3 1874604 1874735 5991 6147 5.8e-26 115.3
```

where columns are:

1. `qseqid` - query (e.g., gene) sequence id
2. `sseqid` - subject (e.g., reference genome) sequence id
3. `pident` - percentage of identical matches
4. `length` - alignment length
5. `mismatch` - number of mismatches
6. `gapopen` - number of gap openings
7. `qstart` - start of alignment in query
8. `qend` - end of alignment in query
9. `sstart` - start of alignment in subject
10. `send` - end of alignment in subject
11. `evalue` - [expect value](https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&PAGE_TYPE=BlastDocs&DOC_TYPE=FAQ#expect)
12. `bitscore` - [bit score](https://www.ncbi.nlm.nih.gov/BLAST/tutorial/Altschul-1.html)

The alignment information produced by LASTZ is a collection. In this collection each element contains alignment data between each of the *E. coli* genomes and our assembly:

![LASTZ collection](../../images/lastz_collection.png "LASTZ produced a collection where each element corresponds to an alignment between an <i>E. coli</i> genome and our assembly. Here one of the elements is expanded (to expand an element simply click on it).").

## Collapsing collection

Collections are a wonderful way to organize large sets of data and parallelize data processing like we did here with LASTZ. However, at this point we need to combine all data into one dataset. Follow the steps below to accomplish this:

> <hands-on-title>Combining collection into a single dataset</hands-on-title>
> 1. {% tool [Collapse Collection](toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/4.2) %} with the following parameters:
>   - *"Collection of files to collapse"*: the output of **LASTZ** (collecion input)
{: .hands_on}

This will produce one gigantic table (over 12 million lines) containing combined LASTZ output for all genomes.

## Getting taste of the alignment data

To make further analyses we need to get an idea about alignment data generated with LASTZ. To do this let's select a random subsample of the large dataset we've generated above. This is necessary because processing the entire dataset will take time and will not give us a better insight anyway. So first we will select 10,000 lines from the alignment data:

> <hands-on-title>Selecting random subset of data</hands-on-title>
> 1. {% tool [Select random lines from a file](random_lines1) %}   with the following parameters:
>   -  *"Randomly select"*: `10000`
>   -  *"from"*: the output from `Collapse Collection`
{: .hands_on}

Now we can visualize this dataset to discover generalities:

> <hands-on-title>Graphing alignment data</hands-on-title>
> 1. Expand random subset of alignment data generated on the previous step by clicking on it.
> 2. You will see "chart" button {% icon galaxy-barchart %}. Click on it.
> 3. In the central panel you will see a list of visualizations. Select **Scatter plot (NVD3)**
> 4. Click **Select data** {% icon galaxy-chart-select-data %}
> 5. Set **Values for x-axis** to `Column: 3` (alignment identity)
> 6. Set **Values for y-axis** to `Column: 4` (alignment length)
> 7. You can also click on configuration button {% icon galaxy-gear %} and specify axis labels etc.
{: .hands_on}

The relationship between the alignment identity and alignment length looks like this (remember that this is only a subsample of the data):

![Identity versus length](../../images/id_vs_len.png "Alignment identity (%) versus length (bp). This graph is truncated at the top")

You can see that most alignments are short and have relatively low identity. Thus we can filter the original dataset by identity and length. Judging from this graph we can select alignment longer than 10,000 bp with identity above 90%.

> <hands-on-title>Filtering data</hands-on-title>
> 1. {% tool [Filter](Filter1) %} data on any column using simple expressions:
>   - *"Filter"*: the full dataset, from the output of the **Collapse Collection** {% icon tool %}.
>   - *"With following condition"*: `c3 >= 90 and c4 >= 10000` (here `c` stands for *column*).
>
>  NOTE: You need to select the full dataset; not the down-sampled one, but [the one generated by the collection collapsing operation](#hands-on-combining-collection-into-a-single-dataset).
>
{: .hands_on}

## Aggregating data

Remember, our objective is to find the genomes that are most similar to ours. Given the alignment data in the table we just created we can define similarity as follows:

> Genomes that have the smallest number of alignment blocks but the highest overall alignment length are most similar to our assembly. This essentially means that they have longest uninterrupted region of high similarity to our assembly.
{: .quote}

However, to extract this information from our data we need to aggregate it. In other words, for each *E. coli* genome we need to calculate the total number of alignment blocks, their combined length, and average identity. The following section explains how to do this:

> <hands-on-title>Aggregating the data</hands-on-title>
> 1. {% tool [Datamash (operations on tabular data)](toolshed.g2.bx.psu.edu/repos/iuc/datamash_ops/datamash_ops/1.1.0) %}   with the following parameters:
>     - *"Input tabular dataset"*: output of the previous `Filter` step.
>     - *"Group by fields"*: `2`. (column 1 contains name of the *E. coli* genome we mapped against)
>     - *"Sort input"*: `Yes`
>     - *"Operation to perform on each group"*:
>         - *"Type"*: `Count`
>         - *"On column"*: `Column: 2`
>     - {% icon param-repeat %} *"Insert operation to perform on each group"*
>       - *"Operation to perform on each group"*:
>           - *"Type"*: `Mean`
>           - *"On column"*: `Column: 3`.
>     - {% icon param-repeat %} *"Insert operation to perform on each group"*
>       - *"Operation to perform on each group"*:
>           - *"Type"*: `Sum`
>           - *"On column"*: `Column: 4`
{: .hands_on}

## Finding closest relatives

The dataset generated above lists each *E. coli* genome accession only once and will have aggregate information for the number of alignment blocks, mean identity, and total length. Let's graph these data:

> <hands-on-title>Graphing aggregated data</hands-on-title>
> 1. Expand the aggregated data generated on the previous step by clicking on it.
> 2. You will see "chart" button {% icon galaxy-barchart %}. Click on it.
> 3. In the central panel you will see a list of visualizations. Select **Scatter plot (NVD3)**
> 4. Click **Select data** {% icon galaxy-chart-select-data %}
> 5. Set **Data point labels** to `Column: 1` (Accession number of each *E. coli* genome)
> 5. Set **Values for x-axis** to `Column: 2` (# of alignment blocks)
> 6. Set **Values for y-axis** to `Column: 4` (Total alignment length)
> 7. You can also click on configuration button {% icon galaxy-gear %} and specify axis labels etc.
{: .hands_on}

The relationship between the number of alignment blocks and total alignment length looks like this:

![Identity versus length](../../images/best_genomes_chart.png "Number of alignment blocks versus total alignment length (bp).")

A group of three dots in the upper left corner of this scatter plot represents genomes that are most similar to our assembly: they have a SMALL number of alignment blocks but HIGH total alignment length. Mousing over these three dots (if you set **Data point labels** correctly in the previous step) will reveal their accession numbers: `LT906474.1`, `CP024090.1`, and `CP020543.1`.

> <warning-title>Things change</warning-title>
> It is possible that when you repeat these steps the set of sequences in NCBI will have changed and you will obtain different accession numbers. Keep this in mind.
{: .warning}

Let's find table entries corresponding to these:

> <hands-on-title>Extracting into about best hits</hands-on-title>
> 1. {% tool [Select lines that match an expression](Grep1) %} with the following parameters:
>   - *"Select lines from"*: to the output from `Datamash`
>   - *"the pattern"*: `LT906474|CP024090|CP020543`. (Here `|` means `or`).
{: .hands_on}

This will generate a short table like this:


CP020543.1 | 11 | 99.926363636364 | 4486976
CP024090.1 | 12 | 99.911666666667 | 4540487
LT906474.1 | 8  | 99.94           | 4575200

From this it appears that `LT906474.1` is closest to our assembly because it has eight alignment blocks, the longest total alignment length (4,575,223) and highest mean identity (99.94%).

# Comparing genome architectures

Now that we know the three genomes most closely related to ours, let's take a closer look at them. First we will re-download sequence and annotation data.

## Getting sequences and annotations

> <hands-on-title>Uploading sequences and annotations</hands-on-title>
> Using the three accession listed above we will fetch necessary data from NCBI. We will use the spreadsheet we uploaded at the start to accomplish this.
>
> 1. {% tool [Upload](upload1) %} the *E. coli* C genome if you have not done so already:
>   - Click **Paste/Fetch data** button (Bottom of the interface box)
>   - **Paste** `https://zenodo.org/record/1306128/files/Ecoli_C_assembly.fna` into the box.
>   - *"Type"*: `fasta`
>   - Click **Start**
>
> 1. {% tool [Select lines that match an expression](Grep1) %} with the following parameters:
>   - *"Select lines from"*: the `genomes.tsv` you uploaded earlier
>   - *"the pattern"*: `LT906474|CP024090|CP020543`
>
> 2. {% tool [Cut](Cut1) %} columns from a table:
>
>    - *"Cut columns"*: `c8,c20`
>    - *"From"*: the output of the **select lines** {% icon tool %}
>
>      It should look like:
>
>      ```
>      GCA_002079225.1	ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/079/225/GCA_002079225.1_ASM207922v1
>      GCA_002761835.1	ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/761/835/GCA_002761835.1_ASM276183v1
>      GCA_900186905.1	ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/900/186/905/GCA_900186905.1_49923_G01
>      ```
>
> 1. Again **Upload** {% icon tool %} data
>
>    2. Switch to the `Rule-based` tab on the right
>
>       - *"Upload data as"*: `Collection(s)`
>       - *"Load tabular data from"*: `History Dataset`
>       - *"Select dataset to load"*: output of the cut tool
>
>       > <tip-title>dataset not there?</tip-title>
>       > If the dataset doesn't appear in the select list, refresh your page.
>       {: .tip}
>
>       > <tip-title>Take a Shortcut</tip-title>
>       > This step is quite long and potentially error prone. If you want to skip those steps, you can copy and paste this bit of text:
>       >
>       > ```json
>       > {"rules":[{"type":"add_column_regex","target_column":1,"expression":".*(\\/GCA.*$)","group_count":1},{"type":"add_column_concatenate","target_column_0":1,"target_column_1":2},{"type":"remove_columns","target_columns":[1,2]},{"type":"add_column_value","value":"_feature_table.txt.gz"},{"type":"add_column_value","value":"_genomic.fna.gz"},{"type":"add_column_concatenate","target_column_0":1,"target_column_1":2},{"type":"add_column_concatenate","target_column_0":1,"target_column_1":3},{"type":"remove_columns","target_columns":[1,2,3]},{"type":"add_column_value","value":"Genes"},{"type":"add_column_value","value":"DNA"},{"type":"add_column_regex","target_column":1,"expression":".*\\/(.*)","group_count":1},{"type":"swap_columns","target_column_0":0,"target_column_1":5},{"type":"remove_columns","target_columns":[5]},{"type":"split_columns","target_columns_0":[1,3],"target_columns_1":[2,4]}],"mapping":[{"type":"list_identifiers","columns":[0],"editing":false},{"type":"url","columns":[1]},{"type":"collection_name","columns":[2]}]}
>       > ```
>       >
>       > You can click the {% icon tool %} next to the header **Rules** {% icon tool %}, and paste the contents there, before clicking **Apply**, checking "Add nametag for name" and then **Upload**.
>       {: .tip}
>
>    3. From **Column**, select `Using a Regular Expression`
>       - *"From Column"*: `B`
>       - Select `Create columns matching expression groups`
>       - *"Regular Expression"*: `.*(\/GCA.*$)`
>       - *"Number of Groups"*: `1`
>       - Click `Apply`
>    4. From **Column**, select `Concatenate Columns`
>       - *"From Column"*: `B`
>       - *"From Column"*: `C`
>       - Click `Apply`
>    5. From **Rules**, select `Remove Columns(s)`
>       - *"From Column"*: `B`, `C`
>       - Click `Apply`
>    6. From **Column**, select `Fixed Value`
>       - *"Value"*: `_feature_table.txt.gz`
>       - Click `Apply`
>    6. From **Column**, select `Fixed Value`
>       - *"Value"*: `_genomic.fna.gz`
>       - Click `Apply`
>    7. From **Column**, select `Concatenate Columns`
>       - *"From Column"*: `B`
>       - *"From Column"*: `C`
>       - Click `Apply`
>    7. From **Column**, select `Concatenate Columns`
>       - *"From Column"*: `B`
>       - *"From Column"*: `D`
>       - Click `Apply`
>    8. From **Rules**, select `Remove Columns(s)`
>       - *"From Column"*: `B`, `C`, `D`
>       - Click `Apply`
>    6. From **Column**, select `Fixed Value`
>       - *"Value"*: `Genes`
>       - Click `Apply`
>    6. From **Column**, select `Fixed Value`
>       - *"Value"*: `DNA`
>       - Click `Apply`
>    3. From **Column**, select `Using a Regular Expression`
>       - *"From Column"*: `B`
>       - Select `Create columns matching expression groups`
>       - *"Regular Expression"*: `.*\/(.*)`
>       - *"Number of Groups"*: `1`
>       - Click `Apply`
>    9. From **Rules** menu, select `Swap Column(s)`
>       - *"Swap Column"*: `A`
>       - *"With Column"*: `F`
>       - Click `Apply`
>    8. From **Rules**, select `Remove Columns(s)`
>       - *"From Column"*: `F`
>       - Click `Apply`
>    9. From **Rules** menu, select `Split Column(s)`
>       - *"Odd Row Column(s)"*: `B`, `D`
>       - *"Even Row Column(s)"*: `C`, `E`
>       - Click `Apply`
>    9. From **Rules** menu, select `Add / Modify Column Definitions`
>       - `Add Definition`, `List Identifier(s)`, Select Column `A`
>       - `Add Definition`, `URL`, Column `B`
>       - `Add Definition`, `Collection Name`, Column `C`
>       - Click `Apply`
>    12. Check **Add nametag for name**
>    12. **Upload**
>
> At the end of this you should have two collections: one containing genomic sequences and another containing annotations.
{: .hands_on}

## Visualizing rearrangements

Now we will perform alignments between our assembly and the three most closely related genomes to get a detailed look at any possible genome architecture changes. We will again use LASTZ:

> <hands-on-title>Aligning again</hands-on-title>
>
> 1. {% tool [LASTZ](toolshed.g2.bx.psu.edu/repos/devteam/lastz/lastz_wrapper_2/1.3.2) %} with the following parameters:
>   - *"Select TARGET sequence(s) to align against"*: `from your history`
>   - {% icon param-collection %} *"Select a reference dataset"*: `DNA`, the *E. coli* genomes we uploaded earlier
>   - {% icon param-file %} *"Select QUERY sequence(s)"*: `E. coli C` fasta file
>   - **Chaining**
>        - *"Perform chaining of HSPs with no penalties"*: `Yes`
>          > <tip-title>What does chaining do?</tip-title>
>          > For more information about chaining [look here](https://lastz.github.io/lastz/#ex_stages)
>          {: .tip}
>
>   - **Output**
>        - *"Specify the output format"*: `Customized general`
>        - *"Select which fields to include"*: select the following
>             - `score` alignment score
>             - `name1` name of the *target* sequence
>             - `strand1` strand for the *target* sequence
>             - `zstart1` 0-based start of alignment in *target*
>             - `end1` end of alignment in *target*
>             - `length1` length of alignment in *target*
>             - `name2` name of *query* sequence
>             - `strand2` strand for the *query* sequence
>             - `zstart2` 0-based start of alignment in *query*
>             - `end2` end of alignment in *query*
>             - `identity` alignment identity
>             - `number` alignment number
>        - *"Create a dotplot representation of alignments?"*: `Yes`
>
> 2. Rename the `LASTZ on collection... mapped reads` something more memorable like `LASTZ Alignments`
>
>    {% snippet faqs/galaxy/collections_rename.md name="LASTZ Alignments" %}
>
{: .hands_on}

Because we chose to produce Dot Plots as well, LASTZ will generate two collections: one containing alignment data and the other containing DotPlots in PNG format:

![Dot Plots](../../images/three_dot_plots.png "Dot Plot representations of alignments between three <i>E. coli</i> genomes and our assembly. Target (X-axis) is indicated above each dot plot. Query (Y-axis) is our assembly. Red circle indicates a region deleted in our assembly.")

A quick conclusion that can be drawn here is that there is a large inversion in CP020543 and deletion in our assembly.

> <details-title>Interpreting Dot Plots</details-title>
> If you are not sure how to interpret Dot Plots here is a great explanation by [Michael Schatz](http://schatz-lab.org/):
>
> ![Interpreting Dot Plots](../../images/dotplot.png "A quick reference to interpreting Dot Plots. Our case is identical to <i>Insertion into Reference</i> shown in the upper left.")
{: .details}

For a moment let's leave LASTZ result and create a browser that would allows us to display our results.

## Producing a Genome Browser for this experiment

The dot plots we've produced above are great, but they are static. It would be wonderful to load these data into a genome browser where one can zoom in and out as well as add tracks such as those containing genes. To create a browser we need a genome and a set of tracks. Tracks are features such as genes or SNPs with start and end positions corresponding to a coordinate system provided by the genome. Thus the first thing to do is to create a *genome* that would represent our experiment. We can create such a genome by simply combining the three genomes of closely related strains with our assembly in a single dataset&mdash;a hybrid genome.

### Collecting the genomes

The first step will be collapsing the collection containing the three genomes into a single file:

> <hands-on-title>Creating a single FASTA dataset with all genomes</hands-on-title>
>
> 1. {% tool [Collapse Collection](toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/4.0) %}
>
>    - {% icon param-collection %} *"Collection of files to collapse"* the three genomes (collection) named `DNA`
>
> 2. Convert the datatype of this output to uncompress it
>
>    {% snippet faqs/galaxy/datasets_convert_datatype.md conversion="Convert compressed to uncompressed" %}
>
> 3. {% tool [Concatenate datasets](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.0) %} tail-to-head (cat):
>    - *"Datasets to concatenate"*: `Collapse collection ... uncompressed`, the output from the uncompression step.
>    - Click **Insert Dataset** button
>        - *"Select"*: the `E. coli C` file from the start of the history
>
> 4. Rename the output to `DNA (E. coli C + Relatives)`
>
>    {% snippet faqs/galaxy/datasets_rename.md name="DNA (E. coli C + Relatives)" %}
{: .hands_on}

The resulting dataset contains four sequences: three genomes plus our assembly.

### Preparing the alignments

Above we computed alignments using LASTZ. Because we ran LASTZ on a collection containing genomic sequences, LASTZ produced a collection as well (actually two collections: one containing alignments an the other with dot plots). To display alignments in the browser we need to do several things:

1. Fix unwanted `%` signs in LASTZ output
2. Create names for alignment blocks
3. Convert LASTZ output into [BED](https://genome.ucsc.edu/FAQ/FAQformat.html#format1) format
4. Create a single BED track containing alignments against all four genomes.

To begin, let's look at the LASTZ output:

1        | 2            | 3 | 4      | 5      | 6      | 7         | 8 | 9      | 10     | 11            | 12     | 13
-------- | ----------   | - | ------ | ------ | ------ | -------   | - | ------ | ------ | -------       | ------ | --
10141727 | `CP020543.1` | + | 48     | 106157 | 106109 | `Ecoli_C` | + | 0      | 106109 | 106107/106109 | 100.0% | 1
    5465 | `CP020543.1` | + | 121267 | 121367 | 100    | `Ecoli_C` | + | 109317 | 109418 | 76/100        | 76.0%  | 2
    4870 | `CP020543.1` | + | 159368 | 159512 | 144    | `Ecoli_C` | + | 128706 | 128828 | 95/115        | 82.6%  | 3

One immediate problem is `%` character in column 12 (alignment identity). We need to remove it as we will use this for the score column of the BED file, and that must be a normal number and not a percentage.

Column 13 of the fields chosen by us for [LASTZ run](#hands-on-aligning-again) is `number`. This is an incrementing number given by LASTZ to every alignment block so it can be uniquely identified. The problem is that by running LASTZ on a collection of three genomes it generated a number for each output independently starting with `1` each time. So these alignments identified are unique within each individual run but are redundant for multiple runs. We can fix that by pre-pending each alignment identified (column 12) with the name of the target sequence (column 2). This would create alignments that are truly unique. For example, in the case of the LASTZ output shown above alignment identifier `1` will become `CP020543.11`, `2` will become `CP020543.12` and so on.

> <comment-title>BED format</comment-title>
> Our goal is to convert this into a format that will be acceptable to the genome browser. One of such formats is [BED](https://genome.ucsc.edu/FAQ/FAQformat.html#format1). In one of its simplest forms (there is one even simpler - 3 column BED) it has six columns:
>
> 1. Chromosome ID
> 2. Start
> 3. End
> 4. Name of the feature
> 5. Score (must be between 0 and 1000)
> 6. Strand (`+`, `-`, or `.` for no strand data).
{: .comment}

> <hands-on-title>Convert LASTZ output to BED</hands-on-title>
>
> 1. {% tool [Replace Text](toolshed.g2.bx.psu.edu/repos/iuc/datamash_ops/datamash_ops/1.1.0) %} in a specific column:
>    - {% icon param-collection %} *"File to process"*: output of LASTZ (`LASTZ Alignments`)
>    - *"in column"*: `Column 12`
>    - *"Find pattern"*: `%`
>    - *"Replace with"*: leave empty
>
> 2. {% tool [Merge Columns together](toolshed.g2.bx.psu.edu/repos/devteam/merge_cols/mergeCols1/1.0.1) %} with the following parameters:
>    - {% icon param-collection %} *"Select data"*: the output of the previous step, `Replace Text on collection ...`
>    - *"Merge column"*: `Column: 2` (this is the Target sequence name)
>    - *"with column"*: `Column: 13` (this is the alignment block created by LASTZ)
>
>    > <details-title>Output information</details-title>
>    > The tool added a new column (Column 14) containing a merge between the target name and alignment id. Now we can differentiate between alignment blocks that exist between, for example, `CP020543.1` and `LT906474.1` because they will have accessions embedded within alignment block IDs. For example, the first alignment between `CP020543.1` and our assembly `Ecoli_C` will have alignment block id `CP020543.11`, while the 225th alignment between `LT906474.1` and `Ecoli_C` will have ID `LT906474.1225`. Because of this we can collapse the entire collection of alignments into a single dataset:
>    {: .details}
>
> 3. {% tool [Collapse Collection](toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/4.0) %} with the following parameters:
>    - {% icon param-collection %} *"Collection of files to collapse"*: the output of the previous step, `Merge Columns on collection...`
>
>    This will produce a single dataset combining all alignment info. We can tell which alignments are between which genomes because we have set identifiers such as `CP020543.13`.
>
> 4. We will reuse this file later so let's rename it `Unprocessed Alignments`
>
>    {% snippet faqs/galaxy/datasets_rename.md name="Unprocessed Alignments" %}
>
> 4. {% tool [Cut](Cut1) %} columns from a table:
>
>    - *"Cut columns"*: `c2,c4,c5,c14,c12,c8`
>    - {% icon param-file %} *"From"*: the output of the previous step (`Unprocessed alignments`)
>
>    > <details-title>Converting to BED</details-title>
>    > Let's look again at the data we generated in the last step:
>    >
>    > 1        | 2          | 4 | 4      | 5      | 6      | 7       | 8 | 9      | 10     | 11            | 12    | 13 | 14
>    > -------- | ---------- | - | ------ | ------ | ------ | ------- | - | ------ | ------ | --            | ----- | -- | -----------
>    > 10141727 | CP020543.1 | + | 48     | 106157 | 106109 | Ecoli_C | + | 0      | 106109 | 106107/106109 | 100.0 | 1  | CP020543.11
>    >     5465 | CP020543.1 | + | 121267 | 121367 | 100    | Ecoli_C | + | 109317 | 109418 | 76/100        | 76.0  | 2  | CP020543.12
>    >     4870 | CP020543.1 | + | 159368 | 159512 | 144    | Ecoli_C | + | 128706 | 128828 | 95/115        | 82.6  | 3  | CP020543.13
>    >
>    > Alignments are regions of high similarity between two sequences. Therefore each alignment block has two sets of coordinates associated with it: start/end in the first sequences (target) and start/end in the second sequence (query). But BED only has one set of coordinates. Thus we can create two BEDs: one using coordinates from the target and the other one from query. The first file will depict alignment data from the standpoint of target sequences `CP020543.1`, `CP024090.1`, `LT906474.1` and the second from the standpoint of query - our own assembly [we called](#hands-on-fixing-assembly) `Ecoli_C`.
>    > In the first BED, column 1 will contain names of targets (`CP020543.1`, `CP024090.1`, and `LT906474.1`).
>    > In the second BED, column 1 will contain name of our assembly: `Ecoli_C`.
>    >
>    > To create the first BED we will cut six columns from the dataset produced at the last step. Specifically, to produce the target BED we will cut columns 2, 4, 5, 14, 12, and 8. To produce the query BED columns 7, 9, 10, 14, 12, 8 will be cut.
>    {: .details}
>
>    > <warning-title>There are multiple <b>CUT</b> tools!</warning-title>
>    > The Hands-On box below uses **Cut** tool. Beware that some Galaxy instances contain multiple **Cut** tools. The one that is used below is called **Cut columns from a table** while the other one, which we will NOT use is called **Cut columns from a table (cut)**. It is a small difference, but the tools are different.
>    {: .warning}
>
>    This will produce a dataset looking like this:
>
>    1          | 2      | 3      | 4           | 5     | 6
>    ---------- | ------ | ------ | ----------- | ----- | -
>    CP020543.1 | 48     | 106157 | CP020543.11 | 100.0 | +
>    CP020543.1 | 121267 | 121367 | CP020543.12 | 76.0  | +
>    CP020543.1 | 159368 | 159512 | CP020543.13 | 82.6  | +
>
>    > <tip-title>Not exactly the same?</tip-title>
>    > Depending on the steps and other choices, the genomes may be in a different order here. This is unimportant, as all of the same alignments are contained in the file, just the ordering is different. As long as these columns look correct (start/end in column 2/3 are reasonable, a number between 0-100 in column 5, and a + or - in column 6) then it is OK.
>    {: .tip}
>
> 5. Rename this "Target Alignments"
>
>    {% snippet faqs/galaxy/datasets_rename.md name="Target Alignments" %}
>
> 6. {% tool [Cut columns from a table](Cut1) %} with the following parameters
>    - *"Cut columns"*: `c7,c9,c10,c14,c12,c8` (look at the data shown above and the definition of BED to see why we make these choices.)
>    - *"From"*: `Unprocessed alignments`, the output of **collection collapse**
>
> 5. Rename this "Query Alignments"
>
>    {% snippet faqs/galaxy/datasets_rename.md name="Query Alignments" %}
>
> 6. {% tool [Concatenate multiple datasets or collections](cat1) %}
>    - *"Concatenate Dataset"*: `Query Alignments`
>    - Click "Insert Dataset" button
>    - *"1: Dataset"*: `Target Alignments`
>
> 7. Change the datatype of the output to BED and rename the output "Target & Query Alignments"
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="bed" %}
>
>    {% snippet faqs/galaxy/datasets_rename.md name="Target & Query Alignments" %}
>
{: .hands_on}

This will produce a dataset looking like this:

1       | 2      | 3      | 4           | 5     | 6
------- | ------ | ------ | ----------- | ----- | -
Ecoli_C | 0      | 106109 | CP020543.11 | 100.0 | +
Ecoli_C | 109317 | 109418 | CP020543.12 | 76.0  | +
Ecoli_C | 128706 | 128828 | CP020543.13 | 82.6  | +


### Extracting Genes

Earlier we [downloaded](#hands-on-uploading-sequences-and-annotations) gene annotations for the three genomes most closely related to our assembly. The data was downloaded as a collection containing annotations for `CP020543.1`, `CP024090.1`, and `LT906474.1`. The annotation data contains multiple columns described by NCBI as follows (you can look at the actual data by finding the annotation collection from above (called `Genes`)):

> Tab-delimited text file reporting locations and attributes for a subset of
> annotated features. Included feature types are: gene, CDS, RNA (all types),
> operon, C/V/N/S_region, and V/D/J_segment.
>
> The file is tab delimited (including a #header) with the following columns:
>
> Column | Definition
> ---    | ---
> 1      | feature: INSDC feature type
> 2      | class: Gene features are subdivided into classes according to the gene biotype computed based on the set of child features for that gene. See the description of the gene_biotype attribute in the GFF3 documentation for more details: ftp://ftp.ncbi.nlm.nih.gov/genomes/README_GFF3.txt ncRNA features are subdivided according to the ncRNA_class. CDS features are subdivided into with_protein and without_protein, depending on whether the CDS feature has a protein accession assigned or not. CDS features marked as without_protein include CDS features for C regions and V/D/J segments of immunoglobulin and similar genes that undergo genomic rearrangement, and pseudogenes.
> 3      | assembly: assembly accession.version
> 4      | assembly_unit: name of the assembly unit, such as "Primary Assembly", "ALT_REF_LOCI_1", or "non-nuclear"
> 5      | seq_type: sequence type, computed from the "Sequence-Role" and "Assigned-Molecule-Location/Type" in the `*_assembly_report.txt` file. The value is computed as: if an assembled-molecule, then reports the location/type value. e.g. chromosome, mitochondrion, or plasmid if an unlocalized-scaffold, then report "unlocalized scaffold on <type>". e.g. unlocalized scaffold on chromosome else the role, e.g. alternate scaffold, fix patch, or novel patch
> 6      | chromosome
> 7      | genomic_accession
> 8      | start: feature start coordinate (base-1). start is always less than end
> 9      | end: feature end coordinate (base-1)
> 10     | strand
> 11     | product_accession: accession.version of the product referenced by this feature, if exists
> 12     | non-redundant_refseq: for bacteria and archaea assemblies, the non-redundant `WP_` protein accession corresponding to the CDS feature. May be the same as column 11, for RefSeq genomes annotated directly with `WP_` RefSeq proteins, or may be different, for genomes annotated with genome-specific protein accessions (e.g. `NP_` or `YP_` RefSeq proteins) that reference a `WP_` RefSeq accession.
> 13     | related_accession: for eukaryotic RefSeq annotations, the RefSeq protein accession corresponding to the transcript feature, or the RefSeq transcript accession corresponding to the protein feature.
> 14     | name: For genes, this is the gene description or full name. For RNA, CDS, and some other features, this is the product name.
> 15     | symbol: gene symbol
> 16     | GeneID: NCBI GeneID, for those RefSeq genomes included in NCBI's Gene resource
> 17     | locus_tag
> 18     | feature_interval_length: sum of the lengths of all intervals for the feature (i.e. the length without introns for a joined feature)
> 19     | product_length: length of the product corresponding to the accession.version in column 11. Protein product lengths are in amino acid units, and do not include the stop codon which is included in column 18. Additionally, product_length may differ from feature_interval_length if the product contains sequence differences vs. the genome, as found for some RefSeq transcript and protein products based on mRNA sequences and also for INSDC proteins that are submitted to correct genome discrepancies.
> 20     | attributes: semi-colon delimited list of a controlled set of qualifiers. The list currently includes: partial, pseudo, pseudogene, ribosomal_slippage, trans_splicing, anticodon=NNN (for tRNAs), old_locus_tag=XXX
>
{: .quote cite="https://ftp.ncbi.nlm.nih.gov/genomes/genbank/README.txt"}


Our objective is to convert these data into BED. In this analysis we want to initially concentrate on protein coding regions. To do this let's select all lines from the annotation datasets that contain the term `CDS`, then
we will produce a collection with three datasets just like the original `Genes` collection but containing only CDS data. Next we need to cut out only those columns that need to be included in the BED format. There is one problem with this. We are trying to convert these data into [6 column BED](#comment-bed-format). In this format the fifth column (score) must have a value between 0 and 1000. To satisfy this requirement we will create a dummy column that will always have a value of `0`.
Finally we can cut necessary columns from these datasets. These columns are 8 (start), 9 (end), 15 (gene symbol), 21 (dummy column we just created), and c10 (strand), and then we can add the genome name.

> <hands-on-title>Extract CDSs from annotation datasets</hands-on-title>
> 1. {% tool [Select lines that match an expression](Grep1) %} with the following parameters:
>    - {% icon param-collection %} *"Select lines from"*: the collection containing annotations, `Genes`
>    - *"the pattern"*: `^CDS`
>
>    This is because we want to retain all lines that begin (`^`) with `CDS`.
>
> 2. {% tool [Add column to an existing dataset](toolshed.g2.bx.psu.edu/repos/devteam/add_value/addValue/1.0.0) %}   with the following parameters:
>    - *"Add this value"*: `0`
>    - {% icon param-collection %} *"to Dataset"*: the collection produced by the previous step (`Select on collection...`)
>
>    This will be used for the "score" field of the BED file since we do not have a proper "score"
>
> 3. {% tool [Cut columns from a table](Cut1) %} with the following parameters:
>
>    We will produce two BED files, one using the product name (e.g. "chromosomal replication initiator protein DnaA") and one using the symbol (e.g. "thrA"). The product name is much more interesting to see in visualisations, but the symbol is more often used in other analyses and we will use that file later. We will start with the product name:
>
>    - *"Cut columns"*: `c8,c9,c14,c21,c10`
>    - {% icon param-collection %} *"From"* the collectio

tools_json:
["Cut", "Cut1", "Upload", "upload1", "Filter sequences by length", "toolshed.g2.bx.psu.edu/repos/devteam/fasta_filter_by_length/fasta_filter_by_length/1.2", "Replace Text", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_line/1.1.2", "LASTZ", "toolshed.g2.bx.psu.edu/repos/devteam/lastz/lastz_wrapper_2/1.3.2", "Collapse Collection", "toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/4.2", "Select random lines from a file", "random_lines1", "Filter", "Filter1", "Datamash (operations on tabular data)", "toolshed.g2.bx.psu.edu/repos/iuc/datamash_ops/datamash_ops/1.1.0", "Select lines that match an expression", "Grep1", "toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/4.0", "Concatenate datasets", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.0", "Merge Columns together", "toolshed.g2.bx.psu.edu/repos/devteam/merge_cols/mergeCols1/1.0.1", "Cut columns from a table", "Concatenate multiple datasets or collections", "cat1", "Add column to an existing dataset", "toolshed.g2.bx.psu.edu/repos/devteam/add_value/addValue/1.0.0", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_column/1.1.3", "Compute sequence length", "toolshed.g2.bx.psu.edu/repos/devteam/fasta_compute_length/fasta_compute_length/1.0.1", "Sort", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_sort_header_tool/1.1.1", "SortBED order the intervals", "toolshed.g2.bx.psu.edu/repos/iuc/bedtools/bedtools_sortbed/2.27.0.0", "ComplementBed Extract intervals not represented by an interval file", "toolshed.g2.bx.psu.edu/repos/iuc/bedtools/bedtools_complementbed/2.27.0.0", "Filter data on any column using simple expressions", "JBrowse", "toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/jbrowse/1.16.8+galaxy1", "Circos: Alignemnts to Links", "toolshed.g2.bx.psu.edu/repos/iuc/circos/circos_aln_to_links/0.69.8+galaxy7", "Circos: Interval to Tiles", "toolshed.g2.bx.psu.edu/repos/iuc/circos/circos_interval_to_tiles/0.69.8+galaxy7", "Circos", "toolshed.g2.bx.psu.edu/repos/iuc/circos/circos/0.69.8+galaxy7", "Intersect intervals find overlapping intervals in various ways", "toolshed.g2.bx.psu.edu/repos/iuc/bedtools/bedtools_intersectbed/2.27.0.2", "Join two Datasets", "join1"]


topic: assembly
tutorial: assembly-with-preprocessing
title: Unicycler assembly of SARS-CoV-2 genome with preprocessing to remove human genome reads
difficulty: intermediate
time_estimation: "4h" # plus additional time for (optional) NCBI SRA downloads
tools_json: []
requirements_json: []
tags_json: ["covid19", "biodiversity"]
content_hash: ab2a461d6e6cd227b9ef23dc30cf479b
last_modified: 2026-05-27T09:01:47.800471
zenodo_link: https://zenodo.org/record/3732359
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/assembly-with-preprocessing/tutorial.html

content:
In some research or clinical contexts it is not possible, or very hard, to
purify DNA/RNA for sequencing from just the specimen of interest.
Instead you will isolate DNA that is contaminated, sometimes heavily, with
DNA/RNA of a different origin.
This is the case for example with microbiome samples, which typically display
considerable contamination with host DNA, or with samples of body fluids for
pathogen detection. Such contamination can pose an issue with certain types of
analyses, in particular with genome assembly.

This tutorial guides you through the preprocessing of sequencing data of
bronchoalveolar lavage fluid (BALF) samples obtained from early COVID-19
patients in China. Since such samples are expected to be contaminated
signficantly with human sequenced reads, the goal is to enrich the data for
SARS-CoV-2 reads by identifying and discarding reads of human origin before
trying to assemble the viral genome sequence.

> <comment-title>The usegalaxy.* COVID-19 analysis project</comment-title>
> This tutorial uses the same data as, and recapitulates to a large extent, the
> [Pre-processing](https://covid19.galaxyproject.org/genomics/1-PreProcessing/)
> and [Assembly](https://covid19.galaxyproject.org/genomics/2-Assembly/) steps
> of the [Genomics](https://covid19.galaxyproject.org/genomics/) section of
> [covid19.galaxyproject.org](https://covid19.galaxyproject.org/).
>
{: .comment}

> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Get data

We are going to create an assembly of the SARS-CoV-2 genome based on mixed
short-reads (Illumina) and long-reads (Nanopore) data from a total of six
different samples, all of which are publicly accessible through the NCBI
and EBI short-reads archives (SRAs).

Since automated data downloads from SRAs can be unreliable at times and could
get broken by changes to the download interface on the NCBI/EBI side, this
tutorial offers two ways to access the sequenced reads input data:

1. Direct download from the NCBI SRA based on accession numbers and using the
   dedicated **Faster Download** tool

   Use this method if it works and is fast enough for you, and if you are
   interested in learning to obtain short-reads data directly from NCBI, in
   general.

2. Download of the same data deposited as a copy at [Zenodo](https://zenodo.org/record/3732359)
   This method uses Galaxy's generic data import functionality, and should be
   very reliable and faster than the download from NCBI.
   It also showcases **rule-based** uploads and demonstrates how they can be
   used to download several datasets and to arrange them into easy to handle
   data structures at the same time.

   > <details-title>Rule-based uploads</details-title>
   > In this tutorial you will only use the features of Galaxy's rule-based
   > uploader that are required to get the input data ready for our analysis,
   > and we will not explain those features in much detail.
   >
   > If, after this first taste, you are interested in a thorough introduction
   > we recommend the advanced tutorial
   > [Collections: Rule Based Uploader](../../../galaxy-interface/tutorials/upload-rules/tutorial.html).
   {: .details}

   Use this method if the direct download from the NCBI SRA does not work, or
   is too slow for your time frame, or if you are interested in advanced use
   of Galaxy's data import functionality.

The corresponding two step-by-step instructions below have been crafted to
produce identically arranged data structures in your history so all subsequent
steps are independent of the data source you choose.

## Get data from NCBI SRA

> <hands-on-title>Data upload to Galaxy from NCBI SRA</hands-on-title>
>
> 1. Create a new history for this tutorial and give it a proper name
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 2. Create a new dataset listing the SRA accession numbers of the Illumina paired-end input data for this tutorial:
>
>    ```
>    SRR10903401
>    SRR10903402
>    SRR10971381
>    ```
>
>    call it, *e.g.*, `Illumina accessions` and set its datatype to `tabular`.
>
>    {% snippet faqs/galaxy/datasets_create_new_file.md format="tabular" %}
>
> 3. Create another new dataset listing the SRA accession numbers of the Nanopore input data for this tutorial:
>
>    ```
>    SRR10948550
>    SRR10948474
>    SRR10902284
>    ```
>
>    call it, *e.g.*, `Nanopore accessions` and set its datatype to `tabular`.
>
> 4. Add `#illumina`/`#nanopore` tags to the datasets
>
>    > <comment-title>Name tags in the analysis</comment-title>
>    > We are going to treat the Illumina- and the Nanopore-sequenced data
>    > separately in this tutorial up to the actual genome assembly step.
>    >
>    > To make it easier to keep track of the two branches of the analysis, we
>    > recommend the use of Galaxy's dataset **name tags**.
>    > A name tag will automatically propagate to any new dataset derived
>    > from the tagged dataset.
>    {: .comment}
>
>    You can create a name tag by attaching a tag starting with `#` to any
>    dataset.
>
>    Name tags are meant to help you identify the origin of datasets quickly.
>    Feel free to either use the suggested names above or choose ones you like.
>
>    {% snippet faqs/galaxy/datasets_add_tag.md %}
>
> 4. Retrieve the Illumina reads data from NCBI:
>
>    Run **Faster Download and Extract Reads in FASTQ** {% icon tool %} with
>    the following parameter settings:
>    - *"select input type"*: `List of SRA accession, one per line`
>      - {% icon param-file %} *"sra accession list"*:
>        the `Illumina accessions` dataset created above
>    - in *"Advanced Options"*
>      - *"Select how to split the spots"*: `--split-3`
>
>    The tool run should generate four new items in your history - three
>    collections and one *log* dataset with a summary of what happened.
>
>    Since all three datasets that we tried to retrieve contain only
>    paired-end reads, only the `Pair-end data` collection is expected to
>    contain downloaded data. Click on the other two collections to verify that
>    they are empty, then delete them from your history (since the collections
>    do not contain any datasets, it does not matter, which delete option -
>    "Collection Only", "Delete Datasets" or "Permanently Delete Datasets" you
>    are choosing when prompted).
>
> 5. Retrieve the Nanopore reads data from NCBI:
>
>    Run **Faster Download and Extract Reads in FASTQ** {% icon tool %} with
>    the following parameter settings:
>    - *"select input type"*: `List of SRA accession, one per line`
>      - {% icon param-file %} *"sra accession list"*:
>        the `Nanopore accessions` dataset created above
>    - in *"Advanced Options"*
>      - *"Select how to split the spots"*: `--split-3`
>
>    As in the previous step, the tool run should generate four new items in
>    your history.
>
>    Since all three datasets that we tried to retrieve in this run contain
>    only single-end reads, only the `Single-end data` collection is expected to
>    contain downloaded data this time. Click on the other two collections to
>    verify that they are empty, then delete them from your history.
>
{: .hands_on}


## Get data from Zenodo

> <hands-on-title>Data upload to Galaxy from Zenodo</hands-on-title>
>
> 1. Create a new history for this tutorial and give it a proper name
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 2. Import Illumina-sequenced reads data from [Zenodo](https://zenodo.org/record/3732359)
>
>    The Zenodo links for the data are these:
>    ```
>    https://zenodo.org/record/3732359/files/SRR10903401_r1.fq.gz
>    https://zenodo.org/record/3732359/files/SRR10903401_r2.fq.gz
>    https://zenodo.org/record/3732359/files/SRR10903402_r1.fq.gz
>    https://zenodo.org/record/3732359/files/SRR10903402_r2.fq.gz
>    https://zenodo.org/record/3732359/files/SRR10971381_r1.fq.gz
>    https://zenodo.org/record/3732359/files/SRR10971381_r2.fq.gz
>    ```
>
>    To upload these data to your Galaxy history in structured form:
>
>      - Copy the above list of links
>      - Open the Galaxy Upload Manager
>        ({% icon galaxy-upload %} on the top right of the tool panel)
>      - In the pop-up window, switch to the **Rule-based** tab and select
>        - *"Upload data as:"*: `Collection(s)`
>        - *"Load tabular data from:"*: `Pasted Table`
>        - Paste the copied links into the text field
>        - Click **Build**
>      - In the next screen, select
>        - {% icon param-repeat %} *"Column"*: `Using a Regular expression`
>          - *"From Column"*: `A`
>          - {% icon param-check %} *"Create columns matching expression groups."*
>          - *"Regular Expression"*: `.+/(SRR\d+)_r(\d).fq.gz`
>          - *"Number of Groups"*: `2`
>          - Click **Apply**
>        - {% icon param-repeat %} *"Rules"*: `Add / Modify Column Definitions`
>          - {% icon param-repeat %} *"Add Definition"*: `URL`
>            - *"URL"*: `A`
>          - {% icon param-repeat %} *"Add Definition"*: `List Identifier(s)`
>            - *"List Identifier(s)"*: `B`
>          - {% icon param-repeat %} *"Add Definition"*: `Paired-end Indicator`
>            - *"Paired-end Indicator"*: `C`
>          - Click **Apply**
>        - *"Type"*: `fastqsanger.gz`
>        - *"Name"*: `Illumina PE data` (or similar)
>        - *"Add nametag for name:"* {% icon param-check %}
>
>          > <comment-title>Name tags in the analysis</comment-title>
>          > We are going to treat the Illumina- and the Nanopore-sequenced data
>          > separately in this tutorial up to the actual genome assembly step.
>          >
>          > To make it easier to keep track of the two branches of the analysis, we
>          > recommend the use of Galaxy's dataset **name tags**.
>          > A name tag will automatically propagate to any new dataset derived
>          > from the tagged dataset.
>          {: .comment}
>
>          Checking this option tells Galaxy to reuse the collection name above
>          as a name tag on the collection.
>
>        - Click **Upload**
>
> 3. Import Nanopore-sequenced reads data from [Zenodo](https://zenodo.org/record/3732359)
>
>    ```
>    https://zenodo.org/record/3732359/files/SRR10902284_ONT.fq.gz
>    https://zenodo.org/record/3732359/files/SRR10948474_ONT.fq.gz
>    https://zenodo.org/record/3732359/files/SRR10948550_ONT.fq.gz
>    ```
>
>    Again, we want to upload this data to our history in structured form.
>    To do so:
>
>      - Copy the above list of links
>      - Open the Galaxy Upload Manager
>        ({% icon galaxy-upload %} on the top right of the tool panel)
>      - In the pop-up window, switch to the **Rule-based** tab and select
>        - *"Upload data as:"*: `Collection(s)`
>        - *"Load tabular data from:"*: `Pasted Table`
>        - Paste the copied links into the text field
>        - Click **Build**
>      - In the next screen, select
>        - {% icon param-repeat %} *"Column"*: `Using a Regular expression`
>          - *"From Column"*: `A`
>          - {% icon param-check %} *"Create columns matching expression groups."*
>          - *"Regular Expression"*: `.+/(SRR\d+)_ONT.fq.gz`
>          - *"Number of Groups"*: `1`
>          - Click **Apply**
>        - {% icon param-repeat %} *"Rules"*: `Add / Modify Column Definitions`
>          - {% icon param-repeat %} *"Add Definition"*: `URL`
>            - *"URL"*: `A`
>          - {% icon param-repeat %} *"Add Definition"*: `List Identifier(s)`
>            - *"List Identifier(s)"*: `B`
>          - Click **Apply**
>        - *"Type"*: `fastqsanger.gz`
>        - *"Name"*: `Nanopore data` (or similar)
>        - *"Add nametag for name:"* {% icon param-check %}
>        - Click **Upload**
>
{: .hands_on}


# Read trimming and quality control

In the end, we do not want to base our mapping on low-quality reads that may
cause misassembly of fragments or introduce sequencing artefacts into the final
assembled sequence. Since quality control, read filtering and read trimming are
all quite fast and computationally cheap operations compared to the read
mapping that we will use to identify and eliminate human reads, it is best to
perform these steps up front.

Due to their very different nature, however, Illumina- and Nanopore-sequenced
reads should be treated rather differently.

## Trimming and filtering of Illumina reads

Galaxy offers a panel of different NGS reads trimming/filtering tools. Here,
we use **fastp** {% icon tool %}, which is straightforward to configure,
and when combined with **MultiQC** {% icon tool %}, enables nice and
easy-to-interpret visualizations of the effects of preprocessing, in particular
for multiple samples.

In the following, we configure the tool to retain reads only if at most 20% of
their bases have a Phred-scaled quality >= 20 and if there length in bases after
trimming of adapter sequences (which the tool auto-detects for us) is at least
50.

The JSON-formatted report produced by the tool, can serve as input to
**MultiQC** {% icon tool %} for a direct visual comparison of key quality
metrics for all samples before and after preprocessing.

> <hands-on-title>Reads preprocessing and quality reporting</hands-on-title>
> 1. **fastp** {% icon tool %} with the following parameters
>    - *"Single-end or paired reads"*: `Paired Collection`
>      - *"Select paired collection(s)"*: the collection of Illumina-sequenced
>        reads as produced in the *Get Data* section
>    - in *"Filter Options"*
>      - in *"Quality filtering options"*
>        - *"Disable quality filtering"*: `No`
>        - *"Qualified quality phred"*: `20`
>        - *"Unqualified percent limit"*: `20`
>      - in *"Length filtering options"*
>        - *"Disable length filtering"*: `No`
>        - *"Length required"*: `50`
>    - in *"Output Options"*
>      - *"Output HTML report"*: `No`
>      - *"Output JSON report"*: `Yes`
>
>    The tool run produces two collections - one with the actual preprocessed
>    reads of all input samples, another one with a JSON-formatted report of
>    the processing for every sample.
>
> 2. **MultiQC** {% icon tool %} with the following parameters
>    - {% icon param-repeat %} *"Results"*
>      - *"Which tool was used generate logs?"*: `fastp`
>        - {% icon param-collection %} *"Output of fastp"*: the collection of
>          JSON-formatted reports, second collection produced by **fastp**
>          {% icon tool %}
>
>    This tool run generates a single webpage output with the combined quality reports
>    for all samples before and after processing with **fastp** {% icon tool %}.
>
{: .hands_on}

## Quality control of Nanopore reads

Nanopore-sequenced reads differ greatly in length from one another and are, on
average, of relatively low quality (in particular when compared to
Illumina-sequenced reads). These properties make it challenging to preprocess
them with standard tools. Quality assessment tools calibrated to work well with
Illumina-sequenced reads are less useful for Nanopore-sequenced reads, too, for
the same reasons. Here we restrict ourselves to a simple quality check with
**NanoPlot** {% icon tool %}, a dedicated QC tool for Nanopore-sequenced reads.

> <hands-on-title>Checking the quality of Nanopore reads with NanoPlot</hands-on-title>
> 1. **NanoPlot** {% icon tool %} with the following parameters
>    - *"Select multifile mode"*: `batch`
>      - *"Type of the file(s) to work on"*: `fastq`
>        - {% icon param-collection %} *"files"*: the collection of
>          Nanopore-sequenced reads as produced in the *Get Data* section
>    - in *"Options for filtering or transforming input prior to plotting"*
>      - *"Logarithmic scaling of lengths in plots."*: `Yes`
>
>    > <question-title></question-title>
>    >
>    > 1. Looking at the three generated quality reports, which of the three
>    >    samples seems to be of better quality overall than the other two,
>    >    and what are some criteria that support this conclusion?
>    >
>    > > <solution-title></solution-title>
>    > > 1. Sample `SRR10948474` has the best overall quality.
>    > >
>    > >    It has both higher average read length and quality than the other
>    > >    two (see the *Summary statistics* table), has a distribution of
>    > >    read qualities that peaks around intermediate quality scores (not
>    > >    low ones as for the other samples), and contains some extra-long
>    > >    (though at least partly rather low-quality) reads. Those last two
>    > >    points become most obvious when looking at the *Read lengths vs
>    > >    Average read quality* plots.
>    >  {: .solution }
>    {: .question}
>
{: .hands_on}

# Subtraction of reads mapping to the human reference genome

## Mapping of Illumina reads

In this tutorial, we are using **Bowtie2** for mapping our short-reads data to
the human genome. *BWA-MEM* would be a good alternative for mapping the 150
nucleotides (see the QC report above) reads from our samples.

According to its authors, the *Minimap2* aligner, which we will be using for
mapping the Nanopore-sequenced data in the next step, is supposed to outcompete
*Bowtie2* and *BWA-MEM* in terms of speed even for Illumina-sequenced reads
of length > 100 nts, but we opt for the conservative approach of using a
widely-used, well-tested tool here.

> <hands-on-title>Mapping with Bowtie2</hands-on-title>
> 1. **Bowtie2** {% icon tool %} with the following parameters
>    - *"Is this single or paired library"*: `Paired-end Dataset Collection`
>       - *"FASTQ Paired Dataset"*: the collection of preprocessed
>         Illumina-sequenced reads, output of **fastp** {% icon tool %}
>       - *"Write unaligned reads (in fastq format) to separate file(s)"*: `No`
>
>         Activating this option may seem attractive since the unaligned reads
>         are what we are interested in, but filtering for those reads with
>         a dedicated tool in a separate step allows us to filter on the
>         properties of the read pairs instead of those of individual reads.
>
>       - *"Write aligned reads (in fastq format) to separate file(s)"*: `No`
>       - *"Do you want to set paired-end options?"*: `No`
>
>     - *"Will you select a reference genome from your history or use a built-in index?"*:
>       `Use a built-in genome index`
>       - *"Select reference genome"*: `Human (Homo sapiens): hg38 Full`
>     - *"Set read groups information?"*: `Do not set`
>     - *"Select analysis mode"*: `Default setting only`
>     - *"Do you want to tweak SAM/BAM Options?"*: `No`
>     - *"Save the bowtie2 mapping statistics to the history"*: `Yes`
>
>    The tool run should produce two collections of output datasets. One with
>    the actual mapped reads of the three samples and one with the corresponding
>    mapping statistics for each sample, which we want to have a brief look at
>    next.
>
> 2. Inspect the `mapping stats` of each sample by clicking on the corresponding collection, then on the {% icon galaxy-eye %} (eye) icon of each individual sample data
>
>    > <question-title></question-title>
>    >
>    > 1. What percentage of reads of each sample has been aligned to the `hg38` reference genome?
>    > 2. Which sample is the least contaminated with human reads?
>    > 3. Which sample contains the highest amount of SARS-CoV2 reads?
>    >
>    > > <solution-title></solution-title>
>    > > 1. The samples have between 13% and 21% of reads aligned to `hg38`.
>    > >    The information can be found on the last line of output for each
>    > >    sample.
>    > > 2. Sample `SRR10971381` is the least contaminated with just above 13%
>    > >    of human reads
>    > > 3. You cannot answer this question with this data. While `SRR10971381`
>    > >    shows the lowest relative contamination with human reads, that
>    > >    does not mean that all other reads are from SARS-CoV-2. They could
>    > >    come from other species (*e.g.*, bacteria or other viruses)
>    > >    contained in this BALF sample.
>    >  {: .solution }
>    {: .question}
>
{: .hands_on}

## Mapping of Nanopore reads

For the mapping of the Nanopore-sequenced data we are using the **Minimap2**
aligner, which is particularly efficient for mapping long reads.

> <hands-on-title>Nanopore reads mapping</hands-on-title>
>
> 1. **Map with minimap2** {% icon tool %} with the following parameters
>    - *"Will you select a reference genome from your history or use a built-in index?"*:
>      `Use a built-in genome index`
>      - *"Using reference genome"*: `Human Dec. 2013 (GRCh38/hg38) (hg38)`
>
>    - *"Single or Paired-end reads"*: `Single`
>      - {% icon param-collection %} *"Select fastq dataset"*: the collection
>        of Nanopore-sequenced reads as set up in the *Get Data* section
>
>    - *"Select a profile of preset options"*: `Oxford Nanopore read to reference mapping. ...`
>
>    This tool run produces one collection with the actual mapped reads for
>    each Nanopore-sequenced sample. Unlike **Bowtie2** it does not have an
>    option to output mapping statistics directly. However, we can generate
>    that information through an extra step.
>
> 2. **Samtools stats** {% icon tool %} with the following parameters
>    - {% icon param-collection %} *"BAM file"*: the collection of mapped
>      Nanopore-sequenced reads, output of **Map with minimap2 {% icon tool %} (step 1)
>    - *"Output"*: `Separate datasets for each statistic`
>      - *"Desired output files"*
>        - {% icon param-check %} *"Summary numbers"*
>
>          These simple summary stats correspond approximately to the
>          statistics generated by **Bowtie2** and are enough for our purpose.
>
>    > <comment-title>Mapping stats for Nanopore-sequenced long reads</comment-title>
>    >
>    > Since, unlike Illumina-generated reads, Nanopore-sequenced reads can
>    > have very different lengths, it makes limited sense to calculate a
>    > ratio of mapped to overall reads for them.
>    >
>    > Instead, `bases mapped` / `total length` should give a more reliable
>    > estimate of which fraction of the data is due to human genome sequence.
>    >
>    > Try to calculate this ratio for the three samples on your own!
>    {: .comment}
>
{: .hands_on}

## Human reads subtraction

At this point you should have two collections of mapped reads - one holding the
mapping results obtained with **Bowtie2** of the three Illumina-sequenced
samples, the other one holding the **minimap2** output for the three
Nanopore-sequenced samples.

Next, we are going to filter the data from both collections to retain only
those reads that were *not* mapped to the human genome, *i.e* those of
potential viral origin.

> <hands-on-title>Mapped reads filtering</hands-on-title>
>
> 1. **Samtools view** {% icon tool %} to filter the Illumina-sequenced reads mapped with Bowtie2:
>    - {% icon param-collection %} *"SAM/BAM/CRAM data set"*: the collection of
>      mapped Illumina-sequenced reads, output of **Bowtie2** {% icon tool %}
>    - *"What would you like to look at?"*: `A filtered/subsampled selection of reads`
>      - in *"Configure filters"*
>        - *"Require that these flags are set"*: `Read is unmapped` and
>          `Mate is unmapped`
>
>      - *"What would you like to have reported?"*: `All reads retained after filtering and subsampling`
>        - *"Produce extra dataset with dropped reads?"*: `No`
>        - *"Output format"*: `BAM (-b)`
>
> 2. **Samtools view** {% icon tool %} to filter the Nanopore-sequenced reads mapped with minimap2:
>    - {% icon param-collection %} *"SAM/BAM/CRAM data set"*: the collection of
>      mapped Nanopore-sequenced reads, output of **minimap2** {% icon tool %}
>    - *"What would you like to look at?"*: `A filtered/subsampled selection of reads`
>      - In *"Configure filters"*
>        - *"Require that these flags are set"*: `Read is unmapped`
>
>      - *"What would you like to have reported?"*: `All reads retained after filtering and subsampling`
>        - *"Produce extra dataset with dropped reads?"*: `No`
>        - *"Output format"*: `BAM (-b)`
>
> 3. (Optional) Remove the database `hg38` attribute from the output files
>
>    > <details-title>Why do this?</details-title>
>    > When we ran the **Bowtie2** {% icon tool %} and **minimap2**
>    > {% icon tool %} mappers before, these tools set the *database* attribute
>    > on their outputs to `hg38` to indicate that the mapped reads in these
>    > outputs have been mapped against this version of the human reference
>    > genome - an important piece of information for further work with those
>    > mapped reads.
>    >
>    > In step 1 and 2 above, however, we have eliminated all mapped reads so
>    > the `database: hg38` info is misleading from this point onward in the
>    > analysis. While not directly harmful, it is best practice to remove this
>    > metadata now.
>    {: .details}
>
>    For the outputs of step 1 and step 2 above, reset the database/build
>    (dbkey) to `unspecified (?)`.
>
>    {% snippet faqs/galaxy/datasets_change_dbkey.md dbkey="unspecified (?)" %}
>
{: .hands_on}


# Format conversion of remaining reads

## Conversion to fastq format

Assembly tools, typically, expect their input data to be fastq-formatted, but
what we have after mapping and filtering is data in BAM format. Hence, we need
to convert the retained Illumina- and Nanopore-sequenced reads back into their
original format before proceeding to assembly.

> <hands-on-title>BAM to fastq format conversion</hands-on-title>
>
> 1. **Samtools fastx** {% icon tool %} to convert the filtered Illumina-sequenced reads to fastq format
>    - {% icon param-collection %} *"BAM or SAM file to convert"*: the collection of
>      filtered Illumina-sequenced reads, output of first **Samtools view** {% icon tool %} run
>    - *"Output format"*: `compressed FASTQ`
>    - *"outputs"*: `READ1` and `READ2`
>
> 2. **Samtools fastx** {% icon tool %} to convert the filtered Nanopore-sequenced reads to fastq format
>    - {% icon param-collection %} *"BAM or SAM file to convert"*: the collection of
>      filtered Nanopore-sequenced reads, output of second **Samtools view** {% icon tool %} run
>    - *"Output format"*: `compressed FASTQ`
>    - *"outputs"*: `unspecific`
>
{: .hands_on}

## Optional: Rearrange the filtered data into the original nested data structure

If you compare the outputs of the last step to the input data we started out
with, you will notice that the Illumina-sequenced data is arranged differently
now than initially. It is arranged now into separate collections of forward and
reverse reads, whereas we started with a single nested collection of the data.

We can easily cast the data back into its original structure with one of
Galaxy's collection manipulation tools, but note that we will not use the
resulting nested collection for this tutorial because the **Unicycler** tool
for assembling the reads would not be able to handle the nested data correctly.

Thus, the following just serves as an illustration and is entirely optional.

> <hands-on-title>Arrange two list collections into a list of pairs</hands-on-title>
>
> 1. **Zip Collection** {% icon tool %} with the following parameters
>    - {% icon param-collection %} *"Input Dataset (Forward)"*: the collection
>      of filtered Illumina-sequenced forward reads in fastq format,
>      first output of first **Samtools fastx** {% icon tool %} run
>    - {% icon param-collection %} *"Input Dataset (Reverse)"*: the collection
>      of filtered Illumina-sequenced reverse reads in fastq format,
>      second output of first **Samtools fastx** {% icon tool %} run
>
{: .hands_on}

## Merging of reads with collection operations

To merge reads from several samples into a combined final assembly, we need to
pass the data to **Unicycler** {% icon tool %} in partially merged form. The
forward and reverse reads of paired-end data should be kept separate, and so
should short and long reads. However, the tool has no option to combine data
from individual samples, so we need to merge the forward, reverse, and the long
reads data, respectively, across samples. Conveniently for us, the outputs of
the earlier **Samtools fastx** {% icon tool %} runs have already returned the
data structured into three corresponding collections for us.

> <hands-on-title>Collapsing each collection into a single dataset</hands-on-title>
>
> 1. **Collapse Collection** {% icon tool %} of Illumina-sequenced *forward* reads
>    - {% icon param-collection %} *"Collection of files to collapse into single dataset"*:
>      the collection of filtered Illumina-sequenced forward reads in fastq format,
>      first output of first **Samtools fastx** {% icon tool %} run
>    - *"Keep one header line"*: `No`
>    - *"Prepend File name"*: `No`
>
> 2. **Collapse Collection** {% icon tool %} of Illumina-sequenced *reverse* reads
>    - {% icon param-collection %} *"Collection of files to collapse into single dataset"*:
>      the collection of filtered Illumina-sequenced reverse reads in fastq format,
>      second output of first **Samtools fastx** {% icon tool %} run
>    - *"Keep one header line"*: `No`
>    - *"Prepend File name"*: `No`
>
> 3. **Collapse Collection** {% icon tool %} of Nanopore-sequenced reads
>    - {% icon param-collection %} *"Collection of files to collapse into single dataset"*:
>      the collection of filtered Nanopore-sequenced reads in fastq format,
>      output of second **Samtools fastx** {% icon tool %} run
>    - *"Keep one header line"*: `No`
>    - *"Prepend File name"*: `No`
>
{: .hands_on}


# SARS-CoV-2 genome assembly

## Optional: Subsampling of reads

The actual assembly of the sequenced reads represents the real bottleneck in
this tutorial. Assembly of the full set of sequences can easily take 10 hours
and would best be conducted overnight.

If you do not have that much time, you should downsample the Illumina-sequenced
combined reads now. Which will reduce the time required to finish the
subsequent assembly step to approximately 1-2 hours.

> <comment-title>If you are in a hurry</comment-title>
> The downsampling parameters below have been chosen to have minimal impact on
> the assembly results. Further speed-ups are certainly possible, but will
> likely lead to poor assembly outcomes.
{: .comment}

> <hands-on-title>Subsampling of paired-end short-reads data</hands-on-title>
>
> 1. **seqtk_sample** {% icon tool %} with the following parameters
>    - {% icon param-files %} *"Input FASTA/Q file"*: The two datasets with the
>      combined Illumina-sequenced forward and reverse reads, outputs of the
>      first and the second run of **Collapse Collection**
>    - *"RNG seed"*: 4
>    - *"Subsample (decimal fraction or number)"*: 0.1
>
{: .hands_on}

## Create assembly

> <hands-on-title>Assembly of SARS-CoV2 genome</hands-on-title>
>
> 1. **Create assemblies with Unicycler** {% icon tool %} with the following parameters
>    - *"Paired or Single end data?"*: `Paired`
>      - {% icon param-file %} *"Select first set of reads"*: the combined
>        Illumina-sequenced forward reads from all samples, output of the
>        first **Collapse Collection** {% icon tool %} run (or first output of
>        **seqtk_sample** {% icon tool %} for a subsample-based assembly)
>      - {% icon param-file %} *"Select second set of reads"*: the combined
>        Illumina-sequenced reverse reads from all samples, output of the
>        second **Collapse Collection** {% icon tool %} run  (or second output
>        of **seqtk_sample** {% icon tool %} for a subsample-based assembly)
>
>    - {% icon param-file %} *"Select long reads. If there are no long reads, leave this empty"*:
>      Nanpore-sequenced reads from all samples, output of the third
>      **Collapse Collection** {% icon tool %} run
>    - *"Select Bridging mode"*: `Normal (moderate contig size and misassembly rate)`
>    - *"Exclude contigs from the FASTA file which are shorter than this length (bp)"*:
>      `100`
>    - *"The expected number of linear (i.e. non-circular) sequences in the assembly"*:
>      `1`
>
{: .hands_on}

## Explore assembly

The **Unicycler** {% icon tool %} run above should produce two output datasets:

- a final assembly in FASTA format
- an assembly graph

Of these, the assembly graph is more information-rich because it not only
contains the sequences of *all* assembled fragments (including the ones shorter
than the threshold length defined for inclusion of the fragments into the FASTA
output), but also indicates the relative average coverage of the fragments by
sequenced reads and how some of the fragments could potentially be bridged
after resolving ambiguities manually.

### Assembly inspection with Bandage

On the downside, the assembly graph format takes some getting used to before
you can make sense out of the information it provides.

This issue can be alleviated through the use of **Bandage**, a package for
exploring assembly graphs through summary reports and visualizations of their
contents.

> <hands-on-title>Assembly stats and visualization with Bandage</hands-on-title>
>
> 1. **Bandage Info** {% icon tool %} with the following parameters
>    - {% icon param-file %} *"Graphical Fragment Assembly"*: the assembly graph dataset produced by
>      **Unicycler**
>    - *"Output the information in a single tab-delimited line starting with the graph file"*:
>      `No`
>
> 2. **Bandage Image** {% icon tool %} with the following parameters
>    - {% icon param-file %} *"Graphical Fragment Assembly"*: the assembly graph dataset produced by
>      **Unicycler**
>    - *"Node name labels?"*: `Yes`
>    - *"Node length labels?"*: `Yes`
>
{: .hands_on}

Let us inspect the summary report produced by **Bandage Info** {% icon tool %}
first:

You may be rather disappointed by the large *percentage of of dead ends* in the
assembly graph (in general, lower is better here), and by the correspondingly
large *node count*. After all, should the viral sequence not be encoded on a
single small contig (a quick check at
[Wikipedia](https://en.wikipedia.org/wiki/Coronavirus#Genome) reveals that
coronaviruses have genomes in the size range of 30kb)?

On the other hand, there is the *Longest node* of 29768 bp of assembled
sequence, which is suspiciously close to the expected genome size, but a much
larger *Estimated sequence length*.

Next, take a look at the assembly graph visualization generated by **Bandage
Image** {% icon tool %} to see if that tells us more:

Indeed, this output shows that **Unicycler** {% icon tool %} managed to
assemble a good number of contigs of moderate size, then had trouble with a
number of really small fragments that it could only assemble with lots of
ambiguities (leading to that ugly clutter of nodes in the top row of the
image). Those small fragments will probably be hard to make sense of, but the
manageable list of moderate-size contigs (nodes 1-23, 25, 26) is encouraging.

Of these, node 1 is the longest node mentioned in the report with a size close
to our expectations.

Node 2 looks peculiar since **Unicycler** claims it is circular, while
Coronavirus genomes are known to be linear.

### Check origin of assembled sequences with BLAST

While we could view the actual contents of the assembly graph output of
**Unicycler** {% icon tool %} and extract node sequences of interest from it
(the longest node and that circular one could be a start), things are much
easier if we work with the FASTA output of **Unicycler** instead.

From the visualization with **Bandage Image** {% icon tool %} we know that the
separately assembled nodes are all longer than 1000 bp. We can extract those
sequences based on the length threshold in Galaxy, then BLAST all retained
sequences in one go.

> <hands-on-title>Filter FASTA sequences by their length</hands-on-title>
>
> 1. **Filter sequences by length** {% icon tool %} with the following parameters
>    - {% icon param-file %} *"Fasta file"*: the FASTA output produced by
>      **Unicycler**
>    - *"Minimal length"*: `1000`
>
>    This outputs a new FASTA datasets with only the sequences satisfying our
>    length threshold.
>
> > <tip-title>Apply length filters after instead of during assembly</tip-title>
> > You may have noted that in the **Unicycler** {% icon tool %} run we kept
> > the tool's *"Exclude contigs from the FASTA file which are shorter than
> > this length (bp)"* option at its default value of `100` instead of using
> > the 1,000 bp threshold there directly to save a step in the analysis.
> >
> > The reason we did this is that normally you will not know the exact length
> > threshold you want until *after* having explored the generated assembly.
> >
> > Length-filtering some FASTA sequences is a trivial process that takes very
> > little time, but you would not want to rerun an hours-long assembly job
> > just because you accidentally stripped some interesting assembled sequences
> > from the output.
> {: .tip}
>
{: .hands_on}

> <hands-on-title>NCBI BLAST of multiple contigs</hands-on-title>
>
> 1. View the output of **Filter sequences by length** {% icon tool %} by
>    clicking the {% icon galaxy-eye %} (eye) icon attached to that dataset.
> 2. Click into the middle panel, which should now display the content of the
>    dataset, select all sequences by pressing <kbd>Ctrl</kbd>+<kbd>A</kbd>,
>    then copy the selection to the clipboard with <kbd>Ctrl</kbd>+<kbd>C</kbd>
> 3. Head over to the
>    [NCBI BLAST](https://blast.ncbi.nlm.nih.gov/Blast.cgi?PAGE=MegaBlast&PROGRAM=blastn)
>    service, paste the copied content into the `Enter Query Sequence` text
>    box and click `BLAST` at the bottom of the form.
> 4. Wait for the *BLAST* run to finish.
> 5. On the results page, look for the drop-down menu next to `Results for`.
>    It lets you toggle the BLAST hits list further down to include only the
>    matches to individual sequences from your multi-sequence query.
>
{: .hands_on}

Now take a bit of time to explore the BLAST hits uncovered for some of your
assembled nodes. Pay attention, specifically, to the node with the longest
sequence (node #1) and the circular node #2, but also investigate the results
for a few others.

> <question-title></question-title>
>
> 1. Which genome is represented by node #1?
> 2. Which genome corresponds to node #2? Does this finding remind you of
>    something you have learnt before?
> 3. What do most other node sequences have in common? Do these additinal
>    findings make sense?
>
> > <solution-title></solution-title>
> > 1. The sequence of node #1 is the assembled SARS-CoV-2 sequence we are
> >    looking for. It is a perfect match to various SARS-CoV-2 genome
> >    sequences found in Genbank over the entire assembled length, and we have
> >    been able to assemble almost the entire genome even from the subsampled
> >    sequencing data.
> > 2. The circular sequence of node #2 corresponds to the 5,386 bp genome of
> >    bacteriophage phiX174. As explained in the more general
> >    [Unicycler assembly](../unicycler-assembly/tutorial.html) tutorial,
> >    this genome is often used as a spike-in in Illumina sequencing.
> >    Finding the complete sequence here is, thus,  another indication that
> >    our analysis worked and produced meaningful results.
> > 3. Almost all other assembled sequences appear to represent parts of
> >    bacterial genomes. The only exceptions are the node #10 sequence, for
> >    which no significant BLAST hits could be found, and the node #11
> >    sequence, which represents a small stretch of left-over human genomic
> >    DNA, which seems to have survived our subtraction approach.
> >
> >    What all the bacterial genomes have in common is that they represent
> >    genera of bacteria that are known to colonize the oral cavity and
> >    mucosa. Since all samples in this analysis are BALF samples the presence
> >    of DNA from such bacteria should not be surprising. In addition, some
> >    members of the identified genera are known as opportunistic pathogens.
> >    In particular, members of the genus *Prevotella* can infect the
> >    respiratory tract and contribute to inflammation under anaerobic
> >    conditions caused by primary infections. Hence, an alternative
> >    explanation for the presence of some of these sequences in the samples
> >    might be that the corresponding bacteria contributed to the clinical
> >    picture of some of the Covid-19 patients they were obtained from.
> {: .solution}
>
{: .question}

# Conclusion


The power of modern genome assembly tools is remarkable, and so is their
robustness in the face of data of metagenomic nature. Assembling reads derived
from a virus and a good handful of copurified bacteria back into separate
contigs is a challenging task, which Unicycler solved without major issues!

Nevertheless, good quality assemblies still rely on proper preprocessing and
filtering to reduce the number of misassembly events, ambiguous assemblies and
of incorporation of sequencing errors into the final assembly.


topic: assembly
tutorial: vgp_workflow_training
title: Using the VGP workflows to assemble a vertebrate genome with HiFi and Hi-C
difficulty: intermediate
time_estimation: 2h
tools_json: ["Quast", "toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1"]
requirements_json: []
tags_json: ["pacbio", "eukaryote", "VGP", "biodiversity"]
content_hash: 60c1d7629b38f23df3495fcaa7be2328
last_modified: 2026-05-27T09:01:47.818367
zenodo_link: https://zenodo.org/record/5887339
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/vgp_workflow_training/tutorial.html

content:
The {VGP}, a project of the {G10K} Consortium, aims to generate high-quality, near error-free, gap-free, chromosome-level, haplotype-phased, annotated reference genome assemblies for every vertebrate species ({% cite Rhie2021 %}). The VGP has developed a fully automated *de-novo* genome assembly pipeline, which uses a combination of three different technologies: Pacbio {HiFi}, {Hi-C} data, and (optionally) Bionano optical map data. The pipeline consists of nine distinct workflows. This tutorial provides a quick example of how to run these workflows for one particular scenario, which is, based on our experience, the most common: assembling genomes using {HiFi} Reads combined with {Hi-C} data (both generated from the same individual).

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Getting started on Galaxy

This tutorial assumes you are comfortable getting data into Galaxy, running jobs, managing history, etc. If you are unfamiliar with Galaxy, we recommend you visit the [Galaxy Training Network](https://training.galaxyproject.org). Consider starting with the following trainings:
- [Introduction to Galaxy]({% link topics/introduction/tutorials/introduction/slides.html %})
- [Galaxy 101]({% link topics/introduction/tutorials/galaxy-intro-101/tutorial.md %})
- [Getting Data into Galaxy]({% link topics/galaxy-interface/tutorials/get-data/slides.html %})
- [Using Dataset Collections]({% link topics/galaxy-interface/tutorials/collections/tutorial.md %})
- [Understanding the Galaxy History System]({% link topics/galaxy-interface/tutorials/history/tutorial.md %})
- [Introduction to Galaxy Analyses]({% link topics/introduction/index.md %})
- [Downloading and Deleting Data in Galaxy]({% link topics/galaxy-interface/tutorials/download-delete-data/tutorial.md %})

# The VGP-Galaxy pipeline

The {VGP} assembly pipeline has a modular organization, consisting in ten workflows (Fig. 1). It can used with the following types of input data:

|  Input data | Assembly quality  | Analysis trajectory <br>([Fig. 1)](#figure-1)|
|------|---------------|-----|
| HiFi | The minimum requirement | A |
| HiFi + HiC| Better continuity | B |
| HiFi + Bionano | Better continuity | C |
| HiFi + Hi-C + Bionano | Even better continuity | D |
| HiFi + parental data| Better haplotype resolution | E |
| HiFi + parental data + Hi-C| Better haplotype resolution and improved continuity | F |
| HiFi + parental + Bionano | Better haplotype resolution and improved continuity | G |
| HiFi + parental data + Hi-C + Bionano | Better haplotype resolution and ultimate continuity | H |

In this table, HiFi and Hi-C data are derived from the individual whose genome is being assembled. "Parental data" refers to high coverage whole genome resequencing data from the parents of the individual being assembled. Assemblies utilizing parental data for phasing are also called "*Trios*". Each combination of input datasets is supported by an *analysis trajectory*: a combination of workflows designed for generating assembly given a particular combination of inputs. These trajectories are listed in the table above and shown in the figure below. 

![The nine workflows of Galaxy assembly pipeline](../../images/vgp_assembly/VGP_workflow_modules.svg "Eight analysis trajectories are possible depending on the combination of input data. A decision on whether or not to invoke Workflow 6 is based on the analysis of QC output of workflows 3, 4, or 5. Thicker lines connecting Workflows 7, 8, and 9 represent the fact that these workflows are invoked separately for each phased assembly (once for maternal and once for paternal).")
<br>
The first stage of the pipeline is the generation of a *k*-mer profile of the raw reads to estimate genome size, heterozygosity, repetitiveness, and error rate necessary for parameterizing downstream workflows. The generation of *k*-mer counts can be done from HiFi data only (Workflow 1) or include data from parental reads for trio-based phasing (Workflow 2). The second stage is contig assembly. In addition to using only {HiFi} reads (Workflow 3), the contig-building (contiging) step can leverage {Hi-C} (Workflow 4) or parental read data (Workflow 5) to produce fully-phased haplotypes (hap1/hap2 or parental/maternal assigned haplotypes), using [`hifiasm`](https://github.com/chhylp123/hifiasm). The contiging workflows also produce a number of critical quality control (QC) metrics such as *k*-mer multiplicity profiles. Inspection of these profiles provides information to decide whether the third stage—purging of false duplication—is required. Purging (Workflow 6), using [`purge_dups`](https://github.com/dfguan/purge_dups) identifies and resolves haplotype-specific assembly segments incorrectly labeled as primary contigs, as well as heterozygous contig overlaps. This increases continuity and the quality of the final assembly. The purging stage is generally unnecessary for trio data, as thses assemblies achieve reliable haplotype resolution using *k*-mer set operations based on parental reads. Generally, HiC-phased contigs also do not need to undergo purging, but if so, then there exists Workflow 6b to purge a single haplotype at a time. The fourth stage, scaffolding, produces chromosome-level scaffolds using information provided by Bionano (Workflow 7), with [`Bionano Solve`](https://Bionano.com/software-downloads/) (optional) and Hi-C (Workflow 8) data and [`YaHS`](https://github.com/c-zhou/yahsscaffolding) algorithms. A final stage of decontamination (Workflow 9) removes exogenous sequences (*e.g.*, viral and bacterial sequences) from the scaffolded assembly. A separate workflow (WF0) is used to assemble and annotate a mitogenome.

> <comment-title>A note on data quality</comment-title>
> We suggest at least 30✕ PacBio HiFi coverage and 60✕ Hi-C coverage (both diploid coverage).
{: .comment}

# Getting the data

The following steps use PacBio {HiFi} and Illumina {Hi-C} data from baker's yeast ([*Saccharomyces cerevisiae*](https://en.wikipedia.org/wiki/Saccharomyces_cerevisiae)). The tutorial represents trajectory **B** from Fig. 1 above. For this tutorial, the first step is to get the datasets from Zenodo. Specifically, we will be uploading two datasets:

1. A set of PacBio {HiFi} reads in `fasta` format. Please note that your HiFi reads received from a sequencing center will usually be fastqsanger.gz format, but the dataset used in this tutorial has been converted to fasta for space..
2. A set of Illumina {Hi-C} reads in `fastqsanger.gz` format.

## Uploading `fasta` datasets from Zenodo

The following two steps demonstrate how to upload three PacBio {HiFi} datasets into your Galaxy history.


> <hands-on-title> Uploading <tt>FASTA</tt> datasets from Zenodo </hands-on-title>
>
> 1. Create a new history for this tutorial
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Copy the following URLs into clipboard.
>
>    (You can do this by clicking on {% icon copy %} button in the right upper corner of the box below. It will appear if you mouse over the box.)
>
>    ```
>    https://zenodo.org/record/6098306/files/HiFi_synthetic_50x_01.fasta
>    https://zenodo.org/record/6098306/files/HiFi_synthetic_50x_02.fasta
>    https://zenodo.org/record/6098306/files/HiFi_synthetic_50x_03.fasta
>    ```
>
> 3. Upload datasets into Galaxy.
>    - set the datatype to `fasta`
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md format="fasta" %}
>
>    {% snippet topics/assembly/tutorials/vgp_genome_assembly/faqs/dataset_upload_fasta_via_urls.md %}
>
{: .hands_on}


## Uploading `fastqsanger.gz` datasets from Zenodo

Illumina {Hi-C} data is uploaded in essentially the same way as shown in the following two steps.

> <warning-title> Make sure you the choose correct format!</warning-title>
> When selecting datatype in "**Type (set all)**" drop-down, make sure you select `fastqsanger` or `fastqsanger.gz` BUT NOT `fastqcssanger` or anything else!
{: .warning}

> <hands-on-title> Uploading <tt>fastqsanger.gz</tt> datasets from Zenodo </hands-on-title>
>
> 1. Copy the following URLs into clipboard.
>     - you can do this by clicking on {% icon copy %} button in the right upper corner of the box below. It will appear if you mouse over the box.
>
>    ```
>    https://zenodo.org/record/5550653/files/SRR7126301_1.fastq.gz
>    https://zenodo.org/record/5550653/files/SRR7126301_2.fastq.gz
>    ```
>
> 2. Upload datasets into Galaxy.
>    - set the datatype to `fastqsanger.gz`
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md format="fasta" %}
>
>    {% snippet topics/assembly/tutorials/vgp_genome_assembly/faqs/dataset_upload_fastqsanger_via_urls.md %}
>
>
>
{: .hands_on}



> <warning-title>These datasets are large!</warning-title>
> Hi-C datasets are large. It will take some time (~15 min) for them to be fully uploaded. Please, be patient.
{: .warning}

## Organizing the data

If everything goes smoothly, your history will look like shown in Fig. 4 below. The three {HiFi} fasta files and the two Hi-C files are better represented as collections: {collection}. Paired data such as Hi-C reads are represented by a specific type of collection called paired collection: {paired-collection}
Also, importantly, the workflows we will be using for the analysis of our data takes collections as inputs (they do not access individual datasets). So let's create collections using steps outlines in the Tips {% icon tip %} that you can find below Fig. 4.

![AfterUpload](../../images/vgp_assembly/making_list.svg "History after uploading HiFi and Hi-C data (left). The creation of lists (collections) combines datasets into two items called 'HiFi data' and 'Hi-C data'(right). See below for instruction on how to make these collections.")

{% snippet faqs/galaxy/collections_build_list.md n=3 %}

{% snippet faqs/galaxy/collections_build_list_paired.md n=2%}

> <details-title>Other ways to upload the data</details-title>
> You can obviously upload your own datasets via URLs as illustrated above or from your own computer. In addition, you can upload data from a major repository called [GenomeArk](https://genomeark.org). GenomeArk is integrated directly into Galaxy Upload. To use GenomeArk following the steps in the Tip {% icon tip %} below:
>
> {% snippet faqs/galaxy/datasets_upload_from_genomeark.md %}
{: .details}


Once we have imported the datasets, the next step is to import the workflows necessary for the analysis of our data from [DockStore](https://dockstore.org).

# Importing workflows

All analyses described in this tutorial are performed using *workflows*--chains of tools--shown in [Fig. 1](#figure-1). Specifically, we will use four workflows corresponding to analysis trajectory **B**: 1, 4, 6, and 8. To use these four workflows you need to import them into your Galaxy account following the steps below. Note: these are not necessarily the latest versions of the actual workflows, but versions that have been tested for this tutorial. To see the latest versions, see the [IWC VGP workflows page](https://iwc.galaxyproject.org/?filter=Vertebrate+Genome+Project/) and click on the workflows for details and links to import them. **Alternatively, for each section of the tutorial, there will be a "Launch [workflow] (View on Dockstore)" link at the beginning, which you can use to launch the workflow.**

# Performing the assembly

Workflows listed in [Fig. 1](#figure-1) support a variety of "analysis trajectories". The majority of species that were sequenced by the {VGP} usually contain {HiFi} reads for the individual being sequenced supplemented with {Hi-C} data. As a result most assemblies performed by us follow the trajectory **B**. This is why this tutorial was designed to follow this trajectory as well.

## Genome profile analysis (WF1)

Before the assembly can be run, we need to collect metrics on the properties of the genome under consideration, such as the expected genome size according to our data. The present pipeline uses **Meryl** for generating the *k*-mer database and **Genomescope2** for determining genome characteristics based on a *k*-mer analysis.

### Launching the workflow

{% snippet faqs/galaxy/workflows_run_ds.md title="Genome profile analysis (WF1)" dockstore_id="github.com/iwc-workflows/kmer-profiling-hifi-VGP1/main" version="v0.9" %}

> <hands-on-title> Running <i>K</i>-mer profile analysis workflow </hands-on-title>
>
> 1. **Identify inputs**
>
>    The profiling workflow takes the following inputs:
>   
>    1. Species Name (Not used in the analysis, for documentation purpose only)
>    2. Assembly Name  (For documentation purpose only)
>    3. {HiFi} reads as a collection
>    4. *K*-mer length
>    5. Ploidy
>
> 2. **Launch *k*-mer profiling workflow**
>
>    1. Click on the **Workflow** menu, located in the top bar
>    2. Click on the {% icon workflow-run %} **Run workflow** buttom corresponding to `kmer-profiling-hifi-VGP1`
>    3. In the **Workflow: kmer-profiling-hifi-VGP1** menu:
>        - *Species Name*: `Saccharomyces_cerevisiae`
>        - *Assembly Name*: `gta26`
>        - {% icon param-collection %} "*Collection of Pacbio Data*": `HiFi data` collection
>        - "*K-mer length*": `31`
>        - "*Ploidy*": `2`
>    4. Click on the {% icon workflow-run %} **Run workflow** button
>   
>    This should look like this:
>
>    ![Parameters of *k*-mer profiling workflow](../../images/vgp_assembly/wf1_launch_ui.png  "Workflow main menu. The workflow menu lists all the workflows that have been imported. It provides useful information for organizing the workflows, such as last update and the tags. The worklows can be run by clicking in the play icon, marked in red in the image.")
>
>    > <comment-title>K-mer length</comment-title>
>    > In this tutorial, we are using a *k*-mer length of 31. This can vary, but the VGP pipeline tends to use a *k*-mer length of 21, which tends to work well for most mammalian-size genomes. There is more discussion about *k*-mer length trade-offs in the extended VGP pipeline tutorial.
>    {: .comment}
>
>
> 3. **Refill your coffee**
>
>    Assembly is not exactly an instantaneous type of analysis - this workflow will take approximately 15 minutes to complete. The same is true for all the analyses in this tutorial.
>
{: .hands_on}

### Interpreting the results

Once the workflow has finished, we can evaluate the transformed linear plot generated by [**Genomescope**](https://github.com/schatzlab/Genomescope), which includes valuable information such as the observed *k*-mer profile, fitted models, and estimated parameters. This file corresponds to the dataset `18` in this [history](https://usegalaxy.org/u/delphinel/h/vertebrate-genome-assembly-training). 

![Genomescope plot described further in caption](../../images/vgp_assembly/genomescope_plot.png "GenomeScope2 <i>k</i>-mer profile. The first peak located at about 25&times; corresponds to the heterozygous peak. The second peak at 50&times; corresponds to the homozygous peak. The plot also includes information about the inferred total genome length (len), genome unique length percent (uniq), overall heterozygosity rate (ab), mean <i>k</i>-mer coverage for heterozygous bases (kcov), read error rate (err), average rate of read duplications (dup) and <i>k</i>-mer size (k).")

This distribution is the result of the Poisson process underlying the generation of sequencing reads. As we can see, the *k*-mer profile follows a bimodal distribution, indicative of a diploid genome. The distribution is consistent with the theoretical diploid model (model fit > 93%). Low frequency *k*-mers are the result of sequencing errors, and are indicated by the red line. Genomescope2 estimated a haploid genome size of around 11.7 Mbp, a value reasonably close to the *Saccharomyces* genome size. We are using the transformed linear plot because it shows the ploidy better by reducing low ccoverage peaks (like *k*-mers containing errors) and increasing higher coverage peaks ({% cite Ranallo_Benavidez_2020 %}). 

It is worth noting that the genome characteristics such as length, error percentage, etc., are based on the GenomeScope2 model, which is the black line in the plot. If the model (black line) does not fit your observed data (blue bars), then these estimated characteristics might be very off. In the case of this tutorial, the model is a good fit to our data, so we can trust the estimates.

The workflow also generate a variety of quality control for your reads:

- A heatmap generated by Mash Dist. This map shows the distance between your samples. This is a sanity check to verify all the read files belong to the same species. The tool developers({% cite ondov2016mash %}) identified the following distance breaks:
  - Same Species: <0.05
  - Same Genus: 0.05 to 0.34
  - Different Genus: >0.34
- A Rdeval report including statistics on reads, such as length, quality and base composition.


> <question-title>Heatmap</question-title>
>
> 1. Look at the heatmap output. What is the highest Mash distance between our read libraries?
> 2. Do all our libraries belong to the same species?
>
> > <solution-title></solution-title>
> > 
> > ![Genomescope plot described further in caption](../../images/vgp_assembly/masmap_heatmap.png "Mash Dist Heatmap. Each square is colored according to the distance between two set of reads. The diagonal is brown since the distance of one set with itself is 0. The closets sets are `01` and `03` in light blue (distance ~0.03), and `02` is a bit more distant with a distance of ~0.04 (dark teal)")
> > 
> > 1. The highest Mash distance between our read libraries is about 0.04 (see color scale legend).
> > 2. Our libraries belong to the same species, since the threshold is 0.05.
> >
> {: .solution}
>
{: .question}

## Assembly (contiging) with `hifiasm` (WF4)

To generate {contigs} we will use the [**hifiasm**](https://github.com/chhylp123/hifiasm) assembler. It is a part of the "Assembly with HiC (WF4)" workflow . This workflow uses **hifiasm** (HiC mode) to generate HiC-phased haplotypes (hap1 and hap2). This is in contrast to its default mode, which generates primary and alternate pseudohaplotype assemblies. This workflow includes three tools for evaluating assembly quality: [**gfastats**](https://github.com/vgl-hub/gfastats), [**BUSCO**](https://busco.ezlab.org/) and [**Merqury**](https://github.com/marbl/merqury).

### Launching the workflow

{% snippet faqs/galaxy/workflows_run_ds.md title="Assembly HiFi-HiC phasing (WF4)" dockstore_id="github.com/iwc-workflows/Assembly-Hifi-HiC-phasing-VGP4/main" version="v0.6" %}

> <hands-on-title> Launching assembly (contiging) workflow </hands-on-title>
>
> 1. **Identify inputs**
>
>    The assembly workflow takes the following inputs:
>
>    1. {HiFi} reads as a collection
>    2. Whether we want to trim the Hi-C reads or not (Sometimes necessary with Arima Hi-C)
>    3. Hi-C reads as a paired collection
>    4. `Genomescope` Summary generated by previous (*k*-mer profiling) workflow
>    5. Meryl *k*-mer database generated by previous (*k*-mer profiling) workflow
>    6. Busco Database
>    7. Busco lineage
>    8. Name for Haplotype 1 (Used to generate QC files)
>    9. Name for Haplotype 2
>    10. Bits for bloom filter: Parameter impacting memory usage. Modify for large genomes.
>    11. Homozygous Read Coverage. Optional: set if you are unsatisfied with the Genomescope model.
>    12. `Genomescope` Model Parameters generated by previous (*k*-mer profiling) workflow
>
> 2. **Launch the workflow**
>
>    1. Click on the **Workflow** menu, located in the top bar
>    2. Click on the {% icon workflow-run %} **Run workflow** button corresponding to `Assembly-Hifi-HiC-phasing-VGP4`
>    3. In the **Workflow: Assembly-Hifi-HiC-phasing-VGP4** menu fill the following parameters:
>        - *Species Name*: `Saccharomyces_cerevisiae`
>        - *Assembly Name*: `gta26`
>        - {% icon param-collection %} "*Pacbio Reads*": `HiFi data` collection
>        - {% icon param-file %} "*Hi-C reads*": `Hi-C data`
>        - {% icon param-file %} "*GenomeScope Summary*": `GenomeScope on data X Summary` (contains tag "`GenomeScopeSummary`")
>        - {% icon param-file %} "*Meryl database*": `Meryl on data X: read-db.mertyldb`: the Meryl *k*-mer database (contains tag "`MerylDatabase`")
>        - "*Database for Busco Lineage*": `Busco v5 Lineage Datasets` (or the latest version available in the instance)
>        - "*Provide lineage for BUSCO (e.g., Vertebrata)*": `Ascomycota`
>        - {% icon param-file %} "*GenomeScope Model Parameters*": `GenomeScope on data X Model parameters` (contains tag "`GenomeScopeParameters`")
>    4. Click on the {% icon workflow-run %} **Run workflow** button
> 
{: .hands_on}

> <comment-title>A note about "Homozygous Read Coverage"</comment-title>
>
>  Hifiasm tries to estimate the homozygous read coverage, but sometimes it can mistake a different peak as the homozygous coverage peak. To get around this, the VGP workflows calculate the homozygous coverage from the haploid coverage estimate from the GenomeScope outputs generated in the *k*-mer profiling workflow. However, sometimes GenomeScope can also misidentify the haploid peak, leading to it feeding an incorrect homozygous read coverage value into hifiasm. *If this is the case, you can run the hifiasm workflows but specify the homozygous read coverage yourself. Otherwise, this parameter is fine to leave blank to let the workflows try to get it right on their own.*
>
{: .comment}



### Interpreting the results

> <warning-title>There will be two assemblies!</warning-title>
> Because we are running hifiasm in HiC-phasing mode, it will produce two assemblies: hap1 and hap2!
{: .warning}

Let's have a look at the stats generated by **gfastats**. This output summarizes some main assembly statistics, such as contig number, N50, assembly length, etc. The workflow provide a joined table to display the statistics for both haplotype assemblies side-by-side. Below we provide a partial output of this file called `Assembly statistics for Hap1 and Hap2` in your history:

>| Statistic | Hap 1 | Hap 2 |
>|-----------|----------:|------:|
>| # contigs | 16 | 17 |
>| Total contig length | 11,304,582 | 12,160,988 |
>| Average contig length | 706,536.38  | 715,352.24 |
>| Contig N50 |  922,430 | 923,452 |
>| Contig auN | 895,015.76  | 904,515.33 |
>| Contig L50 | 5 | 6 |
>| Contig NG50 | 813,311 | 923,452 |
>| Contig auNG | 861,281.68 | 936,364.22 |
>| Contig LG50 | 6 | 6 |
>| Largest contig | 1,532,843 | 1,531,728 |
>| Smallest contig | 185,224 | 85,850 |
{: .matrix}

According to the report, both assemblies are quite similar; the hap1 assembly includes 16 {contigs}, whose cumulative length is around 11.3 Mbp. The hap2 assembly includes 17 contigs, whose total length is 12.1Mbp. Both assemblies come close to the estimated genome size, which is as expected since we used hifiasm-HiC mode to generate phased assemblies, and this lowers the chance of false duplications that can inflate assembly size.

> <comment-title>Are you working with pri/alt assemblies?</comment-title>
> This tutorial uses the hifiasm-HiC workflow, which generates phased hap1 and hap2 assemblies. The phasing helps lower the chance of false duplications, since the phasing information helps the assembler know which genomic variation is heterozygosity at the same locus versus being two different loci entirely. If you are working with primary/alternate assemblies (especially if there is no internal purging in the initial assembly), you can expect higher false duplication rates than we observe here with the yeast HiC hap1/hap2.
{: .comment}

> <question-title></question-title>
>
> 1. What is the longest contig in the hap1 assembly? And in the hap2 one?
> 2. What is the N50 of the hap2 assembly?
>
> > <solution-title></solution-title>
> >
> > 1. The longest contig in the hap1 assembly is 1,532,843 bp, and 1,531,728 bp in the hap2 assembly.
> > 2. The N50 of the hap2 assembly is 923,452 bp.
> >
> {: .solution}
>
{: .question}

Next, we are going to evaluate the outputs generated by **Compleasm**. This tool uses BUSCO to provides qualitative assessment of the completeness of a genome assembly in terms of expected gene content. It relies on the analysis of genes that should be present only once in a complete assembly or gene set, while allowing for rare gene duplications or losses ({% cite Simo2015 %}).
The results are classified into five categories: *complete and single-copy* (S), *complete and duplicated*(D), *fragmented*(F), *incomplete*(I) and *missing*(M).

> <question-title></question-title>
>
> 1. How many BUSCO complete genes have been identified in the hap1 assembly?
> 2. How many BUSCO genes are absent in the hap1 assembly?
>
> > <solution-title></solution-title>
> >
> > See the Compleasm Summary:
> >
> > ```
> > ## lineage: ascomycota_odb10
> > S:86.75%, 1480
> > D:2.70%, 46
> > F:1.29%, 22
> > I:0.00%, 0
> > M:9.26%, 158
> > N:1706
> > ```
> >
> > 1. According to the summary, our assembly contains 1,526 complete BUSCO genes -- this includes ones that are single-copy (S:86.75%, 1480) and ones that are duplicated (D:2.70%, 46).
> > 2. 158 BUSCO genes are missing.
> >
> {: .solution}
>
{: .question}

Despite **BUSCO** being robust for species that have been widely studied, it can be inaccurate when the newly assembled genome belongs to a taxonomic group that is not well represented in [OrthoDB](https://www.orthodb.org/). `Merqury` provides a complementary approach for assessing assembly quality in a reference-free manner via *k*-mer copy number analysis. Specifically, it takes our hap1 as the first genome assembly, hap2 as the second genome assembly, and the merylDB generated previously from our sequencing reads for *k*-mer counts.

By default, `Merqury` generates three collections as output: stats (completeness stats), plots and {QV} stats. Let's first have a look at the copy number (CN) spectrum plot, known as the *spectra-cn* plot. The spectra-cn plot looks at both of your assemblies (here, your two haplotypes) taken *together* (fig. 6a). 

![Figure 6: Merqury spectra-cn plot for initial yeast contigs](../../images/vgp_assembly/yeast_c_merqury_cn.svg "Merqury CN plot for yeast assemblies. The plot tracks the multiplicity of each <i>k</i>-mer found in the read set and colors it by the number of times it is found in a given assembly. Merqury connects the midpoint of each histogram bin with a line, giving the illusion of a smooth curve. <b>a)</b>. <i>K</i>-mer distribution of both haplotypes. <b>b)</b>. <i>K</i>-mer distribution of an individual haplotype (hap2)."){:width="100%"}

Our haplotypes look clean! In the spectra-cn plot for both haplotypes, the peaks are where we should expect them. There is a peak of *k*-mers present at 1-copy (so, only in either hap1 or hap2) with *k*-mer multiplicity of ~25, corresponding to heterozygous coverage. These are our heterozygous alleles being represented only once in our diplolid assemblies, which matches with their haploid coverage. We also have a peak of 2-copy *k*-mers present at diploid coverage, around ~50. This looks good so far, but we should also look at *k*-mer multiplicity for each individual haplotype separately, not just combined as they were in the previous plot. Figure 6b shows that most of the *k*-mers in our assembly are 1-copy -- we have one copy of all the homozygous regions (the ones with 50x coverage) and of about half the heterozygous regions (the ones with 25x coverage). At the 25x coverage point, there is a "read-only" peak, which is expected, as these are the alternative alleles for those heterozygous loci, and those *k*-mers are likely in the other assembly, since they were not missing from the overall spectra-cn plot.


## Hi-C scaffolding (WF8)

In this final stage, we will run the **Scaffolding HiC YAHS (WF8)** workflow, which uses the fact that the contact frequency between a pair of loci strongly correlates with the one-dimensional distance between them (*i.e.*, linear distance on a chromosome). This information allows [**YAHS**](https://github.com/c-zhou/yahs) -- the main tool in this workflow -- to generate scaffolds that are often chromosome-sized.

### Launching Hi-C scaffolding workflow

> <warning-title>The scaffolding workflow is run on <b>ONE</b> haplotype at a time.</warning-title>
> Contiging (VGP4) works with both (hap1/hap2, primary/alternate) assemblies simultaneously. This is not the case for contiging -- it has to be run independently for each haplotype assembly. In this example (below) we run contiging on the hap1 assembly only. You can run the same analysis on the second haplotype by replacing hap1 with hap2. 
{: .warning}

{% snippet faqs/galaxy/workflows_run_ds.md title="Scaffolding HiC (WF8)" dockstore_id="github.com/iwc-workflows/Scaffolding-HiC-VGP8/main" version="v3.5" %}

> <hands-on-title> Launching Hi-C scaffolding workflow </hands-on-title>
>
> 1. **Identify inputs**
>
>    The scaffolding workflow takes the following inputs:
>
>    1. An assembly graph
>    2. Forward Hi-C reads
>    3. Reverse Hi-C reads
>    4. Estimated genome size parsed from GenomeScope summary by the previous run of assembly workflow (VGP4).
>    5. Restriction enzymes used in Hi-C library preparation procedure
>    6. Busco Database
>    7. Busco lineage
>
> 2. **Launch scaffolding workflow (WF8)**
>
>    1. Click on the **Workflow** menu, located in the top bar
>    2. Click on the {% icon workflow-run %} **Run workflow** button corresponding to `Scaffolding-HiC-VGP8`
>    3. In the **Workflow: Scaffolding-HiC-VGP8** menu:
>        - {% icon param-file %} "*input GFA*": `usable hap1 gfa` Output of the contiging workflow (VGP4) with a tag `hic_hap1_gfa` for hap1 assembly.
>        - "*Database for Busco Lineage*": `Busco v5 Lineage Datasets` (or the latest version available in the instance)
>        - "*Provide lineage for BUSCO (e.g., Vertebrata)*": `Ascomycota`
>        - {% icon param-file %} "*HiC forward reads*": `Hi-C forward reads`
>        - {% icon param-file %} "*HiC reverse reads*": `Hi-C reverse reads`
>        - "*Restriction enzymes*": `Dovetail Omni-C: enzyme-free prep` For this tutorial, we'll use the Omni-C option as it is the equivalent of not specifying any restriction enzyme cutsites, but for your own data you would want to select the appropriate option. 
>        - {% icon param-file %} "*Estimated genome size - Parameter File*": `Estimated genome size`: An output of the contiging workflow (VGP4) with a tag `estimated_genome_size`.
>    4. Click on the {% icon workflow-run %} **Run workflow** button
{: .hands_on}

### Interpreting the results

In order to evaluate the Hi-C hybrid scaffolding, we are going to compare the contact maps before and after running the HiC hybrid scaffolding workflow (Fig. below). They will have the following tags:

- Before scaffolding: `pretext_s1`
- After scaffolding: `pretext_s2`

Below is the comparison of the two maps obtained from the yeast tutorial data versus real example from a zebra finch (*Taeniopygia guttata*) genome assembly:

![Pretext final contact map](../../images/vgp_assembly/hi-c_pretext_final.svg "Hi-C maps generated by Pretext before and after scaffolding with Hi-C data. The red circles indicate the differences between the contact maps generated before and after Hi-C hybrid scaffolding. The bottom two panels show results of scaffolding on zebra finch where scaffolding dramatically decreases the number of segments by merging multiple contigs into scaffolds.")

The regions marked with red circles highlight the most notable difference between the two contact maps, where inversion has been fixed.

# Conclusion

To sum up, it is worthwhile to compare the final assembly with the [_S. cerevisiae_ S288C reference genome](https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/146/045/GCF_000146045.2_R64/GCF_000146045.2_R64_assembly_stats.txt):

![Quast plot](../../images/vgp_assembly/quast_plot.png "Cumulative continuity plot comparing assembly generated here (red line) with existing yeast reference (black dotted line). Our assembly is slightly smaller (11,304,507 bp versus 12,071,326. Our assembly is lacking the mitochondrial genome (~86 kb) because the initial data does not include mitochondrial reads. This is partially responsible for this discrepancy. ")

With respect to the total sequence length, we can conclude that the size of our genome assembly is very similar to the reference genome. It is noteworthy that the reference genome consists of 17 sequences, while our assembly includes only 16 chromosomes. This is due to the fact that the reference genome also includes the sequence of the mitochondrial DNA, which consists of 85,779 bp. (The above comparison is performed using {% tool [Quast](toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy1) %} using Primary assembly generated with scaffolding workflow (WF8) and yeast reference.)

![Comparison reference genome](../../images/vgp_assembly/hi-c_pretext_conclusion.svg "Comparison between contact maps generated using the final Primary assembly from this tutorial (left) and the reference genome (right).")

If we compare the contact map of our assembled genome with the reference assembly (Fig. above), we can see that the two are indistinguishable, suggesting that we have generated a chromosome level genome assembly.


topic: assembly
tutorial: mitochondrion-assembly
title: Assembly of the mitochondrial genome from PacBio HiFi reads
difficulty: intermediate
time_estimation: 1H
tools_json: ["MitoHiFi", "toolshed.g2.bx.psu.edu/repos/bgruening/mitohifi/mitohifi/3.2.3+galaxy0"]
requirements_json: []
tags_json: ["biodiversity"]
content_hash: e9dbe499cf93460f88e124ca5be210e3
last_modified: 2026-05-27T09:01:47.815108
zenodo_link: https://zenodo.org/records/13345315
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/mitochondrion-assembly/tutorial.html

content:
This tutorial will show you how to assemble a mitochondrial genome from PacBio HiFi data using MitoHiFi {% cite uliano2023 %}. 
Combined with the tutorials "Using the VGP workflows to assemble a vertebrate genome with HiFi and Hi-C data" and "Decontamination of a genome assembly", this allows you to produce a reference assembly for both the nuclear and the mitochondrial DNA of a vertebrate species. 

This tutorial uses data from the Zebra Finch (*Taeniopygia guttata*) generated by the [Vertebrate Genome Project](https://vertebrategenomesproject.org/). We downsampled the reads that didn't align with the mitochondrial genome so that the tutorial can run faster. 

> <comment-title>Run this analysis on "real" data</comment-title>
>
> If you want to run this analysis on a real sequencing library generated by the [Vertebrate Genome Project](https://vertebrategenomesproject.org/) you can find the PacBio HiFi data on [Genome Ark](https://www.genomeark.org/) as a remote repository and upload it to Galaxy (available on the three main Public Galaxy instances: .org, .eu, .org.au).
>
> {% snippet faqs/galaxy/datasets_import_from_remote_files.md location="Genome Ark -> species -> Taeniopygia_guttata -> bTaeGut2 -> genomic_data -> pacbio_hifi -> *_reads.fasta.gz" %}
>
>
{: .comment}

The assembly is using the wrapped workflow MitoHiFi. 
MitoHiFi: 
- Extracts mitochondrial reads (based on a BLAST against an existing reference mitogenome) and uses Hifiasm {% cite Cheng2021 %} to assemble them. 
- Removes nuclear mitochondrial DNA sequences (NUMTs) from the potential mitogenome contigs
- Generates a circularized and annotated genome for all potential mitogenome contigs
- Selects a representative for the final mitochondrial assembly


> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}


## Get data

> <hands-on-title> Data Upload </hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo]({{ page.zenodo_link }}):
>
>    ```
>    https://zenodo.org/records/13345315/files/PacBio_reads.fastqsanger.gz
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Check that the datatype is `fastqsanger.gz`
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="fastqsanger.gz" %}
>
>
{: .hands_on}

# Download the mitogenome for a related species

To assemble the mitogenome from our PacBio Data,  MitoHiFi needs a reference mitogenome to fish out mitochondrial reads (the reads are blasted against the related reference). To download this reference, we use the tool MitoHiFi with the operation `Find a closely related species`.


> <hands-on-title>  Find a related mitogenome reference  </hands-on-title>
>
> 1. {% tool [MitoHiFi](toolshed.g2.bx.psu.edu/repos/bgruening/mitohifi/mitohifi/3.2.3+galaxy0) %} with the following parameters:
>    - *"Operation type selector"*: `Find a close-related mitochondrial reference genome`
>        - *"Species name"*: `Taeniopygia guttata` Enter the latin name of the species you are assembling
>        - *"Email"*: `your.email@service.com` Enter your email 
>        - *"Minimal appropriate length"*: `15000` As vertebrate mitochondrial genomes are typically at least 14kbp long, we are using a value in this range so that we get complete mitogenome results as our reference. 
>
>
{: .hands_on}


## Assemble the mitochondrial genome with **MitoHiFi**

> <hands-on-title>Assemble the mitochondrial genome</hands-on-title>
>
> 1. Create a collection with your PacBio HiFi Reads
>
>    {% snippet faqs/galaxy/collections_build_list.md name="PacBio Reads" datasets_description="the fastq.gz containing the HiFi reads" n="1" %}
> 
> 2. {% tool [MitoHiFi](toolshed.g2.bx.psu.edu/repos/bgruening/mitohifi/mitohifi/3.2.3+galaxy0) %} with the following parameters:
>    - *"Operation type selector"*: `Run MitoHiFi`
>        - *"Input mode"*: `Pacbio Hifi Reads`
>            - {% icon param-collection %} *"Pacbio Hifi reads"*: `PacBio Reads` (Input dataset collection)
>        - {% icon param-file %} *"Close-related mitogenome in fasta format"*: `MitoHiFi on : reference genome (FASTA)` (output of **MitoHiFi** {% icon tool %})
>        - {% icon param-file %} *"Close-related mitogenome in genbank format"*: `MitoHiFi on : reference genome (genbank)` (output of **MitoHiFi** {% icon tool %})
>        - *"Genetic code"*: `Vertebrate mitochondrial code`
>        - In *"Advanced options"*:
>            - *"Blast percentage identity"*: `70` This setting filters the potential mito contigs -- setting it to 70 means that we are retaining contigs with at least 70% of its length in the BLAST match. This parameter can be lowered if you are expecting more sequence divergence among mitogenomes of your taxa, or vice versa.
>
>
{: .hands_on}


Outputs of MitoHiFi:
- Final mitogenome (FASTA). The mitochondrial genome circularized and rotated to start at tRNA-Phe.
- Final mitogenome (genbank). The final mitogenome annotated in GenBank format.
- Final mitogenome annotation (png). The predicted genes in the final mitogenome.
- Final mitogenome coverage (png). The sequencing coverage along the final mitogenome.
- Contigs stats (TSV). Contains the statistics of your assembled mitogenomes such as the number of genes, size, whether it was circularized or not, if the sequence has frameshifts, and other metrics.
- Reads mapped to close-related mtDNA (FASTA). All and filtered by size.
- Hifiasm contigs (fasta). The results of running Hifiasm on the mitochondrial reads. 

![Image produced by MitoHiFi showing the genes annotated in our mitogenome](../../images/mito_annotation.png "Final mitogenome annotation")



# Conclusion

In this tutorial, we learned how to assemble the mitochondrial genome using PacBio HiFi reads and MitoHiFi. You can try this tutorial on your own data using the full HiFi read set you'd use for the nuclear genome assembly, since the filtering for mitochondrial reads happens within MitoHiFi.


topic: assembly
tutorial: assembly-quality-control
title: Genome Assembly Quality Control
difficulty: intermediate
time_estimation: 2h
requirements_json: []
tags_json: ["assembly", "quality control", "biodiversity"]
content_hash: e28b72c4df2e7a742ddb5dd342e046a2
last_modified: 2026-05-27T09:01:47.800295
zenodo_link: https://zenodo.org/record/6947782
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/assembly-quality-control/tutorial.html

content:
In this tutorial, we will assess the assembly quality of 2 assemblies generated with Hifiasm and Flye using PacBio HiFi reads of a species of fungi, *Saccharomyces cerevisiae* INSC1019 and compare the results with the actual reference genome [*Saccharomyces cerevisiae* S288C](https://www.ncbi.nlm.nih.gov/genome/?term=Saccharomyces%20cerevisiae).

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Get data

We will use long quality reads sequencing data from PacBio HiFi sequencing of *Saccharomyces cerevisiae* INSC1019 genome. This data is a subset of data from ENA repository [SRR13577847](https://www.ebi.ac.uk/ena/browser/view/SRR13577847?show=reads). We will also use the reference genome assembly downloaded from the [NCBI website](https://www.ncbi.nlm.nih.gov/genome/?term=Saccharomyces%20cerevisiae) and we will use it as a comparison with our own assemblies.

## Get data from Zenodo

> <hands-on-title>Data upload from Zenodo</hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo](https://zenodo.org/record/5702408)
>
>    ```
>    https://zenodo.org/record/6947782/files/GCA_000146045.2_genomic.fna
>    https://zenodo.org/record/6947782/files/Scerevisiae-INSC1019.flye.30x.fa
>    https://zenodo.org/record/6947782/files/Scerevisiae-INSC1019.hifiasm.30x.fa
>    https://zenodo.org/record/6947782/files/SRR13577847_subreads.30x.fastq.gz
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Check that the datatype is `fastqsanger.gz` for `SRR13577847_subreads.30x.fastq.gz`
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
>
{: .hands_on}

# Evaluation of assembly contiguity and correctness with **Quast**

A way to calculate metrics assembly is to use **QUAST = QUality ASsessment Tool**. Quast is a tool to evaluate genome assemblies by computing various metrics and to compare genome assembly with a reference genome. The manual of Quast is here: [Quast](http://quast.sourceforge.net/docs/manual.html#sec3)

> <comment-title>QUAST statistics</comment-title>
>
> QUAST will provide us with the following statistics:
>
> - Contigs: The total number of contigs in the assembly.
> - Largest contig: The length of the largest contig in the assembly.
> - Total length: The total number of bases in the assembly.
> - Nx: The largest contig length, *L*, such that contigs of length >= *L* account for at least *x*% of the bases of the assembly.
> - NGx: The contig length such that using equal or longer length contigs produces *x*% of the length of the reference genome, rather than *x*% of the assembly length.
> - GC content: the percentage of nitrogenous bases which are either guanine or cytosine.
>
> If a reference genome is provided, the report also indicates:
> - Genome fraction (%): total number of aligned bases in the reference genome, divided by the genome size
> - Duplication ratio: total number of aligned bases in the assembly, divided by the total number of aligned bases in the reference genome
> - NGA50: NG50 where the lengths of aligned blocks are counted instead of contig lengths. i.e., if a contig has a misassembly with respect to the reference, the contig is broken into smaller pieces. This metric is computed only if a reference genome is provided.
> LGA50: LG50 where aligned blocks are counted instead of contigs. i.e., if a contig has a misassembly with respect to the reference, the contig is broken into smaller pieces
>
> Many other metrics are available when clicking on `Extended report`!
{: .comment}

> <hands-on-title>assembly evaluation with Quast</hands-on-title>
>
> 1. {% tool [Quast](toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy0) %} with the following parameters:
>    - *"Use customized names for the input files?"*: `No, use datasets names`
>    - {% icon param-files %} *"Contigs/scaffolds file"*: `Scerevisiae-INSC1019.flye.30x.fa` and `Scerevisiae-INSC1019.hifiasm.30x.fa`
>    - *"Reads options"*: `Pacbio SMRT reads`
>        - {% icon param-file %} *"FASTQ file"*: `SRR13577847_subreads.30x.fastq.gz`
>    - *"Type of assembly"*: `Genome`
>        - *"Use a reference genome?"*: `Yes`
>            - {% icon param-file %} *"Reference genome"*: `GCA_000146045.2_genomic.fna`
>    - *"Is genome large (>100Mpb)?"*: `No`
>
{: .hands_on}

> <question-title></question-title>
>
> With the informations generated by Quast, compare the two assemblies to the reference genome
>
> 1. Which assembly is closest to the reference genome?
> 2. What is the longest contig of the Flye and Hifiasm assemblies?
> 3. What is the N50 of the Flye assembly?
> 4. Which percentage of reads mapped to each assembly?
>
> > <solution-title></solution-title>
> >
> > 1. The Flye assembly, with a genome fraction of 99.57% while Hifiasm have a low genome fraction (75.15%) which can indicate a problematic assembly.
> > 2. The longest contig is 1,438,238 bp for Flye assembly while is 1,795,653 bp for Hifiasm
> > 3. The N50 of the Flye assembly is 929,061 bp.
> > 4. A total of 99.96% of reads mapped were mapped to the Flye assembly while only 91.02% against the Hifiasm assembly.
> >
> > Without a reference genome, it would have been harder to determine that assembly with Hifiasm is problematic. Metrics on the assembly itself are not always sufficient and therefore it is necessary to use other methodologies to ensure the quality of an assembly.
> {: .solution}
>
{: .question}

# Evaluation of assembly completness

## Core genes completness with **BUSCO**

**BUSCO (Benchmarking Universal Single-Copy Orthologs)** allows a measure for quantitative assessment of genome assembly based on evolutionarily informed expectations of gene content. Details for this tool are here: [Busco website](https://busco.ezlab.org/)

> <hands-on-title>assessing assembly completness with BUSCO</hands-on-title>
>
> 1. {% tool [Busco](toolshed.g2.bx.psu.edu/repos/iuc/busco/busco/5.3.2+galaxy0) %} with the following parameters:
>    - {% icon param-files %} *"Sequences to analyse"*:`Scerevisiae-INSC1019.flye.30x.fa`, `Scerevisiae-INSC1019.hifiasm.30x.fa` and `GCA_000146045.2_genomic.fna`
>    - *"Mode"*: `Genome assemblies (DNA)`
>        - *"Use Augustus instead of Metaeuk"*: `Use Metaeuk`
>    - *"Auto-detect or select lineage?"*: `Select lineage`
>       - *"Lineage"*: `Saccharomycetes`
>    - *"Which outputs should be generated"*: `short summary text` and `summary image`
>
{: .hands_on}

> <question-title></question-title>
>
> Compare the number of BUSCO genes identified in the Flye and Hifiasm assembly to the reference genome. What do you observe ?
>
> > <solution-title></solution-title>
> >
> > Short summary generated by BUSCO indicates that reference genome contains:
> > 1. 2129 Complete BUSCOs (of which 2128 are single-copy and 49 are duplicated),
> > 2. 2 fragmented BUSCOs,
> > 3. 6 missing BUSCOs.
> >
> > Short summary generated by BUSCO indicates that Flye assembly contains:
> > 1. 2127 complete BUSCOs (2127 single-copy and 49 duplicated),
> > 2. 2 fragmented BUSCOs
> > 3. 8 missing BUSCOs.
> >
> > Short summary generated by BUSCO indicates that Hifiasm assembly contains:
> > 1. 1663 complete BUSCOs (1662 single-copy and 27 duplicated),
> > 2. 5 fragmented BUSCOs
> > 3. 469 missing BUSCOs.
> >
> >
> > BUSCO analysis confirms that Flye assembly is better than Hifiasm assembly. Flye is also similar in completness to the reference genome with similar number of complete, fragmented and missing BUSCOs genes.
> >
> {: .solution}
>
{: .question}

Despite BUSCO being robust for species that have been widely studied, it can be inaccurate when the newly assembled genome belongs to a taxonomic group that is not well represented in [OrthoDB](https://www.orthodb.org/). Even in a well-represented taxonomic group, the bias on the selection of reference genomes selected to create OrthoDB can lead to an under-scoring of the newly assembled genome and is dependent on the evolution of the genomes. For example, in microsporidia, basal genomes have much lower scores due to the strong drive on gene loss and gain in these organisms.

## k-mer based assembly evaluation with **Merqury**

Meryl will allow us to generate the k-mer profile by decomposing the sequencing data into k-length substrings, counting the occurrence of each k-mer and determining its frequency. Meryl comprises three main modules: one for generating k-mer databases, one for filtering and combining databases, and one for searching databases.

> <hands-on-title>Generate k-mers count distribution</hands-on-title>
>
> 1. {% tool [Meryl](toolshed.g2.bx.psu.edu/repos/iuc/meryl/meryl/1.3+galaxy6) %} with the following parameters:
>    - *"Operation type selector"*: `Count operations`
>        - *"Count operations"*: `Count: count the ocurrences of canonical k-mers`
>        - {% icon param-file %} *"Input sequences"*: `SRR13577847_subreads.30x.fastq.gz`
>        - *"K-mer size selector"*: `Estimate the best k-mer size`
>            - "*Genome size*": `12000000`
>
{: .hands_on}

Merqury is used for reference-free assembly evaluation based on k-mer set operations. By comparing k-mers in a de novo assembly to those found in unassembled high-accuracy reads, Merqury estimates base-level accuracy and completeness. For trios, Merqury can also evaluate haplotype-specific accuracy, completeness, phase block continuity, and switch errors. Multiple visualizations and tables, such as k-mer spectrum plots, are generated for evaluation of the assembly quality.

> <hands-on-title>K-mer based evaluation with Merqury</hands-on-title>
>
> 1. {% tool [Merqury](toolshed.g2.bx.psu.edu/repos/iuc/merqury/merqury/1.3+galaxy2) %} with the following parameters:
>    - *"Evaluation mode"*: `Default mode`
>        - {% icon param-file %} *"K-mer counts database"*: `Meryldb` (output of **Meryl** {% icon tool %})
>        - *"Number of assemblies"*: `One assembly`
>            - {% icon param-files %} *"Genome assembly"*: `Scerevisiae-INSC1019.flye.30x.fa`, `Scerevisiae-INSC1019.hifiasm.30x.fa` and `GCA_000146045.2_genomic.fna`
>
{: .hands_on}

By default, Merqury generates three collections as output: stats, plots and QV stats. The "stats" and "plots" collection contains the completeness statistics, while the "QV stats" collection contains the quality value statistics.

> <question-title></question-title>
>
> Compare the k-mers plots for the Flye and Hifiasm assembly and the reference genome. What do you observe ?
>
> > <solution-title></solution-title>
> >
> > Merqury analysis confirms that Flye assembly is better than Hifiasm assembly. Hifiasm `assembly.spectra-cn.fl` shows that part of the data is specific to reads (black peak overlapping the red peak) indicating an absence of contigs in the assembly to represent these data.
> >
> > Flye it's similar in completness to the reference genome with a single read peak. We note however that part of the reference genome assembly is specific and absent from the reads (red barplot on the left). This is because the sequencing data is from a different strain (INSC1019) than the reference genome (S288C).
> >
> {: .solution}
>
{: .question}

# Evaluation against reference genome with **Chromeister**

Chromeister is used to generate dotplots to quickly compare sequence sets, even for large genomes. This is provide a synthetic similarity overview, highlighting repetitions, breaks and inversions.

> <hands-on-title>Synteny evaluation with Chromeister</hands-on-title>
>
> 1. {% tool [Chromeister](toolshed.g2.bx.psu.edu/repos/iuc/chromeister/chromeister/1.5.a+galaxy1) %} with the following parameters:
>    - {% icon param-files %} *"Query sequence"*: `Scerevisiae-INSC1019.flye.30x.fa` and `Scerevisiae-INSC1019.hifiasm.30x.fa`
>    - {% icon param-file %} *"Reference sequence"*: `GCA_000146045.2_genomic.fna`
>
{: .hands_on}

> <question-title></question-title>
>
> Compare the dotplots for the Hifiasm assembly and the reference genome. What do you observe ?
>
> > <solution-title></solution-title>
> >
> > ![Chromeister dotplot](../../images/genome-qc/chromeister-dotplot.png)
> >
> > With dotplots, we can quickly detect several inconsistencies between the assembly generated by Hifiasm and the reference genome:
> > - Orange arrows indicate contigs in the reference genome that are absent in the Hifiasm assembly
> > - Purple arrow indicates a contig specific to the Hifiasm assembly. This is not necessarily an inconsistency as it is 2 different strains of the same species.
> > - Brown arrows indicate contigs that are merged into the Hifiasm assembly. In the case of the top arrow, there is a fusion and a partial inversion.
> > - Green arrow indicates a contig partially assembled by Hifiasm.
> >
> {: .solution}
>
{: .question}

# Conclusion


This pipeline shows how to evaluate a genome assembly. Once you are satisfied with your genome sequence, you might want to purge it, make scaffolding and directly starting the annotation process!

tools_json:
["Quast", "toolshed.g2.bx.psu.edu/repos/iuc/quast/quast/5.2.0+galaxy0", "Busco", "toolshed.g2.bx.psu.edu/repos/iuc/busco/busco/5.3.2+galaxy0", "Meryl", "toolshed.g2.bx.psu.edu/repos/iuc/meryl/meryl/1.3+galaxy6", "Merqury", "toolshed.g2.bx.psu.edu/repos/iuc/merqury/merqury/1.3+galaxy2", "Chromeister", "toolshed.g2.bx.psu.edu/repos/iuc/chromeister/chromeister/1.5.a+galaxy1"]


topic: assembly
tutorial: vgp_genome_assembly
title: Vertebrate genome assembly using HiFi, Bionano and Hi-C data - Step by Step
difficulty: intermediate
time_estimation: 5h
requirements_json: []
tags_json: []
content_hash: 3bace83426d865557ef452ab1cfc845e
last_modified: 2026-05-27T09:01:47.818112
zenodo_link: https://zenodo.org/record/5887339
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/assembly/tutorials/vgp_genome_assembly/tutorial.html

content:
Advances in sequencing technologies over the last few decades have revolutionized the field of genomics, allowing for a reduction in both the time and resources required to perform *de novo* genome assembly. Until recently, second-generation sequencing technologies (also known as {NGS}) produced highly accurate but short (up to 800bp) reads. These read lengths were not long enough to cope with the difficulties associated with repetitive regions. Today, so-called {TGS} technologies, also known as {SMRT} sequencing, have become dominant in *de novo* assembly of large genomes. TGS can use native DNA without amplification, reducing sequencing error and bias ({% cite Hon2020 %}, {% cite Giani2020 %}). In 2020, PacBio introduced {HiFi} sequencing, which produces reads 10-25 kbp in length with a minimum accuracy of 99% (Q20). In this tutorial, you will use HiFi reads in combination with data from additional sequencing technologies to generate a high-quality genome assembly.

Deciphering the structural organization of complex vertebrate genomes is currently one of the largest challenges in genomics ({% cite Frenkel2012 %}). Despite the significant progress made in recent years, a key question remains: what combination of data and tools can produce the highest quality assembly? In order to adequately answer this question, it is necessary to analyse two of the main factors that determine the difficulty of genome assembly processes: repetitive content and heterozygosity.

Repetitive elements can be grouped into two categories: interspersed repeats, such as {TE} that occur at multiple loci throughout the genome, and {TR} that occur at a single locus ({% cite Trresen2019 %}). Repetitive elements are an important component of eukaryotic genomes, constituting over a third of the genome in the case of mammals ({% cite SoteroCaio2017 %}, {% cite Chalopin2015 %}). In the case of tandem repeats, various estimates suggest that they are present in at least one third of human protein sequences ({% cite Marcotte1999 %}). {TE} content is among the main factors contributing to the lack of continuity in the reconstruction of genomes, especially in the case of large ones, as TE content is highly correlated with genome size ({% cite SoteroCaio2017 %}). On the other hand, {TR} usually lead to local genome assembly collapse, especially when their length is close to that of the reads ({% cite Trresen2019 %}).

Heterozygosity is also an important factor impacting genome assembly. Haplotype phasing, the identification of alleles that are co-located on the same chromosome, has become a fundamental problem in heterozygous and polyploid genome assemblies ({% cite Zhang2020 %}). When no reference sequence is available, the state-of-the-art strategy consists of constructing a string graph with vertices representing reads and edges representing consistent overlaps. In this kind of graph, after transitive reduction, heterozygous alleles in the string graph are represented by bubbles. When combined with {Hi-C} data, this approach allows complete diploid reconstruction ({% cite DominguezDelAngel2018 %}, {% cite Zhang2020 %}, {% cite Dida2021 %}).

The {G10K} launched the Vertebrate Genome Project ({VGP}), whose goal is generating high-quality, near-error-free, gap-free, chromosome-level, haplotype-phased, annotated reference genome assemblies for every vertebrate species ({% cite Rhie2021 %}). This tutorial will guide you step by step to assemble a high-quality genome using the VGP assembly pipeline, including multiple {QC} evaluations.

> <warning-title>Your results may differ!</warning-title>
>
> Some of your results may slightly differ from the results shown in this tutorial, depending on the versions of the tools used, since algorithms can change between versions.
>
{: .warning}

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Important terms to know

Before getting into the thick of things, let's go over some terms you will often hear when learning about genome assembly. These concepts will be used often throughout this tutorial as well, so please refer to this section as necessary to help your understanding.

**Pseudohaplotype assembly**: A genome assembly that consists of long-phased haplotype blocks separated by regions where the haplotype cannot be distinguished (often homozygous regions). This can result in "switch errors", when the parental haplotypes alternate along the same sequence. These types of assemblies are usually represented by a _primary assembly_ and an _alternate assembly_. (This definition is largely taken from the [NCBI's Genome Assembly Model](https://www.ncbi.nlm.nih.gov/assembly/model/#asmb_def).)

**Primary assembly**: The primary assembly is traditionally the more complete representation of an individual's genome and consists of homozygous regions and one set of loci for heterozygous regions. Because the primary assembly contains both homo- and heterozygous regions, it is more complete than the _alternate assambly_ which often reports only the other allele for heterozygous regions. Thus, the primary assembly is usually what one would use for downstream analyses.

**Alternate assembly**: The alternate assembly consists of the alternate alleles not represented in the _primary assembly_ (heterozygous loci from the other haplotype). These types of sequences are often referred to as haplotigs. Traditionally, the alternate assembly is less complete compared to the primary assembly since homozygous regions are not represented.

**Phasing**: Phasing aims to partition the contigs for an individual according to the haplotype they are derived from. When parental data is available, this is done by identifying parental alleles using read data from the parents. Locally, this is achieved using linkage information in long read datasets. Recent approaches have managed to phase using long-range {Hi-C} linkage information from the same individual ({% cite Cheng2021 %}).

**Assembly graph**: A representation of the genome inferred from sequencing reads. Sequencing captures the genome as many fragmented pieces, instead of whole entire chromosomes at once (we eagerly await the day when this statement will be outdated!). The start of the assembly process pieces together these genome fragments to generate an assembly graph, which is a representation of the sequences and their overlaps. Visualizing assembly graphs can show where homozygous regions branch off into alternate paths on different haplotypes.

**Unitig**: Usually the smallest unit of an assembly graph, consistent with all the available sequencing data. A unitig is often constructed from an unambiguous path in the assembly graph where all the vertices have exactly one incoming and one outgoing edge, except the first vertex can have any number of incoming edges, while the last vertex can have any number of outgoing edges ({% cite Rahman2022 %}). In other words, the internal vertices in the unitig path can only be walked one way, so unitigs represent a path of confident sequence. In the assembly graph, unitig nodes can then have overlap edges with other unitigs.

**Contig**: A contiguous (*i.e.*, gapless) sequence in an assembly, usually inferred algorithmically from the unitig graph.

**False duplications**: Assembly errors that result in one region of the genome being represented twice in the same assembly as two separate regions. Not to be confused with optical or technical duplicates from PCR from short-read sequencing. False duplications can further be classified as either _haplotypic duplications_ or _overlaps_.

**Haplotypic duplication** can happen when a region that is heterozygous in the individual has the two haplotypes showing enough divergence that the assembler fails to interpret them as homologous. For example, say an individual is heterozygous in the region Chr1[1:100] and has Haplotype A from their mother and Haplotype B from their father; a false duplication can arise when the haplotypes are not recognized as being from the same region, and the assembler ends up placing both haplotypes in the same assembly, resulting in Chr1[1:100] being represented twice in one assembly. Ideally, a properly phased assembly would have Haplotype A in one assembly, *e.g.*, the primary, while Haplotype B is in the alternate.

False duplications via **overlaps** result from unresolved overlaps in the assembly graph. In this case, two different contigs end up representing the same sequence, effectively duplicating it. Overlaps happen when a branching point in an assembly graph is resolved such that the contig before the vertex and a contig after the vertex share the same overlapping sequence.

![Types of false duplication.](../../images/vgp_assembly/falseduplications.png "Schematic of types of false duplication. Image adapted from {% cite Rhie2021 %}.")

**Purging**: Purging aims to remove false duplications, collapsed repeats, and very low support/coverage regions from an assembly. When performed on a primary assembly, the haplotigs are removed from the primary and typically placed in the alternate assembly.

**Scaffold**: A scaffold refers to one or more contigs separated by gap (unknown) sequence. Scaffolds are usually generated with the aid of additional information, such as Bionano optical maps, linked reads, Hi-C chromatin information, etc. The regions between contigs are usually of unknown sequence, thus they are represented by sequences of _N_'s. Gap length in the sequence can be sized or arbitrary, depending on the technology used for scaffolding (*e.g.*, optical maps can introduce sized gaps, while scaffolding software using {Hi-C} information usually uses an arbitrary number of N's, such as 500 or 200).

For more about the specific scaffolding technologies used in the VGP pipeline (currently Bionano optical maps and Hi-C chromatin conformation data), please refer to those specific sections within this tutorial.

**HiFi reads**: PacBio {HiFi} reads are the focus of this tutorial. First described in 2019, they have revolutionized genome assembly by combining long (about 10-20 kbp) read lengths with high accuracy (>Q20) typically associated with short-read sequencing ({% cite Wenger2019 %}). These higher read lengths enable HiFi reads to traverse some repeat regions that are problematic to assemble with short reads.

**Ultra-long reads**: Ultra-long reads are typically defined as reads of over 100 kbp, and are usually generated using Oxford Nanopore Technology. Read quality is often lower than HiFi or Illumina (*i.e.*, have a higher error rate), but they are often significantly longer than any other current sequencing technology, and can help assembly algorithms walk complex repeat regions in the assembly graphs.

**Manual curation**: This term refers to manually evaluating and manipulating an assembly based on the raw supporting evidence (*e.g.*, using Hi-C contact map information). The user takes into account the original sequencing data to resolve potential _misassemblies_ and _missed joins_.

**Misassembly**: Misassemblies are a type of assembly error that usually refers to any structural error in the genome reconstruction, *e.g.*, sequences that are not adjacent in the genome being placed next to each other in the sequence. Misassemblies can be potentially identified and remedied by manual curation.

**Missed join**: A missed join happens when two sequences are adjacent to each other in the genome but are not represented contiguously in the final sequence. Missed joins can be identified and remedied in manual curation with Hi-C data.

**Telomere-to-telomere assembly**: Often abbreviated as "T2T", this term refers to an assembly where each chromosome is completely gapless from telomere to telomere. The term usually refers to the recently completed CHM13 human genome ({% cite Nurk2022 %}), though there is an increasing number of efforts to generate T2T genomes for other species.

# VGP assembly pipeline overview

The {VGP} assembly pipeline has a modular organization, consisting of ten workflows (Fig. 1). It can used with the following types of input data:

|  Input data | Assembly quality  | Analysis trajectory <br>([Fig. 2)](#figure-2)|
|------|---------------|-----|
| HiFi | The minimum requirement | A |
| HiFi + HiC | Better haplotype resolution (fewer switch errors) | B |
| HiFi + BioNano | Better contiguity | C |
| HiFi + Hi-C + BioNano | Even better contiguity | D |
| HiFi + parental data | Properly phased | E |
| HiFi + parental data + Hi-C | Better haplotype resolution | F |
| HiFi + parental + BioNano | Properly phased with improved contiguity | G |
| HiFi + parental data + Hi-C + BioNano | Properly phased with even more improved contiguity | H |

In this table, *HiFi* and *Hi-C* refer to HiFi and Hi-C data derived from the individual whose genome is being assembled. **This tutorial assumes you are assembling the genome of one individual; there are special considerations necessary for pooled data that are not covered in this tutorial.** (Note: you can use Hi-C data from another individual of the same species to scaffold, but you *cannot* use that data to phase the contigs in hifiasm.) *Parental data* is high-coverage whole genome resequencing data derived from the parents of the individual being assembled, and is the key component of trio-based genome assembly. Each combination of input datasets is demonstrated in Fig. 2 by an *analysis trajectory*: a combination of workflows designed for generating the best assembly given a particular combination of inputs. These trajectories are listed in the table above and shown in the figure below.

![The nine workflows of Galaxy assembly pipeline](../../images/vgp_assembly/VGP_workflow_modules.svg "Eight analysis trajectories are possible depending on the combination of input data. A decision on whether or not to invoke Workflow 6 is based on the analysis of QC output of workflows 3, 4, or 5. Thicker lines connecting Workflows 7, 8, and 9 represent the fact that these workflows are invoked separately for each phased assembly (once for maternal and once for paternal).")

The stages of genome assembly in the VGP-Galaxy pipeline are generally:

1. ***K*-mer profiling**: the generation of *k*-mer profiles of the raw reads to estimate genome size, heterozygosity, repetitiveness, and error rate. This is useful for getting an idea of the genome that lies within your reads, and is also useful for necessary for parameterizing downstream workflows. The generation of *k*-mer counts can be done from HiFi data only (Workflow 1), or include data from parental reads for trio-based phasing (Workflow 2), if one wants to generate *k*-mer spectra for the individual's parents, as well.

2. **Contig assembly**: In addition to using only {HiFi} reads (Workflow 3), the contig building (contiging) step can leverage {Hi-C} (Workflow 4) or parental read data (Workflow 5) to produce fully-phased haplotypes (hap1/hap2 or parental/maternal assigned haplotypes), using [`hifiasm`](https://github.com/chhylp123/hifiasm). The contiging workflows also produce a number of critical quality control (QC) metrics such as *k*-mer multiplicity profiles. Inspection of these profiles provides information to decide whether the third stage—purging of false duplication—is required.

3. (*Optional*) **Purging**: Purging duplicates (Workflow 6) using [`purge_dups`](https://github.com/dfguan/purge_dups) identifies and resolves haplotype-specific assembly segments incorrectly labeled as primary contigs, as well as heterozygous contig overlaps. This increases contiguity and the quality of the final assembly. The purging stage is generally unnecessary for trio data, as haplotype resolution is attained using set operations done on parental *k*-mers.

4. **Scaffolding**: Scaffolding produces chromosome-level scaffolds using information provided by Bionano (Workflow 7, with [`Bionano Solve`](https://bionano.com/software-downloads/) (optional)) and Hi-C (Workflow 8, with [`YaHS`](https://github.com/c-zhou/yahsscaffolding) algorithms).

5. **Decontamination**: A final step of decontamination (Workflow 9) removes non-target sequences (e.g., contamination as well as mitochondrial sequences) from the scaffolded assembly. A separate workflow (WF0) is used for mitochondrial assembly.

> <comment-title>A note on data quality</comment-title>
> For diploids, we suggest at least 30✕ PacBio HiFi coverage & around 60✕ Hi-C coverage, and up to 60✕ HiFi coverage to accurately assemble highly repetitive regions.
{: .comment}
This training has been organized into four main sections: genome profile analysis, assembly of {HiFi} reads with hifiasm, scaffolding with Bionano optical maps, and scaffolding with {Hi-C} data. Additionally, the **assembly with hifiasm** section has two possible paths in this tutorial: solo contiging or solo w/HiC contiging.

Throughout this tutorial, there will be **detail boxes** with additional background information on the science behind the sequencing technologies and software we use in the pipeline. These boxes are minimized by default, but please expand them to learn more about the data we utilize in this pipeline.

# Get data

In order to reduce computation time, we will assemble samples from the yeast _Saccharomyces cerevisiae_ S288C, a widely used laboratory strain isolated in the 1950s by Robert Mortimer. Using _S. cerevisae_, one of the most intensively studied eukaryotic model organisms, has the additional advantage of allowing us to evaluate the final result of our assembly with great precision. For this tutorial, we generated a set of synthetic HiFi reads corresponding to a theoretical diploid genome.

> <details-title>Generation of synthetic reads</details-title>
> The synthetic HiFi reads were generated by using the [S288C assembly](https://www.ncbi.nlm.nih.gov/genome/15?genome_assembly_id=22535) as reference genome. With this objective we used [HIsim](https://github.com/thegenemyers/HI.SIM), a HiFi shotgun simulator. The commands used are detailed below:
>
> ```bash
> ./FastK -T8 -Nsynthetic -k40 -t4 ./hifi_reads.fastq.gz
> ./FastK -T8 -Nsynthetic -k40 -p:synthetic.ktab .hifi_reads.fastq.gz
> ./Histex -G synthetic > synthetic.histogram
> ./GeneScopeFK.R -i synthetic.histogram -o ./synthetic -k 40 -p 1
> ./HImodel -v -osynthetic -g10:55 -e5 -T8 ./synthetic
> ./HIsim ./GCF_000146045.2_R64_genomic.fasta ./synthetic.model -p.2,.2 -c30 -r1 -U -f -h -e
> ```
> The selected mutation rate was 2%. Note that HIsim generates the synthetic reads in FASTA format. This is perfectly fine for illustrating the workflow, but you should be aware that usually you will work with HiFi reads in FASTQ format.
>
{: .details}

The first step is to get the datasets from Zenodo. Specifically, we will be uploading two datasets:

1. A set of PacBio {HiFi} reads in `fasta` format. Please note that your HiFi reads received from a sequencing center will usually be fastqsanger.gz format, but the dataset used in this tutorial has been converted to fasta for space.
2. A set of Illumina {Hi-C} reads in `fastqsanger.gz` format.

## Uploading `fasta` datasets from Zenodo

The following two steps demonstrate how to upload three PacBio {HiFi} datasets into your Galaxy history.

> <hands-on-title><b>Uploading <tt>FASTA</tt> datasets from Zenodo</b></hands-on-title>
>
> 1. Create a new history for this tutorial
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Copy the following URLs into the clipboard.
>    - you can do this by clicking on {% icon copy %} button in the right upper corner of the box below. It will appear if you mouse over the box.
>
>    ```
>    https://zenodo.org/record/6098306/files/HiFi_synthetic_50x_01.fasta
>    https://zenodo.org/record/6098306/files/HiFi_synthetic_50x_02.fasta
>    https://zenodo.org/record/6098306/files/HiFi_synthetic_50x_03.fasta
>    ```
>
> 3. Upload datasets into Galaxy.
>    - set the datatype to `fasta`
>
> {% snippet faqs/galaxy/datasets_import_via_link.md format="fasta" %}
>
> {% snippet topics/assembly/tutorials/vgp_genome_assembly/faqs/dataset_upload_fasta_via_urls.md %}
>
{: .hands_on}

## Uploading `fastqsanger.gz` datasets from Zenodo

Illumina {Hi-C} data is uploaded in essentially the same way as shown in the following two steps.

> <warning-title>DANGER: Make sure you choose the correct format!</warning-title>
> When selecting datatype in "**Type (set all)**" drop-down, make sure you select `fastaqsanger` or `fastqsanger.gz` BUT NOT `fastqcssanger` or anything else!
{: .warning}

> <hands-on-title><b>Uploading <tt>fastqsanger.gz</tt> datasets from Zenodo</b></hands-on-title>
>
> 1. Copy the following URLs into the clipboard. You can do this by clicking on {% icon copy %} button in the right upper corner of the box below. It will appear if you mouse over the box.
>
>    ```
>    https://zenodo.org/record/5550653/files/SRR7126301_1.fastq.gz
>    https://zenodo.org/record/5550653/files/SRR7126301_2.fastq.gz
>    ```
>
> 2. Upload datasets into Galaxy and set the datatype to `fastqsanger.gz`
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md format="fastqsanger.gz" %}
>
>    {% snippet topics/assembly/tutorials/vgp_genome_assembly/faqs/dataset_upload_fastqsanger_via_urls.md %}
>
> 3. Rename the datasets as follow: 
>     -  Rename `SRR7126301_1.fastq.gz` as `Hi-C_dataset_F`. It contains the forward reads.
>     -  Rename `SRR7126301_2.fastq.gz` as `Hi-C_dataset_R`. It contains the reverse reads.
>
{: .hands_on}

> <warning-title>These datasets are large!</warning-title>
> Hi-C datasets are large. It will take some time (~15 min) for them to be fully uploaded. Please, be patient.
{: .warning}

## Organizing the data

If everything goes smoothly your history will look like shown in the figure below. The three {HiFi} fasta files are better represented as a collection: {collection}. Also, importantly, the workflow we will be using for the analysis of our data takes a collection as input (it does not access individual datasets). So let's create a collection using steps outlined in the Tip {% icon tip %} "Creating a dataset collection":

{% snippet faqs/galaxy/collections_build_list.md %}

The view of your history should transition from what is shown in the left pane below to what looks like the right pane:

![AfterUpload](../../images/vgp_assembly/making_list.svg "History after uploading HiFi and HiC data (left). Creation of a list (collection) combines all HiFi datasets into a single history item called 'HiFi data' (right). See below for instructions on how to make this collection.")

> <details-title>Other ways to upload the data</details-title>
> You can obviously upload your own datasets via URLs as illustrated above or from your own computer. In addition, you can upload data from a major repository called [GenomeArk](https://genomeark.org). GenomeArk is integrated directly into Galaxy Upload. To use GenomeArk follow the steps in the Tip {% icon tip %} below:
>
> {% snippet faqs/galaxy/datasets_upload_from_genomeark.md %}
{: .details}

# HiFi reads preprocessing with **cutadapt**

Adapter trimming usually means trimming the adapter sequence off the ends of reads, which is where the adapter sequence is usually located in {NGS} reads. However, due to the nature of {SMRT} sequencing technology, adapters do not have a specific, predictable location in  {HiFi} reads. Additionally, the reads containing adapter sequences could be of generally lower quality compared to the rest of the reads. Thus, we will use **cutadapt** not to trim, but to remove the entire read if a read is found to have an adapter inside of it.

> <details-title>Background on PacBio HiFi reads</details-title>
>
> PacBio HiFi reads rely on {SMRT} sequencing technology. SMRT is based on real-time imaging of fluorescently tagged nucleotides as they are added to a newly synthesized DNA strand. HiFi further uses multiple subreads from the same circular template to produce one highly accurate consensus sequence (fig. 2).
>
> ![PacBio HiFi technology](../../images/vgp_assembly/pacbio_technology.png "HiFi reads are produced by calling consensus from subreads generated by multiple passes of the enzyme around a circularized template. This results in a HiFi read that is both long and accurate.")
>
> This technology allows to generate long-read sequencing data with read lengths in the range of 10-25 kb and minimum read consensus accuracy greater than 99% (Q20).
>
{: .details}

> <hands-on-title>Primer removal with Cutadapt</hands-on-title>
>
> 1. Run {% tool [Cutadapt](toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt/4.4+galaxy0) %} with the following parameters:
>    - *"Single-end or Paired-end reads?"*: `Single-end`
>    - {% icon param-collection %} *"FASTQ/A file"*: `HiFi_collection`
>    - In *"Read 1 Options"*:
>        - In *"5' or 3' (Anywhere) Adapters"*:
>            - {% icon param-repeat %} *"Insert 5' or 3' (Anywhere) Adapters"*
>                - *"Source"*: `Enter custom sequence`
>                - *"Enter custom 5' or 3' adapter name"*: `First adapter`
>                - *"Enter custom 5' or 3' adapter sequence"*: `ATCTCTCTCAACAACAACAACGGAGGAGGAGGAAAAGAGAGAGAT`
>            - {% icon param-repeat %} *"Insert 5' or 3' (Anywhere) Adapters"*
>                - *"Source"*: `Enter custom sequence`
>                - *"Enter custom 5' or 3' adapter name"*: `Second adapter`
>                - *"Enter custom 5' or 3' adapter sequence"*: `ATCTCTCTCTTTTCCTCCTCCTCCGTTGTTGTTGTTGAGAGAGAT`
>    - In *"Adapter Options"*:
>        - *"Maximum error rate"*: `0.1`
>        - *"Minimum overlap length"*: `35`
>        - *"Look for adapters in the reverse complement"*: `Yes`
>    - In *"Filter Options"*:
>        - *"Discard Trimmed Reads"*: `Yes`
>
>    {% snippet faqs/galaxy/tools_select_collection.md %}
>
> 2. Rename the output file as `HiFi_collection (trimmed)`.
>
>    {% snippet faqs/galaxy/datasets_rename.md %}
>
{: .hands_on}

# Genome profile analysis

Before starting a *de novo* genome assembly project, it is useful to collect metrics on the properties of the genome under consideration, such as the expected genome size, so that you know what to expect from your assembly. Traditionally, DNA flow cytometry was considered the golden standard for estimating the genome size. Nowadays, experimental methods have been replaced by computational approaches ({% cite wang2020estimation %}). One widely used genome profiling methods is based on the analysis of *k*-mer frequencies. It allows one to provide information not only about the genomic complexity, such as the genome size and levels of heterozygosity and repeat content, but also about the data quality.

> <details-title><i>K</i>-mer size, sequencing coverage and genome size</details-title>
>
>*K*-mers are unique substrings of length *k* contained within a DNA sequence. For example, the DNA sequence *TCGATCACA* can be decomposed into five unique *k*-mers that have five bases long: *TCGAT*, *CGATC*, *GATCA*, *ATCAC* and *TCACA*. A sequence of length L will have  L-k+1 *k*-mers. On the other hand, the number of possible *k*-mers can be calculated as  n<sup>k</sup>, where n is the number of possible monomers and k is the *k*-mer size.
>
>
>---------| -------------|-----------------------
>  Bases  |  *k*-mer size  |  Total possible *k*-mers
>---------| -------------|-----------------------
>    4    |       1      |            4
>    4    |       2      |           16
>    4    |       3      |           64
>    4    |      ...     |          ...
>    4    |      10      |    1.048.576
>---------|--------------|-----------------------
>
> Thus, the *k*-mer size is a key parameter, which must be large enough to map uniquely to the genome, but not too large, since it can lead to wasting computational resources. In the case of the human genome, *k*-mers of 31 bases in length lead to 96.96% of unique *k*-mers.
>
> Each unique *k*-mer can be assigned a value for coverage based on the number of times it occurs in a sequence, whose distribution will approximate a Poisson distribution, with the peak corresponding to the average genome sequencing depth. From the genome coverage, the genome size can be easily computed.
{: .details}

## Generation of _k_-mer spectra with **Meryl**

Meryl will allow us to generate the *k*-mer profile by decomposing the sequencing data into *k*-length substrings, counting the occurrence of each *k*-mer and determining its frequency. The original version of Meryl was developed for the Celera Assembler. The current Meryl version comprises three main modules: one for generating *k*-mer databases, one for filtering and combining databases, and one for searching databases. *K*-mers are stored in lexicographical order in the database, similar to words in a dictionary ({% cite Rhie2020 %}).

> <comment-title><i>k</i>-mer size estimation</comment-title>
>
>  Given an estimated genome size (*G*) and a tolerable collision rate (*p*), an appropriate *k* can be computed as $$ k = \log_4\left(\frac{G(1-p)}{p}\right) $$ .
>
{: .comment}

In order to identify some key characteristics of the genome, we do genome profile analysis. To do this, we start by generating a histogram of the *k*-mer distribution in the raw reads (the *k*-mer spectrum). Then, GenomeScope creates a model fitting the spectrum that allows for estimation of genome characteristics. We work in parallel on each set of raw reads, creating a database of each file's *k*-mer counts, and then merge the databases of counts in order to build the histogram.

![Workflow of Kmer counting parallelization, described in the figure caption.](../../images/vgp_assembly/meryl_collections.png "K-mer counting is first done on the collection of FASTA files. Because these data are stored in a collection, a separate `count` job is launched for each FASTA file, thus parallelizing our work. After that, the collection of count datasets is merged into one dataset, which we can use to generate the histogram input needed for GenomeScope.")

> <hands-on-title>Generate <i>k</i>-mers count distribution</hands-on-title>
>
> 1. Run {% tool [Meryl](toolshed.g2.bx.psu.edu/repos/iuc/meryl/meryl/1.3+galaxy6) %} with the following parameters:
>    - *"Operation type selector"*: `Count operations`
>    - *"Count operations"*: `Count: count the occurrences of canonical k-mers`
>    - {% icon param-collection %} *"Input sequences"*: `HiFi_collection (trim)`
>    - *"k-mer size selector"*: `Set a k-mer size`
>    - "*k-mer size*": `31`
>
>    > <comment-title>Selection of <i>k</i>-mer size</comment-title>
>    >
>    > We used 31 as *k*-mer size, as this length has demonstrated to be sufficiently long that most *k*-mers are not repetitive and is short enough to be more robust to sequencing errors. For very large (haploid size > 10 Gb) and/or very repetitive genomes, larger *k*-mer length is recommended to increase the number of unique *k*-mers.
>    {: .comment}
>
> 2. Rename output as `meryldb`
>
> 3. Run {% tool [Meryl](toolshed.g2.bx.psu.edu/repos/iuc/meryl/meryl/1.3+galaxy6) %} again with the following parameters:
>    - *"Operation type selector"*: `Operations on sets of *k*-mers`
>    - *"Operations on sets of k-mers"*: `Union-sum: return k-mers that occur in any input, set the count to the sum of the counts`
>    - {% icon param-file %} *"Input meryldb"*: `Collection meryldb`
>
> 4. Rename it as `Merged meryldb`
>
> 5. Run {% tool [Meryl](toolshed.g2.bx.psu.edu/repos/iuc/meryl/meryl/1.3+galaxy6) %} for the third time with the following parameters:
>    - *"Operation type selector"*: `Generate histogram dataset`
>        - {% icon param-file %} *"Input meryldb"*: `Merged meryldb`
>
> 6. Finally, rename it as `meryldb histogram`.
>
{: .hands_on}

## Genome profiling with **GenomeScope2**

The next step is to infer the genome properties from the *k*-mer histogram generated by Meryl, for which we will use GenomeScope2. GenomeScope2 relies on a nonlinear least-squares optimization to fit a mixture of negative binomial distributions, generating estimated values for genome size, repetitiveness, and heterozygosity rates ({% cite RanalloBenavidez2020 %}).

> <hands-on-title>Estimate genome properties</hands-on-title>
>
> 1. Run {% tool [GenomeScope](toolshed.g2.bx.psu.edu/repos/iuc/genomescope/genomescope/2.0+galaxy2) %} with the following parameters:
>    - {% icon param-file %} *"Input histogram file"*: `meryldb histogram`
>    - *Ploidy for model to use*: `2`
>    - *"k-mer length used to calculate k-mer spectra"*: `31`
>    - In "*Output options*":
>        - Check `Summary of the analysis`
>    - In "*Advanced options*":
>        - *"Create testing.tsv file with model parameters"*: `Yes`
>
{: .hands_on}

Genomescope will generate six outputs:

- **Plots**:
    - Linear plot: *k*-mer spectra and fitted models: frequency (y-axis) versus coverage.
    - Log plot: logarithmic transformation of the previous plot.
    - Transformed linear plot: *k*-mer spectra and fitted models: frequency times coverage (y-axis) versus coverage (x-axis). This transformation increases the heights of higher-order peaks, overcoming the effect of high heterozygosity.
    - Transformed log plot: logarithmic transformation of the previous plot.
- **Model**: this file includes a detailed report about the model fitting.
- **Summary**: it includes the properties inferred from the model, such as genome haploid length and the percentage of heterozygosity. It is worth noting that the genome characteristics such as length, error percentage, etc., are based on the GenomeScope2 model, which is the black line in the plot. If the model (black line) does not fit your observed data (blue bars), then these estimated characteristics might be very off. In the case of this tutorial, the model is a good fit to our data, so we can trust the estimates.

Now, let's analyze the *k*-mer profiles, fitted models and estimated parameters shown below:

![Genomescope plot](../../images/vgp_assembly/genomescope_plot.png "GenomeScope2 31-mer profile. The first peak located at coverage 25✕ corresponds to the heterozygous peak. The second peak at coverage 50✕, corresponds to the homozygous peak. Estimate of the heterozygous portion is 0.576%. The plot also includes information about the inferred total genome length (len), genome unique length percent ('uniq'), overall heterozygosity rate ('ab'), mean *k*-mer coverage for heterozygous bases ('kcov'), read error rate ('err'), and average rate of read duplications ('dup'). It also reports the user-given parameters of *k*-mer size ('k') and ploidy ('p')."){:width="65%"}

This distribution is the result of the Poisson process underlying the generation of sequencing reads. As we can see, the *k*-mer profile follows a bimodal distribution, indicative of a diploid genome. The distribution is consistent with the theoretical diploid model (model fit > 93%). Low frequency *k*-mers are the result of sequencing errors. GenomeScope2 estimated a haploid genome size is around 11.7 Mb, a value reasonably close to *Saccharomyces* genome size. Additionally, it revealed that the variation across the genomic sequences is 0.576%. Some of these parameters can be used later on to parameterize running `purge_dups`. This is covered in the [**solo** contiging section](#solo_hic_switch) section of the tutorial.


# Assembly with **hifiasm**

Once we have finished the genome profiling stage, we can start the genome assembly with hifiasm,  a fast open-source *de novo* assembler specifically developed for PacBio HiFi reads. One of the key advantages of hifiasm is that it allows us to resolve near-identical, but not exactly identical, sequences, such as repeats and segmental duplications ({% cite Cheng2021 %}).

> <details-title>Hifiasm algorithm details</details-title>
>
> By default hifiasm performs three rounds of haplotype-aware error correction to correct sequence errors but keeping heterozygous alleles. A position on the target read to be corrected is considered informative if there are two different nucleotides at that position in the alignment, and each allele is supported by at least three reads.
>
> ![Hifiasm algorithm overview](../../images/vgp_assembly/hifiasm_algorithm.png "Hifiasm algorithm overview. Orange and blue bars represent the reads with heterozygous alleles carrying local phasing information, while green bars come from the homozygous regions without any heterozygous alleles.")
>
> Then, hifiasm builds a phased assembly string graph with local phasing information from the corrected reads. Only the reads coming from the same haplotype are connected in the phased assembly graph. After transitive reduction, a pair of heterozygous alleles is represented by a _bubble_ in the string graph. If there is no additional data, hifiasm arbitrarily selects one side of each bubble and outputs a primary assembly. In the case of a heterozygous genome, the primary assembly generated at this step may still retain haplotigs from the alternate allele.
>
{: .details}

The output of hifiasm will be [GFA](https://github.com/GFA-spec/GFA-spec) files. These differ from FASTA files in that they are a representation of the assembly graph instead of just linear sequences, so the {GFA} contains information about sequences, nodes, and edges (*i.e.*, overlaps). This output preserves the most information about how the reads assemble in graph space, and is useful to visualize in tools such as Bandage; however, our QV tools will expect FASTA files, so we will cover the GFA to FASTA conversion step later.

## `hifiasm` assembly modes

Hifiasm can be run in multiple modes depending on data availability

### **Solo** mode

**Solo**: generates a pseudohaplotype assembly, resulting in a primary & an alternate assembly.
- _Input: PacBio HiFi reads_
- _Output: scaffolded primary assembly, and alternate contigs_
![Diagram for hifiasm solo mode.](../../images/vgp_assembly/hifiasm_solo_schematic.png "The <b>solo</b> pipeline creates <b>primary</b> and <b>alternate</b> contigs, which then typically undergo purging with purge_dups to reconcile the haplotypes. During the purging process, haplotigs are removed from the primary assembly and added to the alternate assembly, which is then purged to generate the final alternate set of contigs. The purged primary contigs are then carried through scaffolding with Bionano and/or Hi-C data, resulting in one final draft primary assembly to be sent to manual curation.")

### **Hi-C** phased mode

**Hi-C-phased**: generates a hap1 assembly and a hap2 assembly, which are phased using the {Hi-C} reads from the same individual.
- _Input: PacBio HiFi & Illumina HiC reads_
- _Output: scaffolded hap1 assembly, and scaffolded hap2 assembly (assuming you run the scaffolding on **both** haplotypes)_
![Diagram for hifiasm hic mode.](../../images/vgp_assembly/hifiasm_hic_schematic.png "The <b>Hi-C-phased</b> mode produces <b>hap1</b> and <b>hap2</b> contigs, which have been phased using the HiC information as described in {% cite Cheng2021 %}. Typically, these assemblies do not need to undergo purging, but you should always look at your assemblies' QC to make sure. These contigs are then scaffolded <i>separately</i> using Bionano and/or Hi-C workflows, resulting in two scaffolded assemblies.")

### **Trio** mode

**Trio**: generates a maternal assembly and a paternal assembly, which are phased using reads from the parents.
- _Input: PacBio HiFi reads from child, Illumina reads from both parents._
- _Output: scaffolded maternal assembly, and scaffolded paternal assembly (assuming you run the scaffolding on **both** haplotypes)_
![Diagram for hifiasm trio mode.](../../images/vgp_assembly/hifiasm_trio_schematic.png "The <b>trio</b> mode produces <b>maternal</b> and <b>paternal</b> contigs, which have been phased using paternal short read data. Typically, these assemblies do not need to undergo purging, but you should always look at your assemblies' QC to make sure. These contigs are then scaffolded <i>separately</i> using Bionano and/or Hi-C workflows, resulting in two scaffolded assemblies.")

No matter which way you run hifiasm, you will have to evaluate the assemblies' {QC} to ensure your genome is in good shape. The VGP pipeline features several reference-free ways of evaluating assembly quality, all of which are automatically generated with our workflows; however, we will run them manually in this tutorial so we can familiarize ourselves with how each QC metric captures a different aspect of assembly quality. If you are interested in running the workflows with automatic QC generation, please see our [corresponding workflow tutorial]({% link topics/assembly/tutorials/vgp_workflow_training/tutorial.md %}) .

## Assembly evaluation

We use several tools for assessing various aspects of assembly quality:

- **gfastats**: manipulation & evaluation of assembly graphs and FASTA files, particularly used for summary statistics (*e.g.*, contig count, N50, NG50, etc.) ({% cite Formenti2022 %}).
![Schematic of N50 calculation.](../../images/vgp_assembly/n50schematic.jpg "<b>N50</b> is a commonly reported statistic used to represent genome contiguity. N50 is calculated by sorting contigs according to their lengths, and then taking the halfway point of the total genome length. The size of the contig at that halfway point is the N50 value. In the pictured example, the total genome length is 400 bp, so the N50 value is 60 because the contig at the halfway point is 60 bp long. N50 can be interpreted as the value where >50% of an assembly's contigs are at that value or higher. Image adapted from <a href='https://www.molecularecologist.com/2017/03/29/whats-n50/'>Elin Videvall at The Molecular Ecologist</a>.")
- **{BUSCO}**: assesses completeness of a genome from an evolutionarily informed functional point of view. BUSCO genes are genes that are expected to be present at single-copy in one haplotype for a certain clade, so their presence, absence, or duplication can inform scientists about if an assembly is likely missing important regions, or if it has multiple copies of them, which can indicate a need for purging ({% cite Simo2015 %}).
- **Merqury**: reference-free assessment of assembly completeness and phasing based on *k*-mers. Merqury compares *k*-mers in the reads to the *k*-mers found in the assemblies, as well as the {CN} of each *k*-mer in the assemblies ({% cite Rhie_merqury %}).

> <comment-title>How do I pick which assembly trajectory to use?</comment-title>
> The ideal scenario would be a trio assembly, where you can use parental data as a ground truth for phasing the haplotypes in the child. Unfortunately, attaining parental samples is difficult and often impossible when studying wild-caught organisms. When parental data is absent, the VGP recommends assembling using {Hi-C} phasing, if possible. This requires the Hi-C data to be derived from the same individual as the HiFi data. If this is not possible, then you cannot use the Hi-C data to phase the contigs, but you can still use it for scaffolding the primary assembly. Refer to the following decision tree for a visual representation of this logic. 
>
> ![Decision tree for picking workflow trajectory when one does not have parental data.](../../images/vgp_assembly/WF3vsWF4_decisiontree.png "If the HiFi and Hi-C data *do not* come from the same individual, then you can use WF3. After that, if you need to purge the primary, then run WF6 and then scaffolding. If the primary does not need purging, you can continue straight to scaffolding the primary. If the HiFi and Hi-C data *do* come from the same individual, then you can use WF4 to obtain hap1 and hap2 assemblies. If either of the assemblies need to be purged, then you can use WF6B to purge an individual haplotype. Then you can scaffold the assemblies separately. Accordingly, if neither need to be purged, then you can proceed to just scaffolding them separately.")
{: .comment}

<div id="solo_hic_switch">
<!-- For use as an anchor -->
</div>

{% include _includes/cyoa-choices.html option1="hic" option2="solo" default="hic"
       text="Use the following buttons to switch between contigging approaches. If you are assembling with only HiFi reads for an individual, then click <b><i>solo</i></b>. If you have HiC reads for the same indiviudal, then click <b><i>hic</i></b>. <b>NOTE: If you want to learn more about purging, then <u>please check out the <i>solo</i> tutorial for details on purging false duplications.</u></b>" %}

<div class = "hic" markdown="1">

<!---- BEGINNING OF HiC SECTION --->
<!---- BEGINNING OF HiC SECTION --->
<!---- BEGINNING OF HiC SECTION --->
<!---- BEGINNING OF HiC SECTION --->
<!---- BEGINNING OF HiC SECTION --->

## HiC-phased assembly with **hifiasm**

If you have the {Hi-C} data for the individual you are assembling with {HiFi} reads, then you can use that information to phase the contigs.

> <hands-on-title>Hi-C-phased assembly with <b>hifiasm</b></hands-on-title>
> 1. Run {% tool [Hifiasm](toolshed.g2.bx.psu.edu/repos/bgruening/hifiasm/hifiasm/0.19.8+galaxy0) %} with the following parameters:
>    - *"Assembly mode"*: `Standard`
>    - {% icon param-file %} *"Input reads"*: `HiFi_collection (trimmed)` (output of **Cutadapt** {% icon tool %})
>    - In *"Options for Hi-C-partition*" select `Specify`
>        - *"Hi-C R1 reads"*: `Hi-C_dataset_F`
>        - *"Hi-C R2 reads"*: `Hi-C_dataset_R`
>
> 2. After the tool has finished running, rename its outputs as follows:
>    - Rename the `Hi-C hap1 balanced contig graph` as `Hap1 contigs graph` and add a `#hap1` tag
>    - Rename the `Hi-C hap2 balanced contig graph` as `Hap2 contigs graph` and  add a `#hap2` tag
>
{: .hands_on}

We have obtained the fully phased contig graphs (as {GFA} files) of hap1 and hap2, but these must be converted to FASTA format for subsequent steps. We will use a tool developed from the VGP: [`gfastats`](https://github.com/vgl-hub/gfastats). `gfastats` is a tool suite that allows for manipulation and evaluation of FASTA and GFA files, but in this instance we will use it to convert our GFAs to FASTA files. Later on, we will use it to generate standard summary statistics for our assemblies.

> <hands-on-title>GFA to FASTA conversion for hifiasm Hi-C assembly</hands-on-title>
>
> 1. Run {% tool [gfastats](toolshed.g2.bx.psu.edu/repos/bgruening/gfastats/gfastats/1.3.6+galaxy0) %} with the following parameters:
>    - {% icon param-files %} *"Input GFA file"*: select `Hap1 contigs graph` and the `Hap2 contigs graph` datasets
>
>    {% snippet faqs/galaxy/tools_select_multiple_datasets.md %}
>
>    - *"Tool mode"*: `Genome assembly manipulation`
>    - *"Output format"*: `FASTA`
>    - *"Generates the initial set of paths*": toggle to `yes`
>
> 2. Rename the outputs as `Hap1 contigs FASTA` and `Hap2 contigs FASTA`
>
{: .hands_on}

> <comment-title>Summary statistics</comment-title>
>
> gfastats will provide us with the following statistics:
>
> - **No. of contigs**: The total number of contigs in the assembly.
> - **Largest contig**: The length of the largest contig in the assembly.
> - **Total length**: The total number of bases in the assembly.
> - **Nx**: The largest contig length, *L*, such that contigs of length > *L* account for at least *x*% of the bases of the assembly.
> - **NGx**: The contig length such that using equal or longer length contigs produces *x*% of the length of the reference genome, rather than *x*% of the assembly length.
> - **GC content**: the percentage of nitrogenous bases which are either guanine or cytosine.
>
{: .comment}

Let's use gfastats to get a basic idea of what our assembly looks like. We'll run gfastats on the {GFA} files because gfastats can report graph-specific statistics as well. After generating the stats, we'll be doing some text manipulations to get the stats side-by-side in a pretty fashion.

> <hands-on-title>Assembly evaluation with gfastats</hands-on-title>
>
> 1. Run assembly statistics generation with {% tool [gfastats](toolshed.g2.bx.psu.edu/repos/bgruening/gfastats/gfastats/1.3.6+galaxy0) %} using the following parameters:
>
>    - {% icon param-files %} *"Input file"*: select `Hap1 contigs graph` and the `Hap2 contigs graph` datasets
>    - *"Tool mode": `Summary statistics generation`
>    - *"Expected genome size"*: `11747160` (remember we calculated this value [earlier using `GenomeScope2`](#genome-profiling-with-genomescope2). It is contained within `GenomeScope2` **Summary** output that should be in your history!)
>    - *"Thousands separator in output"*: Set to "No"
>
>
> 2. Rename outputs of `gfastats` step to as `Hap1 stats` and `Hap2 stats`
>
>    This would generate summary files that look like this (only the first six rows are shown):
>    
>    ```
>    Expected genome size    11747160
>    # scaffolds                    0
>    Total scaffold length          0
>    Average scaffold length      nan
>    Scaffold N50                   0
>    Scaffold auN                0.00
>    ```
>    
>    Because we ran `gfastats` on hap1 and hap2 outputs of `hifiasm` we need to join the two outputs together for easier interpretation:
>
> 3. Run {% tool [Column join](toolshed.g2.bx.psu.edu/repos/iuc/collection_column_join/collection_column_join/0.0.3) %} with the following parameters:
>    - {% icon param-files %} *"Input file"*: select `Hap1 stats` and the `Hap2 stats` datasets. Keep all other settings as they are.
>
> 4. Rename the output as `gfastats on hap1 and hap2 (full)`
>
>    This would generate a joined summary file that looks like this (only the first five rows are shown):
>    
>    ```
>    # gaps               0  0
>    # gaps in scaffolds  0  0
>    # paths              0  0
>    # segments          17 16
>    ```
>    
>    Now let's extract only relevant information by excluding all lines containing the word `scaffold` since there ar

tools_json:
["Cutadapt", "toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt/4.4+galaxy0", "Meryl", "toolshed.g2.bx.psu.edu/repos/iuc/meryl/meryl/1.3+galaxy6", "GenomeScope", "toolshed.g2.bx.psu.edu/repos/iuc/genomescope/genomescope/2.0+galaxy2", "Hifiasm", "toolshed.g2.bx.psu.edu/repos/bgruening/hifiasm/hifiasm/0.19.8+galaxy0", "gfastats", "toolshed.g2.bx.psu.edu/repos/bgruening/gfastats/gfastats/1.3.6+galaxy0", "Column join", "toolshed.g2.bx.psu.edu/repos/iuc/collection_column_join/collection_column_join/0.0.3", "Search in textfiles", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/1.1.1", "Busco", "toolshed.g2.bx.psu.edu/repos/iuc/busco/busco/5.5.0+galaxy0", "Merqury", "toolshed.g2.bx.psu.edu/repos/iuc/merqury/merqury/1.3+galaxy3", "Compute on rows", "toolshed.g2.bx.psu.edu/repos/devteam/column_maker/Add_a_column1/2.0", "Advanced Cut", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cut_tool/1.1.0", "Collapse Collection", "toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/4.2", "Map with minimap2", "toolshed.g2.bx.psu.edu/repos/iuc/minimap2/minimap2/2.17+galaxy4", "Purge overlaps", "toolshed.g2.bx.psu.edu/repos/iuc/purge_dups/purge_dups/1.2.6+galaxy0", "Concatenate multiple datasets or collections", "cat1", "Bionano Hybrid Scaffold", "toolshed.g2.bx.psu.edu/repos/bgruening/bionano_scaffold/bionano_scaffold/3.7.0+galaxy3", "BWA-MEM2", "toolshed.g2.bx.psu.edu/repos/iuc/bwa_mem2/bwa_mem2/2.2.1+galaxy1", "Filter and merge", "toolshed.g2.bx.psu.edu/repos/iuc/bellerophon/bellerophon/1.0+galaxy1", "PretextMap", "toolshed.g2.bx.psu.edu/repos/iuc/pretext_map/pretext_map/0.1.9+galaxy0", "Pretext Snapshot", "toolshed.g2.bx.psu.edu/repos/iuc/pretext_snapshot/pretext_snapshot/0.0.3+galaxy1", "YaHS", "toolshed.g2.bx.psu.edu/repos/iuc/yahs/yahs/1.2a.2+galaxy1"]


