# tutorials

topic: evolution
tutorial: bacterial-comparative-genomics
title: Phylogenetic analysis for bacterial comparative genomics
difficulty: introductory
time_estimation: 1H
requirements_json: []
tags_json: ["microgalaxy"]
content_hash: c2c4c4a1e3adfe80aa8bf4e2090a06d1
last_modified: 2026-05-27T09:01:48.560048
zenodo_link: https://zenodo.org/records/1
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/evolution/tutorials/bacterial-comparative-genomics/tutorial.html

content:
General introduction about the topic and then an introduction of the
tutorial (the questions and the objectives). It is nice also to have a
scheme to sum up the pipeline used during the tutorial. The idea is to
give to trainees insight into the content of the tutorial and the (theoretical
and technical) key concepts they will learn.

You may want to cite some publications; this can be done by adding citations to the
bibliography file (`tutorial.bib` file next to your `tutorial.md` file). These citations
must be in bibtex format. If you have the DOI for the paper you wish to cite, you can
get the corresponding bibtex entry using [doi2bib.org](https://doi2bib.org).

With the example you will find in the `tutorial.bib` file, you can add a citation to
this article here in your tutorial like this:
{% raw %} `{% cite Batut2018 %}`{% endraw %}.
This will be rendered like this: {% cite Batut2018 %}, and links to a
bibliography section which will automatically be created at the end of the
tutorial.


**Please follow our
[tutorial to learn how to fill the Markdown]({{ site.baseurl }}/topics/contributing/tutorials/create-new-tutorial-content/tutorial.html)**

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Galaxy and data preparation

> <hands-on-title> Data Upload </hands-on-title>
>
> 1. Create a new history for this analysis
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Rename the history
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 3. {% tool [Import](upload1) %} the contig file from [Zenodo]({{ page.zenodo_link }}) or from Galaxy shared data libraries:
>
>    ```
>    {{ page.zenodo_link }}/files/DRR187559_contigs.fasta
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
>
{: .hands_on}

# Tree building

> <hands-on-title> Task description </hands-on-title>
>
> 1. {% tool [FASTTREE](toolshed.g2.bx.psu.edu/repos/iuc/fasttree/fasttree/2.1.10+galaxy1) %} with the following parameters:
>    - *"Aligned sequences file (FASTA or Phylip format)"*: `fasta`
>        - {% icon param-file %} *"FASTA file"*: `output` (Input dataset)
>        - *"Set starting tree(s)"*: `No starting trees`
>    - *"Show advanced options"*: `No`
>
>    ***TODO***: *Check parameter descriptions*
>
>    ***TODO***: *Consider adding a comment or tip box*
>
>    > <comment-title> short description </comment-title>
>    >
>    > A comment about the tool or something else. This box can also be in the main text
>    {: .comment}
>
{: .hands_on}

***TODO***: *Consider adding a question to test the learners understanding of the previous exercise*

> <question-title></question-title>
>
> 1. Question1?
> 2. Question2?
>
> > <solution-title></solution-title>
> >
> > 1. Answer for question1
> > 2. Answer for question2
> >
> {: .solution}
>
{: .question}


# Conclusion

Sum up the tutorial and the key takeaways here. We encourage adding an overview image of the
pipeline used.

tools_json:
["Import", "upload1", "FASTTREE", "toolshed.g2.bx.psu.edu/repos/iuc/fasttree/fasttree/2.1.10+galaxy1"]


topic: evolution
tutorial: mtb_transmission
title: Identifying tuberculosis transmission links: from SNPs to transmission clusters
difficulty: intermediate
time_estimation: 2H
requirements_json: []
tags_json: ["prokaryote", "one-health", "microgalaxy"]
content_hash: 7ca2fd02cef3645703565eb18d347501
last_modified: 2026-05-27T09:01:48.562678
zenodo_link: https://zenodo.org/record/6010176
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/evolution/tutorials/mtb_transmission/tutorial.html

content:
In a disease outbreak situation, to understand the dynamics and the size of the outbreak, it is essential to detect transmission clusters to distinguish likely outbreak cases from unrelated background cases. Such detection is nowadays often based on actual sequencing data that enables quantitative conclusions about differences between pathogen isolates.

This tutorial guides you through transmission cluster identification from preprocessed whole-genome sequencing data of MTB strains. It consists of two parts:

1. Part one starts from per-sample lists of mutations/variants (in *variant call format*, VCF), as derived from sequencing data through mapping of sequenced reads to a reference genome followed by variant calling. From the lists of variants, more specifically from the single nucleotide variants found for every sample, you will construct a sample distance matrix and identify likely transmission clusters based on overall sample similarity.

2. Part two starts with drug-resistance profiles of the same samples and combines these profiles with the cluster information obtained in part one to enable reasoning about the validity of identified clusters and about evolution of drug resistance in the samples.

Both the lists of variants and the drug-resistance reports for all samples have been pre-generated for you, and can simply be imported into Galaxy from public sources.

> <comment-title>Origin of the input data</comment-title>
>
> This tutorial does *not* discuss the analysis steps that lead from sequencing data to lists of variants, nor the generation of drug-resistance profiles with the tool TB-Profiler. These topics are covered in the separate tutorial [M. tuberculosis Variant Analysis]({% link topics/variant-analysis/tutorials/tb-variant-analysis/tutorial.md %}) instead.
>
> If, after working through this and the other tutorial, you would like to combine the two to perform the complete analysis from sequenced reads to transmission clusters for the 20 samples used here, you can find their raw sequencing data in this [Zenodo record](https://zenodo.org/record/5911437), and two Galaxy workflows to process the data into VCFs and to generate drug-resistance profiles as [supplementary material](./workflows) to this tutorial.
>
{: .comment}

> <comment-title>Recommended background information</comment-title>
>
> A series of webinars has been produced alongside this tutorial to provide some theoretical background for the topics touched here. Specifically, these are:
>
> - [{% icon video %} Drug resistance prediction](https://youtu.be/Ddwt-_gQR2M)
> - [{% icon video %} Phylogenetic mutations](https://youtu.be/1ps_o5rpnmw)
> - [{% icon video %} The concept of clustering](https://youtu.be/l4cPUECJ7VU)
> - [{% icon video %} Genetic distance thresholds](https://youtu.be/kKNgmpy1N94)
>
> Watching at least some of them before or after doing the tutorial is highly recommended.
>
{: .comment}


> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Analysis Part 1: Identification of transmission clusters from per-sample variants

## Get the data

Any analysis should get its own Galaxy history. So let's start by creating a new one:

> <hands-on-title>Prepare the Galaxy history</hands-on-title>
>
> 1. Create a new history for this analysis
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Give the history a suitable name
>
>    {% snippet faqs/galaxy/histories_rename.md name="MTB transmission clusters tutorial" %}
>
{: .hands_on}

As mentioned in the introduction, mapping and variant calling have already be performed for the 20 samples
that we need to analyze. The result of this preprocessing are 20 VCF files that describe the mutations found
for each of the samples, and which we now need to import into Galaxy.

The fastest way to do so, which lets us build a collection from the 20 datasets with correctly named elements (corresponding to the sample identifiers) directly during data upload is through the so-called *Rule-based uploader*. The Galaxy training material has dedicated tutorials, [Rule Based Uploader]({% link topics/galaxy-interface/tutorials/upload-rules/tutorial.md %}) and [Rule Based Uploader Advanced]({% link topics/galaxy-interface/tutorials/upload-rules-advanced/tutorial.md %}), that introduce this powerful feature in detail.

For the purpose of this tutorial, it is enough if you simply follow the step-by-step instructions provided here exactly.

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Import the following files from [Zenodo]({{ page.zenodo_link }})
>
>    ```
>    https://zenodo.org/record/6010176/files/ERR1203059.vcf
>    https://zenodo.org/record/6010176/files/ERR181435.vcf
>    https://zenodo.org/record/6010176/files/ERR2659153.vcf
>    https://zenodo.org/record/6010176/files/ERR2704678.vcf
>    https://zenodo.org/record/6010176/files/ERR2704679.vcf
>    https://zenodo.org/record/6010176/files/ERR2704687.vcf
>    https://zenodo.org/record/6010176/files/ERR313115.vcf
>    https://zenodo.org/record/6010176/files/ERR551620.vcf
>    https://zenodo.org/record/6010176/files/ERR5987300.vcf
>    https://zenodo.org/record/6010176/files/ERR5987352.vcf
>    https://zenodo.org/record/6010176/files/ERR6362078.vcf
>    https://zenodo.org/record/6010176/files/ERR6362138.vcf
>    https://zenodo.org/record/6010176/files/ERR6362139.vcf
>    https://zenodo.org/record/6010176/files/ERR6362156.vcf
>    https://zenodo.org/record/6010176/files/ERR6362253.vcf
>    https://zenodo.org/record/6010176/files/ERR6362333.vcf
>    https://zenodo.org/record/6010176/files/ERR6362484.vcf
>    https://zenodo.org/record/6010176/files/ERR6362653.vcf
>    https://zenodo.org/record/6010176/files/SRR13046689.vcf
>    https://zenodo.org/record/6010176/files/SRR998584.vcf
>    ```
>
>    1. Copy the links above
>    2. Open the {% tool [Upload](upload1) %} Manager
>    3. In the top row of tabs select **Rule-based**
>    4. Set **Upload data as** to `Collection(s)`
>    5. Paste the copied links into the text field on the right
>    6. Click **Build** to bring up the *Rule Builder* dialog
>    7. Click on the wrench icon {% icon tool %} next to **Rules** in the left panel of the window
>    8. In the text field, erase the prefilled content and copy/paste the following rule definition into it instead:
>
>       ```
>       {
>         "rules": [
>           {
>             "type": "add_column_basename",
>             "target_column": 0
>           },
>           {
>             "type": "add_column_substr",
>             "target_column": 1,
>             "length": 4,
>             "substr_type": "drop_suffix"
>           }
>         ],
>         "mapping": [
>           {
>             "type": "url",
>             "columns": [
>               0
>             ]
>           },
>           {
>             "type": "list_identifiers",
>             "columns": [
>               2
>             ],
>             "editing": false
>           }
>         ]
>       }
>       ```
>    9. Click on **Apply**
>    10. In the bottom row of options, change **Type** from "Auto-detect" to `vcf`
>    11. Set the name of the new collection to `MTB sample variants`
>    12. Click on **Upload**
>
>    It is ok to continue with the next step even while the upload of this data is still ongoing.
>    Just click on the **Close** button of the information pop-up window if it is still open when you are ready to proceed.
> 2. {% tool [Upload](upload1) %} the reference genome that was used for SNP calling from:
>
>    ```
>    https://zenodo.org/record/3497110/files/MTB_ancestor_reference.fasta
>    ```
>
>    and make sure the dataset format is set to `fasta`.
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md format="fasta" %}
>
{: .hands_on}


## Generate a SNP alignment

In this tutorial we aim to calculate the genetic differences (in SNPs) between pairs of MTB genomes.
To do so, we need to compare, between each pair of genomes (thus pairwise), the nucleotides that are
observed at each position. Each time we find a different nucleotide at a given position, we will sum
1 SNP of genetic distance. For example, if two strains have a genetic distance of 5 SNPs, that would
mean that their respective genomes are almost identical, except for 5 positions along all the genome
in which they have different nucleotides.

To do such calculation we need to first build an alignment of all the genomes (a multiple-sequence alignment, or MSA).
Afterwards, we will use specific software to analyze this MSA, count SNPs, and thus calculate the genetic distance between each pair of samples.

![MSA of 5 MTB genomes](./images/MSA.png "A SNP is highlighted in a MSA of five MTB genomes")

### Generate complete genomes

The first step to generate the genomes MSA will be... to get the complete genomes of our samples!
In the [MTB Variant Analysis tutorial]({% link topics/variant-analysis/tutorials/tb-variant-analysis/tutorial.md %}) we have analyzed short-read high-throughput sequencing data (Illumina) to
obtain the respective VCF files that describe the mutations found in each of our samples, as compared
to the reference genome. We can now use these VCF files to build the complete genome of each of our
samples.

#### Filter VCF files for epidemiological/phylogenetic investigation
Interpreting **mixed calls** or **indels** in phylogenetic/epidemiological applications can be very
complicated. That is the reason why we typically use alignments that only contain **fixed SNPs**.

Thus, the first step in this tutorial will be to filter the VCFs so we are sure that
they only contain **fixed SNPs**, which we (somewhat arbitrarily) define as those single-nucleotide variants that are observed at an allele frequency of 90% or greater. We will be using the tool **TB Variant Filter** for this task.

*Note: TB variant Filter refers to SNPs as SNVs. These two short forms are interchangeable, meaning **S**ingle **N**ucleotide **P**olymorphism and **S**ingle **N**ucleotide **V**ariant, respectively.*

> <hands-on-title>Filter VCF files for epidemiological investigation</hands-on-title>
>
> 1. {% tool [TB Variant Filter](toolshed.g2.bx.psu.edu/repos/iuc/tb_variant_filter/tb_variant_filter/0.3.6+galaxy0) %} with the following parameters:
>    - {% icon param-collection %} *"VCF file to be filter"*: `MTB variants per sample`; the uploaded collection of VCF datasets
>    - *"Filters to apply"*: `Only accept SNVs`, `Filter variants by percentage alt allele`
>    - *"Show options for the filters"*: `Yes`
>    - *"Minimum alternate allele percentage to accept"*: `90.0`
>
> > <question-title></question-title>
> >
> > **TB Variant Filter** reads the VCF and output only SNPs that have, at least, 90% frequency.
> > How can the software extract such information from the VCF datasets?
> >
> > > <solution-title></solution-title>
> > >
> > > That information is contained, for each mutation, in the VCF:
> > > - The `TYPE` field within the INFO string will tell us if the mutation is a SNP (TYPE=snp)
> > > - You can look for other types of mutations like insertions (TYPE=ins)
> > > - The `AF` field within the INFO string describes the estimated **A**llele **F**requency
> > > - In the absence of the `AF` field, the allele frequency can be calculated as: `AO` (alternate allele observations) / `DP` (depth of reads at the variant site)
> > {: .solution}
> {: .question}
{: .hands_on}

#### Reconstruct the complete genome of each sample
Our new collection of VCF datasets now only contains **fixed SNPs** that were found in the genome of the respective strains.
Genomic positions not in the VCF mean that, at that particular position, the strain has the
same nucleotide than the reference genome. Knowing this information, one could reconstruct the
complete genome of each strain pretty easily. From the first to the last position in the genome,
one would put the same nucleotide than in the reference if that position is not in the VCF, or the
SNP described in the VCF otherwise. This is exactly what the tool **bcftools consensus** will do for us,
given the reference genome and the VCF of the strain we want to reconstruct the genome for.

> <hands-on-title>Reconstruct the complete genome of each sample</hands-on-title>
>
> 1. {% tool [bcftools consensus](toolshed.g2.bx.psu.edu/repos/iuc/bcftools_consensus/bcftools_consensus/1.15.1+galaxy3) %} with the following parameters:
>    - {% icon param-collection %} *"VCF/BCF Data"*: the collection of filtered variants per sample; output of **TB Variant Filter**
>    - *"Choose the source for the reference genome"*: `Use a genome from the history`
>        - {% icon param-file %} *"Reference genome"*: the uploaded MTB ancestor reference fasta dataset (the reference genome that was used for SNP-calling)
>        - *"Set output FASTA ID from name of VCF"*: `Yes`
>
{: .hands_on}

> <question-title></question-title>
>
> Imagine that we forgot to filter the VCFs to contain only fixed variants, and there are also
> SNPs with frequencies, of 15%, 30%, or 56.78%. Which allele do you think bcftools consensus would
> insert in the genome?
>
> > <solution-title></solution-title>
> > The behaviour of bcftools consensus in this case can be specified with the option `--haplotype`
> > For example, we can set `haplotype=2` so the second allele will be used... wait... what?
> {: .solution}
{: .question}

> <question-title>Second allelle!?</question-title>
>
> What do you think that things like "second allele" or "*The* alterntive allele" mean here?
>
> > <solution-title></solution-title>
> > Many of the bioinformatic programs are developed to analyze eukaryotic genomes, particularly
> > human genomes. That means that these programs have in mind that the genomes
> > are diploid and thus each posible position in the genome has two possible alleles. In
> > bacterial genomics, in contrast, we are **always sequencing a population** of cells with
> > potential genetic diversity (with the exception of single-cell sequencing).
> > That does not mean that we cannot use this type of software, we can (and we do!) but it is
> > good to know what they are meant for, and their possible limitations.
> {: .solution}
{: .question}

### Multiple-sequence alignment (MSA) of all genomes

Multiple sequence alignment is a process in which multiple DNA, RNA or protein sequences are arranged
to find regions of similarity that are supposed to reflect the consequences of different evolutionary
processes (Figure 1). MSAs are used to test hypotheses about these evolutionary processes and to infer phylogenetic
relationships, and for these reasons we build MSA for sequences for which we already assume some sort
of evolutionary relationship. You will learn more on MSAs and phylogenetic inference in the next
tutorial.

Building MSAs of several complete genomes can be a complicated process and computationally demanding. To perform
such a task there are many software packages available like **Muscle**, **MAFFT** or **Clustal**, just to mention some.
Which one are we going to use in this tutorial? Well, we are going to use a trick. We are going to
just stack one genome on top of each other within a text file. (More on why we can do this below).

Our aim is to generate a **multifasta** dataset in which the genomes of our samples are aligned.
Something that looks like this:

```
>Sample 1
TTGACCGATGACCCCGGTTCAGGCTTCACCACAGTGTGGAACGC...
>Sample 2
ATGACCGATGACCCCGGTTCAGGCTTCACCACAGTGTGGAACGC...
>Sample 3
TTCGTTCGATGACCCCGGTTCAGGCTTCACCACAGTGTGGAACG...
```

This could be done manually, by copy-pasting all genomes in a single text file.
However we can do the same with a specific command that *concatenates* datasets.

> <hands-on-title>Concatenate genomes to build a MSA</hands-on-title>
>
> 1. {% tool [Concatenate datasets tail-to-head (cat)](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.1) %} with the following parameters:
>    - {% icon param-collection %} *"Datasets to concatenate"*: the collection of consensus genomes; output of **bcftools consensus**
>
{: .hands_on}

Now we have a multifasta file, where **each position of each genome corresponds to the same position
of the rest of genomes in the file**. This can be seen and used as a multiple-sequence alignment of
all of our genomes! However, it is important that you understand the following question...

> <question-title></question-title>
>
> Generating multiple-sequence alignments can be complicated and computationally demanding, and there are
> many software packages to perform such task. How is it possible that we were able to build a MSA by just
> stacking genomes one on top of each other? Can you think about what makes our case special, so we can
> just use this "trick"?
>
> > <solution-title></solution-title>
> >
> > We have generated the complete genome of each sample by substituting in the reference genome
> > those **SNPs** that we found in that said sample (described in the VCF). Remember that **we
> > removed indels from the VCF when filtering!** Because the complete genomes we generated do not
> > contain insertions or deletions, ALL the genomes have the same length (the length of the
> > reference genome) and each nucleotide corresponds to the same genomic coordinate (the one
> > also in the reference genome). So we are not aligning genomes *per se* and, knowing this, we
> > can build a MSA by just stacking genomes that **are the same length >>AND<< have the same coordinates**.
> {: .solution}
>
{: .question}

### Remove invariant positions

We have generated a MSA that is the basis for the transmission (clustering) and phylogenetic
analysis. Although we could already use this MSA for such analysis, it is common practice to remove
the invariant sites from the alignment. Think that our file now contains 20 genomes of 4.4 Mb each.
MTB genomes have a very low genetic diversity, meaning that in reality, there are only some hundreds or
thousands of SNPs *in total*, because the genomes are >99% identical between them. Identical positions
in a MSA provide no information, so we can remove those and
generate a **SNP alignment** that only contain variant positions with phylogenetic information. By
doing this, we will generate a much smaller file, that will be easier to handle by downstream applications.

We can exemplify this with a couple of pictures:

In the following picture, a polymorphic position in the alignment (SNP)
is highlighted in blue and green. Invariant positions are not highlighted and have an asterisc `*`
on the upper part of the alignment. Most of the MSA is composed of these invariant positions, making
our file larger than necessary.

![MSA of 5 MTB genomes](./images/MSA.png "A SNP is highlighted in a MSA of five MTB genomes")

After removing invariant positions, we end up with a SNP alignment like the following.

![MSA of 5 MTB genomes](./images/SNP_MSA.png "A SNP alignment where all positions are polymorphic")

> <hands-on-title>Removing invariant sites from a MSA</hands-on-title>
>
> 1. {% tool [Finds SNP sites](toolshed.g2.bx.psu.edu/repos/iuc/snp_sites/snp_sites/2.5.1+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"FASTA file"*: the concatenated consensus genomes; output of **Concatenate datasets**
>    - *"Output"*: `Sequence alignment / VCF`
>        - *"Output formats"*: `Multi-FASTA alignment file`
>
{: .hands_on}

In SNP alignments you have to **bear in mind that positions do not longer correspond to genomic
coordinates**, meaning that two contiguous nucleotides may correspond to coordinates thousands of
positions apart.

## Identify transmission clusters

### Calculate pairwise SNP distances

Now we are all set to calculate pairwise SNP distances between samples and decide whether two
patients are within the same transmission cluster or not. Having a SNP alignment, this is fairly
easy. We will use **SNP distance matrix**, that will generate a matrix with pairwise SNP distances.

> <hands-on-title>Distance matrix from SNP alignment.</hands-on-title>
>
> 1. {% tool [SNP distance matrix](toolshed.g2.bx.psu.edu/repos/iuc/snp_dists/snp_dists/0.8.2+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"FASTA multiple sequence alignment"*: the concatenated SNP sites-only sequences; output of **Finds SNP sites**
>
> > <comment-title></comment-title>
> >
> > Have a look at the distance matrix to make sure you understand the whole process. Given that
> > we only have 20 samples, you could already spot some samples that are involved in the same
> > transmission chain (samples with a small number of SNPs between them as explained below).
> {: .comment}
>
{: .hands_on}

### Determine transmission clusters based on a SNP threshold

Now that we have a distance matrix that describes the SNP distance between each pair of samples, we
could already describe the transmission clusters based on a **SNP threshold**, as [explained in the
respective webinar](https://youtu.be/kKNgmpy1N94) . If two samples are at a distance below that threshold, we will say that they
belong to the same transmission cluster, because they are close enough genetically speaking. Again,
we could do this manually, but we are doing bioinformatics, and we want to be able to do the same
analysis regardless of whether we are analyzing two or two million samples. Also, **note that two samples
that are dozens of SNPs apart may belong to the same transmission cluster** if there are other samples
linking them in between as exemplified in the picture below.

![MSA of 5 MTB genomes](./images/Cluster.png "Example of transmission cluster using 10 SNPs threshold")

> <question-title>Very Important Question</question-title>
>
> 1. In the image above exemplifying a transmission cluster, the distance between samples A and E is
> 17 SNPs. Being the other pairwise distances in the figure the same,
> would it be possible that the distance between A and E is different?
>
> > <solution-title></solution-title>
> >
> > 1. The figure used above as an example is a **flagrant oversimplification**. In the figure not all pairwise
> > distances are represented (for example between sample A and C).
> > Most importantly you have to remember that **transmission clusters do not reflect transmission
> > events**.
> > In fact it may happen that transmission does not happen within the cluster we are analyzing!
> > For example, when a patient that is not sampled is the source of infection of all the cases in
> > the cluster (may be a superspreading event). You have to consider that, taking into account the
> > same SNP distances, another possible (yet still oversimplified) scenario could be...
> > ![MSA of 5 MTB genomes](./images/Cluster2.png "Example of transmission cluster using 10 SNPs threshold")
> {: .solution}
{: .question}

### Determine transmission clusters 
The Galaxy team has implemented a tool that reads a SNP distance matrix like the one we just calculated and 
assigns clusters based on a distance threshold. We will use this tool to perform a clustering with a threshold
of 12 SNPs.

> <hands-on-title>Distance matrix from SNP alignment.</hands-on-title>
>
> 1. {% tool [Distance matrix-based hierarchical clustering tool metadata](toolshed.g2.bx.psu.edu/repos/iuc/clustering_from_distmat/clustering_from_distmat/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Distance matrix"*: SNP distance matrix; output of **SNP distance matrix**
>    - *"Clustering method"*: UPGMA (scipy 'average' method)
>    - *"Generate cluster assignments?"*: Yes, and use distance threshold to divide into clusters
>        - *"Distance threshold for sample clustering"*: **12**
>
{: .hands_on}

> <question-title></question-title>
>
> How many transmission clusters did we find? How many samples are linked to recent transmission in our dataset?
>
> > <solution-title>Answer</solution-title>
> > We have found two transmission clusters with respective IDs 1 and 2. Transmission cluster 1
> > is composed of two samples,*ERR5987352* and *ERR6362484* , linked by recent transmission and transmission cluster 2 of three samples, *ERR6362138*, *ERR6362156* and *ERR6362253*.
> {: .solution}
{: .question}

> <question-title></question-title>
>
> Let's assume that we have the isolation dates of samples ERR6362484 and ERR5987352, which
> belong to the same transmission cluster. Sample ERR6362484 was isolated on January 2021, while sample
>  ERR5987352 was isolated on September 2021. Would you be able to determine who was the infector and who the infectee?
>
> > <solution-title></solution-title>
> > **NO**
> >
> >  Isolation dates have been used traditionally to define **index cases** within transmission clusters
> > under the assumption that the most likely scenario is the first isolated sample to be the
> > source of transmission. Today we know that this assumption often leads
> > to misidentification of index cases. **Remember:** we cannot rule out the possibility
> > that patients within the cluster were infected by an index case that was not sampled.
> >
> >  Read {% cite Xu_2019 %} for more information on this topic.
> {: .solution}
{: .question}


# Analysis Part 2: Combining cluster information with drug-resistance reports

Although we have stressed the fact that **clustering cannot be used to delineate
transmission events**, clustering is very useful to investigate outbreaks and determine which cases are
involved in the same transmission chain. We can leverage this information to investigate the relationship
between tuberculosis transmission and particular biological or clinical traits.

In this part of the tutorial, we will investigate the emergence and spread of drug resistance based
on our clustering analysis.

## Get the data

The [MTB Variant Analysis tutorial]({% link topics/variant-analysis/tutorials/tb-variant-analysis/tutorial.md %})
demonstrates the use of **TB-profiler** to generate a report with determinants of drug resistance of a
particular MTB strain, and predict its genotypic drug susceptibility. We have done **exactly the same**
for the 20 samples that we used in the clustering analysis, so we have now the TB-profiler reports for
all of them.

Like in part 1, we can use Galaxy's rule-based uploader functionality to upload the data, build a collection from it and name its elements correctly, all in one step.

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Import the TB-profiler reports for the samples
>
>    ```
>    https://zenodo.org/record/6010176/files/ERR1203059.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR181435.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR2659153.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR2704678.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR2704679.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR2704687.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR313115.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR551620.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR5987300.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR5987352.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR6362078.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR6362138.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR6362139.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR6362156.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR6362253.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR6362333.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR6362484.TBprof.txt
>    https://zenodo.org/record/6010176/files/ERR6362653.TBprof.txt
>    https://zenodo.org/record/6010176/files/SRR13046689.TBprof.txt
>    https://zenodo.org/record/6010176/files/SRR998584.TBprof.txt
>    ```
>
>    1. Copy the links above
>    2. Open the {% tool [Upload](upload1) %} Manager
>    3. In the top row of tabs select **Rule-based**
>    4. Set **Upload data as** to `Collection(s)`
>    5. Paste the copied links into the text field on the right
>    6. Click **Build** to bring up the *Rule Builder* dialog
>    7. Click on the wrench icon {% icon tool %} next to **Rules** in the left panel of the window
>    8. In the text field, erase the prefilled content and copy/paste the following rule definition into it instead:
>
>       ```
>       {
>         "rules": [
>           {
>             "type": "add_column_basename",
>             "target_column": 0
>           },
>           {
>             "type": "add_column_substr",
>             "target_column": 1,
>             "length": 11,
>             "substr_type": "drop_suffix"
>           }
>         ],
>         "mapping": [
>           {
>             "type": "url",
>             "columns": [
>               0
>             ]
>           },
>           {
>             "type": "list_identifiers",
>             "columns": [
>               2
>             ],
>             "editing": false
>           }
>         ]
>       }
>       ```
>    9. Click on **Apply**
>    10. In the bottom row of options, change **Type** from "Auto-detect" to `txt`
>    11. Set the name of the new collection to `TB profiler reports`
>    12. Click on **Upload**
>
{: .hands_on}

## Summarize the data

TB-profiler reports are very useful and comprehensive, and we will use them to better investigate
drug resistance in our dataset. However, it is always useful to summarize the data on a per-sample basis,
on a table, so we can quickly check which strains are, for example, MDR, an which are susceptible to antibiotics.
We would like to generate a table like the following:

| Sample   | DR profile |
|----------|------------|
| Sample A | Sensitive  |
| Sample B | Sensitive  |
| Sample C | MDR        |
| Sample Z | XDR        |

and merge this with the tabular per-sample cluster report that we have generated in the first part of the tutorial.

If we take a look at a TB profiler report we can see that there is one line describing the genotypic drug susceptibility.
Let's have a look at the first part of the TB-profiler report for sample ERR6362653:

```

TBProfiler report
=================

The following report has been generated by TBProfiler.

Summary
-------
ID: tbprofiler
Date: Fri Jan 28 13:14:47 2022
Strain: lineage2.2.1
Drug-resistance: MDR

Lineage report
--------------
Lineage	Estimated Fraction	Family	Spoligotype	Rd
lineage2	1.000	East-Asian	Beijing	RD105
lineage2.2	0.996	East-Asian (Beijing)	Beijing-RD207	RD105;RD207
lineage2.2.1	0.999	East-Asian (Beijing)	Beijing-RD181	RD105;RD207;RD181
```

As you can see, this strain is multi-drug resistant as indicated by (`Drug-resistance: MDR`)
We could then look for this information in each TB-profiler report and generate this table manually,
for example in a spreadsheet. However this is not feasible when analyzing hundreds or thousands of samples
(and very **error-prone!**).

A far more scalable approach is to combine Galaxy tools to automate the necessary data processing steps for us.

### Select the line containing the drug resistance profile in the report of every sample

**grep** is used to search patterns of text within text files. Each time **grep** finds that
pattern, it will print as a result **the complete line** containing such pattern.

If we use **grep** to search for the pattern "`Drug-resistance:`", in a TB-profiler file, we will get
as output the complete line, for example `Drug-resistance: MDR`

> <hands-on-title>Keep only `Drug-resistance:` lines from TB-profiler reports</hands-on-title>
>
> 1. {% tool [Select lines that match an expression](Grep1) %} with the following parameters:
>    - {% icon param-collection %} *"Select lines from"*: the uploaded collection of TB-profiler reports
>    - *"that"*: `Matching`
>    - *"the pattern"*: `Drug-resistance:`
>
{: .hands_on}

### Collapse collection into a table with each line prepended with a sample name

Next, we want to turn the collection of single-line datasets we just created into a single two-column table with
one column of sample IDs and another one with the corresponding drug resistance status.
> <hands-on-title>Prepend the sample name to the DR profile</hands-on-title>
>
> 1. {% tool [Collapse Collection](toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/5.1.0) %} with the following parameters:
>    - {% icon param-collection %} *"Collection of files to collapse into single dataset"*: the collection of single-line drug-resistance reports; output of **Select lines**
>    - *"Prepend File name"*: `Yes`
>      - *"Where to add dataset name"*: `Same line and each line in dataset`
>
{: .hands_on}

### Clean up the table

In this step we will use a simple tool that searches and replaces text to remove the redundant string "Drug-resistance: " from our table by replacing it with *nothing*.

> <hands-on-title>Removing redundant content</hands-on-title>
>
> 1. {% tool [Replace Text](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_line/1.1.2) %} with the following parameters:
>    - {% icon param-file %} *"File to process"*: `Single file` (output of **Collapse Collection** {% icon tool %})
>    - In *"Replacement"*:
>        - {% icon param-repeat %} *"Insert Replacement"*
>          - *"Find pattern"*: `Drug-resistance: `
>
>              Note the space at the end of the pattern!
>          - *"Replace with"*: leave this field empty
>
> 2. Once the tool has finished running, change the format of the output to **tabular**
>
>    Though by now the content of the dataset looks like a nice table, Galaxy still thinks it is in general **txt** format because the original TB-profiler reports were of that format.
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="tabular" %}
>
{: .hands_on}

> <question-title></question-title>
>
> 1. How many MDR strains did we find in the dataset?
> 2. What does it mean to be Pre-MDR?
>
> > <solution-title></solution-title>
> > 1. Eight MDR strains, three of which are pre-XDR because they have additional resistance to fluoroquinolones. (You can look into details by looking into the TB profiler reports).
> > 2. As MDR means to be resistant to INH and RIF, pre-MDR means to be either INH-monoresistant or RIF-monoresistant.
> > If we have a look at the respective TB-profiler reports, we can see that these three strains are RIF-monoresistant.
> {: .solution}
{: .question}

## Combine the drug-resistance and the cluster tables

The two tables we have generated in the two parts of the tutorial share a sample column, which we can use now to join the two tables.

> <hands-on-title>Joining tables</hands-on-title>
>
> 1. {% tool [Join two files](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_easyjoin_tool/1.1.2) %} with the following parameters:
>    - {% icon param-file %} *"1st file"*: transmission clusters report; output of *Distance matrix-based hierarchial clustering*
>    - *"Column to use from 1st file"*: `Column: 1`
>    - {% icon param-file %} *"2nd File"*: final drug-resistance report; output of **Replace Text** above
>    - *"Column to use from 1st file"*: `Column: 1`
>    - *"Output lines appearing in"*: `Both 1st & 2nd file`
>
>
{: .hands_on}


# Interpretation of results

Now that we have performed a clustering analysis and know which DR mutations each strain carries,
we can try to answer a series of questions about how DR may have been emerging and spreading in our study
population.

The final combined table you generated and that we are going to base some questions on, should look similar to this one (header added and sample order rearranged for clarity here):

| Sample       | Cluster_id | DR profile |
|--------------|------------|------------|
| ERR1203059   | -          | Sensitive  |
| ERR181435    | -          | Sensitive  |
| ERR2659153   | -          | Sensitive  |
| ERR2704678   | -          | Sensitive  |
| ERR2704679   | -          | Sensitive  |
| ERR2704687   | -          | Sensitive  |
| ERR313115    | -          | Sensitive  |
| ERR551620    | -          | MDR        |
| ERR5987300   | -          | Pre-XDR    |
| ERR6362078   | -          | MDR        |
| ERR6362139   | -          | Pre-MDR    |
| ERR6362333   | -          | Pre-XDR    |
| ERR6362653   | -          | MDR        |
| SRR13046689  | -          | Other      |
| SRR998584    | -          | Sensitive  |
| ERR5987352   | 1          | Pre-MDR    |
| ERR6362484   | 1          | Pre-MDR    |
| ERR6362138   | 2          | MDR        |
| ERR6362156   | 2          | Pre-XDR    |
| ERR6362253   | 2          | MDR        |

> <question-title></question-title>
>
> Assuming that we have a very good sampling of the outbreak. Which strains **may** represent instances
> of *de novo* evolution of drug resistance and which ones instances of *transmitted* (primary) resistance?
> Remember that you can look at the TB-profiler reports of independent samples for detailed information.
>
> > <solution-title></solution-title>
> > In a simplistic scenario, we could consider clustered strains as instances of transmission and
> > unclustered strains as instances of de novo evolution of DR. Thus, we see that for example there
> > are three MDR strains (ERR551620, ERR6362078, ERR6362653) that are unclustered and therefore may
> > represent cases in which drug resistance evolved independently as a consequence of treatment within the respective
> > patients. However you need to always bear in mind that, although **within our population** those MDR
> > strains do not seem to be linked to transmission, this does NOT rule out the possibility that
> > some of these patients were infected with an MDR strain somewhere else, or that other cases from the cluster where simply not sampled.
> >
> > When looking at clustered strains, distinguishing between transmitted and *de-novo* may be tricky. Note
> > that, for example, for the two RIF-monoresistant strains linked within the same transmission
> > cluster there are, at least, a couple of possible scenarios: one in which a RIF-monoresistant strain
> > evolved in one patient and was transmitted to the other patient afterwards, and one in which both
> > were infected with the same RIF-monoresistant strain from a third patient that we have not sampled.
> > We need to note that, in the first scenario, drug resistance evolved *de novo* in one patient,
> > and was *transmitted* to the other patient, whereas in the second scenario drug resistance was
> > *transmitted* in both cases.
> {: .solution}
{: .question}

> <question-title></question-title>
>
> 1. The same principles as explained above apply to the three MDR strains that are
>    within the same transmission cluster. In this case, however, there is one strain that shows clear
>    evidence of *de-novo* evolution of DR. Do you know which strain and why?
>
> > <solution-title></solution-title>
> > 1. Within this cluster of MDR strains, there is one tagged as Pre-XDR by TB-profiler. If we have
> >    a look at the TB profiler report, we can see that this strain carries an additional mutation in
> >    *gyrA* that confers resistance to fluoroquinolones. This is compatible with an scenario in which
> >    fluoroquinolone resistance evolved independently within this patient after being infected with
> >    the MDR strain.
> {: .solution}
{: .question}

> <question-title></question-title>
>
> There is one strain with a DR profile "other", because it is only resistant to pyrazinamide. This
> strain is not within a transmission cluster. Therefore, we might conclude that pyrazinamide resistance
> most likely evolved *de-novo* in this patient due to antibiotic treatment, but we would be wrong. Do you know why?
>
> > <solution-title></solution-title>
> > The strain is indeed PZA-resistant. And indeed this is strain is NOT linked to transmission
> > within our population. However, if we have a look at the TB-profiler report, we observe that this
> > is a *M. bovis* strain, many of which are known to be intrinsically resistant to PZA.
> {: .solution}
{: .question}

> <question-title></question-title>
>
> 1. Is it possible to find, in the same transmission cluster, two RIF-monoresistant strains that
>    carry different rpoB mutations?
> 2. Is it possible to find, in the same transmission cluster, strains of different MTB sublineages?
>
> > <solution-title></solution-title>
> > 1. Yes, it is **possible**. In that scenario, the **same susceptible strain** would **recently**
> >    have been transmitted to two patients, and RIF resistance would have evolved **independently** in both.
> > 2. No, by definition. Remember that clustering is based on a threshold that we set of genetic
> >    distance measured in SNPs. We want to cluster samples that are genetically so similar that we
> >    can consider them as the same genotype, that is to say, as the same strain. Two different
> >    sublineages, by definition, do not belong to the same genotype and will have a distance in SNPs
> >    between them well beyond any SNP threshold we could use.
> {: .solution}
{: .question}


# Conclusion

You have learned how to perform a clustering analysis to identify patients that are linked by events of
**recent transmission**. Clustering analysis is very useful in outbreak investigation and
can also be used to describe the emergence and spread of drug-resistance within a population. You have
also learned, however, that interpreting clustering results requires careful considerations, given the
limitations of the methodology. Clustering analysis is better complemented with phylogenetic analysis,
which may help overcome some of these limitations.

In the following tutorial you will perform a phylogenetic analysis of these same 20 strains.


# Bonus

You might have noticed that one of the strains analyzed presents thousands of differences (SNPs) to
the reference genome, standing out from the rest of strains. This strain is a *M. canettii* strain,
that was actually not part of the outbreak investigated. However we decided to include it here. Why? Let's find out
in the follow-up tutorial [Tree thinking for tuberculosis evolution and epidemiology]({% link topics/evolution/tutorials/mtb_phylogeny/tutorial.md %}).

tools_json:
["Upload", "upload1", "TB Variant Filter", "toolshed.g2.bx.psu.edu/repos/iuc/tb_variant_filter/tb_variant_filter/0.3.6+galaxy0", "bcftools consensus", "toolshed.g2.bx.psu.edu/repos/iuc/bcftools_consensus/bcftools_consensus/1.15.1+galaxy3", "Concatenate datasets tail-to-head (cat)", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.1", "Finds SNP sites", "toolshed.g2.bx.psu.edu/repos/iuc/snp_sites/snp_sites/2.5.1+galaxy0", "SNP distance matrix", "toolshed.g2.bx.psu.edu/repos/iuc/snp_dists/snp_dists/0.8.2+galaxy0", "Distance matrix-based hierarchical clustering tool metadata", "toolshed.g2.bx.psu.edu/repos/iuc/clustering_from_distmat/clustering_from_distmat/1.1.1", "Select lines that match an expression", "Grep1", "Collapse Collection", "toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/5.1.0", "Replace Text", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_line/1.1.2", "Join two files", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_easyjoin_tool/1.1.2"]


topic: evolution
tutorial: abc_intro_phylo
title: Phylogenetics - Back to basics
difficulty: intermediate
time_estimation: 4H
requirements_json: []
tags_json: ["phylogenetics", "evolution"]
content_hash: 9a582f22e98a3d5ce5e574c75bfdf88e
last_modified: 2026-05-15T22:28:12.948138
zenodo_link: https://zenodo.org/records/11089653
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/evolution/tutorials/abc_intro_phylo/tutorial.html

content:
Phylogenetics is essential for comparing biological species and understanding biodiversity for conservation. This tutorial discusses the basic principles and methods of phylogenetic inference and what you can learn from phylogenetic estimation. It is intended to help you make informed decisions about which methods to use in your research.

{% include _includes/slides-youtube-selector.html id="dmCCQRliEMk" title="Introductory Lecture: Welcome and why phylogenetics?" slides="topics/evolution/tutorials/abc_intro_phylo-intro" %}

Using real-life data and standard tools that are (mostly) available in Galaxy, the tutorial demonstrates the principles behind a variety of methods used to estimate phylogenetic trees from aligned sequence data or distance data.

This is not just a "how to" tutorial, but is instead aimed at giving you a better understanding of the principles of phylogenetics and how the methods work. Maybe you've even built phylogenetic trees before but want to know more about the principles behind the tools.

This tutorial does not cover workflows for taking read data to phylogeny or Bayesian methods. We've included recommended reading and tutorials on these topics in the resources section.

We’ve designed this tutorial with flexibility in mind and so that you can focus on the sections that are relevant to you. It includes videos that explore key concepts, written descriptions of each concept, and exercises that demonstrate the methods in action using tools (mostly) available in Galaxy. The exercises are beginner level, but you should know how molecular sequence data is produced and what it looks like. Depending on how you like to learn, you can choose to watch the videos, read the text, work through the exercises, or a combination of the three!

This tutorial is adapted from a 2019 workshop run by the Australian BioCommons and Professor Michael Charleston (University of Tasmania).

> <agenda-title></agenda-title>
>
> In this tutorial we will deal with:
>
> 1. TOC
> {:toc}
>
> **We will not cover:**
>
> * Workflows from read data to sequences
> * How to get an alignment (much: will use automated methods)
> * Bayesian phylogenetics: MCMC / BEAST / MrBayes
{: .agenda}

<!-- Original notes on scope. Now incorporated into the agenda box

## Scope:
We will cover:

* What a phylogeny is
* Motivation:
	* conservation / biodiversity,
	* fundamental knowledge,
	* designing vaccines
* Why it’s hard to find a phylogeny
	* How long it takes
	* We cannot go back and check
	* The role of simulation
	* The plethora of methods (nucleic acid sequence based, morphometric based (body shape)) making it hard to choose
* Data
	* What a sequence alignment is and how to interpret one;
	* What to do with difficult data (lots of gaps in the alignment)
	* The data we will use: an alignment of malaria sequences.
* Three different main approaches:
	* Distance-based
	* Parsimony
	* Likelihood (simple models only)
  * Phylogenetic networks
* Comparing phylogenies --- what to do when they aren't all the same
* Analysing robustness using bootstrapping
* A little on what can possibly (probably) go wrong?
* An overview of a workflow

We will not cover:

* Workflows from read data to sequences
* How to get an alignment (much: will use automated methods)
* Bayesian phylogenetics: MCMC / BEAST / MrBayes



## Overview

This tutorial has the following structure:

- Introduction and motivation: why is phylogenetic inference important?
- A general overview of phylogenetic inference: from sequence data onward.
- Obtaining the data for this tutorial + exercise
- Sequence alignment (including manual methods, automatic methods, complexity issues / heuristics) + exercise
- Distances based on sequence alignment
- The Neighbor-Joining method & Minimum Evolution, using FastTree2
- Building your first tree (on Galaxy)
- Models of sequence evolution: from the sublime to the ridiculous
- Phylogenetic Networks (**on your own computer, using SplitsTree**), Neighbor-Net
- Assessing the quality of the tree(s): Bootstrapping, branch lengths; conflict in the networks
- Maximum Likelihood with IQTree -->

## What is a Phylogeny?
![Hand drawn phylogenetic tree from Charles Darwin’s notebook with clades A, B, C, D branching from a common ancestor. Prefaced with handwriting that reads ‘I think’ and annotated with other illegible notes.](images/Darwin_tree.png "Charles Darwin's first sketch of an evolutionary tree. Source: Wikimedia commons"){: width="20%"}

A *phylogenetic tree*, also called a *phylogeny*, is usually a tree-like structure, like [Darwin's famous sketch](#figure-1). The leaves or tips of the tree represent extant (living/existing) taxonomic entities like species, genera, or strains (in general called "taxa"). The lines connecting taxa describe the evolutionary relationships between them.
The intersections of lines correspond to hypothetical ancestral taxa. They represent branching events when species split into two new species, or a strain developed a phylogenetic important offshoot strain, etc.

The phylogeny of a group of taxa is the best representation of their evolutionary relationships.
It is also the main basis on which we can build statistics comparing species: without the phylogeny, comparing species (or strains, genera, etc.) is not meaningful.

As Theodosius Dobzhansky famously wrote, **"Nothing in biology makes sense except in the light of evolution"** {% cite 6ef38333-9ecd-3f40-8f3e-5f3d6df3c4b4 %}.


## Why we need phylogenetics

There are many ways in which we can use phylogenetic analyses, from the most fundamental understanding of the evolutionary relationships that exist between a set of species, as in [Charles Darwin's famous sketch](#figure-1), to families:

![Circular phylogenetic tree of hexapods (insects). Clades are colour coded and labelled with common names e.g. ‘Fleas’. Silhouettes of representative species are shown around the outside of the tree.](./images/Hexapoda_phylogenetic_tree.png "Phylogeny of extant hexapod families. {% cite PMID:25275450 %} reproduced under Creative Commons Attribution 4.0 International License")


and much bigger projects across all of life:

![Large and complex phylogenetic tree that includes 92 named bacterial phyla, 26 archaeal phyla and all five of the Eukaryotic supergroups. Major lineages are assigned arbitrary colours and named, with well-characterized lineage names, in italics.](./images/nmicrobiol201648_Fig1_HTML.webp "A modern view of the tree of life based on sequenced genomes. {% cite PMID:27572647 %} reproduced under Creative Commons Attribution 4.0 International License"){: width="800"}

<!-- topics/evolution/tutorials/abc_intro_phylo/images/nmicrobiol201648_Fig1_HTML.webp -->

Aside from gaining a fundamental understanding of biology, other reasons for inferring phylogenetic relationships include:

- Designing vaccines, for example for SARS-CoV2 and influenza;
- Measuring phylogenetic diversity for guiding conservation efforts;
- Understanding coevolution; for example, around 70% of emergent human diseases have come from other species;
- Dating major evolutionary events to study the effects of environmental change on different species.

> <comment-title>Gene trees, species trees reconciliation problem</comment-title>
> It's worth noting that getting the phylogeny from a set of genes – what we often call a **gene tree** – might *not* give us the true phylogeny of the species that house those genes, *even if we get everything right!*
>
> This happens because there are other processes that can influence the so-called "gene tree" such as:
> - lateral gene transfer events
> - gene duplication
> - gene loss and incomplete lineage sorting
> - recombination
>
> **The situation where gene trees and species trees differ is often called the "gene tree / species tree reconciliation problem".**
>
> While it is a fascinating topic, it is beyond the scope of this tutorial. Today we will work under the assumption (which is reasonable for the dataset we will use) that the gene tree will reflect the species relationships.
>
{: .comment}

## Terminology
{% icon video %} From the root to the tips, watch the video to explore common features of phylogenetic trees. See how these features relate to evolutionary patterns and how outgroups can be used to find the root or common ancestor of extant species.

{% include _includes/slides-youtube-selector.html id="gnWlMbGPYx8" title="Lecture: Phylogenetics Terminology" slides="topics/evolution/tutorials/abc_intro_phylo-terminology" %}

Before we start building trees, let's define some terms.

![Schematic of a phylogenetic tree where features such as nodes/taxa, edges/branches are annotated and colour coded. The root of the tree is at the top of the image and the tree branches into two clades as you move towards the bottom of the image. The clades are formed of hypothetical common ancestors and five extant taxa which are labelled as the ‘in group’. Two additional taxa are appended to the right hand side of the tree and are labelled as the outgroup.](images/TreeAnatomyWithOutgroup.png "Anatomy of a phylogenetic tree."){: width="100%"}



It's common to call **phylogenetic tree** a **phylogeny**.

Mathematically, a **tree** is a kind of **graph**, which has objects called **nodes** or **vertices** (lavender, white, and blue boxes in the figure above), connected in pairs by things called **edges** (green and orange lines in the figure above).

Trees are a natural way to think about phylogenetic relationships. The nodes correspond to **taxa**, and the edges, also called **branches**, show the relationships between them, where taxa could be species, or lineages, genera, populations, or even individuals if we are considering something like a genealogy.

Nodes with only one edge attached to them are called **leaves** (or **tips**; in white above) and correspond to taxa with no descendant taxa in the tree. These taxa might be from fossils, or, be currently living, in which case they’re referred to as **extant**.

Internal nodes (in lavender above) correspond to hypothetical common ancestors of the extant taxa: the set of descendants that each one has determines the tree.

Many phylogenies have a special node assigned as the common ancestor of all the taxa represented by the leaves in the tree. This node is called the **root** (in blue above). When this is the case, a natural direction is implied from the root to the tips, going forward in time. We call such trees and phylogenies **rooted**; if there is no root, they are called **unrooted**.

*The majority of phylogenetic inference methods produce unrooted trees, but rooted trees are more useful.*

In a rooted phylogeny, all the leaves that are descendant from any given node form a **monophyletic clade**, or often just “**clade**” (monophyletic means “one tribe (of) origin” from the Greek).

One way to determine where the root of a tree belongs is to include an **outgroup** in the data, which is a set of taxa that are definitely not within the clade of interest (which is then called our **ingroup**) but which share a common ancestor with that clade. A good outgroup won’t be too distantly related to our ingroup, because if it’s too distant, choosing where it should connect to the ingroup will be hard, ultimately resulting in a guess.

You can see in the diagram above that the connection of the ingroup to the outgroup could be from multiple locations. Once the unrooted tree is created, using combined data from ingroup and outgroup taxa, we can confidently say that the root is on the branch connecting our ingroup to our outgroup:

![A schematic of an unrooted phylogenetic tree. It shows the relationship between an ingroup of 5 taxa on the left and an outgroup of two taxa on the right as a series of boxes connected by edges (lines). The taxa appear at approximately the same level across the image. An arrow points to an edge connecting the ingroup and outgroup where it is likely that a common ancestor exists.](images/TreeAnatomyUnrooted.png "Rooting a tree"){: width="100%"}

We can then imagine lifting up the unrooted tree at the branch connecting our outgroup and ingroup -- that is our best guess at the hypothetical ancestor of all our taxa and gives us a good indication of the branching order of our ingroup (and the outgroup):

![A schematic of an unrooted phylogenetic tree. It shows the relationship between an ingroup of 5 taxa on the left and an outgroup of two taxa on the right as a series of boxes connected by edges (lines). An arrow points "Lift here" points to an edge connecting the ingroup and outgroup which is a best guess at the location of the hypothetical ancestor](./images/TreeAnatomyLiftHere.png "'Lifted' tree demonstrating hypothetical ancestor and branching order")

Phylogeny estimation can be thought of as inferring a collection of compatible hypotheses about **monophyly** -- that is, statements that groups of taxa descendant from a common ancestor are each others' closest relatives in the tree.

The tree above is called a **binary tree**, because each internal node branches into *two* descendants. It is a very common assumption that trees are binary, and we make that assumption in this tutorial. In fact, it is often very hard to come to a means by which a phylogeny could be truly *non*-binary: in most cases, this is just due to our inability to resolve the tree completely.

# Building a tree
## Basic Methodology

So, how do we estimate phylogenetic trees?

We start with the leaves of the tree which can be living (extant) or older taxa:

![Five extant taxa (living species) are depicted as five boxes equally spread from left to right. These taxa are contained within a larger blue box which is annotated with the words “Leaves of the tree: these can be extant (living) or older taxa”.](images/WeJustHaveLeaves.png "The 'leaves' of a tree (extant or older taxa) are the starting point for buidling phylogenies"){: width="100%"}

There are several ways to estimate a tree, such as:

1. Go with what we think is the case already (this is not recommended!)
2. Attempt to build a tree based on similarity and dissimilarity, with tools such as Neighbor-Joining (NJ) or FastME (we will do this later in the tutorial)
3. Use a score function, such as Parsimony or Maximum Likelihood, to build potential trees and find the best one (we will do this later too)
4. Something else entirely (Perhaps networks or even inferring evolution based on the parasites of your species of interest!).

**Warning**: Finding an optimal tree is *hard*!

First and foremost, **phylogenetic inference is a statistic _estimation_ process.**

Different estimates of the phylogenetic tree relating a given set of species may differ, even if no errors were made.

It is generally not possible to prove that any tree inferred is *correct* -- since we cannot go back in time and observe speciation events.

> <comment-title>Common evolutionary assumptions used in phylogenetic estimation</comment-title>
>
>  We will (mostly) make these assumptions in this tutorial!
>
> 1. Evolution is “memoryless.” This assumption is that the future evolutionary trajectory of an organism is not affected by its past. This means we can use the powerful mathematics of Markov processes.
>
> 2. Phylogenetic relationships can be correctly represented by a tree! This isn't *always* assumed, but it is very common. Trees are a very attractive representation of evolution, and it is part of our language: "The tree of life" is a common phrase.  However evolution is not always explained by a tree-like, "branching" process as other events such as hybridisation and sharing of genetic material can influence envolutionary processes.
> 3. The molecular clock assumption is that sequences in a clade evolve at about the same rate. This is known to be wrong, but is useful. For instance, there is variation in evolutionary rate between lineages, but if this variation is not significant, we can ignore it and use simpler models, to better leverage the phylogenetic information in the data.
> 4. Lineages don’t interact – once they have speciated, they are independent of each other. This isn't always the case and we know that biological lineages *do* interact with each other -- but our methods are generally not able to manage such complexity. The vast majority of methods make this assumption, particularly if the evolution is also assumed to be tree-like.
>
{: .comment}


### Challenges

Phylogenetic Inference is **Hard**.
One of the things that make it hard is the sheer number of possible trees that can describe relationships among species.

The number of rooted binary trees grows as 1, 3, 15, 105, 945, 10395... in fact the formula for this number for $$n$$ taxa (we use `taxa' in general, since not all phylogenetic inference is at the species level) is $$(2n-3)!! = (2n-3)(2n-5)...(3)(1),$$ which grows as fast as $$2^{n}n!$$.

The table below gives you an idea of the scale we are dealing with.

*The Number of Unrooted Binary Trees*

| *n*          | # trees    | notes       |
|--------------|------------|-------------|
| 3    | $$3$$       | trivial to check |
| 4    | $$15$$       | enumerable by hand |
| 5    | $$105$$       | enumerable by hand on a rainy day |
| 6    | $$945$$       | enumerable by hand during lockdown |
| 7    | $$10395$$       | easily searched by computer |
| 8    | $$135135$$       | a bit more than the number of hairs on your head |
| 9    | $$2027025$$       | population of Sydney living west of Paramatta |
| 10   | $$34459425$$                      | comparable with the number of possible tickets in a typical lottery |
| 20   | $$\approx 8.2\times 10^{21}$$       | getting slow for computers even with branch-and-bound |
| 48   | $$\approx 3.21\times 10^{70}$$       | number of particles in the universe-ish |
| 136  | $$\approx 2.11\times 10^{267}$$       | number of trees to choose from in the first "Out of Africa" data set |
|---|---|---|



## Calculating distances

Building a tree begins with a set of *distances*, which record how different the taxa are from each other.
Distances have very desirable properties, that can be summarised as follows: for any objects $$ x $$, $$y$$, $$z$$, writing $$d(x,y)$$ means the distance from $$x$$ to $$y$$ etc.  These properties are:
 * *non-negativity* -- distances can never be negative, and in fact we treat two things as identical if they have a distance of 0 between them.
 * *symmetry* -- the distance from $$x$$ to $$y$$ is the same as the distance from $$y$$ to $$x$$; that is, $$d(x,y) = d(y,x)$$.
 * the *triangle inequality* -- there are no short-cuts!  The distance from $$x$$ to $$z$$ is always *at most* the distance from $$x$$ to $$y$$ plus that from $$y$$ to $$z$$; that is, $$ d(x,y) + d(y,z) \leq d(x,z)$$.

 In phylogenetics terms, we like distances to represent something like time and we can assign lengths to branches (see the [Tree Anatomy diagram](#figure-4) above).

Distances can be calculated based on a variety of data. Here is a flow-chart of the process:

![Flow chart illustrating how sequence alignment data or dis/similarity measures are used to calculate and calculate phylogenetic distances. Colours and shapes are used to differentiate different sections of the flowchart, guiding the viewer through each step from left to right. The flowchart begins with Sequence Alignment or Dis/Similarity Measures. These are used to form a distance matrix (D) which is used to select two nodes (x and y) forming a new node z. The distance matrix is updated with the new node z until no further nodes can be formed.](images/TreeConstruction.drawio.png "Tree construction flow-chart"){:width="100%"}

The blue boxes on the left show some of the input data forms. The most commonly used kind of data in modern phylogenetics is *aligned molecular sequences* -- typically, DNA, RNA, or Amino Acids (AA) from equivalent (homologous) genes in the species of interest. We focus on this form of molecular phylogenetics in this tutorial.

Other input data forms are distances or dissimilarity measures based on molecular-based measures like DNA-DNA hybridisation, gene presence/absence, and morphology (physical dimensions). We will not discuss this kind of data in this tutorial.

Aligned sequences can be converted into distances (green box above), using models for how the sites (i.e. specific nucleotides or amino acids) have evolved.
These distances can be expressed as a matrix _D_, which becomes the input for distance-based methods.

Distance-based methods (orange boxes) use algorithms to select a pair of taxa, or clades, to join together to make a new clade. Once that decision is made, the two taxa / clades that have been joined are replaced with the clade that the two of them make together as seen in the [figure below](#figure-7).

![Schematic illustrating how taxa are combined into clades. The relationship between three taxa and two clades are represented by green lines. The image consists of two main steps: Taxa 1 and 2 are combined into Inferred Clade 1. Clade 1 and Taxon 3 are then combined to form a new clade. A dashed green line indicates that this process continues until all taxa and clades are joined.](images/JoiningCladesForTreeConstruction.png "Joining clades for tree construction"){: width="60%"}


## Let's build our first tree

In this tutorial we will use a set of *Anolis* lizard DNA sequences, from {% cite 10.1080/106351599260283 %} to trial some phylogenetic methods.
The sequences are from the mitochondrial NADH dehydrogenase subunit 2 gene (ND2) and five transfer RNA (tRNA) genes, with an average sequence length of 1419.109 and a range of [1329,1727].


<!-- I got the sequence lengths from the raw FASTA file using gawk:
gawk 'BEGIN {OFS=","} {print length,$1}' anolis-raw.fst > anolis-sequence-lengths.csv -->
<!-- In fact this is even cleverer:
gawk 'BEGIN {OFS=","} {if (length>100) print length; else print $1}' anolis-raw.fst -->
<!-- The ultimate in collating sequence lengths with the sequence names:
sed -r 'N;s/^(>[A-Za-z2\.]+)\n/\1,/g' anolis-sequence-lengths.txt -->


We are using a relatively small set of sequences to ensure the methods run quickly for the purposes of the tutorial.

In the real world, a phylogenetic analyses often span hundreds, or even thousands, of taxa. Phylogenetic estimation on this many sequences is computationally very intensive, and can take weeks of time even on a high-performance computer.

### Get the data

<!--
* Does it auto-detect ok?
*
 -->
<!-- One sequence is here: https://www.ncbi.nlm.nih.gov/nucleotide/AF055943.2?report=genbank&log$=nuclalign&blast_rank=1&RID=JEX49T9G013 -->

> <hands-on-title>Obtain your data</hands-on-title>
>
> 1. Make sure you have an empty analysis history. Give it a name.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the file anolis-raw.fst from [Zenodo](https://zenodo.org/records/11089653/files/anolis-raw.fst) or from the shared data library.
>    ```
>    https://zenodo.org/records/11089653/files/anolis-raw.fst
>    ```
>    You may need to refresh your history to show the data.
>    If the paste/fetch link does not work for you, you can download it to your own computer and then upload it using the "Upload" button at the top of the left panel in Galaxy.
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
{: .hands_on}

You can click on the 'eye' icon {% icon galaxy-eye %} on the right to see the unaligned data (go ahead!) but the view isn't very informative. This is the raw FASTA file, with the symbols A, C, G, T representing nucleotides. You can see that the sequences are different lengths.

**FASTA** or **Fasta** format (pronounced to rhyme with pasta) format is is commonly used as input to phylogenetic inference programs and has a very simple structure, as follows:

1. Each sequence has a name, which appears on its own line after a '>' sign
2. The next line(s) contain the sequence; continuing either until the next sequence name line, or the end of the file.

The example below has four sequences in it named Taxon_1, Taxon_2, etc, each with a short set of characters representing DNA sequences.


```
>Taxon_1
ACTGCGTTAGGTCTAGCC
>Taxon_2
GATCTACTGCTTTAGGTTGAGCC
>Taxon_3
ACTGCTCTAGCACTGAGCCCA
>Taxon_4
ACTTGGCGTAGCCGGAGGCC
```



<!-- A line beginning with the greater-than sign `>` holds the sequence name or identifier/ID; the other lines hold the sequence data for the sequence ID above. -->

> <tip-title>FASTA format for multiple sequence alignments</tip-title>
>
>The Fasta format can also include symbols such as a question mark '?' for missing data, or hyphen '-' to indicate an insertion or deletion events, collectively *indels*. It is common to represent a multiple sequence alignment in Fasta format with these symbols. It is also a common format for storing high-throughput read data, but *without quality scores* -- if you want to include read quality you would use FASTQ format.
{: .tip}

> <hands-on-title>View your data</hands-on-title>
> Let's view the unaligned sequence in a more understandable form.
>
> 1. Click on the title of your file to see the row of small icons for saving, linking etc:
> ![Screenshot of Galaxy icons. Seven small blue icons are shown on a green background. From left to right they are: floppy disk, link, information, redo, bar chart, flow chart and a question mark.](./images/LittleJobIcons.png)
>
> 2. Click on the **visualise icon** {% icon galaxy-visualise %} and then select the **Multiple Sequence Alignment tool**.
> You should see something like this:
>
> ![Screenshot of sequence visualisation output from Galaxy. Fifteen Anolis DNA sequences are arranged in rows. The nucleotides are colour coded and arranged in columns: A(blue), T (green), C (pink), G (orange). The top half of the image shows approximately 50 bases of each sequence. The lower half of the image shows a zoomed out heatmap-like image of a larger portion of the sequences.](./images/UnalignedAnolis.png){:width="600"}
>
> Play around with the view. You can change colour schemes and add or remove various elements.  Good colour schemes for nucleotide data are "Clustal2" and "nucleotide".
>
> If you cannot see the slider at the top (for moving left or right in the view) you will need to check the "Show residues indices" in the "Vis. elements" drop-down menu.
>
> Do not adjust the scale slider!  Currently, this breaks the page.
{: .hands_on}

> <question-title>Understanding the FASTA file</question-title>
> 1. How many sequences are there in your data?
> 2. How long is the longest sequence, and what is it?
> 3. What about the shortest sequence?
>
> > <solution-title></solution-title>
> > There are 55 sequences.  The longest is from <i>Anolis paternus</i> with length 1729 nucleotides; the shortest is <i>A. luciae</i> with length 1252.
> {: .solution}
{: .question}

# Sequence Alignment

{% icon video %} Watch the video to find out
[0:00] What ‘alignment’ really means and why it is useful for phylogenetics; [6:22] What makes a good alignment; [12:15] How pairwise sequence alignment and dynamic programming work; [33:47] How the multiple sequence alignment algorithms work; and to
[40:00] explore multiple sequence alignment of the Anolis sequences with SeaView.

{% include _includes/slides-youtube-selector.html id="FUc2A2r1LGY" title="Lecture: Sequence alignment" slides="topics/evolution/tutorials/abc_intro_phylo-msa" %}


Molecular sequences must be aligned before they can be used to build phylogenies.

Aligning sequences amounts to finding the nucleotide positions (sites) that we can be confident have the same evolutionary history: they correspond to each other across species and can be considered to have evolved from the same common ancestor.
A good clue to identify these sites, which are called <em>homologous</em>, is that they are well conserved, with only a few changes.

Below is an example of an alignment. On the left are the sequences in Fasta format.
In the middle we see an alignment of those sequences, which has gaps in it that help line up the sites so that more of them agree.
A sign of a "good" alignment is one in which the colours line up vertically. The overhanging parts can be removed as seen in the "trimmed version" on the right.

![Table containing DNA sequence from four taxa arranged in rows. Three columns show the sequences in ‘Unaligned’, ‘Aligned’ and ‘Trimmed’ states. The nucletodies are colour coded A (green), T (pink), C (blue) and G (orange). In the “Unaligned” column, the sequences are continuous. In the ‘Aligned’ column gaps are introduced and represented as dashes so that the sequences align vertically. In the “Trimmed” column overhanging portions of the sequences are greyed out to indicate that they would be trimmed.](./images/ToyAlignmentAndTrim.png "Sequence alignment"){:width="700"}

While aligning *two* sequences is "easy", in the sense that an optimal alignment between two sequences can be found in a reasonable amount of time, optimally aligning multiple sequences is *computationally intractable*. Multiple sequence alignment is a complex process and many methods have been developed to tackle this challenge.

## Aligning sequences with MAFFT

Today you will be aligning sequences using a modern multiple alignment program called **MAFFT**, which is available on Galaxy.

<!-- ## Question: is there a format convention for program names, like Courier Bold or something? -->


> <hands-on-title>Sequence alignment with MAFFT</hands-on-title>
>
> 1. In Galaxy, search for and select the {% tool [MAFFT](toolshed.g2.bx.psu.edu/repos/rnateam/mafft/rbc_mafft/7.508+galaxy1)%} from the tool finder on the left, and run it with the following parameters:
>    - {% icon param-file %} In the MAFFT tool, the **Sequences to align** field should already be filled with your unaligned data. If it isn't, select it using the drop-down menu.
>    - In the **Data type** field select "_Auto detection_".
>    - Leave the **MAFFT flavour** as "_fftns_". This is a good default.
>    - In the **Matrix selection** select "_No matrix_".
> 2. Click "**Run tool**".
{: .hands_on}

Here is a visualisation of the resulting alignment. Note that the colours are now vertically aligned:

![Screenshot of aligned sequences. Multiple DNA sequences are represented in rows and aligned vertically. Vertical columns of purple (G), blue (C), red (T) and green (A) indicate alignment of nucleotides across sequences. Dots replace the letters as an abbreviation where all nucleotides are the same in a given column.](./images/MEGA_alignment.png "Aligned sequences"){:width="600px"}

> <hands-on-title>Inspecting your alignment</hands-on-title>
>
> 1. Click on the title of the completed MAFFT job to show the row of small icons for saving, linking etc.
> ![Screenshot of Galaxy icons. Seven small blue icons are shown on a green background. From left to right they are: floppy disk, link, information, redo, bar chart, flow chart and a question mark](./images/LittleJobIcons.png){:align="center"}
> 2. Click on the **visualise icon** {% icon galaxy-visualise %}
> 3. You will be presented with a couple of options. Select "**Multiple Sequence Alignment**".
> *Your alignment is displayed!*
> The colour scheme is horrible to start with because it's not automatically detecting the data as DNA sequences.  Click on the "Color scheme" button and select "**Clustal2**" for a nicer picture.
> 4. You can click and drag the display of nucleotides in the upper panel, and when you go far enough to the right you'll see some gaps have been introduced by MAFFT.  On either side of each gap you should see that the nucleotides match up pretty well.  This is because the MAFFT algorithm has identified that these site are homologous, and that either an insertion event has happened for a group of sequences, or a deletion happened in the others.
> 5. You may be tempted to play around with "Vis. elements" and show the scale slider - don't! It's currently broken and will crash your browser tab :(.
{: .hands_on}

You should ALWAYS visually check your alignment to see if it makes sense.
A tool you can use on your own computer is [SeaView](https://doua.prabi.fr/software/seaview).



# Distance-based phylogenetic inference
{% icon video %} Watch the video to delve into the mathematics of building phylogenetic trees from distances. [0:00] Why use distances and how they relate to the structure of a phylogenetic tree
[3:05] Types of distances and distance matrices
[11:31] The mathematics behind the Jukes-Cantor/JC69 and HKY85 models [16:04] An example of building a tree from ultrametric distances [21:01] Building a tree from non-clocklike distances with the Neighbour-joining method [25:59] Real life data and limitations for building trees from distances.

{% include _includes/slides-youtube-selector.html id="t68H2Ilc4mI" title="Lecture: Phylogenetic Trees" slides="topics/evolution/tutorials/abc_intro_phylo-trees" %}

We are going to build our first tree, using a very common method called _Neighbor-Joining_. This method was created in the 1980s by Saitou & Nei ({% cite 10.1093/oxfordjournals.molbev.a040454 %}).

<!-- (Saitou, N. & Nei, M. _The Neighbor-Joining method: a new method for reconstructing phylogenetic trees_. 1987, *Molecular Biology and Evolution*  , Vol. **4**(4):406-425); see also Studier, J. A. & Keppler, K. J., _A Note on the Neighbor-Joining Algorithm of Saitou and Nei_, 1988. *Molecular Biology and Evolution* **5**(6): 729-731) -->

## Building a Neighbor-Joining Tree

The Neighbor-Joining (NJ) algorithm is a standard method that takes a set of distances between taxa as input, and sequentially connects them into larger and larger clusters until all taxa have been joined.

NJ is rarely used as a complete tool for phylogenetic analysis. Although it is quite accurate and fast, there are other fast methods that can be then applied to modify the NJ tree and create a better one.

The FastTree2 program that we are using does this. First it creates a "rough" NJ tree, and then modifies it to optimise a quantity called *Minimum Evolution* (ME) (more on this later).  A detailed description of how FastTree works is available from [Microbes online](http://www.microbesonline.org/fasttree/).

> <hands-on-title>Build a Neighbour-Joining Tree with FastTree</hands-on-title>
>
> 1. Search for the {% tool [FastTree](toolshed.g2.bx.psu.edu/repos/iuc/fasttree/fasttree/2.1.10+galaxy1) %} in the tool finder on the left, and run it with the following parameters:
>
>    - *"Aligned sequences file (FASTA or Phylip format)"*: `fasta`
>        - {% icon param-file %} *"FASTA file"*: `outputAlignment` (output of **MAFFT** {% icon tool %})
>    - *"Protein or nucleotide alignment"*: `Nucleotide`
>        - *"Nucleotide evolution model"*: `Jukes-Cantor + CAT`
>    - *"Show advanced options"*: `Yes`
>        - *"Use constant rates?"*: `Use constant rates.`
>        - *"Turn off maximum-likelihood."*: `Yes`
>
> 2. Click on "Run tool".
>
{: .hands_on}

<!--
> Removed and linked to the source instead
>**Heuristic Neighbor-Joining**
>
>First, FastTree uses a heuristic variant of neighbor joining to get a rough topology. During neighbor joining, FastTree stores profiles of internal nodes instead of a distance matrix, which reduces the memory required. FastTree uses a combination of three heuristics to speed up this phase: it remembers the best join for each node, as in fast neighbor-joining; it does a hill-climbing search for better joins from a candidate join, as in relaxed neighbor joining; and it uses the "top hits" heuristic to avoid computing all pairwise distances and to avoid considering all possible joins at every step. It also updates the best join for a node as it comes across them, which reduces the amount of hill-climbing. Another limitation of FastTree's neighbor-joining phase is that it does not correct the distances for multiple substitutions, which exacerbates long-branch attraction. However, this will be corrected in the next stage.
>
>**Minimum Evolution**
>
>FastTree then tries to reduce the length of the tree, using a mix of nearest-neighbor interchanges (NNIs) and subtree-prune-regraft moves (SPRs). These "balanced minimum-evolution" rearrangements are roughly the same as what FastME does, but because FastTree uses profiles instead of distances, it is much faster. By default, FastTree uses 4*log<sub>2</sub>(N) rounds of nearest-neighbor interchanges and 2 rounds of subtree-prune-regraft moves. In each round, it considers every possible NNI in the tree. Because there are too many (O(N<sup>2</sup>)) possible SPR moves, FastTree treats SPR moves as chains of NNIs and only extends the best choice in the chain for chains of length two or greater. In the minimum-evolution framework, if the distances are not too noisy, NNI and SPR moves suffice to reach optimal trees (Desper & Gascuel 2004, Bordewich et al. 2009).
>
>Distances: During these minimum evolution steps, FastTree needs to estimate distances between sequences or profiles. For protein sequences, FastTree estimates distances by using the BLOSUM45 amino acid similarity matrix, and it corrects for multiple substitutions by using the formula $-1.3 \times \log(1-d)$, where $d$ is weighted so that random sequences have an average value of 1. For nucleotide sequences, FastTree uses the Jukes-Cantor distance $-0.75\times\log(1 - \frac{4}{3} d)$, where $d$ is the proportion of positions that differ. When comparing two sequences, positions with gaps are ignored; when comparing two profiles, positions are weighted by their proportions of non-gaps.
>
{: .comment} -->


It won't take very long for FastTree to build your tree.
But when it's done, how can you see it?

Clicking on the 'eye' icon  {% icon galaxy-eye %} of the output doesn't at first appear to be very illuminating: it's just a parenthesised list of taxon names and numbers.
This is **Newick Format**, and it's worth knowing at least a little of what it means.

* Each matched pair of parentheses denotes a **cluster** or **subtree**: "(A,B)" means that A and B are each others' closest relatives (also called _sister taxa_).
* A number after a cluster (so, after a closing parenthesis) is a **label** for that cluster.  In the output from FastTree, this label is an indicator of the support for that branch.
* If there is a colon ':' followed by a number, then this is the **branch length** for the subtree.


![Two rooted phylogenetic trees (a) and (b) are shown. Both trees have three taxa, labelled A, B and C as the leaves and a root y. Taxa A and B form a clade and connect to y via the internal node x. Taxon C connects directly to y. Distances between nodes are shown as numbers. For tree (a): y:C = 6 ; y:x = 2; x:A = 3 ; x:B =2. For tree (b): y:C = 5 ; y:x = 3; x:A = 3 ; x:B =2](./images/NewickExplained.png "Newick Format")

The rooted, 3-taxon trees above have three taxa, labelled A, B and C.  Two of the internal nodes have been labelled (x and y), but it isn't necessary to do so in general (for example, if you wanted to use the label for something like support of each branch, as does FastTree).

In both trees, A and B are sister taxa, and branch lengths are indicated near each branch: you can see how the branch lengths are above each cluster, including the individual taxa (the "leaves" of the tree), but not above the root.

The Newick format for the tree on the left is "((A:3, B:2)x:2, C:6)y;" and for the one on the right it is "((A:3, B:2)x:3, C:5)y;". The number after each colon is the length of the branch above it (closer to the root).

Note that these two trees are very similar: they only differ in the position of the root (y), either being distance 2 from node x, or being 3 from it.  The distance between any two nodes in the tree is the sum of the branch lengths on the path connecting them, so for the trees above, the distance matrix is

|     |  A   |   B  |  C   |
| --- | :--: | :--: | :--: |
|  A  |  0   |  5   |  11  |
|  B  |  5   |  0   |  10  |
|  C  |  11  |  10  |   0  |

Ideally, these will reflect the actual input distances, but such distances are based on messy real data, and do not necessarily obey this ideal.
That is why methods like FastTree are employed to find a tree with the best possible agreement between the distance inferred, and that calculated from sequence data.

> <hands-on-title>Visualising a tree</hands-on-title>
> To visualise your tree in Galaxy:
> 1. Search for the {% tool [Newick Display](toolshed.g2.bx.psu.edu/repos/iuc/newick_utils/newick_display/1.6+galaxy1) %} in the tool finder on the left, and run it with the following parameters:
>
>    - *"Newick file"*: {% icon param-file %}: `tree.nhx` (output of **FastTree** {% icon tool %})
>    - *"Branch support"*: `Display branch support`
>    - *"Branch length"*: `Display branch length`
>    - *"Choose an ouput format"*: `PNG`
> 2. Click on the **display icon** {% icon galaxy-eye %} next to the title of the completed Newick Display job to display your tree.
>
> <!--1. Click on the title of the completed FastTree job to show the row of small icons for saving, linking etc.
> ![Screenshot of Galaxy icons. Seven small blue icons are shown on a green background. From left to right they are: floppy disk, link, information, redo, bar chart, flow chart and a question mark.](./images/LittleJobIcons.png)
>
> 2. Click on the **Visualisation icon** {% icon galaxy-visualise %}
> 3. You will be presented with a couple of options. Select **"Phylogenetic Tree Visualisation"** -- this seems to be the best one.
>
>    *Your tree is displayed!*
>
>    <!-- [SelectTreeVisualisation](./images/PhylogeneticTreeVisualisationOptions.png){:width="400"} -->
>
> <!--5. Open the settings panel via the **Visualisation Settings** {% icon galaxy-vis-config %} at the top right of the central panel. Click on these to reveal the settings and display options. Try out "Circular" and "Radial". -->
>   Notice that there are quite a lot of long branches adjacent to the extant taxa (leaves) and that these branches are much shorter near the centre of the tree.
> Note: **Short branches are much harder to get right.**
>
> Additional visualisation options
> 1. To build a radial tree, rerun Newick Display and choose the *"Draw a radial tree"*: `Yes` option.
> 2. Alternatively you can visualise your tree outside of Galaxy by downloading your FastTree output and using software such as [SplitsTree](https://software-ab.cs.uni-tuebingen.de/download/splitstree4/welcome.html) or [FigTree](http://tree.bio.ed.ac.uk/software/Figtree/). These tools provide additional options for interactively exploring and customising the appearance of your tree.
>
>(If you wish you may of course re-run FastTree and allow it to seek a Maximum Likelihood tree -- maybe once you've learned more about Maximum Likelihood.)
>
>
{: .hands_on}


# Searching for the "best" tree

The other way we can estimate a phylogeny is by choosing some kind of score of "goodness" and then searching the entire set of possible trees for the tree (or trees) that optimises this score.
Note that such scores are "surrogates for truth" in that we *hope* the optimal score will correspond to the true tree, but it is not necessarily the case. In many analyses we therefore use *multiple* methods, in the hope that they will give us the same answer. Minimum Evolution (ME), Maximum Parsimony (MP), and Maximum Likelihood (ML) are common such score functions.

> <comment-title>Data vs Method</comment-title>
> If your conclusion changes based on your reasonable choice of analytical method, then perhaps your data are not adequate.
{: .comment}


## Minimum Evolution (ME)

Minimum Evolution is the idea that the sum of the branch lengths should be as small as possible to still account for the distances between the leaves of the tree, in that the sum of squared differences between the distances implied by the tree and the observed distances from the data, is minimised.  You can read more about this in an article by [Rzhetsky and Nei](https://academic.oup.com/mbe/article/10/5/1073/1037508){%cite 10.1093/oxfordjournals.molbev.a040056 %}.

There are some variations on this ME criterion and FastTree uses an approximation of one of them to find good trees.

## Maximum Parsimony (MP) and Parsimony Length

Most tree estimation methods output trees with branch lengths that correspond to the amount of evolutionary "work" that has to be done to turn one sequence into another.

This can be given as the *minimum number of character state changes required* -- the so-called *parsimony length* -- to convert the (hypothetical) sequence at one end of a branch to that at the other end.
The Maximum Parsimony method is based on this approach.

<!--The parsimony length of a given site pattern in an alignment, given a particular tree, is the minimum number of changes of state that are required to account for the observed characters (e.g. nucleotides) at the leaves of that tree.-->
Finding the parsimony length of a site pattern is easy and fast due to a clever algorithm created by Walter Fitch; hence, finding the score (the "goodness") of a tree is fast. But finding the tree that minimises this score is still computationally intractable, because the space of trees is so huge.

The Maximum Parsimony method for finding the "best" tree is to search tree space for the tree (or trees) that *minimises* the parsimony length of any tree for that alignment. Note that when parsimony is *maximised*, this means the fewest possible changes required, so the *minimum* length.

We do not use the Maximum Parsimony method in this tutorial.

## Maximum Likelihood (ML)
{% icon video %} Watch the video to learn about [0:00] the concept of likelihood and how it lends statistic rigour to phylogenetic analysis as well as [26:45] how it is applied in models like Jukes-Cantor, HKY85 and GTR models to select the “best tree” from your data. [34:16] Wander through tree space to find the best tree using Nearest Neighbour Interchange, Subtree Pruning and Regrafting, and Tree Bisection and Reconnection tree perturbations and [45:40] go “hill climbing” with tree-search algorithms. [48:52] Finally, explore the output of IQTree for the Anolis dataset used in the tutorial.

{% include _includes/slides-youtube-selector.html id="0WzQLAl9h8Q" title="Lecture: Maximum Likelihood & Trees" slides="topics/evolution/tutorials/abc_intro_phylo-mle-trees" %}

Likelihood is the most statistically defensible phylogenetic inference method.
It is based on the idea that the tree that has the highest probability of producing the sequences at the tips of the tree is the tree that is the "most likely" to be correct: this is called the Maximum Likelihood (ML) Tree.

Likelihood is *not* the same as probability, though they are often confused with each other. However, it is *proportional* to the probability that the tree is correct, out of the set of possible trees and models you are considering.

One major, almost ubiquitous, assumption about molecular sequence data is that each site evolves independently of all other sites. Biologically, this isn't always the case, but in practice, this makes things much more tractable, and we still have a good chance of getting the tree(s) right.

Another assumption we make is that the substitution rate -- the rate at which changes of nucleotide at a given position in the sequence happen -- is only dependent on the current state, i.e., we do not care about how a sequence came to be what it is, only what the sequence is now, to determine what are the probable evolutions of it.
This seems much more biologically reasonable and makes this into a Markov process, which in turn enables a lot of calculations to be made simply.

### Searching for trees and their branch lengths

When dealing with nucleotides A, C, G, T, there are 4x4 rate matrices with different names: Jukes-Cantor (JC69) with one parameter, Hasegawa-Kishino-Yano (HKY85) with five parameters, and many more, that define the rates at which nucleotides change.
There are other 20x20 matrices for amino acids, and even 64x64 matrices for codons.

To convert from a rate to a probability, hence giving us a likelihood, requires that we have a branch length. Then, we can calculate the probability under a given model, and after a specific time interval, of going from one nucleotide to another. We multiply these site probabilities to calculate the probability of going from an entire sequence to another.
Thus, looking for the optimal tree under likelihood requires we also search for the best-fit **branch lengths**, as well as looking for the best **tree**.

Maximum Likelihood is therefore the **slowest** tree inference method we discuss in this tutorial.

### Models of sequence evolution

*If you are in a hurry to do the phylogenetic analysis you can skip this section and go to the next [Hands-on: running IQ Tree](#estimating-a-maximum-likelihood-tree-with-iqtree).*


> <details-title>Likelihood models</details-title>
> Likelihood is based on probability, so requires we choose a probabilistic model for the evolution of sequences.
> The simplest model for DNA is that each nucleotide has the same rate of change, and that all nucleotides appear with equal frequency (called the base frequencies) of 25%, 25%, 25%, 25%. This is the Jukes-Cantor (JC69) model published in 1969, and this model has just one parameter.
>
> More biological realism allows for different nucleotide proportions outside the uniform 25% rate. This is the Felsenstein 1981 model, known as F81, and it has three more parameters for the rates (not four: given the first three base frequencies, this defines the other one).
>
> A next level of sophistication is the Hasegawa-Kishino-Yano model (HKY85) published in 1985, which acknowledges that transitions (changes of state within the purines A, G or within the pyrimidines C, T) occur more rea

tools_json:
["MAFFT", "toolshed.g2.bx.psu.edu/repos/rnateam/mafft/rbc_mafft/7.508+galaxy1", "FastTree", "toolshed.g2.bx.psu.edu/repos/iuc/fasttree/fasttree/2.1.10+galaxy1", "Newick Display", "toolshed.g2.bx.psu.edu/repos/iuc/newick_utils/newick_display/1.6+galaxy1", "IQTree", "toolshed.g2.bx.psu.edu/repos/iuc/iqtree/iqtree/2.1.2+galaxy2"]


topic: computational-chemistry
tutorial: md-simulation-namd
title: Running molecular dynamics simulations using NAMD
difficulty: intermediate
time_estimation: 3H
tools_json: []
requirements_json: []
tags_json: []
content_hash: 1b287bd7876a95eaa003dd35a43127a8
last_modified: 2026-05-15T22:28:10.769240
zenodo_link: https://zenodo.org/record/3234841
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/computational-chemistry/tutorials/md-simulation-namd/tutorial.html

content:
In this tutorial we will perform a simulation with the popular [NAMD](http://www.ks.uiuc.edu/Research/namd/) molecular dynamics software. Please note NAMD tools are not currently available on a public Galaxy server due to licensing issues. If you are interested in following this tutorial, you will need to download the [BRIDGE docker container](https://github.com/scientificomputing/BRIDGE) and [download NAMD](https://www.ks.uiuc.edu/Development/Download/download.cgi?PackageName=NAMD) yourself.


> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}



This tutorial is made up of two parts. In the first section, we will look at preparation of a system (solvation, charge neutralisation, energy minimisation) using CHARMM. In the second section, we will perform an equilibration and production simulation, using NAMD. If you already completed the [Setting up molecular systems]({% link topics/computational-chemistry/tutorials/setting-up-molecular-systems/tutorial.md %}) tutorial, which covers the use of the CHARMM graphical user interface (GUI), you have already prepared your system, so go straight to the [second section](#md-simulations-with-namd), using the files you prepared earlier.

The process can be accomplished by selecting each tool from the tools menu, or by importing the workflow. The workflow method is most efficient and the individual tools used in the workflows are discussed below. The entire workflow (preparation + simulation) is shown below for [CHARMM and NAMD](#workflows).



> <details-title>NVT, NPT and statistical mechanics theory</details-title>
>
> [See Statistical Mechanics, McQuarrie for more in depth theory ISBN:9781891389153](https://books.google.co.za/books/about/Statistical_Mechanics.html?id=itcpPnDnJM0C&redir_esc=y)
>
{: .details}

# System preparation with CHARMM

If you already prepared your system using the CHARMM-GUI, and saved the output files, you can skip this section.

## Setup

Initially, we need to prepare a protein-ligand system in CHARMM. 

This tool will:
- solvate the protein-ligand complex, using the TIP3P water model
- neutralise the system, using 0.05M NaCl
- conduct a short energy minimisation 

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this tutorial.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the files from the Zenodo link provided.
>    ```
>    https://zenodo.org/record/3234841/files/cbh1test.crd
>    https://zenodo.org/record/3234841/files/cbh1test.psf
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
> 3. Rename the datasets.
>
>    {% snippet faqs/galaxy/datasets_rename.md %}
>
> 4. Check that the datatype is correct. The crd file should have the CRD datatype and the psf file the PSF datatype.
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
>
{: .hands_on}

> <hands-on-title>initial processing</hands-on-title>
>
> Run **System Setup** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"psf input"*: Protein structure file of the protein-ligand system. (Input dataset)
>    - {% icon param-file %} *"crd input"*: Coordinate file of the protein-ligand system. (Input dataset)
>    - *"Buffer"*: Edge to edge distance between the protein and the edge of the water box in angstroms
>    - *"Custom topology and parameter files for ligands"* (optional)
>
{: .hands_on}

### Energy minimisation

The setup provides us with the CRD and PSF files needed to perform a simulation. In addition, we have a PRM file which defines the parameters of the unit cell. Now, we need to perform energy minimisation. This relaxes the system, removing any steric clashes or unusual geometry which could artificially raise the energy.

This tools will:
- Minimise energy using a steepest descent algorithm followed by Adopted Newton Raphson (using the defined number of steps)
- Set up periodic boundaries and generate Particle Mesh Ewald (PME)
- generate reference structures for restraints in NAMD (if selected)

> <hands-on-title>energy minimization</hands-on-title>
>
> Run **Energy Minimizer** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"system_setup_crd"*: Coordinate file generated by the setup tool. (Input dataset)
>    - {% icon param-file %} *"system_setup_psf"*: Protein structure file generated by the setup tool. (Input dataset)
>    - {% icon param-file %} *"waterbox parameters input"*: Water box parameter file generated by the setup tool. (Input dataset)
>    - *"Minimization steps"*: `1000`
>    - *"Create reference structures for RMSD restraints for NAMD?"*: `No`
>    - *"Custom topology and parameter files for ligands"*: `No`
>
{: .hands_on}


# MD simulations with NAMD

At this point we are ready to run the simulation, which uses NAMD as a molecular dynamics engine. An NVT simulation is followed by an NPT simulation.


### NVT

Classical NVT dynamics, maintaining constant **n**umber of particles, **v**olume and **t**emperature.

This tool runs classical molecular dynamics simulations in NAMD using an NVT ensemble. User can run the simulation in small time intervals. The coordinates, velocities and the extended system files can be use to restart the simulations. If required, harmonic restraints can be used to maintain the protein shape. These restraints, in particular RMSD harmonic restraints can be added with the NAMD collective variable module.

> <hands-on-title>NVT dynamics</hands-on-title>
>
> 1. **NAMD MD Simulator (NVT)** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"xplor psf input"*: PSF file from CHARMM preparation (Input dataset)
>    - {% icon param-file %} *"pdb input"*: PDB file from CHARMM preparation (Input dataset)
>    - {% icon param-file %} *"PME grid specs"*: Generated by the setup tool (Input dataset)
>    - {% icon param-file %} *"waterbox prm input"*: Water box parameter file generated by the setup tool. (Input dataset)
>    - *"Temperature (K)"*: `300`
>    - *"Are you restarting a simulation?"*: `No`
>    - *"Use Harmonic restraints simulation?"*: `No`
>    - *"Custom parameter file for ligands"*: `No`
>    - *"DCD Frequency (ps)"*: `1` (Frequency to record frames in the DCD trajectory)
>    - *"Simulation Time (ps)"*: `10`
>    - *"Number of processors"*: `4`
{: .hands_on}


### NPT
Classical NPT dynamics, maintaining constant **n**umber of particles, **p**ressure and **t**emperature.

This tool runs classical molecular dynamics simulations in NAMD using an NPT ensemble. User can run the simulation in small time intervals. The coordinates, velocities and the extended system files can be use to restart the simulations. Harmonic restraints can be used. NAMD collective variable module is used to give RMSD harmonic restraints.

> <hands-on-title>NPT dynamics</hands-on-title>
>
> 1. **NAMD MD Simulator (NPT)** {% icon tool %} with the following parameters:
>    - {% icon param-file %} *"xplor psf input"*: PSF file from CHARMM preparation (Input dataset)
>    - {% icon param-file %} *"pdb input"*: PDB file from CHARMM preparation (Input dataset)
>    - {% icon param-file %} *"PME grid specs"*: Generated by the setup tool (Input dataset)
>    - {% icon param-file %} *"waterbox prm input"*: Water box parameter file generated by the setup tool. (Input dataset)
>    - *"Temperature (K)"*: `300`
>    - *"Pressure (bar)"*: `1.01325`
>    - *"Are you restarting a simulation?"*: `Yes`
>    - *"Coordinates from the previous simulation"*:  from the NVT simulation
>    - *"Velocities from the previous simulation"*:  from the NVT simulation
>    - *"Extended system of the previous simulation"*: from the NVT simulation
>    - *"Use Harmonic restraints simulation?"*: `No`
>    - *"Custom parameter file for ligands"*: `No`
>    - *"DCD Frequency (ps)"*: `1` (Frequency to record frames in the DCD trajectory)
>    - *"Simulation Time (ps)"*: `15`
>    - *"Number of processors"*: `4`
{: .hands_on}

# Workflows

Both the CHARMM preparatory workflow and the NAMD simulation workflow are available as an alternative to executing individual tools.

![Snapshot of CHARMM and NAMD analysis workflow]({% link topics/computational-chemistry/images/NAMD_CHARMMGUI_workflow.png %} "A simple simulation workflow starting with CHARMM for setup and NAMD to continue the production simulation")

![Snapshot of NAMD analysis workflow]({% link topics/computational-chemistry/images/NAMD_workflow.png %} "A simple NAMD simulation workflow")

# Conclusion

After completing the steps, or running the workflow, we have successfully produced a trajectory (the xtc file) which describes the atomic motion of the system. This can be viewed using molecular visualization software or analysed further; please visit the visualization and [analysis]({% link topics/computational-chemistry/tutorials/analysis-md-simulations/tutorial.md %}) tutorials for more information.


topic: computational-chemistry
tutorial: med-chem-data
title: Data management in Medicinal Chemistry
difficulty: introductory
time_estimation: 1H
requirements_json: ["type: \"internal\""]
tags_json: ["fair", "data-management", "medicinal-chemistry", "computational-chemistry"]
content_hash: 90fafe76f1a52d7c8bc52e11c9055bb1
last_modified: 2026-05-15T22:28:10.769710
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/computational-chemistry/tutorials/med-chem-data/tutorial.html

content:
The development of medicinal chemistry is advancing very rapidly. Big pharmaceutical companies, research institutes and universities are working on ground-breaking solutions to help patients combat all kinds of diseases. During that development process, tons of data are generated – not only from the lab environment but also from clinical trials. Given that the discovery of more potent, safer and cheaper drugs is the ultimate goal of all research bodies, we should all focus on making the data we gather FAIR: **F**indable, **A**ccessible, **I**nteroperable, and **R**eusable to push the boundaries of drug development even further. 

With the currently available methods such as artificial intelligence, machine learning, many toolkits, software and access to various databases, managing big data is now inherently linked to medicinal chemistry and helps to make this area as efficient as it can be. 

In this tutorial, we will therefore explore some concepts related to medicinal chemistry, explore the available chemical and pharmacological databases, and perform some basic data analyses using the Galaxy interface. 

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}


# Principles of medicinal chemistry research

## Drug-likeness
There are many factors that medicinal chemists take into account while designing new drugs. It is essential to estimate the properties of the molecule before synthesising it in the lab, so the structures of drug candidates are usually compared to the existing drugs by so-called drug-likeness. This includes intrinsic properties of a compound that will lead to favourable ADMET (**A**bsorption, **D**istribution, **M**etabolism, **E**xcretion, **T**oxicity) characteristics. Below are some properties usually assessed:
- Lipophilicity
- Size
- Polarity
- Insolubility
- Unsaturation
- Flexibility

Additionally, there have been some rules developed to help estimate drug-likeness, amongst which:
- Lipinski rule ({% cite Lipinski2001 %})
- Ghose rule ({% cite Ghose1999 %})
- Veber rule ({% cite Veber2002 %})
- Egan rule ({% cite Egan2000 %})
- Muegge rule ({% cite Muegge2001 %})
- Abbott bioavailability score ({% cite Martin2005 %})


As you see, there are a number of characteristics to consider before even synthesising the molecule in a lab. One of the online tools that help to summarise all the most important information about intrinsic properties and drug-likeness of compounds is called [SwissADME](http://www.swissadme.ch/) and is a free web tool to evaluate pharmacokinetics, drug-likeness and medicinal chemistry friendliness of small molecules ({% cite Daina2017 %}). 

Below we briefly discuss two probably the most well-known rules: Lipinski's and Veber's rules.

## The Lipinski rule of 5

This rule of thumb developed back in 1997 by Christopher Lipinski and colleagues tries to predict the likelihood that a given small molecule can be orally active. The Lipinski rule of 5 favours molecules as potential oral drug candidates if:
-	the molecular mass is less than 500
-	the calculated logarithm of the octanol−water partition coefficient (clogP) is less than 5
-	they have up to 5 H-bond donors
-	they have up to 10 H-bond acceptors

However, nowadays there are more and more drugs being developed which don’t comply with those rules and regardless are still effective. There are voices from the scientific community, pointing out that “We are in danger of repeating our past mistakes if we assume these new modalities are not 'drug-like' and cannot be oral drugs because they are not [rule of 5] compliant” (Michael Shultz from Novartis, cited in {% cite Mullard2018 %} where the authors re-assess the rule of 5). Then, in {% cite OHagan2014 %} we read "This famous "rule of 5" has been highly influential in this regard, but only about 50 % of orally administered new chemical entities actually obey it." 


## Veber's rule

In ({% cite Veber2002 %}) we read that the commonly applied molecular weight cutoff at 500 does not itself significantly separate compounds with poor oral bioavailability from those with acceptable values. The authors explain that on average both the number of rotatable bonds and polar surface area or hydrogen bond count tend to increase with molecular weight which may in part explain the success of the molecular weight parameter in predicting oral bioavailability. Their observations suggest that compounds will have a high probability of good oral bioavailability if they meet the two criteria: 
- 10 or fewer rotatable bonds - increased rotatable bond count has a negative effect on the permeation rate
- a polar surface area no greater than 140 Å <sup>2</sup> (or 12 or fewer H-bond donors and acceptors) - reduced polar surface area correlates better with increased permeation rate than does lipophilicity


# Big data for drug discovery

Have you ever wondered how new drugs are developed? It is actually quite a long and complex pathway and not only takes lots of time but also resources. The scheme below will help you understand the scale of that process.

![A scheme showing drug discovery pipeline: starting from target identification, hit, lead, lead optimisation, pre-clinical testing, clinical evaluation in phases I - III, and finally regulatory approval. Lead optimisation also includes iterative steps of designing, making, testing and analysing the proposed compounds.]({% link topics/computational-chemistry/images/medchem.png %} "A scheme showing drug discovery pipeline")

As you can see, to get just one drug to the market, often thousands of structures are assessed to find the most effective and safe compound. That's lots of information being generated in the whole process! The information can be stored in repositories and databases so that subsequent drug discovery process in a similar area is easier, faster and cheaper. Managing big data and using machine learning and computational chemistry methods is crucial in the lead optimisation step. By reviewing chemical databases and studying molecular docking simulations, we can save much time and resources to synthesise only those compounds that are the most promising based on *in-silico* methods. Even after synthesising the optimised 'leads', we still get lots of data from tests and analyses. By using appropriate cheminformatic tools and software, we can use the results to drive forward this iterative process of lead optimisation.


## Ten Vs of big data for drug discovery

Hopefully, now you have an idea of what the process of drug discovery looks like. To understand why managing data in medicinal chemistry is so challenging, let's have a closer look at the "Ten Vs" of big data {% cite Zhao2020 %}:

- **V**olume: size of data. 
- **V**elocity: speed of new data generation. 
- **V**ariety: various formats of data. 
- **V**eracity: quality of data. 
- **V**alidity: authenticity of data. 
- **V**ocabulary: terminology of data. 
- **V**enue: platform of data generation. 
- **V**isualization: view of data. 
- **V**olatility: duration of data usefulness. 
- **V**alue: the potential of data usefulness to reduce the cost of drug discovery and development.

Medicinal chemists must keep in mind those features while both searching for data and publishing it. In this way we can focus on the main aspects of data management, try to improve the availability and normalisation of data, and be aware of limitations of repositories and inconsistencies in data quality. 

## FAIR MedChem
As you saw before, there are many properties that the compounds can be searched by, such as molecular weight, H-bond donors or acceptors, polarity, etc. Sometimes there is a need to assess other molecular properties but it might happen that not all the information is included in the database. This is one of the reasons why new repositories are being developed – they are more specific and gather particular properties of interest. How much easier the life of scientists could be if the relevant data was publicly available, well-ordered and contained the needed metadata? By submitting the data with the necessary information to the repository is a good way to make the data FAIR. This will make it:
- **F**indable, as the data will be given specific identifiers
- **A**ccessible, as the data will be available online, open and free where possible
-	**I**nteroperable, as the repository will often enforce the use of formalised, consistent language
-	**R**eusable, as the data will be released under a license with detailed provenance
In the repositories 


# Chemical and pharmacological databases

Currently, there are lots of publicly available databases storing information about hit molecules, chemical structures and properties, drug targets, pharmaceuticals… In the recent paper {% cite Zhao2020 %} the authors collated relevant databases to advance computer-aided drug discovery and divided them into several main groups:
- Chemical collections
- Drug / drug-like compounds
- Drug targets, including genomics and proteomics data
- Biological data from assay screening, metabolism, and efficacy studies
- Drug liabilities and toxicities
- Clinical databases

Below you will find the databases listed under corresponding categories, all taken from {% cite Zhao2020 %} paper. 

There are many more databases available, and many are still being developed. They are often quite specific and contain certain types of compounds (eg. [PROTACs](http://cadd.zju.edu.cn/protacdb/) or are aimed at a particular disease. If the listed databases are not specific enough for your research, you can try luck by searching smaller, more specific databases. 

## Chemical collections

| Database | Description | Size (as of 29 October 2019) |
|:---:|:---:|:---:|
| [Enamine REAL Database](https://enamine.net/hit-finding/compound-collections/real-database) | Tool used to find new hit molecules using large-scale virtual screening and for searching analogues to hit molecules | >700 million compounds that comply with ‘rule of 5’ and Veber criteria |
| [ZINC](http://zinc.docking.org/) | Contains compound information including 2D/3D structure, purchasability,   target, and biology-related information | >230 million compounds in 3D formats and >750 million compounds for analogue searching |
| [PubChem](https://pubchem.ncbi.nlm.nih.gov) | Contains chemical molecule (mostly small molecule) information, including chemical structures, identifiers, chemical and physical properties,   biological activities, safety and toxicity data | 97 million compounds, 236 million substances, 268 million bioactivities |
| [ChemSpider](https://www.chemspider.com/) | Free chemical structure database providing fast access to >78 million   structures, properties, and associated information | >78 million compound structures |
| [SCUBIDOO](http://kolblab.org/scubidoo/index.php) | Freely accessible database that currently holds 21 million virtual products originating from a small library of building blocks and collection of robust organic reactions | 21 million virtual products |
| [ChEMBL](https://www.ebi.ac.uk/chembl/) | Manually curated database of bioactive molecules with drug-like properties; brings together chemical, bioactivity, and genomic data to aid translation of genomic information into effective new drugs. | >1.9 million compounds, 1.1 million pieces of assay information |
| [TCM-Mesh](http://mesh.tcm.microbioinformatics.org/) | Integration of a database and a data-mining system for network pharmacology analysis of all respects of traditional Chinese medicine,   including herbs, herbal ingredients, targets, related diseases, adverse effect, and toxicity | 383 840 compounds, 6235 herbs |
| [Super Natural II](http://bioinf-applied.charite.de/supernatural_new/index.php) | Contains natural compounds, including information about corresponding 2D   structures, physicochemical properties, predicted toxicity class and potential vendors | 325 508 natural compounds |
| [BIAdb](https://webs.iiitd.edu.in/raghava/biadb/index.html) | Comprehensive database of benzylisoquinoline alkaloids, containing information about 846 unique benzylisoquinoline alkaloids | ~846 unique benzylisoquinoline alkaloids |

## Drug / drug-like compounds

| Database | Description | Size (as of 29 October 2019) |
|:---:|:---:|:---:|
| [AICD](http://956023.ichengyun.net/AICD/index.php) | Anti-Inflammatory Compounds Database (AICD) deposits compounds with potential antiinflammation activities | 79 781 small molecules |
| [DrugBank](https://go.drugbank.com/) | Unique bioinformatics and cheminformatics resource that combines detailed drug data with comprehensive drug target information | 13 441 drug entries |
| [ReFRAME](https://reframedb.org/) | Screening library of 12 000 molecules assembled by combining three databases (Clarivate Integrity, GVK Excelra GoStar, and Citeline   Pharmaprojects) to facilitate drug repurposing | 12 000 molecules |
| [SuperDRUG2](http://cheminfo.charite.de/superdrug2/) | Contains approved/marketed drugs with regulatory details, chemical structures (2D and 3D), dosage, biological targets, physicochemical properties, external identifiers, adverse effects, and PK data | >4600 active pharmaceutical ingredients |
| [Drugs@FDA database](https://www.fda.gov/drugs/drug-approvals-and-databases/drugsfda-data-files) | Information about drugs from FDA | ~23 391 drug application records |
| [e-Drug3D](https://chemoinfo.ipmc.cnrs.fr/MOLDB/index.php) | Contains 1930 molecular structures approved by FDA between 1939 and 2019   with a molecular weight <2000 | 1930 drugs |

## Drug targets, including genomics and proteomics data

| Database | Description | Size (as of 29 October 2019) |
|:---:|:---:|:---:|
| [BindingDB](https://www.bindingdb.org/bind/index.jsp) | Public, web-accessible database of measured binding affinities, focusing chiefly on interactions of proteins considered to be candidate drug targets with ligands that are small,   drug-like molecules | 1 756 093 binding data, for 7371 protein targets and 780 240 small molecules |
| [Supertarget](http://insilico.charite.de/supertarget/index.php?site=home) | An extensive web resource for analyzing drugtarget interactions. | 332 828 drug-target interactions |
| [Ligand Expo](http://ligand-expo.rutgers.edu/index.html) | Provides chemical and structural information about small molecules within structure entries of Protein Data   Bank. | 30 440 entries of ligand |
| [PDBeChem](https://www.ebi.ac.uk/pdbe-srv/pdbechem/) | Consistent and enriched library of ligands, small molecules, and monomers referenced as residues and hetgroups in PDB entries | >29 922 ligands |
| [PDBbind-CN](http://www.pdbbind-cn.org/) | Provides essential linkage between energetic and structural information of biomolecular complexes, which is helpful for various computational and statistical studies on molecular recognition in biological systems | 19 588 biomolecular complexes |
| [STITCH](http://stitch.embl.de/) | Database integrating information about interactions from metabolic pathways, crystal structures, binding experiments, and drug–target relationships | Interactions between 300 000 small molecules and 2.6 million proteins from 1133 organisms |
| [BioGRID](https://thebiogrid.org/) | The Biological General Repository for Interaction Datasets is an open-access database on protein, genetic, and chemical interactions for humans and all major model organisms | 1 753 686 protein and genetic interactions, 28 093 chemical associations and 874 796 posttranslational modifications from major model organisms |
| [Binding MOAD](http://bindingmoad.org/) | Created from a subset of Protein   Data Bank (PDB), containing every high-quality example of ligand-protein binding. | 36 047 protein-ligand structures, and 13 353 binding data |
| [GPCRdb](https://gpcrdb.org/) | Contains data from GPCRs, including crystal structures, sequence alignments, and receptor mutations; can be visualized in interactive diagrams; provides online analysis tools | 15 149 proteins, and 144 917 ligands |
| [Guide to Pharmacology](https://www.guidetopharmacology.org/) | IUPHAR/BPS guide to pharmacology is an open-access, expert-curated database of molecular interactions between ligands and their targets. | 2937 targets, and 9859 ligands |
| [GLASS](https://zhanglab.ccmb.med.umich.edu/GLASS/) | GPCR-Ligand Association (GLASS)   database is a manually curated repository for experimentally validated   GPCR–ligand interactions; along with relevant GPCR and chemical information,   GPCRligand association data are extracted and integrated into GLASS from literature and public databases | ~ 277 651 unique ligands and 3048 GPCRs |

## Biological data from assay screening, metabolism, and efficacy studies

| Database | Description | Size (as of 29 October 2019) |
|:---:|:---:|:---:|
| [HMDB](https://hmdb.ca/about) | Freely available electronic database containing detailed information about small-molecule metabolites found in the human body | 114 162 metabolite entries |
| [SMPDB](http://smpdb.ca/) | Small Molecule Pathway Database (SMPDB) is an interactive, visual database containing >30 000 small-molecule pathways found in humans only | >30 000 small-molecule pathways |
| [TTD](http://db.idrblab.net/ttd/) | Therapeutic Target Database (TTD) is a database providing information about known and explored therapeutic protein and nucleic acid targets,   targeted disease, pathway information and corresponding drugs directed at each of these targets | 2589 targets, and 31 614 drugs |
| [BioCyc](https://biocyc.org/) | Collection of 7615 pathway/genome databases; each database in BioCyc collection describes genome and metabolic pathways of a single organism | 7615 pathway/genome databases |
| [BiGG](http://bigg.ucsd.edu/) | Metabolic reconstruction of human metabolism designed for systems biology simulation and metabolic flux balance modelling | 2004 proteins, 2766 metabolites, and 3311 metabolic and transport reactions |
| [BRENDA](https://www.brenda-enzymes.org/) | Main collection of enzyme functional data available to scientific   community | At least 40 000 different enzymes from >6900 different organisms |
| [Reactome](https://reactome.org/) | Curated, peer-reviewed knowledgebase of biological pathways, including metabolic pathways, and protein trafficking and signalling pathways | >9600 proteins, 9800 reactions, and 2000 pathways for humans |
| [BioModels Database](https://www.ebi.ac.uk/biomodels/) | Repository of computational models of biological processes; models   described from literature are manually curated and enriched with   cross-references | 6753 patient-derived genome-scale metabolic models, 112 898 metabolic   models etc. |
| [KEGG](https://www.genome.jp/kegg) | Database resource that integrates genomic, chemical, and systemic functional information | 18 652 metabolites |
| [CARLSBAD](http://carlsbad.health.unm.edu/carlsbad/?mode=home) | Database and knowledge inference system that integrates multiple bioactivity data sets to provide researchers with novel capabilities for mining and exploration of available structure-activity relationships (SAR)   throughout chemical biology space. | 932 852 CARLSBAD activities, 890 323 unique structure–target pairs, 3542   targets, 435 343 unique structures |
| [WOMBAT](http://dud.docking.org/wombat/) | Contains 331 872 entries, representing 1966 unique targets, with bioactivity annotations | 268 246 unique structures |
| [Open NCI Database](https://cactus.nci.nih.gov/ncidb2.2/) | Maintained by the National Cancer Institute; contains small-molecule information such as names, biological activities, and structures; a useful resource for researchers working in cancer/AIDS fields | >250 000 compounds |
| [NPACT](http://crdd.osdd.net/raghava/npact/) | Provides information on the plant-derived natural compounds, including structure, properties (physical, elemental, and topological), cancer type,   cell lines, inhibitory values (IC50, ED50, EC50, GI50), molecular targets,   commercial suppliers, and drug-likeness of compounds | 1574 entries |
| [PKPB_DB](https://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=204443) | Contains physiological parameter values for humans from early childhood through senescence; intended to be used in physiologically based (PB) PK modelling; also contains similar data for animals (primarily rodents) | N/A |

## Drug liabilities and toxicities

| Database | Description | Size (as of 29 October 2019) |
|:---:|:---:|:---:|
| [T3DB](http://www.t3db.ca/) | Unique bioinformatics resource that combines detailed toxin data with comprehensive toxin target information | 3678 toxins |
| [DrugMatrix](https://ntp.niehs.nih.gov/data/drugmatrix/) | One of world’s largest toxicogenomic reference resources | ~ 600 drug molecules and 10 000 genes |
| [ACToR](https://actor.epa.gov/actor/home.xhtml) | Includes computational toxicology information about compounds, including   HTS, chemical exposure, sustainable chemistry (chemical structures and physicochemical properties) and virtual tissue data | >500 000 chemicals |
| [SkinSensDB](https://cwtung.kmu.edu.tw/skinsensdb/) | Contains curated data from published AOP-related skin sensitization   assays | 710 unique chemicals |
| [SIDER](http://sideeffects.embl.de/download/) | Contains information on marketed medicines and their recorded adverse drug reactions, including frequency, drug and adverse effect classifications | 1430 drugs with 5868 side effect information |
| [LTKB Benchmark Dataset](https://www.fda.gov/science-research) | Contains drugs with potential to cause drug-induced liver injury in humans; established using FDA-approved prescription drug labels;   liver-toxicity-knowledge-base-ltkb/ltkb-benchmark-dataset | 287 prescription drugs |
| [CTD](http://ctdbase.org/) | Comparative Toxicogenomics Database (CTD) is a premier public resource for literature-based, manually curated associations between chemicals, gene products, phenotypes, diseases, and environmental exposures | 13 378 unique chemicals and related information |

## Clinical databases

| Database | Description | Size (as of 29 October 2019) |
|:---:|:---:|:---:|
| [ClinicalTrials.gov](https://clinicaltrials.gov/) | Database of privately and publicly funded clinical studies conducted around the world | ~ 324 429 research studies in all 50 US states and 209 countries |
| [AACT database](https://aact.ctti-clinicaltrials.org/) | Publicly available relational database that contains all information   (protocol and result data elements) about every study registered in   ClinicalTrials.gov. Content is downloaded from ClinicalTrials.gov daily and loaded into AACT | ~ 324 429 research studies in all 50 US states and 209 countries |
| [EORTC Clinical Trials Database](https://www.eortc.org/clinical-trials-database/) | Contains information about EORTC clinical trials and clinical trials from other organizations with EORTC participation | N/A |
| [Exposome-Explorer](http://exposome-explorer.iarc.fr/) | Contains detailed information on the nature of biomarkers, populations and subjects where measured, samples analyzed, methods used for biomarker analyses, concentrations in biospecimens, correlations with external exposure measurements, and biological reproducibility over time | 908 biomarkers |
| [PharmaGKB](https://www.pharmgkb.org/) | A pharmacogenomics knowledge resource that encompasses clinical information about drug molecules | 733 drugs with their clinical information |


# Data management in chemistry using Galaxy

## Chemical file formats

Depending on the type of analysis you are going to perform, you will work with different file formats. Below are the most important data types, commonly used in cheminformatics, that you will likely see in Galaxy:
- SDF (.sd, .sdf) - stands for Structure-Data Format, and SDF files wrap the molfile (MDL Molfile) format. Stores information about the chemical structure and associated data of compounds in plain text. Files in SDF format can encode single or multiple molecules that are then delimited by lines consisting of four dollar signs. SDF files are formatted ASCII files that store information about the positions of the individual atoms (either in 2D or 3D space) that make up the molecule. The data on connectivity and hybridization state are also encoded, although their use is less frequent and often inconsistent.
- MOL (.mol) - an MDL Molfile for holding information about the atoms, bonds, connectivity and coordinates of a molecule. 
- MOL2 (.mol2) - a Tripos Mol2 file is a complete, portable representation of a SYBYL molecule. It is an ASCII file which contains all the information needed to
reconstruct a SYBYL molecule
- CML (.cml) - Chemical Markup Language (ChemML or CML) is an approach to managing molecular information using tools such as XML and Java. It supports a wide range of chemical concepts including molecules, reactions, spectra and analytical data, computational chemistry, chemical crystallography and materials
- InChI (IUPAC International Chemical Identifier) - a textual identifier for chemical substances, designed to provide a standard way to encode molecular information. The identifiers describe chemical substances in terms of layers of information — the atoms and their bond connectivity, tautomeric information, isotope information, stereochemistry, and electronic charge information ({% cite Heller2015 %})
- SMILES (.smi) - the **s**implified **m**olecular-**i**nput **l**ine-**e**ntry **s**ystem (SMILES) is a specification in the form of a line notation for describing the structure of chemical species using short ASCII strings. A linear text format which can describe the connectivity and chirality of a molecule ({% cite Weininger1988 %}) 
- PDB - the Protein Data Bank (PDB) file format is a textual file format describing the three-dimensional structures of molecules held in the [Protein Data Bank](https://www.rcsb.org/), now succeeded by the mmCIF format. It contains a description and annotation of protein and nucleic acid structures including atomic coordinates, secondary structure assignments, as well as atomic connectivity. In addition, experimental metadata is stored. ({% cite Berman2007 %})
- GRO (.gro) - a plain text file storing spatial coordinates and velocities (if available) of atoms during a molecular dynamics simulation, utilised by [GROMACS](https://manual.gromacs.org/archive/5.0.3/online/gro.html)

To visualise the structures held by the files with the positions of the individual atoms, you can use (NGL Viewer)[https://nglviewer.org/ngl/].

If you plan to work with molecular dynamics simulations, there are also some MD trajectory file formats that you might want to get familiar with. In Galaxy, you can convert between xtc, trr, dcd and netcdf files using a tool called {% tool [MDTraj file converter](toolshed.g2.bx.psu.edu/repos/chemteam/md_converter/md_converter/1.9.6+galaxy0) %}.

## Galaxy tools 

In Galaxy Chemical Toolbox there are dozens of tools that can be used for various analyses. Below we will only show a few, mostly related to data import, format conversion and some functions linked to what was discussed previously. 

Let's start with importing publicly available data. [Protein Data Bank](https://www.rcsb.org/) stores thousands of three-dimensional structural data of proteins nucleic acids and other biological molecules.

> <hands-on-title>Import PDB file from Protein Data Bank</hands-on-title>
>
> 1. Open [Protein Data Bank](https://www.rcsb.org/)
> 2. Type your search term in the box *"Enter search term(s), Entry ID(s), or sequence"*: `MAO B`
> ![A screenshot showing a search box in the upper right corner of the Protein Data Bank website.]({% link topics/computational-chemistry/images/pdb_search.jpg %} "Search box of the Protein Data Bank")
>3. Choose the structure of interest and copy its accession code: 2BK3
> ![A screenshot showing some of the resulting structures with their accession codes.]({% link topics/computational-chemistry/images/pdb_results.jpg %} "How to get accession code of the resulting structures.")
>4. Switch to Galaxy
>5. {% tool [Get PDB file](toolshed.g2.bx.psu.edu/repos/bgruening/get_pdb/get_pdb/0.1.0) %} with the following parameters:
>    - *"PDB accession code"*: `2BK3`
>
{: .hands_on}

This is often the first step of docking studies that you can learn about in other tutorials, for example in [Protein-ligand docking tutorial]({% link topics/computational-chemistry/tutorials/cheminformatics/tutorial.md %}).

You can import any molecule to Galaxy using SMILES notation. Below we show an example with benzenesulfonyl chloride and ethylamine. 

> <hands-on-title>Import SMILES</hands-on-title>
>
> 1. Copy SMILES of your molecule(s) of interest. In this example, we use benzenesulfonyl chloride (SMILES: C1=CC=C(C=C1)S(=O)(=O)Cl) and ethylamine (SMILES: CCN).
> 2. Click on the **{% icon galaxy-upload %} Upload data** button in the tools panel on the left-hand side.
> 3. Click on the **{% icon galaxy-wf-edit %} Paste/Fetch data** button twice. Two boxes will appear.
> 4. In the first box, under **"Name"** section, enter `Benzenesulfonyl chloride` and under **"Type"**: `smi`
> 5. Below paste the SMILES of benzenesulfonyl chloride: `C1=CC=C(C=C1)S(=O)(=O)Cl`
> 6. In the second box, under **"Name"** section, enter `Ethylamine` and under **"Type"**: `smi`
> 7. Below paste the SMILES of ethylamine: `CCN`
> 8. Click the **"Start"** button, then close the dialogue box. Your files are being added to your history!
>    
{: .hands_on}

As you learned from the previous section, there are many different formats used in computational chemistry. Of course, Galaxy allows you to interconvert between them - just have a look below!

> <hands-on-title>Convert the file format</hands-on-title>
>
> 1. {% tool [Compound conversion](toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy0) %} with the following parameters:
>    - {% icon param-files %} *"Molecular input file"*: click on {% icon param-files %} *Multiple datasets* icon and choose both `Benzenesulfonyl chloride` and `Ethylamine`
>    - *"Output format"* - you have many options to choose from! Not only the formats but also the associated parameters. Just scroll the list and pick the one that is relevant to your downstream analysis. Here we use `MDL MOL format (sdf, mol)`
> 2. Rename {% icon galaxy-pencil %} the corresponding files `Benzenesulfonyl chloride SDF` and `Ethylamine SDF`. Before renaming the files, make sure to check either the starting dataset numbers or the atoms in the file in order not to confuse the two files! 
>
{: .hands_on}

We intentionally chose to work on benzenesulfonyl chloride and ethylamine. Do you know why? Well, if you think about the synthesis of sulfonamides, that's exactly what you need - a sulfonyl chloride and an amine. So... let's do some computational synthesis! 

> <hands-on-title>Run the reaction</hands-on-title>
>
> 1. {% tool [Reaction maker](toolshed.g2.bx.psu.edu/repos/bgruening/ctb_im_rxn_maker/ctb_im_rxn_maker/1.1.4+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `Benzenesulfonyl chloride SDF`
>    - *"Reagent options"*: Sulfonamide
>    - {% icon param-file %} *"Reagent file"*: `Ethylamine SDF`
>
{: .hands_on}

> <tip-title> Reaction maker tool works also with SMILES!</tip-title>
>
> Above we converted the *.smi* file into *.sdf* to show how {% icon tool %} **Compound conversion** tool works. However, it is worth pointing out that {% icon tool %} **Reaction maker** works also with SMILES files and the conversion happens automatically so that your *.smi* dataset is imported as *.sdf* input.
>
{: .tip} 

> <details-title>Which reactions I can run using this tool?</details-title>
>
> {% icon tool %} **Reaction maker** works with more reaction types, such as:
> - Amides synthesis
> - Nucleophilic aromatic substitution (SNAr mechanism)
> - Urea
> - Suzuki coupling
> - Sulfonamide
> - Reductive amination
> - N-alkylation
> - Ether coupling
> - Ester coupling
> - Benzimidazole
> - Triazole
> - Benzoxazole
> - Sarah Cu
> - Sarah Quat Am
>
{: .details}


The output is the *.txt* log file and the SDF file with the product molecule. If you have a look at that file, you'll see that there is just a list of atoms with their corresponding coordinates. To draw the resulting structure, we can use another tool.

> <hands-on-title>Visualisation of compounds</hands-on-title>
>
> 1. {% tool [Visualisation](toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_svg_depiction/openbabel_svg_depiction/3.1.1+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Molecular input file"*: `SDF output for Reaction maker`
>    - *"Property to display under the molecule"* - there are many properties that you can choose from! Please note that sometimes if not enough information is provided, then the property might not be displayed. Let's choose a basic but useful property: `Molecular weight`
>    - *"Format of the resulting picture"*: `SVG`
>
{: .hands_on}

Since we spoke about the drug-likeness in the previous section, let's see how it works in practice, on the example of our "synthesised" molecule.

> <hands-on-title>Estimate the drug-likeness</hands-on-title>
>
> 1. {% tool [Drug-likeness](toolshed.g2.bx.psu.edu/repos/bgruening/qed/ctb_silicos_qed/2021.03.4+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Molecule data in SDF or SMILES format"*: `SDF output for Reaction maker`
>    - *"Method"* - two possible methods to weight the features are available - one based on max. weight (QEDw,max), and the other on mean weight (QEDw,mo). There is also an option to leave features unweighted (QEDw,u) and we'll choose this one: `unweighted (QEDw,u)`
>    - *"Include the descriptor names as header"*: {% icon param-toggle %} `Yes`
>
{: .hands_on}

> <details-title>What are the assessed properties?</details-title>
>
> The eight properties used are molecular weight (MW), octanol–water partition coefficient (ALOGP), number of hydrogen bond donors (HBDs), number of hydrogen bond acceptors (HBAs), molecular polar surface area (PSA), number of rotatable bonds (ROTBs), number of aromatic rings (AROMs) and number of structural alerts (ALERTS).
>
{: .details}

In medicinal chemistry, we often base the new structures on scaffolds of existing drugs. Therefore, it is quite useful to be able to find similar structures - below is an example of how to do it in Galaxy.

> <hands-on-title>Search ChEMBL database for similar compounds</hands-on-title>
>
> 1. {% tool [Search ChEMBL database](toolshed.g2.bx.psu.edu/repos/bgruening/chembl/chembl/0.10.1+galaxy4) %} with the following parameters:
>    - *"SMILES input type"*: `File`
>    - {% icon param-file %} *"Input file"*: `Benzenesulfonyl chloride`
>    - *"Search type"*: `Substructure`
> 2. Rename {% icon galaxy-pencil %} the output file `Benzenesulfonyl chloride substructures`
>    
{: .hands_on}


> <tip-title>Search ChEMBL database tool works also with SDF!</tip-title>
>
> Even though the input format of the above tool is SMILES, it can automatically interconvert between *.sdf* and *.smi* formats, so you don't have to use {% icon tool %} **Compound conversion** tool beforehand. 
> 
{: .tip} 

> <details-title>Parameters of Search ChEMBL database tool</details-title>
>
>  {% icon tool %} **Search ChEMBL database** tool allows for searching the ChEMBL database for compounds which resemble a SMILES string. Two search options are possible:
> - similarity (searches for compounds which are similar to the input within a specified Tanimoto cutoff)
> - substructure (searches for compounds which contain the input substructure)
>   
> Results can be filtered for compounds which are
> - approved drugs
> - biotherapeutic
> - natural products
> - fulfil all of the Lipinski rule of five criteria
>
{: .details}

Let's repeat this step, but with Lipinski's Rule of Five which you learned about in the previous section of this tutorial!

> <hands-on-title>Search ChEMBL database for similar compounds using the Lipinski rule of five</hands-on-title>
>
> 1. Click the {% icon galaxy-refresh %} **Repeat** button on the previous dataset.
> 2. You will be redirected to {% tool [Search ChEMBL database](toolshed.g2.bx.psu.edu/repos/bgruening/chembl/chembl/0.10.1+galaxy4) %}. Leave all the parameters as before, except for one:
>    - *"Filter for Lipinski's Rule of Five"*: {% icon param-toggle %} `Yes`
> 2. Rename {% icon galaxy-pencil %} the output file `Benzenesulfonyl chloride Lipinski substructures`
>
{: .hands_on}


> <question-title></question-title>
>
> How many molecules have been filtered out?
>
> > <solution-title></solution-title>
> >
> > If you click on the dataset, you will see a short summary of how many molecules have been found.
> > When we didn't apply any filters, the tool found 45 molecules. After applying Lipinski's Rule of Five filter, 36 molecules satisfied the rules. 
> > ![Image showing the preview of the output datasets from searching benzenesulfonyl chloride substructures in the ChEMBL database, with and without Lipinski's Rule of Five filter.]({% link topics/computational-chemistry/images/substructures.png %} "Preview showing the number of molecules found in the ChEMBL database, before and after applying Lipinski's Rule of Five filter")
> >
> {: .solution}
>
{: .question}


## Galaxy workflow summary

Those are just a few tools, but they offer many parameters that you can tune depending on your analysis. Additionally, we provide you with an [example history](https://usegalaxy.eu/u/j.jakiela/h/data-management-in-medicinal-chemistry-tutorial) and the dedicated [workflow](https://usegalaxy.eu/u/j.jakiela/w/data-management-in-medicinal-chemistry-workflow) of this small tutorial. You are more than welcome to explore other tools in the Galaxy Chemical Toolbox though!

# Data-driven medicinal chemistry - yesterday, today and tomorrow

## Why do we need big data in med-chem?
In the article by {% cite Lusher2014 %} the role of the medicinal chemist is to make decisions about which of the infinite possibilities of new compounds should be made next. As the amount and variety of data on which to base these decisions grows, so too must the data analysis skills of the medicinal chemist. The authors claim that modern medicinal chemists should be able to recognise sources of relevant information, prepare raw data, use statistical tools, extract meaningful information, interpret results, recognise potential problems and make visualisations to communicate their findings to improve the quality of compounds being produced in research. According to the authors, this situation will require improved education and increased access to data and information management tools. This is where Galaxy comes in, offering not only a platform for data analysis with a plethora of the most commonly used tools in the field but also educational materials allowing anyone to learn and excel in analysis. 
 
The privilege of applying computational methods in medicinal chemistry pipelines allows to focus on the most promising compounds and to remove unsuitable ones before the stage of chemical synthesis. Not only is this approach more efficient but also more sustainable and can reduce the costs of the synthetic stage of drug discovery ({% cite Brown2018 %}).

Another important role that big data plays in drug discovery is target identification (and validation) which might be currently one the biggest challenges in medicinal chemistry. By introducing genomic data in drug discovery, we can produce more specific and effective medicines. Understanding the underlying causes of the disease and the biological targets are crucial in designing new drugs and hence should be inherently linked with this process, where possible. 

## Data challenges 
It is important to understand the limitations of the databases we use since if data is not well understood, the processing and analysis drawn may ultimately be flawed, following the ‘garbage in, garbage out’ principle. Here are some challenges mentioned in {% cite Brown2018 %} that you can explore more in-depth by referring to that article.
-	Errors
-	Reproducibility
-	Standardisation
-	Formatting of data
-	Information held in silos
-	Loss of contextual information
-	Limited and biased data

## Machine learning in med-chem
Given the huge number of molecules in the various databases, applying machine learning in cheminformatics and building predictive models of the physiochemical properties of molecules is more and more popular. It helps to predict absorption, distribution, metabolism, excretion, toxicity (ADMET), likelihood of interaction with the drug targets as well as off-target effects. It can be useful in elucidating complex protein-protein or drug-drug interaction networks, in understanding structure-activity relationships (SARs) and when combined with data from the ‘omics’ revolution. This approach also helps to identify bioisosteres and introduce the idea of scaffold hopping in molecular design ({% cite Brown2018 %}).

<!---
TO FURTHER IMPROVE THE TUTORIAL:
expand this section, include examples
-->

## Knowledge sharing
It is not only about gathering and managing the data but also building a culture of knowledge sharing. We work in big, international teams with well-established methods and well-equipped labs. There are so many opportunities for knowledge capture and exchange! The challenge emerges here though – how to efficiently connect and help each other? This is the question that many pharmaceutical companies try to answer, such as Merck & Co. who describe their approach to knowledge management and report on the multiple enduring and complementary teams and initiatives to capture and share knowledge ({% cite Beshore2022 %}).

tools_json:
["MDTraj file converter", "toolshed.g2.bx.psu.edu/repos/chemteam/md_converter/md_converter/1.9.6+galaxy0", "Get PDB file", "toolshed.g2.bx.psu.edu/repos/bgruening/get_pdb/get_pdb/0.1.0", "Compound conversion", "toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy0", "Reaction maker", "toolshed.g2.bx.psu.edu/repos/bgruening/ctb_im_rxn_maker/ctb_im_rxn_maker/1.1.4+galaxy0", "Visualisation", "toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_svg_depiction/openbabel_svg_depiction/3.1.1+galaxy0", "Drug-likeness", "toolshed.g2.bx.psu.edu/repos/bgruening/qed/ctb_silicos_qed/2021.03.4+galaxy0", "Search ChEMBL database", "toolshed.g2.bx.psu.edu/repos/bgruening/chembl/chembl/0.10.1+galaxy4"]


topic: computational-chemistry
tutorial: zauberkugel
title: Protein target prediction of a bioactive ligand with Align-it and ePharmaLib
difficulty: intermediate
time_estimation: 2H
requirements_json: []
tags_json: []
content_hash: dd9d303b8f33a04b9d6ace64efc196f4
last_modified: 2026-05-15T22:28:10.773482
zenodo_link: https://zenodo.org/record/6055897
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/computational-chemistry/tutorials/zauberkugel/tutorial.html

content:
Historically, the pharmacophore concept was formulated in 1909 by the German physician and Nobel prize laureate Paul Ehrlich ({% cite Ehrlich1909 %}). According to the [International Union of Pure and Applied Chemistry (IUPAC)](https://iupac.org/), a pharmacophore is defined as “an ensemble of steric and electronic features that is necessary to ensure the optimal supramolecular interactions with a specific biological target and to trigger (or block) its biological response” ({% cite Wermuth1998 %}). Starting from the cocrystal structure of a non-covalent protein–ligand complex (e.g. Figure 1), pharmacophore perception involves the extraction of the key molecular features of the bioactive ligand at the protein–ligand contact interface into a single model ({% cite Moumbock2019 %}). These pharmacophoric features mainly include: H-bond acceptor (HACC or A), H-bond donor (HDON or D), lipophilic group (LIPO or H), negative center (NEGC or N), positive center (POSC or P), and aromatic ring (AROM or R) moieties. Moreover, receptor-based excluded spheres (EXCL) can be added in order to mimic spatial constraints of the binding pocket (Figure 2). Once a pharmacophore model has been generated, a query can be performed either in a forward manner, using several ligands to search for novel putative hits of a given target, or in a reverse manner, by screening a single ligand against multiple pharmacophore models in search of putative protein targets ({% cite Steindl2006 %}).

![PDB ID: 4MVF]({% link topics/computational-chemistry/images/4MVF-STU.png %} "Crystal Structure of *Plasmodium falciparum* calcium-dependent protein kinase 2 (CDPK2) complexed with staurosporine (STU) with PDB ID: [4MVF](https://www.rcsb.org/structure/4mvf). Image generated using Maestro (Schrödinger LLC, NY).")

Bioactive compounds often bind to several target proteins, thereby exhibiting polypharmacology. However, experimentally determining these interactions is laborious, and structure-based virtual screening of bioactive compounds could expedite drug discovery by prioritizing hits for experimental validation. The recently reported ePharmaLib ({% cite Moumbock2021 %}) dataset is a library of 15,148 e-pharmacophores modeled from solved structures of pharmaceutically relevant protein–ligand complexes of the screening Protein Data Bank (sc-PDB, {% cite Desaphy2014 %}). ePharmaLib can be used for target fishing of phenotypic hits, side effect predictions, drug repurposing, and scaffold hopping.

![STU]({% link topics/computational-chemistry/images/STU.png %} "Depiction of the 2D structure of staurosporine (left) and 3D structure (right) with key pharmacophoric features extracted from the STU–CDPK2 complex (PDB ID: [4MVF](https://www.rcsb.org/structure/4mvf)). Image generated using Maestro (Schrödinger LLC, NY).")

In this tutorial, you will perform pharmacophore-based target prediction of a bioactive ligand known as staurosporine (Figure 2) with the ePharmaLib subset representing *Plasmodium falciparum* protein targets (138 pharmacophore models) and the open-source pharmacophore alignment program [Align-it](https://anaconda.org/bioconda/align_it), formerly known as PHARAO ({% cite Taminau2008 %}).

> <details-title>Pharmacology of staurosporine</details-title>
>
>  Staurosporine (PDB hetID: [STU](https://www.rcsb.org/ligand/STU)) is an indolocarbazole secondary metabolite isolated from several bacteria of the genus Streptomyces. It displays diverse biological activities such as anticancer and antiparasitic activities ({% cite Nakano2009 %}).
{: .details}

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}


# Create a history

As a first step, we create a new history for the analysis.

> <hands-on-title>Hands-on 1: Create history</hands-on-title>
>
> 1. Create a new history.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Rename it to `Staurosporine target prediction`.
>
>    {% snippet faqs/galaxy/histories_rename.md name = "Staurosporine target prediction" %}
>
{: .hands_on}


# Get data

For this exercise, we need two datasets: the ePharmaLib pharmacophore library (PHAR format) and a query ligand structure file (SMI format). 

## Fetching the ePharmaLib dataset

Firstly, we will retrieve the concatenated ePharmaLib subset representing *P. falciparum* protein targets. 

> <hands-on-title>Hands-on 2: Upload ePharmaLib</hands-on-title>
>
> 1. Upload the dataset from the [Zenodo](https://zenodo.org) link provided to your Galaxy history.
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md link="https://zenodo.org/record/6055897/files/ePharmaLib_PHARAO_plasmodium.phar" %}
>
>    > <comment-title>ePharmaLib versions</comment-title>
>    >
>    > Two versions of the ePharmaLib (PHAR & PHYPO formats) have been created for use with the pharmacophore alignment programs [Align-it](https://anaconda.org/bioconda/align_it) and [Phase](https://www.schrodinger.com/products/phase), respectively. Both versions can be broken down into small datasets. e.g. for human targets. They are freely available at [Zenodo](https://zenodo.org) under the link:
>    >	```
>    >	https://zenodo.org/record/6055897
>    >  ```
>    {: .comment}
>
> 2. Change the datatype from `tabular`to `phar`. This step is essential, as Galaxy does not automatically detect the datatype for PHAR files.
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="phar" %}
>
> 3. You can view the contents of the downloaded PHAR file by pressing the **eye icon** (View data) for this dataset. 
>
>    > <details-title>What is a PHAR file?</details-title>
>    >
>    > A PHAR file is essentially a series of lines containing the three-dimensional coordinates of pharmacophoric features and excluded spheres. The first column specifies a feature type (e.g. HACC is a hydrogen bond acceptor). Subsequent columns specify the position of the feature center in a three-dimensional space. Individual pharmacophores are separated by lines containing four dollar signs (`$$$$`). The pharmacophores of the ePharmaLib dataset were labeled according to the following three-component code *PDBID-hetID-UniprotEntryName*.
>    {: .details}
{: .hands_on}

## Creating a query ligand structure file

In this step, we will manually create an SMI file containing the SMILES of staurosporine.

> <details-title>What are SMILES and the SMI file format?</details-title>
>
> The simplified molecular-input line-entry system (SMILES) is a string notation for describing the 2D chemical structure of a compound. It only states the atoms present in a compound and the connectivity between them. As an example, the SMILES string of acetone is `CC(=O)C`. SMILES strings can be imported by most molecule editors and converted into either two-dimensional structural drawings or three-dimensional models of the compounds, and vice versa. For more information on how the notation works, please consult the [OpenSMILES specification](http://opensmiles.org/opensmiles.html) or the description provided by [Wikipedia](https://en.wikipedia.org/wiki/Simplified_molecular-input_line-entry_system). 
>
{: .details}

> <hands-on-title>Hands-on 3: Create an SMI file</hands-on-title>
>
>
> 1. Create a new file using the Galaxy upload manager, with the following contents. Make sure to select the datatype (with **Type**) as `smi`. This step is essential, as Galaxy does not automatically detect the datatype for SMI files.
>
>    ```
>    C[C@@]12[C@@H]([C@@H](C[C@@H](O1)N3C4=CC=CC=C4C5=C6C(=C7C8=CC=CC=C8N2C7=C53)CNC6=O)NC)OC	staurosporine
>    ```
>
>    {% snippet faqs/galaxy/datasets_create_new_file.md format="smi" %}
>
> > <tip-title>SMILES generation</tip-title>
> >
> > A SMILES string can automatically be generated from a ligand name or 2D structure with a desktop molecule editor such [ChemDraw®](https://perkinelmerinformatics.com/products/research/chemdraw/) and [Marvin®](https://chemaxon.com/products/marvin), or with web-based molecule editors such as [PubChem Sketcher](https://pubchem.ncbi.nlm.nih.gov//edit3/index.html) and [ChemDraw® JS](https://chemdrawdirect.perkinelmer.cloud/js/sample/index.html). Moreover, the pre-computed SMILES strings of a large number of bioactive compounds can be retrieved from chemical databases such as [PubChem](https://pubchem.ncbi.nlm.nih.gov/). e.g.
> >
> > ```
> > https://pubchem.ncbi.nlm.nih.gov/compound/44259#section=Isomeric-SMILES&fullscreen=true
> > ```
> >
> {: .tip}
>
{: .hands_on}

> <question-title></question-title>
>
> Why do we specifically use a so-called isomeric SMILES string?
>
> > <solution-title></solution-title>
> >
> > Staurosporine is a chiral molecule possessing four chiral centers. The SMILES notation allows the specification of configuration at tetrahedral centers and double bond geometry, by marking atoms with `@` or `@@`. These are structural features that cannot be specified by connectivity alone, and therefore SMILES which encode this information are termed isomeric SMILES. A notable feature of these rules is that they allow rigorous partial specification of chirality.
> >
> {: .solution}
>
{: .question}


# Pre-processing

Prior to pharmacophore alignment, the predominant ionization state(s) of the query ligand as well as its 3D conformers should be generated. Also, the pharmacophore dataset will be split into a collection of individual pharmacophore files.
 
## Ligand hydration

More often than not, the bioactive form of a compound is its predominant form at physiological pH (7.4). In this step, we predict the most probable ionization state(s) of the query ligand at pH 7.4 with the cheminformatics toolkit OpenBabel ({% cite OBoyle2011 %}).

> <hands-on-title>Hands-on 4: Add hydrogen atoms</hands-on-title>
>
> 1. {% tool [Add hydrogen atoms](toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_addh/openbabel_addh/3.1.1+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"Molecular input file"*: `staurosporine.smi` (from Hands-on 3)
>    - *"Add hydrogens to polar atoms only (i.e. not to carbon atoms)"*: `Yes`
>
> 2. Rename the output to `staurosporine_hydrated`.
>
>    {% snippet faqs/galaxy/datasets_rename.md name = "staurosporine_hydrated" %}
>
{: .hands_on}

> <question-title></question-title>
>
> Nitrogen-containing functional groups are known to be basic. Which of them present in staurosporine (Figure 2) do you expect to be protonated at pH 7.4, and which not? And why?
>
> > <solution-title></solution-title>
> >
> > Only the secondary N-methylamino group will be protonated because indoles, much like aromatic amides, are typically not basic.
> >
> {: .solution}
>
{: .question}

## Splitting ePharmaLib into individual pharmacophores

The ePharmaLib subset representing *P. falciparum* protein targets (*ePharmaLib_PHARAO_plasmodium.phar*) is a concatenated file containing 148 individual pharmacophore files. To speed up our analysis, it is preferable to split the dataset into individual files in order to perform several pharmacophore alignments in parallel, using Galaxy's collection functionality.

> <hands-on-title>Hands-on 5: Splitting ePharmaLib</hands-on-title>
>
> 1. {% tool [Split file](toolshed.g2.bx.psu.edu/repos/bgruening/split_file_to_collection/split_file_to_collection/0.5.0) %} with the following parameters:
>    - *"Select the file type to split"*: `Generic`
>        - {% icon param-file %} *"File to split"*: `ePharmaLib_PHARAO_plasmodium.phar` (from Hands-on 2)
>        - *"Method to split files"*: `Specify record separator as regular expression`
>            - *"Regex to match record separator"*: `\$\$\$\$`
>            - *"Split records before or after the separator?"*: `After`
>        - *"Specify number of output files or number of records per file?"*: `Number of records per file ('chunk mode')`
>        - *"Base name for new files in collection"*: `epharmalib`
>        - *"Method to allocate records to new files"*: `Maintain record order`
>
> 2. Rename the output to `ePharmaLib_PLAF_split`.
>
>    {% snippet faqs/galaxy/datasets_rename.md name = "ePharmaLib_PLAF_split" %}
>
{: .hands_on}

## Ligand conformational flexibility

To reduce the calculation time, the Align-it ({% cite Taminau2008 %}) tool performs rigid alignment rather than flexible alignment. Conformational flexibility of the ligand is accounted for by introducing a preliminary step, in which a set of energy-minimized conformers for the query ligand are generated with the RDConf ({% cite rdconf %}) tool (using the RDKit ({% cite landrum2013rdkit %}) toolkit). 

> <hands-on-title>Hands-on 6: Low-energy ligand conformer search</hands-on-title>
>
> 1. {% tool [RDConf: Low-energy ligand conformer search](toolshed.g2.bx.psu.edu/repos/bgruening/rdconf/rdconf/2020.03.4+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `staurosporine_hydrated` (from Hands-on 4)
>    - *"Maximum number of conformers to generate per molecule"*: `100`
>
> 2. Rename the output to `staurosporine_3D_conformers`.
>
>    {% snippet faqs/galaxy/datasets_rename.md name = "staurosporine_3D_conformers" %}
>
>    > <comment-title>RDConf</comment-title>
>    >
>    > It is recommended to use the default settings, except for the number of conformers which should be changed to 100. As a rule of thumb, a threshold of 100 conformers appropriately represents the conformational flexibility of a compound with less than 10 rotatable bonds. The output SDF (structure data file) format encodes three-dimensional atomic coordinates of each conformer, separated by lines containing four dollar signs (`$$$$`).
>    {: .comment}
>
{: .hands_on}

> <question-title></question-title>
>
> Have a look at the contents of the created collection `staurosporine_3D_conformers`. Why were less than 100 conformers were generated for staurosporine?
>
> > <solution-title></solution-title>
> >
> > Staurosporine is a fused 8-ring system with only two rotatable bonds, due to its planar aromatic 5-ring indolocarbozole scaffold which confers a high structural rigidity upon the compound, i.e. it exists in relatively few energetically distinct 3D conformations.
> >
> {: .solution}
>
{: .question}


# Pharmacophore alignment

In this step, the ligand conformer dataset (SDF format) is converted on-the-fly to a pharmacophore dataset (PHAR format) and simultaneously aligned to the individual pharmacophores of the ePharmaLib dataset in a batch mode with Align-it ({% cite Taminau2008 %}). The pharmacophoric alignments and thus the predicted targets are ranked in terms of a scoring metric: `Tversky index` = [0,1]. The higher the Tversky index, the higher the likelihood of the predicted protein–ligand interaction.

> <hands-on-title>Hands-on 7: Pharmacophore alignment</hands-on-title>
>
> 1. {% tool [Pharmacophore alignment](toolshed.g2.bx.psu.edu/repos/bgruening/align_it/ctb_alignit/1.0.4+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Defines the database of molecules that will be used to screen"*: `staurosporine_3D_conformers` (from Hands-on 7)
>    - {% icon param-file %} *"Reference molecule"*: `ePharmaLib_PLAF_split` (from Hands-on 5)
>    - *"No normal information is included during the alignment"*: `Yes`
>    - *"Disable the use of hybrid pharmacophore points"*: `Yes`
>    - *"Only structures with a score larger than this cutoff will be written to the files"*: `0.0`
>    - *"Maximum number of best scoring structures to write to the files"*: `1`
>    - *"This option defines the used scoring scheme"*: `TVERSKY_REF`
>
{: .hands_on}

# Post-processing

The above pharmacophore alignment produces three types of outputs: the aligned pharmacophores (PHAR format), aligned structures (SMI format), and alignment scores (tabular format). Of these results, only the alignment scores are of interest and will be post-processed prior to analysis.

## Concatenating the pharmacophore alignment scores

The alignment score of the best ranked ligand conformer aligned against each ePharmaLib pharmacophore is stored in an individual file. In total, this job generates a collection of 138 output files which should be concatenated in a single file, for a better overview of the predictions.

> <hands-on-title>Hands-on 8: Concatenating the scores</hands-on-title>
>
> 1. {% tool [Concatenate datasets](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Datasets to concatenate"*: `scores` (from Hands-on 7)
>
> 2. Rename the output to `concatenated_scores`.
>
>    {% snippet faqs/galaxy/datasets_rename.md name = "concatenated_scores" %}
>
{: .hands_on}

## Ranking the predicted protein targets

The resulting `concatenated_scores` needs to be re-sorted according to the alignment metric, the Tversky index, i.e. the 10th column. The pharmacophores of the ePharmaLib dataset were labeled according to the following three-component code *PDBID-hetID-UniprotEntryName*. The contents of the `concatenated_scores` are as follows:

	------    ---------------------------------------------------------------------
	column    Content
	------    ---------------------------------------------------------------------
	     1    Id of the reference structure
	     2    Maximum volume of the reference structure
	     3    Id of the database structure
	     4    Maximum volume of the database structure
	     5    Maximum volume overlap of the two structures
	     6    Overlap between pharmacophore and exclusion spheres in the reference
	     7    Corrected volume overlap between database pharmacophore and reference
	     8    Number of pharmacophore points in the processed pharmacophore
	     9    TANIMOTO score
	    10    TVERSKY_REF score
	    11    TVERSKY_DB score
	------    --------------------------------------------------------------------- 

> <hands-on-title>Hands-on 9: Sort Dataset</hands-on-title>
>
> 1. {% tool [Sort](sort1) %} with the following parameters:
>    - {% icon param-file %} *"Sort Dataset"*: `concatenated_scores` (from Hands-on 8)
>    - *"on column"*: `c10`
>
> 2. Rename the output to `final_target_prediction_scores`.
>
>    {% snippet faqs/galaxy/datasets_rename.md name = "final_target_prediction_scores" %}
>
> 3. You can view the contents of the collection `final_target_prediction_scores` by pressing the **eye icon** (View data).
>
> The top-ranked protein of our target prediction experiment is *4mvf-STU-CDPK2_PLAFK* (Figures 1 & 2) with a Tversky index = 0.73. The general observation that can be made from this ranking of protein hits is the high self-retrieval rate of known targets, which demonstrates the high prediction accuracy of the method. The higher the Tversky index, the higher the likelihood of the predicted protein–ligand interaction; with a value of 0.5 corresponding to a 50% likelihood.
>
{: .hands_on}

> <question-title></question-title>
> 
> Why was a perfect pharmacophore alignment (Tversky index =  1) not achieved for the top-ranked protein target for which the cocrystallized ligand is staurosporine (*STU*)?
>
> > <solution-title></solution-title>
> >
> > A perfect pharmacophore alignment because a computational conformer generator (here RDConf in Hands-on 6) is unlikely to be able to reproduce a crystallographic (native) ligand pose with 100% accuracy.
> >
> {: .solution}
>
{: .question}

# One-step Zauberkugel workflow vs. multi-step workflow

For pharmacophore-based protein target prediction, you can choose to use Galaxy tools separately and in succession as described above, or alternatively use the one-step Zauberkugel workflow as described below (Figure 3).

> <hands-on-title>Upload the Zauberkugel workflow</hands-on-title>
>
> Upload the Zauberkugel workflow from the following URL:
>
> {% snippet faqs/galaxy/workflows_run_trs.md path="topics/computational-chemistry/tutorials/zauberkugel/workflows/main_workflow.ga" title="Zauberkugel Workflow" %}
>
> The Zauberkugel workflow requires only two inputs; the ligand structure file (SMI format) and the ePharmaLib dataset (PHAR format). The output of the prediction of human targets of staurosporine performed with the ePharmaLib human target subset (<https://zenodo.org/record/6055897>) and this workflow is available as a [Galaxy history](https://usegalaxy.eu/u/aurelien_moumbock/h/zauberkugel).
{: .hands_on}

![Snapshot of Zauberkugel workflow]({% link topics/computational-chemistry/images/zauberkugel.png %} "Zauberkugel — protein target prediction of a bioactive ligand with Align-it and ePharmaLib")

# Further analysis

To obtain a docking pose of a protein–ligand interaction predicted from pharmacophore-based protein target prediction, follow the [Protein–ligand docking]({% link topics/computational-chemistry/tutorials/cheminformatics/tutorial.md %}) Galaxy training.


# Conclusion

tools_json:
["Add hydrogen atoms", "toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_addh/openbabel_addh/3.1.1+galaxy1", "Split file", "toolshed.g2.bx.psu.edu/repos/bgruening/split_file_to_collection/split_file_to_collection/0.5.0", "RDConf: Low-energy ligand conformer search", "toolshed.g2.bx.psu.edu/repos/bgruening/rdconf/rdconf/2020.03.4+galaxy0", "Pharmacophore alignment", "toolshed.g2.bx.psu.edu/repos/bgruening/align_it/ctb_alignit/1.0.4+galaxy0", "Concatenate datasets", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.1", "Sort", "sort1"]


topic: computational-chemistry
tutorial: md-simulation-gromacs
title: Running molecular dynamics simulations using GROMACS
difficulty: intermediate
time_estimation: 2H
requirements_json: []
tags_json: []
content_hash: 82dbe41d5a05b5657d77bf39e9c23394
last_modified: 2026-05-15T22:28:10.768279
zenodo_link: https://zenodo.org/record/2598415
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/computational-chemistry/tutorials/md-simulation-gromacs/tutorial.html

content:
Molecular dynamics (MD) is a method to simulate molecular motion by iterative application of Newton's laws of motion. It is often applied to large biomolecules such as proteins or nucleic acids.

Multiple packages exist for performing MD simulations. One of the most popular is the open-source GROMACS, which is the subject of this tutorial. Other MD packages which are also wrapped in Galaxy are [NAMD]({% link topics/computational-chemistry/tutorials/md-simulation-namd/tutorial.md %}) and CHARMM (available in the [docker container](https://github.com/scientificomputing/BRIDGE)).

This is a introductory guide to using GROMACS ({% cite abraham15 %}) in Galaxy to prepare and perform molecular dynamics on a small protein. For the tutorial, we will perform our simulations on hen egg white lysozyme.

> <comment-title>More information</comment-title>
> This guide is based on the GROMACS tutorial provided by Justin Lemkul [here](http://www.mdtutorials.com/gmx/lysozyme/index.html) - please consult it if you are interested in a more detailed, technical guide to GROMACS.
{: .comment}

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}



# Process

Prior to performing simulations, a number of preparatory steps need to be executed.

The process can be divided into multiple stages:
 1. Setup (loading data, solvation i.e. addition of water and ions)
 2. Energy minimization of the protein
 3. Equilibration of the solvent around the protein (with two ensembles, NVT and NPT)
 4. Production simulation, which produces our trajectory.

The trajectory is a binary file that records the atomic coordinates at multiple time steps, and therefore shows the dynamic motion of the molecule. Using visualization software, we can display this trajectory as a film displaying the molecular motion of the protein. We will discuss each step making up this workflow in more detail.


# Getting data
To perform simulation, an initial PDB file is required. This should be 'cleaned' of solvent and any other non-protein atoms. Solvent will be re-added in a subsequent step.

A prepared file is available via Zenodo. Alternatively, you can prepare the file yourself. Download a PDB structure file from the [Protein Data Bank](https://www.rcsb.org/) and remove the unwanted atoms using the grep text processing tool. This simply removes the lines in the PDB file that refer to the unwanted atoms.


> <hands-on-title>Upload an initial structure</hands-on-title>
> 1. First of all, create a new history and give it a name.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Use the {% tool [Get PDB](toolshed.g2.bx.psu.edu/repos/bgruening/get_pdb/get_pdb/0.1.0) %} tool to download a PDB file for simulation:
>    - *"PDB accession code"*: `1AKI`
>
> 3. Use the {% tool [Search in textfiles (grep)](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/1.1.1) %} text processing tool to remove all lines that refer to non-protein atoms.
>    - *"Select lines from"*: uploaded PDB file
>    - *"that"*: `Don't Match`
>    - *"Regular Expression"*: `HETATM`
>
{: .hands_on}

> <comment-title>Alternative upload</comment-title>
> As an alternative option, if you prefer to upload the cleaned file directly from Zenodo, you can do so with the following link:
> ```
> https://zenodo.org/record/2598415/files/1AKI_clean.pdb
> ```
>
> {% snippet faqs/galaxy/datasets_import_via_link.md %}
{: .comment}

> <details-title>Background: What is the PDB (Protein Data Bank) and format?</details-title>
>
> The Protein Data Bank (PDB) format contains atomic coordinates of biomolecules and provides a standard representation for macromolecular structure data derived from X-ray diffraction and NMR studies. Each structure is stored under a four-letter accession code. For example, the PDB file we will use is assigned the code [1AKI](https://www.rcsb.org/pdb/explore/explore.do?structureId=1AKI).
>
> More resources:
>
>  -  Multiple structures are stored and can be queried at [https://www.rcsb.org/](https://www.rcsb.org/)
>  - Documentation describing the PDB file format is available from the wwPDB at [http://www.wwpdb.org/documentation/file-format.php](http://www.wwpdb.org/documentation/file-format.php).
{: .details}

## Lysozyme
The protein we will look at in this tutorial is hen egg white [lysozyme](https://en.wikipedia.org/wiki/Lysozyme), a widely studied enzyme which is capable of breaking down the polysaccharides of many bacterial cell walls. It is a small (129 residues), highly stable globular protein, which makes it ideal for our purposes.

![Structure of lysozyme openly available from https://commons.wikimedia.org/wiki/File:Lysozyme.png]({% link topics/computational-chemistry/images/Lysozyme.png %} "Structure of lysozyme")

# Setup

The **GROMACS initial setup** {% icon tool %} tool uses the PDB input to create three files which will be required for MD simulation.

Firstly, a topology for the protein structure is prepared. The topology file contains all the information required to describe the molecule for the purposes of simulation - atom masses, bond lengths and angles, charges. Note that this automatic construction of a topology is only possible if the building blocks of the molecules (i.e. amino acids in the case of a protein) are precalculated for the given force field. A force field and water model must be selected for topology calculation. Multiple choices are available for each; we will use the OPLS/AA force field and SPC/E water model. Secondly, a GRO structure file is created, storing the structure of the protein. Finally, a 'position restraint file' is created which will be used for NVT/NPT equilibration. We will return to this later.

In summary, the initial setup tool will:
- create a 'topology' file
- convert a PDB protein structure into a GRO file, with the structure centered in a simulation box (unit cell)
- create a position restraint file

After these files have been generated, a further step is required to define a simulation box (unit cell) in which the simulation can take place. This can be done with the **GROMACS structure configuration** {% icon tool %} tool. It also defines the unit cell 'box', centered on the structure. Options include box dimensions and shape; here, while a cuboidal box may be most intuitive, rhombic dodecahedron is the most efficient option, as it can contain the protein using the smallest volume, thus reducing the simulation resources devoted to the solvent.

> <hands-on-title>perform initial processing</hands-on-title>
>
> 1. Run {% tool [GROMACS initial setup](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_setup/gmx_setup/2020.4+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"PDB input file"*: `1AKI_clean.pdb` (Input dataset)
>    - *"Water model"*: `SPC/E`
>    - *"Force field"*: `OPLS/AA`
>    - *"Ignore hydrogens"*: `No`
>    - *"Generate detailed log"*: `Yes`
>
> > <question-title></question-title>
> >
> > Why is it necessary to provide an input structure containing no non-protein molecules?
> >
> > > <solution-title></solution-title>
> > > Automatic topology construction only succeeds if the components of the structure are recognized. For example, providing a structure of a protein in complex with a non-protein ligand or cofactor will result in an error.
> > {: .solution}
> {: .question}
>
> 2. Run {% tool [GROMACS structure configuration](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_editconf/gmx_editconf/2020.4+galaxy0) %} with the following parameters:
>    - *"Input structure"*: GRO output from initial setup tool
>    - *"Output format"*: `GRO file`
>    - *"Configure box?"*: `Yes`
>    - *"Box dimensions in nanometers"*: `1.0`
>    - *"Box type"*: `Rectangular box with all sides equal`
>    - *"Generate detailed log"*: `Yes`
{: .hands_on}


# Solvation

The next stage is protein solvation, performed using **GROMACS solvation and adding ions** {% icon tool %}. Water molecules are added to the structure and topology files to fill the unit cell. At this stage sodium or chloride ions are also automatically added to neutralize the charge of the system. In our case, as lysozyme has a charge of +8, 8 chloride anions are added.

![Solvated protein]({% link topics/computational-chemistry/images/solvated_protein.png %} "Solvated protein in a cubic unit cell")

> <hands-on-title>solvation</hands-on-title>
>
> {% tool [GROMACS solvation and adding ions](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_solvate/gmx_solvate/2020.4+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"GRO structure file"*: GRO structure file produced by the structure configuration tool
>    - {% icon param-file %} *"Topology (TOP) file"*: Topology produced by setup
>    - *"Water model for solvation"*: `SPC`
>    - *"Add ions to neutralise system?"*: `Yes, add ions`
>    - *"Specify salt concentration (sodium chloride) to add, in mol/liter"*: `0`
>    - *"Generate detailed log"*: `Yes`
>
{: .hands_on}

# Energy minimization

To remove any steric clashes or unusual geometry which would artificially raise the energy of the system, we must relax the structure by running an energy minimization (EM) algorithm.

Here, and in the later steps, two options are presented under 'Parameter input'. Firstly, the default setting, which we will use for this tutorial, requires options to be selected through the Galaxy interface. Alternatively, you can choose to upload an MDP (molecular dynamics parameters) file to define the simulation parameters. Using your own MDP file will allow greater customization, as not all parameters are implemented in Galaxy (yet); however, it requires a more advanced knowledge of GROMACS. Description of all parameters can be found [here](http://manual.gromacs.org/documentation/2018/user-guide/mdp-options.html).

> <hands-on-title>energy minimization</hands-on-title>
>
> {% tool [GROMACS energy minimization](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_em/gmx_em/2020.4+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"GRO structure file"*: GRO structure file produced by solvation tool
>    - {% icon param-file %} *"Topology (TOP) file"*: Topology
>    - *"Parameter input"*: `Use default (partially customisable) setting`
>    - *"Choice of integrator"*: `Steepest descent algorithm` (most common choice for EM)
>    - *"Neighbor searching"*: `Generate a pair list with buffering` (the '[Verlet scheme](https://en.wikipedia.org/wiki/Verlet_list)')
>    - *"Electrostatics"*: `Fast smooth Particle-Mesh Ewald (SPME) electrostatics`
>    - *"Distance for the Coulomb cut-off"*: `1.0`
>    - *"Cut-off distance for the short-range neighbor list"*: `1.0` (but irrelevant as we are using the Verlet scheme)
>    - *"Short range van der Waals cutoff"*: `1.0`
>    - *"Number of steps for the MD simulation"*: `50000`
>    - *"EM tolerance"*: `1000`
>    - *"Maximum step size"*: `0.01`
>    - *"Generate detailed log"*: `Yes`
>
{: .hands_on}

# Equilibration

At this point equilibration of the solvent around the solute (i.e. the protein) is necessary. This is performed in two stages: equilibration under an NVT ensemble, followed by an NPT ensemble. Use of the NVT ensemble entails maintaining constant **n**umber of particles, **v**olume and **t**emperature, while the NPT ensemble maintains constant **n**umber of particles, **p**ressure and **t**emperature. (The NVT ensemble is also known as the isothermal-isochoric ensemble, while the NPT ensemble is also known as the isothermal-isobaric ensemble).

During the first equilibration step (NVT), the protein must be held in place while the solvent is allowed to move freely around it. This is achieved using the position restraint file we created in system setup. When we specify this restraint, protein movement is not totally forbidden, but is energetically punished. During the second NPT step, we remove the restraints.

## NVT equilibration
Firstly, we perform equilibration using classical NVT dynamics.

> <hands-on-title>NVT dynamics</hands-on-title>
>
> {% tool [GROMACS simulation](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_sim/gmx_sim/2020.4+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"GRO structure file"*: GRO structure file
>    - {% icon param-file %} *"Topology (TOP) file"*: Topology
>    - *"Use a checkpoint (CPT) file"*: `No CPT input`
>    - *"Produce a checkpoint (CPT) file"*: `Produce CPT output`
>    - *"Apply position restraints"*: `Apply position restraints`
>    - {% icon param-file %} *"Position restraint file"*: Position restraint file produced by 'Setup' tool.
>    - *"Ensemble"*: `Isothermal-isochoric ensemble (NVT).`
>    - *"Trajectory output"*: `Return no trajectory output` (we are not interested in how the system evolves to the equilibrated state, merely the final structure)
>    - *"Structure output"*: `Return .gro file`
>    - *"Parameter input"*: `Use default (partially customisable) setting`
>    - *"Choice of integrator"*: `A leap-frog algorithm for integrating Newton’s equations of motion` (A basic leap-frog integrator)
>    - *"Bond constraints"*: `Bonds with H-atoms` (bonds involving H are constrained)
>    - *"Neighbor searching"*: `Generate a pair list with buffering` (the '[Verlet scheme](https://en.wikipedia.org/wiki/Verlet_list)')
>    - *"Electrostatics"*: `Fast smooth Particle-Mesh Ewald (SPME) electrostatics`
>    - *"Temperature"*: `300`
>    - *"Step length in ps"*: `0.002`
>    - *"Number of steps that elapse between saving data points (velocities, forces, energies)"*: `5000`
>    - *"Distance for the Coulomb cut-off"*: `1.0`
>    - *"Cut-off distance for the short-range neighbor list"*: `1.0` (but irrelevant as we are using the Verlet scheme)
>    - *"Short range van der Waals cutoff"*: `1.0`
>    - *"Number of steps for the NVT simulation"*: `50000`
>    - *"Generate detailed log"*: `Yes`
{: .hands_on}

## NPT equilibration
Having stabilized the temperature of the system with NVT equilibration, we also need to stabilize the pressure of the system. We therefore equilibrate again using the NPT (constant number of particles, pressure, temperature) ensemble.

Note that we can continue where the last simulation left off (with new parameters) by using the checkpoint (CPT) file saved at the end of the NVT simulation.

> <hands-on-title>NPT dynamics</hands-on-title>
>
> {% tool [GROMACS simulation](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_sim/gmx_sim/2020.4+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"GRO structure file"*: GRO structure file
>    - {% icon param-file %} *"Topology (TOP) file"*: Topology
>    - *"Use a checkpoint (CPT) file"*: `Continue simulation from a CPT file.`
>    - {% icon param-file %} *"Checkpoint (CPT) file"*: Checkpoint file produced by NVT equilibration
>    - *"Produce a checkpoint (CPT) file"*: `Produce CPT output`
>    - *"Apply position restraints"*: `No position restraints`
>    - {% icon param-file %} *"Position restraint file"*: None
>    - *"Ensemble"*: `Isothermal-isobaric ensemble (NPT).`
>    - *"Trajectory output"*: `Return no trajectory output`
>    - *"Structure output"*: `Return .gro file`
>    - *"Parameter input"*: `Use default (partially customisable) setting`
>    - *"Choice of integrator"*: `A leap-frog algorithm for integrating Newton’s equations of motion` (A basic leap-frog integrator)
>    - *"Bond constraints"*: `Bonds with H-atoms` (bonds involving H are constrained)
>    - *"Neighbor searching"*: `Generate a pair list with buffering` (the '[Verlet scheme](https://en.wikipedia.org/wiki/Verlet_list)')
>    - *"Electrostatics"*: `Fast smooth Particle-Mesh Ewald (SPME) electrostatics`
>    - *"Temperature"*: `300`
>    - *"Step length in ps"*: `0.002`
>    - *"Number of steps that elapse between saving data points (velocities, forces, energies)"*: `5000`
>    - *"Distance for the Coulomb cut-off"*: `1.0`
>    - *"Cut-off distance for the short-range neighbor list"*: `1.0` (but irrelevant as we are using the Verlet scheme)
>    - *"Short range van der Waals cutoff"*: `1.0`
>    - *"Number of steps for the NPT simulation"*: `50000`
>    - *"Generate detailed log"*: `Yes`
{: .hands_on}

> <question-title></question-title>
>
> Why is the position of the protein restrained during equilibration?
>
> > <solution-title></solution-title>
> > The purpose of equilibration is to stabilize the temperature and pressure of the system; these are overwhelmingly dependent on the solvent. Structural changes in the protein are an additional complicating variable, which can more simply be removed by restraining the protein.
> {: .solution}
{: .question}


# Production simulation
Now that equilibration is complete, we can release the position restraints. We are now finally ready to perform a production MD simulation.

> <hands-on-title>Production simulation</hands-on-title>
>
> 1. {% tool [GROMACS simulation](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_sim/gmx_sim/2020.4+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"GRO structure file"*: GRO structure file
>    - {% icon param-file %} *"Topology (TOP) file"*: Topology
>    - *"Use a checkpoint (CPT) file"*: `Continue simulation from a CPT file.`
>    - {% icon param-file %} *"Checkpoint (CPT) file"*: Checkpoint file produced by NPT equilibration
>    - *"Produce a checkpoint (CPT) file"*: `No CPT output`
>    - *"Apply position restraints"*: `No position restraints`
>    - *"Ensemble"*: `Isothermal-isobaric ensemble (NPT).`
>    - *"Trajectory output"*: `Return .xtc file (reduced precision)` (this time, we save the trajectory)
>    - *"Structure output"*: `Return .gro file`
>    - *"Parameter input"*: `Use default (partially customisable) setting`
>    - *"Choice of integrator"*: `A leap-frog algorithm for integrating Newton’s equations of motion` (A basic leap-frog integrator)
>    - *"Bond constraints"*: `Bonds with H-atoms` (bonds involving H are constrained)
>    - *"Neighbor searching"*: `Generate a pair list with buffering` (the '[Verlet scheme](https://en.wikipedia.org/wiki/Verlet_list)')
>    - *"Electrostatics"*: `Fast smooth Particle-Mesh Ewald (SPME) electrostatics`
>    - *"Temperature"*: `300`
>    - *"Step length in ps"*: `0.002`
>    - *"Number of steps that elapse between saving data points (velocities, forces, energies)"*: `5000`
>    - *"Distance for the Coulomb cut-off"*: `1.0`
>    - *"Cut-off distance for the short-range neighbor list"*: `1.0` (but irrelevant as we are using the Verlet scheme)
>    - *"Short range van der Waals cutoff"*: `1.0`
>    - *"Number of steps for the simulation"*: `500000`
>    - *"Generate detailed log"*: `Yes`
{: .hands_on}

# Workflow

A GROMACS workflow is provided for this tutorial. Overall, the workflow takes a PDB (Protein Data Bank) structure file as input and returns a MD trajectory.

![GROMACS workflow]({% link topics/computational-chemistry/images/workflow_gromacs.png %} "The basic GROMACS workflow")

# Conclusion


After completing the steps, or running the workflow, we have successfully produced a trajectory (the xtc file) which describes the atomic motion of the system. This can be viewed using molecular visualization software or analysed further; please visit the visualization and [analysis]({% link topics/computational-chemistry/tutorials/analysis-md-simulations/tutorial.md %}) tutorials for more information.

![Trajectory]({% link topics/computational-chemistry/images/traj.gif %} "Trajectory produced using the GROMACS workflow, visualized with the NGL viewer")

tools_json:
["Get PDB", "toolshed.g2.bx.psu.edu/repos/bgruening/get_pdb/get_pdb/0.1.0", "Search in textfiles (grep)", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/1.1.1", "GROMACS initial setup", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_setup/gmx_setup/2020.4+galaxy0", "GROMACS structure configuration", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_editconf/gmx_editconf/2020.4+galaxy0", "GROMACS solvation and adding ions", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_solvate/gmx_solvate/2020.4+galaxy1", "GROMACS energy minimization", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_em/gmx_em/2020.4+galaxy0", "GROMACS simulation", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_sim/gmx_sim/2020.4+galaxy1"]


topic: computational-chemistry
tutorial: cheminformatics
title: Protein-ligand docking
difficulty: intermediate
time_estimation: 3H
requirements_json: []
tags_json: []
content_hash: beb50048d0d8c33427c80fa17633137a
last_modified: 2026-05-27T09:01:47.900393
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/computational-chemistry/tutorials/cheminformatics/tutorial.html

content:
Cheminformatics is the use of computational techniques and information about molecules to solve problems in chemistry. This involves a number of steps: retrieving data on chemical compounds, sorting data for properties which are of interest, and extracting new information. This tutorial will provide a brief overview of all of these, centered around protein-ligand docking, a molecular modelling technique. The purpose of protein-ligand docking is to find the optimal binding between a small molecule (ligand) and a protein. It is generally applied to the drug discovery and development process with the aim of finding a potential drug candidate. First, a target protein is identified. This protein is usually linked to a disease and is known to bind small molecules. Second, a 'library' of possible ligands is assembled. Ligands are small molecules that bind to a protein and may interfere with protein function. Each of the compounds in the library is then 'docked' into the protein to find the optimal binding position and energy.

Docking is a form of molecular modelling, but several simplifications are made in comparison to methods such as molecular dynamics. Most significantly, the receptor is generally considered to be rigid, with covalent bond lengths and angles held constant. Charges and protonation states are also not permitted to change. While these approximations reduce accuracy to some extent, they increase computational speed, which is necessary to screen a large compound library in a realistic amount of time.

In this tutorial, you will perform docking of ligands into the N-terminus of Hsp90 (heat shock protein 90). The tools used for docking are based on the open-source software [AutoDock Vina](http://vina.scripps.edu/) ({% cite Trott2009 %}).

> <details-title>Biological background</details-title>
>
> The 90 kDa heat shock protein (Hsp90) is a chaperone protein responsible for catalyzing the conversion of a wide variety of proteins to a functional form; examples of the Hsp90 clientele, which totals several hundred proteins, include nuclear steroid hormone receptors and protein kinases. The mechanism by which Hsp90 acts varies between clients, as does the client binding site; the process is dependent on post-translational modifications of Hsp90 and the identity of co-chaperones which bind and regulate the conformational cycle. 
>
>
> Due to its vital biochemical role as a chaperone protein involved in facilitating the folding of many client proteins, Hsp90 is an attractive pharmaceutical target. In particular, as protein folding is a potential bottleneck to slow cellular reproduction and growth, blocking Hsp90 function using inhibitors which bind tightly to the ATP binding site could assist in treating cancer; for example, the antibiotic geldanamycin and its analogs are under investigation as possible anti-tumor agents. 
>
> See {% cite Pearl2006 %} for a review on the structure and mechanism of the Hsp90 protein. Alternatively, read more at [PDB-101](https://pdb101.rcsb.org/motm/108) and [Wikipedia](https://en.wikipedia.org/wiki/Hsp90).
{: .details}



> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Download data

For this exercise, we need two datasets: a protein structure and a library of compounds. We will download the former directly from the Protein Data Bank; the latter will be created by searching the ChEMBL database ({% cite Gaulton2016 %}).

## Get data

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Search Galaxy for the {% tool [Get PDB](toolshed.g2.bx.psu.edu/repos/bgruening/get_pdb/get_pdb/0.1.0) %} tool. Request the accession code ```2brc```.
> 3. Rename the dataset to 'Hsp90 structure'
> 4. Check that the datatype is correct (PDB file).
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
>
{: .hands_on}

![Hsp90 N-terminus structure]({% link topics/computational-chemistry/images/2brc.png %} "Structure of Hsp90 N-terminus, as recorded on the PDB. Visualization produced using VMD  ({% cite Humphrey1996 %}).")

# Separating protein and ligand structures

You can view the contents of the downloaded PDB file by pressing the 'View data' icon in the history pane. After the header section (about 500 lines), the atoms of the protein and their coordinates are listed. The lines begin with ```ATOM```. At the end of the file, the atomic coordinates of the ligand and the solvent water molecules are also listed, labelled ```HETATM```. We will use the grep tool to separate these molecules into separate files, and then convert the ligand file into SDF/MOL format using the 'Compound conversion' tool, which is based on OpenBabel, an open-source library for analyzing chemical data ({% cite OBoyle2011 %}).

> <details-title>What is grep?</details-title>
>
> Grep is a command-line tool for searching text files for lines which match a search query.
>
> There is a Galaxy tool available based on grep, which we will apply to the downloaded PDB tool.
>
{: .details}

> <hands-on-title>Separate protein and ligand</hands-on-title>
>
> 1. {% tool [Search in textfiles (grep)](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/9.5+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Select lines from"*: Downloaded PDB file 'Hsp90 structure'
>    - {% icon param-file %} *"that"*: `Don't match`
>    - {% icon param-file %} *"Regular Expression"*: `HETATM`
>    - All other parameters can be left as their defaults. 
>    - Rename the dataset **'Protein (PDB)'**.
>
>    The result is a file with all non-protein (`HETATM`) atoms removed.
> 2. {% tool [Search in textfiles (grep)](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/9.5+galaxy0) %} with the following parameters. Here, we use grep again to produce a file with only non-protein atoms.
>    - {% icon param-file %} *"Select lines from"*: Downloaded PDB file 'Hsp90 structure' 
>    - {% icon param-file %} *"that"*: `Match`
>    - {% icon param-file %} *"Regular Expression"*: `CT5` (the name of the ligand in the PDB file)
>    - All other parameters can be left as their defaults. 
>    - Rename the dataset **'Ligand (PDB)'**.
>
>    This produces a file which only contains ligand atoms.
> 3. {% tool [Compound conversion](toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy1) %} - interconvert between various chemistry and molecular modeling data files with the following parameters:
>    - {% icon param-file %} *"Molecular input file"*: Ligand PDB file created in step 2.
>    - {% icon param-file %} *"Output format"*: `MDL MOL format (sdf, mol)`
>    - {% icon param-file %} *"Add hydrogens appropriate for pH"*: `7.4`
>    - All other parameters can be left as their defaults. 
>    - Change the datatype to 'mol' and rename the dataset **'Ligand (MOL)'**.
>
>    Applying this tool will generate a representation of the structure of the ligand in MOL format.
>
{: .hands_on}

At this stage, separate protein and ligand files have been created. Next, we want to generate a compound library we can use for docking.


## Creating and processing the compound library

In this step we will create a compound library, using data from the ChEMBL database.

> <details-title>What chemical databases are available?</details-title>
>
> Multiple databases are available online which provide access to chemical information, e.g. chemical structures, reactions, or literature. In this tutorial, we use a tool which searches the [ChEMBL](https://www.ebi.ac.uk/chembl/) database. There are also Galaxy tools available for downloading data from [PubChem](https://pubchem.ncbi.nlm.nih.gov/). 
>
> Other notable databases include [Drugbank](https://www.drugbank.ca/) and [ChemSpider](http://www.chemspider.com/).
>
{: .details}

We will generate our compound library by searching ChEMBL for compounds which have a similar structure to the ligand in the PDB file we downloaded in the first step. There is a Galaxy tool for accessing ChEMBL which requires data input in SMILES format; thus, the first step is to convert the 'Ligand' PDB file to a SMILES file. Then the search is performed, returning a SMILES file. For docking, we would like to convert to SDF format, which we can do once again using the 'Compound conversion' tool.

> <hands-on-title>Generate compound library</hands-on-title>
>
> 1. {% tool [Compound conversion](toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy1) %} - interconvert between various chemistry and molecular modeling data files with the following parameters:
>    - {% icon param-file %} *"Molecular input file"*: 'Ligand' PDB file
>    - {% icon param-file %} *"Output format"*: `SMILES format (SMI)`
>    - Leave all other options as default.
> 2. Rename the output of the 'compound conversion' step to **'Ligand SMILES'**.
> 3. {% tool [Search ChEMBL database](toolshed.g2.bx.psu.edu/repos/bgruening/chembl/chembl/0.10.1+galaxy4) %} with the following parameters:
>    - {% icon param-file %} *"SMILES input type"*: File
>    - {% icon param-file %} *"Input file"*: 'Ligand SMILES' file
>    - {% icon param-file %} *"Search type"*: `Similarity`
>    - {% icon param-file %} *"Tanimoto cutoff score"*: `40`
>    - {% icon param-file %} *"Filter for Lipinski's Rule of Five"*: `Yes`
>    - All other parameters can be left as their defaults.
>
>    > <question-title></question-title>
>    >
>    > Why are compounds filtered for Lipinski's Rule of Five?
>    >
>    > > <solution-title></solution-title>
>    > > Lipinski's rule of five is an empirical rule which can be used to determine the 'druglikeness' of a molecule. The rule consists of four criteria which relate to the pharmacokinetics of the molecule. The rule is discussed in ({% cite Lipinski2004 %}), or you can also read more on [Wikipedia](https://en.wikipedia.org/wiki/Lipinski%27s_rule_of_five).
>    > {: .solution}
>    {: .question}
>
>    Optional: experiment with different combinations of options - adding different filters, adjusting the Tanimoto coefficient.
> 4. Check that the datatype is correct (smi). This step is essential, as Galaxy does not automatically detect the datatype for SMILES files.
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
> 5. Rename dataset **'Compound library'**.
>
{: .hands_on}

> <tip-title>Problems using the ChEMBL tool?</tip-title>
> A number of users encounter issues with the ChEMBL tool - sometimes the tool fails, or the output is returned successfully but is empty. If this happens to you, try the following:
> 1.  Rerun the tool - if a transient error on the ChEMBL server was at fault, this might be enough to fix it.
> 1.  Try modifying some of the parameters. For example, reducing the Tanimoto coefficient should increase the number of compounds returned.
> 1.  If all else fails, you can use the following list of SMILES:
>
>     ```
>     Cc1n[nH]c(c2ccc(O)c(Cl)c2)c1c3ccc4OCCOc4c3	CHEMBL187670
>     COc1ccc(cc1)c2c(C)[nH]nc2c3ccc(O)cc3O	CHEMBL192894
>     COc1ccc(c(O)c1)c2onc(C)c2c3ccc4OCCOc4c3	CHEMBL1541585
>     CCOc1ccc(c(O)c1)c2[nH]nc(C)c2c3ccccc3OC	CHEMBL1504505
>     CN(CCc1c(C)n[nH]c1C)Cc2cn(C)nc2c3ccc4OCCOc4c3	CHEMBL1560480
>     COc1ccc(c(O)c1)c2[nH]nc(C)c2c3ccc4OCCCOc4c3	CHEMBL362893
>     CCCc1c(OCCCOc2cc(O)c(cc2CC)c3cc[nH]n3)ccc4CCC(Oc14)C(=O)O	CHEMBL81401
>     Cc1cccc(n1)c2[nH]nc(C)c2c3ccnc4ccccc34	CHEMBL129153
>     COc1ccc(cc1OC)c2c(C)n[nH]c2c3ccc(O)cc3O	CHEMBL1595327
>     COc1ccc(c(O)c1)c2noc(C)c2c3ccc4ccccc4n3	CHEMBL1486235
>     CCc1cc(c(O)cc1O)c2[nH]nc(C)c2c3ccc4OCCOc4c3	CHEMBL399530
>     Cc1[nH]nc(c2cc(Cl)c(O)cc2O)c1c3ccc4OCCOc4c3	CHEMBL191074
>     COc1ccc(c(O)c1)c2[nH]ncc2c3ccc4OCCOc4c3	CHEMBL1415374
>     Cc1n[nH]c(c2cc(Cl)ccc2O)c1c3ccc4OCCOc4c3	CHEMBL187678
>     Cc1noc(c2ccc(O)cc2O)c1c3ccc4OCCOc4c3	CHEMBL582320
>     CCOC(=O)c1oc(cc1)c2c(C)[nH]nc2c3cc(CC)c(O)cc3O	CHEMBL3932805
>     NC(=O)c1ccc2[nH]nc(c3ccc4OCCOc4c3)c2c1	CHEMBL3900406
>     COc1ccc(cc1OC)c2cc([nH]n2)c3c(O)c(OC)c4occc4c3OC	CHEMBL1351838
>     CCCc1cc(c(O)cc1OC)c2[nH]ncc2c3ccc4OCCCOc4c3	CHEMBL1443258
>     Oc1cc(O)c(cc1Cl)c2[nH]ncc2c3ccc4OCCOc4c3	CHEMBL191228
>     CCc1cc(c(O)cc1O)c2n[nH]cc2c3ccc4OCCOc4c3	CHEMBL3187010
>     Oc1ccccc1c2cc([nH]n2)c3ccc4OCCOc4c3	CHEMBL1567097
>     CCCc1cc(c(O)cc1O)c2[nH]ncc2c3ccc4OCCCOc4c3	CHEMBL1578064
>     COc1ccc(c(O)c1)c2[nH]nc(C)c2c3ccc(OC)c(OC)c3	CHEMBL1335688
>     COc1ccc(cc1)c2c(N)n[nH]c2c3cc(OC)c4OCCOc4c3	CHEMBL2408971
>     CCC(C)c1cc(c(O)cc1O)c2[nH]ncc2c3ccc4OCOc4c3	CHEMBL187674
>     Cc1noc(c2ccc(O)cc2O)c1c3ccc4OCCCOc4c3	CHEMBL587334
>     OCc1cn(nc1c2ccc3OCCOc3c2)c4ccccc4	CHEMBL1549407
>     Oc1ccc(c(O)c1)c2[nH]ncc2c3cccc4cccnc34	CHEMBL1305951
>     Cc1n[nH]c(c2ccc(O)cc2O)c1c3ccc4OCCOc4c3	CHEMBL188965
>     Cc1[nH]nc(c2ccc3OCC(=O)Nc3c2)c1c4ccc(F)cc4	CHEMBL3337723
>     CCOc1ccc(c(O)c1)c2n[nH]c(C)c2c3ccc(OC)cc3	CHEMBL1698243
>     COc1ccc(cc1)c2cc([nH]n2)c3c(O)c(OC)c4occc4c3OC	CHEMBL1402615
>     CCc1cc(c(O)cc1O)c2[nH]ncc2c3ccc4OCOc4c3	CHEMBL1412538
>     CCOc1cc(O)c(cc1CC)c2nc(N)ncc2c3ccc4OCCOc4c3	CHEMBL547662
>     COc1ccc(cc1)c2c(N)onc2c3cc(OC)c4OCCOc4c3	CHEMBL3113121
>     CCCc1cc(c(O)cc1O)c2n[nH]cc2c3ccc4OCCOc4c3	CHEMBL3956397
>     CCC(C)c1cc(c(O)cc1O)c2[nH]nc(C)c2c3ccc4OCCOc4c3	CHEMBL190919
>     CCC(C)c1cc(c(O)cc1OC)c2[nH]ncc2c3ccc4OCCOc4c3	CHEMBL435501
>     Cn1cc(CNCc2c[nH]nc2c3ccc(F)cc3)c(n1)c4ccc5OCCOc5c4	CHEMBL1537178
>     Oc1ccc(F)cc1c2cc([nH]n2)c3ccc4OCCOc4c3	CHEMBL1451528
>     CCc1cc(c(O)cc1OCC(=O)O)c2n[nH]c(C)c2c3ccc4OCCCOc4c3	CHEMBL3952001
>     Cc1[nH]nc(c2ccc(O)c(O)c2O)c1c3ccc(Cl)cc3	CHEMBL1092945
>     COc1cc(cc(OC)c1OC)c2n[nH]nc2c3ccc4OCCOc4c3	CHEMBL3740841
>     Oc1ccc(c(O)c1)c2n[nH]cc2c3ccc4OCOc4c3	CHEMBL577176
>     ```
>
> Don't worry if you can't get it to work - successfully generating this list is a very minor part of the tutorial!
{: .tip}


There are some other tools available, which will not be used in this tutorial, which help to develop a more focused compound library. For example, the 'Natural product likeness calculator' and 'Drug-likeness' tools assign a score to compounds based on how similar they are to typical natural products and drugs respectively, which could then be used to filter the library. If you are interested, you can try testing them out on the library just generated.

> <tip-title>Generating a compound library</tip-title>
> If you try using this tutorial using your own data, you might encounter some issues. Important things to remember:
> * If you encounter an error, check the SMILES file only has a single column. Additional columns can be removed using the 'Cut' tool.
> * If the output file is empty, it may be that the ChEMBL database doesn't have any compounds similar to the input. Consider lowering the Tanimoto coefficient to 70 if this is the case and removing filters (including the Lipinski RO5 filter). If this doesn't help, you will have to use another source of chemical data (e.g. PubChem).
> * Finally, please remember this step is totally optional if you already have a list of compounds for docking (in SMILES or another format). In this case you can upload them to Galaxy and continue with the next step.
{: .tip}

> <details-title>What are SMILES and SDF formats?</details-title>
>
> SMILES and SD-files both represent chemical structures. A SMILES file represents the 2D structure of a molecule as a chemical graph. In other words, it states only the atoms and the connectivity between them. An example of a SMILES string (taken from the ligand in the PDB file) is `c1c2OCCOc2ccc1c1c(C)[nH]nc1c1cc(CC)c(O)cc1O`. For more information on how the notation works, please consult the [OpenSMILES specification](http://opensmiles.org/opensmiles.html) or the description provided by [Wikipedia](https://en.wikipedia.org/wiki/Simplified_molecular-input_line-entry_system). A more comprehensive alternative to the SMILES system is the International Chemical Identifier (InChI).
>
> Neither SMILES nor InChI format contain the three-dimensional structure of a molecule. By contrast, the SDF (structure data file) format encodes three-dimensional atomic coordinates of a structure, similar to a PDB file.
>
> In a previous step, we also generated a MOL file - this format is closely related to the SDF format. The difference is that MOL files can store only a single molecule, whereas SD-files can encode single or multiple molecules. Multiple molecules are separated by lines containing four dollar signs (`$$$$`).
>
> For docking, we need the three-dimensional coordinates of the ligand; thus, we want to convert from SMILES to SDF format.
{: .details}

# Prepare files for docking

A processing step now needs to be applied to the protein structure and the docking candidates - each of the structures needs to be converted to PDBQT format before using the AutoDock Vina docking tool.

Further, docking requires the coordinates of a binding site to be defined. Effectively, this defines a 'box' in which the docking software attempts to define an optimal binding site. In this case, we already know the location of the binding site, since the downloaded PDB structure contained a bound ligand. There is a tool in Galaxy which can be used to automatically create a configuration file for docking when ligand coordinates are already known.

> <hands-on-title>Generate PDBQT and config files for docking</hands-on-title>
>
> 1. {% tool [Prepare receptor](toolshed.g2.bx.psu.edu/repos/bgruening/autodock_vina_prepare_receptor/prepare_receptor/1.5.7+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Select a PDB file"*: 'Protein' PDB file.
>    - Rename to 'Prepared receptor'
> 2. {% tool [Compound conversion](toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy1) %} - interconvert between various chemistry and molecular modeling data files with the following parameters:
>    - {% icon param-file %} *"Molecular input file"*: 'Compound library' file.
>    - {% icon param-file %} *"Output format"*: `MDL MOL format (sdf,mol)`
>    - {% icon param-file %} *"Generate 3D coordinates"*: `Yes`
>    - {% icon param-file %} *"Add hydrogens appropriate for pH"*: `7.4`
>    - Leave all other options unchanged.
>    - Rename to 'Prepared ligands'
> 3. {% tool [Calculate the box parameters using RDKit](toolshed.g2.bx.psu.edu/repos/bgruening/autodock_vina_prepare_box/prepare_box/2021.03.5+galaxy0) %} for an AutoDock Vina job from a ligand or pocket input file (confounding box) with the following parameters:
>    - {% icon param-file %} *"Input ligand or pocket"*: `Ligand (MOL)` file.
>    - {% icon param-file %} *"x-axis buffer"*: `5`
>    - {% icon param-file %} *"y-axis buffer"*: `5`
>    - {% icon param-file %} *"z-axis buffer"*: `5`
>    - {% icon param-file %} *"Random seed"*: `1`
>    - Rename to 'Docking config file'.
{: .hands_on}

Perhaps you are interested in a system which does not have a ligand within the binding site (an apoprotein). In this case you need to run the fpocket tool to identify potential binding sites in the protein structure - take a look at the following (optional) section.


> <details-title>How to find the binding site of an apoprotein?</details-title>
>
> If the structure contains no ligand in complex with the protein (i.e. apoprotein), there is an additional step of identifying the binding site. Software is available for automatic detection of pockets which may be promising candidates for ligand binding sites. For example, let's try out the fpocket tool ({% cite LeGuilloux2009 %}) on the Hsp90 structure, imagining we don't have access to the file containing the ligand coordinates.
> > <hands-on-title>Finding binding sites using fpocket</hands-on-title>
> >
> > 1. {% tool [fpocket](toolshed.g2.bx.psu.edu/repos/bgruening/fpocket/fpocket/3.1.4.2+galaxy0) %} with the following parameters:
> >   - {% icon param-file %} *"Input file"*: `Protein (PDB)` file.
> >   - {% icon param-file %} *"Type of pocket to detect"*: `Small molecule binding sites`
> >   - {% icon param-file %} *"Output files"*: select `PDB files containing the atoms in contact with each pocket`, `Log file containing pocket properties`.
> >
> > 2. {% tool [Calculate the box parameters for an AutoDock Vina job](toolshed.g2.bx.psu.edu/repos/bgruening/autodock_vina_prepare_box/prepare_box/2021.03.4+galaxy0) %} with the following parameters:
> >    - {% icon param-file %} *"Input ligand or pocket"*: `pocket2` PDB file from the `Atoms in contact with each pocket` collection.
> >    - {% icon param-file %} *"x-axis buffer"*: `5`
> >    - {% icon param-file %} *"y-axis buffer"*: `5`
> >    - {% icon param-file %} *"z-axis buffer"*: `5`
> >    - {% icon param-file %} *"Exhaustiveness (optional)"*: `1`
> >    - {% icon param-file %} *"Random seed"*: `1`
> >    - Rename to 'Docking config file derived from pocket'.
> >
> >  The fpocket tool generates two different outputs: a `Pocket properties` log file containing details of all the pockets which fpocket found in the protein. The second output is a collection (a list) containing one PDB file for each of the pockets. Each of the PDB files contains only the atoms in contact with that particular pocket. Note that fpocket assigns a score to each pocket, but you should not assume that the top scoring one is the only one where compounds can bind! For example, the pocket where the ligand in the `2brc` PDB file binds is ranked as the second-best according to fpocket.
> {: .hands_on}
>
> You can compare the config file generated with the one generated from the ligand directly - if you picked the right pocket (`pocket2`) the coordinates should be pretty similar.
>
{: .details}


# Docking

Now that the protein and the ligand library have been correctly prepared and formatted, docking can be performed.

> <hands-on-title>Perform docking</hands-on-title>
>
> 1. {% tool [VINA Docking](toolshed.g2.bx.psu.edu/repos/bgruening/autodock_vina/docking/1.2.3+galaxy0) %} tool to perform protein-ligand docking with Autodock Vina with the following parameters:
>    - {% icon param-file %} *"Receptor"*: 'Protein receptor' file.
>    - {% icon param-file %} *"Ligands"*: 'Prepared ligands' file. 
>    - {% icon param-file %} *"Specify pH value for ligand protonation"*: `7.4` 
>    - {% icon param-file %} *"Specify parameters"*: 'Upload a config file to specify parameters' 
>    - {% icon param-file %} *"Box configuration"*: 'Docking config file'
>    - {% icon param-file %} *"Exhaustiveness"*: leave blank (it was specified in the previous step)
{: .hands_on}

The output consists of a collection, which contains an SDF output file for each ligand, containing multiple docking poses and scoring files for each of the ligands. We will now perform some processing on these files which extracts scores from the SD-files and selects the best score for each. 

# Optional: cheminformatics tools applied to the compound library

The ChemicalToolbox contains a large number of cheminformatics tools. This section will demonstrate some of the useful functionalities available. If you are just interested in docking, feel free to skip this section - or, just try out the tools which look particularly interesting.

(This section can also be completed while waiting for the docking, which can take some time to complete.)

## Visualization

It can be useful to visualize the compounds generated. There is a tool available for this in Galaxy based on OpenBabel.

> <hands-on-title>Visualization of chemical structures</hands-on-title>
>
> 1. {% tool [Visualisation](toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_svg_depiction/openbabel_svg_depiction/3.1.1+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"Molecular input file"*: Compound library
>    - {% icon param-file %} *"Embed molecule as CML"*: `No`
>    - {% icon param-file %} *"Draw all carbon atoms"*: `No`
>    - {% icon param-file %} *"Use thicker lines"*: `No`
>    - {% icon param-file %} *"Property to display under the molecule"*: `Molecule title`
>    - {% icon param-file %} *"Sort the displayed molecules by"*: `Molecular weight`
>    - {% icon param-file %} *"Format of the resultant picture"*: `SVG`
>
{: .hands_on}

This produces an SVG image of all the structures generated ordered by molecular weight.

![Image showing structures of compounds from ChEMBL]({% link topics/computational-chemistry/images/compound_library.png %} "Structures of the compounds from ChEMBL.")

### Calculation of fingerprints and clustering

In this step, we will group similar molecules together. A key tool in cheminformatics for measuring molecular similarity is fingerprinting, which entails extracting chemical properties of molecules and storing them as a bitstring. These bitstrings can be easily compared computationally, for example with a clustering method. The fingerprinting tools in Galaxy are based on the Chemfp tools ({% cite Dalke2013 %}).

Before clustering, let's label each compound. To do so add a second column to the SMILES compound library containing a label for each molecule. The ```Ligand SMILES``` file is also labelled something like ```/data/dnb02/galaxy_db/files/010/406/dataset_10406067.dat``` (the exact name will vary) and we would like to give it a more useful name. When labelling is complete, we can concatenate (join together) the library file with the original SMILES file for the ligand from the PDB file.

> <hands-on-title>Calculate molecular fingerprints</hands-on-title>
> 1. {% tool [Replace](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_find_and_replace/9.5+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"File to process"*: `Ligand SMILES`.
>    - {% icon param-file %} *"Find pattern"*: add the current label of the SMILES here. You can find it by clicking the 'view' button next to the `Ligand SMILES` dataset - it will look something like `/data/dnb02/galaxy_db/files/010/406/dataset_10406067.dat`.
>    - {% icon param-file %} *"Replace with"*: `ligand`
> 2. {% tool [Concatenate datasets](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/1.0.0) %} tail-to-head with the following parameters:
>    - {% icon param-file %} *"Datasets to concatenate"*: Output of the previous step.
>    - Click on **Insert Dataset** and in the new selection box which appears, select 'Compound library'.
>    - Run the step and rename the output dataset 'Labelled compound library'. 
> 3. {% tool [Molecule to fingerprint](toolshed.g2.bx.psu.edu/repos/bgruening/chemfp/ctb_chemfp_mol2fps/1.5) %} conversion to several different fingerprint formats with the following parameters:
>    - {% icon param-file %} *"Molecule file"*: 'Labelled compound library' file.
>    - {% icon param-file %} *"Type of fingerprint"*: `Open Babel FP2 fingerprints`
>    - Rename to 'Fingerprints'.
{: .hands_on}

Taylor-Butina clustering  ({% cite Butina1999 %}) provides a classification of the compounds into different groups or clusters, based on their structural similarity. This methods shows us how similar the compounds are to the original ligand, and after docking, we can compare the results to the proposed clusters to observe if there is any correlation.

![Image showing a Fingerprinting System]({% link topics/computational-chemistry/images/fingerprints.png %} "A simple fingerprinting system. Each 1 or 0 in the bitstring corresponds to the presence or absence of a particular feature in the molecule. In this case, the presence of phenyl, amine and carboxylic acid groups are encoded.")

> <hands-on-title>Cluster molecules using molecular fingerprints</hands-on-title>
> 1. {% tool [Taylor-Butina clustering](toolshed.g2.bx.psu.edu/repos/bgruening/chemfp/ctb_chemfp_butina_clustering/1.5) %} of molecular fingerprints with the following parameters:
>    - {% icon param-file %} *"Fingerprint dataset"*: 'Fingerprints' file.
>    - {% icon param-file %} *"threshold"*: `0.8`
> 2. {% tool [NxN clustering](toolshed.g2.bx.psu.edu/repos/bgruening/chemfp/ctb_chemfp_nxn_clustering/1.5.1) %} of molecular fingerprints with the following parameters:
>    - {% icon param-file %} *"Fingerprint dataset"*: 'Fingerprints' file.
>    - {% icon param-file %} *"threshold"*: `0.0`
>    - {% icon param-file %} *"Format of the resulting picture"*: `SVG`
>    - {% icon param-file %} *"Output options"*: `Image`
{: .hands_on}

The image produced by the NxN clustering shows the clustering in the form of a dendrogram, where individual molecules are represented as vertical lines and merged into clusters. Merges are represented by horizontal lines. The y-axis represents the similarity of data points to each other; thus, the lower a cluster is merged, the more similar the data points are which it contains. Clusters in the dendogram are colored differently. For example, all molecules connected in red are similar enough to be grouped into the same cluster. 

![NxN clustering]({% link topics/computational-chemistry/images/nxn.png %} "Dendrogram produced by NxN clustering. The library used to produce this image is generated with a Tanimoto cutoff of 80; here 15 search results are shown, plus the original ligand contained in the PDB file.")

> <details-title>Further investigation (optional)</details-title>
>
> * Try generating fingerprints using some of the other nine different protocols available and monitor how this affects the clustering.
>
> * For both the Taylor-Butina and NxN clustering, a threshold has to be set. Try varying this value and observe how the clustering results are affected.
>
{: .details}

# Post-processing and plotting

From our collection of SD-files, we first extract all stored values into tabular format and then combine the files together to create a single tabular file.

> <hands-on-title>Process SD-files</hands-on-title>
>
> 1. {% tool [Extract values from an SD-file](toolshed.g2.bx.psu.edu/repos/bgruening/sdf_to_tab/sdf_to_tab/2020.03.4+galaxy0) %}  into a tabular file using RDKit with the following parameters:
>    - {% icon param-file %} *"Input SD-file"*: Collection of SD-files generated by the docking step. (Remember to select the 'collection' icon!)
>    - {% icon param-file %} *"Include the property name as header"*: `Yes` 
>    - {% icon param-file %} *"Include SMILES as column in output"*: `Yes`
>    - {% icon param-file %} *"Include molecule name as column in output"*: `Yes` 
>    - Leave all other paramters unchanged.
> 2. {% tool [Collapse Collection](toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/5.1.0) %} into single dataset in order of the collection with the following parameters:
>    - {% icon param-file %} *"Collection of files to collapse into single dataset"*: Collection of tabular files generated by the previous step.
>    - {% icon param-file %} *"Keep one header line"*: `Yes` 
>    - {% icon param-file %} *"Prepend File name"*: `No` 
>
>    {% snippet faqs/galaxy/tools_select_collection.md datatype="datatypes" %}
>
> 3. {% tool [Compound conversion](toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy1) %} - interconvert between various chemistry and molecular modeling data files with the following parameters:
>    - {% icon param-file %} *"Molecular input file"*: choose one of the SD-files from the collection generated by the docking step.
>    - {% icon param-file %} *"Output format"*: `Protein Data Bank format (pdb)` 
>    - {% icon param-file %} *"Split multi-molecule files into a collection"*: `Yes`
>    - Leave all other parameters unchanged. 
{: .hands_on}

We now have a tabular file available which contains all poses calculated for all ligands docked, together with scores and RMSD values for the deviation of each pose from the optimum. We also have PDB files for some of the docking poses which can be inspected using the NGLViewer visualization embedded in Galaxy.

> <details-title>Further investigation (optional)</details-title>
>
> There are visualizations available in Galaxy for producing various types of plots - for example, a scatter plot of RMSD (compared to optimal docking pose) against docking score can be calculated very easily:
> 
> ![Scatter plot of RMSD against docking score]({% link topics/computational-chemistry/images/scatterplot.png %} "Scatter plot of RMSD against docking score.")
> 
> The plot shows a correlation between the two variables, as expected, though only a slight one, given the narrow range of docking scores in the dataset and the structural similarity of the ligands tested.
> 
> A more advanced exercise would be to generate molecular descriptors for the molecule (there are three tools available for this in Galaxy: RDKit, Mordred and PaDEL) and to identify those which are particularly predictive of docking score, either by inspecting plots or using the statistics tools built into Galaxy.
{: .details}

Use the NGLviewer to inspect the protein and various ligand poses generated by docking. This can be done using either the visualization of NGLViewer in Galaxy, or via the [NGL website](http://nglviewer.org/ngl/).

![Two docking poses for a ligand bound to Hsp90]({% link topics/computational-chemistry/images/activesite.gif %} "Two docking poses for a ligand bound to the active site of Hsp90. One (docking score -8.4) can be seen to be bound deeper in the active site than the other (docking score -5.7), which is reflected in the difference between the docking scores.")

tools_json:
["Get PDB", "toolshed.g2.bx.psu.edu/repos/bgruening/get_pdb/get_pdb/0.1.0", "Search in textfiles (grep)", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/9.5+galaxy0", "Compound conversion", "toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy1", "Search ChEMBL database", "toolshed.g2.bx.psu.edu/repos/bgruening/chembl/chembl/0.10.1+galaxy4", "Prepare receptor", "toolshed.g2.bx.psu.edu/repos/bgruening/autodock_vina_prepare_receptor/prepare_receptor/1.5.7+galaxy0", "Calculate the box parameters using RDKit", "toolshed.g2.bx.psu.edu/repos/bgruening/autodock_vina_prepare_box/prepare_box/2021.03.5+galaxy0", "fpocket", "toolshed.g2.bx.psu.edu/repos/bgruening/fpocket/fpocket/3.1.4.2+galaxy0", "Calculate the box parameters for an AutoDock Vina job", "toolshed.g2.bx.psu.edu/repos/bgruening/autodock_vina_prepare_box/prepare_box/2021.03.4+galaxy0", "VINA Docking", "toolshed.g2.bx.psu.edu/repos/bgruening/autodock_vina/docking/1.2.3+galaxy0", "Visualisation", "toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_svg_depiction/openbabel_svg_depiction/3.1.1+galaxy1", "Replace", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_find_and_replace/9.5+galaxy0", "Concatenate datasets", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/1.0.0", "Molecule to fingerprint", "toolshed.g2.bx.psu.edu/repos/bgruening/chemfp/ctb_chemfp_mol2fps/1.5", "Taylor-Butina clustering", "toolshed.g2.bx.psu.edu/repos/bgruening/chemfp/ctb_chemfp_butina_clustering/1.5", "NxN clustering", "toolshed.g2.bx.psu.edu/repos/bgruening/chemfp/ctb_chemfp_nxn_clustering/1.5.1", "Extract values from an SD-file", "toolshed.g2.bx.psu.edu/repos/bgruening/sdf_to_tab/sdf_to_tab/2020.03.4+galaxy0", "Collapse Collection", "toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/5.1.0"]


topic: computational-chemistry
tutorial: analysis-md-simulations
title: Analysis of molecular dynamics simulations
difficulty: intermediate
time_estimation: 1H
requirements_json: []
tags_json: []
content_hash: 8dc11b900377cd7a09d9d8b0ea41c8e4
last_modified: 2026-05-15T22:28:10.764477
zenodo_link: https://zenodo.org/record/2537734
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/computational-chemistry/tutorials/analysis-md-simulations/tutorial.html

content:
Molecular dynamics simulations return highly complex data. The Cartesian positions of each atom of the system (thousands or even millions) are recorded at every time step of the trajectory; this may again be thousands to millions of steps in length. Therefore, some kind of further analysis is needed to extract useful information from the data.

In this tutorial, we illustrate some of the analytical tools able to investigate conformational changes by analysis of a typical short protein simulation, such as for CBH1.

There are other analysis tools available; you are encouraged to try these out too.

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Get data

The data required can be generated by completing the [NAMD simulation tutorial]({% link topics/computational-chemistry/tutorials/md-simulation-namd/tutorial.md %}).  Access it from your history. Alternatively, download the data from the Zenodo link provided.

> <hands-on-title>Upload cellulose simulation trajectory</hands-on-title>
>
> 1. Create a new history
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the files from [Zenodo](https://zenodo.org/record/2537734), or from your history, if you completed the previous [NAMD simulation tutorial]({% link topics/computational-chemistry/tutorials/md-simulation-namd/tutorial.md %}):
>
>    ```
>    https://zenodo.org/record/2537734/files/cbh1test.dcd
>    https://zenodo.org/record/2537734/files/cbh1test.pdb
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Rename the dcd file 'CBH1 trajectory' and rename the pdb file 'CBH1 structure' 
>
>    {% snippet faqs/galaxy/datasets_rename.md %}
>
{: .hands_on}


# Analysis with BIO3D

We'll carry out some basic analysis by calculating RMSD, RMSF and PCA. The tools use the [Bio3D package](http://thegrantlab.org/bio3d), developed by the [Grant lab](http://thegrantlab.org/).


## RMSD

RMSD, or root-mean-square deviation, is a standard measure of structural distance between coordinates. It measures the average distance between a group of atoms (e.g. backbone atoms of a protein). If we calculate RMSD between two sets of atomic coordinates - for example, two time points from the trajectory - the value is a measure of how much the protein conformation has changed. [Wikipedia](https://en.wikipedia.org/wiki/Root-mean-square_deviation_of_atomic_positions) provides more information.

> <hands-on-title>Calculate RMSD</hands-on-title>
>
> {% tool [RMSD Analysis](toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_rmsd/bio3d_rmsd/2.3.4) %} with the following parameters:
>    - {% icon param-file %} *"dcd trajectory input"*: Trajectory file
>    - {% icon param-file %} *"pdb input"*: Structure file
>    - *"Select domains"*: `Calpha` (calculate RMSD only for the C-alpha domain of the protein)
>
{: .hands_on}

![Snapshot of RMSD plot]({% link topics/computational-chemistry/images/RMSD_plot_33.png %} "RMSD plot for a short CBH1 simulation")
![Snapshot of RMSD histogram]({% link topics/computational-chemistry/images/RMSD_Histogram_Plot_34.png %} "RMSD histogram for a short CBH1 simulation")

> <question-title></question-title>
>
> What do the features in the RMSD plot tell us?
>
> > <solution-title></solution-title>
> > The increase in the RMSD plot with time shows the protein steadily deviates from its original conformation.
> >
> > The three peaks visible in the histogram suggests the presence of three main conformations which are accessed during the trajectory.
> {: .solution}
{: .question}


## RMSF

The root-mean-square fluctuation (RMSF) measures the average deviation of a particle (e.g. a protein residue) over time from a reference position (typically the time-averaged position of the particle). Thus, RMSF analyzes the portions of structure that are fluctuating from their mean structure the most (or least).

> <hands-on-title>Calculate RMSF</hands-on-title>
>
> 1. {% tool [RMSF Analysis](toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_rmsf/bio3d_rmsf/2.3.4) %} with the following parameters:
>    - {% icon param-file %} *"dcd trajectory input"*: Trajectory file
>    - {% icon param-file %} *"pdb input"*: Structure file
>    - *"Select domains"*: `Calpha` (calculate RMSF only for the C-alpha domain of the protein)
>
{: .hands_on}

![Snapshot of RMSF plot]({% link topics/computational-chemistry/images/rmsf_plot_23.png %} "RMSF plot for a short CBH1 simulation")

> <question-title></question-title>
>
> What can we learn from the features in the RMSF plot?
>
> > <solution-title></solution-title>
> > Higher RMSF values most likely are loop regions with more conformational flexibility, where the structure is not as well defined.
> >
> > This allows a link with experimental spectroscopic techniques which detect the secondary structure of a protein.
> {: .solution}
{: .question}


## PCA

Principal component analysis (PCA) converts a set of correlated observations (movement of all atoms in protein) to a set of principal components which are linearly independent (or uncorrelated). Mathematically, it is a transformation of the data to a new coordinate system, in which the first coordinate represents the greatest variance, the second coordinate represents the second most variance, and so on.

You can read more about PCA on [Wikipedia](https://en.wikipedia.org/wiki/Principal_component_analysis). In a nutshell, PCA takes a complex dataset with many variables and tries to distill the variables down to a few 'principal components' which still preserve most of the differences between the data.

In summary:
- The **PCA** {% icon tool %} tool will calculate and return a PCA to determine the relationship between statistically meaningful conformations (major global motions) sampled during the trajectory. THe tool returns several images of the PCA and the raw data in tab-separated format.
- The **PCA visualization** {% icon tool %} tool will carry out PCA and return a trajectory of the selected principle component. This trajectory is useful for visualisation and further investigating the interesting modes and changes that occur within a selected principle component.


> <hands-on-title>Calculate PCA</hands-on-title>
> 1. {% tool [PCA](toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_pca/bio3d_pca/2.3.4) %} with the following parameters:
>    - {% icon param-file %} *"dcd trajectory input"*: Trajectory file
>    - {% icon param-file %} *"pdb input"*: Structure file
>    - *"Use singular value decomposition (SVD) instead of default eigenvalue decomposition ?"*: `No`
>    - *"Select domains"*: `Calpha`
>
> 1. {% tool [PCA visualization](toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_pca/bio3d_pca/2.3.4) %} with the following parameters:
>    - {% icon param-file %} *"dcd trajectory input"*: Trajectory file
>    - {% icon param-file %} *"pdb input"*: Structure file
>    - *"Use singular value decomposition (SVD) instead of default eigenvalue decomposition ?"*: `No`
>    - *"Select domains"*: `Calpha`
>    - *"Principal component id"*: `1`
>
> PCA visualisation: This tool can generate small trajectories of the first three principal components. The .pdb of the .nc files can be visualized using a visualization software such as VMD.
>
{: .hands_on}

![Snapshot of PCA plot]({% link topics/computational-chemistry/images/PCA_plot_46.png %} "PCA plot for a short CBH1 simulation")

> <question-title></question-title>
>
> What do the features in the RMSD plot tell us? Do the principal coordinates have a meaning?
>
> > <solution-title></solution-title>
> > Here, PCA shows the statistically meaningful conformations in the CBH1 trajectory. The principal motions within the trajectory and the vital motions needed for conformational changes can be identified. Two distinct groupings along the PC1 plane, indicating a non-periodic conformational change, are identified. The groupings along the PC2 and PC3 planes do not completely cluster separately, implying that these global motions are periodic. The PC1 is linked to an active site motion that limits the motion to a key glycosidic bond.
> {: .solution}
{: .question}

## Workflow vs. individual tools
You can choose to use the tools one by one as described above, or alternatively combine into a single analysis using the workflow provided.
![Snapshot of conformational analysis workflow]({% link topics/computational-chemistry/images/workflow_confanalysis.png %} "A simple analysis workflow")

> <hands-on-title>Upload a workflow</hands-on-title>
>
> 1. Click on 'Workflow' in the toolbar at the top of the main Galaxy page. In the upper right corner of the central pane, click the 'Upload or import workflow' icon.
>
> 2. Enter the 'Archived workflow URL' and click 'Import workflow'.
>    ```
>    https://raw.githubusercontent.com/galaxyproject/training-material/master/topics/computational-chemistry/tutorials/analysis-md-simulations/workflows/main_workflow.ga
>    ```
>
{: .hands_on}


# Further analysis

Further analyses are available; try out the [MDAnalysis workflow]({% link topics/computational-chemistry/tutorials/analysis-md-simulations/workflows/advanced_workflow.ga %}), which includes a Ramachandran plot and various timeseries.

![MDAnalysis workflow]({% link topics/computational-chemistry/images/workflow_mdanalysis.png %} "MD analysis workflow")

# Conclusion

tools_json:
["RMSD Analysis", "toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_rmsd/bio3d_rmsd/2.3.4", "RMSF Analysis", "toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_rmsf/bio3d_rmsf/2.3.4", "PCA", "toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_pca/bio3d_pca/2.3.4", "PCA visualization"]


topic: computational-chemistry
tutorial: setting-up-molecular-systems
title: Setting up molecular systems
difficulty: intermediate
time_estimation: 2H
tools_json: []
requirements_json: []
tags_json: []
content_hash: 3442aad658d25c78ffeb83bd3b81772d
last_modified: 2026-05-15T22:28:10.772851
zenodo_link: https://zenodo.org/record/2600690
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/computational-chemistry/tutorials/setting-up-molecular-systems/tutorial.html

content:
In this tutorial, we'll cover the basics of molecular modelling by setting up a protein in complex with a ligand and uploading the structure to Galaxy. This tutorial will make use of CHARMM-GUI. Please note that the follow-up to this tutorial (located in [Running molecular dynamics simulations using NAMD]({% link topics/computational-chemistry/tutorials/md-simulation-namd/tutorial.md %})) requires access to NAMD Galaxy tools, which can be accessed using the [Docker container](https://github.com/scientificomputing/BRIDGE) but are currently not available on any public Galaxy server.

> <comment-title>Audience</comment-title>
> This tutorial is intended for those who are new to the computational chemistry tools in Galaxy.
{: .comment}

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Cellulase and cellulose

To start we'll look at the PDB and find the entry for a fungal enzyme that cleaves cellulose. The enzyme is 7CEL, a hydrolase as seen in [the figure.](#figure-1)

![Snapshot of 7CEL pdb with octaose ligand]({% link topics/computational-chemistry/images/enzyme.jpg %} "7CEL Cellulase with a short chain cellulose (octaose) ligand")

In this section we'll access the PDB, download the correct structure, import it and view in Galaxy.

> <details-title>Background: What is the PDB (Protein Data Bank) and format?</details-title>
>
> The Protein Data Bank (PDB) format contains atomic coordinates of biomolecules and provides a standard representation for macromolecular structure data derived from X-ray diffraction and NMR studies. Each structure is stored under a four-letter accession code. For example, the PDB file we will use is assigned the code [7CEL](https://www.rcsb.org/pdb/explore/explore.do?structureId=7CEL)).
>
> More resources:
>
>  -  Multiple structures are stored and can be queried at [https://www.rcsb.org/](https://www.rcsb.org/)
>  - Documentation describing the PDB file format is available from the wwPDB at [http://www.wwpdb.org/documentation/file-format.php](http://www.wwpdb.org/documentation/file-format.php).
{: .details}


> <details-title>Background: Why choose a cellulase?</details-title>
>
> Using enzymes to break down abundant cellulose into disaccharide units (cellobiose) is a method to optimise the
> biofuel process. {% cite barnett11 %}
>
> More resources:
>
>  - [https://en.wikipedia.org/wiki/Cellulase](https://en.wikipedia.org/wiki/Cellulase)
>  - [https://en.wikipedia.org/wiki/Biofuel](https://en.wikipedia.org/wiki/Biofuel)
>  - {% cite Payne_2015 %}
>  - {% cite Barnett_2010 %}
{: .details}

## Get data

The 7CEL [PDB](https://files.rcsb.org/download/7CEL.pdb) does not include a complete 8 unit substrate and some modelling is required. The correctly modelled substrate is provided for this tutorial.
> <details-title>More details about the modelling done</details-title>
> - VMD (visualisation software) was used for atomic placement and CHARMM was used for energy minimisation.
> - The PDB structure contains a mutation at position 217 (glumatate to glutamine). Our structure reverses this.
> - The ligand was modelled separately and inserted into the binding site.
>
{: .details}

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this tutorial.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the files from the Zenodo link provided.
>
>    ```
>    https://zenodo.org/record/2600690/files/7cel_modeled.pdb?download=1
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
> 3. Rename the datasets.
> 4. Check that the datatype is correct. The file should have the PDB datatype.
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="pdb" %}
>
{: .hands_on}

# Modelling with CHARMM-GUI
It is convenient to set up the molecular system outside Galaxy using a tool such as CHARMM-GUI. Alternative methods are possible - see the [GROMACS tutorial]({% link topics/computational-chemistry/tutorials/md-simulation-gromacs/tutorial.md %}) for an example. {% cite jo17 %}

> <tip-title>Viewing figures</tip-title>
> * Some of the figures are screenshots and it may be difficult to make out details
> * Right-click on the image and choose 'Open image in new tab' to view
> * Zoom in and out as needed to see the content
{: .tip}

![CHARMM-GUI interface]({% link topics/computational-chemistry/images/charmmgui.png %} "The CHARMM-GUI interface")

Go to the correct section depending on which MD engine you will be using.

## CHARMM

### Upload the PDB to CHARMM-GUI
[Navigate to CHARMM-GUI](http://www.charmm-gui.org/?doc=input/pdbreader) and use the Input Generator, specifically the PDB Reader tool and upload the Cellulase PDB file. Press 'Next Step: Select Model/Chain' in the bottom right corner.

> <hands-on-title>Upload the PDB to CHARMM-GUI</hands-on-title>
>
> 1. Retrieve the modelled PDB structure from [Zenodo](https://doi.org/10.5281/zenodo.2600690).
> 2. Upload the PDB and choose CHARMM format.
> ![Snapshot of CHARMM-GUI PDB reader section]({% link topics/computational-chemistry/images/charmmgui-reader.png %} "The CHARMM-GUI PDB Reader tool")
{: .hands_on}

### Select both protein and ligand models

> <hands-on-title>Generate PDB file</hands-on-title>
> Two model chains are presented for selection: the protein (PROA) and the hetero residue, which is the ligand or glycan in this case (HETA). Select both, and press 'Next Step: Generate PDB' in the bottom right corner.
> ![Snapshot of CHARMM-GUI model section]({% link topics/computational-chemistry/images/charmmgui-modelchain.png %} "Select both ligand and protein models in CHARMM-GUI")
{: .hands_on}

### Manipulate the system
> <hands-on-title>Make necessary modifications</hands-on-title>
> Rename the hetero chain to BGLC and add ten disulfide bonds to the protein, as shown in the figure. Then press 'Next Step: Generate PDB' in the bottom right corner.
> ![Snapshot of CHARMM-GUI renaming section]({% link topics/computational-chemistry/images/charmmgui-manipulate.png %} "Rename the chains in CHARMM-GUI")
{: .hands_on}

### Download the output
> <hands-on-title>Download CHARMM output</hands-on-title>
> The output is a .tgz file (a tarball or zipped tarball). Inside the archive you will see all inputs and outputs from CHARMM-GUI.
> ![Snapshot of CHARMM-GUI CHARMM output section]({% link topics/computational-chemistry/images/charmmgui-charmmoutput.png %} "CHARMM output from CHARMM-GUI")
{: .hands_on}

> <tip-title>What is a .tgz file?</tip-title>
>
> This is a compressed file which contains all the output files created by the CHARMM-GUI. To access them, the .tgz file needs to be decompressed. There should be a tool available on your operating system for this. If you prefer to use the command line, tar will work fine on Linux or Mac `tar -zxvf example.tgz`. On Windows use [7zip](https://www.7-zip.org/download.html), or download Git for windows and use Git Bash.
{: .tip}


### Upload to Galaxy
> <hands-on-title>Upload files to Galaxy</hands-on-title>
> Upload the step1_pdbreader.psf and step1_pdbreader.crd files to your Galaxy instance and run the system setup tool.
{: .hands_on}

## NAMD

### Upload the PDB to CHARMM-GUI

> <hands-on-title>Upload the PDB to CHARMM-GUI</hands-on-title>
> Retrieve the modelled PDB structure from [Zenodo](https://doi.org/10.5281/zenodo.2600690).
> [Navigate to CHARMM-GUI](http://www.charmm-gui.org/?doc=input/mdsetup) and use the Input Generator, specifically the Solution Builder tool. Upload the PDB file, selecting 'CHARMM' as the file format. Press 'Next Step: Select Model/Chain' in the bottom right corner.
> ![Snapshot of CHARMM-GUI Solution Builder tool ]({% link topics/computational-chemistry/images/charmmgui-mdsimulator-solution-builder.png %} "The CHARMM-GUI Solution Builder tool")
{: .hands_on}


### Select both protein and ligand models
> <hands-on-title>Generate PDB file</hands-on-title>
> Two model chains are presented for selection: the protein (PROA) and the hetero residue, which is the ligand or glycan in this case (HETA). Select both, and press 'Next Step: Generate PDB' in the bottom right corner.
> ![Snapshot of CHARMM-GUI model section]({% link topics/computational-chemistry/images/charmmgui-modelchain.png %} "Select both ligand and protein models in CHARMM-GUI")
{: .hands_on}


### Manipulate the system
> <hands-on-title>Make necessary modifications</hands-on-title>
> Rename the hetero chain to BGLC and add disulfide bonds.Press 'Next Step: Generate PDB' in the bottom right corner.
> ![Snapshot of CHARMM-GUI renaming section]({% link topics/computational-chemistry/images/charmmgui-manipulate.png %} "Rename the chains in CHARMM-GUI")
{: .hands_on}


### Set up the waterbox and add ions
> <hands-on-title>Solvate the protein</hands-on-title>
> Set up a waterbox. Use a size of 10 angstroms and choose a cubic box ('rectangular' option).
> ![Snapshot of CHARMM-GUI waterbox section]({% link topics/computational-chemistry/images/charmm-gui-waterbox-from-solution-builder.png %} "Setting up a waterbox in CHARMM-GUI")
{: .hands_on}

> <question-title></question-title>
>
> Why is 10 angstrom a fair choice for the buffer?  Why choose 0.15M NaCl?
>
> > <solution-title></solution-title>
> > Under periodic boundary conditions, we need to ensure the protein can never interact with its periodic image, otherwise artefacts are introduced. Allowing 10 angstroms between the protein and the box edge ensures the two images will always be at minimum 20 angstroms apart, which is sufficient.
> >
> > Some of the residues on the protein surface are charged and counter-ions need to be present nearby to neutralise them. Failure to explicitly model salt ions may destabilise the protein.
> {: .solution}
{: .question}


### Generate the FFT automatically

> <hands-on-title>Generate the FFT</hands-on-title>
> Particle Mesh Ewald (PME) summation is the method being used to calculate long-range interactions in this system. To improve the computational time a Fast Fourier Transform (FFT) is used. A detailed discussion of FFT will not be presented here; there are many articles on the subject. Try [Wikipedia](https://en.wikipedia.org/wiki/Ewald_summation) and {% cite Toukmaji_1996 %}.
> ![Snapshot of CHARMM-GUI FFT section]({% link topics/computational-chemistry/images/charmmgui-fft.png %} "Setting up a FFT in CHARMM-GUI")
{: .hands_on}

### Download the output
> <hands-on-title>Solvate the protein</hands-on-title>
> The output is a .tgz file (a tarball or zipped tarball). Inside the archive you will see all inputs and outputs from CHARMM-GUI.
> ![Snapshot of CHARMM-GUI NAMD output section]({% link topics/computational-chemistry/images/charmmgui-namdoutput.png %} "NAMD output from CHARMM-GUI")
{: .hands_on}

> <tip-title>What is a .tgz file?</tip-title>
>
> This is a compressed file and needs to be uncompressed using the correct tool.
> On Linux or Mac: tar will work fine `tar -zxvf example.tgz`.
> On Windows use [7zip](https://www.7-zip.org/download.html) or download Git for windows and use Git Bash.
{: .tip}


### Upload to Galaxy
> <hands-on-title>Upload files to Galaxy</hands-on-title>
> Upload the following files to your Galaxy instance and ensure the correct datatype is selected:
> - step3_pbcsetup.psf -> xplor psf input (psf format)
> - step3_pbcsetup.pdb -> pdb input (pdb format)
> - Checkfft.str -> PME grid specs (txt format)
> - step2.1_waterbox.prm -> waterbox prm input (txt format)
{: .hands_on}

You are now ready to run the NAMD workflow, which is discussed in another [tutorial]({% link topics/computational-chemistry/tutorials/md-simulation-namd/tutorial.md %}).


# Conclusion


{% icon trophy %} Well done! You have started modelling a cellulase protein and uploaded it into Galaxy. The next step is running molecular dynamics simulations ([tutorial]({% link topics/computational-chemistry/tutorials/md-simulation-namd/tutorial.md %}))


topic: computational-chemistry
tutorial: covid19-docking
title: Virtual screening of the SARS-CoV-2 main protease with rxDock and pose scoring
difficulty: intermediate
time_estimation: 2H # Just 1 week (if you have 5000 CPUs) ;)
requirements_json: []
tags_json: ["covid19", "one-health"]
content_hash: d5d56a976a1103a9934c2c8ff5f6460c
last_modified: 2026-05-15T22:28:10.766509
zenodo_link: https://zenodo.org/record/3730474
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/computational-chemistry/tutorials/covid19-docking/tutorial.html

content:
This tutorial provides a companion to the work performed in March 2020 by InformaticsMatters, the Diamond Light Source, and the European Galaxy Team to perform virtual screening on candidate ligands for the SARS-CoV-2 main protease (MPro). This work is described [in our dedicated site](https://covid19.galaxyproject.org/cheminformatics).

In this tutorial, you will perform protein-ligand docking to MPro using rxDock ({% cite rdock %}), a version of the popular rDock software, and score the results using two different methods. The same tools will be used as in the original study, but with a smaller dataset.

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Background

Early in March 2020, the Diamond Light Source completed a successful fragment screen on MPro, which provided 55 fragment hits (see their [press release](https://www.diamond.ac.uk/covid-19/for-scientists/Main-protease-structure-and-XChem.html) here). In an effort to identify candidate molecules for binding, InformaticsMatters, the XChem group and the European Galaxy team joined forces to construct and execute a Galaxy workflow for performing and evaluating molecular docking on a massive scale.

An initial list of 41,000 candidate molecules was assembled by using the Fragalysis fragment network to elaborate from the initial fragment hits, as described [in their documentation](https://diamondlightsource.atlassian.net/wiki/spaces/FRAG/pages/8323192/The+Astex+Fragment+network). These were used as inputs for the docking and scoring workflow. The workflow consists of the following steps, each of which was carried out using tools installed on the European Galaxy server:
1. Charge enumeration of the 41,000 candidate molecules selected based on the fragment hits.
2. Generation of 3D conformations based on SMILES strings of the candidate molecules.
3. Docking of molecules into each of the MPro structures using rxDock.
4. Evaluation of the docking poses using a TransFS, a deep learning approach ({% cite transfs %}) developed by the XChem group and collaborators, and SuCOS scoring ({% cite sucos %}), which compares the poses with the structures of the original fragment hits.

The original study required almost 20 years of CPU time, not counting GPU resources consumed. This is obviously not reproducible as a tutorial. Therefore, we will repeat the workflow with a small library of just 100 molecules, on a single MPro fragment structure. Links will be provided to original Galaxy histories, with notes to explain where and why things were done differently to the tutorial.

![MPro structure, with a fragment bound]({% link topics/computational-chemistry/images/mpro.png %} "Structure of MPro, with a fragment bound. <a href="https://usegalaxy.eu/u/sbray/v/mpro-x0072">Click to view</a> in NGL. ({% cite ngl %})")

# Get data

We require three datasets for the simulation and analysis:
1. A list of 100 ligand candidates. These are the molecules which will be docking into the protein binding site.
2. A PDB file of the receptor MPro protein (without ligand or solvent).
3. A list of fragment hits (17 in total) in SDF format.

> <details-title>Differences with the original study</details-title>
>
> Of the initial 55 fragment hits, 17 were chosen for further study. From these, 41,587 compounds were generated using the Fragalysis fragment network for further study. The 100 compounds used in the tutorial are taken from this list.
>
>
> Starting data is available from this Galaxy history: [https://usegalaxy.eu/u/sbray/h/mpro-raw-data](https://usegalaxy.eu/u/sbray/h/mpro-raw-data).
> 
> 
> This history contains 103 files. One of these (`Initial candidates for docking`) contains the 41k candidate compounds in SMILES format. The remaining 102 files (all with names beginning with `Mpro-x...`) provide structural information on the fragment hits - 6 files per hit (hence 17 x 6 = 102). 
>
>
> The identity of the files is as follows:
>
> - the `*_0.mol` files contain the fragment structure in mol format.
> - the `*_0.pdb` files contain the fragment structure in pdb format.
> - the `*_0_apo.pdb` files contain the protein with solvent, but without ligand
> - the `*_0_apo-desolv.pdb` files contain the protein without either solvent or ligand
> - the `*_0_apo-solv.pdb` files contain only solvent
> - the `*_0_bound.pdb` file contain everything (protein, ligand and solvent)
>
> The PDB file of the receptor that we are using is `Mpro-x0195_0_apo-desolv.pdb`. In other words, the structure is derived from just one fragment hit. In the original study, however, all compounds were docked against all of the fragment hit structures.
{: .details}

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo](https://zenodo.org/record/3730474):
>
>    ```
>    https://zenodo.org/record/3730474/files/candidates.smi
>    https://zenodo.org/record/3730474/files/Mpro-x0195_0_apo-desolv.pdb
>    https://zenodo.org/record/3730474/files/hits.sdf
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Rename the datasets `Candidates SMILES`, `Receptor PDB` and `Hits SDF` respectively.
> 4. Check that the datatypes (`smi`, `pdb`, and `sdf` respectively) are correct. In particularly, check the `Candidates SMILES` file, as the SMILES datatype is not detected automatically by Galaxy.
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
{: .hands_on}

# Preparation for docking

Before docking, the candidate ligands need to be prepared for docking with the following steps: 1) charge enumeration, 2) generation of three-dimensional structures, and 3) splitting the SD-file into a collection.

> <details-title>Differences with the original study</details-title>
>
> This stage is carried out as described here, except of course with the full set of 42,000 compounds. See [the docking prep workflow](https://covid19.galaxyproject.org/cheminformatics/1-DockingPrep/) for more details.
{: .details}

## Charge enumeration

Many of the compounds may contain functional groups which can exist in multiple charge states, and this will affect the quality of binding to the receptor dramatically. Therefore, we perform 'charge enumeration', which means that we generate all charge forms of the compounds within a certain pH range.

> <hands-on-title>Charge enumeration</hands-on-title>
>
> 1. {% tool [Enumerate charges](toolshed.g2.bx.psu.edu/repos/bgruening/enumerate_charges/enumerate_charges/2020.03.4+galaxy0) %} with the following parameters:
>    - *"Input molecule data"*: `Candidate SMILES`
>    - *"Minimum pH"*: `4.4`
>    - *"Maximum pH"*: `10.4`
> 2. Rename the output file `Enumerated candidates SMILES`.
>
>    > <comment-title></comment-title>
>    >
>    > The **Enumerate charges** {% icon tool %} tool is based on the Dimorphite-DL program. ({% cite Ropp2019 %})
>    {: .comment}
>
{: .hands_on}

The output is another SMILES file, with several hundred entries.

## Generate three-dimensional conformations

So far our list of enumerated candidate compounds is still in SMILES format; we need to produce three-dimensional structures in SDF format for docking. This can be done with the **Compound conversion** {% icon tool %} tool.

If you are not familiar with SMILES and SDF formats, consult the introductory [protein-ligand docking tutorial](../cheminformatics/tutorial.html) for more details.

> <hands-on-title>Convert to SDF format</hands-on-title>
>
> 1. {% tool [Compound conversion](toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy0) %} with the following parameters:
>    - *"Molecular input file"*: `Enumerated candidates` dataset.
>    - *"Output format"*: `MDL MOL format (sdf, mol)`
>        - *"Generate 3D coordinates"*: `Yes`
> 2. Rename the output file `Enumerated candidates SDF`.
>
>    > <comment-title></comment-title>
>    >
>    > The **Compound conversion** {% icon tool %} tool is based on the OpenBabel toolkit. ({% cite OBoyle2011 %})
>    {: .comment}
>
{: .hands_on}

## Splitting the SD-file into a collection

The next stage is to split the SD-file with the candidate ligands into a set of smaller SD-files.


> <question-title></question-title>
>
> Why is splitting the file necessary?
>
> > <solution-title></solution-title>
> >
> > The rxDock tool performs one docking at a time (more technically: the task is not parallelized, as it uses only a single CPU). Therefore, splitting the large SD-file into many small files allows the work to be carried out by multiple Galaxy jobs in parallel, so it completes faster.
> >
> > In the original study, this kind of parallelization was absolutely essential because of the enormous dataset; at some points, there were 5,000 docking jobs running concurrently on the European Galaxy server. Even on the much smaller scale of this tutorial, we can speed things up considerably using this trick.
> >
> {: .solution}
{: .question}

> <hands-on-title>Split the SD-files</hands-on-title>
>
> 1. {% tool [Split file to dataset collection](toolshed.g2.bx.psu.edu/repos/bgruening/split_file_to_collection/split_file_to_collection/0.5.0) %} with the following parameters:
>    - *"Select the file type to split"*: `SD-files`
>    - *"SD-file to split"*: `Enumerated candidates SDF`
>        - *"Specify number of output files or number of records per file?"*: `Number of output files`
>            - *"Number of new files"*: `10`
>        - *"Method to allocate records to new files"*: `Alternate output files`
>
>
{: .hands_on}

# Active site preparation

The active site also needs to be prepared for docking, using the following steps: 1) conversion to MOL2 format, and 2) generation of the active site using the **rbcavity** {% icon tool %} tool.

> <details-title>Differences with the original study</details-title>
>
> This stage was carried out as described here. However, it was repeated for each of the fragment hit structures, not just the `Mpro-x0195_0_apo-desolv.pdb` file used here. See [the active site prep workflow](https://covid19.galaxyproject.org/cheminformatics/2-ActiveSitePrep/) for more details.
{: .details}

## Convert protein structure to MOL2 format

The receptor file we are using is in PDB format, but the rxDock tool we use for docking requires an input in MOL2 format. Therefore, we first convert the file.

> <hands-on-title>Conversion to MOL2 format</hands-on-title>
>
> 1. {% tool [Compound conversion](toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy0) %} with the following parameters:
>    - *"Molecular input file"*: `Receptor PDB` dataset.
>    - *"Output format"*: `Sybyl Mol2 format (mol2)`
> 2. Rename the output file `Receptor MOL2`.
>
>
{: .hands_on}

## Generate Frankenstein ligand

For docking with rxDock, a file needs to be created defining the active site. This requires two input files - one for the protein and one for the ligand. We want an active site generation that takes into account the features of all 17 fragments, and therefore need to generate a 'Frankenstein ligand' which possesses the properties of all the fragments. A very simple Galaxy tool is available for this.

> <question-title></question-title>
>
> What is a 'Frankenstein ligand' and why do we need it?
>
> > <solution-title></solution-title>
> >
> > The Frankenstein ligand combines the atoms of all the fragments and therefore occupies the entire space of the binding site. Therefore, it is the best choice for cavity definition. See the [information provided by InformaticsMatters](https://www.informaticsmatters.com/blog/2018/11/23/cavities-and-frankenstein-molecules.html) for more details.
> >
> {: .solution}
>
{: .question}

> <hands-on-title>Generate Frankenstein ligand</hands-on-title>
>
> 1. {% tool [Create Frankenstein ligand](toolshed.g2.bx.psu.edu/repos/bgruening/ctb_frankenstein_ligand/ctb_frankenstein_ligand/2013.1-0+galaxy0) %} with the following parameters:
>    - *"Input file"*: `Hits SDF`
> 2. Rename the file to `Frankstein SDF`.
>
{: .hands_on}

## Generate active site definition

The active site can now be generated using the **rbcavity** {% icon tool %} tool, which requires the receptor in MOL2 format as input as well as a single reference ligand in Mol/SDF format. We use the Frankenstein ligand as the reference.

> <hands-on-title>Active site preparation</hands-on-title>
>
> 1. {% tool [rxDock cavity definition](toolshed.g2.bx.psu.edu/repos/bgruening/rxdock_rbcavity/rxdock_rbcavity/0.1.5) %} with the following parameters:
>    - *"Receptor"*: `Receptor MOL2`
>    - *"Reference ligand"*: `Frankenstein SDF`
>    - *"Mapper sphere radius"*: `3.0`
>    - *"Mapper small sphere radius"*: `1.0`
>    - *"Mapper minimum volume"*: `100`
>    - *"Mapper volume increment"*: `0`
>    - *"Mapper grid step"*: `0.5`
>    - *"Cavity weight"*: `1.0`
>
> 2. Rename the output file `Active site`.
>    > <comment-title></comment-title>
>    >
>    > The meanings of these parameters are too complex to go into in this tutorial. If you are interested, see the [rDock documentation](http://rdock.sourceforge.net/wp-content/uploads/2015/08/rDock_User_Guide.pdf) for more details.
>    {: .comment}
>
{: .hands_on}

# Docking and scoring

Docking and scoring are now performed, using the following steps: 1) docking using rxDock, 2) recombining the results into a single SDF file, 3) TransFS scoring, and 4) SuCOS scoring.

> <details-title>Differences with the original study</details-title>
>
> This section in the original study differed from this tutorial in the following ways:
> 1. Docking was performed on over 100,000 enumerated candidates, rather than the 300 used here.
> 2. 25 different poses were generated per candidate, rather than 5, as in this tutorial.
> 3. Because of the large number of poses to score (more than a million), the scoring steps were parallelized by splitting into collections. This is skipped in the tutorial.
> 4. The entire process was repeated 17 times, using a different fragment hit as the receptor structure each time.
> See [the Docking](https://covid19.galaxyproject.org/cheminformatics/3-Docking/) for more details. A full list of Galaxy histories generated is listed [the histories page](https://covid19.galaxyproject.org/cheminformatics/Histories/).
{: .details}

## Docking with rxDock

> <hands-on-title>Docking</hands-on-title>
>
> 1. {% tool [rxDock docking](toolshed.g2.bx.psu.edu/repos/bgruening/rxdock_rbdock/rxdock_rbdock/0.1.5) %} with the following parameters:
>    - *"Receptor"*: `Receptor MOL2`
>    - *"Active site"*: `Active site`
>    - *"Ligands"*: `Split file` collection
>    - *"Number of dockings"*: `5`
>    - *"Number of best poses"*: `5`
>
>    > <comment-title></comment-title>
>    >
>    > For more information about docking, check out the [introductory tutorial](../cheminformatics/tutorial.html). It uses a different tool, AutoDock Vina, rather than rxDock, but the general principles are the same.
>    {: .comment}
>
{: .hands_on}

## Collapse collection to a single file 

Having created a collection to parallelize the docking procedure, we can now recombine the results to a single file.

> <hands-on-title>Collapse collection</hands-on-title>
>
> 1. {% tool [Collapse Collection](toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/4.2) %} with the following parameters:
>    - *"Collection of files to collapse into single dataset"*: Output of docking step
> 2. Rename to `Docked poses SDF`.
>
{: .hands_on}

The output file should contain around 1,900 docked poses in SDF format.

## Pose scoring with TransFS

In this step, we carry out scoring of the poses using TransFS. This is a deep learning approach developed at the University of Oxford, employing augmentation of training data with incorrectly docked ligands to prompt the model to learn from protein-ligand interactions. ({% cite transfs %})

The TransFS scoring returns a value (saved as `<TransFSScore>` in the SDF file) between 0 (poor) and 1 (good).

> <hands-on-title>TransFS scoring</hands-on-title>
>
> 1. {% tool [XChem TransFS pose scoring](toolshed.g2.bx.psu.edu/repos/bgruening/xchem_transfs_scoring/xchem_transfs_scoring/0.4.0) %} with the following parameters:
>    - *"Receptor"*: `Receptor PDB`
>    - *"Ligands"*: `Docked poses SDF`
>    - *"Distance to waters"*: `2`
>
{: .hands_on}

## Pose scoring with SuCOS

This step involves scoring of the poses from each molecule against the original fragment screening hit ligands using the SuCOS MAX shape and feature overlay algorithm. ({% cite sucos %}) The conformation and position of the poses are compared to known structures (i.e. the fragment hits) to determine a score.

SuCOS scoring returns a value (saved as `<Max_SuCOS_Score>` in the SDF file) between 0 (poor) and 1 (good).

> <hands-on-title>SuCOS scoring</hands-on-title>
>
> 1. {% tool [Max SuCOS score](toolshed.g2.bx.psu.edu/repos/bgruening/sucos_max_score/sucos_max_score/2020.03.4+galaxy0) %} with the following parameters:
>    - *"Ligands to be scored"*: Output of the TransFS step
>    - *"Set of clusters to score against"*: `Hits SDF`
> 2. Rename the output file to `Scored poses`.
{: .hands_on}

# Compound selection

The aim of the original study was to select 500 candidate molecules for synthesis and experimental study. In order to do this, the data for all fragment hits had to be combined (i.e. so that each compound was assigned the lowest score from all the fragments). The resulting table was then compared with a list of compounds available from [Enamine](https://enamine.net/) and [Chemspace](https://chem-space.com/) and the 500 highest scoring matching compounds were selected for purchase.

This step is skipped in the tutorial, as only 100 compounds were tested, using only a single fragment hit structure. If you want, though, check out the [history](https://usegalaxy.eu/u/timdudgeon/h/top-500-enamine--chemspace-bb) and [workflow](https://usegalaxy.eu/u/timdudgeon/w/filter-results) used.

# Conclusion


This tutorial guided you through docking and scoring of a small set of compounds to the MPro protein. Hopefully, you have a better understanding of how docking can be practically used, as well as how the original study was performed.

tools_json:
["Enumerate charges", "toolshed.g2.bx.psu.edu/repos/bgruening/enumerate_charges/enumerate_charges/2020.03.4+galaxy0", "Compound conversion", "toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy0", "Split file to dataset collection", "toolshed.g2.bx.psu.edu/repos/bgruening/split_file_to_collection/split_file_to_collection/0.5.0", "Create Frankenstein ligand", "toolshed.g2.bx.psu.edu/repos/bgruening/ctb_frankenstein_ligand/ctb_frankenstein_ligand/2013.1-0+galaxy0", "rxDock cavity definition", "toolshed.g2.bx.psu.edu/repos/bgruening/rxdock_rbcavity/rxdock_rbcavity/0.1.5", "rxDock docking", "toolshed.g2.bx.psu.edu/repos/bgruening/rxdock_rbdock/rxdock_rbdock/0.1.5", "Collapse Collection", "toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/4.2", "XChem TransFS pose scoring", "toolshed.g2.bx.psu.edu/repos/bgruening/xchem_transfs_scoring/xchem_transfs_scoring/0.4.0", "Max SuCOS score", "toolshed.g2.bx.psu.edu/repos/bgruening/sucos_max_score/sucos_max_score/2020.03.4+galaxy0"]


topic: computational-chemistry
tutorial: htmd-analysis
title: High Throughput Molecular Dynamics and Analysis
difficulty: advanced
time_estimation: 3H
requirements_json: []
tags_json: []
content_hash: caaeb3e479e626c5c53b726ad523ea2f
last_modified: 2026-05-27T09:01:47.900774
zenodo_link: https://zenodo.org/records/3813283
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/computational-chemistry/tutorials/htmd-analysis/tutorial.html

content:
This tutorial provides an introduction to using high-throughput molecular dynamics to study protein-ligand interaction, as applied to the N-terminal domain of Hsp90 (heat shock protein 90).


> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}


# Background

## What is high-throughput molecular dynamics?
Molecular dynamics (MD) is a method to simulate molecular motion by iterative application of Newton's laws of motion. It is often applied to large biomolecules such as proteins or nucleic acids. A common application is to assess the interaction between these macromolecules and a number of small molecules (e.g. potential drug candidates). This tutorial provides a guide to setting up and running a high-throughput workflow for screening multiple small molecules, using the open-source GROMACS tools provided through the Galaxy platform. Following simulation, the trajectory data is analyzed using a range of tools to investigate structural properties and correlations over time.


## Why is Hsp90 interesting to study?
The 90 kDa heat shock protein (Hsp90) is a chaperone protein responsible for catalyzing the conversion of a wide variety of proteins to a functional form; examples of the Hsp90 clientele, which totals several hundred proteins, include nuclear steroid hormone receptors and protein kinases ({% cite Pearl2006 %}). The mechanism by which Hsp90 acts varies between clients, as does the client binding site; the process is dependent on post-translational modifications of Hsp90 and the identity of co-chaperones which bind and regulate the conformational cycle ({% cite Schopf2017 %}).

Due to its vital biochemical role as a chaperone protein involved in facilitating the folding of many client proteins, Hsp90 is an attractive pharmaceutical target. In particular, as protein folding is a potential bottleneck to cellular reproduction and growth, blocking Hsp90 function using inhibitors which bind tightly to the ATP binding site of the NTD could assist in treating cancer; for example, the antibiotic geldanamycin and its analogs are under investigation as possible anti-tumor agents ({% cite Stebbins1997 %}, {% cite Hermane2019 %}).

In the structure which will be examined during this tutorial, the ligand of concern is a resorcinol, a common class of compounds with affinity for the Hsp90 N-terminal domain. It is registered in the PubChem database under the compound ID 135508238 ({% cite ligand_resorcinol %}). As can be seen by viewing the PDB structure, the resorcinol part of the structure is embedded in the binding site, bound by a hydrogen bond to residue aspartate-93. The ligand structure also contains a triazole and a fluorophenyl ring, which lie nearer to the surface of the protein.

![Image showing Hsp90 structure, with a ligand bound]({% link topics/computational-chemistry/images/hsp90lig.png %} "Structure of Hsp90, with a ligand bound. <a href="https://usegalaxy.eu/u/sbray/v/hsp90-lig">Click to view</a> in NGL. ({% cite ngl %})")


## Get data

As a first step, we create a new Galaxy history and then we download a crystal structure for the Hsp90 protein from the Protein Data Bank (PDB). The structure is provided under accession code `6HHR` ({% cite Schuetz2018 %}) and shows Hsp90 in complex with a ligand belonging to the resorcinol class.

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Search Galaxy for the {% tool [Get PDB](toolshed.g2.bx.psu.edu/repos/bgruening/get_pdb/get_pdb/0.1.0) %} tool. Request the accession code ```6hhr```.
> 3. Rename the dataset to 'Hsp90 structure'
> 4. Check that the datatype is correct (PDB file).
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
>
{: .hands_on}


# Simulation

## Topology generation

Now we have downloaded a PDB structure of the protein we wish to study, we will start preparing it for MD simulation; this process may also be referred to as parameterization or topology generation.

GROMACS distinguishes between constant and dynamic attributes of the atoms in the system. The constant attributes (e.g. atom charges, bonds connecting atoms) are listed in the topology (TOP file), while dynamic attributes (attributes that can change during a simulation, e.g. atom position, velocities and forces) are stored in structure (PDB or GRO) and trajectory (XTC and TRR) files.

The PDB file we start from only explicitly states atom element (i.e.carbon, oxygen, and so on) and 3D Cartesian coordinates of each atom; additionally, it will usually not include hydrogen atoms. Therefore, before beginning simulation, we need to calculate the rest of the information contained within the topology file. Parameterization needs to be done separately for the ligand and protein. Therefore, the first step is to separate the PDB file into two sets of coordinates - one for the ligand and one for the protein.


### Extract protein and ligand coordinates

Parameterization needs to be done separately for the ligand and protein. Therefore, the first step is to separate the PDB file into two sets of coordinates - one for the ligand and one for the protein. Here, we can make use of the simple text manipulation tools integrated into Galaxy.

> <question-title></question-title>
>
> 1. Why do protein and ligand need to be parameterized separately?
>
> > <solution-title></solution-title>
> >
> > 1. Protein and small molecules are constructed differently. A protein is made up of 20 different building blocks (amino acids) - therefore, to construct a protein topology, amino acid topologies simply need to be combined appropriately. By contrast, the structure of small molecules is far more flexible and needs to be calculated for each different structure.
> >
> {: .solution}
>
{: .question}

> <hands-on-title>Separate protein and ligand coordinates</hands-on-title>
>
> 1. {% tool [Search in textfiles (grep)](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/1.1.1) %} with the following parameters:
>    - *"Select lines from"*: 'Hsp90 structure'
>    - *"that"*: `Don't Match`
>    - *"Regular Expression"*: `HETATM`
> 2. Rename output to 'Protein (PDB)'
> 3. {% tool [Search in textfiles (grep)](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/1.1.1) %} with the following parameters:
>    - *"Select lines from"*: 'Hsp90 structure'
>    - *"that"*: `Match`
>    - *"Regular Expression"*: `AG5E`
> 4. Rename output to 'Ligand (PDB)'
>
{: .hands_on}

Here, we simply filter the original PDB twice: once for lines which do not match `HETATM`, which returns a PDB file containing only protein, not ligand and solvent; and once for lines which match the ligand's identity code `AG5E`, which returns a PDB file containing only the ligand.

If you inspect the PDB file, you will see that two different ligands are listed: `AG5E` and `BG5E`. These are the same compound (`G5E`); they refer to two possible conformations it could adopt, both of which would be compatible with the crystallographic data. We choose to take the coordinates from `AG5E` for our simulations, though you could just as well enter `BG5E` instead. For the purpose of this tutorial, you should consider this as a minor technical detail arising from the nature of protein crystallography.

### Set up protein topology

Firstly, we need to calculate the topology for the protein file. We will use the **GROMACS initial setup** {% icon tool %} tool.

> <hands-on-title>Generate protein topology</hands-on-title>
>
> 1. {% tool [GROMACS initial setup](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_setup/gmx_setup/2020.4+galaxy0) %} with the following parameters:
>    - *"PDB input file"*: 'Protein (PDB)' file
>    - *"Force field"*: `AMBER99SB`
>    - *"Water model"*: `TIP3P`
>    - *"Generate detailed log"*: `Yes`
>
>    > <comment-title></comment-title>
>    > A force field is essentially a function to calculate the potential energy of a system, based on various empirical parameters (for the atoms, bonds, charges, dihedral angles and so on). There are a number of families of forcefields; some of the most commonly used include CHARMM, AMBER, GROMOS and OpenFF (for a recent, accessible overview see ({% cite Lemkul2020 %}).
>    >
>    >
>    > A wide range of models exist for modeling water. Here we are using the common TIP3P model, which is an example of a 'three-site model' - so-called because the molecule is modeled using three points, corresponding to the three atoms of water. (Four- and five-site models include additional 'dummy atoms' representing the negative charges of the lone pairs of the oxygen atom).
>    {: .comment}
>
{: .hands_on}

The tool produces four outputs: a GRO file (containing the coordinates of the protein), a TOP file (containing other information, including on charges, masses, bonds and angles), an ITP file (which will be used to restrain the protein position in the equilibration step later on), and a log for the tool.

Please note all GROMACS tools output a log. Generally, you only need to look at this when a job fails. These provide useful information for debugging if we encounter any problems.


### Generate a topology for the ligand

To generate a topology for the ligand, we will use the **acpype** {% icon tool %} tool ({% cite SousadaSilva2012 %}). This provides a convenient interface to the AmberTools suite and allows us to easily create the ligand topology in the format required by GROMACS.

Inspecting the contents of the `Ligand (PDB)` file shows that it contains no hydrogen atoms. These need to be added before the topology can be calculated. The **Compound conversion** {% icon tool %} (which is based on OpenBabel) can be used to achieve this.

> <hands-on-title>Generate ligand topology</hands-on-title>
>
> 1. {% tool [Compound conversion](toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy0) %} with the following parameters:
>    - *"Molecular input file"*: 'Ligand (PDB)'
>    - *"Output format"*: `Protein Data Bank format (pdb)`
>    - *"Add hydrogens appropriate for pH"*: `7.0`
> 2. Rename the output file to `Hydrated ligand (PDB)`.
> 3. {% tool [Generate MD topologies for small molecules](toolshed.g2.bx.psu.edu/repos/chemteam/ambertools_acpype/ambertools_acpype/21.3+galaxy0) %} with the following parameters:
>    - *"Input file"*: 'Hydrated ligand (PDB)'
>    - *"Charge of the molecule"*: `0`
>    - *"Multiplicity"*: `1`
>    - *"Force field to use for parameterization"*: `gaff`
>    - *"Charge method"*: `bcc (default)`
>    - *"Save GRO file?"*: `Yes`
>
{: .hands_on}

Here, we use GAFF (general AMBER force field), which is a generalized AMBER force field which can be applied to almost any small organic molecule.

We select charge and multiplicity as appropriate. The ligand studied here is neutral, so the charge is 0. The multiplicity is 1, which will be the case for every simple organic molecule we encounter; only if we deal with more exotic species such as metal complexes or carbenes will we need to consider higher values.

Having generated topologies, we now need to combine them, define the box
which contains the system, add solvent and ions, and perform an energy
minimization step.

<!-- ## Solvation and energy minimization -->


## Combine topology and GRO files

While we have separate topology and structure files for both protein and ligand, we need to combine them into a single set of files to continue with the simulation setup. A dedicated Galaxy tool is provided for this, using the Python library ParmEd ({% cite Swails2016 %}).

> <hands-on-title>Combine GRO and topology files</hands-on-title>
>
> 1. {% tool [Merge GROMACS topologies](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_merge_topology_files/gmx_merge_topology_files/3.2.0+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Protein topology (TOP) file"*: `TOP` file created by the **GROMACS initial setup** tool
>    - {% icon param-file %} *"Ligand topology (TOP or ITP file)"*: `Topology` created by the **acpype** tool
>    - {% icon param-file %} *"Protein structure (GRO) file"*: `GRO` file created by the **GROMACS initial setup** tool
>    - {% icon param-file %} *"Ligand structure (GRO) file"*: `Structure file (GRO format)` created by the **acpype** tool
>
{: .hands_on}


## Create the simulation box

The next step, once combined coordinate (GRO) and topology (TOP) files have been created, is to create a simulation box in which the system is situated.

> <hands-on-title>Create simulation box</hands-on-title>
>
> 1. {% tool [GROMACS structure configuration](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_editconf/gmx_editconf/2020.4+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Input structure"*: `System GRO file` (Input dataset)
>    - *"Configure box?"*: `Yes`
>        - *"Box dimensions in nanometers"*: `1.0`
>        - *"Box type"*: `Triclinic`
>    - *"Generate detailed log"*: `Yes`
>
>
>    > <comment-title></comment-title>
>    >
>    > This tool returns a new GRO structure file, containing the same coordinates as before, but defining a simulation box such that every atom is a minimum of 1 nm from the box boundary. A distance of at least 1 nm is recommended to avoid interactions between the protein and its mirror image. On the other hand, increasing the box size too far will increase the simulation time, due to the greater number of solvent molecules which need to be treated. A variety of box shapes are available to choose: we select triclinic, as it provides the most efficient packing in space and thus fewer computational resources need to be devoted to simulation of solvent.
>    {: .comment}
>
{: .hands_on}



## Solvation

The next step is solvation of the newly created simulation box - as we are simulating under biological conditions, we use water as the solvent. Note that the system is charged (depending on the pH) - the solvation tool also adds sodium or chloride ions (replacing existing water molecules) as required to neutralize this.

> <hands-on-title>Solvation</hands-on-title>
>
> 1. {% tool [GROMACS solvation and adding ions](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_solvate/gmx_solvate/2020.4+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"GRO structure file"*: output of **GROMACS structure configuration** {% icon tool %}
>    - {% icon param-file %} *"Topology (TOP) file"*: `TOP` output of **Merge GROMACS topologies**
>    - *"Water model for solvation"*: `SPC (generic three-point model)`
>    - *"Add ions to neutralize system"*: `Yes, add ions`
>    - *"Specify salt concentration (sodium chloride) to add, in mol/liter"*: `0`
>    - *"Generate detailed log"*: `Yes`
> 2. Rename the outputs to `Solvated GRO` and `Solvated TOP`.
>
{: .hands_on}


## Energy minimization

After the solvation step, parameterization of the system is complete and preparatory simulations can be performed. The first of theses is energy minimization, which can be carried out using the **GROMACS energy minimization** {% icon tool %} tool.

> <question-title></question-title>
>
> 1. What is the purpose of energy minimization?
>
> > <solution-title></solution-title>
> >
> > 1. Running an energy minimization (EM) algorithm relaxes the structure by removing any steric clashes or unusual geometry which would artificially raise the energy of the system.
> >
> {: .solution}
>
{: .question}


> <hands-on-title>Energy minimization</hands-on-title>
>
> 1. {% tool [GROMACS energy minimization](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_em/gmx_em/2020.4+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"GRO structure file."*: GRO output of **GROMACS solvation and adding ions** {% icon tool %}
>    - {% icon param-file %} *"Topology (TOP) file."*: TOP output of **GROMACS solvation and adding ions** {% icon tool %}
>    - *"Parameter input"*: `Use default (partially customisable) setting`
>        - *"Number of steps for the MD simulation"*: `50000`
>        - *"EM tolerance"*: `1000.0`
>    - *"Generate detailed log"*: `Yes`
>    - Rename output to `Minimized GRO file`
>
{: .hands_on}

The EM tolerance here refers to the maximum force which will be allowed in a minimized system. The simulation will be terminated when the maximum force is less than this value, or when 50000 steps have elapsed.

As an aside, we can use the `Extract energy components` tool to plot the convergence of the potential energy during the minimization.

> <hands-on-title>Checking EM convergence</hands-on-title>
>
> 1. {% tool [Extract energy components with GROMACS](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_energy/gmx_energy/2020.4+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"EDR file."*: EDR output of **GROMACS energy minimization** {% icon tool %}
>    - *"Terms to calculate"*: `Potential`
>    - *"Output format"*: `Galaxy tabular`
> 2. On the output tabular file, click on the 'Visualize this data' icon. This provides a range of visualization options. Select 'Line chart (jqPlot)'.
> 3. In the visualization window which appears, click on `Select data.` Enter the following parameters:
>    - *"Provide a label"*: `Energy potential`
>    - *"Values for x-axis"*: `Column: 1`
>    - *"Values for y-axis"*: `Column: 2`
>
{: .hands_on}

The resulting plot should resemble the figure below. The system first drops rapidly in energy, before slowly converging on the minimized state.

![Snapshot of Energy potential during the EM simulation]({% link topics/computational-chemistry/images/empot.png %} "Energy potential during the EM simulation. <a href="https://usegalaxy.eu/u/sbray/v/em-pot">Click to view</a> as a Galaxy visualization")

## Equilibration

We now carry out equilibration in two stages: NVT and NPT. This is discussed at greater length in the basic GROMACS tutorial. Equilibration requires restraining the protein structure - we use the ITP file produced by the initial setup tool for this.

Simulation under the NVT ensemble allows the solvent to be brought to the desired temperature, while simulation under the NPT ensemble bring the solvent to the correct pressure.

> <comment-title>More detail about equilibration</comment-title>
>
> At this point equilibration of the solvent around the solute (i.e. the protein) is necessary. This is performed in two stages: equilibration under an NVT (or isothermal-isochoric) ensemble, followed by an NPT (or isothermal-isobaric) ensemble. Use of the NVT ensemble entails maintaining constant number of particles, volume and temperature, while the NPT ensemble maintains constant number of particles, pressure and temperature.
>
> For equilibration, the protein must be held in place while the solvent is allowed to move freely around it. This is achieved using the position restraint file (ITP) we created in system setup. When we specify this restraint, protein movement is not forbidden, but is energetically penalized.
>
{: .comment}


> <hands-on-title>NVT equilibration</hands-on-title>
>
> 1. {% tool [GROMACS simulation](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_sim/gmx_sim/2020.4+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"GRO structure file"*: `Minimized GRO file` (from energy minimization step)
>    - {% icon param-file %} *"Topology (TOP) file"*: `Solvated TOP`
>    - In *"Inputs"*:
>        - {% icon param-file %} *"Position restraint (ITP) file"*: ITP file produced by initial setup step.
>    - In *"Outputs"*:
>        - *"Trajectory output"*: `Return .xtc file (reduced precision)`
>        - *"Structure output"*: `Return .gro file`
>        - *"Produce a checkpoint (CPT) file"*: `Produce CPT output`
>        - *"Produce an energy (EDR) file"*: `Produce EDR output`
>    - In *"Settings"*:
>        - *"Parameter input"*: `Use default (partially customisable) setting`
>            - *"Bond constraints (constraints)"*: `All bonds (all-bonds).`
>            - *"Temperature /K"*: `300`
>            - *"Step length in ps"*: `0.001`
>            - *"Number of steps that elapse between saving data points (velocities, forces, energies)"*: `1000`
>            - *"Number of steps for the simulation"*: `50000`
>    - *"Generate detailed log"*: `Yes`
>
>
{: .hands_on}

Once the NVT equilibration is complete, it is worth using the `Extract energy components` tool again to check whether the system temperature has converged on 300 K. This can be done as described for energy minimization, this time specifying `Temperature` under `Terms to calculate` rather than `Potential`. The plot should show the temperature reaching 300 K and remaining there, albeit with some fluctuation.

Having stabilized the temperature of the system with NVT equilibration,
we also need to stabilize the pressure of the system. We therefore
equilibrate again using the NPT (constant number of particles, pressure,
temperature) ensemble.

Note that we can continue where the last simulation left off (with new
parameters) by using the checkpoint (CPT) file saved at the end of the
NVT simulation.

> <hands-on-title>NPT equilibration</hands-on-title>
>
> 1. {% tool [GROMACS simulation](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_sim/gmx_sim/2020.4+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"GRO structure file"*: GRO output of **GROMACS simulation** {% icon tool %} (NVT equilibration)
>    - {% icon param-file %} *"Topology (TOP) file"*: `Solvated TOP`
>    - In *"Inputs"*:
>        - {% icon param-file %} *"Checkpoint (CPT) file"*: Output of **GROMACS simulation** {% icon tool %} (NVT equilibration))
>        - {% icon param-file %} *"Position restraint (ITP) file"*: ITP file produced by initial setup step.
>    - In *"Outputs"*:
>        - *"Trajectory output"*: `Return .xtc file (reduced precision)`
>        - *"Structure output"*: `Return .gro file`
>        - *"Produce a checkpoint (CPT) file"*: `Produce CPT output`
>        - *"Produce an energy (EDR) file"*: `Produce EDR output`
>    - In *"Settings"*:
>        - *"Ensemble"*: `Isothermal-isobaric ensemble (NPT)`
>        - *"Parameter input"*: `Use default (partially customisable) setting`
>            - *"Bond constraints (constraints)"*: `All bonds (all-bonds).`
>            - *"Temperature /K"*: `300`
>            - *"Step length in ps"*: `0.001`
>            - *"Number of steps that elapse between saving data points (velocities, forces, energies)"*: `1000`
>            - *"Number of steps for the simulation"*: `50000`
>    - *"Generate detailed log"*: `Yes`
>
>
{: .hands_on}

After the NPT equilibration is complete, **Extract energy components** {% icon tool %} can be used again to view the pressure of the system. This is done as described for energy minimization, specifying `Pressure` under `Terms to calculate`. The plot should show convergence on 1 bar and remain there, although some fluctuation is expected.

## Production simulation

We can now remove the restraints and continue with the production simulation. The simulation will run for 1 million steps, with a step size of 1 fs, so will have a total length of 1 ns. This is rather short compared to the state-of-the-art, but sufficient for the purposes of a tutorial. For longer-scale simulations, the tool can be used multiple times (with the checkpoint file) to continue the existing simulation.

> <hands-on-title></hands-on-title>
>
> 1. {% tool [GROMACS simulation](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_sim/gmx_sim/2020.4+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"GRO structure file"*: Output of **GROMACS simulation** {% icon tool %} (NPT equilibration)
>    - {% icon param-file %} *"Topology (TOP) file"*: `Solvated TOP`
>    - In *"Inputs"*:
>        - {% icon param-file %} *"Checkpoint (CPT) file"*: Output of **GROMACS simulation** {% icon tool %} (NPT simulation))
>    - In *"Outputs"*:
>        - *"Trajectory output"*: `Return .xtc file (reduced precision)`
>        - *"Structure output"*: `Return .gro file`
>        - *"Produce a checkpoint (CPT) file"*: `Produce CPT output`
>    - In *"Settings"*:
>        - *"Ensemble"*: `Isothermal-isobaric ensemble (NPT)`
>        - *"Parameter input"*: `Use default (partially customisable) setting`
>            - *"Temperature /K"*: `300`
>            - *"Step length in ps"*: `0.001`
>            - *"Number of steps that elapse between saving data points (velocities, forces, energies)"*: `1000`
>            - *"Number of steps for the simulation"*: `1000000`
>    - *"Generate detailed log"*: `Yes`
>
>
{: .hands_on}


# Analysis

After the completion of the simulation, the following questions arise: 1) is the simulation converged enough, and 2) what interesting molecular properties are observed. To answer these questions, an analysis of the GROMACS simulation outputs (structure and trajectory file) will be carried out using Galaxy tools developed for computational chemistry ({% cite senapathi_biomolecular_2019 %}) based on popular analysis software, such as MDAnalysis ({% cite michaudagrawal_mdanalysis_2011 %}), MDTraj ({% cite mcgibbon_mdtraj_2015 %}), and  Bio3D ({% cite skjaerven_integrating_2014 %}). These tools output both tabular files as well as a variety of attractive plots.


## Create PDB file needed by most analysis tools

Before beginning a detailed analysis, the structure and trajectory files generated previously need to be converted into different formats. First, convert the structural coordinates of the system in GRO format into PDB format. This PDB file will be used by most analysis tools as a starting structure.  Next, convert the trajectory from XTC to DCD format, as a number of tools (particularly those based on Bio3D) only accept trajectories in DCD format.

> <hands-on-title>Convert coordinate format</hands-on-title>
>
> 1. {% tool [GROMACS structure configuration](toolshed.g2.bx.psu.edu/repos/chemteam/gmx_editconf/gmx_editconf/2020.4+galaxy0) %} with the following parameters:
>    - *"Output format"*: `PDB file`
>    - *"Configure box?"*: `No`
>
>    > <comment-title></comment-title>
>    >
>    > This tool can also be used to carry out initial setup (as discussed in the simulation methods section) for GROMACS simulations and convert from PDB to GRO format.
>    {: .comment}
>
{: .hands_on}


## Convert trajectory to DCD format

Convert from XTC to DCD format. A number of the analysis tools being used have been built to analyse trajectories in CHARMM's DCD format.

> <hands-on-title>Convert trajectory format</hands-on-title>
>
> 1. {% tool [MDTraj file converter](toolshed.g2.bx.psu.edu/repos/chemteam/md_converter/md_converter/1.9.6+galaxy0) %} with the following parameters:
>    - *"Output format"*: `DCD file`
>
>    > <comment-title></comment-title>
>    >
>    > This tool can also be used to interconvert between several trajectory formats.
>    {: .comment}
>
{: .hands_on}


## RMSD analysis - protein

The Root Mean Square Deviation (RMSD) and Root Mean Square Fluctuation (RMSF) are calculated to check the stability and conformation of the protein and ligand through the course of the simulation.
RMSD is a standard measure of structural distance between coordinate
sets that measures the average distance between a group of atoms. The
RMSD of the Cα atoms of the protein backbone is calculated here and
is a measure of how much the protein conformation has changed between different time points in the trajectory.


> <hands-on-title>RMSD Analysis</hands-on-title>
>
> 1. {% tool [RMSD Analysis](toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_rmsd/bio3d_rmsd/2.3.4) with the following parameters:
>    - *"Select domains"*: `C-alpha`
>
>    > <comment-title></comment-title>
>    >
>    > Note that for more complex systems, you may need to consider a more focused selection. For example, if you have a ligand that is a protein consider modifying this selection.
>    {: .comment}
>
{: .hands_on}

![RMSD timeseries Hsp90](../../images/htmd_analysis_rmsd1_series.png "RMSD timeseries for the Hsp90 Cα atoms")

![RMSD histogram Hsp90](../../images/htmd_analysis_rmsd1_histo.png "RMSD histogram for the Hsp90 Cα atoms")

The RMSD time series for the protein shows a thermally stable and equilibrated structure that plateaus at 1.0Å with an average RMSD between 0.8Å and 1.0Å. There are no large conformational changes during the simulation. The RMSD histogram confirms this. Note these graphs are automatically created by Galaxy as part of the tool's outputs.


## RMSD analysis - ligand

Calculating the RMSD of the ligand is necessary to check if it is stable in the active site and to identify possible binding modes. If the ligand is not stable, there will be large fluctuations in the RMSD.

For the RMSD analysis of the ligand, the `Select domains` parameter of the tool can for convenience be set to `Ligand`; however, this automatic selection sometimes fails. The other alternative, which we apply here, is to specify the `Residue ID` in the textbox provided. In this example the ligand's Residue ID is `G5E`. The output is the requested RMSD data as a time series, the RMSD plotted as a time series and as a histogram.


> <hands-on-title>RMSD analysis</hands-on-title>
>
> 1. {% tool [RMSD Analysis](toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_rmsd/bio3d_rmsd/2.3.4) with the following parameters:
>    - {% icon param-file %} *"DCD trajectory input"*: `output` (output of **MDTraj file converter** {% icon tool %})
>    - {% icon param-file %} *"PDB input"*: `output` (output of **GROMACS structure configuration** {% icon tool %})
>    - *"Select domains"*: `Residue ID`
>        - *"Residue ID"*: `G5E`
>
>
{: .hands_on}

In our case the ligand is stable with a single binding mode. The RMSD fluctuates around 0.3Å, with a slight fluctuation near the end of the simulation. This is more clearly seen in the histogram. The conformation seen during simulation is very similar to that in the crystal structure and the ligand is stable in the active site.

![RMSD timeseries Hsp90 ligand](../../images/htmd_analysis_rmsd2_series.png "RMSD timeseries for the Hsp90 Residue ID G5E (ligand)")

![RMSD histogram Hsp90 ligand](../../images/htmd_analysis_rmsd2_histo.png "RMSD histogram for the Hsp90 Residue ID G5E (ligand)")


## RMSF analysis

The Root Mean Square Fluctuation (RMSF) is valuable to consider, as it represents the deviation at a reference position over time. The fluctuation in space of particular amino acids in the protein are considered. The Cα of the protein, designated by `C-alpha`, is a good selection to understand the change in protein structure. Depending on the system these fluctuations can be correlated to experimental techniques including Nuclear Magnetic Resonance (NMR) and M\"{o}ssbauer spectroscopy ({% cite berjanskii_nmr_2006 %}, {% cite kuzmanic_determination_2010 %}). The output from the tools is the requested RMSF data and the RMSF plotted as a time series.

> <hands-on-title>RMSF analysis</hands-on-title>
>
> 1. {% tool [RMSF Analysis](toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_rmsf/bio3d_rmsf/2.3.4) with the following parameters:
>    - {% icon param-file %} *"DCD trajectory input"*: `output` (output of **MDTraj file converter** {% icon tool %})
>    - {% icon param-file %} *"PDB input"*: `output` (output of **GROMACS structure configuration** {% icon tool %})
>    - *"Select domains"*: `C-alpha`
>
>
{: .hands_on}

![RMSF Hsp90](../../images/htmd_analysis_rmsf.png "RMSF(Å) vs the residue position. Large fluctuations occur at various positions, which correspond to flexible loop regions on the surface of the protein.")

When considering the RMSF, fluctuations greater than 1.0Å are of interest; for example see the fluctuations near residue positions 50, 110 and 160.  Inspecting the structure with molecular visualization software such as VMD, these can be seen to correspond to flexible loop regions on the protein surface. In addition, very large fluctuations are seen for the C-terminus; this is common and no investigation is needed.

Note that the first few residues of this protein are missing in the PDB, and therefore residue position 0 in the RMSF corresponds to position 17 in the Hsp90 FASTA primary sequence. This is a fairly common problem that can occur with molecular modeling of proteins, where there may be missing residues at the beginning or within the sequence.



## PCA analysis

Principal component analysis (PCA) converts a set of correlated
observations (movement of selected atoms in protein) to a set of principal
components (PCs) which are linearly independent (or uncorrelated). Here several related tools are used.
The PCA tool calculates the PCA in order to determine the relationship between statistically meaningful conformations (major global motions) sampled during the trajectory. The Cα carbons of the protein backbone are again a good selection for this purpose.  Outputs include the PCA raw data and figures of the relevant principal components (PCs) as well as an eigenvalue rank plot which is used to visualize the proportion of variance due to each principal component (remembering that the PCs are ranked eigenvectors based on the variance).
Having discovered the principal components usually these are visualized. The PCA visualization tool will create trajectories of specific principal components which can be viewed in a molecular viewer such as VMD ({% cite hump_vmd_1996 %}) or NGL viewer ({% cite Rose2018ngl %}). We also consider the PCA cosine content which when close to 1 indicates that the simulation is not converged and a longer simulation is needed. For values below 0.7, no statement can be made about convergence or lack thereof.

> <hands-on-title>PCA</hands-on-title>
>
> 1. {% tool [PCA](toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_pca/bio3d_pca/2.3.4) %} with the following parameters:
>    - {% icon param-file %} *"DCD trajectory input"*: `output` (output of **MDTraj file converter** {% icon tool %})
>    - {% icon param-file %} *"PDB input"*: `output` (output of **GROMACS structure configuration** {% icon tool %})
>    - *"Select domains"*: `C-alpha`
>
>
{: .hands_on}


![PCA Hsp90](../../images/htmd_analysis_pca.png "PCA results which include graphs of PC2 vs PC1, PC2 vs PC3, PC3 vs PC1  colored from blue to red in order of time, and an eigenvalue rank plot. In the eigenvalue plot the cumulative variance is labeled for each data point.")

The first three principal components are responsible for 32.8% of the total variance, as seen in the eigenvalue rank plot. The first principal component (PC1) accounts for 15.4% of the variance (see PC1 vs PC2 and eigenvalue rank plots). Visualization of PC1 using VMD shows a rocking motion and wagging of the C-terminus.


> <hands-on-title>PCA cosine content calculation</hands-on-title>
>
> 1. {% tool [Cosine Content](toolshed.g2.bx.psu.edu/repos/chemteam/mdanalysis_cosine_analysis/mdanalysis_cosine_analysis/1.0.0+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"DCD/XTC trajectory input"*: `output` (output of **MDTraj file converter** {% icon tool %})
>    - {% icon param-file %} *"PDB/GRO input"*: `output` (output of **GROMACS structure configuration** {% icon tool %})
>
>
{: .hands_on}

The PCA cosine content of the dominant motion related to PC1 is 0.93, indicating that the simulation is not fully converged. This is expected due to the short simulation length. For production level simulations, it is the norm to extend simulations to hundreds of nanoseconds in length, if not microseconds. As this tutorial is designed to be carried out on public webservers, we limit simulations to 1 ns, as we cannot provide a large amount of computational resources for training purposes.

> <hands-on-title>PCA visualisation</hands-on-title>
>
> 1. {% tool [PCA visualization](toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_pca/bio3d_pca/2.3.4) %} with the following parameters:
>    - {% icon param-file %} *"DCD trajectory input"*: `output` (output of **MDTraj file converter** {% icon tool %})
>    - {% icon param-file %} *"PDB input"*: `output` (output of **GROMACS structure configuration** {% icon tool %})
>    - *"Select domains"*: `C-alpha`
>
>
{: .hands_on}

![PC1 Hsp90 gif](../../images/htmd_analysis_pc1_hsp90.gif "PC1 motion for Hsp90")

## Hydrogen bond analysis

Hydrogen bonding interactions contribute to binding and are worth investigating, in particular persistent hydrogen bonds. All possible hydrogen bonding interactions between the two selected regions, here the protein and the ligand, are investigated over time using the VMD hydrogen bond analysis tool included in Galaxy. Hydrogen bonds are identified and in the output the total number of hydrogen bonds and  occupancy over time is returned.

> <hands-on-title>Hydrogen bond analysis</hands-on-title>
>
> 1. {% tool [Hydrogen Bond Analysis using VMD](toolshed.g2.bx.psu.edu/repos/chemteam/vmd_hbonds/vmd_hbonds/1.9.3) %} with the following parameters:
>    - {% icon param-file %} *"DCD/XTC trajectory input"*: `output` (output of **MDTraj file converter** {% icon tool %})
>    - {% icon param-file %} *"PDB/GRO input"*: `output` (output of **GROMACS structure configuration** {% icon tool %})
>    - *"Selection 1"*: `protein`
>    - *"Selection 2"*: `resname G5E`
>
>
{: .hands_on}

The active site of this protein is quite hydrophobic, yet multiple hydrogen bonds were identified. The hydrogen bond between aspartate-93 and the ligand (as identified in the crystal structure) was found to be persistent, meeting the hydrogen bond criteria for 89.22% of the simulation. A hydrogen bond between the ligand and the carbonyl group of glycine-97 was found to have a 15.27% occupancy. Hydrogen bonding interactions with threonine-184, asparagine-51 and lysine-58 were also observed but these are not persistent and only present for a minority of the simulation. These values can be accessed from the 'Percentage occupancy of the H-bond' output of the hydrogen bond analysis tool.


# Optional: Automating high throughput calculations
Up until this step, Galaxy tools have been applied sequentially to datasets. This is useful to gain an understanding of the steps involved, but becomes tedious if the workflow needs to be run on multiple protein-ligand systems. Fortunately, Galaxy allows entire workflows to be executed with a single mouse-click, enabling straightforward high-throughput analyses.

We will demonstrate the high-throughput capabilities of Galaxy by running the workflow detailed so far on a further three ligands.

> <hands-on-title>High-throughput MD</hands-on-title>
>
> 1. Create a new history for running the high-throughput workflow and name it `Hsp90 HTMD simulation`
> 2. Upload the SD-file containing the new ligand structures from Zenodo  and rename it `Ligands (SDF)`
> 3. Import the simulation workflow from the European ({%cite eu_htmd_simulation_workflow %}) or the South African Galaxy server ({%cite  za_htmd_simulation_workflow %}).
> 4. Run the imported workflow with the following parameters:
>    - *"Send results to a new history"*: `Yes`
>    - *"History name results to a new history"*: `Hsp90 HTMD analysis`
>    - *"GRO input"*: 'Collection of GRO files produced by simulation workflow'
>    - *"XTC input"*: 'Collection of XTC files produced by simulation workflow'
>
{: .hands_on}


This process runs the entire simulation and analysis procedure described so far on the new set of ligands. It uses Galaxy's [collection feature]({% link topics/galaxy-interface/tutorials/collections/tutorial.md %}) to organize the data; each item in the history is a collection (essentially a directory containing multiple individual datasets) containing one file corresponding to each of the input ligands.

Note that the SD-file needs to contain ligands with the correct 3D coordinates for MD simulation. The easiest way to obtain these is using a molecular docking tool such as Autodock Vina ({% cite Trott2009 %}) or rDock ({% cite Ruiz2014 %}); tutorials and workflows are available for both of these from the Galaxy Training Network. As an example, the history in which the SD-file used in the HTMD workflow is generated (using AutoDock Vina) is provided ({% cite eu_6hhr %}).

Apart from manual setups or collections, there are several other alternatives which are helpful in scaling up workflows. Galaxy supports and provides training material for converting [histories to workflows]({% link topics/galaxy-interface/tutorials/history-to-workflow/tutorial.md %}), using [multiple histories]({% link topics/galaxy-interface/tutorials/history/tutorial.md %}), and the [Galaxy Application Programming Interface (API)]({% link topics/dev/tutorials/bioblend-api/slides.html %}). For beginners and users who prefer a visual interface, automation can be done using multiple histories and collections with the standard Galaxy user interface.

If you are able to write small scripts, you can automate everything you have learned here with the Galaxy API. This allows you to interact with the server to automate repetitive tasks and create more complex workflows (which may have repetition or branching). The simplest way to access the API is through the Python library BioBlend ({% cite sloggett_bioblend %}). An example Python script, which uses BioBlend to run the GROMACS simulation workflow for each of a list of ligands, is given in the hands-on box below.

> <hands-on-title>Bioblend script</hands-on-title>
>
>    ```
>from bioblend import galaxy
>
># Server and account details
>API_KEY = 'YOUR USEGALAXY.EU API KEY'
>gi = galaxy.GalaxyInstance(key=API_KEY,
>    url='https://usegalaxy.eu/')
>
># ID for GROMACS workflow
>workflow_id = 'adc6d049e9283789'
>
># Dataset IDs for ligands to dock
>ligands = {
># ligand_name: dataset ID,
>'lig1': '11ac94870d0bb33a79c5fa18b0fd3b4c',
># ...
>}
>
># Loop over ligands, invoking workflow
>for name, _id in ligands.items():
>    inv = gi.workflows.invoke_workflow(
>        workflow_id,
>        inputs={
>            '1': {'src': 'hda', 'id': _id}
>        },
>        history_name=f'HTMD run on {name}'
>    )
>    ```
>
{: .hands_on}

Another option is to use the Planemo command-line tool. Planemo has a very wide range of functions, including the ability to run Galaxy workflows from the command line. For a full overview, see the [tutorial](https://planemo.readthedocs.io/en/latest/running.html).

Assuming you have Planemo installed, you can create a file with the name `htmd-job.yml` with the following contents:

```
SDF file with (docked) ligands:
  class: File
  galaxy_id: '11ac94870d0bb33a79c5fa18b0fd3b4c'
  # path: ligands.sdf

```

You can specify the dataset either using its Galaxy ID, or with the path of a local file. The Galaxy ID is associated with the specific server you are using - in
this case `usegalaxy.eu`. You can get this ID by clicking on the `View details` icon of a dataset in your Galaxy history - it is provided in the `Dataset Information`
table as `History Content API ID`. Then run:

```
planemo run adc6d049e9283789 htmd-job.yml --galaxy_url https://usegalaxy.eu --galaxy_user_key YOUR_USEGALAXY_EU_API_KEY --history_name "HTMD with planemo"
```

Here `adc6d049e9283789` is the Galaxy ID associated with the workflow on `usegalaxy.eu`, but the `planemo run` command can also use a local workflow file.

If you now return to the web-browser, you should see a new history should have been created named `HTMD with planemo` and it should begin to fill with datasets.


# Conclusion


This tutorial provides a guide on how to study protein-ligand interaction using molecular dynamics in Galaxy. Performing such analyses in Galaxy makes it straightforward to set up, schedule and run workflows, removing much of the difficulty from MD simulation. Thus, the technical barrier to performing high-throughput studies is greatly reduced. Results are structured in the form of Galaxy histories or collections, and include ready-plotted diagrams, which ensure data can be easily understood and reproduced if necessary. Apart from streamlining the process for existing MD users, this tutorial should also prove useful as a pedagogical guide for educating students or newcomers to the field.

After completing the tutorial, the user will be familiar at a basic level with a range of MD analysis techniques, and understand the steps required for a typical MD simulation. Thus, they will be equipped to apply these tools to their own problems.


# Glossary and abbreviations

* Concepts:
  * **MD** - molecular dynamics simulation.
  * **NVT** - isothermal-isochoric ensemble, in which number of atoms (N), volume (V) and temperature (T) are kept constant. 
  * **NPT** - isothermal-isobaric ensemble, in which number of atoms (N), pressure (P) and temperature (T) are kept constant. 
  * **RMSD** - Root Mean Square Deviation
  * **RMSF** - Root Mean Square Fluctuation
  * **PCA** - Principal Component Analysis
* Software:
  * **GROMACS** (Groningen Machine for Chemical Simulation) - a molecular dynamics package mainly designed for simulations of proteins, lipids and nucleic acids. 
  * **NAMD** (Nanoscale Molecular Dynamics) - a molecular dynamics software designed for high-performance simulation of large biomolecular systems.
  * **CHARMM** (Chemistry at Harvard Macromolecular Mechanics) - a molecular modeling and simulation program for many-particle systems. It includes a comprehensive set of force fields to simulate biomolecules.
  * **VMD** (Visual Molecular Dynamics) - a molecular visualisation program. 
  * **ACPYPE** (AnteChamber Python Parser interface) - a tool that simplifies the generation of small molecule topologies and parameters for a variety of molecular dynamics programs, like GROMACS and CHARMM. It is based on the Antechamber software provided by AmberTools.
* Filetypes:
  * **PDB** - a commonly used filetype for storing 3-dimensional structural data of large biomolecules such as proteins and nucleic acids. Also stands for the Protein Data Bank, an online repository for such data.
  * **GRO** - a filetype similar to PDB, which contains a molecular structure in a format supported by GROMACS.
  * **DCD** - a trajectory file format supported by CHARMM and NAMD.
  * **TOP** - a GROMACS filetype for representing molecular topologies.
  * **ITP** - another GROMACS filetype for representing molecular topologies.
  * **TRR** - a lossless GROMACS trajectory file, storing coordinates, forces and velocity. 
  * **XTC** - a compressed GROMACS trajectory file, storing coordinates. 
* Force fields and water molecules:
  * **GROMOS** (GROningen Molecular Simulation) - the name of a force field for molecular dynamics simulation.
  * **AMBER** (Assisted Model building with Energy Refinement) - a set of molecular mechanical force fields for the simulation of biomolecules.
  * **OpenFF** (Open Force Field) - a Python toolkit used for the development and application of modern molecular mechanics force fields.
  * **GAFF** (General AMBER Force Field) - a force field developed to augment the other AMBER force fields, allowing augmentation of most pharmaceutical molecules. 
  * **TIP3P** - a water model that specifies rigid water molecules with charges and Lennard-Jones parameters assigned to each of the 3 atoms.
* Biological concepts:
  * **Hsp90** - Heat shock protein 90, a chaperone protein.
  * **ATP** - Adenosine triphosphate, the primary carrier of energy in cells.
  * **NTD** - N-Terminal Domain, the start of a protein or polypeptide chain where the free amine group is located.
  * **NMR** (Nuclear Magnetic Resonance) - an experimental technique for determining 3D molecular structure.

tools_json:
["Get PDB", "toolshed.g2.bx.psu.edu/repos/bgruening/get_pdb/get_pdb/0.1.0", "Search in textfiles (grep)", "toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/1.1.1", "GROMACS initial setup", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_setup/gmx_setup/2020.4+galaxy0", "Compound conversion", "toolshed.g2.bx.psu.edu/repos/bgruening/openbabel_compound_convert/openbabel_compound_convert/3.1.1+galaxy0", "Generate MD topologies for small molecules", "toolshed.g2.bx.psu.edu/repos/chemteam/ambertools_acpype/ambertools_acpype/21.3+galaxy0", "Merge GROMACS topologies", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_merge_topology_files/gmx_merge_topology_files/3.2.0+galaxy0", "GROMACS structure configuration", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_editconf/gmx_editconf/2020.4+galaxy0", "GROMACS solvation and adding ions", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_solvate/gmx_solvate/2020.4+galaxy1", "GROMACS energy minimization", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_em/gmx_em/2020.4+galaxy0", "Extract energy components with GROMACS", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_energy/gmx_energy/2020.4+galaxy0", "GROMACS simulation", "toolshed.g2.bx.psu.edu/repos/chemteam/gmx_sim/gmx_sim/2020.4+galaxy1", "MDTraj file converter", "toolshed.g2.bx.psu.edu/repos/chemteam/md_converter/md_converter/1.9.6+galaxy0", "RMSD Analysis", "toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_rmsd/bio3d_rmsd/2.3.4", "RMSF Analysis", "toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_rmsf/bio3d_rmsf/2.3.4", "PCA", "toolshed.g2.bx.psu.edu/repos/chemteam/bio3d_pca/bio3d_pca/2.3.4", "Cosine Content", "toolshed.g2.bx.psu.edu/repos/chemteam/mdanalysis_cosine_analysis/mdanalysis_cosine_analysis/1.0.0+galaxy0", "PCA visualization", "Hydrogen Bond Analysis using VMD", "toolshed.g2.bx.psu.edu/repos/chemteam/vmd_hbonds/vmd_hbonds/1.9.3"]


topic: visualisation
tutorial: jbrowse2
title: Genomic Data Visualisation with JBrowse2
difficulty: intermediate
time_estimation: 1h
tools_json: ["JBrowse2", "toolshed.g2.bx.psu.edu/repos/fubar/jbrowse2/jbrowse2/3.7.0+galaxy0"]
requirements_json: []
tags_json: ["gmod", "jbrowse2"]
content_hash: b87a1fb09e7f118d67d5368111eb1758
last_modified: 2026-05-27T09:01:50.060282
zenodo_link: https://doi.org/10.5281/zenodo.19554505
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/visualisation/tutorials/jbrowse2/tutorial.html

content:
> [JBrowse2](https://pmc.ncbi.nlm.nih.gov/articles/PMC10108523/) is a fast, embeddable genome browser built completely with JavaScript
> and HTML5, with optional run-once data formatting tools written in Perl.
>
{: .quote cite="https://jbrowse.org/jb2/"}

JBrowse is an open-source genomic browser developed to view and explore genomoc data interactively. The first version, released in 2009, gained popularity due to its simplicity and its ability to display data in various formats: GFF3/GTF for annotations, BAM for RNA-seq alignements, BigWig for coverage, and VCF for variants.

With JBrowse2, released in 2020, the possibilities have been significantly expanded. This new version now allows for visualization of Hi-C data, offering a 3D representation of chromosomal interactions, a major asset forstudying genome structure.

JBrowse2 also introduces circular view, ideal for analyzing bacterial genomes or plasmids, as well as dotplots, which facilitate the comparison of syntenic regions across species. These new features enable a more in-depth and visual analysis of relationship between genomes.

Another key advantage is support for the CRAM format, a more compact alternative to BAM. This format reduces storage space while retaining the same functionality, which is particularly useful for large datasets.

For more information on JBrowse2, please visit this [website](https://jbrowse.org/jb2/).

In this tutorial, we will use JBrowse2, which is available on Galaxy.


> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Data Upload

> <hands-on-title>Getting the data</hands-on-title>
>
> 1. Create and name a new history for this tutorial.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the datasets we will visualize:
>
>    ```
>    https://zenodo.org/records/19554505/files/data.bw
>    https://zenodo.org/records/19554505/files/merlin-sample.cram
>    https://zenodo.org/records/19554505/files/merlin.fa
>    https://zenodo.org/records/19554505/files/merlin.gtf
>    https://zenodo.org/records/19554505/files/merlin.hic
>    https://zenodo.org/records/19554505/files/test.vcf
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
{: .hands_on}

The data for this tutorial is a dataset from *Citrobacter phage Merlin*.

# Simple Gene Tracks

If you have used JBrowse1 before, using JBrowse2 on Galaxy is very similar. If this is your first time, don't worry, it's easy to use. We'll start by adding the structural annotation as a track in JBrowse2.

If you’re not familiar with structural annotation, we recommend reading the [general introduction to annotation]({% link topics/genome-annotation/tutorials/introduction/slides.md %}#1) and following the tutorials on annotating genomes (e.g. [Bacterial Genome Annotation]({% link topics/genome-annotation/tutorials/bacterial-genome-annotation/tutorial.md %}), or [Genome annotation with Helixer]({% link topics/genome-annotation/tutorials/helixer/tutorial.md %})).

> <hands-on-title>Build the JBrowse2</hands-on-title>
>
> 1. {% tool [JBrowse2](toolshed.g2.bx.psu.edu/repos/fubar/jbrowse2/jbrowse2/3.7.0+galaxy0) %} with the following parameters:
>    - *"Reference genome to display"*: `Use a genome from history`
>        - {% icon param-file %} *"Select the reference genome"*: `merlin.fa`
>    - Click on *"Insert Track Category"*:
>        - {% icon param-repeat %} *"Track Category Label"*: call it `Structural annotation`
>            - Click on *"Insert Track"*:
>                    - *"Track Type"*: `GFF/GFF3/BED Features`
>                        - {% icon param-file %} *"GFF/GFF3/BED Track Data"*: `merlin.gtf`
> 2. Execute
>
> 3. View the output
>
> 4. Turn on both tracks of data.
>
>    > <tip-title>Slect or remove tracks</tip-title>
>    >
>    > * To view the tracks, you need to select them.
>    > * To do this, click on **Open track selector** in the top-left corner, as shown in blue in the screenshot.
>    > * A list of all your tracks will appear on the right. You can select or deselect tracks to view.
>    > ![sequencing tracks in JBrowse2](../../images/jbrowse2_select-remove_tracks.png "Select or remove some tracks")
>    >
>    {: .tip}
>
>
> 5. Navigate along the genome. Feel free to zoom in on specific areas.
>
>    > <tip-title> Navigating in Jbrowse </tip-title>
>    > - To **navigate along the genome**, use your mouse by left-clicking and dragging. Arrows for moving are also available.
>    > - To **zoom in** on an area, you have several options. The first is to use the magnifying glass icon. The second option is to select your zoom level using your mouse.
>    > - If an area interests you, you can highlight it. Select the area in question, right-click, and select **“Bookmark region”**. You can change the color if you want to distinguish between different areas.
>    {: .tip}
>
> 6. By clicking on a gene, you will be able to get information about it, such as the type (CDS, exon, gene), its length and its position.
>
>    ![sequencing tracks in JBrowse2](../../images/jbrowse2-gtf-track.png "All of the sequencing tracks in JBrowse2. Try exploring!")
>    ![sequencing tracks in JBrowse2](../../images/jbrowse2-gtf-track_info.png "Example of information that can be obtained by clicking on a gene")
>
{: .hands_on}


If you are not familiar with JBrowse2, here are a few important points:
- To **navigate along the genome**, use your mouse by left-clicking and dragging. Arrows for moving are also available.
- To **zoom in** on an area, you have several options. The first is to use the magnifying glass icon. The second option is to select your zoom level using your mouse.
- If an area interests you, you can highlight it. Select the area in question, right-click, and select **“Bookmark region”**. You can change the color if you want to distinguish between different areas.

# Complex Gene Tracks

As mentioned in the introduction, JBrowse2 supports a wide range of input formats for tracks. We previously launched JBrowse2 with a single track, so let’s launch it with multiple tracks.

> <hands-on-title>Build a JBrowse2 with multiple tracks</hands-on-title>
>
> 1. {% tool [JBrowse2](toolshed.g2.bx.psu.edu/repos/fubar/jbrowse2/jbrowse2/3.7.0+galaxy0) %} with the following parameters:
>    - *"Reference genome to display"*: `Use a genome from history`
>        - {% icon param-file %} *"Select the reference genome"*: `merlin.fa`
>    - Click on *"Insert Track Category"*:
>        - {% icon param-repeat %} *"Track Category Label"*: call it `Structural annotation`
>            - Click on *"Insert Track"*:
>                    - *"Track Type"*: `GFF/GFF3/BED Features`
>                        - {% icon param-file %} *"GFF/GFF3/BED Track Data"*: `merlin.gtf`
>    - Click on *"Insert Track Category"*:
>        - {% icon param-repeat %} *"Track Category Label"*: call it `Coverage`
>            - Click on *"Insert Track"*:
>                 - *"Track Type"*: `BigWig`
>                    - {% icon param-file %} *"BigWig Track Data"*: `data.bw`
>    - Click on *"Insert Track Category"*:
>        - {% icon param-repeat %} *"Track Category Label"*: call it `Alignement RNA-seq`
>            - Click on *"Insert Track"*:
>                 - *"Track Type"*: `CRAM`
>                    - {% icon param-file %} *"CRAM Track Data"*: `merlin-sample.cram`
>    - Click on *"Insert Track Category"*:
>        - {% icon param-repeat %} *"Track Category Label"*: call it `VCF SNPs`
>            - Click on *"Insert Track"*:
>                 - *"Track Type"*: `VCF SNPs`
>                    - {% icon param-file %} *"CRAM Track Data"*: `test.vcf`
>
> 2. Execute
> 3. View the output
>
>    ![complex gene tracks in JBrowse2](../../images/JBrowse2_complex_gene_tracks.png "Example view of JBrowse2 with multiple tracks (reference genome, structural annotation, RNA-seq alignment, etc.).")
>
{: .hands_on}

# Hi-C Data Visualization

The .hic file is a standard format used to store Hi-C interaction matrices. These matrices represent physical contacts between chromosomal regions within a genome, thereby revealing its 3D spatial organization. Unlike linear data (such as GFF annotations or BAM alignments), Hi-C provides insight into how regions of the genome are organized and physically interact in space, for example, to form loops or structural domains.


> <hands-on-title>Build the JBrowse2 with Hi-C data</hands-on-title>
>
> 1. {% tool [JBrowse2](toolshed.g2.bx.psu.edu/repos/fubar/jbrowse2/jbrowse2/3.7.0+galaxy0) %} with the following parameters:
>    - *"Reference genome to display"*: `Use a genome from history`
>        - {% icon param-file %} *"Select the reference genome"*: `merlin.fa`
>    - Click on *"Insert Track Category"*:
>        - {% icon param-repeat %} *"Track Category Label"*: call it `Hi-C`
>            - Click on *"Insert Track"*:
>                    - *"Track Type"*: `HiC`
>                        - {% icon param-file %} *"Hi-C Track Data"*: `merlin.hic`
> 2. Execute
>
> 3. View the contents of the file
>
>    ![Screenshot of JBrowse2](../../images/jbrowse2-hic-track.png "Hi-C heatmap showing chromosomal interactions in the 18,724–43,842 bp region of the *Citrobacter phage Merlin* genome.")
>
{: .hands_on}

In this image, we can see the chromosomal interactions in the 18,724–43,842 bp region of the *Citrobacter phage Merlin* genome.

How should we interpret our results?

A heatmap contains several patterns:
- The diagonal line shows local interactions between neighboring regions on the linear genome.
- The red blocks indicate long-range interactions between distant regions on the linear sequence. These blocks are often symmetrical and form an “X” or “checkerboard” pattern. This may correspond to:
  - DNA loops formed during replication or packaging of the viral genome.
  - Interactions between regulatory elements (e.g., promoters) that can influence gene transcription.
- Arcs connect regions in physical interaction, which can help us understand how the viral genome is organized within the capsid.


To explore other examples of Hi-C data (such as eukaryotic genomes or more complex matrices), we invite you to check out the [official JBrowse 2 demo](https://jbrowse.org/code/jb2/latest/?config=test_data%2Fconfig_demo.json&session=local-FcsRCX4o_A).

# Conclusion

This does not exhaustively cover JBrowse2, and the tool is more extensible than can be easily documented, but hopefully these examples are illustrative and can give you some ideas about your next steps. If you'd like to see more examples of visualizations, you can find them on the [JBrowse2 website](https://jbrowse.org/jb2/demos/).


topic: visualisation
tutorial: circos
title: Visualisation with Circos
difficulty: intermediate
time_estimation: 2h
requirements_json: []
tags_json: []
content_hash: 1c66d6f1f5a93d9c886c2e6a7c61a080
last_modified: 2026-05-15T22:28:18.635278
zenodo_link: https://zenodo.org/record/4494146
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/visualisation/tutorials/circos/tutorial.html

content:
Circos ({% cite krzywinski2009circos %}) is a software package for visualizing data in a circular layout. This makes Circos ideal for exploring relationships between objects or positions. Circos plots have appeared in thousands of scientific publications. Although originally designed for visualizing genomic data, it can create figures from data in any field. In this tutorial we discuss the Galactic Circos ({% cite Rasche_2020 %}) tool which enables you to access Circos via the convenient Galaxy interface, and even produce plots as part of your workflows!

![Panel of example Circos images](../../images/circos-sample-panel.png)

In this tutorial you will learn how to create such publication-ready Circos plots within Galaxy, and hopefully you can draw inspiration from these for developing your own plots.

> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}


# Background

Circos supports various different plot types, such as histograms, scatter plots and heat maps. Each Circos plot may contain multiple tracks containing different sub-plots, making it ideal for visualisation of high-dimensional data.


## Circos is an Iterative Process

Publication quality circos plots are *rarely* produced on the first try. Developing a quality Circos plot involves a lot of trial and error to find the best way to convey specific pieces of your data to your audience. Usually you will build up the circos plot one track at a time, and play around with different parameter settings until the plot looks exactly like you want it to:

![Gif of a circos plots through the development process](../../images/circos2.gif "Circos plots are an iterative process, requiring many iterative steps, each improving your plot and getting you closer to a final image.")


Circos is an extremely flexible but also very complex tool. The Galaxy Circos tool covers the most commonly used Circos features, but in order to avoid becoming too complex, it does not expose every single configuration option available in Circos. However, the Galaxy Circos tool allows you to download the full set of configuration files it uses, allowing you to manually tweak the plot further.


> <comment-title>Circos tutorials</comment-title>
>
> To learn more about using Circos outside of Galaxy (e.g. for tweaking the
> Circos configuration output by the Galaxy tool), there are a wide range
> of tutorials available from the [Circos website](http://circos.ca)
{: .comment}


# Circos Basics

## Ideogram

The ideogram depicts your major data classes. For genomics data this is usually chromosomes, but could also be species, or genes, or another resolution level depending on what relationships you want to show. For non-genomic data this could be individuals in a population, countries, or any other major facet of your data that you want to use for grouping.

![Examples of ideograms in Circos](../../images/circos/ideograms_examples.png "The ideogram shows the objects for which you will plot data. Cytogenic bands can be shown as needed and when available."){: width="70%"}


## Data Tracks

Within this ideogram, we can plot *data tracks*. There are different plot types available, such as scatterplots, histograms, heatmaps, and link tracks. Below is a list of the main track types with example images:

<table style="table-layout: fixed">
	<thead>
		<tr>
			<th width="300">Example</th>
			<th>Description</th>
		</tr>
	</thead>
	<tbody>
		<tr>
			<td><img src="../../images/circos/data_tracks_scatter.png" alt="Example of a Circos scatter track"/></td>
			<td>
				<b>Scatter plot</b> track type. All data points are indicated by a glyph (shape).
				<br /><br />

				<u>Data format</u>: four or five columns, tab separated. While Circos accepts space or tab separated, to use Galaxy's data manipulation tools you will need tab separated data. The attributes colum is completely optional but can be used to override colours, or provide extra context like "strand" or something else on which it might be diserable to filter. The format is <code>key=value,key2=value2,...</code>
				<table>
					<thead><tr><th>Chromosome</th><th>Start</th><th>End</th><th>Value</th><th>Attributes (optional)</th></tr></thead>
						<tbody>
						<tr>
							<td>chrX</td><td>0</td><td>7499999</td><td>50</td><td></td>
						</tr>
						<tr>
							<td>chrX</td><td>7500000</td><td>14999999</td><td>100</td><td>fill_color=red</td>
						</tr>
						<tr>
							<td>chrX</td><td>15000000</td><td>22499999</td><td>-25</td><td>fill_color=(140,104,137)</td>
						</tr>
					</tbody>
				</table>
			</td>
		</tr>
		<tr>
			<td><img src="../../images/circos/data_tracks_line.png" alt="Example of a Circos line track" /></td>
			<td><strong>Line plot</strong> track type. Data points are connected by a line. <br/><br/><u>Data format</u>: identical to scatter plot.</td>
		</tr>
		<tr>
			<td><img src="../../images/circos/data_tracks_histogram.png" alt="Example of a Circos Histogram track" /></td>
			<td><strong>Histogram</strong> track type. Data points are connected to form a step-like trace. <br/><br/><u>Data format</u>: identical to scatter plot.</td>
		</tr>
		<tr>
			<td><img src="../../images/circos/data_tracks_heatmap.png" alt="Example of a Circos Heatmap track" /></td>
			<td><strong>Heatmap</strong> track type. <br/><br/><u>Data format</u>: identical to scatter plot.</td>
		</tr>
		<tr>
			<td><img src="../../images/circos/data_tracks_tiles.png" alt="Example of a Circos tile track" /></td>
			<td>
				<strong>Tile</strong> track type. This is useful to indicate a range of the chromosome, for example to show genes, <span class="notranslate">reads</span>, repeat regions, etc.
				<br /><br />

				<u>Data format</u>: four or five columns, similar to scatter plot but instead of a value or a score, we have a text label that can be displayed.
				<table>
					<thead><tr><th>Chromosome</th><th>Start</th><th>End</th><th>Label</th><th>Attributes (optional)</th></tr></thead>
						<tbody>
						<tr>
							<td>chr1</td><td>10119128</td><td>10345073</td><td>C_994</td><td>id=Conrad_994</td>
						</tr>
						<tr>
							<td>chr1</td><td>10171406</td><td>10415218</td><td>M_1535</td><td>id=McCarroll_1535</td>
						</tr>
						<tr>
							<td>chr1</td><td>103351772</td><td>103448763</td><td>M_5507</td><td>id=Mills_5507</td>
						</tr>
					</tbody>
				</table>
			</td>
		</tr>
		<tr>
			<td><img src="../../images/circos/data_tracks_text.png" alt="Example of a Circos text track" /></td>
			<td><strong>Text</strong> track type. Text may also be added to a plot, for instance to indicate names of impacted genes. <br/><br/><u>Data format</u>: identical to tiles plot.</td>
		</tr>
		<tr>
			<td><img src="../../images/circos/data_tracks_links.png" alt="Example of a Circos link track" /></td>
			<td>
				<strong>Link</strong> track type. Relationship between objects can be indicated by a line between them.
				<br /><br />

				<u>Data format 1</u>: six or seven columns, including the optional attribute column.
				<table>
					<thead>
						<tr>
							<th>Chromosome 1</th><th>Start 1</th><th>End 1</th>
							<th>Chromosome 2</th><th>Start 2</th><th>End 2</th>
							<th>Attributes (optional)</th>
						</tr>
					</thead>
					<tbody>
						<tr>
							<td>chrX</td><td>87655109</td><td>107655109</td><td>chr12</td><td>109275831</td><td>129275831</td><td>color=blue,value=0</td>
						</tr>
						<tr>
							<td>chrX</td><td>73701156</td><td>93701156</td><td>chr22</td><td>26513447</td><td>46513447</td><td>color=green,value=1</td>
						</tr>
						<tr>
							<td>chr12</td><td>121879007</td><td>132349534</td><td>chr7</td><td>43840633</td><td>63840633</td><td>color=red,value=2</td>
						</tr>
					</tbody>
				</table>

				<u>Data format 2</u>: four or five columns, including the optional attribute column.
				<table>
					<thead>
						<tr>
							<th>Link ID</th><th>Chromosome</th><th>Start</th><th>End</th><th>Attributes (optional)</th>
						</tr>
					</thead>
					<tbody>
						<tr>
							<td>link_0</td><td>chrX</td><td>87655109</td><td>107655109</td><td>color=red</td>
						</tr>
						<tr>
							<td>link_0</td><td>chr12</td><td>109275831</td><td>129275831</td><td>color=blue</td>
						</tr>
						<tr>
							<td>link_1</td><td>chrX</td><td>73701156</td><td>93701156</td><td>color=green</td>
						</tr>
						<tr>
							<td>link_1</td><td>chr22</td><td>26513447</td><td>46513447</td><td>color=black</td>
						</tr>
					</tbody>
				</table>
			</td>
		</tr>
		<tr>
			<td><img src="../../images/circos/data_tracks_ribbons.png" alt="Example of a Circos link track with ribbons" /></td>
			<td><strong>Ribbons</strong> are a type of link track. These can be coloured and twisted as desired. <br/><br/><u>Data format</u>: identical to links.</td>
		</tr>
	</tbody>
</table>

Tracks can be customized a lot, some relevant concepts are:
  - **Radius** of the track determines its location between the center (0) and the ideogram (1).
  - **Rules** can be defined to change for example the color of data points depending on the value of the data points.
  - **Axes and backgrounds** can be drawn on a data track


There are a *lot* of further customizations available within Circos, but in this tutorial, we will start with the basics.


# Tutorial Overview

We will now illustrate Circos further with a number of example plots. Each of these can be run independently of each other, so feel free to pick an example that suits your interest. If this is your first time using Circos, we suggest doing the examples in order.

| Link to Section     | Preview       | Note                                  |
| ------------------- |:-------------:| :------------------------------------ |
| [Cancer Genomics](#example-cancer-genomics) | ![VCaP cancer Circos plot](../../images/circos/vcap.png){: width="45%"} | Create a plot using data from a cancer cell line. This tutorial will guide you through the iterative process of building your first Circos image, and covers some cancer background as well. |
| [Presidential Debate](#example-presidential-debate) | ![Presidential Debate NYT Circos plot](../../images/circos/candidatenetworkdiagram.png){: width="45%"} | To show that Circos can be used for non-genomics data as well, this example recreates a plot that appeared in the New York times, visualizing data of the presidential debates. |
| [ENCODE cover](#example-nature-cover-encode) | ![Nature Cover ENCODE](../../images/nature_cover_encode.png){: width="30%"} | Recreate a Nature Cover! |




# Example: Cancer Genomics

In this section, we will recreate a Circos plot of the VCaP cancer cell line presented in {% cite alves2013gene %}. In this study, data from various sources were combined into a single integrative Circos plot.


![VCaP cancer Circos plot](../../images/circos/vcap.png "Circos plot of the VCaP cancer cell line displaying from the outside in: copy number variation, B-allele frequency, structural variants")


This plot has 4 tracks:
 - Structural variants (2 tracks, data obtained from whole-genome NGS sequencing data)
 - B-allele Frequency (obtained from SNP array data)
 - Copy Number (obtained from SNP array data)


In this section we will reproduce this Circos plot step by step.

## Data upload


> <hands-on-title>Obtaining our data</hands-on-title>
>
> 1. Make sure you have an empty analysis history. Give it a name.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. {% tool [Import Data](upload1) %}
>    - Import the sample data files to your history, either from a shared data library (if available), or from Zenodo using the following URLs:
>
>    ```
>    https://zenodo.org/record/4494146/files/VCaP_Copy-Number.tsv
>    https://zenodo.org/record/4494146/files/VCaP_B-allele-Frequency.tsv
>    https://zenodo.org/record/4494146/files/VCaP-highConfidenceJunctions.tsv
>    https://zenodo.org/record/4494146/files/hg18_karyotype_withbands.txt
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
{: .hands_on}


## Ideogram

As the first step to this Circos plot, let's configure the ideogram (set of chromosomes to draw). You can use one of the built-in genomes, or you can supply your own karyotype file.

> <hands-on-title>Set ideogram configuration</hands-on-title>
>
> 1. {% tool [Circos](toolshed.g2.bx.psu.edu/repos/iuc/circos/circos/0.69.8+galaxy12) %} visualizes data in a circular layout with the following parameters:
>    - In *"Karyotype"*:
>        - *"Reference Genome Source"*: `Custom Karyotype`
>            - {% icon param-file %} *"Karyotype Configuration"*: `hg18_karyotype_withbands.txt`
>    - In *"Ideogram"*:
>        - *"Spacing Spacing Between Ideograms (in chromosome units)"*: `50`
>        - *"Radius"*: `0.85`
>        - *"Thickness"*: `45`
>        - In *"Labels"*:
>            - *"Label Font Size"*: `64`
>        - In *"Cytogenic Bands"*:
>            - *"Bands transparency"*: `2`
>            - *"Band Stroke Thickness"*: `1`
>
> 2. **Rename** {% icon galaxy-pencil%} the output `Circos Plot ideogram`
>
>    {% snippet faqs/galaxy/datasets_rename.md name="Circos Plot ideogram" %}
>
{: .hands_on}

You should now have a plot that looks like this:

![Circos output with only the ideogram](../../images/circos/cancer_ideogram.png){: width="50%"}

We will use this as the basis for our plot, and add data tracks one at a time.


> <comment-title>Advanced info: defining your karyotype file</comment-title>
>
> Since our data uses `hg18` reference genome, we supply a corresponding karyotype file:
>
> ```
> chr - chr1 1 0 247249719 chr1
> chr - chr2 2 0 242951149 chr2
> chr - chr3 3 0 199501827 chr3
> chr - chr4 4 0 191273063 chr4
> chr - chr5 5 0 180857866 chr5
> chr - chr6 6 0 170899992 chr6
> chr - chr7 7 0 158821424 chr7
> chr - chr8 8 0 146274826 chr8
> chr - chr9 9 0 140273252 chr9
> chr - chr10 10 0 135374737 chr10
> chr - chr11 11 0 134452384 chr11
> chr - chr12 12 0 132349534 chr12
> chr - chr13 13 0 114142980 chr13
> chr - chr14 14 0 106368585 chr14
> chr - chr15 15 0 100338915 chr15
> chr - chr16 16 0 88827254 chr16
> chr - chr17 17 0 78774742 chr17
> chr - chr18 18 0 76117153 chr18
> chr - chr19 19 0 63811651 chr19
> chr - chr20 20 0 62435964 chr20
> chr - chr21 21 0 46944323 chr21
> chr - chr22 22 0 49691432 chr22
> chr - chrX x 0 154913754 chrx
> chr - chrY y 0 57772954 chry
> ```
>
> Chromosome definitions are formatted as follows: `chr - ID LABEL START END COLOR`
>
> - The first two fields are always `chr`, indicating that the line defines a chromosome, and `-`. The second field defines the parent structure and is used only for band definitions.
> - The `ID` is the identifier used in data files whereas the `LABEL` is the text that will appear next to the ideogram on the image.
> - The `start` and `end` values define the size of the chromosome. The karyotype file should store the entire chromsome size, not just the region you wish to draw.
> There are other parameters we can use to draw only a subset of the data (e.g. just one chromosome).
> - The `color` parameter is optional, to use built-in color scheme, use `chr1, chr2. etc` again in this column
>
> More information about this format (including karyotype definitions for several species) can be found on the [Circos website](http://circos.ca/documentation/tutorials/ideograms/karyotypes/)
>
> > <question-title></question-title>
> >
> > 1. Why didn't we use the `hg18` built-in genome (using the `Locally Cached` option)?
> >
> > > <solution-title></solution-title>
> > >
> > > 1. These built-in definitions often include more than the canonical chromosomes (chr1-chr22, chrX, chrY), which we might not want to plot. For example, using
> > >    The full definition of `hg18` built-in to Galaxy, we get the following ideogram:
> > >
> > >    ![Example of ideogram generated using built-in genome definition](../../images/circos/cancer_ideogram_cached.png){: width="85%"}
> > >
> > >    We could still use this karyotype, but we would have to limit the chromosomes to be drawn in the Circos settings (we will cover this later). In this example however, we supply our own karyotype file defining only the canonical chromosomes.
> > {: .solution}
> {: .question}
>
{: .comment}


## Structural Variations

The first data track we will configure, will be the structural variants (SVs) using the *link* track type in Circos. We will colour the links differently depending on whether the SVs are intrachromosomal (within a single chromosome) or interchromosomal (between different chromosomes).


> <comment-title>Background: Structural Variants</comment-title>
>
> Structural variants (SVs) are large-scale genomic rearrangements. SVs involve large segments of DNA (>50 bp) that are deleted, duplicated, translocated or inverted.
>
> ![Overview of types of SVs](../../images/circos/sv-types.jpg "Different types of SVs")
>
> Image Credit: {% cite alkan2011genome%}
>
> One of the first observations of SVs in the human genome is known as the [Philadelphia Chromosome](https://en.wikipedia.org/wiki/Philadelphia_chromosome), a SV observed in leukemia. In this mutation, a translocation of genetic material  occurs between chromosomes 9 and 22, resulting in a fusion between genes *BCR* and *ABL1*, causing the production of a hybrid protein, impairing various signalling pathways and causing the cell to divide uncontrollably.
>
> ![the Philedelphia chromosome](../../images/circos/sv.jpg "The Philadelphia chromosome"){: width="70%"}
>
> In cancer analyses it is therefore often useful to examine SVs and look for potential fusion genes that may affect cell function.
>
{: .comment}


SVs are usually described in terms of the SV **breakpoints** (or **junctions**); sets of genomic locations which are separated by a large distance on the reference genome, but have become adjacent in the sample through the occurrence of structural variants. Unfortunately, there is no standard file format for SV data, with different SV callers outputting different formats. Therefore, our first step will be to transform our input dataset to the Circos format for link tracks.


### SV File Format

```
#ASSEMBLY_ID	GS000008107-ASM
#SOFTWARE_VERSION	2.0.2.22
#GENERATED_BY cgatools
#GENERATED_AT	2012-Mar-07 19:33:32.930656
#FORMAT_VERSION	2.0
#GENOME_REFERENCE	NCBI build 36
#SAMPLE	GS00669-DNA_D02
#TYPE	JUNCTIONS
#DBSNP_BUILD	dbSNP build 130
#GENE_ANNOTATIONS	NCBI build 36.3
>Id	LeftChr	LeftPosition	LeftStrand	LeftLength	RightChr	RightPosition	RightStrand	RightLength	StrandConsistent	Interchromosomal	Distance	DiscordantMatePairAlignments	JunctionSequenceResolved	TransitionSequence	TransitionLength	LeftRepeatClassification	RightRepeatClassification	LeftGenes	RightGenes	XRef	DeletedTransposableElement	KnownUnderrepresentedRepeat	FrequencyInBaselineGenomeSet	AssembledSequence	EventId	Type	RelatedJunctions
```


### Circos Input Format

```
chromosome - start - end - chromosome - start - end
```

So in order to convert this to Circos format, we need to
- Remove header lines (lines starting with `#`)
- Select the columns containing the chromosomes and positions of the breaks (junctions)


{% snippet faqs/galaxy/analysis_cut.md %}

> <hands-on-title>Prepare input data</hands-on-title>
>
> 1. {% tool [Select](Grep1) %} lines that match an expression with the following parameters:
>    - {% icon param-file %} *"Select lines from"*: `VCaP highConfidenceJunctions.tsv`
>    - *"that"*: `NOT Matching`
>    - *"the pattern"*: `^[#><]`
>
> 2. {% tool [Cut](Cut1) %} columns from a table with the following parameters:
>    - *"Cut columns"*: `c2,c3,c3,c6,c7,c7`
>    - {% icon param-file %} *"From"*: output of **Select** {% icon tool %}
>
> 3. **Rename** {% icon galaxy-pencil %} this output to `SVs Circos.tsv`
>
{: .hands_on}

Now that we have the correct format, we can plot our data in Circos. We will plot the SVs as links; showing which parts of genome have been fused together in our sample.

Given that Circos is a very complex with dozens of parameters to set, we re-run previous circos runs to build on the existing configuration we have done, without losing the progress and having to re-specify parameters every time.

> <hands-on-title>Add Circos link track for SVs</hands-on-title>
>
> 1. Click **Re-run** {% icon galaxy-refresh %} on the previous Circos {% icon tool %} run (`Circos Plot ideogram`)
>
> 2. Add a new Link Track for the SV data, colouring by SV type:
>    - In *"Link Tracks"*:
>       - In *"Link Data"*:
>           - {% icon param-repeat %} *"Insert Link Data"*
>               - *"Inside Radius"*: `0.95`
>               - {% icon param-file %} *"Link Data Source"*: `SVs Circos.tsv`
>               - *"Link Type"*: `basic`
>               - *"Thickness"*: `3.0`
>               - *"Bezier Radius"*: `0.5`
>               - In *"Rules"*:
>                   - In *"Rule"*:
>                       - {% icon param-repeat %} *"Insert Rule"*
>                           - In *"Conditions to Apply"*:
>                               - {% icon param-repeat %} *"Insert Conditions to Apply"*
>                                   - *"Condition"*: `Interchromosomal`
>                           - In *"Actions to Apply"*:
>                               - {% icon param-repeat %} *"Insert Actions to Apply"*
>                                   - *"Action"*: `Change Fill Colour`
>                                   - *"Change fill Color"*: {% color_picker #ff0000 %} (red)
>
> 2. **Rename** {% icon galaxy-pencil%} the output `Circos Plot SVs`
>
{: .hands_on}


Your output should look something like this:

![The plot with an SV track](../../images/circos/cancer_svs1.png "SVs on the VCaP cell line. Red lines indicate <i>interchromosomal</i> SVs, where pieces originating from different chromosomes have fused together. Black lines show breaks withing in single chromosome."){: width="60%"}

> <question-title></question-title>
>
> 1. Are there more interchromosomal or intrachromosomal SVs?
> 2. Which chromosome appears to have the most SVs?
>
> > <solution-title></solution-title>
> >
> > 1. Interchromosomal SVs (between different chromosomes) are coloured red in this
> >    plot, while SVs within a single chromosome are coloured black. By plotting the data with Circos, you can now easily see
> >    at a glance that there are more intrachromosomal SVs (black) than interchromosomal SVs (red).
> > 2. Chromosome 5 appears to have a lot more SVs than the other chromosomes (it looks almost completely black!)
> >
> {: .solution}
{: .question}

We see from this image that chromosome 5 has an unusually large number of SVs, let's look at that chromosome more closely, by limiting the chromosomes Circos should draw:


> <hands-on-title>Plot only Chromosome 5</hands-on-title>
>
> 1. Hit **Rerun** {% icon galaxy-refresh %} on the previous Circos {% icon tool %} run
>
> 2. **Change** the following tool parameters:
>    - In *"Ideogram"*:
>       - *"Limit/Filter Chromosomes"*: `chr5`
>       - *"Spacing Between Ideograms (in chromosome units)"*: `0.5`
>
{: .hands_on}

You should see a plot like:

![Circos plot of chromosome 5 SVs](../../images/circos/cancer_svs_chr5.png "Chromosome 5 of the VCaP cancer cell line. The <a href="https://en.wikipedia.org/wiki/Locus_(genetics)">q arm</a> of this chromosome appears to be affected by an unusually large number of SVs"){: width="50%"}



> <question-title></question-title>
>
> 1. Are there indeed significantly more SVs on chromosome 5 than on the other chromosomes? (hint: plot some of the other chromosomes as well)
> 2. Are the SVs equally distributed over chromosome 5? Can you think of an explanation for this?
>
> > <solution-title></solution-title>
> >
> > 1. Yes, plotting for example only chromosome 1 (left) and comparing this with the chromosome 5 plot (right), reveals that chr5 has abnormally high number of SVs compared to the other chromosomes
> >
> >    ![Circos plot of chromosome 5 SVs](../../images/circos/cancer_svs_chr1.png){: width="40%"}
> >    ![Circos plot of chromosome 1 SVs](../../images/circos/cancer_svs_chr5.png){: width="40%"}
> >
> >
> > 2. No, only part of chromosome 5 appears to be affected. It turns out that this region is exactly one arm of the chromosome.
> >    This could be caused by a phenomenon known as *chromothripsis* (see next box).
> >
> >    ![Chromosome arms](../../images/circos/chromosome-arms.jpg "The different arms of a chromosome. The short arm is termed p, the long arm is q. In our sample, the 5q arm appears to be affected by chromothripsis"){: width="50%"}
> >
> {: .solution}
{: .question}



> <comment-title>Background: Chromothripsis</comment-title>
>
> **Chromothripsis** is a phenomenon whereby (part of) a chromosome is shattered in a single catastrophic event, and subsequently imprecisely stitched
> together by the cell's repair mechanisms. This leads to a huge number of SV junctions.
>
> ![Chromothripsis](../../images/circos/chromothripsis.png "Chromothripsis is a scattering of the DNA, followed by an imprecise repair process, leading to many structural rearrangements."){: width="60%"}
>
> > **Characteristics of chromothripsis:**
> >
> > 1. **Large numbers of complex rearrangements** in localised regions of single chromosomes or chromosome arms (showed by high density and clustered breakpoints) which suggests that chromosomes need to be condensed e.g. in mitosis for chromothripsis to occur.
> > 2. Low **copy number states- alternation between 2 states** (sometimes 3) suggesting that rearrangements occurred in a short period of time.
> > 3. In chromothriptic areas you get **alternation of regions which retain heterozygosity-two copy (no loss or gain), with regions that have loss of heterozygosity- one copy** (heterozygous deletion). This suggest that the rearrangements took place at a time that both parental copies of the chromosome were present and hence early on the development of the cancer cell.
> >
> {: .quote cite="https://en.wikipedia.org/wiki/Chromothripsis"}
>
{: .comment}

By visualizing the SVs, we have observed characteristic 1 of the list above; large number of complex rearrangements in a localised regon of a single chromosome arm, one of the main features of chromothripsis. In order to confirm we are indeed dealing with chromothripsis, we will next look plot copy number data and B-allele frequency data (both obtained from microarrays) to ascertain whether we observe the expected patterns in copy number states and heterozygosity.


## Copy Number Variation

Next, we will create a track displaying copy number. This data comes from Affymetrix SNP arrays.


> <comment-title>Background: Copy Number Variation (CNV)</comment-title>
>
> The human genome is a *diploid* genome, meaning there are 2 copies of each chromosome, one paternal, and one maternal. This means that for any given gene, humans have two different copies of it in our genome.
>
> <br>
>
> Some structural variants will lead to a change in this copy number, for example duplications and deletions. Other SVs (such as inversions and translocations) do not result in a change in copy number, since the piece of DNA is just moved, but the number of copies of it remains the same.
>
> ![SVs lead to CNVs](../../images/circos/cancer-cnv-intro.png "Duplications and deletions lead to a change in copy number, measurable by a change in read depth. Other SVs such as inversions are copy-number neutral.")
>
> In a healthy diploid genome, we expect the copy number to be around 2 in most places, with occasional duplications and deletions which are part of
> the normal variation within the human population. In highly rearranged genomes such as cancer we expect to see a lot more copynumber variation.
>
{: .comment}


> <comment-title>Background: DNA Microarrays</comment-title>
>
> **Microarrays** are used to measure the expression levels of large number of genes simultaneously, or to genotype multiple regions of a genome. In this example in our tutorial, we have data from a **SNP array**. This type of microarray detects the presence and proportion (homozyogous/heterozygous) of a
> wide range of SNPs (Single Nucleotide Polymorphisms) known to exist within the population. A set of **probes** targeting positions of a large number of known SNPs
> are used to detect the presence or absence of the SNPs in the sample.
>
> <br>
> Each SNP location is covered by 2 probes (one for the reference allele, and one for the variant allele). By comparing their combined intensity to the expected intensity (e.g. the sample average), a measure known as the **Log R ratio**, we can learn something about copy number. The resulting plots often look something like this:
>
>   ![Example of a Log R ration plot](../../images/circos/cnv-plot.png){: width="50%"}
>
> With a value of 0 indicating the normalized copy number (2 in the case of a diploid genome), and significant diversions from this expected value
> point to copy number gains or losses (the figure above shows 1 region with a copy number loss).
>
{: .comment}


Let's look at our file format (VCaP-copynumber.txt`):

```
Chromosome	Start	End	Value	Array
chr1	10004	10004	0.07110633	0
chr1	28663	28663	0.2057637	0
chr1	46844	46844	0.2016204	0
chr1	59415	59415	0.1775235	0
chr1	72017	72017	-0.1353417	0
```

The `Value` column indicates the copy number state, and is always between -1 and 1. the position has expected copy-number (`0`), indicating 2 copies in the case of diploid genomes, or whether it has a copy number loss (negative values) or a gain in copy number (positive values).

This is pretty close to the format expected by Circos for 2D data tracks (`chr - start - end - value`), all we need to do to prepare this file is remove the first header line, and remove column 5. Furthermore, because this file is quite large, we do a subsampling down to 25000 lines; this is enough to get a genome-level overview of the data, but small enough that Circos will complete plotting quickly.



> <hands-on-title>Prepare CNV input file</hands-on-title>
>
> 1. {% tool [Remove beginning](Remove beginning1) %} of a file with the following parameters:
>    - *"Remove first"*: `1`
>    - {% icon param-file %} *"from"*: `VCaP copy number.tsv`
>
> 2. {% tool [Cut](Cut1) %} columns from a table  with the following parameters:
>    - *"Cut columns"*: `c1,c2,c3,c4`
>    - {% icon param-file %} *"From"*: output of **Remove beginning** {% icon tool %}
>
> 3. {% tool [Select random lines](random_lines1) %} with the following parameters:
>    - *"Randomly select"*: `25000`
>    - {% icon param-file %} *"from"*: output of **Cut** {% icon tool %}
>
> 4. **Rename** {% icon galaxy-pencil %} the output file `cnv-circos.txt`
>
{: .hands_on}


Now that our file is prepared, we can add a track to our Circos image. We will create a scatterplot, and colour each data point depending on copy number state (green=gain, red=loss)


> <hands-on-title>Add Copy Number track to Circos</hands-on-title>
>
> 1. Hit **Rerun** {% icon galaxy-refresh %} on the Circos plot containing the SV track {% icon tool %} run (`Circos Plot SVs`)
>
> 2. Add a new scatterplot track to the image
>    - In *"2D Data Tracks"*:
>        - {% icon param-repeat %} *"Insert 2D Data Plot"*
>            - *"Outside Radius"*: `0.95`
>            - *"Inside Radius"*: `0.8`
>            - *"Plot Type"*: `Scatter`
>                - *"Scatter Plot Data Source"*: `cnv-circos.txt`
>                - In *"Plot Format Specific Options"*:
>                    - *"Glyph Size"*: `4`
>                    - *"Fill Color"*: {% color_picker #7f7f7f %} (gray)
>                    - *"Stroke Thickness"*: `0`
>            - *"Minimum / maximum options"*: `Supply min/max values`
>                - *"Minimum value"*: `-1.0`
>                - *"Maximum value"*: `1.0`
>
> 3. **Examine** {% icon galaxy-eye %} the resulting plot
>
>    > <question-title></question-title>
>    >
>    > 1. Examine the resulting plot, what do you see?
>    > 2. How could we solve this?
>    >
>    > > <solution-title></solution-title>
>    > >
>    > > 1. We see the new track, but it overlaps with the SV track. This is because we used the same `radius` parameter.
>    > >    This parameter determines the position of the track within the plot.
>    > >
>    > >    ![Circos plot of chromosome 5 SVs](../../images/circos/cancer-overlap.png){: width="80%"}
>    > >
>    > >
>    > > 2. To fix this, we can rerun the Circos tool, and change the radius of the link track (SVs) to be inside the new copynumber track (<`0.8`).
>    > >    We will do this in the next step. This is what we mean by Circos being an iterative process; the tool is too complex to define a multitrack plot
>    > >    all at once, rather, you build it up step by step and frequently check the output.
>    > >
>    > {: .solution}
>    {: .question}
>
> 4. **Rerun** {% icon galaxy-refresh %} the tool, changing the following parameters.
>    - In *"Link Tracks"*:
>        - In *"1: Link Data"*:
>            - *"Inside Radius"*: `0.75`
>
>
{: .hands_on}

You should see a plot that looks like:

![Circos plot with CNV track](../../images/circos/cancer-cnv.png){: width="75%"}


Now that we are happy with the placement of our track, let's tweak it a bit more. Let's colour positions showing a significant copy number loss (< `-0.15`) red, and positions with a copy number gain (> `0.15`) green, leaving everything inbetween gray (expected copy number):


> <hands-on-title>Colour data points by copy number state</hands-on-title>
>
> 1. Hit **Rerun** {% icon galaxy-refresh %} on the previous Circos {% icon tool %} run
>
> 2. In the **2D data track** of the CNV track we just created, add the following rules:
>    - In *"Rules"*:
>        - {% icon param-repeat %} *"Insert Rule"*
>            - In *"Conditions to Apply"*:
>                - {% icon param-repeat %} *"Insert Conditions to Apply"*
>                    - *"Condition"*: `Based on value (ONLY for scatter/histogram/heatmap/line)`
>                        - *"Points above this value"*: `0.15`
>            - In *"Actions to Apply"*:
>                - {% icon param-repeat %} *"Insert Actions to Apply"*
>                    - *"Action"*: `Change Fill Color for all points`
>                        - *"Fill Color"*: {% color_picker #00b050 %} (green)
>        - {% icon param-repeat %} *"Insert Rule"*
>            - In *"Conditions to Apply"*:
>                - {% icon param-repeat %} *"Insert Conditions to Apply"*
>                    - *"Condition"*: `Based on value (ONLY for scatter/histogram/heatmap/line)`
>                       - *"Points below this value"*: `-0.15`
>            - In *"Actions to Apply"*:
>                - {% icon param-repeat %} *"Insert Actions to Apply"*
>                    - *"Action"*: `Change Fill Color for all points`
>                        - *"Fill Color"*: {% color_picker #ff0000 %} (red)
>
{: .hands_on}

You should now see a plot like this:

![Circos plot with SNV track with rules defined](../../images/circos/cancer-cnv-full.png){: width="75%"}

Sometimes it can also be nice to see the axes of the plot, to more accurately judge the values of the different data points.
We can do this as follows:

> <hands-on-title>Add plot axes</hands-on-title>
>
> 1. Hit **Rerun** {% icon galaxy-refresh %} on the previous Circos {% icon tool %} run
>
> 2. In the **2D data track** of the CNV track we just created, add plot axes as follows:
>    - In *"Axes"*:
>        - In *"Axis"*:
>            - {% icon param-repeat %} *"Insert Axis"*
>                - *"Spacing"*: `0.25`
>                - *"Color"*: {% color_picker #7f7f7f %} (gray)
>                - *"y0"*: -1
>                - *"y1"*: 1
> 3. **Rename** {% icon galaxy-pencil%} the output `Circos Plot CopyNumber`
>
{: .hands_on}


You should now see a plot with axes:

![Circos plot with SNV track with rules and axes defined](../../images/circos/cancer-cnv-full-axes.png){: width="75%"}

<!-- TODO: update image when axes are fixed (should be able to go to -1) -->




## B-allele Frequency

Next, we will visualize the B-allele frequency (also known as minor allele frequency)

> <comment-title>Background: B-allele Frequency (BAF)</comment-title>
>
> The B-allele frequency is closely related to copy number. There are many nuances to the measurement of B-allele frequency,
> but roughly speaking it indicates the frequency (ratio) of the non-reference allele of the SNP within the sample
>
> This can be used to estimate copy number changes; in a diploid genome we expect to observe 3 states:
>  1. SNP is present in 100% of the probes (**homozygous variant**)
>  2. SNP is present in 0% of the probes (**homozygous reference**)
>  3. SNP is present in 50% of the probes (**heterozygous**)
>
> <br>
>
> By plotting this percentage, we get our B-allele frequency plot:
>
> ![Example CNV and BAF plots for different copy number states](../../images/circos/cnv-baf-intro.png "expected B-allele frequency plot (top) and Log R ratio plot (bottom) for different copy number states")
>
> When these SNPs are detected at different ratios, it may indicate copy number variation. For example, a region displaying SNP ratios of 33% and 66% may indicate a copy number of 3 for that region (see image above).
>
{: .comment}


Now we will add such a B-allele frequency plot as track in our Circos visualization.
The data we will use for this is also obtained from SNP array data, and looks like this:

```
Chromosome	Start	End	Value
chr1	10004	10004	0.9956236
chr1	28663	28663	0.005509489
chr1	46844	46844	0.488594
chr1	59415	59415	0.570193
chr1	72017	72017	0.006410222
chr1	97215	97215	0.0
chr1	110905	110905	0.9918569
[..]
```

Note that the B-allele frequency value is always between 0 and 1.

We will make another scatterplot, so our data should be in the same format as the copynumber track: `chr - start - end - value`. Luckily, this data is already in the correct format, all we have to do is remove the header line! We will also subset the data again by selecting lines randomly from the file.


> <hands-on-title>Prepare the B-allele frequency table</hands-on-title>
>
> 1. {% tool [Remove beginning](Remove beginning1) %} with the following parameters:
>    - *"Remove first"*: `1`
>    - {% icon param-file %} *"from"*: `VCaP_B-allele frequence.tsv`
>
> 2. {% tool [Select random lines](random_lines1) %} with the following parameters:
>    - *"Randomly select"*: `25000`
>    - {% icon param-file %} *"from"*: output of **Remove** {% icon tool %}
>    - *"Set a random seed"*: `Don't set seed`
>
> 3. **Rename** {% icon galaxy-pencil %} this file to `baf-circos.tsv`
>
{: .hands_on}

Now are data is ready to be plotted in Circos. We will plot this track directly inside the CNV track, which means we will have to change the radius of the SV link track again as well.

> <hands-on-title>Add B-allele Frequency track to Circos</hands-on-title>
>
> 1. Hit **Rerun** {% icon galaxy-refresh %} on the previous Circos {% icon tool %} run (`Circos Plot CopyNumber`)
>
> 2. Add a new scatterplot track to the image
>    - In *"2D Data Tracks"*:
>        - {% icon param-repeat %} *"Insert 2D Data Plot"*
>            - *"Outside Radius"*: `0.75`
>            - *"Inside Radius"*: `0.6`
>            - *"Plot Format"*: `Scatter`
>                - {% icon param-file %} *"Scatter Plot Data Source"*: `baf-circos.tsv` (output of **Select random lines** {% icon tool %})
>                - In *"Plot Format Specific Options"*:
>                    - *"Glyph Size"*: `4`
>                    - *"Fill Color"*: {% color_picker #7f7f7f %} (gray)
>                    - *"Stroke Thickness"*: `0`
>            - *"Minimum / maximum options"*: `Supply min/max values`
>                - *"Minimum value"*: `0.0`
>                - *"Maximum value"*: `1.0`
>            - In *"Axes"*:
>                - In *"Axis"*:
>                    - {% icon param-repeat %} *"Insert Axis"*
>                        - *"Spacing"*: `0.25`
>    - In *"Link Tracks"*:
>        - In *"1: Link Data"*:
>            - *"Inside Radius"*: `0.55`
>
> 3. **Rename** {% icon galaxy-pencil %} this plot to `Circos Plot BAF`
>
{: .hands_on}

You should see a plot that looks like this:

![Circos output with BAF track](../../images/circos/cancer-baf.png){: width="60%"}


Great! we can see our B-allele frequency plot track added.


> <question-title></question-title>
>
> 1. Look at the B-allele frequency track, try to identify chromosome(s) having a copy number of:
>    1. `CN=2` (diploid)
>    2. `CN=1` (haploid)
>    3. `CN=3` (triploid)
>
>    Hint: ![Example CNV and BAF plots for different copy number states](../../images/circos/cnv-baf-intro.png)
>
> 2. Do you see anything other than these states?
>
> > <solution-title></solution-title>
> >
> > 1. Compare the B-allele frequency plot to the expected plot shown above for the different copynumber states.
> >    1. Chromosome 12 appears completely diploid
> >    2. Chromosomes 16 and X appear to have only 1 copy (no heterozygosity and a loss in copynumber as shown by the CNV track)
> >    3. Chromosomes 1,2, and 3 show a pattern consistent with `CN=3`
> >
> > 2. Chromosome 5 shows a lot of changes in B-allele frequency. Chromosome 19 displays a pattern that could potentially indicate 4 copies (B-allele frequencies of 0, 0.25, 0.5, 0.75 and 1)
> {: .solution}
{: .question}


## Optional: Final Tweaking of Circos plot

You may have noticed, that by moving the link track closer to the center repeatedly, the track of intrachromosomal links has become rather narrow.
There is a parameter of the link track type called *Bezier*, which controls how tightly the links arc (i.e. how close to the center they reach. By playing around with this parameter, we can fine a more pleasing

> <hands-on-title>Change Bezier radius</hands-on-title>
>
> 1. Hit **Rerun** {% icon galaxy-refresh %} on the previous Circos {% icon tool %} run (`Circos Plot BAF`)
>
> 2. Change the Bezier parameter of the SV track:
>    - In *"Link Tracks"*:
>        - In *"1: Link Data"*:
>            - *"Bezier Radius"*: `0.25`
>
{: .hands_on}

![Final Circos image](../../images/circos/cancer-all.png){: width="60%"}


Another thing you may have noticed, is that in the original image we showed at the start of this section, the red links (interchromosomal SVs) were displayed as a completely different track. To do this, instead of creating a single track with a rule to change the colour of a subset of the data, we can make 2 separate tracks, with rules to only plot a subset of the data.

> <question-title>Exercise: Split SV track into two</question-title>
>
> 1. Try to split the link track in two so that it matches the original image. This may take some trial and error. The full configuration is shown in the answer box below, but we provede some hints if you want to try it yourself first:
>    1. **Change the existing link track:**
>       1. instead of a rule to change the colour of *interchromosomal* SVs, change their visibility (hide them)
>    2. **Add a new link track**
>       1. Choose an appropriate radius
>       2. Set the link colour to red
>       3. Add a rule to hide the *intrachromosomal* SVs
>
>   ![VCaP cancer Circos plot](../../images/circos/vcap.png "The original image from the paper. Try to make the two link tracks look like this"){: width="40%"}
>
> > <solution-title></solution-title>
> >
> > The full configuration  of the two link tracks is:
> >
> > > <hands-on-title>Hands on: Two link tracks</hands-on-title>
> > >
> > > 1. Hit **Rerun** {% icon galaxy-refresh %} on the previous Circos {% icon tool %} run
> > >
> > > 2. Configure two separate link tracks:
> > >    - In *"Link Tracks"*:
> > >        - In *"Link Data"*:
> > >            - {% icon param-repeat %} *"Insert Link Data"*
> > >                - *"Inside Radius"*: `0.55`
> > >                - {% icon param-file %} *"Link Data Source"*: `SVs Circos.tsv`
> > >                - *"Link Type"*: `basic`
> > >                - *"Thickness"*: `1.0`
> > >                - *"Bezier Radius"*: `0.25`
> > >                - In *"Advanced Settings"*:
> > >                    - *"Bezier Radius Purity"*: `1.0`
> > >                    - *"Perturb links?"*: `no`
> > >                - In *"Rules"*:
> > >                    - In *"Rule"*:
> > >                        - {% icon param-repeat %} *"Insert Rule"*
> > >                            - In *"Conditions to Apply"*:
> > >                                - {% icon param-repeat %} *"Insert Conditions to Apply"*
> > >                                    - *"Condition"*: `Interchromosomal`
> > >                            - In *"Actions to Apply"*:
> > >                                - {% icon param-repeat %} *"Insert Actions to Apply"*
> > >                                    - *"Action"*: `Change Visibility`
> > >                                    - *"Show"*: `No`
> > >            - {% icon param-repeat %} *"Insert Link Data"*
> > >                - *"Inside Radius"*: `0.3`
> > >                - {% icon param-file %} *"Link Data Source"*: `SVs Circos.tsv`
> > >                - *"Link Type"*: `basic`
> > >                - *"Link Color"*: {% color_picker #ff0000 %} (red)
> > >                - *"Thickness"*: `2.0`
> > >                - *"Bezier Radius"*: `0.0`
> > >                - In *"Rules"*:
> > >                    - In *"Rule"*:
> > >                        - {% icon param-repeat %} *"Insert Rule"*
> > >                            - In *"Conditions to Apply"*:
> > >                                - {% icon param-repeat %} *"Insert Conditions to Apply"*
> > >                                    - *"Condition"*: `Intrachromosomal`
> > >                            - In *"Actions to Apply"*:
> > >                                - {% icon param-repeat %} *"Insert Actions to Apply"*
> > >                                    - *"Action"*: `Change Visibility`
> > >                                    - *"Show"*: `No`
> > >
> > {: .hands_on}
> >
> {: .solution}
{: .question}


Awesome! You have now created a publication-quality Circos plot within Galaxy! There are more example plots in the sections below if you would like to get some more practice with the tool.


# Example: Presidential Debate

Circos was originally developed for genomics data, and a lot of the terminology in the tool is reminiscent of genomics (karyotype, ideogram, chromosome), but Circos can be used to plot any type of data. To illustrate this, the next example involves recreating a plot that appeared in [an article in the New York Times](http://www.nytimes.com/interactive/2007/12/15/us/politics/DEBATE.html), visualizing the 2008 presidential debates.

![Original plot from the NYT article](../../images/circos/presidential_debate_nyt.png "Plot showing how many times each presidential candidate mentioned each other candidate in the debates leading up to the 2008 US presidential election."){: width="80%"}


> <comment-title>Note</comment-title>
>
> This tutorial is based on one of the [tutorials on the Circos website](http://circos.ca/documentation/tutorials/recipes/naming_names/).
>
{: .comment}


Since we could not obtain the original datasets used to generate this image, we will re-create a similar plot using an artificial dataset:

![Presidential plot we will create this tutorial](../../images/circos/presidential_debate_plot.png){: width="60%"}


## Get Data

First, let's get the data we need for this plot:

> <hands-on-title>Obtaining our data</hands-on-title>
>
> 1. Make sure you have an empty analysis history. Give it a name.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. **Import Data.**
>    - Import the sample data files to your history, either from a shared data library (if available), or from Zenodo using the following URLs:
>
>    ```
>    https://zenodo.org/record/4494146/files/debate_karyotype.txt
>    https://zenodo.org/record/4494146/files/debate_links.tab
>    https://zenodo.org/record/4494146/files/debate_slices.tab
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
{: .hands_on}


## Ideogram

Ideogra

tools_json:
["Import Data", "upload1", "Circos", "toolshed.g2.bx.psu.edu/repos/iuc/circos/circos/0.69.8+galaxy12", "Select", "Grep1", "Cut", "Cut1", "Remove beginning", "Remove beginning1", "Select random lines", "random_lines1"]


topic: visualisation
tutorial: circos-microbial
title: Ploting a Microbial Genome with Circos
difficulty: intermediate
time_estimation: 30m
requirements_json: ["type: \"internal\""]
tags_json: []
content_hash: 3f0f67d223e8073c13ec661e57872343
last_modified: 2026-05-15T22:28:18.630314
zenodo_link: https://doi.org/10.5281/zenodo.3591856
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/visualisation/tutorials/circos-microbial/tutorial.html

content:
Once you're comfortable with [Circos in Galaxy]({% link topics/visualisation/tutorials/circos/tutorial.md %}), you might want to explore some real world use cases with Circos such as making a simple Genome Annotation plot, like one might want to publish alongside their genome annotation publication

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

There are a few common data formats and potentially required transformations for working with genome annotation data in Galaxy:

Data type | Transformations | Visualisation Options
--- | --- | ---
Genome Annotations | Circos Interval to Tiles, Circos Interval to Text Labels | Tiles (classic gene box-type viz), Text labels (used for annotating the important genes names), or a histogram (feature density)
Sequencing | BAM Coverage → Circos BigWig to Scatter | Histogram is probably the most common visualisation option
Variants | From BAM + variant calling, cut tool | Select specific columns of the VCF file (i.e. chromosome, position, position, quality)

## Get data

> <hands-on-title>Data Upload</hands-on-title>
>
> 1. Create and name a new history for this tutorial.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the datasets we will visualize:
>
>    ```
>    https://zenodo.org/record/3591856/files/genome.fa
>    https://zenodo.org/record/3591856/files/dna%20sequencing%20coverage.bw
>    https://zenodo.org/record/3591856/files/RNA-Seq%20coverage%201.bw
>    https://zenodo.org/record/3591856/files/RNA-Seq%20coverage%202.bw
>    https://zenodo.org/record/3591856/files/genes%20(NCBI).gff3
>    https://zenodo.org/record/3591856/files/variants.vcf
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
{: .hands_on}

This is a subset[^1] of *Escherichia coli str. K-12 substr. MG1655*, and associated sequencing data.

[^1]: reduced for faster plotting and faster data download

And we'll be using that data to make the following plot:

![plot of NC_000913 showing a range of datasets, from outside to in: a pink band representing the genome, two rna seq datasets and a dna seq coverage dataset showing coverage spikes around 10 and 20kb. Next inside is an overlapping set of genes presumably on the plus strand in dark blue overlapped with lilac triangles representing variants. Next a set of gene names, and finally a set of minus strand genes before a GC skew map in red and blue.](images/plot.png "Resulting Circos Plot showing a set of gene annotations, RNA-Seq, DNA-Seq coverage, as well as variants.")

## Fast Option: Using a Workflow

If you would like to try the 'fast' option, once you've imported the datasets, you can run the following workflow:

> <hands-on-title>Run workflow</hands-on-title>
>
> 1. **Import the workflow** into Galaxy
>
>    {% snippet faqs/galaxy/workflows_run_trs.md path="topics/visualisation/tutorials/circos-microbial/workflows/main_workflow.ga" title="Circos for E. Coli" %}
>
> 2. Run the workflow {% icon workflow %} using the following parameters:
>
>    - {% icon param-file %} *"Genome"*: `genome.fa`
>    - {% icon param-file %} *"Genes"*: `genes (NCBI).gff3`
>    - {% icon param-file %} *"RNA Seq Coverage (1)"*: `RNA-Seq Coverage 1.bw`
>    - {% icon param-file %} *"RNA Seq Coverage (2)"*: `RNA-Seq Coverage 2.bw`
>    - {% icon param-file %} *"DNA Sequencing Coverage"*: `dna sequencing coverage.bw`
>    - {% icon param-file %} *"Variants"*: `variants.vcf`
>
>    {% snippet faqs/galaxy/workflows_run.md %}
>
{: .hands_on}

Alternatively you can run the pre-processing steps and configure Circos manually as follows:

## Manual Configuration

We'll calculate the GC skew first from the genome sequence:

> <hands-on-title>GC Skew</hands-on-title>
>
> 1. {% tool [GC Skew](toolshed.g2.bx.psu.edu/repos/iuc/circos/circos_gc_skew/0.69.8+galaxy9) %} with the following parameters:
>    - *"Source for reference genome"*: `Use a genome from history`
>        - {% icon param-file %} *"Select a reference genome"*: `genome.fa` (Input dataset)
>    - *"Window size"*: `200`
>
>    > <comment-title>Window size</comment-title>
>    > The optimal window size is sometimes a process of trial and error to find the right balance between too many datapoints, and the expected smooth curve that should appear indicating forward or reverse strand genes.
>    {: .comment}
>
{: .hands_on}

### Preparing BigWig Files

With that file available, we're ready to convert these into a format Circos can understand. Natively we store the files in BigWig because it's a very space efficient format, however Circos only processes text files, and expects a dataset with the following structure:

Column | Value
--- | ---
1 | Chromosome name
2 | Start
3 | End
4 | value

so we'll use a tool to convert them into the Circos-preferred format.

> <hands-on-title>Dataset Pre-processing</hands-on-title>
>
> 1. {% tool [Circos: bigWig to Scatter](toolshed.g2.bx.psu.edu/repos/iuc/circos/circos_wiggle_to_scatter/0.69.8+galaxy9) %} with the following parameters:
>    - {% icon param-files %} *"Data file"*:
>      - output of **GC Skew** {% icon tool %}
>      - RNA-Seq coverage 1.bw (Uploaded Dataset)
>      - RNA-Seq coverage 2.bw (Uploaded Dataset)
>      - DNA sequencing coverage.bw (Uploaded Dataset)
>
>    > <comment-title>Multi-select to automate processing</comment-title>
>    > Multi-select allows you to easily process several datasets at once in Galaxy
>    {: .comment}
>
{: .hands_on}

> <comment-title>Creating BigWig files from coverage</comment-title>
> You can use a tool like {% tool [bamCoverage: generates a coverage bigWig file from a given BAM or CRAM file](toolshed.g2.bx.psu.edu/repos/bgruening/deeptools_bam_coverage/deeptools_bam_coverage/3.5.4+galaxy0) %} to create a bigWig file from a BAM or CRAM sequencing dataset.
{: .comment}


### Preparing Variant Calls

Variant calls in a `vcf` format can easily be transformed into the same format as we converted the BigWigs to.

> <hands-on-title>Dataset Pre-processing</hands-on-title>
>
> 1. {% tool [Cut](Cut1) %} with the following parameters:
>    - *"Cut columns"*: `c1,c2,c2,c6`
>    - {% icon param-file %} *"From"*: `variants.vcf` (Uploaded dataset)
>
>    > <question-title></question-title>
>    >
>    > 1. Why these columns? What do they represent?
>    > 1. Why is `c2` selected twice?
>    >
>    > > <solution-title></solution-title>
>    > >
>    > > 1. `c1` is the chromosome name, `c2` is the position of the variant, and `c6` is the quality column.
>    > > 2. `c2` is used twice because in Circos there are no 'point' values, everything has a start and end. So here we re-use the start position to represent a 1 base long feature.
>    > >
>    > {: .solution}
>    >
>    {: .question}
>
{: .hands_on}

### Preparing Gene Annotations

Gene annotations (gff3, bed, gtf), known as "intervals" in the Circos world, can be converted into a couple different formats, namely [text labels](http://circos.ca/documentation/tutorials/quick_start/text/) and [tiles](http://circos.ca/documentation/tutorials/quick_start/tiles/).

> <hands-on-title>Prepare gene calls</hands-on-title>
>
> 1. {% tool [Circos: Interval to Circos Text Labels](toolshed.g2.bx.psu.edu/repos/iuc/circos/circos_interval_to_text/0.69.8+galaxy9) %} with the following parameters:
>    - *"Data Format"*: `GFF3`
>        - {% icon param-file %} *"GFF3 File"*: `genes (NCBI).gff3` (Input dataset)
>        - *"GFF3 Attribute to pull value from"*: `Name`
>
> 1. {% tool [GFF-to-BED](gff2bed1) %} with the following parameters:
>    - {% icon param-file %} *"Convert this dataset"*: `genes (NCBI).gff3` (Input dataset)
>
> 1. {% tool [Circos: Interval to Tiles](toolshed.g2.bx.psu.edu/repos/iuc/circos/circos_interval_to_tile/0.69.8+galaxy9) %} with the following parameters:
>    - *"Data Format"*: `BED6+`
>        - {% icon param-file %} *"BED File (BED6+ only)"*: output of **GFF-to-BED** {% icon tool %}
>
>
{: .hands_on}


### Making the Plot

With our:

- gene calls
- variant calls
- and sequencing depth

We're ready to run Circos! As this is a 'near-final' circos plot it's requires complicated configuration. Normally you would reach configuration like this with a lot of iterations. It took the tutorial author around 20 executions of the Circos tool to produce this plot.

> <hands-on-title>Circos</hands-on-title>
>
> 1. {% tool [Circos](toolshed.g2.bx.psu.edu/repos/iuc/circos/circos/0.69.8+galaxy9) %} with the following parameters:
>    - In *"Karyotype"*:
>        - *"Reference Genome Source"*: ` FASTA File from History (can be slow, generate a length file to improve execution time.)`
>            - {% icon param-file %} *"Source FASTA Sequence"*: `genome.fa` (Uploaded dataset)
>    - In *"Ideogram"*:
>        - *"Chromosome units"*: `Kilobases`
>        - *"Spacing Between Ideograms (in chromosome units)"*: `0.3`
>        - *"Thickness"*: `10.0`
>        - In *"Labels"*:
>            - *"Label Font Size"*: `48`
>    - In *"2D Data Tracks"*:
>        - In *"2D Data Plot"*:
>            - {% icon param-repeat %} *"Insert 2D Data Plot"*
>                - *"Outside Radius"*: `0.98`
>                - *"Inside Radius"*: `0.92`
>                - *"Plot Type"*: `Histogram`
>                    - {% icon param-file %} *"Histogram Data Source"*: output of **Circos: bigWig to Scatter**  on RNA Seq Coverage 2 {% icon tool %}
>                    - In *"Plot Format Specific Options"*:
>                        - *"Fill Color"*: `#f08fa4`
>
>            - {% icon param-repeat %} *"Insert 2D Data Plot"*
>                - *"Outside Radius"*: `0.92`
>                - *"Inside Radius"*: `0.86`
>                - *"Plot Type"*: `Histogram`
>                    - {% icon param-file %} *"Histogram Data Source"*: output of **Circos: bigWig to Scatter**  on RNA Seq Coverage 1 {% icon tool %}
>                    - In *"Plot Format Specific Options"*:
>                        - *"Fill Color"*: `#8ff0a4`
>
>            - {% icon param-repeat %} *"Insert 2D Data Plot"*
>                - *"Outside Radius"*: `0.86`
>                - *"Inside Radius"*: `0.8`
>                - *"Plot Type"*: `Histogram`
>                    - {% icon param-file %} *"Histogram Data Source"*: output of **Circos: bigWig to Scatter**  on DNA sequencing coverage {% icon tool %}
>                    - In *"Plot Format Specific Options"*:
>                        - *"Fill Color"*: `#ffbe6f`
>
>            - {% icon param-repeat %} *"Insert 2D Data Plot"*
>                - *"Outside Radius"*: `0.79`
>                - *"Inside Radius"*: `0.6`
>                - *"Z-index"*: `10` (This is used to plot *over* the genes which are added later.)
>                - *"Plot Type"*: `Scatter`
>                    - {% icon param-file %} *"Scatter Data Source"*: output of **cut**  on variants.vcf {% icon tool %}
>                    - In *"Plot Format Specific Options"*:
>                        - *"Glyph"*: `Triangle`
>                        - *"Glyph Size"*: `6`
>                        - *"Fill Color"*: `#dc8add`
>                        - *"Stroke Thickness"*: `0`
>                - In *"Axes"*:
>                    - In *"Axis"*:
>                        - {% icon param-repeat %} *"Insert Axis"*
>                            - *"Radial Position"*: `Absolute position (values match data values)`
>                                - *"Spacing"*: `5000.0`
>                            - *"y1"*: `40000.0`
>                            - *"Color"*: `#1a5fb4`
>                            - *"Color Transparency"*: `0.4`
>
>            - {% icon param-repeat %} *"Insert 2D Data Plot"*
>                - *"Outside Radius"*: `0.6`
>                - *"Inside Radius"*: `0.55`
>                - *"Plot Type"*: `Text Labels`
>                    - {% icon param-file %} *"Text Data Source"*: output of **Circos: Interval to Text**  on genes (NCBI).gff {% icon tool %}
>                    - In *"Plot Format Specific Options"*:
>                        - *"Label Size"*: `18`
>                        - *"Show Link"*: `No`
>                        - *"Snuggle Labels"*: `Yes`
>
>            - {% icon param-repeat %} *"Insert 2D Data Plot"*
>                - *"Outside Radius"*: `0.7`
>                - *"Inside Radius"*: `0.6`
>                - *"Plot Type"*: `Tiles`
>                    - {% icon param-file %} *"Tile Data Source"*: output of **Circos: Interval to Tiles**  on genes (NCBI).gff {% icon tool %}
>                    - In *"Plot Format Specific Options"*:
>                        - *"Fill Color"*: `#1c71d8`
>                        - *"Overflow Behavior"*: `Hide: overflow tiles are not drawn`
>                - In *"Rules"*:
>                    - In *"Rule"*:
>                        - {% icon param-repeat %} *"Insert Rule"*
>                            - In *"Conditions to Apply"*:
>                                - {% icon param-repeat %} *"Insert Conditions to Apply"*
>                                    - *"Condition"*: `Based on qualifier value (when available)`
>                                        - *"Qualifier name"*: `strand`
>                                        - *"Condition"*: `Less than (numeric)`
>                                        - *"Qualifier value to compare against"*: `0`
>                            - In *"Actions to Apply"*:
>                                - {% icon param-repeat %} *"Insert Actions to Apply"*
>                                    - *"Action"*: `Change Visibility`
>            - {% icon param-repeat %} *"Insert 2D Data Plot"*
>                - *"Outside Radius"*: `0.53`
>                - *"Inside Radius"*: `0.45`
>                - *"Plot Type"*: `Tiles`
>                    - {% icon param-file %} *"Tile Data Source"*: output of **Circos: Interval to Tiles**  on genes (NCBI).gff {% icon tool %}
>                    - In *"Plot Format Specific Options"*:
>                        - *"Overflow Behavior"*: `Hide: overflow tiles are not drawn`
>                - *"Orient Inwards"*: `Yes`
>                - In *"Rules"*:
>                    - In *"Rule"*:
>                        - {% icon param-repeat %} *"Insert Rule"*
>                            - In *"Conditions to Apply"*:
>                                - {% icon param-repeat %} *"Insert Conditions to Apply"*
>                                    - *"Condition"*: `Based on qualifier value (when available)`
>                                        - *"Qualifier name"*: `strand`
>                                        - *"Condition"*: `Greater than (numeric)`
>                                        - *"Qualifier value to compare against"*: `0`
>                            - In *"Actions to Apply"*:
>                                - {% icon param-repeat %} *"Insert Actions to Apply"*
>                                    - *"Action"*: `Change Visibility`
>                        - {% icon param-repeat %} *"Insert Rule"*
>                            - In *"Conditions to Apply"*:
>                                - {% icon param-repeat %} *"Insert Conditions to Apply"*
>                                    - *"Condition"*: `Apply to Every Point`
>                            - In *"Actions to Apply"*:
>                                - {% icon param-repeat %} *"Insert Actions to Apply"*
>                                    - *"Action"*: `Change Fill Color for all points`
>                                        - *"Fill Color"*: `#99c1f1`
>            - {% icon param-repeat %} *"Insert 2D Data Plot"*
>                - *"Outside Radius"*: `0.45`
>                - *"Inside Radius"*: `0.35`
>                - *"Plot Type"*: `Histogram`
>                    - {% icon param-file %} *"Histogram Data Source"*: output of **Circos: bigWig to Scatter** on the GC Skew Plot {% icon tool %}
>                    - In *"Plot Format Specific Options"*:
>                        - *"Fill Color"*: `#ff5757`
>                - In *"Rules"*:
>                    - In *"Rule"*:
>                        - {% icon param-repeat %} *"Insert Rule"*
>                            - In *"Conditions to Apply"*:
>                                - {% icon param-repeat %} *"Insert Conditions to Apply"*
>                                    - *"Condition"*: `Based on value (ONLY for scatter/histogram/heatmap/line)`
>                                        - *"Points below this value"*: `0.0`
>                            - In *"Actions to Apply"*:
>                                - {% icon param-repeat %} *"Insert Actions to Apply"*
>                                    - *"Action"*: `Change Fill Color for all points`
>                                        - *"Fill Color"*: `#5092f7`
>    - In *"Ticks"*:
>        - *"Skip first label"*: `Yes`
>        - In *"Tick Group"*:
>            - {% icon param-repeat %} *"Insert Tick Group"*
>                - *"Tick Spacing"*: `10.0`
>                - *"Tick Size"*: `20.0`
>                - *"Show Tick Labels"*: `Yes`
>            - {% icon param-repeat %} *"Insert Tick Group"*
>                - *"Tick Size"*: `15.0`
>                - *"Show Tick Labels"*: `No`
>            - {% icon param-repeat %} *"Insert Tick Group"*
>                - *"Tick Spacing"*: `0.25`
>                - *"Color"*: `#9a9996`
>                - *"Show Tick Labels"*: `No`
>
>    > <comment-title>Circos is complicated</comment-title>
>    > Please check your parameters carefully, and expect that mistakes can be made. Just re-run the tool and modify your parameters!
>    > And while this example is probably very overwhelming, when you create a
>    > Circos plot from scratch, it will be less overwhelming; it'll be your
>    > data which you know better, and you'll add one track at a time.
>    >
>    > ![gif of a circos plot being iteratively modified to reach something that looks like the final plot](images/montage.gif)
>    {: .comment}
>
{: .hands_on}

Congratulations on plotting a microbial genome subset in Circos!

## Conclusion

Plotting with Circos is essentially infinitely customisable but here we offer suggestions for a default plotting workflow.

tools_json:
["GC Skew", "toolshed.g2.bx.psu.edu/repos/iuc/circos/circos_gc_skew/0.69.8+galaxy9", "Circos: bigWig to Scatter", "toolshed.g2.bx.psu.edu/repos/iuc/circos/circos_wiggle_to_scatter/0.69.8+galaxy9", "bamCoverage: generates a coverage bigWig file from a given BAM or CRAM file", "toolshed.g2.bx.psu.edu/repos/bgruening/deeptools_bam_coverage/deeptools_bam_coverage/3.5.4+galaxy0", "Cut", "Cut1", "Circos: Interval to Circos Text Labels", "toolshed.g2.bx.psu.edu/repos/iuc/circos/circos_interval_to_text/0.69.8+galaxy9", "GFF-to-BED", "gff2bed1", "Circos: Interval to Tiles", "toolshed.g2.bx.psu.edu/repos/iuc/circos/circos_interval_to_tile/0.69.8+galaxy9", "Circos", "toolshed.g2.bx.psu.edu/repos/iuc/circos/circos/0.69.8+galaxy9"]


