# tutorials

topic: proteomics
tutorial: DIA_Analysis_MSstats
title: Statistical analysis of DIA data
difficulty: intermediate
time_estimation: 1H
tools_json: []
requirements_json: []
tags_json: ["[DIA]"]
content_hash: 753b8637c0645052cdd959290bf8f26b
last_modified: 2026-02-26T21:12:17.457996
zenodo_link: https://zenodo.org/record/4307758
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/DIA_Analysis_MSstats/tutorial.html

content:
This training covers the statistical analysis of data independent acquisition (DIA) mass spectrometry (MS) data, after successfull identification and quantification of peptides and proteins. We therefore recommend to first go through the [DIA library generation tutorial]({{site.baseurl}}/topics/proteomics/tutorials/DIA_lib_OSW/tutorial.html) as well as the [DIA analysis tutorial]({{site.baseurl}}/topics/proteomics/tutorials/DIA_Analysis_OSW/tutorial.html), which teach the principles and characteristics of DIA data analysis.

Modern mass spectrometry approaches enables the identification and quantification of thousands of proteins and tens of thousands of peptides in single measurements. This provides immense potential to in-depth explorative analysis of a variety of biological samples. However, often the number of available samples is limited leading to large proteomic datasets with only a few numbers of replicates or samples per condition. Thus, the statistical analysis remains challenging in such in-depth proteomic studies.

Here we will use **MSstats**, which enables the statistical analysis and processing of proteomic data ({% cite Choi2014 %}).

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

## Get data
> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this tutorial and give it a meaningful name
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the DIA analysis results, the sample annotation and the comparison matrix from [Zenodo](https://zenodo.org/record/4307758)
>    ```
>    https://zenodo.org/record/4307758/files/PyProphet_export.tabular
>    https://zenodo.org/record/4307758/files/Sample_annot_MSstats.txt
>    https://zenodo.org/record/4307758/files/Comp_matrix_HEK_Ecoli.txt
>    https://zenodo.org/record/4307758/files/PyProphet_msstats_input.tabular
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Once the files are green, rename the sample annotation file in 'Sample_annot_MSstats', the comparison matrix file in 'Comp_matrix_HEK_Ecoli' and the two DIA analysis results files in 'PyProphet_export' and 'PyProphet_msstats_input'
>
>    {% snippet faqs/galaxy/datasets_rename.md %}
>
{: .hands_on}


# Statistical analysis with **MSstats**

> <hands-on-title>Performing statistical analysis using <b>MSstats</b> and the tabular output from <b>PyProphet export</b></hands-on-title>
>
> 1. {% tool [MSstats](toolshed.g2.bx.psu.edu/repos/galaxyp/msstats/msstats/3.20.1.0) %} with the following parameters:
>    - *"input source"*: `OpenSWATH`
>        - {% icon param-file %} *"OpenSWATH_input"*: `PyProphet_export`
>        - {% icon param-file %} *"OpenSWATH_annotation"*: `Sample_annot_MSstats`
>    - *"Compare Groups"*: `Yes`
>        - {% icon param-file %} *"Comparison Matrix"*: `Comp_matrix_HEK_Ecoli`
>
>    > <comment-title>data Process and group comparison</comment-title>
>    >During the data process step the peptide intensities are normalized and protein inference is performed. Using a predefined comparison matrix multiple comparisons can be performed.
>    {: .comment}
>
{: .hands_on}


> <question-title></question-title>
>
> 1. How many lines does the PyPyprophet_export.tabular file have? How many lines does the ProcessedData have and do you notice any differences in their structure or format?
> 2. How many proteins were used for the Group comparison? (see ComparisonResult)
>
> > <solution-title></solution-title>
> >
> > 1. The PyPyprophet_export.tabular has appr. 230.000 lines whereas the ProcessedData has over 1 mio lines and is in the so called long format. Here every individual transition (single m/z value) is reported per row.
> > 2. The ComparisonResult has 5022 lines, meaning over 5000 Proteins were compared between the two different Spike-in conditions.
> >
> {: .solution}
>
{: .question}


> <tip-title>Continue with results from Zenodo</tip-title>
>
> In case the **MSstats** run is not yet finished, the results can be downloaded from Zenodo to be able to continue the tutorial
> 1. Import the files from [Zenodo](https://zenodo.org/record/4307758)
>    ```
>    https://zenodo.org/record/4307758/files/MSstats_ComparisonResult_export_tabular.tsv
>    ```
{: .tip}

## Detailed investigation of Ecoli identifications and quantifications

> <hands-on-title>Investigating Ecoli proteins in the MSstats comparison results</hands-on-title>
>
> 1. {% tool [Select](Grep1) %} with the following parameters:
>    - {% icon param-file %} *"Select lines from"*: `MSstats_ComparisonResult_export_tabular` (output of **MSstats** {% icon tool %})
>    - *"the pattern"*: `(ECOLI)|(log2FC)`
>
> 2. {% tool [Filter](Filter1) %} with the following parameters:
>    - {% icon param-file %} *"Filter"*: `Select_Ecoli` (output of **Select** {% icon tool %})
>    - *"With following condition"*: `c7!='NA'`
>    - *"Number of header lines to skip"*: `1`
>
> 3. {% tool [Histogram](toolshed.g2.bx.psu.edu/repos/devteam/histogram/histogram_rpy/1.0.4) %} with the following parameters:
>    - {% icon param-file %} *"Dataset"*: `Filter_Ecoli` (output of the previous **Filter** {% icon tool %})
>    - *"Numerical column for x axis"*: `Column: 3`
>    - *"Number of breaks (bars)"*: `25`
>    - *"Plot title"*: `Distribution of Ecoli Protein log FC values`
>    - *"Label for x axis"*: `log2 Fold Change`
>
>
>    > <comment-title>Extracting Ecoli information</comment-title>
>    >First we only select rows containing specific terms such as "Ecoli" from the complete ComparisonResults file. Afterwards, the table is filtered to containg only proteins with valid statistical information (e.g. p-value). Using the log2 Fold change values from all remaining Ecoli proteins we can observe the distribution of log2FC values from the comparison of the two Spike-in conditions.
>    {: .comment}
>
{: .hands_on}

> <question-title></question-title>
>
> 1. How many Ecoli proteins were identified and for how many was the p-value for the comparison of the two Spike-in conditions computed?
> 2. How does the distribution of the log2FC values look like? Which Spike-in contained higher amounts of Ecoli and is it possible to see how much more Ecoli was spiked-in?
>
> > <solution-title></solution-title>
> >
> > 1. In total, over 800 Ecoli proteins were identified from which 500 have a p-value for the comparison of the two Spike-in conditions.
> > 2. We can see a gaussian distribution of the log2FC values around a positive value of 3. Since we compared Spike_in_2 / Spike_in_1 we can directly see that Spike_in_2 contained higher amounts of Ecoli. Furthermore, since the apex of the distribution is around 3 and we compared log2 intensities, we could estimate that Spike_in_2 contained approx. 8-times more Ecoli than Spike_in_1.
> >
> {: .solution}
>
{: .question}


# Statistical analysis with **MSstats**

> <hands-on-title>Performing statistical analysis using <b>MSstats</b> and the msstats_input from <b>PyProphet export</b></hands-on-title>
>
> 1. {% tool [MSstats](toolshed.g2.bx.psu.edu/repos/galaxyp/msstats/msstats/3.20.1.0) %} with the following parameters:
>    - *"input source"*: `MStats 10 column format`
>        - {% icon param-file %} *"MSstats 10-column input"*: `PyProphet_msstats_input`
>    - *"Compare Groups"*: `Yes`
>
>
>    > <comment-title>MSstats input format</comment-title>
>    >For the statistical analysis using MSstats the input must be in the long format, containing all relevant information in 10 predefined columns. The conversion of the **PyProphet export** output can either be done using **MSstats** (as we did above), or during the **Pyprophet export** step by using another R package called swath2stats ({% cite Blattmann2016 %}). Prior to the conversion the data can be processed and filtered using the swath2stats functionalities.
>    {: .comment}
>
{: .hands_on}

> <question-title></question-title>
>
> 1. How many lines does the `PyProphet_msstats_input.tabular` file have? How many lines does the ProcessedData have and do you notice any differences in their structure or format?
> 2. How many proteins were used for the Group comparison? And do you already see a difference to the first **MSstats** step?
>
> > <solution-title></solution-title>
> >
> > 1. The `PyProphet_msstats_input.tabular` has over 870.000 lines and the ProcessedData has over 1 mio lines. Here both files are in the long format, in which every transition is reported per row.
> > 2. Here the ComparisonResult has only 3871 lines, meaning that almonst 1200 fewer proteins were used in the comparison of the two spike-in conditions.
> >
> {: .solution}
>
{: .question}

> <tip-title>Continue with results from Zenodo</tip-title>
>
> In case the **MSstats** run is not yet finished, the results can be downloaded from Zenodo to be able to continue the tutorial
> 1. Import the files from [Zenodo](https://zenodo.org/record/4307758)
>    ```
>    https://zenodo.org/record/4307758/files/MSstats_ComparisonResult_msstats_input.tsv
>    ```
{: .tip}

## Detailed investigation of Ecoli identifications and quantifications

> <hands-on-title>Investigating Ecoli proteins in the MSstats comparison results</hands-on-title>
>
> 1. {% tool [Select](Grep1) %} with the following parameters:
>    - {% icon param-file %} *"Select lines from"*: `MSstats_ComparisonResult_msstats_input` (output of the second **MSstats** {% icon tool %})
>    - *"the pattern"*: `(ECOLI)|(log2FC)`
>
> 2. {% tool [Filter](Filter1) %} with the following parameters:
>    - {% icon param-file %} *"Filter"*: `Select_Ecoli` (output of the second **Select** {% icon tool %})
>    - *"With following condition"*: `c7!='NA'`
>    - *"Number of header lines to skip"*: `1`
>
> 3. {% tool [Histogram](toolshed.g2.bx.psu.edu/repos/devteam/histogram/histogram_rpy/1.0.4) %} with the following parameters:
>    - {% icon param-file %} *"Dataset"*: `Filter_Ecoli` (output of the previous **Filter** {% icon tool %})
>    - *"Numerical column for x axis"*: `Column: 3`
>    - *"Number of breaks (bars)"*: `25`
>    - *"Plot title"*: `Distribution of Ecoli Protein log FC values`
>    - *"Label for x axis"*: `log2 Fold Change`
>
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. How many Ecoli proteins were identified and for how many was the p-value for the comparison of the two Spike-in conditions computed? Are there any differences compared to the selected and filtered results from the previous **MSstats** step?
> 2. How does the distribution of the log2FC values look like? Are there any differences compared to the selected and filtered results from the previous **MSstats** step?
>
> > <solution-title></solution-title>
> >
> > 1. In total, over 600 Ecoli proteins were identified from which 500 have a p-value for the comparison of the two Spike-in conditions. Here we identify 200 Ecoli proteins less than before, however, the number of proteins for which a p-value was calculated differs only slightly.
> > 2. Generally, the two log2FC distribution look very similar, showing a gaussian distribution of the log2FC values around a positive value of 3. There seems to be a slight difference of the apex of the distribution, in the first **MSstats** analysis it seems to be higher than 3, whereas in the second **MSstats** analysis the apex seems to be lower than 3.
> >
> {: .solution}
>
{: .question}



# Conclusion


![DIA_MSstats_pipe](../../images/DIA_MSstats_sum.png "All-in one workflow for DIA analysis in Galaxy. The statistical analysis using MSstats is highlighted in green.")

Using **MSstats** we were able to identify and quantify differentially regulated proteins between two Spike-in conditions in a HEK/Ecoli Benchmark DIA datatset. Furthermore, the preprocessing of the proteomic data prior to the statistical analysis can directly impact results. Thus, it might be beneficial to try various ways of intermediate data processing and statistical analysis to increase the sensitivity and specificity of the investigation.

![DIA_MSstats_galaxy](../../images/DIA_MSstats_galaxy.png "Galaxy worfklow containing all necessary steps for the statistical analysis of DIA data using MSstats")


topic: proteomics
tutorial: maxquant-msstats-dda-lfq
title: MaxQuant and MSstats for the analysis of label-free data
difficulty: intermediate
time_estimation: 2H
tools_json: []
requirements_json: []
tags_json: ["[label-free]"]
content_hash: f828dce4d0a4491cdc0ad71461e31ef6
last_modified: 2026-02-26T21:12:17.492222
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/maxquant-msstats-dda-lfq/tutorial.html

content:
Modern mass spectrometry-based proteomics enables the identification and quantification of thousands of proteins. Therefore, quantitative mass spectrometry represents an indispensable technology for biological and clinical research. Statistical analyses are required for the unbiased answering of scientific questions and to uncover all important information in the proteomic data. Classical statistical approaches and methods from other omics technologies are not ideal because they do not take into account the speciality of mass spectrometry data that include several thousands of proteins but often only a few dozens of samples (referred to as ‘curse of dimensionality’) and stochastic data properties that reflect sample preparation and spectral acquisition (Choi 2014).

In this training we will cover the full analysis workflow from label-free, data dependent acquisition (DDA) raw data to statistical results. We’ll use two popular quantitative proteomics software: MaxQuant and MSstats. MaxQuant allows protein identification and quantification for many different kinds of proteomics data (Cox and Mann 2008).  In case you have no previous experience with MaxQuant, we recommend to go through the [MaxQuant beginners tutorial]({{site.baseurl}}/topics/proteomics/tutorials/maxquant-label-free/tutorial.html) before. MSstats provides statistical functionalities to find differentially abundant peptides or proteins from data dependent acquisition (DDA), data independent acquisition (DIA) or single reaction monitoring (SRM) proteomic experiments.
The training dataset consists of a skin cancer cohort of 19 patients, which is a subset of a published study {% cite Fll2018 %}. One fifth of all non melanoma skin cancers are cutaneous squamous cell carcinomas (cSCC) that mainly derive from exposure to ultraviolet light. Most cSCC have a good prognosis but the few metastasizing cSCC have dramatically increased mortality. Here, we compare these metastasizing cSCC to cSCC in patients with the genetic disease recessive dystrophic epidermolysis bullosa (RDEB). RDEB is a genetic skin blistering and extracellular matrix disease caused by collagen VII deficiency. To investigate molecular differences between these two aggressive cSCCs with different origin, we used global proteomic analysis of formalin-fixed paraffin-embedded human cSCC tissues.


> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Get data


The annotation file, group comparison file and FASTA file for this training is deposited at [Zenodo](https://zenodo.org/records/17177151). It is of course possible to use another FASTA file with human proteome sequences, but to ensure that the results are compatible we recommend to use the provided FASTA file. MaxQuant not only adds known contaminants to the FASTA file, but also generates the “decoy” hits for false discovery rate estimation itself, therefore the FASTA file is not allowed to have decoy entries. To learn more about FASTA files, have a look at [Protein FASTA Database Handling tutorial]({{site.baseurl}}/topics/proteomics/tutorials/database-handling/tutorial.html). The raw data is available via the [PRIDE repository](https://www.ebi.ac.uk/pride/archive/projects/PXD006914). As this is a real life study, the raw data sizes are large and computation time in MaxQuant is long. To save time and storage capacity, you can skip downloading the raw data and the MaxQuant run and instead continue with the MaxQuant outputs which we provide later on. In this case skip the data upload steps 5-8 which are only necessary for the MaxQuant run.


> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this tutorial and give it a meaningful name
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the FASTA database, annotation file and comparison matrix from [Zenodo](https://zenodo.org/records/17177151)
>
>    ```
>    https://zenodo.org/records/17177151/files/input_protein_database.fasta
>    https://zenodo.org/records/17177151/files/input_annotation_file.tabular
>    https://zenodo.org/records/17177151/files/input_comparison_matrix.tabular
>    ```
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Once the files are green, rename the fasta file into 'protein database', the annotation file into 'annotation file' and the comparison matrix file into 'comparison matrix'.
>
>    {% snippet faqs/galaxy/datasets_rename.md %}
>
> 4. You can save time and storage capacity when running this tutorial by skipping the first step and not running MaxQuant. In that case, upload the MaxQuant results listed in the MaxQuant analysis section. If you want to run MaxQuant, import the raw data from [PRIDE](https://www.ebi.ac.uk/pride/archive/projects/PXD006914) in a collection:
>    1. Open the upload menu and slect the "Rule Based" tab.
>    2. Paste the data urls:
>       ```
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment105_metast_cSCC1.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment106_metast_cSCC2.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment107_metast_cSCC3.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment109_metast_cSCC4.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment116_metast_cSCC5.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment117_metast_cSCC6.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment118_metast_cSCC7.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment119_metast_cSCC8.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment120_metast_cSCC9.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment121_metast_cSCC10.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment122_metast_cSCC11.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment123_metast_cSCC12.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment124_metast_cSCC13.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment110_RDEB_cSCC1.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment111_RDEB_cSCC2.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment112_RDEB_cSCC3.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment113_RDEB_cSCC4.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment114_RDEB_cSCC5.raw
>       https://ftp.pride.ebi.ac.uk/pride-archive/2017/11/PXD006914/Experiment115_RDEB_cSCC6.raw
>        ```
>    3. Upload Type `Collections`
>    4. Click on `Build`
>    5. We need our datasets to be named '**metast_cSCC1.raw**', '**metast_cSCC2.raw**', etc.. The naming for the raw files have to be exactly this way to later match the file names provided in the MSstats annotation file.
>    6. Select `+ Column`, `using a Regular Expression`, then "Create columns matching expression groups". Use the expression `Experiment[0-9]+_(.+)` and Click `Apply` at the end. This will extract the file names from the url.
>    7. Add column with the fixed value `thermo.raw` to specify the format of the files
>    8. Add column with the fixed value `raw_files`. This will be the collection name.
>    9. We will now  add columns definitions: Select `+ Rules`, then `Add/Modify Column definitions`. Select `+ Add Defenitions` and Set the URL as column `A`, List Identifiers as column `B`, Collection Name as column `D`, and Type as column `C`. Then `Apply`.
>    10. Click on `Upload` to start the upload.
>    ![upload](../../images/maxquant-msstats-lfq/upload.png "Rule Based Upload")
>
{: .hands_on}

# MaxQuant analysis

The run time of **MaxQuant** {% icon tool %} depends on the number and size of the input files and on the chosen parameters. The run of the training datasets will take a few hours, but the training can be directly continued with the MaxQuant result files from Zenodo. We start the MaxQuant run with the default parameters, with a few adjustments. Protein level quantification parameters do not really matter here, because MSstats will use feature quantifications and perform protein summarization based on them. A quality control report is generated with the PTXQC functionality {% cite Bielow2015 %} that is directly implemented in the MaxQuant Galaxy tool. To continue with statistical analysis in MSstats, the Protein Groups and the Evidence files are needed from MaxQuant.

> <hands-on-title>Optional: MaxQuant analysis</hands-on-title>
>
> 1. {% tool [MaxQuant](toolshed.g2.bx.psu.edu/repos/galaxyp/maxquant/maxquant/2.0.3.0+galaxy0) %} with the following parameters:
>    - In *"Input Options"*:
>        - {% icon param-file %} *"FASTA files"*: `protein database`
>    - In *"Search Options"*:
>    - *"minimum unique peptides"*: `1`
>    - *"Match between runs"*: `yes`
>    - In *"Parameter Group"*:
>        - {% icon param-collection %} *"Infiles"*: `raw_files`
>    - *"Generate PTXQC (proteomics quality control pipeline) report? (experimental setting)"*: `True`
>    - In *"Output Options"*:
>        - *"Select the desired outputs."*: `Protein Groups` `Evidence`
>
{: .hands_on}

Because the MaxQuant run takes really long, we recommend to download the MaxQuant results from Zenodo and continue with the tutorial.

> <hands-on-title>Load MaxQuant results from Zenodo</hands-on-title>
>
> 1. Import the files from [Zenodo](https://zenodo.org/records/17177151)
>
>    ```
>    https://zenodo.org/records/17177151/files/MaxQuant%20Evidence.tabular
>    https://zenodo.org/records/17177151/files/MaxQuant%20Protein%20Groups.tabular
>    https://zenodo.org/records/17177151/files/PTXQC%20report.pdf
>    ```
> 2. Rename the files `Evidence`, `Protein Groups`, and `PTXQC report` respectively.
>
{: .hands_on}


> <question-title></question-title>
>
> 1. How many proteins and features were identified in total?
> 2. In which columns (number) are the potential contaminants in the protein group and evidence file respectively?
> 3. How large is the proportion of potential contaminants?
>
> > <solution-title></solution-title>
> >
> > 1. 2512 protein groups and ~240000 features were found in total (number of lines of protein group and evidence files)
> > 2. They are in column 138 (protein groups) and 50 (evidence)
> > 3. Up to ~60% of the samples intensities derive from potential contaminants (PTXQC plots page 6)
> >
> {: .solution}
>
{: .question}


# MSstats analysis

The protein groups and evidence files of MaxQuant can directly be input into MSstats. MSstats automatically removes all proteins that are labelled as contaminants ('+' sign in the column 'potential contaminant' of both MaxQuant outputs). However, in this skin dataset we expect that the skin proteins are part of the sample and not a contamination. Therefore, we keep all human contaminants by first removing non human proteins (by selecting only lines that contain the word 'HUMAN' or a word from the header line) and then for the human potential contaminants remove the '+' (replacing '+' with ''(empty field) in the 'potential contaminant' column) to keep them for the analysis.

We use the modified MaxQuant protein groups and evidence files as input in MSstats. In addition, an annotation file that describes the experimental design and a comparison matrix is needed. Please start the MSstats run first and while it is running you can find more details on its parameters below.

> <hands-on-title>MSstats Analysis</hands-on-title>
>
> 1. {% tool [Select](Grep1) %} with the following parameters:
>    - {% icon param-file %} *"Select lines from"*: `Protein Groups` (output of **MaxQuant** {% icon tool %})
>    - *"the pattern"*: `(HUMAN)|(Majority)`
>    - Once finished, rename the output `select protein groups`
> 2. {% tool [Select](Grep1) %} with the following parameters:
>    - {% icon param-file %} *"Select lines from"*: `Evidence` (output of **MaxQuant** {% icon tool %})
>    - *"the pattern"*: `(HUMAN)|(Sequence)`
>    - Once finished, rename the output `select evidence`
> 3. {% tool [Replace Text in a specific column](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_column/9.5+galaxy2) %} with the following parameters:
>    - {% icon param-file %} *"File to process"*: `select protein groups` (output of **Select** {% icon tool %})
>    - In *"Replacement"*:
>            - *"in column"*: `Column: 138`
>            - *"Find pattern"*: `+`
>    - Once finished, rename the file into `protein groups input for MSstats`
> 4. {% tool [Replace Text in a specific column](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_column/9.5+galaxy2) %} with the following parameters:
>    - {% icon param-file %} *"File to process"*: `select evidence` (output of **Select** {% icon tool %})
>    - In *"Replacement"*:
>            - *"in column"*: `Column: 50`
>            - *"Find pattern"*: `+`
>    - Once finished, rename the file into `evidence input for MSstats`
> 5. {% tool [MSstats](toolshed.g2.bx.psu.edu/repos/galaxyp/msstats/msstats/4.0.0+galaxy1) %}  with the following parameters:
>    - *"input source"*: `MaxQuant`
>        - {% icon param-file %} *"evidence.txt - feature-level data"*: `evidence input for MSstats` (output of **Replace** {% icon tool %})
>        - {% icon param-file %} *"proteinGroups.txt - protein-level data"*: `protein groups input for MSstats` (output of **Replace** {% icon tool %})
>        - {% icon param-file %} *"annotation file"*: `annotation file`
>        - *"Select Protein ID in evidence.txt"*: `Leading razor protein column`
>        - In *"MaxQtoMSstatsFormat Options"*:
>            - *"Remove the proteins which have only 1 peptide and charge"*: `Yes`
>    - In *"dataProcess Options"*:
>        - *"Select outputs"*: `MSstats log` `MSstats FeatureLevelData` `MSstats ProteinLevelData` `Sample Quantification Matrix Table` `Group Quantification Matrix Table`
>        - In *"dataProcess Plot Options"*:
>            - *"Select visualization outputs"*: `MSstats QCPlot`
>            - *"Select protein IDs to draw plots"*: `Option for QC plot: "allonly" will generate one QC plot with all proteins`
>    	     - In *"Advanced visualization parameters"*:
>                - *"Angle of labels represented each condition at the top of graph"*: `0`
>    - *"Compare Groups"*: `Yes`
>        - {% icon param-file %} *"Comparison Matrix"*: `comparison_matrix`
>        - *"Select outputs"*: `MSstats ComparisonResult`
>        - In *"Comparison Visualization Options"*:
>            - *"Select visualization outputs"*: `MSstats VolcanoPlot`
>    	     - In *"Advanced visualization parameters"*:
>                - *"Involve fold change cutoff or not for volcano plot or heatmap"*: `1.5`
>                - *"For volcano plot or heatmap, logarithm transformation of adjusted p-valuewith base 2 or 10"*: `2`
>                - *"Display protein names in Volcano Plot"*: `No`
>
{: .hands_on}


> <question-title></question-title>
>
> 1. How many proteins were removed as potential non human contaminants?
> 2. How many proteins were included into the statistical analysis?
>
> > <solution-title></solution-title>
> >
> > 1. 27 (2518 lines in protein group file minus 2491 lines after select)
> > 2. 1693 (MSstats log)
> >
> {: .solution}
>
{: .question}



## More details on MSstats

MSstats  is designed for statistical modelling of mass spectrometry based proteomic data {% cite Choi2014 %}.
Proteomic data analysis requires statistical approaches that reduce bias and inefficiencies and distinguish systematic variation from random artifacts {% cite Kll2011 %}.

MSstats is directly compatible with the output of several quantitative proteomics software. In addition to the results of the proteomics software an annotation file is needed as input. The annotation file describes the experimental design such as the conditions, biological and technical replicates. To be compatible with MaxQuant results, an additional column with the label type is needed, which only contains L (light) in DDA experiments. A wrong setup of the annotation file is the most common source of errors in MSstats, thus we collected more information in the box below to allow you to adjust the annotation file when analyzing your own experiments.

> <tip-title>Generating the MSstats annotation file</tip-title>
>
> For label-free MaxQuant data, the annotation file should have 5 columns with exactly these headers: Raw.file, Condition, BioReplicate, Run; IsotopeLabelType
> 1. Raw.file: The names must match exactly to the file names in the MaxQuant evidence.txt "Raw file" column. (e.g. "file1.raw.thermo").
> 2. Condition: The conditions which will be compared in the statistical modelling. They are not allowed to start with a number or contain any special characters except for '_'.
> 3. BioReplicate: This column should contain a unique identifier for each biological replicate in the experiment. For example, in a clinical proteomic investigation this should be a unique patient id. If technical replicates are present, all samples from the same biological replicate should have the same id but different run ids. MSstats automatically detects the presence of technical replicates and accounts for them in the model-based analysis.
> 4. Run: This column contains the identifier of a mass spectrometry run. Each mass spectrometry run should have a unique identifier (number or name), regardless of the origin of the biological sample.
> 5. IsotopeLabelType: This is L (light) for all MaxQuant DDA experiments.
>
{: .tip}

MSstats will compare all conditions that are indicated in the comparison matrix. The comparison matrix has to be setup correctly to avoid errors and wrong statistical modelling. It contains a first column that gives the comparisons a name and one column per condition. This matrix is filled with 1 and -1 to specify the conditions that are compared in each comparison and with 0 for conditions that are not part of the comparison.

> <tip-title>Generating the MSstats comparison matrix</tip-title>
>
> 1. The first column of the comparison matrix contains the names of the comparisons and should have 'names' as header. These names will be used in all MSstats output files, therefore it is important that the names are meaningful and reflect the actual comparison (see below)
> 2. An additional column for each condition that is present in the data. This means each condition present in the annotation file has to be a separate column even when the condition will not be used for any comparison. The header should contain the condition name exactly as written in the annotation file.
> 3. Fill the matrix: Use 1 and -1 to indicate the conditions to compare and 0 for conditions that are not compared. Multiple groups can be combined by using 0.5.
> 4. Example: to compare condition1 with condition2: write 1 into the column of condition1 and write -1 in the column of condition2. In the first column of the matrix name this comparison condition1-condition2. The naming of the comparison should reflect the direction of the comparison and thus always have the condition that is set to 1 first and the condition that is set to -1 second (condition1-condition2 and NOT condition2-condition1).
>
{: .tip}

The first analysis step in MSstats is the conversion of the input data into an MSstats compatible table. For this step several parameters to filter and adjust the input data can be selected. We keep the default parameters and only change one parameter in order to remove proteins which have only a single peptide measurement.
Next, data processing optimizes the data for statistical modelling via log-transformation, and normalization of intensities, feature selection, missing value imputation, and run-level summarization.
Log- transformation is performed to transform multiplicative signals to additive signals which are compatible with linear statistical models and bring the intensity distribution close to a normal distribution. Furthermore, it changes the dependence of variances from the intensity values: in the raw data larger intensities have larger variances but after log transformation lower intensities have larger variances.

Normalization aims to make the intensities of different runs more comparable to each other. The default normalization method, equalize medians, assumes that the majority of proteins do not change across runs and shifts all intensities of a run by a constant to obtain equal median intensities across runs.
A feature in label-free DDA data corresponds to a peptide at a given charge state (m/z value), resulting from the identification of MS2 spectra combined with the quantitative information from the MS1 scans. Feature selection allows the use of either all, only the most abundant features or only high quality peptides for protein summarization.

Missing values and noisy features with outliers are typical in label-free DDA datasets but influence protein summarization. Therefore, it is recommended to perform missing value imputation. Missing values are reported differently in different Softwares. MaxQuant reports them as NA and MSstats assumes that missing intensity values from MaxQuant mean that the intensity was below the limit of quantification. This means the values are not missing for random but for the reason of low abundance. Therefore, the values are only partially known and  called “censored”. This may also be the case for very low intensity values, which might not be reliable. The percentile that is not trusted and should be considered a censored value is defined via the “Maximum quantile for deciding censored missing values” parameter. Censored values are replaced by an intensity that is generated via an accelerated failure time model (AFT). Alternatively censored values may be replaced by the minimum value of the features, runs or both as defined in the “Cutoff value for censoring”. Runs with no intensity measurement for a protein will be removed for any further calculation on this protein.

Protein summarization is by default performed via Tukey’s median polish for robust parameter estimation with median across rows and columns. Run-level summaries are later used for statistical group comparison.

Any two groups can be compared to find differentially abundant proteins between them. MSstats uses a family of linear mixed models that are automatically adjusted for the comparison type according to the information in the annotation file, such as conditions, biological and technical replicates and runs. This allows comparison of groups with different sizes; comparison of the mean of some groups, paired designs and time course experiments.


# Follow up on MSstats results

We obtain several output files from MSstats. MSstats log file contains the MSstats report with warnings and information about the analysis steps.
The MSstats QCPlot visualizes the log transformed intensities for all proteins and runs.
The volcano plot plots the statistical result as transformed p-values vs. the log2 fold change. A fold change of 1.5 means that a protein is 50% more abundand in one condition than the other. The log2 fold change is 0.58.

![qc plot](../../images/maxquant-msstats-lfq/qc_plot.png "QC plot of all proteins")

![volcano plot](../../images/maxquant-msstats-lfq/volcano_plot.png "Volcano plot showing p-values and log10 fold changes for all proteins. Dashed line indicates p-value of 0.05 and log2 fold change of ± 0.58")

The FeatureLevelData file contains the transformed, normalized and imputed intensities for each peptide in each run. ProteinLevelData data summarizes intensities per run and protein.
We’ll count and visualize the number of features per run and calculate the distribution of proteins per sample.

> <hands-on-title>Follow up on MSstats results</hands-on-title>
>
> 1. {% tool [Summary Statistics](Summary_Statistics1) %} with the following parameters:
>    - {% icon param-file %} *"Summary statistics on"*: `ProteinLevelData` (output of **MSstats** {% icon tool %})
>    - *"Column or expression"*: `c8`
> 2. {% tool [Datamash](toolshed.g2.bx.psu.edu/repos/iuc/datamash_ops/datamash_ops/datamash_ops/1.9+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Input tabular dataset"*: `ProteinLevelData` (output of **MSstats** {% icon tool %})
>    - *"Group by fields"*: `4`
>    - *"Sort input"*: `Yes`
>    - *"Input file has a header line"*: `Yes`
>    - *"Print header line"*: `Yes`
>    - In *"Operation to perform on each group"*:
>        - *"Type"* : `count`
>        - *"On column"*: `Column: 1`
> 3. Click on {% icon galaxy-barchart %} “Visualize this data” on the **Datamash** {% icon tool %} result.
>   - Select `Bar, Line and Scatter`
>   - Click  `Expand` {% icon galaxy-vis-config %} at the top right of the chart.
>   - *"Provide a title"*: `Number of features per sample`
>   - *"Column of x-axis values"*: `Column: 1`
>   - Save {% icon galaxy-save %} (file is saved under "Visualization" --> "Saved Visualizations")
>
{: .hands_on}


> <question-title></question-title>
>
> 1. Which sample has the lowest amount of proteins after protein summarization?
> 2. In the complete experiment, how many features has a protein on average?
>
> > <solution-title></solution-title>
> >
> > 1. RDEB cSCC4
> > ![Number of proteins per sample](../../images/maxquant-msstats-lfq/features_sample.png "Number of proteins per sample (run)")
> > 2. Around 6 features per protein (mean in summary statistics).
> >
> {: .solution}
>
{: .question}


# Filtering MSstats results

The comparison result table summarizes the statistical results per protein and comparison. First, we keep only the Uniprot ID in column 1 to make the ID less cluttered. This is done by deleting everything before the first pipe '|' and everything after the second pipe '|'.
Then we keep only statistically significant proteins that means they have an adjusted p-value below 0.05.
Next, we separate up- and down-regulated proteins by filtering for a positive and negative log2FC.
The Sample Quantification Matrix Table contains the summarized intensities per protein and sample.
In order to make its IDs compatible with the ones from the comparison result at a later step, we keep only the Uniprot ID as well.


> <hands-on-title>Filtering MSstats results</hands-on-title>
>
> 1. {% tool [Replace Text in a specific column](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_column/9.5+galaxy2) %} with the following parameters:
>    - {% icon param-file %} *"File to process"*: `Comparison Result` (output of **MSstats** {% icon tool %})
>    - In *"Replacement"*:
>        - *"in column"*: `Column: 1`
>        - *"Find pattern"*: `sp\|`
>    - {% icon param-repeat %} *"Insert Replacement"*
>        - *"in column"*: `Column: 1`
>        - *"Find pattern"*: `\|.*`
>    - Once finished, rename the output `replaced comparison result`
> 2. {% tool [Filter data on any column using simple expressions](Filter1) %} with the following parameters:
>    - {% icon param-file %} *"Filter"*: `replaced comparison result` (output of **Replace Text** {% icon tool %})
>    - *"With following condition"*: `c8<0.05`
>    - *"Number of header lines to skip"*: `1`
>    - Once finished, rename the file into `significant proteins`
> 3. {% tool [Filter on any column using simple expressions](Filter1) %} with the following parameters:
>    - {% icon param-file %} *"Filter"*: `significant proteins` (output of **Filter** {% icon tool %})
>    - *"With following condition"*: `c3>0.58`
>    - *"Number of header lines to skip"*: `1`
> 4. Add a tag `#metastasized` to the filtered file and rename it into `metastasized filtered`
>
>    {% snippet faqs/galaxy/datasets_add_tag.md %}
>
> 5. {% tool [Filter on any column using simple expressions](Filter1) %} with the following parameters:
>    - {% icon param-file %} *"Filter"*: `significant proteins` (output of the first **Filter** {% icon tool %})
>    - *"With following condition"*: `c3<-0.58`
>    - *"Number of header lines to skip"*: `1`
> 6. Add a tag `#rdeb` to the filtered file and rename it into `rdeb filtered`
>
{: .hands_on}


> <question-title></question-title>
>
> 1. Why do we filter for the adjusted p-value?
> 2. How many proteins have a p-value below 0.05?
>
> > <solution-title></solution-title>
> >
> > 1. Adjusted p-values control for the multiplicity of testing. Since we fit a separate model, and conduct a separate comparison for each protein, the number of tests equals the number of comparisons. A 0.05 cutoff of an adjusted p-value controls the False Discovery Rate in the collection of tests over all the proteins at 5%. Since they account for the multiplicity, adjusted p-values are more conservative (i.e. it is more difficult to detect a change).
> > 2. 198 in total (first filtering step); 178 are upregulated in metastasized cSCC (metastasized filtered) and 15 are upregulated in RDEB cSCC (rdeb filtered).
> >
> {: .solution}
>
{: .question}

# Finding differentially abundant proteins

For each condition we select only the significant proteins, which are proteins with a p-value above 0 and below 0.05. Proteins with a p-value of 0 are missing in one condition and are therfore discarded in the next steps. We'll keep only the column with the Uniprot ID and extract the average protein intensities per sample from the sample quantification matrix file and vizualize them at heatmap. We do the exact same steps for both conditions, therefore, each time you start a tool (except for the replace step) you can use the multiple input file to start the step for the metastasized and rdeb files at the same time.

> <hands-on-title>filter differentially abundant proteins</hands-on-title>
>
> 1. {% tool [Filter on any column using simple expressions](Filter1) %} with the following parameters:
>    - {% icon param-file %} *"Filter"*: `metastasized filtered` (output of **Filter** {% icon tool %})
>    - *"With following condition"*: `c8>0`
>    - *"Number of header lines to skip"*: `1`
>    - Rename the file into `significant metastasized`
> 2. {% tool [Cut](Cut1) %} with the following parameters:
>    - *"Cut columns"*: `c1`
>    - {% icon param-file %} *"From"*: `significant metastasized ` (output of last **Filter** {% icon tool %})
>    - Rename the file into `metastasized cut`
> 3. {% tool [Filter on any column using simple expressions](Filter1) %} with the following parameters:
>    - {% icon param-file %} *"Filter"*: `rdeb filtered` (output of **Filter** {% icon tool %})
>    - *"With following condition"*: `c8>0`
>    - *"Number of header lines to skip"*: `1`
>    - Rename the file into `significant rdeb`
> 4. {% tool [Cut](Cut1) %} with the following parameters:
>    - *"Cut columns"*: `c1`
>    - {% icon param-file %} *"From"*: `significant rdeb` (output of last **Filter** {% icon tool %})
>    - Rename the file into `rdeb cut`
> 5. {% tool [Replace Text in a specific column](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_column/9.5+galaxy2) %} with the following parameters:
>    - {% icon param-file %} *"File to process"*: `Sample Quantification Matrix` (output of **MSstats** {% icon tool %})
>    - In *"Replacement"*:
>        - *"in column"*: `Column: 1`
>        - *"Find pattern"*: `sp\|`
>    - {% icon param-repeat %} *"Insert Replacement"*
>        - *"in column"*: `Column: 1`
>        - *"Find pattern"*: `\|.*`
>    - Rename the file into `replaced sample quantification matrix`
> 6. {% tool [Join two files](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_easyjoin_tool/9.5+galaxy2) %} with the following parameters:
>    - {% icon param-file %} *"1st file"*: `replaced sample quantification matrix` (output of **Replace text** {% icon tool %})
>    - *"Column to use from 1st file"*: `Column: 1`
>    - {% icon param-file %} *"2nd File"*: `metastasized cut` (output of **Cut** {% icon tool %})
>    - *"Column to use from 2nd file"*: `Column: 1`
>    - *"First line is a header line"*: `Yes`
>    - Rename the file into `metastasized join`
> 7. {% tool [Join two files](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_easyjoin_tool/9.5+galaxy2) %} with the following parameters:
>    - {% icon param-file %} *"1st file"*: `replaced sample quantification matrix` (output of **Replace Text** {% icon tool %})
>    - *"Column to use from 1st file"*: `Column: 1`
>    - {% icon param-file %} *"2nd File"*: `rdeb cut` (output of **Cut** {% icon tool %})
>    - *"Column to use from 2nd file"*: `Column: 1`
>    - *"First line is a header line"*: `Yes`
>    - Rename the file into `rdeb join`
> 8. {% tool [heatmap2](toolshed.g2.bx.psu.edu/repos/iuc/ggplot2_heatmap2/ggplot2_heatmap2/3.2.0+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"Input should have column headers - these will be the columns that are plotted"*: `metastasized join` (output of **Join** {% icon tool %})
>    - *"Plot title"*: `Upregulated proteins in metastasized cSCC`
>    - *"Scale data on the plot (after clustering)"*: `Scale my data by row`
>    - *"Enable data clustering"*: `No`
>    - *"Output format "*: `PNG`
> 9. {% tool [heatmap2](toolshed.g2.bx.psu.edu/repos/iuc/ggplot2_heatmap2/ggplot2_heatmap2/3.2.0+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"Input should have column headers - these will be the columns that are plotted"*: `rdeb join` (output of **Join** {% icon tool %})
>    - *"Plot title"*: `Upregulated proteins in RDEB cSCC`
>    - *"Scale data on the plot (after clustering)"*: `Scale my data by row`
>    - *"Enable data clustering"*: `No`
>    - *"Output format "*: `PNG`
>
{: .hands_on}

> <question-title></question-title>
>
> 1. How many proteins are differentially abundant?
>
> > <solution-title></solution-title>
> >
> > 1. 125 are upregulated in metastasized cSCC and 14 are upregulated in RDEB cSCC (number of lines minus 1 in the first filtering step per condition).
> >
> {: .solution}
>
{: .question}

# Follow up on differentially abundant proteins

In addition we retrieve for each Uniprot ID the corresponding protein names from uniprot to allow an easier interpretation.

> <hands-on-title>MSstats visualizations</hands-on-title>
>
> 1. {% tool [UniProt ID mapping and retrieval](toolshed.g2.bx.psu.edu/repos/bgruening/uniprot_rest_interface/uniprot/0.7) %} (Do not use 0.6, you may need to change the version on some instances) with the following parameters:
>    - {% icon param-file %} *"Input file with IDs"*: `metastasized join` (output of **Join** {% icon tool %})
>    - *"ID column"*: `Column: 1`
>    - *"Do you want to map IDs or retrieve data from UniProt"*: `Retrieve: request entries by uniprot accession using batch retrieval`
>    - Rename the file into `metastasized uniprot`
> 2. {% tool [UniProt ID mapping and retrieval](toolshed.g2.bx.psu.edu/repos/bgruening/uniprot_rest_interface/uniprot/0.7) %} (Do not use 0.6, you may need to change the version on some instances) with the following parameters:
>    - {% icon param-file %} *"Input file with IDs"*: `rdeb join` (output of **Join** {% icon tool %})
>    - *"ID column"*: `Column: 1`
>    - *"Do you want to map IDs or retrieve data from UniProt"*: `Retrieve: request entries by uniprot accession using batch retrieval`
>    - Rename the file into `rdeb uniprot`
> 3. {% tool [FASTA-to-Tabular](toolshed.g2.bx.psu.edu/repos/devteam/fasta_to_tabular/fasta2tab/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Convert these sequences"*: `metastasized uniprot` (output of **UniProt** {% icon tool %})
>    - *"How many columns to divide title string into?"*: `2`
> 4. {% tool [FASTA-to-Tabular](toolshed.g2.bx.psu.edu/repos/devteam/fasta_to_tabular/fasta2tab/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Convert these sequences"*: `rdeb uniprot` (output of **UniProt** {% icon tool %})
>    - *"How many columns to divide title string into?"*: `2`
>
{: .hands_on}

Three of the differentially abundant proteins found here were also found and stained with antibodies in the original publication: Collagen XIV which is higher in RDEB cSCC than in metastasizing cSCC and Serum amyloid P-component as well as X-ray repair cross-complementing protein 6, which are both higher in metastasizing cSCC than in RDEB cSCC. Collagen XIV is a fibril associated collagen which may have tissue stabilizing function in the dermis. The upregulation of collagen XIV as well as other collagens in RDEB cSCC could be a compensation effort for the impaired collagen VII in RDEB tissues. Collagen VII is indeed only found in some RDEB samples and with such low intensities that it appears as upregulated in metastasized cSCC.

![col14 staining](../../images/maxquant-msstats-lfq/col14_ihc.png "Immunoflourescence staining of collagen XIV in RDEB and metastasizing cSCC skin tissues")


topic: proteomics
tutorial: neoantigen-fragpipe-discovery
title: Neoantigen 2: Database merge and FragPipe discovery
difficulty: intermediate
time_estimation: 3H
tools_json: []
requirements_json: []
tags_json: ["[label-free]"]
content_hash: 73d9829cd9631019c2c1e71df894ab5e
last_modified: 2026-02-26T21:12:17.500486
zenodo_link: https://zenodo.org/records/14374118
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/neoantigen-fragpipe-discovery/tutorial.html

content:
In this tutorial, we will guide you through a bioinformatics workflow aimed at merging neoantigen databases with known human protein sequences, preparing the data for proteomics analysis using FragPipe. This workflow involves processing FASTA files, filtering for unique sequences, and validating the FASTA databases before using FragPipe to perform peptide identification and validation of neoantigens.

Throughout the tutorial, you will learn how to integrate multiple datasets, ensuring that you can perform analyses such as the identification of potential neoantigens, which are critical for cancer immunotherapy and vaccine development. The tools and steps covered here are important for any bioinformatics pipeline dealing with proteomics and neoantigen discovery.

![Discovery Overview workflow]({% link topics/proteomics/images/neoantigen/FragPipe_Discovery_1.PNG %})
> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Neoantigen Fragpipe Discovery

This tutorial guides users through the process of performing database searching or neoantigen protein/peptide discovery. It encompasses essential bioinformatics steps to identify and variant-specific peptides for immunological studies.The overview is divided into 2 major stages: (A) Merging all the variant databases and Validating the sequences, (B) Performing database searching using Fragpipe to discover neoantigen peptides. Below is an overview of each major stages


- **A: Merging Databases**
   1. **Get Data** The first step involves gathering and uploading the necessary proteomics data files into the analysis environment. These files typically contain protein sequences or raw spectrum data that will be processed throughout the tutorial. Proper data organization and tagging are essential to ensure smooth workflow execution.
   2. **Merging FASTA Files and Filtering for Unique Sequences** In this step, multiple FASTA files containing protein sequences are merged into a single file. After merging, sequences are filtered to retain only the unique ones, ensuring that redundancy is removed and only relevant protein data is used for downstream analysis.

- **B: Discovery search**
   1. **Validating FASTA Databases** Once the FASTA files are merged and filtered, it's important to validate the database to ensure that the protein sequences are correctly formatted and usable for analysis. This step helps identify and correct any issues in the dataset before performing more complex analysis tasks.
   2. **Running FragPipe for Neoantigen Discovery** FragPipe, a proteomics analysis tool, is then employed to process the data further. This involves peptide identification, protein quantification, and running specialized workflows such as nonspecific-HLA searches to identify potential neoantigens that may be targeted for immunotherapy.
   3. **Collapsing Datasets and Selecting Relevant Data** After the analysis, the resulting datasets are collapsed into a single dataset to simplify further processing. This step helps streamline the data, making it easier to select and focus on the relevant sequences that match the biological question being addressed.
   4. **Querying Tabular Results for Further Analysis** In the final step, tabular results from the analysis are queried using SQL-like commands to filter and extract the most relevant data. This allows for focused analysis on specific protein sequences or neoantigens that were identified, enabling further downstream analysis and interpretation.


# A: Merging Databases

![Database Merging]({% link topics/proteomics/images/neoantigen/FragPipe_Discovery_2.PNG %})

## Get data

> <hands-on-title> Data Upload </hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo]({{ page.zenodo_link }}) or from
>    the shared data library (`GTN - Material` -> `{{ page.topic_name }}`
>     -> `{{ page.title }}`):
>
>    ```
>    https://zenodo.org/records/14374118/files/Experimental-Design-Fragpipe.tabular
>    https://zenodo.org/records/14374118/files/Arriba-Fusion-Database.fasta
>    https://zenodo.org/records/14374118/files/Human_cRAP_Non_normal_transcripts_dB.fasta
>    https://zenodo.org/records/14374118/files/STS_26T_2_Eclipse_02102024.raw
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
> 3. Rename the datasets
> 4. Check that the datatype
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
>
> 5. Add to each database a tag corresponding to ...
>
>    {% snippet faqs/galaxy/datasets_add_tag.md %}
>
{: .hands_on}



## Merging FASTA Files and Filtering for Unique Sequences

Next, we will merge the FASTA files, ensuring that any redundant sequences are removed. This step ensures that we only work with unique sequences, improving the quality and accuracy of the subsequent analysis. In this step, we combine the fusion database generated from the Arriba Pipeline (first neoantigen workflow) with the non-reference database created from HISAT, Freebayes, CustomPRODB, and the Stringtie Pipeline (second neoantigen workflow). Once merging is done, we validate the database to ensure that the sequences are in the right format.

> <hands-on-title> FASTA Merge Files and Filter Unique Sequences</hands-on-title>
>
> 1. {% tool [FASTA Merge Files and Filter Unique Sequences](toolshed.g2.bx.psu.edu/repos/galaxyp/fasta_merge_files_and_filter_unique_sequences/fasta_merge_files_and_filter_unique_sequences/1.2.0) %} with the following parameters:
>    - *"Run in batch mode?"*: `Merge individual FASTAs (output collection if input is collection)`
>        - In *"Input FASTA File(s)"*:
>            - {% icon param-repeat %} *"Insert Input FASTA File(s)"*
>                - {% icon param-file %} *"FASTA File"*: `Human_cRAP_Non_normal_transcripts_dB.fasta` (Input dataset)
>        - In *"Input FASTA File(s)"*:
>            - {% icon param-repeat %} *"Insert Input FASTA File(s)"*
>                - {% icon param-file %} *"FASTA File"*: `Arriba-Fusion-Database.fasta` (Input dataset)
>
>
{: .hands_on}


## Sequence database parsing with Validate FASTA Database

> <hands-on-title> Validate FASTA Database </hands-on-title>
>
> 1. {% tool [Validate FASTA Database](toolshed.g2.bx.psu.edu/repos/galaxyp/validate_fasta_database/validate_fasta_database/0.1.5) %} with the following parameters:
>    - {% icon param-file %} *"Select input FASTA dataset"*: `output` (output of **FASTA Merge Files and Filter Unique Sequences** {% icon tool %})
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why is it important to validate a FASTA database before using it for further analysis in a proteomics pipeline?
> 2. What potential issues might be identified by the Validate FASTA Database tool?
>
> > <solution-title></solution-title>
> >
> > 1. Validating a FASTA database ensures that the sequences are correctly formatted and that no errors are present. It helps ensure the accuracy of the data used in downstream analysis, which is critical for correct protein identification and neoantigen discovery.
> > 2. The Validate FASTA Database tool might identify issues such as incorrect sequence formatting, missing information (e.g., headers or sequence data), or invalid characters in the sequences, which could cause errors during further analysis.
> >
> {: .solution}
>
{: .question}

# B: Discovery search
##  Running FragPipe for Neoantigen Discovery
The FragPipe tool is used for processing and analyzing proteomics data in mass spectrometry experiments. This tool integrates multiple components such as MSFragger, IonQuant, and Percolator, allowing users to perform tasks such as peptide-spectrum match (PSM) identification, protein quantification, and post-translational modification (PTM) analysis. In this task, FragPipe is being used to identify potential neoantigens by analyzing mass spectrometry (MS) data and correlating it with a validated FASTA sequence database.
The FragPipe tool is executed with the MS data and a validated FASTA sequence database to identify peptides, proteins, and potential neoantigens from the raw mass spectrometry data. Users must provide MS spectrum files, a manifest file, and the validated FASTA database (created in the previous step using the Validate FASTA Database tool). The tool runs a series of analyses, including non-specific enzyme digestion, quantification of protein abundance, and validation of peptide-spectrum matches (PSMs) with Percolator and Protein Prophet. Output includes identified peptides, proteins, and quantification results, which are essential for downstream neoantigen discovery.

In this workflow, FragPipe is used after FASTA database validation to ensure that the sequence database is correctly formatted and ready for mass spectrometry analysis. It integrates several tools in a single workflow to process the raw MS data, identify peptides and proteins, and provide validation for the results. It also supports isobaric and label-free quantification methods for protein and peptide quantification, important for understanding relative protein abundance and identifying potential neoantigens.

![Fragpipe-discovery]({% link topics/proteomics/images/neoantigen/FragPipe_Discovery_3.PNG %})


## Creating a FragPipe Manifest (Experimental Design) File

The **FragPipe Manifest Generator** creates the experimental design file required by FragPipe, mapping each raw LC–MS/MS file to its experimental factors (e.g., experiment and bioreplicate). This manifest ensures consistent labeling of runs for downstream identification and quantification. In this workflow, we generate a simple design with a single experiment and a single bioreplicate, and we explicitly set the scan data type to **DDA** to match acquisition.

> <hands-on-title> FragPipe Manifest Generator </hands-on-title>
> 
> 1. {% tool [FragPipe Manifest Generator](toolshed.g2.bx.psu.edu/repos/galaxyp/fragpipe/fragpipe_manifest_generator/23.0+galaxy0) %} with the following parameters:
>    - {% icon param-file %} “Proteomics spectrum files” (mzML, mzXML, or Thermo .raw): select the MS file for this run.
>    - In *"Assign experiments"*: `Enter column values`
>        - In *"Comma-separated experiment numbers"*: `1`
>    - In *"Assign bioreplicates"*: `Enter column values`
>        - In *"Comma-separated bioreplicate numbers"*: `1`
>    - In *“Assign scan_data_types"*: `Assign to all scan files`
>        - In *"Data Type”*: `DDA`
>
{: .hands_on}

> <question-title></question-title>
>
> 1. What does the FragPipe manifest control in downstream analysis?  
> 2. How should experiments vs. bioreplicates be assigned for multi-condition studies?
>
> > <solution-title></solution-title>
> >
> > 1. The manifest defines how runs are grouped and compared (e.g., by experiment/condition and bioreplicate), ensuring FragPipe/Philosopher interpret files consistently for identification and quantification.  
> > 2. Use distinct **experiment numbers** for biological conditions (e.g., 1,1,2,2 for Control vs. Treatment) and distinct **bioreplicate numbers** to label independent replicates within each condition (e.g., 1,2 within each experiment). This preserves replicate structure for proper statistics and reporting.  
> >
> {: .solution}
>
{: .question}


> <tip-title>Mixed Data Types</tip-title>
>
> If your acquisition includes mixed data types (e.g., DIA or IMS-DDA), set the Data Type per file accordingly. Ensure raw file names are stable and meaningful—renaming files after manifest creation can break downstream steps.
>
> 
{: .tip}


> <hands-on-title> Fragpipe </hands-on-title>
>
> 1. {% tool [FragPipe -  Academic Research and Education User License (Non-Commercial)](toolshed.g2.bx.psu.edu/repos/galaxyp/fragpipe/fragpipe/23.0+galaxy0) %} with the following parameters:
>    - *"I understand that these tools, including MSFragger, IonQuant, Bruker, and Thermo Raw File Reader, are available freely for academic research and educational purposes only, and agree to the following terms."*: `Yes`
>    - {% icon param-file %} *"Proteomics Spectrum files"*: `STS_26T_2_Eclipse_02102024.raw` (Input dataset)
>    - {% icon param-file %} *"Manifest file"*: `Experimental-Design-Fragpipe.tabular` (Input dataset)
>    - {% icon param-file %} *"Proteomics Search Database in FASTA format"*: `goodFastaOut` (output of **Validate FASTA Database** {% icon tool %})
>    - *"Split database"*: `200`
>    - *"Workflow"*: `Nonspecific-HLA`
>        - In *"MSFragger Options"*:
>            - In *"Search Tolerances"*:
>                - *"Set Precursor Mass tolerances"*: `Use default`
>                - *"Set Precursor True tolerance"*: `Use default`
>                - *"Set Fragment Mass tolerances"*: `ppm`
>            - In *"In-silico Digestion Parameters"*:
>                - *"Protein Digestion Enzyme"*: `nonspecific`
>                - *"Second Enzyme Digest"*: `No`
>                - *"Maximum length of peptides to be generated during in-silico digestion"*: `20`
>            - In *"Variable Modifications"*:
>                - *"Select Variable Modifications"*: `Oxidation of M (15.99491461956) modaa`
>            - In *"Spectrum Processing"*:
>                - *"Precursor mass mode"*: `corrected`
>                - *"Precursor Charge Override"*: `Use default`
>        - In *"Validation"*:
>            - *"Run Validation"*: `Yes`
>                - *"PSM Validation"*: `Run MSBooster and Percolator`
>                - *"Run Protein Prophet"*: `Yes`
>                - *"Generate Philosopher Reports"*: `Yes`
>        - In *"Quant (MS1)"*:
>            - *"Perform Label-Free Quantification"*: `Use workflow default`
>        - In *"PTMs"*:
>            - *"Run PTM Shepherd"*: `no`
>        - In *"Quant (Isobaric)"*:
>            - *"Perform Isobaric Quantification"*: `no`
>
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. Why do we need to use a validated FASTA database with FragPipe?
> 2. What is the significance of running Percolator and Protein Prophet in the FragPipe workflow?
>
> > <solution-title></solution-title>
> >
> > 1. A validated FASTA database ensures that the protein sequences used in the analysis are correctly formatted and free from errors. This improves the accuracy of peptide identification and ensures reliable downstream analysis for neoantigen discovery.
> > 2. Percolator is used to validate peptide-spectrum matches (PSMs) by improving the accuracy of identification through machine learning-based scoring. Protein Prophet is used for protein identification validation, providing confidence levels for protein inference. Both tools help ensure high-quality, reliable results in proteomics analysis.
> >
> {: .solution}
>
{: .question}

## Collapsing Datasets and Selecting Relevant Data

The Collapse Collection tool is used to combine multiple input files into a single dataset, which is especially useful in workflows where various outputs, such as peptide or protein data, need to be merged for further analysis. In this specific workflow, the tool consolidates peptide data produced by FragPipe into one unified collection. This step is important because it simplifies data handling and organization, making downstream processes like querying or visualization much easier and more efficient. Collapsing the collection reduces the complexity of dealing with multiple files and minimizes the risk of errors. It also ensures that all related data is available in a single dataset, speeding up subsequent analysis and improving overall workflow efficiency. By merging the results into one file, researchers can focus on analysis rather than managing fragmented data.

In this workflow, we aim to consolidate the peptide output from FragPipe. In the current version of FragPipe, the peptide output is generated in a collection format. We need to collapse this collection to create a single tabular output. The next step is to select only relevant variant sequences, hence we remove all the contaminants and known HUMAN sequences from the output.

> <hands-on-title> Collapse Collection </hands-on-title>
>
> 1. {% tool [Collapse Collection](toolshed.g2.bx.psu.edu/repos/nml/collapse_collections/collapse_dataset/5.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Collection of files to collapse into single dataset"*: `output_peptide` (output of **FragPipe -  Academic Research and Education User License (Non-Commercial)** {% icon tool %})
>
>
{: .hands_on}

> <hands-on-title> Removing contaminants and known proteins with Select </hands-on-title>
>
> 1. {% tool [Select](Grep1) %} with the following parameters:
>    - {% icon param-file %} *"Select lines from"*: `output` (output of **Collapse Collection** {% icon tool %})
>    - *"that"*: `NOT Matching`
>    - *"the pattern"*: `(HUMAN)|(contam_)|(con_)`
>    - *"Keep header line"*: `Yes`
>
>
{: .hands_on}

> <hands-on-title> Extracting Peptide Candidates using Query Tabular </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `output` (output of **Select** {% icon tool %})
>    - *"SQL Query to generate tabular output"*:
> ``` sql
> SELECT DISTINCT c1
> FROM t1
> ```
>    - *"include query result column headers"*: `No`
>
>
{: .hands_on}


> <hands-on-title> Remove beginning of file </hands-on-title>
>
> 1. {% tool [Remove beginning](Remove beginning1) %} with the following parameters:
>    - {% icon param-file %} *"from"*: `output` (output of **Select tool** {% icon tool %})
>
{: .hands_on}



# Conclusion

In this workflow, tools like FragPipe, Collapse Collection, and Query Tabular are used to process, merge, and filter proteomics data for efficient analysis. FragPipe allows for in-depth proteomic analysis, producing results that are consolidated using Collapse Collection into a unified dataset, simplifying further analysis. The Query Tabular tool then enables targeted querying of this dataset, filtering it based on specific conditions to refine the data for downstream analysis. Together, these tools streamline the workflow, enhancing the ability to manage, manipulate, and extract meaningful insights from complex biological data. This step-by-step process exemplifies how bioinformatics tools can be integrated to handle large-scale data, making the analysis more efficient and focused on specific research questions.


# Rerunning on your own data

To rerun this entire analysis at once, you can use our workflow. Below we show how to do this:

> <hands-on-title>Running the Workflow</hands-on-title>
>
>  1. **Import the workflow** into Galaxy:
>     - [Neoantigen Database Search](https://tinyurl.com/ipepgen-dbsearch-wf)
>
> 3. Run **Workflow** {% icon workflow %} using the following parameters:
>     - *"Send results to a new history"*: `No`
>     - {% icon param-file %} *"Non-Reference protein database"*: `Human_cRAP_Non_reference_transcripts_dB.fasta`
>     - {% icon param-file %} *"Fusion protein database"*: `Arriba-Fusion-Database.fasta`
>     - {% icon param-file %} *"Input raw file"*: `STS_26T_2_Eclipse_02102024.raw`
>     - {% icon param-file %} *"Experimental design file for Fragpipe"*: `Experimental-Design-Fragpipe.tabular`
>
>    {% snippet faqs/galaxy/workflows_run.md %}
>
> 
> 
> 
> 
> <comment-title>DISCLAIMER</comment-title>
>
> - If any step in this workflow fails, please ensure that the input files have been correctly generated and formatted by the preceding tools. Workflow failures often result from improperly called or incomplete input data rather than errors in the workflow itself. Users are responsible for verifying their input before troubleshooting workflow issues.
>
> {: .comment}
>
>
{: .hands_on}

# Are you feeling adventurous? ✨

## One-Click Neoantigen Workflow

This new [One-Click Neoantigen Workflow](https://usegalaxy.eu/u/galaxyp/w/ipepgen-one-click-workflow) brings together all key modules of the neoantigen discovery process into a single, streamlined analysis within Galaxy. Instead of launching each tutorial separately, users can now execute the entire end-to-end pipeline—from database creation to HLA binding prediction—with just one click —without ever leaving Galaxy.

# Disclaimer

Please note that all the software tools used in this workflow are subject to version updates and changes. As a result, the parameters, functionalities, and outcomes may differ with each new version. Additionally, if the protein sequences are downloaded at different times, the number of sequences may also vary due to updates in the reference databases or tool modifications. We recommend the users to verify the specific versions of software tools used to ensure the reproducibility and accuracy of results.


topic: proteomics
tutorial: proteogenomics-dbcreation
title: Proteogenomics 1: Database Creation
difficulty: intermediate
time_estimation: 30m
tools_json: []
requirements_json: []
tags_json: ["[proteogenomics]"]
content_hash: 4d056c2c1def56d424df1c490d27991b
last_modified: 2026-02-26T21:12:17.510341
zenodo_link: https://doi.org/10.5281/zenodo.1489208
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/proteogenomics-dbcreation/tutorial.html

content:
**Proteogenomics** involves the use of mass spectrometry (MS) based proteomics data against genomics and transcriptomics data to identify peptides and to understand protein-level evidence of gene expression. In the first section of the tutorial, we will create a protein database (FASTA) using RNA-sequencing files (FASTQ) and then perform sequence database searching using the resulting FASTA file with the MS data to identify peptides corresponding to novel proteoforms. Then, we will assign the genomic coordinates and annotations for these identified peptides and visualize the data for its spectral quality and genomic localization

![Novel Peptides](../../images/potential_novel_publication.png)


Proteogenomics integrates **RNA-Seq** data for generating customized protein sequence databases with mass spectrometry-based proteomics data, which are matched to these databases to identify protein sequence variants. ({% cite Chambers_2017 %}).

![Proteogenomics](../../images/workflow_objective1.png)



> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
> 1. TOC
> {:toc}
>
{: .agenda}

# Overview

This tutorial focuses on creating a **FASTA** database generated from RNA-seq data. There are two outputs from this workflow: (1) a **sequence database** consisting of variants and known reference sequences and (2) mapping files containing **genomic** and **variant** mapping data.

![Database creation](../../images/database_creation.png)

The first part of the tutorial deals with creating the FASTA file containing sequences with single amino acid variants (SAVs), insertions and deletions (indels). The second part of the workflow helps in creating a FASTA file with transcript assemblies (splicing variants).


# Data upload

In this tutorial, protein and the total RNA sample was obtained from the early development of B-cells from mice. It was obtained at two developmental stages of B-cells: *Ebf1* -/- pre-pro-B and *Rag2* -/- pro-B. Please refer to the original study  [Heydarian, M. et al.](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4276347/)for details.


> <hands-on-title>data upload and organization</hands-on-title>
>
> 1. Create a new history and name it something meaningful (e.g. *Proteogenomics DB creation*)
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the Uniprot FASTA, FASTQ file and the GTF file from Zenodo [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.13270741.svg)](https://doi.org/10.5281/zenodo.13270741)
>    ```
>    https://zenodo.org/records/1489208/files/Trimmed_ref_5000_uniprot_cRAP.fasta
>    https://zenodo.org/record/1489208/files/FASTQ_ProB_22LIST.fastqsanger
>    https://zenodo.org/record/1489208/files/Mus_musculus.GRCm38.86.gtf
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Rename the datasets with more descriptive names (strip off the url prefixes)
>
>    {% snippet faqs/galaxy/datasets_rename.md %}
>
> 4. Make sure that the datatype of file `FASTQ_ProB_22List.fastqsanger` is set to `fastqsanger`
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="fastqsanger" %}
>
> 5. Make sure the Database/Build (dbkey) is set to `Mouse.Dec 2011 (GRCm38/mm10)(mm10)`
>
>    {% snippet faqs/galaxy/datasets_change_dbkey.md dbkey="Mouse.Dec 2011 (GRCm38/mm10)(mm10)"%}
>
> 6. **Note**: If you are running the workflow taken from the **GTN**, then make sure the tool "**Tabular-to-FASTA**" (tool number 26)
>    has Title Column labeled as "1" and Sequence Column as "2".
{: .hands_on}

# Change the chromosome names in the Ensembl GTF to match a UCSC genome reference

UCSC prefaces chromosome names with chr while Ensembl does not. To perform this action, we use the Replace Text in a specific column - a [regular expression](https://docs.python.org/3/library/re.html) tool.

> <hands-on-title>Replace Text in a specific column</hands-on-title>
>  1. {% tool [Replace Text in a specific column](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_column/9.5+galaxy0) %} with the following parameters:
>      - {% icon param-file %} *"File to process"*: `Mus_musculus.GRCm38.86.gtf`
>      - Change numbered chromosome names
>          - *"in column"*: `Column: 1`
>          - *"Find pattern"*: `^([1-9][0-9]*)$`
>          - *"Replace with*: `chr\\1`
>      - {% icon param-repeat %} *Insert Replacement* to add another replacement option to change XY chromosomes
>          - *"in column"*: `Column: 1`
>          - *"Find pattern"*: `^([XY])$`
>          - *"Replace with*: `chr\\1`
>      - {% icon param-repeat %} *Insert Replacement* to add another replacement option to change mitochondrial chromosome
>          - *"in column"*: `Column: 1`
>          - *"Find pattern"*: `^MT$`
>          - *"Replace with*: `chrM`
>
>  2. Rename the output to `Mus_musculus.GRCm38.86.fixed.gtf`
>
> > <comment-title>Note on chromosome name changes</comment-title>
> >  This step is necessary to help identify and correct errors due to inputs having non-identical chromosome identifiers and/or different chromosome sequence content. [Galaxy Support](https://galaxyproject.org/support/chrom-identifiers/)
> {: .comment}
{: .hands_on}


# Aligning FASTQ files to the mouse genome

The first tool in the workflow is the [**HISAT2**](http://ccb.jhu.edu/software/hisat) alignment tool. It maps next-generation sequence (NGS) reads to the reference genome. This tool requires an RNA-seq file (.FASTQ) and a reference genome file in Gene transfer format (GTF). This .gtf file is obtained from the Ensembl database. When successful, this tool outputs a .bam file (binary version of a SAM: **S**equence **A**lignment/**M**ap).

> <hands-on-title>Alignment with HISAT2</hands-on-title>
>
>  1. {% tool [HISAT2](toolshed.g2.bx.psu.edu/repos/iuc/hisat2/hisat2/2.2.1+galaxy1) %} with the following parameters:
>    - *"Source for the reference genome"*: `Use a built-in genome`
>    - *"Select a reference genome"*: `Mouse (Mus Musculus): mm10`
>    - *"Single-end or paired-end reads"*: `Single end`
>    - {% icon param-file %} *"Input FASTQ files"*: `FASTQ_ProB_22LIST.fastqsanger`
>    - *"Specify strand information"*: `Unstranded`
> 2. Inspect {% icon galaxy-eye %} the resulting files
> 3. Rename the output to `HISAT_Output.BAM`
>
> > <comment-title>Note on strandedness</comment-title>
> > Note that if your reads are from a stranded library, you need to choose the appropriate
> > setting for "Specify strand information" mentioned above. For single-end reads, use `F` or `R`.
> > `F` means that a read corresponds to a forward transcript. `R` means that a read corresponds to the reverse
> > complemented counterpart of a transcript.
> >
> > For paired-end reads, use either `FR` or `RF`. With this
> > option being used, every read alignment will have an XS attribute tag: `+` means a read
> > belongs to a transcript on the positive `+` strand of the genome. `-` means a read belongs to a
> > transcript on the negative `-` strand of the genome. (TopHat has a similar option, `--library--type` option,
> > where fr - first strand corresponds to R and RF; fr - second strand corresponds to F and FR.)
> {: .comment}
>
{: .hands_on}


# Variant Analysis

## Variant Calling

[FreeBayes]( https://github.com/ekg/freebayes) is a Bayesian genetic variant detector designed to find small polymorphisms, specifically SNPs (single-nucleotide polymorphisms), indels (insertions and deletions), MNPs (multi-nucleotide polymorphisms), and complex events (composite insertion and substitution events) smaller than the length of a short-read sequencing alignment.

![Variant calling](../../images/variant_calling.png)

Provided with some BAM dataset(s) and a reference sequence, FreeBayes will generate a VCF dataset describing SNPs, indels, and complex variants in samples in the input alignments. By default, FreeBayes will consider variants supported by at least two observations in a single sample (`-C`) and also by at least 20% of the reads from a single sample (`-F`). These settings are suitable for low to high depth sequencing in haploid and diploid samples, but users working with polyploid or pooled samples may adjust them depending on the characteristics of their sequencing data.

FreeBayes is capable of calling variant haplotypes shorter than a read length where multiple polymorphisms segregate on the same read. The maximum distance between polymorphisms phased in this way is determined by the `--max-complex-gap`, which defaults to 3bp. In practice, this can comfortably be set to half the read length. Ploidy may be set to any level (`-p`), but by default all samples are assumed to be diploid. FreeBayes can model per-sample and per-region variation in copy-number (`-A`) using a copy-number variation map.

FreeBayes can act as a frequency-based pooled caller and can describe variants and haplotypes in terms of observation frequency rather than called genotypes. To do so, use `--pooled-continuous` and set input filters to a suitable level. Allele observation counts will be described by AO and RO fields in the VCF output.


> <hands-on-title>Variant Calling with FreeBayes</hands-on-title>
>
> 1. {% tool [FreeBayes](toolshed.g2.bx.psu.edu/repos/devteam/freebayes/freebayes/1.3.9+galaxy0) %} with the following parameters:
>    - *"Choose the source for the reference genome"*: `Locally cached file`
>      - *"Run in batch mode?"*: `Run Individually`
>      - {% icon param-file %}  *"BAM dataset:*: `HISAT_Output.BAM`
>      - *"Using reference genome"*: `Mouse (Mus Musculus): mm10`
>    - *"Limit variant calling to a set of regions?"*: `Do not Limit`
>    - *"Choose parameter selection level"*: `Simple diploid calling`
>
> 2. Inspect {% icon galaxy-eye %} the resulting files.
>
> Note: FreeBayes may take some time...be patient!!!!
{: .hands_on}

> <comment-title>FreeBayes options</comment-title>
>
>    Galaxy allows five levels of control over FreeBayes options, provided by the Choose
>    parameter selection level menu option. These are:
>    1. Simple diploid calling: The simplest possible FreeBayes application. Equivalent to using
>    FreeBayes with only a BAM input and no other parameter options.
>
>    2. Simple diploid calling with filtering and coverage: Same as #1 plus two additional options:
>     (1) `-0` (standard filters: `--min-mapping-quality 30 --min-base-quality 20 --min-supporting-allele
>     -qsum 0 --genotype-variant-threshold 0`) and (2) `--min-coverage`.
>
>    3. Frequency-based pooled calling: This is equivalent to using FreeBayes with the following
>    options: `--haplotype-length 0 --min-alternate-count 1 --min-alternate-fraction 0 --pooled
>    -continuous  --report- monomorphic`. This is the best choice for calling variants in mixtures
>    such as viral, bacterial, or organellar genomes.
>
>    4. Frequency-based pooled calling with filtering and coverage: Same as #3 but adds `-0` and
>    `--min-coverage` like in #2.
>
>  Complete list of all options: Gives you full control by exposing all FreeBayes options as Galaxy parameters.
{: .comment}

## Variant annotation and Genome Mapping

CustomProDB ({% cite Wang_2013 %}) generates custom protein FASTA sequences files from exosome or transcriptome data. Once Freebayes creates the .vcf file, CustomProDB uses this file to generate a custom protein FASTA file from the transcriptome data. For this tool, we use Ensembl 89 mmusculus (GRm38.p5) (dbsnp142) as the genome annotation. We create three FASTA files from CustomProDB: (1) a variant FASTA file for short indels, (2) a Single Amino acid Variant (SAV) FASTA file, an SQLite database file (genome mapping and variant mapping) for mapping proteins to a genome and (3) an RData file for variant protein coding sequences.
The reference protein set can be filtered by transcript expression level (RPKM calculated from a BAM file), and variant protein forms can be predicted based on variant calls (SNPs and indels reported in a VCF file).

Annotations CustomProDB depends on a set of annotation files (in RData format) to
create reference and variant protein sequences. Galaxy administrators can use the CustomProDB
data manager to create these annotations to make them available for users.

![customproDB](../../images/CustomProDB.png)


> <hands-on-title>Generate protein FASTAs from exosome or transcriptome data</hands-on-title>
>
> 1. {% tool [CustomProDB](toolshed.g2.bx.psu.edu/repos/galaxyp/custom_pro_db/custom_pro_db/1.22.0) %} with the following parameters:
>    - *"Will you select a genome annotation from your history or use a built-in annotation?"*: `Use built in genome annotation`
>      - *"Using reference genome"*: `Ensemble 89 mmusculus (GRm38.p5) (dbsnp142)`
>      - {% icon param-file %} *"BAM file"*: `HISAT_Output.BAM`
>      - {% icon param-file %} *"VCF file"*: `Freebayes.vcf`
>    - *"Create a variant FASTA for short insertions and deletions"*: `Yes`
>    - *"Create SQLite files for mapping proteins to genome and summarizing variant proteins"*: `Yes`
>    - *"Create RData file of variant protein coding sequences"*: `Yes`
>
> 2. Inspect {% icon galaxy-eye %} the resulting files
{: .hands_on}

There are 6 files are generated through the CustomProDB tool:
- rpkm.fasta: reference proteins filtered by transcript expression level (RPKM calculated from a BAM file)
- snv.fasta: Single Nucleotide Variants detected by FreeBayes (.vcf file)
- indel.fasta: Insertions/Deletions detected by FreeBayes (.vcf file)
- Genomic_mapping.sqlite: mapping proteins to genome
- Variant_annotation.sqlite: used to visualize the variants in the Multi-omics Visualization Platform
- RData: set of annotation files (in RData format) to create reference and variant protein sequences


For mapping proteins to genome and an RData file for variant protein coding sequences. Similar to the genomic mapping, a variant mapping file
is also generated from CustomProDB. This SQLite file is also converted to tabular format and made SearchGUI-compatible. This
variant annotation file will be used to visualize the variants in the Multi-omics Visualization Platform (in-house visualization platform developed by Galaxy-P senior developers).

# Transcript Assembly

## RNA-seq to transcripts

[StringTie](http://ccb.jhu.edu/software/stringtie/) is a fast and highly efficient assembler of RNA-Seq alignments into potential transcripts. It uses a network flow algorithm as well as an optional *de novo* assembly step to assemble and quantitate full-length transcripts representing multiple splice variants for each gene locus.

Its input can include not only the alignments of raw reads used by other transcript assemblers, but also alignments of longer sequences that have been assembled from those reads. To identify differentially expressed genes between experiments, StringTie's output can be processed by specialized software like Ballgown, Cuffdiff or other programs (DESeq2, edgeR, etc.).


> <hands-on-title>Transcript assembly with StringTie</hands-on-title>
>
> 1. {% tool [StringTie](toolshed.g2.bx.psu.edu/repos/iuc/stringtie/stringtie/2.2.3+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Input mapped reads"*: `HISAT_Output.BAM`
>    - *"Specify strand information"*: `Unstranded`
>    - *"Use a reference file to guide assembly?"*: `Use Reference GTF/GFF3`
>      - *"Reference file"*: `Use file from History`
>      - {% icon param-file %} *"GTF/GFF3 dataset to guide assembly"*: `Mus_musculus.GRCm38.86.fixed.gtf`
> 2. Inspect {% icon galaxy-eye %} the resulting files.
> 3. Rename the output to `Stringtie_output.gtf`
>
>    {% snippet faqs/galaxy/datasets_rename.md %}
>
> 4. Make sure the datatype is `gtf`
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md %}
>
{: .hands_on}

StringTie accepts a BAM (or SAM) file of paired-end RNA-seq reads, which must be sorted by genomic location (coordinate position). This file contains spliced read alignments and can be produced directly by programs such as HISAT2. We recommend using HISAT2 as it is a fast and accurate alignment program. Every spliced read alignment (i.e. an alignment across at least one junction) in the input BAM file must contain the tag XS to indicate the genomic strand that produced the RNA from which the read was sequenced. Alignments produced by HISAT2 (when run with the `--dta` option) already include this tag, but if you use a different read mapper
you should check that this XS tag is included for spliced alignments.

> <details-title>Note about parameter settings</details-title>
> Be sure to run HISAT2 with the `--dta` option for alignment (under 'Spliced alignment options'),
> for the best results.
>
> Also note that if your reads are from a stranded library, you need to choose the appropriate
> setting for 'Specify strand information' above. As, if Forward (FR) is selected, StringTie will
> assume the reads are from a `--fr` library, while if Reverse (RF) is selected, StringTie will
> assume the reads are from a `--rf` library, otherwise it is assumed that the reads are from an
> unstranded library (the widely-used, although now deprecated, TopHat had a similar `--library-type`
> option, where fr-firststrand corresponded to RF and fr-secondstrand corresponded to FR).
> If you don't know whether your reads are from a stranded library or not, you could use the
> tool 'RSeQC Infer Experiment' to try to determine.
>
> As an option, a reference annotation file in GTF/GFF3 format can be provided to StringTie. In
> this case, StringTie will prefer to use these "known" genes from the annotation file, and for
> the ones that are expressed it will compute coverage, TPM and FPKM values. It will also produce
> additional transcripts to account for RNA-seq data that aren't covered by (or explained by) the
> annotation. Note that if option `-e` is not used, the reference transcripts need to be fully covered
> by reads in order to be included in StringTie's output. In that case, other transcripts assembled
> from the data by StringTie and not present in the reference file will be printed as well.
>
> We highly recommend that you provide annotation if you are analyzing a genome that is well
> annotated, such as human, mouse, or other model organisms.
{: .details}

## Evaluate the assembly with annotated transcripts

[GffCompare](https://ccb.jhu.edu/software/stringtie/gffcompare.shtml) compares and evaluates the accuracy of RNA-Seq transcript assemblers (Cufflinks, Stringtie). * collapse (merge) duplicate transcripts from multiple GTF/GFF3 files (e.g. resulted from assembly of different samples) * classify transcripts from one or multiple GTF/GFF3 files as they relate to reference transcripts provided in a annotation file (also in GTF/GFF3 format)

The original form of this program is also distributed as part of the Cufflinks suite, under the name [CuffCompare](http://cole-trapnell-lab.github.io/cufflinks/cuffcompare/). Most of the options and parameters of CuffCompare are supported by GffCompare, while new features will likely be added to GffCompare in the future.

> <hands-on-title>compare assembled transcripts against a reference annotation</hands-on-title>
>
> 1. {% tool [GffCompare](toolshed.g2.bx.psu.edu/repos/iuc/gffcompare/gffcompare/0.12.6+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"GTF inputs for comparison"*`Stringtie_output.gtf`
>    - *"Use Reference Annotation"*: `yes`
>      - *"Choose the source for the reference annotation"*: `History`
>        - {% icon param-file %} *"Reference Annotation"*: `Mus_musculus.GRCm38.86.gtf`
>    - *"Strict match"*: `yes`
>
{: .hands_on}

> <details-title>GffCompare vs CuffCompare</details-title>
> A notable difference between GffCompare and CuffCompare is that when a single query GTF/GFF file is
> given as input along with a reference annotation (`-r` option), GFFCompare switches into "annotation mode"
> and it generates a `.annotated.gtf` file instead of the `.merged.gtf` produced by CuffCompare with the same
> parameters. This file has the same general format as CuffCompare's .merged.gtf file (with "class
> codes" assigned to transcripts as per their relationship with the matching/overlapping reference,
> transcript) but the original transcript IDs are preserved, so GffCompare can thus be used as a
> simple way of annotating a set of transcripts.
>
> Another important difference is that the input transcripts are no longer discarded when they
> are found to be "intron redundant", i.e. contained within other, longer isoforms. CuffCompare
> had the `-G` option to prevent collapsing of such intron redundant isoforms into their longer
> "containers", but GffCompare has made this the default mode of operation (hence the `-G` option is
> no longer needed and is simply ignored when given).
>
{: .details}

## Translate transcripts

> <hands-on-title>Translate transcripts to protein sequence</hands-on-title>
>
> First, we must convert the GffCompare annotated GTF file to BED format.
>
> 1. {% tool [Convert gffCompare annotated GTF to BED](toolshed.g2.bx.psu.edu/repos/galaxyp/gffcompare_to_bed/gffcompare_to_bed/0.2.1) %} with the following parameters:
>  - {% icon param-file %} *"GTF annotated by gffCompare"*: `output from gff compare`
>  - *"filter GffCompare class_codes to convert"*:
>    - {% icon param-check %} `j : Potentially novel isoform (fragment): at least one splice junction is shared with a reference transcript`
>    - {% icon param-check %} `e : Single exon transfrag overlapping a reference exon and at least 10 bp of a reference intron, indicating a possible   pre-mRNA fragment.`
>    - {% icon param-check %} `i : A transfrag falling entirely within a reference intron`
>    - {% icon param-check %} `p : Possible polymerase run-on fragment (within 2Kbases of a reference transcript)`
>    - {% icon param-check %} `u : Unknown, intergenic transcript`
>
>    Next, we translate transcripts from the input BED file into protein sequences.
>
> 2. {% tool [Translate BED transcripts cDNA in 3frames or CDS](toolshed.g2.bx.psu.edu/repos/galaxyp/translate_bed/translate_bed/0.1.0) %} with the following parameters:
>   - {% icon param-file %} *"A BED file with 12 columns"*: `Convert GffCompare-annotated GTF to BED`
>   - *"Source for Genomic Sequence Data"*: `Locally cached twobit`
>   - *"Select reference 2bit file"*: `mm10`
>   - *"Fasta ID Options"*
>     - *"fasta ID source, e.g. generic"*: `generic`
>
>    Finally, we convert a BED format file of the proteins from a proteomics search database into a tabular format for the Multiomics Visualization Platform (MVP).
>
> 3. {% tool [bed to protein map](toolshed.g2.bx.psu.edu/repos/galaxyp/bed_to_protein_map/bed_to_protein_map/0.2.0) %} with the following parameters:
>   - {% icon param-file %} *"A BED file with 12 columns, thickStart and thickEnd define protein coding region"*: `Translate cDNA_minus_CDS`
>
{: .hands_on}

> <comment-title>Note on data formats</comment-title>
> - The tabular output can be converted to a sqlite database using the Query_Tabular tool.
> - The sqlite table should be named: feature_cds_map.
> - The names for the columns should be: name, chrom, start, end, strand, cds_start, cds_end
> - This SQL query will return the genomic location for a peptide sequence in a protein
>   (multiply the amino acid position by 3 for the cds location)
{: .comment}

# Creating FASTA Databases

In this section we will perform the following tasks:
- The **Protein Database Downloader** tool is used to download the FASTA database from UniProt and cRAP (**c**ommon **R**epository of **A**dventitious **P**roteins) database containing known/reference mouse proteins. For the purpose of this tutorial, we will use the `Reference_5000_uniprot_cRAP.fasta` file as this database.
- The **FASTA Merge Files and Filter Unique Sequences** tool is used to merge the databases obtained from CustomProDB and "Translate BED" tool
along with the UniProt and cRAP databases.
- The **Regex Text Manipulation** tool is used to manipulate the FASTA file to make it SearchGUI-compatible.

> <hands-on-title></hands-on-title>
>
> 1. {% tool [FASTA Merge Files and Filter Unique Sequences](toolshed.g2.bx.psu.edu/repos/galaxyp/fasta_merge_files_and_filter_unique_sequences/fasta_merge_files_and_filter_unique_sequences/1.2.0) %} with the following parameters:
>   - *"Run in batch mode?"*: `Merge individual FASTAs (output collection if input is collection)`
>   - Click {% icon param-repeat %} *"Insert Input FASTA File(s)"* three times so that there are a total of three *"Input FASTA File(s)"* blocks.
>   - Use the following parameters:
>     - {% icon param-file %} *"Input FASTA File(s)"* : 
>       ```
>       1.HISAT2_Output.rpkm
>       2.HISAT2_Output.snv
>       3.HISAT2_Output.indel
>       ```
>     - *"How are sequences judged to be unique?"*: `Accession and Sequence`
>     - *"Accession Parsing Regular Expression"*: `^>([^ ]+).*$`
>
> 2. Rename the output `Merged and Filtered FASTA from CustomProDB`
>
{: .hands_on}
> <comment-title>Tool parameters explained</comment-title>
> This tool concatenates FASTA database files together.
> - If the uniqueness criterion is "Accession and Sequence", only the first appearence of each unique sequence will appear in the output. Otherwise, duplicate sequences are allowed, but only the first appearance of each accession will appear in the output.
> - The default accession parser will treat everything in the header before the first space as the accession.
{: .comment}

> <comment-title>CustomProDB generated protein Accession names need to be changed for PeptideShaker</comment-title>
> PeptideShaker does not allow greaterthan sign, comma, or spaces in the protein accession name.
> PeptideShaker also requires the FASTA ID to start with a source, e.g. *>sp|Q8C4J7|* for SwissProt entries.  Nonstandard entries must be prefixed *generic|*.
> We will also need to similarily modify the accessions in the genomic mapping and and variant annotation outputs. For eg: ENSEMBL-PRO (output from CustomProDB) and STRG (from StringTie) are not standard format.
{: .comment}

> <hands-on-title>FASTA to Tabular</hands-on-title>
> 1. {% tool [FASTA-to-Tabular](toolshed.g2.bx.psu.edu/repos/devteam/fasta_to_tabular/fasta2tab/1.1.1) %} with the default parameters
>    - {% icon param-file %} *"Convert these sequences"*: `Merged and Filtered FASTA from CustomProDB`
>
> 2. {% tool [Column Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/galaxyp/regex_find_replace/regexColumn1/1.0.3) %} with the following parameters:
>    - {% icon param-file %} *"Select cells from"*: `FASTA-to-Tabular output` from previous step
>    - *"Using"*: `Column: 1`
>    - {% icon param-repeat %} **Insert Check**
>      - *"Find Regex"* : `^(ENS[^_]+_\d+:)([ACGTacgt]+)>([ACGTacgt]+)\s*`
>      - *"Replacement"*: `\1\2_\3`
>    - {% icon param-repeat %} **Insert Check**
>      - *"Find Regex"*:  `,([A-Y]\d+[A-Y]?)\s*`
>      - *"Replacement"*: `.\1`
>    - {% icon param-repeat %} **Insert Check**
>      - *"Find Regex"*: `^(ENS[^ |]*)\s*`
>      - *"Replacement"*: `generic|\1`
>
> 2. {% tool [Tabular-to-FASTA](toolshed.g2.bx.psu.edu/repos/devteam/tabular_to_fasta/tab2fasta/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Tab-delimited file"*: `Column Regex Find And Replace Output`
>    - *"Title column"*: `Column: 1`
>    - *"Sequence Column"*:`Column: 2`
>    - Rename it as `CustomProDB Merged Fasta`
>
{: .hands_on}

> <comment-title><b>FASTA-to-Tabular</b> and <b>Tabular-to-FASTA</b> versions</comment-title>
> Please make sure the tool version is 1.1.1
> Also, while running the workflow, the Title column for **Tabular-to-FASTA** might come empty, so please make sure you select Title Column as "1".
{: .comment}

![Fasta sequence](../../images/Fasta_sequence.png)


For visualization purposes we also use the concatenate tool to concatenate the genomic mapping with the protein mapping dataset. This output will be used for visualization in MVP to view the genomic coordinates of the variant peptide.

We also need to modify the CustomProDB protein accessions the same as was done for the FASTA database.

An SQLite database containing the genomic mapping SQLite, variant annotation and information from the protein mapping file is concatenated to form a single genomic mapping SQLite database later used as an input for the 'Peptide Genomic Coordinate' tool. For that we need to follow the steps below:

## Genomic mapping database

> <hands-on-title>Create Database for genomic mapping</hands-on-title>
>
> 1. {% tool [SQLite to tabular for SQL query](toolshed.g2.bx.psu.edu/repos/iuc/sqlite_to_tabular/sqlite_to_tabular/3.2.1) %} with the following parameters:
>   - {% icon param-file %} *"SQLite Database"*: `genomic_mapping.sqlite` from CustomProDB
>   - {% icon param-text %} *"Query"*:
>      ```
>      SELECT pro_name, chr_name, cds_chr_start - 1, cds_chr_end,strand,cds_start - 1, cds_end
>      FROM genomic_mapping
>      ORDER BY pro_name, cds_start, cds_end
>      ```
>   - {% icon param-file %} *"Omit column headers from tabular output"*: `No`
>
> 2. Rename the output as `genomic_mapping_sqlite`
>
>    {% snippet faqs/galaxy/datasets_rename.md %}
>
>    The output is further processed so that the results are compatible with the Multiomics Visualization Platform.
>    We need to modify the CustomProDB protein accessions the same as was done for the FASTA database.
>
> 3. {% tool [Column Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/galaxyp/regex_find_replace/regexColumn1/1.0.3) %} with the following parameters:
>    - {% icon param-file %} *"Select cells from"*: `genomic_mapping_sqlite' (tabular)`
>    - *"Using"*: `column 1`
>    - {% icon param-repeat %} **Insert Check**
>      - *"Find Regex"* : `^(ENS[^_]+_\d+:)([ACGTacgt]+)>([ACGTacgt]+)\s*`
>      - *"Replacement"*: `\1\2_\3`
>    - {% icon param-repeat %} **Insert Check**
>      - *"Find Regex"*:  `,([A-Y]\d+[A-Y]?)\s*`
>      - *"Replacement"*: `.\1`
>    - {% icon param-repeat %} **Insert Check**
>      - *"Find Regex"*: `^(ENS[^ |]*)\s*`
>      - *"Replacement"*: `\1`
>
> 4. Rename the output as `SAV_INDEL`
>
>     This tool goes line by line through the specified input file and if the text in the selected column matches a specified regular expression pattern, it replaces the text with the specified replacement.
>
>     Next, we will concatenate the output from this tool with the "Bed to protein map" output.
>
> 5. {% tool [Concatenate multiple datasets](toolshed.g2.bx.psu.edu/repos/artbio/concatenate_multiple_datasets/cat_multi_datasets/1.4.3) %} with the following parameters:
>   - {% icon param-files %} *"Concatenate Datasets"*: Select the output from the previous tool named as `SAV_INDEL` and the output from the `bed to protein map` tool and run the tool.
>   - Rename the output as `Genomic_Protein_map`
>
>     Now, we load this tabular datasets into an SQLite database.
>
> 6. {% tool [Query Tabular using SQLite](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - {% icon param-repeat %} **Insert Database Table**
>      - {% icon param-file%} *"Tabular Dataset for Table"*: `Genomic_Protein_map`
>      - Section **Table Options**:
>        - *"Specify Name for Table"*: `feature_cds_map`
>        - *"Specify Column Names (comma-separated list)"*: `name,chrom,start,end,strand,cds_start,cds_end`
>        - {% icon param-repeat %} **Insert Table Index**:
>          - *"This is a unique index"*: `No`
>          - *"Index on columns"*: `name,cds_start,cds_end`
>        - *"Save the sqlite database in your history"*:`Yes`
>
> 7. Rename the output as `genomic_mapping_sqlitedb`
>
>    ![genomic mapping](../../images/genomic_mapping_file.png)
>
{: .hands_on}


## Variant Annotations database

We will repeat the process for the variant annotations (this file will be used as a input to the new updates that will be made on the Multiomics Visualization platform).  We need to modify the CustomProDB protein accessions the same as was done for the FASTA database.

> <hands-on-title>Create database for variant annotations</hands-on-title>
>
> 1. {% tool [SQLite to tabular for SQL query](toolshed.g2.bx.psu.edu/repos/iuc/sqlite_to_tabular/sqlite_to_tabular/3.2.1) %} with the following parameters:
>  - {% icon param-file %} *"SQLite Database"*: `variant_annotations.sqlite` from CustomProDB
>  -  *"Query"*:
>    ```
>    SELECT var_pro_name,pro_name,cigar,annotation
>    FROM variant_annotation
>    ```
> 2. Rename the output as `variant_annotation_sqlite`
> 
>    We will subject the output to text manipulation so that the results are compatible with the Multiomics Visualization Platform.
>
> 3. {% tool [Column Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/galaxyp/regex_find_replace/regexColumn1/1.0.3) %} with the following parameters:
>    - {% icon param-file %} *"Select cells from"*: `variant_annotations_sqlite' (tabular)`
>    -  *"Using:"* `Column: 1`
>    - {% icon param-repeat %} **Insert Check**
>      -  *"Find Regex"*: `^(ENS[^_]+_\d+:)([ACGTacgt]+)>([ACGTacgt]+)\s*`
>      -  *"Replacement"*: `\1\2_\3`
>    - {% icon param-repeat %} **Insert Check**:
>      -  *"Find Regex"*: `,([A-Y]\d+[A-Y]?)\s*`
>      -  *"Replacement"*: `.\1`
>    - {% icon param-repeat %} **Insert Check**:
>      -  *"Find Regex"*: `^(ENS[^ |]*)\s*`
>      -  *"Replacement"*:  `\1`
> 
> 4. Rename the output as `variant_annotation`
> 
>    Next, we will load this tabular dataset into an SQLite database.
>
> 5. {% tool [Query Tabular using SQLite](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - {% icon param-repeat %} **Insert Database Table**
>      - {% icon param-file %} *"Tabular Dataset for Table"*: `variant_annotation`
>      - Section **Table Options**:
>        -  *"Specify Name for Table"*: `variant_annotation`
>        -  *"Specify Column Names (comma-separated list)"*: `name,reference,cigar,annotation`
>        - {% icon param-repeat %} **Insert Table Index**
>          -  *"This is a unique index"*: `No`
>          -  *"Index on columns"*: `name,cigar`
>      -  *"Save the sqlite database in your history"*:`Yes`
> 6. Rename the output as `Variant_annotation_sqlitedb`
>
{: .hands_on}

These SQLite databases, which contain the genomic mapping SQLite and variant
annotation information from the protein mapping file, will be utilized
by MVP to visualize the genomic loci of any variant peptides.


![SNP variant](../../images/viewing_SNP_Variant_IGV.png)

Finally, we can create a database which can be used to search Mass spectrometry (raw/mgf) files to identify peptides that match to this database.

To do so:

> <hands-on-title></hands-on-title>
>
> 1. {% tool [FASTA Merge Files and Filter Unique Sequences](toolshed.g2.bx.psu.edu/repos/galaxyp/fasta_merge_files_and_filter_unique_sequences/fasta_merge_files_and_filter_unique_sequences/1.2.0) %} with the following parameters:
>    - *"Run in batch mode?"*: `Merge individual FASTAs (output collection if input is collection)`
>    - Click {% icon param-repeat %} *"Insert Input FASTA File(s)"* three times so that there are a total of three *"Input FASTA File(s)"* blocks.
>    - {% icon param-files %} *"Input FASTA File(s)"* : `Custom ProDB Merged Fasta File output`
>    - {% icon param-files %} *"Input FASTA File(s)"* : `Translate BED transcripts Fasta output`
>    - {% icon param-files %} *"Input FASTA File(s)"* : `Reference_5000_uniprot_cRAP.fasta`
>    - *"How are sequences judged to be unique?"*: `Accession and Sequence`
>    - *"Accession Parsing Regular Expression"*: `^>([^ ]+).*$`
>
{: .hands_on}

> <comment-title>Tool parameters explained</comment-title>
> This tool concatenates FASTA database files together.
> - If the uniqueness criterion is "Accession and Sequence", only the first appearence of each unique sequence will appear in the output. Otherwise, duplicate sequences are allowed, but only the first appearance of each accession will appear in the output.
> - The default accession parser will treat everything in the header before the first space as the accession.
{: .comment}

## List of Reference Protein Accession names

Generate a list of Reference Proteins. Identify peptides that are contained in the reference proteins will be filtered out in the next tutorial.

> <hands-on-title>FASTA to Tabular</hands-on-title>
> 1. {% tool [FASTA-to-Tabular](toolshed.g2.bx.psu.edu/repos/devteam/fasta_to_tabular/fasta2tab/1.1.1) %} with the default parameters:
>    - {% icon param-file %} *"Convert these sequences"*: `HISAT_Output.rpkm`
>
> 2. {% tool [Filter Tabular](toolshed.g2.bx.psu.edu/repos/iuc/filter_tabular/filter_tabular/3.3.1) %} with these filters:
>    - {% icon param-file %} *"Tabular Dataset to filter"*: output of `FASTA-to-Tabular` on HISAT_Output.rpkm
>    - {% icon param-repeat %} Insert Filter Tabular Input Lines
>      - *"Filter by"*: `select columns`
>      - *"enter column numbers to keep"*: `1`
>    - {% icon param-repeat %} Insert Filter Tabular Input Lines
>      - *"Filter by"*: `regex replace value in column`
>      - *"enter column number to replace"*: `1`
>      - *"regex pattern"*: `^([^ |]+).*$`
>      - *"replacement expression"*: `\1`
>
> 3. {% tool [FASTA-to-Tabular](toolshed.g2.bx.psu.edu/repos/devteam/fasta_to_tabular/fasta2tab/1.1.1) %} with the default parameters:
>    - {% icon param-file %} *"Convert these sequences"*: `Trimmed_ref_5000_uniprot_cRAP.fasta`
>
> 4. {% tool [Filter Tabular](toolshed.g2.bx.psu.edu/repos/iuc/filter_tabular/filter_tabular/3.3.1) %} with these filters:
>    - {% icon param-file %} *"Tabular Dataset to filter"*: output of `FASTA-to-Tabular` on Trimmed_ref_5000_uniprot_cRAP.fasta
>    - {% icon param-repeat %} Insert Filter Tabular Input Lines
>      - *"Filter by"*: `select columns`
>      - *"enter column numbers to keep"*: `1`
>    - {% icon param-repeat %} Insert Filter Tabular Input Lines
>      - *"Filter by"*: `regex replace value in column`
>      - *"enter column number to replace"*: `1`
>      - *"regex pattern"*: `^[^|]+[|]([^| ]+).*$`
>      - *"replacement expression"*: `\1`
>
> 5. {% tool [Concatenate multiple datasets](toolshed.g2.bx.psu.edu/repos/artbio/concatenate_multiple_datasets/cat_multi_datasets/1.4.3) %} with the following parameters:
>    - {% icon param-files %} *"Concatenate Datasets"*: Select the output from the previous 2 "Filter Tabular" outputs.
>
{: .hands_on}

> <comment-title><b>FASTA-to-Tabular</b> and <b>Tabular-to-FASTA</b> versions</comment-title>
> Please make sure the tool version is 1.1.1
{: .comment}

> <comment-title>Tool versions</comment-title>
> - All the tools mentioned in this tutorial are subjected to change when the tool version is upgraded .
{: .comment}

# Conclusion

This completes the walkthrough of the proteogenomics database creation workflow. This tutorial is a guide to have a database and mapping files ready for  Database searching and novel peptide analysis. Researchers can use this workflow with their data also, please note that the tool parameters, reference genomes and the workflow will be needed to be modified accordingly.

This workflow was developed by the Galaxy-P team at the University of Minnesota. For more information about Galaxy-P or our ongoing work, please visit us at [galaxyp.org](https://galaxyp.org)

{: .comment}


topic: proteomics
tutorial: proteogenomics-dbsearch
title: Proteogenomics 2: Database Search
difficulty: intermediate
time_estimation: 15m
tools_json: []
requirements_json: []
tags_json: ["[proteogenomics]"]
content_hash: 4d5acbd26a08115453081ce40ed489eb
last_modified: 2026-02-26T21:12:17.511585
zenodo_link: https://doi.org/10.5281/zenodo.1489208
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/proteogenomics-dbsearch/tutorial.html

content:
In this tutorial, we perform proteogenomic database searching using the Mass Spectrometry data. The inputs for performing the proteogenomic database searching are the peaklist MGF files and the FASTA database file. The FASTA database is obtained by running the first workflow “Uniprot_cRAP_SAV_indel_translatedbed.FASTA”. The second workflow focuses on performing database search of the peak list files (MGFs).

![Workflow](../../images/second_workflow.png)


> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
> 1. TOC
> {:toc}
>
{: .agenda}



# Pretreatments

> <hands-on-title>data upload and organization</hands-on-title>
>
> 1. Create a **new history** and name it something meaningful (e.g. *Proteogenomics DB search*)
> 2. Import the four MGF MS/MS files and the Trimmed_ref_5000_uniprot_cRAP.FASTA sequence file from Zenodo.[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.13270741.svg)](https://doi.org/10.5281/zenodo.13270741)
>    ```
>    https://zenodo.org/record/1489208/files/Mo_Tai_Trimmed_mgfs__Mo_Tai_iTRAQ_f4.mgf
>    https://zenodo.org/record/1489208/files/Mo_Tai_Trimmed_mgfs__Mo_Tai_iTRAQ_f5.mgf
>    https://zenodo.org/record/1489208/files/Mo_Tai_Trimmed_mgfs__Mo_Tai_iTRAQ_f8.mgf
>    https://zenodo.org/record/1489208/files/Mo_Tai_Trimmed_mgfs__Mo_Tai_iTRAQ_f9.mgf
>    https://zenodo.org/records/15359565/files/Uniprot_cRAP_SAV_indel_translatedbed.fasta
>    https://zenodo.org/records/13270741/files/Reference_Protein_Accessions.tabular
>
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Rename the datasets to something more recognizable (strip the URL prefix)
> 4. Verify that the `Reference_Protein_Accessions.tabular` format is `tabular`.
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="tabular" %}
>
> 5. Build a **Dataset list** for the four MGF files, name it as `Mo_Tai_MGFs`
>
>    {% snippet faqs/galaxy/collections_build_list.md %}
>
{: .hands_on}


# Match peptide sequences

The search database labeled `Uniprot_cRAP_SAV_indel_translatedbed.FASTA` is the input database that
will be used to match MS/MS to peptide sequences via a sequence database search.

For this, the sequence database-searching program called [SearchGUI](https://compomics.github.io/projects/searchgui.html) will be used.The generated dataset collection of the three *MGF files* in the history is used as the MS/MS input. We will walk through a number of these settings in order to utilize SearchGUI on these example MGF files.



## SearchGUI

> <hands-on-title>SearchGUI</hands-on-title>
>
> 1. {% tool [Identification Parameters](toolshed.g2.bx.psu.edu/repos/galaxyp/peptideshaker/ident_params/4.0.41+galaxy1) %} with the following parameters:
>    - *"Fixed Modifications"*: `Carbamidomethylation of C, ITRAQ-4Plex of K, ITRAQ-4Plex of peptide N-term`
>    - *"Variable modifications"*: `Oxidation of M, ITRAQ-4Plex of Y`
>    - *"Fragment Tolerance (Daltons)"*: `0.05` (this is high resolution MS/MS data)
>    - *"Minimum charge"*:`2`
>    - *"Maximum charge"*:`6`
>    - Section **Search Engine Options**:
>      - *"X!Tandem Options"*: `Advanced`
>      - *"X!Tandem: Advanced Search Related"*: `Set Advanced Search Parameters`
>        - *"X!Tandem: Quick Acetyl"*: `No`
>        - *"X!Tandem: Quick Pyrolidone"*: `No`
>      - Section **X!Tandem peptide model refinement**
>        - *"X!Tandem: Maximum Valid Expectation Value, refinement"* : `100`
>
>
> 1. {% tool [Search GUI](toolshed.g2.bx.psu.edu/repos/galaxyp/peptideshaker/search_gui/4.0.41+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"Identification Parameters file"*: `Identification Parameters: PAR file`
>    - {% icon param-file %} *"Protein Database"*: `Uniprot_cRAP_SAV_indel_translatedbed.FASTA` (Or however you named the `FASTA` file)
>
>        > <comment-title></comment-title>
>        >    The "Uniprot_cRAP_SAV_indel_translatedbed" FASTA database is obtained when you run the first proteogenomics workflow *"Proteogenomics 1: Database Creation"*.
>        >    Please make sure to run the 1st workflow.
>        {: .comment}
>
>    - {% icon param-collection %} *"Input Peak lists (mgf)"*: `Mo_Tai_MGFs` dataset collection.
>
>      > <tip-title>Select dataset collections as input</tip-title>
>      >
>      > * Click the **Dataset collection** icon on the left of the input field:
>      >
>      > * Select the appropriate dataset collection from the list
>      {: .tip}
>
>    - Section **Search Engine Options**:
>      - {% icon param-check %} *"DB-Search Engines"*: `X!Tandem`
>
>        > <comment-title></comment-title>
>        >    The section **Search Engine Options** contains a selection of sequence database searching
>        >    programs that are available in SearchGUI. Any combination of these programs can be used for
>        >    generating PSMs from MS/MS data. For the purpose of this tutorial, **X!Tandem** we will be
>        >    used.
>        {: .comment}
>
>
> Once the database search is completed, the SearchGUI tool will output a file (called a
> SearchGUI archive file) that will serve as an input for the next section, PeptideShaker.
> Rename the output as "**Compressed SearchGUI results**"
>
>
> > <comment-title></comment-title>
> > Note that sequence databases used for proteogenomics are usually much larger than
> > the excerpt used in this tutorial. When using large databases, the peptide identification
> > step can take much more time for computation. In proteogenomics, choosing the optimal
> > database is a crucial step of your workflow.
> {: .comment}
>
{: .hands_on}



## PeptideShaker

[PeptideShaker](https://compomics.github.io/projects/peptide-shaker.html) is a post-processing software tool that
processes data from the SearchGUI software tool. It serves to organize the Peptide-Spectral
Matches (PSMs) generated from SearchGUI processing and is contained in the SearchGUI archive.
It provides an assessment of confidence of the data, inferring proteins identified from the
matched peptide sequences and generates outputs that can be visualized by users to interpret
results. PeptideShaker has been wrapped in Galaxy to work in combination with SearchGUI
outputs.

> <comment-title>File Formats</comment-title>
>   There are a number of choices for different data files that can be generated using
>   PeptideShaker. A compressed file can be made containing all information needed to
>   view them results in the standalone PeptideShaker viewer. A `mzidentML` file can
>   be generated that contains all peptide sequence matching information and can be
>   utilized by compatible downstream software. Other outputs are focused on the inferred
>   proteins identified from the PSMs, as well as phosphorylation reports, relevant if
>   a phosphoproteomics experiment has been undertaken.
{: .comment}


> <hands-on-title>PeptideShaker</hands-on-title>
>
> 1. {% tool [Peptide Shaker](toolshed.g2.bx.psu.edu/repos/galaxyp/peptideshaker/peptide_shaker/2.0.33+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"Compressed SearchGUI results"*: The `Search GUI` archive file
>    - *"Include the protein sequences in mzIdentML"*: `No`
>    - *"Contact Information"*: `Specify Contact Information`
>      - Enter your own informations
>    - *"Output options"*: Select the `PSM Report` (Peptide-Spectral Match) and the `Certificate of Analysis`
>
>     > <comment-title>Certificate of Analysis</comment-title>
>     >
>     > The "Certificate of Analysis" provides details on all the parameters
>     > used by both SearchGUI and PeptideShaker in the analysis. This can be downloaded from the
>     > Galaxy instance to your local computer in a text file if desired.
>     >
>     {: .comment}
>
> 2. Inspect {% icon galaxy-eye %} the resulting files
>
{: .hands_on}

A number of new items will appear in your History, each corresponding to the outputs selected in the PeptideShaker parameters. The Peptide Shaker’s PSM report is used as an input for the BlastP analysis. Before performing BlastP analysis. The Query Tabular tool and few test manipulation tools are used to remove spectra that belongs to the reference proteins. The output tabular file “Peptides_for_Blast-P_analysis” will contain only those spectra that did not belong to any known proteins.

# Create a SQLite database for peptide, protein and genomic annotation visualization

The mzidentml output from the Peptide shaker is converted into an sqlite database file by using the mz to sqlite tool. This sqlite output is used to open the Multi-omics visualization platform, wherein you can view the spectra of the peptides using Lorikeet parameters. To open the MVP viewer, click on the “Visualize" {% icon galaxy-visualise %}  icon and select "MVP Application" ( this will open the interactive multi-omics viewer)


> <hands-on-title>mz to sqlite</hands-on-title>
>
> This tool extracts mzidentml and its associated proteomics datasets into a sqlite db
>
> 1. {% tool [mz to sqlite](toolshed.g2.bx.psu.edu/repos/galaxyp/mz_to_sqlite/mz_to_sqlite/2.1.1+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"Proteomics identification files"*: `PeptideShaker_mzidentml`
>    - {% icon param-collection %} *"Proteomics Spectrum files"*: `Mo_Tai_MGFs`
>    - {% icon param-file %} *"Proteomics Search Database Fasta"*: `Uniprot_cRAP_SAV_indel_translatedbed.FASTA`
>
>    ![mz2sqlite](../../images/mz2sqlite.png){:width="20%"}
>
{: .hands_on}

The next step is to remove known peptides from the list of PSMs that we acquired from the Peptide Shaker results. For that, we need to perform Query tabular to extract the list of known peptides from the UniProt and cRAP database.


## Query Tabular

> <hands-on-title>Remove Reference proteins</hands-on-title>
>
>  1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>
>     - {% icon param-repeat %} **Insert Database Table** (b): `PSM report`
>       - Section **Filter Dataset Input**:
>         - {% icon param-repeat %} **Insert Filter Tabular Input Lines**
>         - *"Filter by"*:  `skip leading lines`
>         - *"Skip lines"*: `1`
>
>     - Section **Table Options**:
>       - *"Specify Name for Table"*: `psms`
>       - *"Use first line as column names"* : `No`
>       - *"Specify Column Names (comma-separated list)"*:`id,Proteins,Sequence`
>       - *"Only load the columns you have named into database"*: `Yes`
>       - {% icon param-repeat %} **Table Index**
>         - *"This is a unique index"*: `No`
>         - *"Index on Columns"*: `id`
>
>     - {% icon param-repeat %} **Insert Database Table** (c):`PSM report`
>
>     - Section **Filter Dataset Input**
>       - {% icon param-repeat %} **Insert Filter Tabular Input Lines**
>         - *"Filter by"*:  `skip leading lines`
>         - *"Skip lines"*: `1`
>       - {% icon param-repeat %} **Insert Filter Tabular Input Lines**
>         - *"Filter by"*:  `select columns`
>         - *"Enter column numbers to keep"*: `1,2`
>       - {% icon param-repeat %} **Insert Filter Tabular Input Lines**
>         - *"Filter by"*:  `normalize list columns,replicate rows for each item in the list`
>         - *"Enter column numbers to normalize"*: `2`
>         - *"List item delimiter in column"*: `,`
>
>     - Section **Table Options**:
>       - *"Specify Name for Table"*: `prots`
>       - *"Use first line as column names"* : `No`
>       - *"Specify Column Names (comma-separated list)"*:`id,prot`
>       - *"Only load the columns you have named into database"*: `Yes`
>       - {% icon param-repeat %} **Insert Table Index**:
>         - *"This is a unique index"*: `No`
>         - *"Index on Columns"*: `prot,id`
>
>     - {% icon param-repeat %} **Insert Database Table** (a): `Reference_Protein_Accessions`
>     - Section **Table Options**:
>       - *"Tabular Dataset for Table"*: `Uniprot`
>       - *"Use first line as column names"* : `No`
>       - *"Specify Column Names (comma-separated list)"*:`prot`
>       - {% icon param-repeat %} **Insert Table Index**:
>         - *"This is a unique index"*: `No`
>         - *"Index on Columns"*: `prot`
>
>
>       > <comment-title></comment-title>
>       >
>       > By default, table columns will be named: c1,c2,c3,...,cn (column names for a table must be unique).
>       > You can override the default names by entering a comma separated list of names, e.g. `,name1,,,name2`
>       > would rename the second and fifth columns.
>       > Check your input file to find the settings which best fits your needs.
>       >
>       {: .comment}
>
>
>     - *"Save the sqlite database in your history"*: `No`
>
>       > <comment-title>Querying an SQLite Database</comment-title>
>       > **Query Tabular** can also use an existing SQLite database.
>       > Activating `Save the sqlite database in your history`
>       > will store the generated database in the history, allowing to reuse it directly.
>       {: .comment}
>
>     - *"SQL Query to generate tabular output"*:
>       ```
>       SELECT psms.*
>       FROM psms
>       WHERE psms.id NOT IN
>       (SELECT distinct prots.id
>       FROM prots JOIN uniprot ON prots.prot = uniprot.prot)
>       ORDER BY psms.id
>       ```
>
>     - *"include query result column headers"*: `Yes`
>
>  - Click **Run Tool** and inspect the query results file after it turns green.
>
>
{: .hands_on}

The output from this step is that the resultant peptides would be those which doesn't belong in the Uniprot or cRAP database. The query tabular tool is used again to create a tabular output containing peptides ready for Blast P analysis.


> <hands-on-title>Query Tabular</hands-on-title>
>
>  1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>
>     - {% icon param-repeat %} **Insert Database Table** (b): `Query results`
>     - Section **Filter Dataset Input**:
>       - {% icon param-repeat %} **Insert Filter Tabular Input Lines**
>         - *"Filter by"*:  `skip leading lines`
>         - *"Skip lines"*: `1`
>
>
>     - Section **Table Options**:
>       - *"Specify Name for Table"*: `psm`
>       - *"Use first line as column names"* : `No`
>       - *"Specify Column Names (comma-separated list)"*:`id,Proteins,Sequence`
>       - *"Only load the columns you have named into database"*: `Yes`
>
>     - *"SQL Query to generate tabular output"*:
>       ```
>       SELECT Sequence || ' PSM=' || group_concat(id,',') || ' length='
>       || length(Sequence) as "ID",Sequence
>       FROM  psm
>       WHERE length(Sequence) >6
>       AND length(Sequence) <= 30
>       GROUP BY Sequence
>       ORDER BY length(Sequence),Sequence
>       ```
>     - *"include query result column headers"*: `Yes`
>
>     - Click **Run Tool** and inspect the query results file after it turns green.
>
> 2. Rename {% icon galaxy-pencil %} the output as `Peptides_for_Blast-P_analysis`
>
> ![QT](../../images/QT_output.png)
>
{: .hands_on}


# Getting data Blast-P ready


BlastP search is carried out with the PSM report (output from PeptideShaker). Before, BlastP analysis the “Peptides_for_Blast-P_analysis” is first converted from Tabular format to FASTA file format which can be easily read by the BlastP algorithm. This is done with the help of “Tabular to FASTA” conversion tool.
The short BlastP uses parameters for short peptide sequences (8-30 aas). Please use the rerun option to look at the parameters used.

> <hands-on-title>Tabular to FASTA (version 1.1.1)</hands-on-title>
> 1. {% tool [Tabular-to-FASTA](toolshed.g2.bx.psu.edu/repos/devteam/tabular_to_fasta/tab2fasta/1.1.1) %} with the following parameters:
>    - *"Tab-delimited file"*: `Peptides_for_Blast-P_analysis`
>    - *"Title column"*: `1`
>    - *"Sequence Column"*:`2`
>
{: .hands_on}



The output FASTA file is going to be subjected to BLAST-P analysis.

> <comment-title>Tool Versions</comment-title>
>
> The tools are subjected to changes while being upgraded.
> Thus, running the workflow provided with the tutorial, the user might need to make sure they are using the latest version including updating the parameters.
>
{: .comment}



# Conclusion


This completes the walkthrough of the proteogenomics database search workflow. This tutorial is a guide to performing database searching with mass spectrometry files and having peptides ready for Blast-P analysis, you can perform follow-up analysis using the next GTN "Proteogenomics Novel Peptide Analysis".
Researchers can use this workflow with their data also, please note that the tool parameters, reference genomes, and the workflow will need to be modified accordingly.

This workflow was developed by the Galaxy-P team at the University of Minnesota. For more information about Galaxy-P or our ongoing work, please visit us at [galaxyp.org](https://galaxyp.org)


{: .comment}


topic: proteomics
tutorial: ml-modeling-of-anti-cancer-peptides
title: Machine Learning Modeling of Anticancer Peptides
difficulty: intermediate
time_estimation: 30m
tools_json: []
requirements_json: []
tags_json: ["[ML, cancer]"]
content_hash: 83a93f624836d92e4b42beaf3e680b18
last_modified: 2026-02-26T21:12:17.498739
zenodo_link: https://doi.org/10.5281/zenodo.4111092
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/ml-modeling-of-anti-cancer-peptides/tutorial.html

content:
Biological molecules such as proteins, peptides, DNA, and RNA can be represented by their biochemical or sequence-based properties. These properties can be utilized to deduce biological meanings using ML modeling. A descriptor or feature is the quantitative or qualitative measure of a property that is associated with a sequence. For example, a chemical compound can be described via its charge chemical formula, molecular weight, number of rotatable bonds, etc. Similarly, several properties can be deduced from the biological sequence that can be utilized to describe a biological activity such as anticancer property. Properties associated with a group of peptide sequences such as overall charge, hydrophobicity profile, or k-mer composition can be utilized to build an ML model and this model can be used to predict biological properties of unknown peptides. Several computational methods have been proven very useful in the initial screening and prediction of peptides for various biological properties. These methods have emerged as effective alternatives to the lengthy and expensive traditional experimental approaches. Finding ACPs through wet-lab methods is costly and time-consuming; thus, the development of an efficient computational approach is useful to predict potential ACP peptides before wet-lab experimentation. In this tutorial, we will be discussing how peptide-based properties like charge, hydrophobicity, the composition of amino acids, etc. can be utilized to predict the biological properties of peptides. Additionally, we will learn how to use different utilities of the Peptide Design and Analysis Under Galaxy (PDAUG) package to calculate various peptide-based descriptors and use these descriptors for ML modeling. We will use CTD (composition, transition, and distribution) descriptor to define peptide sequences in the training set and will test 6 different ML algorithms. We will also assess the effect of normalization on the accuracy of ML models.



![Peptide descriptors](../../images/PDAUG_ML_1.png "ML algorithms use numerical representation of a sequence-based properties for model building")

In **Figure 1** features or descriptors are represented with (F1, F2, F3, etc.) and in binary classification, usually, class labels are represented by 0 or 1.


### Easy access to tools, workflows and data from the docker image

An easy way to install and use the PDAUG toolset, and follow this tutorial is via a prebuilt docker image equipped with a PDAUG toolset, workflow, and data library. A prebuilds docker image can be downloaded and run by typing a simple command at the terminal after installing docker software on any operating system.

> <hands-on-title>Easy access of tools, workflows and data from docker image</hands-on-title>
>
> 1. Downloading the docker image from the docker hub using `docker pull jayadevjoshi12/galaxy_pdaug:latest` command.
> 2. Running the container with latest PDAUG tools `docker run -i -t -p 8080:80 jayadevjoshi12/galaxy_pdaug:latest`.
> 3. Workflow is available under the workflow section, use `admin` as username and `password` as a password to login as an administrator of your galaxy instance.
> 4. Use `admin` as username and `password` as a password to login galaxy instance, which is available at [localhost](http://127.0.0.1:8080) to access workflow and data.
>
{: .hands_on}



> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

## Training data set

A high-quality dataset was retrieved from a previously published work {% cite hajisharifi_predicting_2014 %}.  In the, ML balance training set (an equal number of positive and negative samples) is always recommended. However, an imbalance training set can also be handled, and sometimes, to assess the robustness of the model an imbalanced training set is intentionally introduced.  The objective of this tutorial is to provide a basic introduction of ML in peptide research hence we will use a balanced training dataset with an equal number of positive (ACPs) and negative (non-ACPs) data.  A simple python code was applied to randomly select and removed several non-ACPs, which reduces their number from 205 to 138. The final training set contains 138 ACPs and 138 non-ACPs. The length distribution of the positive dataset is somewhat different from the negative dataset. Peptide length is an important feature in determining biological activity. Based on their length we assess the differences between ACPs and non-ACPs, and we found that except for a few outliers both ACPs and non-ACPs show a mean length of  40 and 32 respectively.



### Get data

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo](https://zenodo.org/record/4111092#.X712_6pKhhE) or from the shared data library
>
>    ```
>    https://zenodo.org/record/4111092/files/ACPs.fasta
>    https://zenodo.org/record/4111092/files/non_ACPs.fasta
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
> 3. Rename the datasets to their basename (ACPs.fasta, non_ACPs.fasta)
> 4. Check that the datatype is correctly set to fasta
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="fasta" %}
>
>
{: .hands_on}



## Calculating Peptide Descriptors

In this step we will calculate CTD descriptos. Composition describptors are defined as the number of amino acids of a particular property divided by total number of amino acids.  Transition descriptors are representd as the number of transition from a particular property to different property divided by (total number of amino acids − 1). Distribution descriptors are derived by chain length and the amino acids of a particular property are located on this length {% cite Govindan_Nair_2013 %}.


> <hands-on-title>Calculating CTD descriptors for ACPs and non-ACPs</hands-on-title>
>
> 1. {% tool [PDAUG Sequence Property Based Descriptors](toolshed.g2.bx.psu.edu/repos/jay/pdaug_sequence_property_based_descriptors/pdaug_sequence_property_based_descriptors/0.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Input fasta file"*: `ACPs.fasta` (output of **Input dataset** {% icon tool %})
>    - *"DesType"*: `CTD`
>
> 1. {% tool [PDAUG Sequence Property Based Descriptors](toolshed.g2.bx.psu.edu/repos/jay/pdaug_sequence_property_based_descriptors/pdaug_sequence_property_based_descriptors/0.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Input fasta file"*: `non_ACPs.fasta` (output of **Input dataset** {% icon tool %})
>    - *"DesType"*: `CTD`
>
>
{: .hands_on}


## Preparing a traning data set

We will combine the ACPs and non-ACPs data set as a single tabular data and will add the class label.

### Adding class labels to the training data


In Binary classification, ML algorithms classify the elements of a set into two groups based on a classification rule. Binary classes are usually represented by `0` and `1`. In our example, we are finding peptides with anticancer properties, therefore we can denote all the peptides with anticancer properties as `1` and peptides with non-anticancer properties as `0`.  In general, samples represented with `1` also describe as positive data, and samples with `0` labels as negative data.  Data with multi-class classification problems can also be represented by `0, 1, 2, 3,` etc. In addition to this, in ML the class labels can also be represented by a string such as "anticancer" and "non-anticancer" or "treated" and "untreated".  However, there are several ML tools and libraries prefer numerical class label over a string therefore, in this tutorial, `0` and `1` will be used as class labels.



> <hands-on-title>Adding class labels to the tabular data</hands-on-title>
>
> 1. {% tool [PDAUG Add Class Label](toolshed.g2.bx.psu.edu/repos/jay/pdaug_addclasslabel/pdaug_addclasslabel/0.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `PDAUG Sequence Property Based Descriptors on data 1 - CTD (tabular)` (output of **Peptide Sequence Descriptors** {% icon tool %})
>    - *"Class Label"*: `1`
>
> 2. {% tool [PDAUG Add Class Label](toolshed.g2.bx.psu.edu/repos/jay/pdaug_addclasslabel/pdaug_addclasslabel/0.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `PDAUG Sequence Property Based Descriptors on data 2 - CTD (tabular)` (output of **Peptide Sequence Descriptors** {% icon tool %})
>    - *"Class Label"*: `0`
>
>
{: .hands_on}


### Merging ACPs and non-ACPs samples to create a traning dataset

In previous steps, we labeled positive data or ACPs as "1", and negative data or non-ACPs as "0", now we can combine these two tabular datafiles as one training dataset.  In this step, we will merge these tabular detasets as a labeled training data set.

> <hands-on-title>Merging two tabular files</hands-on-title>
>
> 1. {% tool [PDAUG Merge Dataframes](toolshed.g2.bx.psu.edu/repos/jay/pdaug_merge_dataframes/pdaug_merge_dataframes/0.1.0) %} with the following parameters:
>    - {% icon param-files %} *"Input files"*: `PDAUG Add Class Label on data 3 - (tabular)` (output of **Add Class Label** {% icon tool %}), `PDAUG Add Class Label on data 4 - (tabular)` (output of **Add Class Label** {% icon tool %})
>
>
>
{: .hands_on}


## Applying 6 different ML algorithms on the training data set

In this step, we will apply six ML algorithms Linear Regression Classifier (LRC), Random Forest Classifier(RFC), Gaussian naive Bayes Classifier (GBC), Decision Tree Classifier (DTC), Stochastic Gradient Descent Classifier (SGDC) & Support Vector Machine Classifier (SVMC) with 10 fold cross-validation on the training data. In cross-validation, positive and negative data are randomly divided into 10 parts each set has the 10th part of active as well as inactive peptides. The algorithm was trained on the 9 sets and the prediction was made on the remaining 10th set. This process was repeated for every set. Thus the final performance scores are calculated as a mean of all the folds. We used min-max to normalize the data before ML modeling. The entire workflow was applied to the four descriptor sets and accuracy was estimated based on accuracy, precision, recall, f1, and AUC.


> <hands-on-title>Applying 6 ML algorithms on the training data set</hands-on-title>
>
> 1. {% tool [PDAUG ML Models](toolshed.g2.bx.psu.edu/repos/jay/pdaug_ml_models/pdaug_ml_models/0.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `PDAUG Merge Dataframes on data 6 and data 5 - (tabular)` (output of **Merge dataframes** {% icon tool %})
>    - *"Select Machine Learning algorithms"*: `LRC`
>        - *"Select advanced parameters"*: `No, use program defaults.`
>    - *"Choose the Test method"*: `Internal`
>    - *"Cross validation"*: `10`
>
> 2. {% tool [PDAUG ML Models](toolshed.g2.bx.psu.edu/repos/jay/pdaug_ml_models/pdaug_ml_models/0.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `PDAUG Merge Dataframes on data 6 and data 5 - (tabular)` (output of **Merge dataframes** {% icon tool %})
>    - *"Select Machine Learning algorithms"*: `RFC`
>        - *"Specify advanced parameters"*: `No, use program defaults.`
>    - *"Choose the Test method"*: `Internal`
>    - *"Cross validation"*: `10`
>
> 3. {% tool [PDAUG ML Models](toolshed.g2.bx.psu.edu/repos/jay/pdaug_ml_models/pdaug_ml_models/0.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `PDAUG Merge Dataframes on data 6 and data 5 - (tabular)` (output of **Merge dataframes** {% icon tool %})
>    - *"Select Machine Learning algorithms"*: `GBC`
>        - *"Specify advanced parameters"*: `No, use program defaults.`
>    - *"Choose the Test method"*: `Internal`
>    - *"Cross validation"*: `10`
>
> 4. {% tool [PDAUG ML Models](toolshed.g2.bx.psu.edu/repos/jay/pdaug_ml_models/pdaug_ml_models/0.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `PDAUG Merge Dataframes on data 6 and data 5 - (tabular)` (output of **Merge dataframes** {% icon tool %})
>    - *"Select Machine Learning algorithms"*: `DTC`
>        - *"Specify advanced parameters"*: `No, use program defaults.`
>    - *"Choose the Test method"*: `Internal`
>    - *"Cross validation"*: `10`
>
> 5. {% tool [PDAUG ML Models](toolshed.g2.bx.psu.edu/repos/jay/pdaug_ml_models/pdaug_ml_models/0.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `PDAUG Merge Dataframes on data 6 and data 5 - (tabular)` (output of **Merge dataframes** {% icon tool %})
>    - *"Select Machine Learning algorithms"*: `SGDC`
>        - *"Specify advanced parameters"*: `No, use program defaults.`
>    - *"Choose the Test method"*: `Internal`
>    - *"Cross validation"*: `10`
>
> 6. {% tool [PDAUG ML Models](toolshed.g2.bx.psu.edu/repos/jay/pdaug_ml_models/pdaug_ml_models/0.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Input file"*: `PDAUG Merge Dataframes on data 6 and data 5 - (tabular)` (output of **Merge dataframes** {% icon tool %})
>    - *"Select Machine Learning algorithms"*: `SVMC`
>        - *"Specify advanced parameters"*: `No, use program defaults.`
>    - *"Choose the Test method"*: `Internal`
>    - *"Cross validation"*: `10`
>
{: .hands_on}

## Results assessment

### Merging results in one file

In previous steps we have trained the ML models, these models return a TSV  that captures performance measures of these algorithms. We used the `Merge Data Frame` tool to combine these results as one file in this step.  

> <hands-on-title>Merging result as one tabular file</hands-on-title>
>
> 1. {% tool [PDAUG Merge Dataframes](toolshed.g2.bx.psu.edu/repos/jay/pdaug_merge_dataframes/pdaug_merge_dataframes/0.1.0) %} with the following parameters:
>    - {% icon param-files %} *"Input files"*: `PDAUG ML Models on data 7 - LRC (tabular)` (output of **ML Models** {% icon tool %}), `PDAUG ML Models on data 7 - RFC (tabular)` (output of **ML Models** {% icon tool %}), `PDAUG ML Models on data 7 - GBC (tabular)` (output of **ML Models** {% icon tool %}), `PDAUG ML Models on data 7 - DTC (tabular)` (output of **ML Models** {% icon tool %}), `PDAUG ML Models on data 7 - SGDC (tabular)` (output of **ML Models** {% icon tool %}), `PDAUG ML Models on data 7 - SVMC (tabular)` (output of **ML Models** {% icon tool %})
>
>
{: .hands_on}


### Creating a final heat map to assess the results

In the final step, a heat map will be generated which represents performance measures of various algorithms. We applied five different performance measures, accuracy, recall, F1-score, precision, and mean AUC (Area Under Curve) score.


> <hands-on-title>Plotting the results</hands-on-title>
>
> 1. {% tool [PDAUG Basic Plots](toolshed.g2.bx.psu.edu/repos/jay/pdaug_basic_plots/pdaug_basic_plots/0.1.0) %} with the following parameters:
>    - *"Data plotting method"*: `Heat Map`
>        - {% icon param-file %} *"Input file"*: `PDAUG Merge Dataframes on data 18, data 16, and others - (tabular)` (output of **PDAUG Merge Dataframes** {% icon tool %})
>        - *"Index Column"*: `Algo`
>        - *"Label for x-axis"*: `Performance Measures`
>        - *"Label for y-axis"*: `ML algorithms`
>
>
{: .hands_on}

![ML results](../../images/ML_HEATMAP.png "Heatmap represents the performance of 6 ML algorithms")

The performance of ML algorithms can be assessed by commonly used performance measures represented in **Figure 2**.

- **Accuracy** is described as correctly predicted instances and calculated based on true positive (TP) and true negative (TN) divided by TP, TN, false positive (FP), and false-negative (FN).
- **AUC** is Area under ROC curve, where ROC is a receiver operating characteristic. AUC represents the area covered by ROC.
- **F1** measures also an important estimate in model accuracy and can be defined as a harmonic mean of precision.
- **Precision** also known as the probability of positive values (PPV), is summarised as the probability of currently predicted positive instances and estimated based on TP  and FP.
- **Recall** also known as sensitivity, is defined as the estimation of the percentage of the correctly predicted positive instances and is also calculated with TP  and FP.


The value for each of these estimates falls between 0 and 1, and larger values indicating a better performance and accuracy. The brighter yellow color shows high-performance while the blue color shows a lower score.  Heat map suggests that algorithms GBC, LRC, and SVMC show high performs in comparison to the other three. Better performance means these classifiers have been able to classify ACPs and non-ACPs more accurately than others. DTC shows an intermediate performance while RFC and SGDC performed poorly on this data set. Finally, we learn how to calculate features, how to utilize these features to build ML models, and how we can assess the performance of an ML model. In the future, advanced parameters of this algorithm can be assessed to improve the performance of the models. Additionally, several features can be calculated other than CTD and utilized to build ML models and assess performance.


# Conclusion


In this tutorial, we learn how to utilize the quantitative properties of peptide sequences and apply the ML algorithms to predict the biological properties of the peptide sequence.

![Used workflow](../../images/MLWorkFlow.png "Workflow")


topic: proteomics
tutorial: ntails
title: Detection and quantitation of N-termini (degradomics) via N-TAILS
difficulty: intermediate
time_estimation: 1h
tools_json: []
requirements_json: []
tags_json: ["[]"]
content_hash: 2f1bf92e016b4ba851edf3995571e659
last_modified: 2026-02-26T21:12:17.505091
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/ntails/tutorial.html

content:
N-Tails is a special Proteomics technique to analyze peptide abundancy changes of protein N-termini. Prior to the MS measurement, N-Tails enriches unmodified, as well as acetylated N-termini. Both common and "unusual" N-termini are identified, where "unusual" means that the protein N-terminus was changes.
This is best explained by an example: directly after translation, a protein has exactly one N-terminus. When a protease is cutting the protein in half, each half has its own N-terminus. While the N-terminus of the first half protein is exactly the same as the one of the full protein precursor ("native N-terminus"), the N-terminus of the second half is different ("neo-N-terminus") and depends on the amino acid sequence where the protein was cut.
The N-Tails technique includes the use of heavy isotope dimethyl labelling.

The figure below illustrates the mechanism of N-Tails. It was originally published by Stefan Tholen (doctoral thesis, not available online). Further reading on N-Tails and other N-terminal techniques, see {% cite Tholen2013 %}.

![N-Tails technique](../../images/wf_ntails_technique.png "Overview of the N-Tails technique")

The N-Tails technique was originally designed to research protease biology and has most often been used in this field. It was originally published in {% cite Kleifeld2010 %}.

> <comment-title>Interpretation of N-Tails results</comment-title>
>
> Be careful when interpreting the results of N-Tails experiments. While the technique **is** fit to identify direct protease substrates, it does not discriminate direct from indirect ("downstream") effects. Thus, most of the identified N-termini will **not** be direct protease substrates, even if their change in protein abundance is statistically significant. To identify direct protease substrates, you have to further validate substrate candidates by comparing the prime and non-prime amino acids of each identified N-terminus with the protease cleavage motif. The information can be extracted from the peptide IDs, but this step is so far not included in the workflow.
>
> Lacking discrimination between direct and indirect effects is a general restriction also in other N-terminal screening techniques (e.g. COFRADIC), and is not specific for the N-Tails technique.
{: .comment}

This workflow was originally built in the OpenMS framework "TOPPAS" and published in [Lai, Weisser et al., MCP, 2016](https://www.ncbi.nlm.nih.gov/pubmed/27087653). It was converted to OpenMS v2.1, rebuild for the Galaxy framework and tested on the original dataset by Melanie Föll. It was designed for data analysis of a three samples combined in one MS run, a technique based on dimethyl stable isotope labeling (SIL). For more information on SIL, consult [this tutorial]({{site.baseurl}}/topics/proteomics/tutorials/protein-quant-sil/tutorial.html). The original data were generated using pre-fractionation. Thus, peptides of one biological experiment are measured in multiple consecutive MS runs (one run per fraction).

The figure below gives an overview of the used Galaxy nodes. For further description of the workflow, please consider the [original publication](https://www.ncbi.nlm.nih.gov/pubmed/27087653).

![N-Tails Galaxy Workflow](../../images/wf_ntails_tripleDimethylLabel.png "N-Tails Galaxy Workflow")

Notice that the given digestion enzyme is "ArgC", even if the proteins were digested using trypsin. Due to the used labelling method prior to digestion, lysine ("K") residues are dimethylated. Therefore, trypsin will not cut c-terminal of lysine, but only c-terminal of arginine in a N-TAILS experiment. This resembles the ArgC specificity and generally results in longer peptides [Rogers and Overall, MCP, 2013](http://www.mcponline.org/content/12/12/3532.full).

## Input

The workflow needs two input files:

  1) A collection of mzML files (multiple fractions of the same experiment).
  2) A FASTA protein database for the organism of interest. For more information on protein databases, consult [this tutorial]({{site.baseurl}}/topics/proteomics/tutorials/database-handling/tutorial.html)

## Customizing the Workflow

- *Running the workflow on a non-prefractionated sample:* Simply use only one file as an input.
- *Running the workflow on a double dimethyl labeling (only light and heavy labeling):* remove the third **MSGFPlusAdapter** {% icon tool %} and the following **PeptideIndexer** {% icon tool %} . Make sure that the mass changes are correctly given in the **MSGFPlusAdapter** {% icon tool %} .

## Citation

If you use this workflow directly, or any derivative of it, in work leading to a scientific publication,
please cite:

Lai, Z.W., Weisser, J., Nilse, L., Costa, F., Keller, E., Tholen, M., Kizhakkedathu, J.N., Biniossek, M., Bronsert, P., and Schilling, O. (2016). Formalin-Fixed, Paraffin-Embedded Tissues (FFPE) as a Robust Source for the Profiling of Native and Protease-Generated Protein Amino Termini. Mol. Cell. Proteomics 15, 2203–2213.


topic: proteomics
tutorial: protein-id-sg-ps
title: Peptide and Protein ID using SearchGUI and PeptideShaker
difficulty: introductory
time_estimation: 45m
tools_json: []
requirements_json: []
tags_json: ["[DDA, HeLa]"]
content_hash: bc1c4fe108d52e7ffcd24b230e569fff
last_modified: 2026-02-26T21:12:17.507619
zenodo_link: https://zenodo.org/record/546301
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/protein-id-sg-ps/tutorial.html

content:
Identifying the proteins contained in a sample is an important step in any proteomic experiment. However, in most experimental set ups, proteins are digested to peptides before the LC-MS/MS analysis. In this so-called "bottom-up" procedure, only peptide masses are measured. Therefore, protein identification cannot be performed directly from raw data, but is a multi-step process:

1. Raw data preparation
2. Peptide-to-Spectrum matching
3. Peptide inference
4. Protein inference

A plethora of software solutions exist for each step. In this tutorial, we will show how to
use the [ProteoWizard](http://proteowizard.sourceforge.net/) tool MSconvert and the [OpenMS](https://openms.de) tool [PeakPickerHiRes](http://ftp.mi.fu-berlin.de/pub/OpenMS/release-documentation/html/TOPP_PeakPickerHiRes.html) for step 1, and the [Compomics](https://compomics.com/) tools [SearchGUI](https://compomics.github.io/projects/searchgui.html) and [PeptideShaker](https://compomics.github.io/projects/peptide-shaker.html), for the steps 2-4.

For an alternative identification pipeline using only tools provided by the [OpenMS software suite](https://openms.de), please consult [this tutorial]({{site.baseurl}}/topics/proteomics/tutorials/protein-id-oms/tutorial.html).

# Input data


As an example dataset, we will use an LC-MS/MS analysis of HeLa cell lysate published
in [Vaudel et al., 2014, Proteomics](https://www.ncbi.nlm.nih.gov/pubmed/24678044). Detailed information
about the dataset can be found on [PRIDE](https://www.ebi.ac.uk/pride/archive/projects/PXD000674).
For step 2, we will use a validated human Uniprot FASTA database without appended decoy sequences.
If you already completed the tutorial on [Database Handling]({{site.baseurl}}/topics/proteomics/tutorials/database-handling/tutorial.html) you can use the constructed database priot to the **DecoyDatabase** {% icon tool %} step. You can find a prepared database, as well as the input proteomics data in different file formats on [Zenodo](https://zenodo.org/record/796184).

> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}


# Preparing Raw Data

Raw data conversion is the first step of any proteomic data analysis. The most common converter is msconvert from the [ProteoWizard software suite](http://proteowizard.sourceforge.net/), the format to convert to is mzML. SearchGUI needs `MGF` format as input, but as we need the `mzML` format for several other tasks, we will convert to `mzML` first. Due to licensing reasons, msconvert runs only on windows systems and will not work on most Galaxy servers.

Depending on your machine settings, raw data will be generated either in profile mode or centroid mode. For most peptide search engines, the tandem mass spectrometry (MS2) data have to be converted to centroid mode, a process called "peak picking" or "centroiding".
Machine vendors offer algorithms to extract peaks from profile raw data. This is implemented in ***msconvert*** {% icon tool %} and can be run in parallel to the mzML conversion. However, the OpenMS tool ***PeakPickerHiRes*** {% icon tool %} is reported to generate slightly better results ([Lange et al., 2006, Pac Symp Biocomput](https://www.ncbi.nlm.nih.gov/pubmed/17094243)) and is therefore recommended for quantitative studies ([Vaudel et al., 2010, Proteomics](https://www.ncbi.nlm.nih.gov/pubmed/19953549)).
If your data were generated on a low resolution mass spectrometer, use ***PeakPickerWavelet*** {% icon tool %} instead.

> <hands-on-title>File Conversion and Peak Picking</hands-on-title>
>
> We provide the [input data](https://zenodo.org/record/796184) in the original `raw` format and also already converted to `MGF` and `mzML` file formats. If ***msconvert*** {% icon tool %} does not run on your Galaxy instance, please download the preconverted `mzML` as an input.
>
> 1. Create a new history for this Peptide and Protein ID exercise.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Load the example dataset into your history from Zenodo: [raw](https://zenodo.org/record/892005/files/qExactive01819.raw) [mzML](https://zenodo.org/record/892005/files/qExactive01819_profile.mzml)
> 3. Rename the dataset to something meaningful.
> 4. (*optional*) Run ***msconvert*** {% icon tool %} on the test data to convert to the `mzML` format.
> 5. Run ***PeakPickerHiRes*** {% icon tool %} on the resulting file. Click `+ Insert param.algorithm_ms_levels` and change the entry to "2". Thus, peak picking will only be performed on MS2 level.
> 6. Run ***FileConverter*** {% icon tool %} on the picked mzML. In the **Advanced Options** set the **Output file type** to `MGF`.
>
>   > <comment-title>Local Use of MSConvert</comment-title>
>   > The vendor libraries used by msconvert are only licensed for Windows systems and are therefore rarely implemented in Galaxy instances. If ***msconvert*** {% icon tool %} is not available in your Galaxy instance, please install the software on a Windows computer and run the conversion locally. You can find [a detailed description of the necessary steps](https://compomics.com/bioinformatics-for-proteomics/identification/) ("Peak List Generation"). Afterwards, upload the resulting mzML file to your Galaxy history.
>  {: .comment}
{: .hands_on}

# Peptide and Protein Identification
Mass spectrometry experiments identify peptides by isolating them, ioinizing and subsequently colliding them with a gas for fragmentation. This method generates a spectrum of peptide fragment masses for each isolated peptide - an MS2 spectrum. To find out the peptide sequences, the MS2 spectrum is compared to a theoretical spectrum generated from a protein database. This step is called peptide-to-spectrum (also: spectrum-to-sequence) matching. Accordingly, a peptide that is successfully matched to a sequence is termed PSM (Peptide-Spectrum-Match). There can be multiple PSMs per peptide, if the peptide was fragmented several times. Different peptide search engines have been developed to fulfill the matching procedure.

It is generally recommended to use more than one peptide search engine and use the combined results for the final peptide inference ([Shteynberg et al., 2013, Mol. Cell. Proteomics](https://www.ncbi.nlm.nih.gov/pubmed/23720762)). Again, there are several software solutions for this, e.g. iProphet (TPP) or ConsensusID (OpenMS). In this tutorial we will use ***Search GUI*** {% icon tool %}, as it can automatically search the data using several search engines. Its partner tool ***Peptide Shaker*** {% icon tool %} is then used to combine and evaluate the search engine results.

In bottom-up proteomics, it is necessary to combine the identified peptides to proteins. This is not a trivial task, as proteins are redundant in most eukaryotic organisms. Thus, not every peptide can be assigned to only one protein. Luckily, the ***Peptide Shaker*** {% icon tool %} already takes care of protein inference and even gives us some information on validity of the protein identifications. We will discuss validation in a [later step](#evaluation-of-peptide-and-protein-ids) of this tutorial.

> <hands-on-title>Peptide and Protein Identification</hands-on-title>
>
> 1. Copy the prepared protein database from the tutorial [Database Handling](../database-handling/tutorial.html) into your current history by using the multiple history view or upload the [ready-made database from Zenodo](https://zenodo.org/record/892005/files/Human_database_%28cRAP_and_Mycoplasma_added%29.fasta).
> 2. Open ***Search GUI*** {% icon tool %} to search the mgf file against the protein database. In the **`Search Engine Options`** select `X!Tandem` and `MS-GF+`. In the **`Protein Modification Options`** add the **`Fixed Modifications`**: `Carbamidomethylation of C` and the **`Variable Modifications`**: `Oxidation of M`.
> 3. Run ***Peptide Shaker*** {% icon tool %} on the Search GUI output. Enable the following outputs: `Zip File for import to Desktop App`, `mzidentML File`, `PSM Report`, `Peptide Report`, `Protein Report`.
>
>   > <comment-title>Search GUI Parameters</comment-title>
>   > We ran ***Search GUI*** {% icon tool %} with default settings. When you are processing files of a different experiment, you may need to adjust some of the parameters.
>   > **Search GUI** bundles numerous peptide search engines for matching MS/MS to peptide sequences within a database. In practice, using 2-3 different search engines offers high confidence while keeping analysis time reasonable. In our hands, X! tandem, MS-GF+, OMSSA and Comet search algorithms offer good results.
>   > The **`Precursor Options`** have to be adjusted to the mass spectrometer which was used to generate the files. The default settings fit a  high resolution Orbitrap instrument.
>   > In the **`Advanced Options`** you may set much more detailed settings for each of the used search engines. When using X!Tandem, we recommend to switch off the advanced X!Tandem options **`Noise suppression`**, **`Quick Pyrolidone`** and **`Quick Acetyl`**. When using MSGF, we recommend to select the correct **`Instrument type`**.
>   {: .comment}
>
>   > <comment-title>PeptideShaker Outputs</comment-title>
>   > Peptide Shaker offers a variety of outputs.
>   > The `Zip File for import to Desktop App` can be downloaded to view and evaluate the search results in the Peptide Shaker viewer ([Download](https://compomics.github.io/projects/peptide-shaker.html)).
>   > The several `Reports` contain tabular, human-readable information.
>   > Also, an `mzidentML` (= `mzid`) file can be created that contains all peptide sequence matching information and can be utilized by compatible downstream software.
>   > The `Certificate of Analysis` provides details on all parameters settings of both Search GUI and Peptide Shaker used for the analysis.
>   {: .comment}
>
>   > <question-title></question-title>
>   > 1. How many peptides were identified? How many proteins?
>   > 2. How many peptides with oxidized methionine were identified?
>   >
>   >  > <solution-title></solution-title>
>   >  > 1. You should have identified 3,325 peptides and 1,170 proteins.
>   >  > 2. 328 peptides contain an oxidized methionine (MeO). To get to this number, you can use ***Select*** {% icon tool %} on the Peptide Report and search for either "Oxidation of M" or "M\<ox\>".
>   >  {: .solution }
>   {: .question}
{: .hands_on}

# Analysis of Contaminants
The FASTA database used for the peptide to spectrum matching contained some entries that were not expected to stem from the HeLa cell lysate, but are common contaminations in LC-MS/MS samples. The main reason to add those is to avoid misidentification of the spectra to other proteins. However, it also enables you to check for contaminations in your samples. **CAVE:** in human samples, many proteins that are common contaminants may also stem from the real sample. The real source of such human proteins might require advanced investigation.

> <hands-on-title>Analysis of Contaminants</hands-on-title>
>
> 1. Run ***Select*** {% icon tool %} on the Peptide Shaker Protein Report to select all lines that match the pattern "CONTAMINANT".
> 2. Remove all contaminants from your protein list by running ***Select*** {% icon tool %} on the Peptide Shaker Protein Report. Select only those lines that **do not** match the pattern "CONTAMINANT".
>
>   > <question-title></question-title>
>   > 1. Which contaminants did you identify? Where do these contaminations come from?
>   > 2. What other sources of contaminants exist?
>   > 3. How many mycoplasma proteins did you identify? Does this mean that the analyzed HeLa cells were infected with mycoplasma?
>   > 4. How many false positives do we expect in our list? How many of these are expected to match mycoplasma proteins?
>   >
>   >  > <solution-title></solution-title>
>   >  > 1. TRY_BOVIN is bovine trypsin. It was used to degrade the proteins to peptides. ALBU_BOVIN is bovine serum albumin. It is added to cell culture medium in high amounts.
>   >  > 2. Contaminants often stem from the experimenter, these are typically keratins or other high-abundant human proteins. Basically any protein present in the room of the mass spectrometer might get into the ion source, if it is airborne. As an example, sheep keratins are sometimes found in proteomic samples, stemming from clothing made of sheep wool.
>   >  > 3. There should be five _Mycoplasma_ proteins in your protein list. However, all of them stem from different _Mycoplasma_ species. Also, every protein was identified by one peptide only. You can see this in column 17-19 of your output. These observations make it quite likely that we might have identified false positives here.
>   >  > 4. As we were allowing for a false discovery rate of 1 %, we would expect 12 false positive proteins in our list.
>   >  >    False positives are expected to be randomly assigned to peptides in the FASTA database. Our database consists of about 20,000 human proteins and 4,000 mycoplasma proteins. Therefore, we would expect 17 % (= 2) of all false positives matching to mycoplasma proteins.
>   >  {: .solution }
>   {: .question}
{: .hands_on}


# Evaluation of Peptide and Protein IDs
***Peptide Shaker*** {% icon tool %} provides you with validation results for the identified PSM, peptides and proteins. It classifies all these IDs in the categories "Confident" or "Doubtful". On each level, the meaning of these terms differs to some extent:

- **PSMs** are marked as "Doubtful" when the measured MS2 spectrum did not fit well to the theoretical spectrum.
- **Peptides** have a combined scoring of their PSMs. They are marked as "Doubtful", when the score is below a set threshold. The threshold is defined by the false discovery rate (FDR).
- **Proteins** are marked as "Doubtful", when they were identified by only a single peptide or when they were identified solely by "Doubtful" peptides.

> <hands-on-title>Evaluation of Peptide and Protein IDs</hands-on-title>
>
> 1. Remove all "Doubtful" proteins from your protein list by running ***Select*** {% icon tool %} on the Peptide Shaker Protein Report. Select only those lines that **do not** match the pattern "Doubtful".
>
>   > <question-title></question-title>
>   > 1. How to exclude mycoplasma proteins?
>   > 2. How many "Confident" non-contaminant proteins were identified?
>   >
>   >  > <solution-title></solution-title>
>   >  > 1. Add another ***Select*** {% icon tool %} matching the pattern "HUMAN".
>   >  > 2. You should have identified 582 human non-contaminant proteins that were validated to be "Confident".
>   >  {: .solution }
>   {: .question}
{: .hands_on}

# Premade Workflow


[A premade workflow for this tutorial is available](workflows/wf_proteinID_SG_PS.ga)

# Further Reading


- [Search GUI and Peptide Shaker tutorials at Compomics](https://compomics.com/bioinformatics-for-proteomics/)
- [Using Search GUI and Peptide Shaker in Galaxy](https://drive.google.com/file/d/0B6bIeOvjBkbWVnhMLWxXdGVUY3M/view)


topic: proteomics
tutorial: multiGSEA-tutorial
title: Multiomics data analysis using MultiGSEA
difficulty: intermediate
time_estimation: 1H
tools_json: []
requirements_json: []
tags_json: ["multi-omics", "transcriptomics", "proteomics", "metabolomics"]
content_hash: 1369c234ea00c35a1ce1de884515ebe4
last_modified: 2026-02-26T21:12:17.499604
zenodo_link: https://zenodo.org/records/14216972
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/multiGSEA-tutorial/tutorial.html

content:
The multiGSEA package was designed to run a robust GSEA-based pathway enrichment for multiple omics layers (Canzler and Hackermüller, 2020) {% cite Canzler2020 %}. The enrichment is calculated for each omics layer separately and aggregated p-values are calculated afterwards to derive a composite multi-omics pathway enrichment.

Pathway definitions can be downloaded from up to eight different pathway databases by means of the graphite Bioconductor package (Sales, Calura, and Romualdi 2018). Feature mapping for transcripts and proteins is supported towards Entrez Gene IDs, Uniprot, Gene Symbol, RefSeq, and Ensembl IDs. The mapping is accomplished through the AnnotationDbi package (Pagès et al. 2019) and currently supported for 11 different model organisms including human, mouse, and rat. ID conversion of metabolite features to Comptox Dashboard IDs (DTXCID, DTXSID), CAS-numbers, Pubchem IDs (CID), HMDB, KEGG, ChEBI, Drugbank IDs, or common metabolite names is accomplished through the AnnotationHub package metabliteIDmapping. This package provides a comprehensive ID mapping for more than 1.1 million entries.

This tutorial covers a simple example workflow illustrating how the multiGSEA package works. The omics data sets that will be used throughout the example were originally provided by {% cite quiros2017multi %}. In their publication the authors analyzed the mitochondrial response to four different toxicants, including Actinonin, Diclofenac, FCCB, and Mito-Block (MB), within the transcriptome, proteome, and metabolome layer.
In this tutorial we will solely focus on the Actinonin data set.

For more background information you can read following articles: 

- Multiomics analysis by Quiros {% cite quiros2017multi %}

- Methods for combining p-values by Loughin {% cite LOUGHIN2004467 %}

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Preparing the Data

To perform pathway enrichment with MultiGSEA, you'll need omics datasets in the file type TSV . Each individual data set contains four columns representing the feature (denoted as Symbol), the log2 fold change (logFC), the p-value (pValue), and the adjusted p-values (adj.pValue). We'll use example data provided on Zenodo.

## Get data

### Data Upload

> <hands-on-title> Getting datasets </hands-on-title>
> 1. Create a new history for this tutorial.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Import the datasets from [Zenodo]({{ page.zenodo_link }})  into your Galaxy instance:
>    ```
>    https://zenodo.org/records/14216972/files/transcriptome.tsv
>    https://zenodo.org/records/14216972/files/proteome.tsv
>    https://zenodo.org/records/14216972/files/metabolome.tsv
>    ```
{: .hands_on}


# Running MultiGSEA

In this step, you'll use the MultiGSEA tool to perform GSEA-based pathway enrichment on the uploaded datasets.

> <hands-on-title> Task description </hands-on-title>
>
> 1. Run {% tool [multiGSEA](toolshed.g2.bx.psu.edu/repos/iuc/multigsea/multigsea/1.12.0+galaxy0) %} with the following parameters
>    - *"Select transcriptomics data"*: `Enabled`
>        - {% icon param-file %} *"Transcriptomics data"*: `Transcriptomics`
>        - {% icon param-select %} *"Gene ID format in transcriptomics data"*: `SYMBOL`
>    - *"Select proteomics data"*: `Enabled`
>        - {% icon param-file %} *"Proteomics data"*: `Proteomics`
>        - {% icon param-select %} *"Gene ID format in proteomics data"*: `SYMBOL`
>    - *"Select metabolomics data"*: `Enabled`
>        - {% icon param-file %} *"Metabolomics data"*: `Metabolomics`
>        - {% icon param-select %} *"Metabolite ID format"*: `HMDB`
>    - *"Supported organisms"*: `Homo sapiens (Human)`.
>    - *"Pathway databases"*: `Kegg`, `Reactome`
>    - *"Combine p-values method"*: `Stouffer`
>    - *"P-value correction method"*: `BH`
>
>    > <tip-title>About the parameters</tip-title>
>    > - **Pathway databases**: Databases often contain their own format in which pathway definitions are provided. So you can select a relevant database. For the tutorial we choose the preset "Kegg" and "Reactome".
>    > - **Combine p-values method**: Choose a method (here `Stouffer` for balanced weighting). To more comprehensively measure a pathway response, multiGSEA provides different approaches to compute an aggregated p value over multiple omics layers. Because no single approach for aggregating p values performs best under all circumstances, {% cite LOUGHIN2004467 %} proposed basic recommendations on which method to use depending on structure and expectation of the problem. If small p values should be emphasized, Fisher’s method should be chosen. In cases where p values should be treated equally, Stouffer’s method is preferable. If large p values should be emphasized, the user should select Edgington’s method. Figure 2 indicates the difference between those three methods.  
>    >   ![P-Value](../../images/p-value.png "P-value methods")
>    > - **P-value correction method** Type I and type II errors depend on each other and thus reducing type I errors through a p value adjustment will likely increase the chance of making a type II error and an appropriate trade-off has to be made. Choose one of the different methods for controlling false discovery rate: For the tutorial choose `BH` (Benjamini-Hochberg).
>    {: .tip}
>
{: .hands_on}

# Mitochondrial stress activates metabolic pathways of amino acid biosynthesis.

After we performed pathway enrichment on our data we want to continue our analysis by filtering the outputs, e.g. by p-value.

><hands-on-title> Filtering by values</hands-on-title>
>
> We are going to use the tool {% tool [Filter data on any column using simple expressions](Filter1) %} with the following parameters:
> - {% icon param-file %} *"Filter"*: Select the output dataset of the multiGSEA tool.
> - *"With following condition"*: `c9<=0.01`
>
{: .hands_on}


><hands-on-title> Filtering by keyword</hands-on-title>
>
>Mitochondrial stress triggers the activation of amino acid biosynthesis and related metabolic pathways, as highlighted by Quiros et al. (Ref findest du in der Vignette), who identified up-regulation of several amino acid related pathways.
>These findings align with our results using a multi-omics approach with multiGSEA
>Our results also reveal the enrichment of amino acid-related pathways. This can be see, e.g., by searching for pathways containing the word "amino" in the name. We can search for this using the regular expression `"\bamino\b"` (`\b` is a special character marking a word border).
>
>
>For further filtering we are going to use the tool {% tool [Search in textfiles](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_grep_tool/9.3+galaxy1) %} with the following parameters:
> - {% icon param-file %} *"Selectlines from"*: Select the output of the last filter.
> - *"that"*: Choose "Match"
> - *"Type of regex*": Choose "Basic (-G)"
> - *"Regular Expression"*: \bamino\b
> - *"Match type"*: "case insensitive"
> - *"Show lines preceding the matched line"*: 0
> - *"Show lines trailing the matched line"*: 0
> - "*Output*": "text file (for further processing)"
>
>This should produce a file with the following content:
>```
>(KEGG) Amino sugar and nucleotide sugar metabolism	0.258278145695364	0.571272307893517	8.40563134115856e-07	4.03302191748788e-05	2.34233775080415e-05	0.00310502061987873	8.75854833022965e-05	0.00347274191804089
>(KEGG) Biosynthesis of amino acids	3.7749282501799e-05	0.00138827212066317	1.53729180439681e-10	2.45864202583197e-08	0.0794621026894866	0.333364038319823	1.48946778740876e-07	1.13741176493032e-05
>(REACTOME) Amino acid and derivative metabolism	4.7058253543142e-06	0.000231903073460604	7.70587893652191e-22	1.84864035687161e-18	0.055363321799308	0.286607081313329	9.03772696553135e-14	2.2775071953139e-11
>(REACTOME) Response of EIF2AK4 (GCN2) to amino acid deficiency	6.42075812177139e-07	4.52021371772706e-05	3.76589836665495e-10	5.31434716565013e-08	NA	NA	3.35403180955872e-11	4.83629358772636e-09
>
>```
>
{: .hands_on}

> <question-title></question-title>
>
> 1. What file format is required for the input data in MultiGSEA?
> 2. What is the purpose of the “Combine p-values method” parameter, and which method was selected in this tutorial?
> 3. Why is it important to select pathway databases (e.g., KEGG) when using MultiGSEA?
>
> > <solution-title></solution-title>
> >
> > 1. The required file format is TSV.
> > 2. The “Combine p-values method” parameter is used to aggregate p-values across omics layers. In this tutorial, the method Stouffer was selected to apply balanced weighting.
> > 3. Selecting pathway databases ensures that the analysis uses appropriate and relevant pathway definitions for enrichment.
> >
> {: .solution}
>
{: .question}


# Conclusion

In this tutorial, you explored the capabilities of MultiGSEA for performing pathway enrichment analysis across multiple omics layers, including transcriptomics, proteomics, and metabolomics data. By following the steps, you learned how to:

 - Prepare and upload the required omics datasets.
 - Configure and execute the MultiGSEA tool within Galaxy.
 -  Combine p-values from different omics layers to derive a unified perspective on pathway enrichment.


topic: proteomics
tutorial: neoantigen-peptide-verification
title: Neoantigen 3: PepQuery2 Verification
difficulty: intermediate
time_estimation: 3H
tools_json: []
requirements_json: []
tags_json: ["[label-free]"]
content_hash: 20011f7e7e14ff951eb611f67e7b6531
last_modified: 2026-02-26T21:12:17.503320
zenodo_link: https://zenodo.org/records/15090517
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/neoantigen-peptide-verification/tutorial.html

content:
Verification is an essential step in the process of analyzing neoantigens to ensure that the identified peptides or proteins are accurate and reliable. Neoantigens, which are novel antigens arising from tumor-specific mutations, are crucial for cancer immunotherapy. However, the prediction and identification of these neoantigens must be rigorously validated to ensure they are truly present and capable of eliciting an immune response. Without proper verification, the therapeutic potential of neoantigens cannot be realized, leading to wasted resources and failed treatments.

In this tutorial, we will guide you through a series of bioinformatics tools used to verify novel peptides and proteins predicted from mass spectrometry data. These tools help confirm the presence and specificity of the identified neoantigens by comparing them to protein databases, checking for validation against spectral data, and performing sequence analysis.

![Validation Overview workflow]({% link topics/proteomics/images/neoantigen/PepQuery2_Verification_1.PNG %})


> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

## PepQuery2 Verification Workflow

This workflow involves several key steps to validate neoantigens identified through mass spectrometry data. Each step leverages bioinformatics tools to ensure that the predicted peptides or proteins are accurate and can be used for further therapeutic research. Below are the detailed steps of this workflow:

1. **Get Data** The first step in the workflow involves gathering and uploading the necessary mass spectrometry (MS) data files. These files, which include raw spectra or peptide data, are essential for identifying potential neoantigens. Proper data organization and renaming ensure that the workflow executes smoothly.
2. **Preprocessing Raw MS Data with msconvert** Next, the raw MS data files are preprocessed using msconvert to convert them into a suitable format for analysis, specifically the Mascot Generic Format (MGF). This step is critical for transforming the data into a structured format that can be analyzed using peptide validation tools.
3. **Peptide Validation with PepQuery2** Once the data is preprocessed, PepQuery2 is employed to validate the peptides. This tool compares the identified peptides against a protein reference database, using MS/MS data to confirm their authenticity. It ensures that the identified peptides are truly valid and not false positives, which is essential for identifying reliable neoantigens.
4. **Filtering Tabular Data** After peptide validation, Query Tabular is used to filter the results. This step narrows down the data by focusing on specific criteria such as peptide match status and spectrum quality, ensuring only the most relevant peptides are retained for further analysis.
5. **Converting Data to FASTA Format** The validated and filtered peptides are then converted into a FASTA format using the Tabular-to-FASTA tool. This format is commonly used for sequence analysis and provides a standardized structure for further processing of the data.
6. **BLAST-P Sequence Alignment** In the next step, the peptides are subjected to sequence alignment using NCBI BLAST+ blastp against a protein database. This step helps confirm whether the peptides are novel or match known proteins, adding an extra layer of validation to ensure their potential as therapeutic neoantigens.
7. **Refining Data with Query Tabular** Following the BLAST alignment, Query Tabular is used again to filter out peptides that do not meet the criteria or are not novel. This step helps ensure that only the most promising neoantigen candidates remain for further investigation.
8. **Summary of Results** The final step involves summarizing the results, which include the validated and refined peptides identified as potential neoantigens. These findings are organized and presented for interpretation, providing valuable insights into the therapeutic potential of the identified neoantigens for immunotherapy.


## Get data

> <hands-on-title> Data Upload </hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo]({{ page.zenodo_link }}) or from
>    the shared data library (`GTN - Material` -> `{{ page.topic_name }}`
>     -> `{{ page.title }}`):
>
>    ```
>    https://zenodo.org/records/14375467/files/Human-TaxID.txt
>    https://zenodo.org/records/14375467/files/HUMAN_CRAP.fasta
>    https://zenodo.org/records/14375467/files/NeoAntigen-Candidates.tabular
>    https://zenodo.org/records/14375467/files/STS_26T_2_Eclipse_02102024.raw
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
> 3. Rename the datasets
> 4. Check that the datatype
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
>
> 5. Add to each database a tag corresponding to ...
>
>    {% snippet faqs/galaxy/datasets_add_tag.md %}
>
{: .hands_on}


# Performing validation of peptides using PepQuery

![Validation-with-PepQuery]({% link topics/proteomics/images/neoantigen/PepQuery2_Verification_2.PNG %})

## Preprocessing Raw MS Data with msconvert
msconvert is a tool used to preprocess raw mass spectrometry (MS) data files by converting them into a compatible format for downstream analysis. In this workflow, msconvert is employed to convert the input unrefined MS data into Mascot Generic Format (MGF), a standardized format for storing peptide and protein data. Additionally, during this step, peak picking is applied to identify the most relevant peaks from the raw spectra, which improves the quality of the data. However, the charge state recalculation is disabled as it's not necessary for the subsequent analysis. By converting the raw data into MGF format, msconvert prepares the dataset for further processing in tools like PepQuery2 and BLAST, ensuring that the data is in the right format for peptide identification and validation.

> <hands-on-title> msconvert</hands-on-title>
>
> 1. {% tool [msconvert](toolshed.g2.bx.psu.edu/repos/galaxyp/msconvert/msconvert/3.0.20287.2) %} with the following parameters:
>    - {% icon param-file %} *"Input unrefined MS data"*: `STS_26T_2_Eclipse_02102024.raw` (Input dataset)
>    - *"Do you agree to the vendor licenses?"*: `Yes`
>    - *"Output Type"*: `mgf`
>    - In *"Data Processing Filters"*:
>        - *"Apply peak picking?"*: `Yes`
>        - *"(Re-)calculate charge states?"*: `no`
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. What does "peak picking" mean in the context of msconvert?
> 2. Why is the charge state recalculation turned off in msconvert?
>
> > <solution-title></solution-title>
> >
> > 1. Peak picking refers to the process of identifying the most significant peaks in the mass spectrometry data, which correspond to the ions of interest. This step is important because it helps filter out noise and retain only the relevant peaks that will be used for peptide identification.
> > 2. The charge state recalculation is turned off because the data being processed is already assumed to have accurate charge assignments from the initial raw spectra. Recalculating the charge states could lead to unnecessary complications or errors in the context of this workflow, which focuses on peptide identification rather than charge state assignment.
> >
> {: .solution}
>
{: .question}

## Peptide Validation with PepQuery2

PepQuery2 is a tool used to validate novel peptides and proteins by searching mass spectrometry data against a reference protein database. In this workflow, PepQuery2 is used for peptide validation, specifically focusing on the detection of novel peptides or proteins. The input data includes a list of peptides (obtained from a previous step) and a protein reference database, which is also provided from the workflow history. The spectrum file, generated by msconvert, is used to search for peptide-spectrum matches (PSMs). The tool is configured with specific parameters, such as peptide charge and length, and the fragmentation method (CID/HCD). The output will help identify novel peptides that may not be present in the reference database, aiding in the discovery of potential new biomarkers or proteins.



> <hands-on-title> PepQuery2 </hands-on-title>
>
> 1. {% tool [PepQuery2](toolshed.g2.bx.psu.edu/repos/galaxyp/pepquery2/pepquery2/2.0.2+galaxy2) %} with the following parameters:
>    - *"Validation Task Type"*: `novel peptide/protein validation`
>    - In *"Input Data"*:
>        - *"Input Type"*: `peptide`
>            - *"Peptides?"*: `Peptide list from your history`
>                - {% icon param-file %} *"Peptide Sequences (.txt)"*: `NeoAntigen-Candidates.tabular` (Input dataset)
>        - *"Protein Reference Database from"*: `history`
>            - {% icon param-file %} *"Protein Reference Database File"*: `HUMAN_CRAP.fasta` (Input dataset)
>        - *"MS/MS dataset to search"*: ` Spectrum Datasets from history`
>            - {% icon param-file %} *"Spectrum File"*: `output` (output of **msconvert** {% icon tool %})
>        - *"Report Spectrum Scan as"*: `spectrum title in MGF`
>    - In *"Modifications"*:
>        - *"Fixed modification(s)"*: `0: NO Modifications`
>        - *"Variable modification(s)"*: `2: Oxidation of M [15.99491461956]`
>    - In *"Digestion"*:
>        - *"Enzyme"*: `Non enzyme`
>    - In *"Mass spectrometer"*:
>        - In *"Tolerance"*:
>            - *"Precursor Unit"*: `ppm`
>        - In *"PSM"*:
>            - *"Fragmentation Method"*: `CID/HCD`
>            - *"Minimum Charge"*: `1`
>            - *"Maximum Charge"*: `3`
>            - *"Minimum length of peptide"*: `9`
>            - *"Maximum length of peptide"*: `11`
>    - *"Use fast mode for searching"*: `Yes`
>
>
{: .hands_on}



> <question-title></question-title>
>
> 1. Why is "Non enzyme" selected as the digestion method in PepQuery2?
> 2. What is the significance of selecting "CID/HCD" as the fragmentation method?
>
> > <solution-title></solution-title>
> >
> > 1. "Non enzyme" is selected because, in this workflow, the peptides are already prepared and may not require digestion by a specific enzyme. This option allows for the validation of the peptides without applying any enzyme-specific cleavage rules.
> > 2. CID (Collision-Induced Dissociation) and HCD (Higher Energy Collisional Dissociation) are fragmentation techniques used to break peptides into smaller ions for analysis. These methods are commonly employed to improve the identification of peptides in mass spectrometry by generating clearer and more interpretable spectra, which is essential for accurate peptide validation.
> >
> {: .solution}
>
{: .question}

## Filtering Tabular Data with **Query Tabular**

Query Tabular is a tool used to query tabular datasets using SQL-like commands. In this step, the tool is used to filter and extract specific data from the results generated by PepQuery2 (specifically the psm_rank_txt dataset). The query provided selects specific columns from the table where a condition is met. This allows the user to filter the peptide-spectrum matches (PSMs) based on certain criteria, such as identifying PSMs that were validated with a "Yes" condition. The tool helps organize the data into a more manageable and relevant format for further analysis.

> <hands-on-title> Query tabular </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `psm_rank_txt` (output of **PepQuery2** {% icon tool %})
>    - *"SQL Query to generate tabular output"*:
> ``` sql
> SELECT c1,c4
> FROM t1
> WHERE (c20 = 'Yes')
> ```
>    - *"include query result column headers"*: `No`
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why is the option "include query result column headers" set to 'No'?
> 2. What does the SQL query SELECT c1, c4 FROM t1 WHERE (c20 = 'Yes') do?
>
> > <solution-title></solution-title>
> >
> > 1. The option is set to 'No' because, in this case, the user may not need column headers in the output file. This can be useful when the output is being further processed or integrated into another system where the headers are not required.
> > 2. This SQL query selects specific columns (c1 and c4) from the table t1 where the value in column c20 is equal to 'Yes'. Essentially, it filters the dataset to retrieve only the rows where a certain condition (e.g., peptide validation) has been met.
> >
> {: .solution}
>
{: .question}

# Extracting Novel Neoantigen Peptides after BLAST-P

![Novel-Peptides]({% link topics/proteomics/images/neoantigen/PepQuery2_Verification_3.PNG %})

## Converting Data to FASTA Format with **Tabular-to-FASTA**
Tabular-to-FASTA is a tool used to convert tabular data into the FASTA format, commonly used for storing sequence data. In this step, the tool takes the output of the Query Tabular step, which contains the filtered peptide data, and converts it into a FASTA format. The parameters specify which columns in the tabular data correspond to the sequence (c1) and the title (c['2']). The output will be a FASTA file where each peptide sequence is accompanied by its associated title or identifier. This conversion is crucial for further analysis, such as alignment or database searches, using sequence data in FASTA format. This database is being generated for BLAST-P searching.

> <hands-on-title> Tabular to FASTA </hands-on-title>
>
> 1. {% tool [Tabular-to-FASTA](toolshed.g2.bx.psu.edu/repos/devteam/tabular_to_fasta/tab2fasta/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Tab-delimited file"*: `output` (output of **Query Tabular** {% icon tool %})
>    - *"Title column(s)"*: `2`
>    - *"Sequence column"*: `1`
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why is there a need to convert the files to FASTA for BLASTP searches?
>
> > <solution-title></solution-title>
> >
> > 1. We need to convert data to FASTA format for BLASTP because it is the required input format for the tool. FASTA provides a standardized way to represent protein sequences with a header and sequence lines, making it compatible with BLASTP’s alignment algorithms. This format allows BLASTP to properly parse, compare, and search the query sequence against protein databases.
> >
> {: .solution}
>
{: .question}

## BLAST-P Sequence Alignment
In this step, the NCBI BLAST+ blastp tool is used for performing protein sequence alignment. It compares the query protein sequence (generated from the Tabular-to-FASTA tool) against a locally installed BLAST database of protein sequences. The alignment is optimized for shorter protein queries (less than 30 residues) using the blastp-short option. The tool uses a PAM30 scoring matrix, along with specific gap costs, and generates tabular output with extended 25 columns, allowing users to examine the results in detail.


- **Parameters:**
  - **Protein query sequence(s):** The input protein sequence in FASTA format.
  - **Subject database/sequences:** The BLAST database against which the query sequence will be compared.
  - **Type of BLAST:** Optimized for short protein queries.
  - **Expectation value cutoff:** Specifies the cutoff for statistical significance.
  - **Advanced options:** Includes scoring matrix (PAM30), gap costs, and word size for the BLAST algorithm.

> <hands-on-title> NCBI BLAST+ blastp </hands-on-title>
>
> 1. {% tool [NCBI BLAST+ blastp](toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastp_wrapper/2.14.1+galaxy2) %} with the following parameters:
>    - {% icon param-file %} *"Protein query sequence(s)"*: `output` (output of **Tabular-to-FASTA** {% icon tool %})
>    - *"Subject database/sequences"*: `Locally installed BLAST database`
>        - *"Protein BLAST database"*: `NCBI NR (03 Sep 2023)`
>    - *"Type of BLAST"*: `blastp-short - BLASTP optimized for queries shorter than 30 residues`
>    - *"Set expectation value cutoff"*: `200000.0`
>    - *"Output format"*: `Tabular (extended 25 columns)`
>    - *"Advanced Options"*: `Show Advanced Options`
>        - *"Scoring matrix and gap costs"*: `PAM30`
>            - *"Gap Costs"*: `Existence: 9  Extension: 1`
>        - *"Maximum number of HSPs (alignments) to keep for any single query-subject pair"*: `1`
>        - *"Word size for wordfinder algorithm"*: `2`
>        - *"Multiple hits window size: use 0 to specify 1-hit algorithm, leave blank for default"*: `15`
>        - *"Minimum score to add a word to the BLAST lookup table."*: `16`
>        - *"Composition-based statistics"*: `0: No composition-based statistics`
>        - *"Restrict search of database to a given set of ID's"*: `Taxonomy identifiers (TaxId's)`
>            - {% icon param-file %} *"Restrict search of database to list of TaxId's"*: `Human-TaxID.txt` (Input dataset)
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. What is the significance of using the blastp-short option in this tool?
> 2. Why is the PAM30 matrix used in this step?
>
> > <solution-title></solution-title>
> >
> > 1. The blastp-short option is used because the query protein sequence is shorter than 30 residues, and this setting optimizes the alignment for such sequences.
> > 2. The PAM30 matrix is a substitution matrix suitable for shorter protein sequences and is specifically designed for those with fewer than 50 amino acids.
> >
> {: .solution}
>
{: .question}

## Refining Data with **Query Tabular**
The Query Tabular tool is used in this workflow to filter and organize data from previous steps. In this case, it processes the output from the NCBI BLAST+ blastp tool, which contains protein sequence alignment results. The tool allows the user to execute SQL-like queries on the tabular data, enabling filtering and sorting based on specific criteria. For example, the query selects distinct peptides from the pep table that are not perfectly aligned (i.e., with a sequence identity of 100%) and have certain mismatches or gaps in the alignment. The filtered results are then used for further analysis.

The tool helps to refine the data by removing sequences that meet specific criteria, such as perfect alignment, and ensuring that the remaining sequences meet the necessary quality standards for further exploration. The output of this tool is the novel peptides identified from the datasets.

> <hands-on-title> Query Tabular</hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `output1` (output of **NCBI BLAST+ blastp** {% icon tool %})
>            - In *"Table Options"*:
>                - *"Specify Name for Table"*: `blast`
>                - *"Specify Column Names (comma-separated list)"*: `qseqid,sseqid,pident,length,mismatch,gapopen,qstart,qend,sstart,send,evalue,bitscore,sallseqid,score,nident,positive,gaps,ppos,qframe,sframe,qseq,sseq,qlen,slen,salltitles`
>                - In *"Table Index"*:
>                    - {% icon param-repeat %} *"Insert Table Index"*
>                        - *"Index on Columns"*: `qseqid`
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `output` (output of **Query Tabular** {% icon tool %})
>            - In *"Table Options"*:
>                - *"Specify Name for Table"*: `pep`
>                - *"Specify Column Names (comma-separated list)"*: `pep,seq`
>                - In *"Table Index"*:
>                    - {% icon param-repeat %} *"Insert Table Index"*
>                        - *"Index on Columns"*: `pep`
>    - *"SQL Query to generate tabular output"*:
> ``` sql
> SELECT DISTINCT pep.*
> FROM pep
> JOIN blast ON pep.pep = blast.qseq
> WHERE pep.pep NOT IN
> (SELECT qseq
> FROM blast
> WHERE pident = 100)
> AND (blast.pident < 100
> OR blast.gapopen >= 1
> OR blast.length < blast.qlen)
> ORDER BY pep.pep
> ```
>    - *"include query result column headers"*: `No`
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. Why is the query filtering out sequences with 100% identity from the BLAST results?
> 2. What is the significance of using SQL-like queries in Query Tabular?
>
> > <solution-title></solution-title>
> >
> > 1. Sequences with 100% identity are filtered out to focus on those that show variability in the alignment, which might indicate novel or significant biological differences worth exploring.
> > 2. SQL-like queries allow for efficient data manipulation and filtering, enabling users to extract only the most relevant results based on complex conditions (such as mismatches, gaps, or alignment length).
> >
> {: .solution}
>
{: .question}


# Conclusion

This workflow effectively integrates several bioinformatics tools to process, analyze, and refine protein sequence data. Starting from raw MS data, through peptide identification and database querying, to sequence alignment and filtering, each tool serves a specific purpose in ensuring that only the most relevant sequences are selected for further analysis. By utilizing Query Tabular in particular, users can apply custom SQL-like queries to filter out sequences based on specific criteria, which helps focus on the most promising data. The final results in this workflow are the neoantigen peptides which will be then used for further interpretation and exploration, enabling researchers to gain valuable insights into protein sequences with high confidence and precision. This approach highlights the power of combining multiple tools and techniques in a streamlined bioinformatics workflow to tackle complex data analysis challenges.

# Rerunning on your own data

To rerun this entire analysis at once, you can use our workflow. Below we show how to do this:


> <hands-on-title>Running the Workflow</hands-on-title>
>
>  1. **Import the workflow** into Galaxy:
>     - [Neoantigen Peptide Verification](https://tinyurl.com/ipepgen-pepquery-wf)
> 
> 3. Run **Workflow** {% icon workflow %} using the following parameters:
>     - *"Send results to a new history"*: `No`
>     - {% icon param-file %} *"Candidate_Neoantigens"*: `NeoAntigen-Candidates.tabular`
>     - {% icon param-file %} *"Human Uniprot (with isoforms) and CRAP Database"*: `HUMAN_CRAP.fasta`
>     - {% icon param-file %} *"Input raw file(s)"*: `STS_26T_2_Eclipse_02102024.raw`
>     - {% icon param-file %} *"Human Taxonomy ID"*: `Human-TaxID.txt`
>
>    {% snippet faqs/galaxy/workflows_run.md %}
>    
> <comment-title>DISCLAIMER</comment-title>
>
> - If any step in this workflow fails, please ensure that the input files have been correctly generated and formatted by the preceding tools. Workflow failures often result from improperly called or incomplete input data rather than errors in the workflow itself. Users are responsible for verifying their input before troubleshooting workflow issues.
>
> {: .comment}
>
>
{: .hands_on}

# Are you feeling adventurous? ✨

## One-Click Neoantigen Workflow

This new [One-Click Neoantigen Workflow](https://usegalaxy.eu/u/galaxyp/w/ipepgen-one-click-workflow) brings together all key modules of the neoantigen discovery process into a single, streamlined analysis within Galaxy. Instead of launching each tutorial separately, users can now execute the entire end-to-end pipeline—from database creation to HLA binding prediction—with just one click —without ever leaving Galaxy.

# Disclaimer

Please note that all the software tools used in this workflow are subject to version updates and changes. As a result, the parameters, functionalities, and outcomes may differ with each new version. Additionally, if the protein sequences are downloaded at different times, the number of sequences may also vary due to updates in the reference databases or tool modifications. We recommend the users to verify the specific versions of software tools used to ensure the reproducibility and accuracy of results.


topic: proteomics
tutorial: clinical-mp-2-discovery
title: Clinical Metaproteomics 2: Discovery
difficulty: intermediate
time_estimation: 3H
tools_json: []
requirements_json: []
tags_json: ["[label-TMT11]"]
content_hash: 6dfe42e96d7b7bfa9a0f8ec8e07c93a9
last_modified: 2026-02-26T21:12:17.463276
zenodo_link: https://doi.org/10.5281/zenodo.10105821
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/clinical-mp-2-discovery/tutorial.html

content:
This tutorial can be followed with any user-defined database but would work best if the clinical metaproteomics database generation module was used (see [Database Generation tutorial](https://github.com/subinamehta/training-material/blob/main/topics/proteomics/tutorials/clinical-mp-database-generation/tutorial.md)). The MetaNovo tool generates a more manageable database that contains identified proteins. The MetaNovo-generated database merged with Human SwissProt (reviewed only) and contaminants (cRAP) databases to generate a compact database (~21.2k protein sequences) that will be used for peptide identification.


# Peptide identification
The MSMS data will be searched against the compact database `Human UniProt Microbial Proteins (from MetaNovo) and cRAP` to identify peptide and protein sequences via sequence database searching. For this tutorial, two peptide identification programs will be used: SearchGUI/PeptideShaker and MaxQuant. However, you could use other software too, such as Fragpipe or Scribe. For the purpose of this tutorial, a dataset of the 4 RAW/MGF files will be used as the MS/MS input.


![Discovery Workflow]({% link topics/proteomics/images/clinical-mp/clinical-mp-discovery.JPG %})


> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Database Searching
This step is to identify proteins based on mass spectrometry data. The algorithms identify peptides in the spectra and search a protein sequence database to match observed peptide data with theoretical peptide masses and spectra. Scoring and false discovery rate control help assess the reliability of matches, followed by protein inference to determine the proteins present in the sample. These algorithms are essential for interpreting mass spectrometry data, aiding in protein identification, quantification, and insights into biological processes and disease mechanisms in proteomics research.

## Get data

> <hands-on-title> Data Upload </hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo]({{ page.zenodo_link }}) or from
>    the shared data library (`GTN - Material` -> `{{ page.topic_name }}`
>     -> `{{ page.title }}`):
>
>    ```
>    https://zenodo.org/records/10105821/files/Human_UniProt_Microbial_Proteins_(from_MetaNovo)_and_cRAP.fasta
>    https://zenodo.org/records/10105821/files/PTRC_Skubitz_Plex2_F10_9Aug19_Rage_Rep-19-06-08.raw
>    https://zenodo.org/records/10105821/files/PTRC_Skubitz_Plex2_F11_9Aug19_Rage_Rep-19-06-08.raw
>    https://zenodo.org/records/10105821/files/PTRC_Skubitz_Plex2_F13_9Aug19_Rage_Rep-19-06-08.raw
>    https://zenodo.org/records/10105821/files/PTRC_Skubitz_Plex2_F15_9Aug19_Rage_Rep-19-06-08.raw
>    https://zenodo.org/records/10105821/files/Experimental-Design_Discovery_MaxQuant.tabular
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
> 3. Rename the datasets
> 4. Check that the datatype
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
>
> 5. Add to each database a tag corresponding to user.
> 6. Create a dataset collection of all the raw files and MGF files.
>
>    {% snippet faqs/galaxy/datasets_add_tag.md %}
>
{: .hands_on}

# Import Workflow


> <hands-on-title>Running the Workflow</hands-on-title>
>
> 1. **Import the workflow** into Galaxy:
>
>    {% snippet faqs/galaxy/workflows_run_trs.md path="topics/proteomics/tutorials/clinical-mp-2-discovery/workflows/WF2_Discovery-Workflow.ga" title="Discovery Workflow" %}
>
> 2. Run **Workflow** {% icon workflow %} using the following parameters:
>    - *"Send results to a new history"*: `No`
>    - {% icon param-file %} *" RAW files"*: `RAW dataset collection`
>    - {% icon param-file %} *" Human UniProt Microbial Proteins (from MetaNovo) and cRAP"*: `Human_UniProt_Microbial_Proteins_(from_MetaNovo)_and_cRAP.fasta`
>    - {% icon param-file %} *" Experimental Design Discovery MaxQuant"*: `Experimental-Design_Discovery_MaxQuant.tabular`
>
>    {% snippet faqs/galaxy/workflows_run.md %}
>
{: .hands_on}


# Peptide identification
Using the compact database generated by MetaNovo as the input database, we will match MS/MS data to peptide sequences via sequence database searching.

For this tutorial, two peptide identification programs will be used: SearchGUI/PeptideShaker and MaxQuant. For both programs, the created dataset of the four MS datasets  in the history will be used as the MS/MS input. The RAW MS/MS data files will be converted into mascot generic format (MGF) files as that is the standard format in which MS/MS searches are performed.

Peptides identified from each program will be verified with the PepQuery tool to generate a master list of confident verified microbial peptides.

## Appending decoy sequenced to FASTA database with **FastaCLI**

Using the FastaCLI tool, decoy sequences will be appended to the FASTA database. Decoy sequences are protein sequences are not expected to be present in samples. For more information on how to generate and append decoy sequences, see [GTN Protein FASTA Database Handling](https://training.galaxyproject.org/archive/2019-12-01/topics/proteomics/tutorials/database-handling/tutorial.html#creating-a-decoy-database).

> <hands-on-title> FastaCLI </hands-on-title>
>
> 1. {% tool [FastaCLI](toolshed.g2.bx.psu.edu/repos/galaxyp/peptideshaker/fasta_cli/4.0.41+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"Protein Database"*: `output` (Input dataset)
>
>
{: .hands_on}

## Converting RAW files to MGF files with **msconvert**

The msconvert tool allows for the conversion of mass spectrometry data files between different formats, such as thermo.raw, mgf, or mzml.

> <hands-on-title> msconvert: RAW to MGF </hands-on-title>
>
> 1. {% tool [msconvert](toolshed.g2.bx.psu.edu/repos/galaxyp/msconvert/msconvert/3.0.20287.2) %} with the following parameters:
>    - {% icon param-collection %} *"Input unrefined MS data"*: `output` (Input dataset collection)
>    - *"Do you agree to the vendor licenses?"*: `Yes`
>    - *"Output Type"*: `mgf`
>    - In *"Data Processing Filters"*:
>        - *"Apply peak picking?"*: `Yes`
>        - *"(Re-)calculate charge states?"*: `no`
>
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. Why do we need to use MGF instead of RAW files for Search GUI?
>
> > <solution-title></solution-title>
> >
> > 1. SearchGUI is compatible only with MGF files, hence you have to use msconvert or Thermofile converter tools to convert the RAW format to MGF fomat.
> >
> {: .solution}
>
{: .question}

## Perform Database searching with **Search GUI**
SearchGUI is a database-searching tool that comprises different search engines to match sample MS/MS spectra to known peptide sequences. In our analysis, we will use X!Tandem and MS-GF+ as search algorithms within SearchGUI for matching spectra from mass spectrometry data against peptides from the protein sequence database.

The SearchGUI tool will perform a database search based on the parameters we've set and will generate a file (called a SearchGUI archive file) that will serve as the input for the PeptideShaker tool. The SearchGUI archive file contains Peptide-Spectral Matches (PSMs), and PeptideShaker is a post-processing software that will assess the confidence of the data. PeptideShaker also infers the identities of proteins based on the matched peptide sequences, and users are able to visualize these outputs to interpret results.  More information about database searching using SearchGUI and PeptideShaker is accessible at [Metaproteomics tutorial](https://gxy.io/GTN:T00221).

> <hands-on-title> Peptide discovery using SearchGUI </hands-on-title>
>
> 1. {% tool [Search GUI](toolshed.g2.bx.psu.edu/repos/galaxyp/peptideshaker/search_gui/4.0.41+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"Identification Parameters file"*: `Identification_Parameters_File` (output of **Identification Parameters** {% icon tool %})
>    - {% icon param-file %} *"Fasta file"*: `input_database_concatenated_target_decoy` (output of **FastaCLI** {% icon tool %})
>    - {% icon param-file %} *"Input Peak Lists"*: `output` (output of **msconvert** {% icon tool %})
>    - *"SearchGUI Options"*: `Default`
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why do we need to add decoy sequences to our FASTA database for Search GUI? And how many do we need to add?
>
> > <solution-title></solution-title>
> >
> > 1. Adding decoy sequences helps in FDR estimation, discriminating true positives from false positives, and quality control of the data. The number of decoy sequences you need to add to your database depends on the desired FDR level you want to achieve. A common practice is to use a 1:1 ratio of target sequences to decoy sequences. In other words, for every real protein sequence in your database, you would add a decoy sequence. This allows you to estimate the FDR at 1%, 5%, or any other chosen threshold.
> >
> {: .solution}
>
{: .question}
> <question-title></question-title>
>
> 1. What is the Identification Parameters tool?
>
> > <solution-title></solution-title>
> >
> > 1. Identification Parameters tool is an input required by the search GUI tool, it contains all the parameters required to run the search algorithms.
> >
> {: .solution}
>
{: .question}

## Post-processing of SearchGUI output using with **Peptide Shaker**

> <hands-on-title> Peptide Shaker </hands-on-title>
>
> 1. {% tool [Peptide Shaker](toolshed.g2.bx.psu.edu/repos/galaxyp/peptideshaker/peptide_shaker/2.0.33+galaxy1) %} with the following parameters:
>    - {% icon param-file %} *"Compressed SearchGUI results"*: `searchgui_results` (output of **Search GUI** {% icon tool %})
>    - In *"Exporting options"*:
>        - *"Follow-up analysis export options"*: `Do not export`
>        - *"Identification features reports to be generated"*: `PSM Report` `Peptide Report` `Protein Report` `Certificate of Analysis`
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. What are the differences between the following reports from PeptideShaker: PSM report, Peptide report, and Protein report?
>
> > <solution-title></solution-title>
> >
> > 1. PSM reports focus on individual peptide-spectrum matches, providing detailed information about each spectrum and its assigned peptide sequence. Peptide reports summarize information about unique peptides and their properties. Protein reports, on the other hand, focus on proteins, including protein inference, grouping, and quantification, making them more suitable for understanding the overall protein composition in a sample. These reports serve different purposes in proteomic data analysis and are used to extract various levels of information from mass spectrometry results.
> >
> {: .solution}
>
{: .question}

##  Using Text Manipulation Tools to Manage Microbial Outputs from SearchGUI/PeptideShaker

> <hands-on-title> Selecting microbial peptides from SearchGUI/PeptideShaker with Select tool </hands-on-title>
>
> 1. {% tool [Select](Grep1) %} with the following parameters:
>    - {% icon param-file %} *"Select lines from"*: `output_peptides` (output of **Peptide Shaker** {% icon tool %})
>    - *"that"*: `NOT Matching`
>    - *"the pattern"*: `(_HUMAN)|(_REVERSED)|(REV_)|(CON)|(con)`
>    - *"Keep header line"*: `Yes`
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. What is the purpose of this step?
>
> > <solution-title></solution-title>
> >
> > 1. This step is to extract microbial peptides or to remove any peptides that match humans, reverse, contaminants, etc.
> >
> {: .solution}
>
{: .question}

> <hands-on-title> Selecting microbial PSMs from SearchGUI/PeptideShaker with Select </hands-on-title>
>
> 1. {% tool [Select](Grep1) %} with the following parameters:
>    - {% icon param-file %} *"Select lines from"*: `output_psm` (output of **Peptide Shaker** {% icon tool %})
>    - *"that"*: `NOT Matching`
>    - *"the pattern"*: `(_HUMAN)|(_REVERSED)|(REV_)|(CON)|(con)`
>    - *"Keep header line"*: `Yes`
>
>
{: .hands_on}


> <hands-on-title> Filtering confident microbial peptides from SGPS  with Filter </hands-on-title>
>
> 1. {% tool [Filter](Filter1) %} with the following parameters:
>    - {% icon param-file %} *"Filter"*: `out_file1` (output of **Select** {% icon tool %})
>    - *"With following condition"*: `c17=='Confident'`
>    - *"Number of header lines to skip"*: `1`
>
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. In the Filtering steps, what does “Confidence” mean quantitatively, i.e. what is the percentage cutoff?
>
> > <solution-title></solution-title>
> >
> > 1. The term "Confidence" in the context of proteomic data analysis often refers to a measure of how reliable or trustworthy a particular protein or peptide identification is. However, the specific numerical value or percentage cutoff for confidence can vary depending on the software or approach you are using and the goals of your analysis. In many proteomics studies, researchers use a false discovery rate (FDR) to set a quantitative confidence threshold. Here we have set it as 1%FDR, which means that you're accepting only 1% or less of your reported identifications as likely to be false positives.
> >
> {: .solution}
>
{: .question}


> <hands-on-title> Filtering confident microbial PSMs from SGPS with Filter </hands-on-title>
>
> 1. {% tool [Filter](Filter1) %} with the following parameters:
>    - {% icon param-file %} *"Filter"*: `out_file1` (output of **Select** {% icon tool %})
>    - *"With following condition"*: `c24=='Confident'`
>    - *"Number of header lines to skip"*: `1`
>
>
{: .hands_on}


We will generate and merge the Human SwissProt Protein Database and contaminants (cRAP) and convert the resulting FASTA file to a tabular file that will be used in the Query Tabular tool to generate distinct microbial peptides from SearchGUI/PeptideShaker.

> <hands-on-title> Merging Human SwissProt and cRAP databases for Query Tabular with FASTA Merge Files and Filter Unique Sequences </hands-on-title>
>
> 1. {% tool [FASTA Merge Files and Filter Unique Sequences](toolshed.g2.bx.psu.edu/repos/galaxyp/fasta_merge_files_and_filter_unique_sequences/fasta_merge_files_and_filter_unique_sequences/1.2.0) %} with the following parameters:
>    - *"Run in batch mode?"*: `Merge individual FASTAs (output collection if input is collection)`
>        - In *"Input FASTA File(s)"*:
>            - {% icon param-repeat %} *"Insert Input FASTA File(s)"*
>                - {% icon param-file %} *"FASTA File"*: `Human Swissprot Protein Database` (output of **Protein Database Downloader** {% icon tool %})
>                - - {% icon param-file %} *"FASTA File"*: `Contaminants cRAP database` (output of **Protein Database Downloader** {% icon tool %})
>
>
{: .hands_on}


> <hands-on-title> Converting FASTA sequences to TAB-delimited file with FASTA-to-Tabular </hands-on-title>
>
> 1. {% tool [FASTA-to-Tabular](toolshed.g2.bx.psu.edu/repos/devteam/fasta_to_tabular/fasta2tab/1.1.0) %} with the following parameters:
>    - {% icon param-file %} *"Convert these sequences"*: `output` (output of **FASTA Merge Files and Filter Unique Sequences** {% icon tool %})
>
>
{: .hands_on}


> <hands-on-title> Filtering out accession numbers from TAB-delimited file  with Filter Tabular </hands-on-title>
>
> 1. {% tool [Filter Tabular](toolshed.g2.bx.psu.edu/repos/iuc/filter_tabular/filter_tabular/3.3.0) %} with the following parameters:
>    - {% icon param-file %} *"Tabular Dataset to filter"*: `output` (output of **FASTA-to-Tabular** {% icon tool %})
>    - In *"Filter Tabular Input Lines"*:
>        - {% icon param-repeat %} *"Insert Filter Tabular Input Lines"*
>            - *"Filter By"*: `select columns`
>                - *"enter column numbers to keep"*: `1`
>        - {% icon param-repeat %} *"Insert Filter Tabular Input Lines"*
>            - *"Filter By"*: `regex replace value in column`
>                - *"enter column number to replace"*: `1`
>                - *"regex pattern"*: `^[^|]+[|]([^| ]+).*$`
>                - *"replacement expression"*: `\1`
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. What’s the difference between a FASTA and Tabular output?
>
> > <solution-title></solution-title>
> >
> > 1. FASTA Output: Typically used to report identified peptide or protein sequences, which are useful for building or updating sequence databases, for downstream sequence analysis, or for re-searching against the sequences.
> > Tabular Output: Used for presenting various information related to identified peptides or proteins, such as accession numbers, scores, abundance values, and other attributes. Tabular output facilitates data analysis, comparisons, and custom data processing.
> >
> >
> {: .solution}
>
{: .question}


> <hands-on-title> Querying protein accession numbers and peptide sequences of confident microbial PSMs (from SGPS) with Query Tabular </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.0) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `out_file1` (output of **Filter** {% icon tool %})
>            - In *"Filter Dataset Input"*:
>                - In *"Filter Tabular Input Lines"*:
>                    - {% icon param-repeat %} *"Insert Filter Tabular Input Lines"*
>                        - *"Filter By"*: `skip leading lines`
>                            - *"Skip lines"*: `1`
>                    - {% icon param-repeat %} *"Insert Filter Tabular Input Lines"*
>                        - *"Filter By"*: `prepend a line number column`
>            - In *"Table Options"*:
>                - *"Specify Name for Table"*: `psms`
>                - *"Specify Column Names (comma-separated list)"*: `ln,id,Proteins,Sequence`
>                - *"Only load the columns you have named into database"*: `Yes`
>                - In *"Table Index"*:
>                    - {% icon param-repeat %} *"Insert Table Index"*
>                        - *"Index on Columns"*: `ln`
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `out_file1` (output of **Filter** {% icon tool %})
>            - In *"Filter Dataset Input"*:
>                - In *"Filter Tabular Input Lines"*:
>                    - {% icon param-repeat %} *"Insert Filter Tabular Input Lines"*
>                        - *"Filter By"*: `skip leading lines`
>                            - *"Skip lines"*: `1`
>                    - {% icon param-repeat %} *"Insert Filter Tabular Input Lines"*
>                        - *"Filter By"*: `prepend a line number column`
>                    - {% icon param-repeat %} *"Insert Filter Tabular Input Lines"*
>                        - *"Filter By"*: `normalize list columns, replicates row for each item in list`
>                            - *"enter column numbers to normalize"*: `3`
>            - In *"Table Options"*:
>                - *"Specify Name for Table"*: `prots`
>                - *"Specify Column Names (comma-separated list)"*: `ln,id,prot`
>                - *"Only load the columns you have named into database"*: `Yes`
>                - In *"Table Index"*:
>                    - {% icon param-repeat %} *"Insert Table Index"*
>                        - *"This is a unique index"*: `Yes`
>                        - *"Index on Columns"*: `prot,ln`
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `output` (output of **Filter Tabular** {% icon tool %})
>            - In *"Table Options"*:
>                - *"Specify Name for Table"*: `Uniprot`
>                - *"Specify Column Names (comma-separated list)"*: `prot`
>                - In *"Table Index"*:
>                    - {% icon param-repeat %} *"Insert Table Index"*
>                        - *"Index on Columns"*: `prot`
>    - *"SQL Query to generate tabular output"*: `SELECT id,Proteins,Sequence FROM psms WHERE psms.ln NOT IN (SELECT distinct prots.ln FROM prots JOIN Uniprot ON prots.prot = Uniprot.prot) ORDER BY psms.ln`
>    - *"include query result column headers"*: `Yes`
>
>
{: .hands_on}


> <hands-on-title> Cutting out peptide sequences from Query Tabular with Cut </hands-on-title>
>
> 1. {% tool [Cut](Cut1) %} with the following parameters:
>    - *"Cut columns"*: `c3`
>    - {% icon param-file %} *"From"*: `output` (output of **Query Tabular** {% icon tool %})
>
>
{: .hands_on}


> <hands-on-title> Grouping distinct (unique) peptides from SGPS with Group </hands-on-title>
>
> 1. {% tool [Group](Grouping1) %} with the following parameters:
>    - {% icon param-file %} *"Select data"*: `out_file1` (output of **Cut** {% icon tool %})
>    - *"Group by column"*: `c1`
>
>
{: .hands_on}


## Perform peptide discovery with **MaxQuant**
MaxQuant is an MS-based proteomics platform that is capable of processing raw data and provides improved mass precision and high precursor mass accuracy (HPMA), which resulted in increased protein identification and more in-depth proteomic analysis. Raw MS/MS spectra will be searched against the reduced MetaNovo-generated database (~21.2k sequences). More information about analysis using MaxQuant is available, including [Label-free data analysis](https://gxy.io/GTN:T00218) and [MaxQuant and MSstats for the analysis of TMT data](https://gxy.io/GTN:T00220).

> <hands-on-title> Peptide discovery using MaxQuant </hands-on-title>
>
> 1. {% tool [MaxQuant](toolshed.g2.bx.psu.edu/repos/galaxyp/maxquant/maxquant/2.0.3.0+galaxy0) %} with the following parameters:
>    - In *"Input Options"*:
>        - {% icon param-file %} *"FASTA files"*: `output` (Input dataset)
>    - In *"Search Options"*:
>        - {% icon param-file %} *"Specify an experimental design template (if needed). For detailed instructions see the help text."*: `output` (Input dataset)
>        - *"minimum peptide length"*: `8`
>        - *"Match between runs"*: `Yes`
>        - *"Maximum peptide length for unspecific searches"*: `50`
>    - In *"Protein quantification"*:
>        - *"Use only unmodified peptides"*: `Yes`
>            - *"Modifications used in protein quantification"*: `Oxidation (M)`
>        - In *"LFQ Options"*:
>            - *"iBAQ (calculates absolute protein abundances by normalizing to copy number and not protein mass)"*: `No`
>    - In *"Parameter Group"*:
>        - {% icon param-repeat %} *"Insert Parameter Group"*
>            - {% icon param-collection %} *"Infiles"*: `output` (Input dataset collection)
>            - *"fixed modifications"*: `Carbamidomethyl (C)`
>            - *"variable modifications"*: `Oxidation (M)`
>            - *"enzyme"*: `Trypsin/P`
>            - *"Quantitation Methods"*: `reporter ion MS2`
>                - *"isobaric labeling"*: `TMT11plex`
>                - *"Filter by PIF"*: `True`
>    - *"Generate PTXQC (proteomics quality control pipeline) report? (experimental setting)"*: `False`
>    - In *"Output Options"*:
>        - *"Select the desired outputs."*: `Protein Groups` `mqpar.xml` `Peptides` `MSMS` `msms scans` `summary` `MaxQuant and PTXQC log` `yaml config file`
>
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. What is the Experimental Design file for MaxQuant?
> 
> > <solution-title></solution-title>
> >
> > 1. In MaxQuant, the **Experimental Design** file is used to specify the experimental conditions, sample groups, and the relationships between different samples in a proteomics experiment. This file is a crucial component of the MaxQuant analysis process because it helps the software correctly organize and analyze the mass spectrometry data. The **Experimental Design** file typically has a ".txt" extension and is a tab-delimited text file. Here's what you might include in an Experimental Design file for MaxQuant: **Sample Names** (You specify the names of each sample in your experiment. These names should be consistent with the naming conventions used in your raw data files.), **Experimental Conditions** (You define the experimental conditions or treatment groups associated with each sample. For example, you might have control and treated groups, and you would assign the appropriate condition to each sample.), **Replicates** (You indicate the replicates for each sample, which is important for assessing the statistical significance of your results. Replicates are typically denoted by numeric values (e.g., "1," "2," "3") or by unique identifiers (e.g., "Replicate A," "Replicate B")), **Labels** (If you're using isobaric labeling methods like TMT (Tandem Mass Tag) or iTRAQ (Isobaric Tags for Relative and Absolute Quantitation), you specify the labels associated with each sample. This is important for quantification.), **Other Metadata** (You can include additional metadata relevant to your experiment, such as the biological source, time points, or any other information that helps describe the samples and experimental conditions.)
> >
> {: .solution}
>
{: .question}


## Using Text Manipulation Tools to Manage Microbial Outputs from MaxQuant

> <hands-on-title> Selecting microbial peptides from MaxQuant with Select </hands-on-title>
>
> 1. {% tool [Select](Grep1) %} with the following parameters:
>    - {% icon param-file %} *"Select lines from"*: `peptides` (output of **MaxQuant** {% icon tool %})
>    - *"that"*: `NOT Matching`
>    - *"the pattern"*: `(_HUMAN)|(_REVERSED)|(REV_)|(CON)|(con)`
>    - *"Keep header line"*: `Yes`
>
>
{: .hands_on}


> <hands-on-title> Cutting out microbial peptide sequences with Cut </hands-on-title>
>
> 1. {% tool [Cut](Cut1) %} with the following parameters:
>    - *"Cut columns"*: `c1`
>    - {% icon param-file %} *"From"*: `out_file1` (output of **Select** {% icon tool %})
>
>
{: .hands_on}


> <hands-on-title> Remove header line from MaxQuant peptide output with Remove beginning </hands-on-title>
>
> 1. {% tool [Remove beginning](Remove beginning1) %} with the following parameters:
>    - {% icon param-file %} *"from"*: `out_file1` (output of **Cut** {% icon tool %})
>
>
{: .hands_on}


> <hands-on-title> Grouping distinct (unique) peptide sequences from MaxQuant with Group </hands-on-title>
>
> 1. {% tool [Group](Grouping1) %} with the following parameters:
>    - {% icon param-file %} *"Select data"*: `out_file1` (output of **Remove beginning** {% icon tool %})
>    - *"Group by column"*: `c1`
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. How case-sensitive is the Group tool? Can I only group by column values, and not row values?
>
> > <solution-title></solution-title>
> >
> > 1. You can make it case sensitive, by default it is not. The tool here does column grouping only.
> >
> {: .solution}
>
{: .question}


## Process SGPS and MaxQuant peptides to compile one list of unique microbial peptides


> <hands-on-title> Concatenate SGPS and MaxQuant peptides into a singular database with Concatenate datasets </hands-on-title>
>
> 1. {% tool [Concatenate datasets](toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.1) %} with the following parameters:
>    - {% icon param-files %} *"Datasets to concatenate"*: `out_file1` (output of **Group** {% icon tool %}), `out_file1` (output of **Group** {% icon tool %})
>
>
{: .hands_on}


> <hands-on-title> Group the peptides from SGPS and MaxQuant to remove duplicates with Group </hands-on-title>
>
> 1. {% tool [Group](Grouping1) %} with the following parameters:
>    - {% icon param-file %} *"Select data"*: `out_file1` (output of **Concatenate datasets** {% icon tool %})
>    - *"Group by column"*: `c1`
>
>
{: .hands_on}


# Conclusion
By following this tutorial, you have effectively conducted a search of your MS/MS data against the compact database and successfully retrieved reliable microbial peptides. After identifying these microbial peptides with the assistance of MaxQuant and SearchGUI, the next step is to verify the presence of these peptides. This compiled list of unique peptides will serve as the input for PepQuery to validate the confident identification of microbial peptides with the help of the verification workflow.


topic: proteomics
tutorial: metaquantome-taxonomy
title: metaQuantome 3: Taxonomy
difficulty: intermediate
time_estimation: 1h
tools_json: []
requirements_json: []
tags_json: ["[microgalaxy]"]
content_hash: 24f76067d1156c57b60722271da918cc
last_modified: 2026-02-26T21:12:17.497970
zenodo_link: https://doi.org/10.5281/zenodo.4110725
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/metaquantome-taxonomy/tutorial.html

content:
metaQuantome software suite {% cite Easterly2019 %} was developed by the {% cite Galaxy-P %} for quantitative and statistical analysis of metaproteomics data. For taxonomic and functional expression analysis within the microbial community, metaQuantome leverages peptide-level quantitative information to generate visual outputs for data interpretation. It also generates outputs that help in understanding the taxonomic contribution to a selected function as well as functions expressed by selected taxonomic group.

In this tutorial, we will learn specifically about the metaQuantome Taxonomy workflow. In particular, we will learn about how peptide-level quantitation and associated taxonomic information can be used to generate bar plots (for taxonomic composition), volcano plots (to detect differentially expressed taxa) and heatmap cluster analysis.

To demonstrate the use of this workflow, we have used a thermophilic biogas reactor dataset wherein municipal food waste and manure is digested to generate methane gas ({% cite Delogu2020 %}). After one round in the reactor, the microbial community was simplified and enriched via serial dilution. This inoculum was then transferred to a solution of cellulose from Norwegian Spruce and incubated at 65°C. Triplicate samples were taken in a time series from 0 to 43 hours after inoculation and mass spectrometry data was acquired on a Q-Exactive (Thermo) mass spectrometer. For this training, we have chosen three time points (8 hours, 18 hours and 33 hours) from this dataset.

![MetaQuantome-taxonomy-workflow](../../images/mQ-taxa-workflow.jpg){: width="100%"}

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}


# Pretreatments

The first step in this tutorial is to get the data from the Zenodo link provided and make sure that it is in the correct format.


## Get data

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Create a new history for this tutorial and give it a meaningful name.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 2. Import the files from [Zenodo]({{ page.zenodo_link }}): a Functional File and an Intensity file.
>
>
>    ```
>    https://zenodo.org/record/4110725/files/Intensity-File.tabular
>    https://zenodo.org/record/4110725/files/Taxonomy-File.tabular
>
>    ```
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    Alternatively, import the files from the shared data library (`GTN - Material` -> `{{ page.topic_name }}` -> `{{ page.title }}`)
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
>
>
{: .hands_on}

# Download metaQuantome Databases

> <hands-on-title>Run metaQuantome databases</hands-on-title>
>
> 1. {% tool [metaQuantome: databases](toolshed.g2.bx.psu.edu/repos/galaxyp/metaquantome_db/metaquantome_db/2.0.0-0) %} with the following parameters:
>    - {% icon param-select %} *"Databases to Download"*: `Select All` (NCBI Taxonomy Database, Gene Ontology Database and Enzyme Commission (EC) database)
>
>
>    > <comment-title></comment-title>
>    >
>    > metaQuantome uses freely available bioinformatic databases to expand your set of direct annotations.
>    >
>    >
>    > The databases are:
>    >
>    > 1. NCBI taxonomy database. consists of a list of all currently identified taxa and the relationships between them.
>    >
>    > 2. Gene Ontology (GO) term database. metaQuantome uses the go-basic.obo file and the metagenomics slim GO (subset of the full GO). More details are
>    > available at [GO term website](http://geneontology.org/docs/download-ontology/)
>    >
>    > 3. ENZYME database with Enzyme Classification (EC) numbers. This database classifies enzymes and organizes the relationships between them.
>    >
>    {: .comment}
>
{: .hands_on}


> <question-title></question-title>
>
> 1. Why is it necessary to download metaQuanome databases?
> 2. Can these databases be downloaded from other sources?
>
> > <solution-title></solution-title>
> >
> > 1. metaQuantome databases help the metaQuantome:expand tool to co-relate information your have provided in your input files according to the GO term, taxa or EC databases and its relationship .
> > 2. For metaQuantome, we use freely available databases. This module downloads the most recent releases of the specified databases and stores them in a single file, which can then be accessed by the rest of the metaQuantome modules. For reference, the taxonomy database is the largest (~500 Mb), while the GO and EC databases are smaller: ~34 Mb and ~10Mb, respectively. Also, note that the databases will be stored in the history so that the date of download can be referenced later. Thus, the databases will not be modified, except for the NCBI database.
> >
> {: .solution}
>
{: .question}

# Create metaQuantome sample file

The create samples file module is used to generate the samples file input file for the metaQuantome workflow. This input file is used to specify the column names used for each experimental group. These column names are referenced when handling the input data and performing statistical analysis.

> <hands-on-title>Create an experimental Design file for sorting samples</hands-on-title>
>
> 1. {% tool [metaQuantome: create samples file](toolshed.g2.bx.psu.edu/repos/galaxyp/metaquantome_sample/metaquantome_sample/2.0.0-0) %} with the following parameters:
>    - {% icon param-select %} *"Sample file creation method"*: `Manually specify experimental conditions and samples`
>    - In *"Insert Samples"*:
>        - *"1.Samples"*:
>            - *"Group Name"*: `T2`
>               - *"Column"*: `T2_A1,T2_B1`
>        - *"2.Samples"*:
>            - *"Group Name"*: `T4`
>               - *"Column"*: `T4A_1,T4B_1`
>        - *"3.Samples"*:
>            - *"Group Name"*: `T7`
>               - *"Column"*: `T7A_1,T7B_1`
>
>
{: .hands_on}


# Run metaQuantome


## metaQuantome: expand

The expand module is the first analysis step in the metaQuantome analysis workflow, and can be run to analyze differently expressed Taxa in the samples.
In taxonomy mode, the following information is required apart from metaQuantome databases and samples file: a tab-separated taxonomy annotation file, with a peptide column and a taxonomy annotation column. The taxonomic annotations should be the lowest common ancestor (LCA) for each peptide, preferably given as NCBI taxonomy IDs.

- The name of the peptide column in the taxonomic annotation file (`peptide`)
- The name of the taxonomy annotation column in the taxonomy annotation file (`taxon_id`)
- The name of the peptide column in the Intensity file (`peptide`)

> <hands-on-title>Run metaQuantome expand</hands-on-title>
>
> 1. {% tool [metaQuantome: expand](toolshed.g2.bx.psu.edu/repos/galaxyp/metaquantome_expand/metaquantome_expand/2.0.0-0) %} with the following parameters:
>    - {% icon param-file %} *"Database Archive File"*: `metaQuantome databases` (output of **metaQuantome: database** {% icon tool %})
>    - {% icon param-file %} *"Samples file"*: `metaQuantome: create samples_file` (output of **metaQuantome: create samples file** {% icon tool %})
>    - *"Mode"*: `Taxonomic analysis`
>        - {% icon param-file %} *"Taxonomy file"*: `Taxonomy File` (Input dataset)
>        - *"Taxonomy file: peptide column name"*: `peptide`
>        - *"Taxonomy column name"*: `taxon_id`
>    - {% icon param-file %} *"Intensity file"*: `Intensity File` (Input dataset)
>        - *"Intensity file: peptide column name"*: `peptide`
>
> 2. **Rename** {% icon galaxy-pencil %} the output file to `metaQuantome expand`.
>
>
>    > <comment-title></comment-title>
>    >
>    > The structure of the output file depends on the analysis mode and the experimental design, but the columns generally look like this, with one row for each term:
>    >
>    > | term id | info about term. (one or more columns) | mean term intensity (by sample group)            | term intensity (by sample)                                                      | number of unique peptides (by sample) | number of sample children in each sample |
>    > |---------|----------------------------------------|--------------------------------------------------|---------------------------------------------------------------------------------|---------------------------------------|------------------------------------------|
>    > | term1   | name, rank, etc.                       | note that this is the log2 of the mean intensity | this is the log2 of term intensity in each sample. Missing data is coded as NA. | integer. 0 is coded as NA             | integer. 0 is coded as NA                |
>    > |         |                                        |                                                  |                                                                                 |                                       |                                          |
>    {: .comment}
>
>    {: .comment}
>
{: .hands_on}


## metaQuantome: filter

The filter module is the second step in the metaQuantome workflow. The filter module filters the expanded terms to those that are representative of the data according to the sample parameters the user has specified.

> <hands-on-title>Filtering the expanded data</hands-on-title>
>
> 1. {% tool [metaQuantome: filter](toolshed.g2.bx.psu.edu/repos/galaxyp/metaquantome_filter/metaquantome_filter/2.0.0-0) %} with the following parameters:
>    - *"Mode"*: `Taxonomic analysis`
>    - {% icon param-file %} *"Samples file"*: `metaQuantome:create samples_file` (output of **metaQuantome: create samples file** {% icon tool %})
>    - {% icon param-file %} *"metaquantome expand file"*: `metaQuantome:expand` (output of **metaQuantome: expand** {% icon tool %})
>    - *"min_peptides"*: `1`
>    - *"min_pep_nsamp"*: `1`
>    - *"min_children_non_leaf"*: `2`
>    - *"min_child_nsamp"*: `1`
>    - *"qthreshold"*: `2`
>
> 2. **Rename** {% icon galaxy-pencil %} the output file to `metaQuantome:Filter`.
>
>
>    > <comment-title></comment-title>
>    >
>    > To learn more about the filter module please read the metaQuantome paper published by Galaxy-P {% cite Easterly2019 %}.
>    {: .comment}
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Can we change the filter parameters?
>
> > <solution-title></solution-title>
> >
> > 1. The parameters can be changed according to the experimental design or depending on the data.
> >
> {: .solution}
>
{: .question}

## metaQuantome: stat

> <hands-on-title>Statistical analysis of the filtered data on multiple conditions.</hands-on-title>
>
> 1. {% tool [metaQuantome: stat](toolshed.g2.bx.psu.edu/repos/galaxyp/metaquantome_stat/metaquantome_stat/2.0.0-0) %} with the following parameters:
>    - {% icon param-file %} *"Samples file"*: `metaQuantome:create samples_file` (output of **metaQuantome: create samples file** {% icon tool %})
>    - {% icon param-file %} *"metaquantome expanded or filtered file"*: `metaQuantome:Filter` (output of **metaQuantome: filter** {% icon tool %})
>    - *"Mode"*: `Taxonomic analysis`
>    - *"Name of the control sample group of control (used as denominator for fold change evaluation)."*: `T2`
>    - *"Statistical test to perform"*: `standard t-test (parametric)`
>    - *"Perform paired tests"*: `Yes`
>
>
>
> 2. **Rename** {% icon galaxy-pencil %} the output file to `metaQuantome:stat`.
>
>    > <comment-title></comment-title>
>    >
>    > In this tutorial, we have used the sample T2 as the control group for fold change evaluation. The users have the ability to choose their own control sample group. The initial version of the metaQuantome stat module could only process 2 conditions at a time. However, the new update of metaQuantome has the ability to perform statistical analysis across multiple conditions.
>    {: .comment}
>
{: .hands_on}


# Visualize your Data

The  outputs of the visualization module of metaQuantome are high-quality, publication-ready visualizations: barplots for the analysis of a single sample or experimental condition and differential abundance analysis, volcano plots, heatmaps, and principal components analysis for comparisons between two or more experimental conditions.
Here were are showing 2 visualizations: **Barplot and Volcano Plot**. The Heatmap and PCA plot for multiple conditions are under development.
There are two outputs of the visualization tool : an **HTML file (figure) and a tabular output containing the plot data**.

## metaQuantome: visualize

> <hands-on-title>Bar chart visualization of Taxonomy in T2 sample.</hands-on-title>
>
> 1. {% tool [metaQuantome: visualize](toolshed.g2.bx.psu.edu/repos/galaxyp/metaquantome_viz/metaquantome_viz/2.0.0-0) %} with the following parameters:
>    - {% icon param-file %} *"Tabular file from metaQuantome stats or metaQuantome filter"*: `metaQuantome:stat` (output of **metaQuantome: stat** {% icon tool %})
>    - {% icon param-file %} *"Samples file"*: `metaQuantome:create samples_file` (output of **metaQuantome: create samples file** {% icon tool %})
>    - *"Mode"*: `Taxonomic analysis`
>        - *"Plot type"*: `Bar Chart (bar)`
>            - *"Taxonomic rank to restrict to in the plot"*: `genus`
>            - *"Mean intensity column name"*: `T2_mean`
>
>
>    > <comment-title></comment-title>
>    > The user has a choice to run Bar plot for T2, T4 and T7 to know the top 5 or 10 most differently expressed taxonomy.
>    >
>    {: .comment}
>
{: .hands_on}


![T2_taxa](../../images/T2_taxa.png "Bar Chart visualization of Top 5 genus present in T2 sample."){: width="85%"}

> <question-title></question-title>
>
> 1. Can we select any color for the bar chart?
> 2. Can we increase the number of terms to display?
>
> > <solution-title></solution-title>
> >
> > 1. Yes, the available colors are blue, orange, yellow, violet, black and red.
> > 2. Yes, the user can choose their own number, however, the larger the display numbers are the dimensions of the image has to be adjusted accordingly.
> >
> {: .solution}
>
{: .question}


> <hands-on-title>Bar chart visualization of Taxonomy in T7 sample.</hands-on-title>
>
> 1. {% tool [metaQuantome: visualize](toolshed.g2.bx.psu.edu/repos/galaxyp/metaquantome_viz/metaquantome_viz/2.0.0-0) %} with the following parameters:
>    - {% icon param-file %} *"Tabular file from metaQuantome stats or metaQuantome filter"*: `metaQuantome:stat` (output of **metaQuantome: stat** {% icon tool %})
>    - {% icon param-file %} *"Samples file"*: `metaQuantome:create samples_file` (output of **metaQuantome: create samples file** {% icon tool %})
>    - *"Mode"*: `Taxonomic analysis`
>        - *"Plot type"*: `Bar Chart (bar)`
>            - *"Taxonomic rank to restrict to in the plot"*: `genus`
>            - *"Mean intensity column name"*: `T7_mean`
>
>
>
{: .hands_on}

![T7_taxa](../../images/T7_taxa.png "Bar Chart visualization of Top 5 genus present in T7 sample."){: width="85%"}



> <hands-on-title>Bar chart visualization of Taxonomy in T4 sample.</hands-on-title>
>
> 1. {% tool [metaQuantome: visualize](toolshed.g2.bx.psu.edu/repos/galaxyp/metaquantome_viz/metaquantome_viz/2.0.0-0) %} with the following parameters:
>    - {% icon param-file %} *"Tabular file from metaQuantome stats or metaQuantome filter"*: `metaQuantome:stat` (output of **metaQuantome: stat** {% icon tool %})
>    - {% icon param-file %} *"Samples file"*: `metaQuantome:create samples_file` (output of **metaQuantome: create samples file** {% icon tool %})
>    - *"Mode"*: `Taxonomic analysis`
>        - *"Plot type"*: `Bar Chart (bar)`
>            - *"Taxonomic rank to restrict to in the plot"*: `genus`
>            - *"Mean intensity column name"*: `T4_mean`
>
>
{: .hands_on}

![T4_taxa](../../images/T4_taxa.png "Bar Chart visualization of Top 5 genus present in T4 sample."){: width="85%"}

## metaQuantome: visualize Volcano Plots

> <hands-on-title>Volcano Plot visualization of the data T4 and T2.</hands-on-title>
>
> 1. {% tool [metaQuantome: visualize](toolshed.g2.bx.psu.edu/repos/galaxyp/metaquantome_viz/metaquantome_viz/2.0.0-0) %} with the following parameters:
>    - {% icon param-file %} *"Tabular file from metaQuantome stats or metaQuantome filter"*: `metaQuantome:stat` (output of **metaQuantome: stat** {% icon tool %})
>    - {% icon param-file %} *"Samples file"*: `metaQuantome:create samples_file` (output of **metaQuantome: create samples file** {% icon tool %})
>    - *"Mode"*: `Taxonomic analysis`
>        - *"Plot type"*: `Volcano Plot (volcano)`
>            - *"Name of the fold change column in the stat dataframe"*: `log2fc_T4_over_T2`
>            - *"Name of the Corrected p-value column in the stat dataframe"*: `corrected_p_T4_over_T2`
>        - {% icon param-select %} *"Flip the fold change (i.e., multiply log fold change by -1)"*: `No`
>
>
>    > <comment-title></comment-title>
>    >
>    > Here, we show metaQuantome's Taxonomy differential abundance volcano plot. The user may select the significance threshold (0.05 by default), and terms with statistically significant fold changes are colored green and labeled. However, we can currently only compare two conditions at a time. Here we have compared T4 and T2 time points. The user can also perform comparison of T7 and T2.
>    {: .comment}
>
{: .hands_on}


![T2-T4-taxa-volcano](../../images/T2-T4-taxa-volcano.png "Volcano plot for differentially expressed taxa comparing sample T4 and T2."){: width="100%"}

> <hands-on-title>Volcano Plot visualization of the data T7 and T2.</hands-on-title>
>
> 1. {% tool [metaQuantome: visualize](toolshed.g2.bx.psu.edu/repos/galaxyp/metaquantome_viz/metaquantome_viz/2.0.0-0) %} with the following parameters:
>    - {% icon param-file %} *"Tabular file from metaQuantome stats or metaQuantome filter"*: `metaQuantome:stat` (output of **metaQuantome: stat** {% icon tool %})
>    - {% icon param-file %} *"Samples file"*: `metaQuantome:create samples_file` (output of **metaQuantome: create samples file** {% icon tool %})
>    - *"Mode"*: `Taxonomic analysis`
>        - *"Plot type"*: `Volcano Plot (volcano)`
>            - *"Name of the fold change column in the stat dataframe"*: `log2fc_T7_over_T2`
>            - *"Name of the Corrected p-value column in the stat dataframe"*: `corrected_p_T7_over_T2`
>
{: .hands_on}

![T2-T7-taxa-volcano](../../images/T2-T7-taxa-volcano.png "Volcano plot for differentially expressed taxa comparing sample T7 and T2."){: width="100%"}

> <question-title></question-title>
>
> 1. Can we choose to Flip the fold change?
>
> > <solution-title></solution-title>
> >
> > 1. Certainly, the user has the ability to flip the fold changes.
> >
> {: .solution}
>
{: .question}

# Conclusion


This completes the walkthrough of the metaQuantome taxonomy workflow. This tutorial is a guide to run the metaQuantome modules and can be used for metaproteomics research. Here, we have incorporated only two visualization modules in this workflow but we will make the heatmap and PCA plot also available. Researchers can use this workflow with their data also, please note that the tool parameters and the workflow will be needed to be modified accordingly. Also, refer to the metaQuantome data creation workflow to understand how to make files metaQuantome compatible.

This workflow was developed by the Galaxy-P team at the University of Minnesota. For more information about Galaxy-P or our ongoing work, please visit us at [galaxyp.org](https://galaxyp.org)

{: .comment}


topic: proteomics
tutorial: neoantigen-non-reference-database-generation
title: Neoantigen 1b: Non-Reference-Database-Generation
difficulty: intermediate
time_estimation: 3H
tools_json: []
requirements_json: []
tags_json: ["[label-free]"]
content_hash: ab008deed7dbc3797d01934654bba89f
last_modified: 2026-02-26T21:12:17.502568
zenodo_link: https://zenodo.org/records/14372521
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/neoantigen-non-reference-database-generation/tutorial.html

content:
Proteogenomics leverages mass spectrometry (MS)-based proteomics data alongside genomics and transcriptomics data to identify neoantigens—unique peptide sequences arising from tumor-specific mutations. In the initial section of this tutorial, we will construct a customized protein database (FASTA) using RNA-sequencing files (FASTQ) derived from tumor samples. Following this, we will conduct sequence database searches using the resultant FASTA file and MS data to identify peptides corresponding to novel proteoforms, specifically focusing on potential neoantigens. We will then assign genomic coordinates and annotations to these identified peptides and visualize the data, assessing both spectral quality and genomic localization.

In this framework, Proteogenomics incorporates RNA-Seq data to generate tailored protein sequence databases, enabling the identification of protein sequence variants, including neoantigens, through mass spectrometry analysis ({% cite Chambers_2017 %}).

![Non-Reference_Protein_Database overview workflow]({% link topics/proteomics/images/neoantigen/Non-Normal_Protein_Database_1.PNG %})

In this workflow, (A) Generation of variant database, (B) Generation of assembled protein database, and (C) Merging all databases with known HUMAN protein sequences.
> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Overview of Non-Reference Neoantigen Database Workflow

This tutorial guides users through the process of generating a non-reference variant database. It encompasses essential bioinformatics steps to identify and prepare variant-specific peptides for immunological studies. Below is an overview of each major stage:

1. **Get Data.** The workflow begins with uploading raw sequencing data, followed by a quality assessment to ensure data integrity. This step establishes a solid foundation for subsequent analyses by addressing any issues in the initial dataset.
2. **Variant Detection and Mapping.** Next, the RNA sequencing data is aligned to a reference genome using tools like HISAT2 and StringTie. Alignment events are detected with specialized tools like Freebayes, CustomProDB, and GFFcompare, which identify non-reference gene transcripts. These tools analyze the resulting alignments to characterize the gene segments in CDS, single nucleotide variants, indels, UTRs, or frameshifts.
3. **Text reformatting and Database generation.** Once variants are identified, we generate a customized database and apply various reformatting techniques to tag it, ensuring optimal usability for downstream processing.
4. **Addition of known protein sequences.** Known proteomics databases are added to the variant database to create a comprehensive database.
5. **Final Database Construction.** The workflow concludes with applying regex adjustments and other formatting functions to standardize the output. This process culminates in creating a comprehensive database of potential non-reference protein sequences, making them ready for experimental validation and clinical exploration.


# Get data

> <hands-on-title> Data Upload </hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo]({{ page.zenodo_link }}) or from
>    the shared data library (`GTN - Material` -> `{{ page.topic_name }}`
>     -> `{{ page.title }}`):
>
>    ```
>    https://zenodo.org/records/14372521/files/Homo_sapiens.GRCh38_canon.106.gtf
>    https://zenodo.org/records/14372521/files/HUMAN_CRAP.fasta
>    https://zenodo.org/records/14372521/files/RNA-Seq_Reads_1.fastqsanger.gz
>    https://zenodo.org/records/14372521/files/RNA-Seq_Reads_2.fastqsanger.gz
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
> 3. Rename the datasets
> 4. Check that the datatype
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
>
> 5. Add to each database a tag corresponding to ...
>
>    {% snippet faqs/galaxy/datasets_add_tag.md %}
>
{: .hands_on}


# Preprocessing

## Convert compressed file to uncompressed

Uncompressing data is a crucial first step in many bioinformatics workflows because raw sequencing data files, especially from high-throughput sequencing, are often stored in compressed formats (such as `.gz` or `.zip`) to save storage space and facilitate faster data transfer. Compressed files need to be uncompressed to make the data readable and accessible for analysis tools, which generally require the data to be in plain text or other compatible formats. By uncompressing these files, we ensure that downstream applications can efficiently process and analyze the raw sequencing data without compatibility issues related to compression. In this workflow, we do that for both forward and reverse files.

> <hands-on-title> Converting compressed to uncompressed </hands-on-title>
>
> 1. {% tool [Convert compressed file to uncompressed.](CONVERTER_gz_to_uncompressed) %} with the following parameters:
>    - {% icon param-file %} *"Choose compressed file"*: `RNA-Seq_Reads_1.fastqsanger.gz` (Input dataset)
>
> 2. {% tool [Convert compressed file to uncompressed.](CONVERTER_gz_to_uncompressed) %} with the following parameters:
>    - {% icon param-file %} *"Choose compressed file"*: `RNA-Seq_Reads_2.fastqsanger.gz` (Input dataset)
>
>
{: .hands_on}

# Extracting Single amino acid variants with HISAT and Freebayes
![A:Generating variant protein sequence database]({% link topics/proteomics/images/neoantigen/Non-Normal_Protein_Database_2.PNG %})
## Aligning to the reference genome with HISAT2
HISAT2 is a fast and efficient tool used in bioinformatics workflows to align sequence reads to a reference genome. In this task, HISAT2 is used to align paired-end reads against the human genome version GRCh38 (hg38). This alignment is essential for downstream analyses such as variant calling or transcript quantification. HISAT2 is configured to use default alignment and scoring options to ensure simplicity and speed, which is often suitable for general-purpose analyses.

In this workflow, HISAT2 serves the critical role of mapping raw sequencing data (reads) to a reference genome. This step is a foundation for understanding genetic variation and gene expression in the sample. By aligning the reads to a reference, HISAT2 provides a structured output that can be further analyzed in various bioinformatics applications.


> <hands-on-title> HISAT2 </hands-on-title>
>
> 1. {% tool [HISAT2](toolshed.g2.bx.psu.edu/repos/iuc/hisat2/hisat2/2.2.1+galaxy1) %} with the following parameters:
>    - *"Source for the reference genome"*: `Use a built-in genome`
>        - *"Select a reference genome"*: `Human Dec. 2013 (GRCh38/hg38) (hg38)`
>    - *"Is this a single or paired library"*: `Paired-end`
>        - {% icon param-file %} *"FASTA/Q file #1"*: `RNA-Seq_Reads_1.fastqsanger` (output of **Convert compressed file to uncompressed.** {% icon tool %})
>        - {% icon param-file %} *"FASTA/Q file #2"*: `RNA-Seq_Reads_2.fastqsanger` (output of **Convert compressed file to uncompressed.** {% icon tool %})
>        - *"Paired-end options"*: `Use default values`
>    - In *"Advanced Options"*: Keep all default settings
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why do we need to align sequencing reads to a reference genome in this workflow?
> 2. What do paired-end reads represent, and why are they important in this alignment?
>
> > <solution-title></solution-title>
> >
> > 1. Aligning sequencing reads to a reference genome is essential for identifying the location of each read within the genome, which enables analysis of gene expression levels, detection of mutations, and understanding of overall genome structure in the sample.
> > 2. Paired-end reads are sequences generated from both ends of a DNA fragment, providing more information about the fragment's size and alignment accuracy. They improve alignment quality and facilitate more accurate mapping, especially for repetitive regions, by providing context from both ends of each read.
> >
> {: .solution}
>
{: .question}

## Variant Calling with FreeBayes
FreeBayes is a variant calling tool used in bioinformatics to identify genetic variants (such as SNPs, insertions, and deletions) from aligned sequence data. In this step, FreeBayes takes the output from the HISAT2 alignment (BAM or CRAM file) and identifies variations in the aligned reads compared to the human reference genome GRCh38. By running in "Simple diploid calling" mode, FreeBayes is configured to detect variants in diploid samples, assuming two sets of chromosomes as typically found in humans.


In this workflow, FreeBayes performs the essential function of variant calling, which is critical for identifying genetic differences that could be associated with diseases, traits, or other biological characteristics. The output from FreeBayes can then be used for downstream analyses such as functional annotation or association studies.

> <hands-on-title> FreeBayes </hands-on-title>
>
> 1. {% tool [FreeBayes](toolshed.g2.bx.psu.edu/repos/devteam/freebayes/freebayes/1.3.6+galaxy0) %} with the following parameters:
>    - *"Choose the source for the reference genome"*: `Locally cached`
>        - *"Run in batch mode?"*: `Run individually`
>            - {% icon param-file %} *"BAM or CRAM dataset"*: `output_alignments` (output of **HISAT2** {% icon tool %})
>        - *"Using reference genome"*: `Human Dec. 2013 (GRCh38/hg38) (hg38)`
>    - *"Limit variant calling to a set of regions?"*: `Do not limit`
>    - *"Read coverage"*: `Use defaults`
>    - *"Choose parameter selection level"*: `1. Simple diploid calling`
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why is variant calling important after aligning reads to a reference genome?
> 2. What does "Simple diploid calling" mean in the context of FreeBayes?
>
> > <solution-title></solution-title>
> >
> > 1. Variant calling is important because it identifies specific genetic differences between the sample and the reference genome. These differences can provide insights into genetic variation, which may be associated with certain phenotypes, diseases, or treatment responses.
> > 2. "Simple diploid calling" refers to the assumption that each sample has two copies of each chromosome (diploid). FreeBayes uses this assumption to identify variants present in either one or both copies, which is appropriate for most human genetic analyses.
> >
> {: .solution}
>
{: .question}


## Customized Database generation using CustomProDB

CustomProDB is a bioinformatics tool used to generate custom protein databases that incorporate specific genetic variants found in a sample. In this task, CustomProDB utilizes genome annotation data (GRCh38/hg38), a BAM file from HISAT2 alignment, and a VCF file from FreeBayes variant calling to create a variant-aware protein database. This database includes the sample’s unique variant proteins, which is essential for personalized proteomics research.

In this workflow, CustomProDB plays a critical role in translating genetic variants identified by FreeBayes into custom protein sequences. This variant-specific database is valuable for applications in proteomics, as it allows for the detection of variant-specific peptides in mass spectrometry data. The generated outputs, including a variant FASTA file and mapping files, support downstream analyses, such as studying how genetic variations may affect protein function or abundance.

> <hands-on-title> CustomProDB </hands-on-title>
>
>
> 1. {% tool [CustomProDB](toolshed.g2.bx.psu.edu/repos/galaxyp/custom_pro_db/custom_pro_db/1.22.0) %} with the following parameters:
>    - *"Will you select a genome annotation from your history or use a built-in annotation?"*: `Use a built-in genome annotation`
>        - *"Select genome annotation"*: `Human (Ensembl 78 hsapiens) (hg38/GRCh38)`
>        - {% icon param-file %} *"BAM file"*: `output_alignments` (output of **HISAT2** {% icon tool %})
>        - {% icon param-file %} *"VCF file"*: `output_vcf` (output of **FreeBayes** {% icon tool %})
>    - *"Create a variant FASTA for short insertions and deletions"*: `Yes`
>    - *"Create SQLite files for mapping proteins to genome and summarizing variant proteins"*: `Yes`
>    - *"Create RData file of variant protein-coding sequences"*: `Yes`
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why is it necessary to create a custom protein database with genetic variants?
> 2. What is the purpose of creating SQLite and RData files in CustomProDB?
>
> > <solution-title></solution-title>
> >
> > 1. A custom protein database allows for the detection of sample-specific protein variants, which may affect protein structure and function. This is especially useful in personalized medicine, where identifying unique variants helps understand individual differences in disease mechanisms or treatment responses.
> > 2. The SQLite files map proteins to genome regions and summarize variant proteins, which supports efficient data storage and retrieval for variant analysis. The RData file contains coding sequences for variant proteins, which is useful for further statistical and bioinformatic analyses in R. Although we are not using this in our current workflow, the users can definitely use it to manipulate in their workflows.
> >
> {: .solution}
>
{: .question}

## Converting FASTA database from CustomPRODB to tabular for text processing with FASTA-to-Tabular
FASTA-to-Tabular is a tool that converts FASTA-formatted sequence files into tabular format, where each sequence entry is represented as a row with separate columns for identifiers and sequences. In this task, FASTA-to-Tabular processes the variant FASTA file generated by CustomProDB to create a tabular representation of the protein variants. This format is often easier to analyze or integrate into downstream data workflows.

In this workflow, FASTA-to-Tabular enables the conversion of variant protein sequences into a structured tabular format, which is helpful for subsequent data processing and analysis. This format allows researchers to efficiently filter, sort, or query specific sequence information and simplifies integration with other data analysis tools or databases. We do this for the indels, single nucleotide variants and rpkm databases.

> <hands-on-title> INDEL - FASTA-to-Tabular </hands-on-title>
>
> 1. {% tool [FASTA-to-Tabular](toolshed.g2.bx.psu.edu/repos/devteam/fasta_to_tabular/fasta2tab/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Convert these sequences"*: `output_indel` (output of **CustomProDB** {% icon tool %})
>
>
{: .hands_on}

> <hands-on-title> SAV - FASTA-to-Tabular </hands-on-title>
>
> 1. {% tool [FASTA-to-Tabular](toolshed.g2.bx.psu.edu/repos/devteam/fasta_to_tabular/fasta2tab/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Convert these sequences"*: `output_sav` (output of **CustomProDB** {% icon tool %})
>
>
{: .hands_on}

> <hands-on-title> RPKM - FASTA-to-Tabular </hands-on-title>
>
> 1. {% tool [FASTA-to-Tabular](toolshed.g2.bx.psu.edu/repos/devteam/fasta_to_tabular/fasta2tab/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Convert these sequences"*: `output_rpkm` (output of **CustomProDB** {% icon tool %})
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why is it helpful to convert FASTA sequences to a tabular format?
> 2. What information is typically retained when converting a FASTA file to tabular format?
>
> > <solution-title></solution-title>
> >
> > 1. Converting FASTA sequences to tabular format makes it easier to visualize, filter, and manage the data. The tabular structure allows researchers to perform sorting, searching, and data manipulation more efficiently, which is beneficial for downstream analysis.
> > 2. The conversion usually retains the sequence identifiers and the sequences themselves. Each sequence entry in the FASTA file is represented in two columns: one for the identifier and another for the actual sequence, enabling easy access to both key elements of the data.
> >
> {: .solution}
>
{: .question}

## Manipulating the headers with Column Regex Find And Replace

Column Regex Find And Replace is a tool that applies regular expression (regex) patterns to specified columns in a tabular dataset to find and replace text patterns. In this task, the tool is used on the output from the FASTA-to-Tabular step to standardize or format specific patterns in the data. By applying regex patterns to column 1, the tool identifies and modifies specific sequence identifiers or annotations, making them easier to interpret or use in further analyses.

In this workflow, Column Regex Find And Replace cleans and formats the data in a way that makes identifiers or variant descriptions consistent. This is important for data compatibility, especially when the data needs to be used across different tools or integrated into larger datasets. It ensures that all sequence labels or variant annotations follow a uniform format, which reduces errors in downstream analyses.

{% snippet faqs/galaxy/analysis_regular_expressions.md %}

> <hands-on-title> INDEL-Column Regex Find And Replace </hands-on-title>
>
> 1. {% tool [Column Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/galaxyp/regex_find_replace/regexColumn1/1.0.3) %} with the following parameters:
>    - {% icon param-file %} *"Select cells from"*: `output` (output of **FASTA-to-Tabular** {% icon tool %})
>    - *"using column"*: `1`
>    - In *"Check"*:
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(ENS[^_]+_\d+:)([ACGTacgt]+)>([ACGTacgt]+)\s*`
>            - *"Replacement"*: `\1\2_\3`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `,([A-Y]\d+[A-Y]?)\s*`
>            - *"Replacement"*: `.\1`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(ENS[^ |]*)\s*`
>            - *"Replacement"*: `\1`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(.*)$`
>            - *"Replacement"*: `generic|INDEL_\1`
>
> 2. Rename as "Indel-FASTA"
>
> 
{: .hands_on}

> <hands-on-title> SAV-Column Regex Find And Replace </hands-on-title>
>
> 1. {% tool [Column Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/galaxyp/regex_find_replace/regexColumn1/1.0.3) %} with the following parameters:
>    - {% icon param-file %} *"Select cells from"*: `output` (output of **FASTA-to-Tabular** {% icon tool %})
>    - *"using column"*: `1`
>    - In *"Check"*:
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(ENS[^_]+_\d+:)([ACGTacgt]+)>([ACGTacgt]+)\s*`
>            - *"Replacement"*: `\1\2_\3`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `,([A-Y]\d+[A-Y]?)\s*`
>            - *"Replacement"*: `.\1`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(ENS[^ |]*)\s*`
>            - *"Replacement"*: `\1`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(.*)$`
>            - *"Replacement"*: `generic|SAV_\1`
>
> 2. Rename as "SAV-FASTA"
> 
{: .hands_on}

> <hands-on-title> RPKM -Column Regex Find And Replace </hands-on-title>
>
> 1. {% tool [Column Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/galaxyp/regex_find_replace/regexColumn1/1.0.3) %} with the following parameters:
>    - {% icon param-file %} *"Select cells from"*: `output` (output of **FASTA-to-Tabular** {% icon tool %})
>    - *"using column"*: `1`
>    - In *"Check"*:
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(ENS[^_]+_\d+:)([ACGTacgt]+)>([ACGTacgt]+)\s*`
>            - *"Replacement"*: `\1\2_\3`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `,([A-Y]\d+[A-Y]?)\s*`
>            - *"Replacement"*: `.\1`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(ENS[^ |]*)\s*`
>            - *"Replacement"*: `\1`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(.*)$`
>            - *"Replacement"*: `generic|RPKM_\1`
>
> 2. Rename as "RPKM-FASTA"
> 
{: .hands_on}

> <question-title></question-title>
>
> 1. Why use regular expressions to modify text in a column?
> 2. What is the purpose of the `generic|INDEL_\1` replacement pattern ( or any other pattern) in this tool?
>
> > <solution-title></solution-title>
> >
> > 1. Regular expressions allow for flexible, precise pattern matching, enabling the user to find and replace complex text patterns within a column. This is useful for standardizing data formats, extracting relevant portions, or appending additional information to entries.
> > 2. The `generic|INDEL_\1` replacement pattern prepends `generic|INDEL_` to each entry in the column, standardizing the label format and clarifying that each entry is related to an insertion or deletion (INDEL) variant. This makes it clear in downstream analysis that the sequence is variant-specific.
> >
> {: .solution}
>
{: .question}

## Converting the manipulated tabular files to FASTA with Tabular-to-FASTA
Tabular-to-FASTA is a tool that converts tabular data back into FASTA format, where each entry has a title and a sequence. In this step, the tool uses the processed output from the Column Regex Find And Replace step to create a FASTA file. Column 1 is set as the title (identifier) for each sequence, and column 2 contains the sequence data. This conversion is useful when the data needs to be returned to a standard sequence format for further bioinformatics analyses.

In this workflow, Tabular-to-FASTA converts the formatted tabular data back into a FASTA file, making it compatible with tools that require FASTA input for further analysis. This step enables the standardized, cleaned sequences from previous steps to be utilized in additional bioinformatics workflows or databases, maintaining the variant-specific information in a commonly used format. We do this for all the tabular files (SAV, INDEL, and RPKM).

> <hands-on-title> INDEL-Tabular-to-FASTA </hands-on-title>
>
> 1. {% tool [Tabular-to-FASTA](toolshed.g2.bx.psu.edu/repos/devteam/tabular_to_fasta/tab2fasta/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Tab-delimited file"*: `out_file1` (output of **Column Regex Find And Replace** {% icon tool %})
>    - *"Title column(s)"*: `1`
>    - *"Sequence column"*: `2`
>
> 2. Rename as "Indel-FASTA"
>
> 
{: .hands_on}

> <hands-on-title> SAV-Tabular-to-FASTA </hands-on-title>
>
> 1. {% tool [Tabular-to-FASTA](toolshed.g2.bx.psu.edu/repos/devteam/tabular_to_fasta/tab2fasta/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Tab-delimited file"*: `out_file1` (output of **Column Regex Find And Replace** {% icon tool %})
>    - *"Title column(s)"*: `1`
>    - *"Sequence column"*: `2`
>
> 2. Rename as "SAV-FASTA"
>
> 
{: .hands_on}

> <hands-on-title> RPKM-Tabular-to-FASTA </hands-on-title>
>
> 1. {% tool [Tabular-to-FASTA](toolshed.g2.bx.psu.edu/repos/devteam/tabular_to_fasta/tab2fasta/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Tab-delimited file"*: `out_file1` (output of **Column Regex Find And Replace** {% icon tool %})
>    - *"Title column(s)"*: `1`
>    - *"Sequence column"*: `2`
>
> 2. Rename as "RPKM-FASTA"
>
>    
{: .hands_on}

> <question-title></question-title>
>
> 1. Why is it important to convert tabular data back to FASTA format?
> 2. What information is typically included in the title and sequence columns in a FASTA file?
>
> > <solution-title></solution-title>
> >
> > 1. Converting tabular data back to FASTA format makes it compatible with a wide range of bioinformatics tools that require FASTA input. It allows the sequence data, now standardized and labeled, to be used in additional analyses, such as protein structure prediction or functional annotation.
> > 2. The title column usually contains a unique identifier or label for each sequence, often including relevant annotations. The sequence column contains the actual DNA, RNA, or protein sequence. Together, these provide essential details for recognizing and analyzing each sequence entry in downstream applications.
> >
> {: .solution}
>
{: .question}

## Merging Single amino acid variant databases with FASTA Merge Files and Filter Unique Sequences
FASTA Merge Files and Filter Unique Sequences is a tool that combines multiple FASTA files into a single file and removes any duplicate sequences, keeping unique entries. In this task, the tool takes the FASTA file generated from the Tabular-to-FASTA step and merges it with any other FASTA files in the input list. The tool then filters the sequences to ensure that unique sequences are retained in the final output, which is important for reducing redundancy in the dataset.

In this workflow, FASTA Merge Files and Filter Unique Sequences consolidate all sequence data into a single, non-redundant FASTA file. This step is essential for removing duplicate sequences, which helps streamline the dataset for further analysis. A unique sequence file reduces computational load and minimizes potential biases in downstream applications that could be affected by redundant data. We are merging the indel, sav, and rpkm databases in this step.


> <hands-on-title> FASTA Merge Files and Filter Unique Sequences </hands-on-title>
>
> 1. {% tool [FASTA Merge Files and Filter Unique Sequences](toolshed.g2.bx.psu.edu/repos/galaxyp/fasta_merge_files_and_filter_unique_sequences/fasta_merge_files_and_filter_unique_sequences/1.2.0) %} with the following parameters:
>    - *"Run in batch mode?"*: `Merge individual FASTAs (output collection if the input is a collection)`
>        - In *"Input FASTA File(s)"*:
>            - {% icon param-repeat %} *"Insert Input FASTA File(s)"*
>                - {% icon param-file %} *"FASTA File"*: `SAV-FASTA` (output of **Tabular-to-FASTA** {% icon tool %})
>        - In *"Input FASTA File(s)"*:
>            - {% icon param-repeat %} *"Insert Input FASTA File(s)"*
>                - {% icon param-file %} *"FASTA File"*: `INDEL-FASTA` (output of **Tabular-to-FASTA** {% icon tool %})
>        - In *"Input FASTA File(s)"*:
>            - {% icon param-repeat %} *"Insert Input FASTA File(s)"*
>                - {% icon param-file %} *"FASTA File"*: `RPKM-FASTA` (output of **Tabular-to-FASTA** {% icon tool %})>
> 
> 2. Rename as `non-reference_CustomProDB_FASTA`
>    
{: .hands_on}

> <question-title></question-title>
>
> 1. Why is it useful to filter out duplicate sequences from a FASTA file?
> 2. What does “Run in batch mode” mean in the context of merging FASTA files?
>
> > <solution-title></solution-title>
> >
> > 1. Filtering out duplicate sequences ensures that unique sequences are included, which reduces redundancy in the dataset. This can enhance computational efficiency, prevent overrepresentation of certain sequences, and ensure that analyses are based on a unique set of entries.
> > 2. In this context, “Run in batch mode” means merging multiple FASTA files into a single collection, even if the input consists of multiple individual FASTA files. This mode consolidates the files to make them easier to manage and analyze as a single dataset.
> >
> {: .solution}
>
{: .question}

# Extracting Assembled sequences with Stringtie and GFF compare
![Assembled sequence database generation]({% link topics/proteomics/images/neoantigen/Non-Normal_Protein_Database_3.PNG %})

## Assemble with StringTie

StringTie is a tool used for assembling RNA-Seq alignments into potential transcripts, estimating their abundances, and creating GTF files of transcript structures. In this task, StringTie takes short reads mapped by HISAT2 as input and uses a reference GTF/GFF3 file to guide the assembly of transcripts. This guidance enables StringTie to produce accurate transcript structures that can be useful for differential expression analysis and downstream functional analyses.

In this workflow, StringTie is responsible for reconstructing transcript structures based on RNA-Seq alignments. By using a reference file to guide the assembly, StringTie improves the accuracy of the transcript annotations, which is valuable for comparing expression levels across conditions or identifying novel transcript variants. This step provides a foundation for gene expression quantification and other transcriptome analyses.

> <hands-on-title> StringTie </hands-on-title>
>
> 1. {% tool [StringTie](toolshed.g2.bx.psu.edu/repos/iuc/stringtie/stringtie/2.2.3+galaxy0) %} with the following parameters:
>    - *"Input options"*: `Short reads`
>        - {% icon param-file %} *"Input short mapped reads"*: `output_alignments` (output of **HISAT2** {% icon tool %})
>    - *"Use a reference file to guide assembly?"*: `Use reference GTF/GFF3`
>        - *"Reference file"*: `Use a file from history`
>            - {% icon param-file %} *"GTF/GFF3 dataset to guide assembly"*: `Homo_sapiens.GRCh38_canon.106.gtf` (Input dataset)
>        - *"Output files for differential expression?"*: `No additional output`
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. Why is a reference GTF/GFF3 file used to guide the assembly in StringTie?
> 2. What type of input does StringTie require to perform transcript assembly?
>
> > <solution-title></solution-title>
> >
> > 1. A reference GTF/GFF3 file provides known transcript information, helping StringTie to assemble transcripts more accurately. This guidance ensures that the assembled transcript structures are consistent with existing gene models, enhancing the reliability of the results and enabling easier comparison across different samples.
> > 2. StringTie requires mapped short reads as input, typically in BAM format, which contain RNA-Seq alignments. These mapped reads are used to reconstruct transcript structures, and a reference annotation file can be optionally provided to guide the assembly process.
> >
> {: .solution}
>
{: .question}


## Accuracy of transcript predictions with GffCompare

GffCompare is a tool used to compare and evaluate the accuracy of transcript predictions by comparing GTF/GFF files with reference annotations. In this task, GffCompare takes the transcript assembly output from StringTie and compares it against a reference annotation file. This allows users to identify novel transcripts, assess the accuracy of the assembly, and evaluate the completeness of the transcript predictions relative to known annotations.

In this workflow, GffCompare assesses the quality of the transcript assembly produced by StringTie by aligning it with a known reference annotation. This step helps determine which assembled transcripts match known genes, which are new variants, and which might represent entirely novel transcripts. By providing accuracy metrics and categorizing the transcripts, GffCompare facilitates the interpretation of transcriptomic data, especially in studies aiming to identify new gene isoforms or validate assembled structures.

> <hands-on-title> GffCompare </hands-on-title>
>
> 1. {% tool [GffCompare](toolshed.g2.bx.psu.edu/repos/iuc/gffcompare/gffcompare/0.12.6+galaxy0) %} with the following parameters:
>    - {% icon param-file %} *"GTF inputs for comparison"*: `output_gtf` (output of **StringTie** {% icon tool %})
>    - *"Use reference annotation"*: `Yes`
>        - *"Choose the source for the reference annotation"*: `History`
>            - {% icon param-file %} *"Reference annotation"*: `Homo_sapiens.GRCh38_canon.106.gtf` (Input dataset)
>    - *"Use sequence data"*: `No`
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why is it important to compare assembled transcripts to a reference annotation?
> 2. What information can GffCompare provide about the assembled transcripts?
>
> > <solution-title></solution-title>
> >
> > 1. Comparing assembled transcripts to a reference annotation helps identify which transcripts correspond to known genes, which ones are novel isoforms, and which are new transcript structures. This process validates the transcript assembly and provides confidence in the biological relevance of the findings.
> > 2. GffCompare provides information on how well the assembled transcripts match the reference annotation, categorizing transcripts as known, novel isoforms, or completely new structures. It also provides accuracy metrics, which are useful for evaluating the quality of the transcript assembly.
> >
> {: .solution}
>
{: .question}



## BED file conversion with Convert gffCompare annotated GTF to BED
Convert gffCompare Annotated GTF to BED is a tool that converts GTF files annotated by GffCompare into BED format. BED (Browser Extensible Data) files are commonly used for displaying transcript annotations in genome browsers, providing a simpler and more compact representation than GTF files. In this task, the tool converts GffCompare’s annotated GTF output, allowing visualization and further analyses of transcript annotations in a format compatible with many genomic tools and browsers.

In this workflow, converting GffCompare's annotated GTF output to BED format allows the user to visualize and analyze the assembled and annotated transcripts in genome browsers. This step facilitates the interpretation of transcript locations, boundaries, and structures in an accessible and efficient manner, especially for assessing the genomic context and identifying patterns across the genome.

> <hands-on-title> Convert gffCompare annotated GTF to BED </hands-on-title>
>
> 1. {% tool [Convert gffCompare annotated GTF to BED](toolshed.g2.bx.psu.edu/repos/galaxyp/gffcompare_to_bed/gffcompare_to_bed/0.2.1) %} with the following parameters:
>    - {% icon param-file %} *"GTF annotated by gffCompare"*: `transcripts_annotated` (output of **GffCompare** {% icon tool %})
>    - *"filter gffCompare class_codes to convert"*:  
`j : Potentially novel isoform (fragment): at least one splice junction is shared with a reference transcript`  
`e : Single exon transfrag overlapping a reference exon and at least 10 bp of a reference intron, indicating a possible pre-mRNA fragment.`  
`i : A transfrag falling entirely within a reference intron`  
`p : Possible polymerase run-on fragment (within 2Kbases of a reference transcript)`  
`u : Unknown, intergenic transcript`  
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why is BED format useful for viewing transcript annotations?
> 2. What does the option "filter gffCompare class_codes to convert" allow you to do?
>
> > <solution-title></solution-title>
> >
> > 1. BED format provides a simplified, compact way to represent transcript locations, which is widely compatible with genome browsers. This format allows easy visualization of transcript structures, making it efficient for examining the genomic context of the assembled transcripts.
> > 2. The "filter gffCompare class_codes to convert" option lets users specify which types of transcript classes (such as known or novel isoforms) to include in the BED output. This filter is useful for focusing on specific categories of transcripts, such as novel structures, for further analysis or visualization.
> >
> {: .solution}
>
{: .question}



## Translating BED to FASTA sequences with Translate BED transcripts
Translate BED transcripts is a tool that translates BED files containing transcript annotations into FASTA sequences. This tool uses a reference genomic sequence (in this case, a 2bit file) to extract the nucleotide sequences corresponding to the regions defined in the BED file. The output is a FASTA file, which contains the translated sequences of the annotated transcripts, allowing further analysis of their sequence composition.

In this workflow, translating BED files to FASTA sequences is essential for obtaining the actual nucleotide sequences of the annotated transcripts. This step enables the user to analyze the sequences further, for example, by identifying functional regions, and sequence motifs, or conducting downstream analysis like mutation detection or variant calling.

> <hands-on-title> Translate BED transcripts </hands-on-title>
>
> 1. {% tool [Translate BED transcripts](toolshed.g2.bx.psu.edu/repos/galaxyp/translate_bed/translate_bed/0.1.0) %} with the following parameters:
>    - {% icon param-file %} *"A BED file with 12 columns"*: `output` (output of **Convert gffCompare annotated GTF to BED** {% icon tool %})
>    - *"Source for Genomic Sequence Data"*: `Locally cached twobit`
>        - *"Select reference 2bit file"*: `hg38`
>    - In *"Fasta ID Options"*:
>        - *"fasta ID source, e.g. generic"*: `generic`
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why do we use a 2bit reference file in the Translate BED Transcripts tool?
> 2. What is the role of the "Fasta ID source" option in this tool?
>
> > <solution-title></solution-title>
> >
> > 1. The 2bit reference file provides a compact and efficient representation of the reference genome sequence, which is used to extract the nucleotide sequences corresponding to the BED annotations. It allows the tool to quickly and accurately retrieve the relevant genomic sequences for translation into FASTA format.
> > 2. The "Fasta ID source" option specifies the format for the FASTA sequence identifiers. The "generic" option assigns a simple generic identifier to each sequence. This helps in organizing the sequences in a standardized manner, especially if specific identifiers are not available or needed for further analysis.
> >
> {: .solution}
>
{: .question}

## Generating a genomic coordinate file with bed to protein map
The bed to protein map tool translates the genomic coordinates in a BED file into protein sequences. The input BED file should have 12 columns, where the "thickStart" and "thickEnd" fields define the protein-coding regions. This tool uses these coordinates to map the nucleotide sequences (from the BED regions) into protein sequences, assuming the provided BED file contains protein-coding regions.

This tool is important for converting the genomic annotations (in BED format) that correspond to protein-coding regions into actual protein sequences. This step is critical for downstream protein analysis, such as protein function prediction, domain identification, or understanding the consequences of genetic mutations at the protein level.

> <hands-on-title> bed to protein map </hands-on-title>
>
> 1. {% tool [bed to protein map](toolshed.g2.bx.psu.edu/repos/galaxyp/bed_to_protein_map/bed_to_protein_map/0.2.0) %} with the following parameters:
>    - {% icon param-file %} *"A BED file with 12 columns, thickStart and thickEnd define protein coding region"*: `translation_bed` (output of **Translate BED transcripts** {% icon tool %})
>
>
{: .hands_on}

> <question-title></question-title>
>
> 1. What do the "thickStart" and "thickEnd" fields in the BED file represent in the context of protein mapping?
> 2. Why is it important to ensure that the BED file contains the correct "protein-coding regions" for this tool to function properly?
>
> > <solution-title></solution-title>
> >
> > 1. The "thickStart" and "thickEnd" fields in the BED file specify the start and end coordinates of the protein-coding regions within the genome. These fields define the portion of the transcript that will be used for translating nucleotide sequences into protein sequences.
> > 2. The accuracy of the protein sequences produced by the bed-to-protein map tool depends on correctly identifying protein-coding regions in the BED file. If the file includes non-coding regions or incorrectly defined coordinates, the resulting protein sequences may not be accurate, which could lead to erroneous conclusions in downstream analyses.
> >
> {: .solution}
>
{: .question}


# Consolidating Chromosomal Coordinates for Somatic Mutations

In this section, we combine genomic mapping results, variant annotations, and transcript-level BED information into a single unified table. This consolidated file provides the complete chromosomal locations and functional context for all somatic mutations identified in the workflow, ensuring that downstream analyses have access to harmonized and comprehensive coordinate information.

## Extracting Genomic Coordinates Using Genomic_sqlite-to-tabular

The **Genomic_sqlite-to-tabular** tool allows users to query a genomic mapping SQLite database and extract relevant information into a readable tabular format. In the context of neoantigen discovery, this step is critical for linking validated peptides back to their precise genomic coordinates. The mapping information enables downstream interpretation of variant location, coding sequence position, strand orientation, and the broader genomic context of each neopeptide.

By executing an SQL query against the `genomic_mapping` table within the SQLite database, this tool retrieves protein names, chromosome identifiers, coding sequence coordinates, and strand information. These genomic annotations are essential for variant interpretation, peptide validation, and visualizing peptide-to-genome relationships in downstream analyses.

> <hands-on-title> Genomic_sqlite-to-tabular </hands-on-title>
>
> 1. {% tool [SQLite to tabular](toolshed.g2.bx.psu.edu/repos/galaxyp/sqlite_to_tabular/sqlite_to_tabular/2.0.0) %} with the following parameters:
>    - {% icon param-file %} *"SQLite Database"* (sqlite): `genomic_mapping.sqlite`  
>    - *"SQL query"*:
>      ```
>      SELECT pro_name,
>             chr_name,
>             cds_chr_start - 1,
>             cds_chr_end,
>             strand,
>             cds_start - 1,
>             cds_end
>      FROM genomic_mapping
>      ORDER BY pro_name, cds_start, cds_end
>      ```
>
> 
{: .hands_on}

> <question-title></question-title>
>
> 1. Why do we extract genomic coordinates for validated neopeptides?
> 2. What is the significance of subtracting **1** from certain coordinate fields in the SQL query?
>
> > <solution-title></solution-title>
> >
> > 1. Extracting genomic coordinates enables precise mapping of neopeptides back to the genome, supporting annotation of variant locations, integration with proteogenomic pipelines, and verification against reference genomes. This information is important for understanding how genetic variants alter protein sequences and contribute to neoantigen formation.
> > 2. Subtracting **1** adjusts the coordinates to **0-based indexing**, which is required by many downstream tools and genomic browsers. This ensures consistency when merging or comparing genomic ranges across tools that use different coordinate conventions.
> >
> {: .solution}
>
{: .question}

## Extracting Variant Annotations Using Variant_sqlite-to-tabular

The **Variant_sqlite-to-tabular** tool is used to query the variant annotation SQLite database and convert key information into a tabular format. This step is essential for linking validated neopeptides to the underlying genomic variants that generated them. The resulting table provides protein-level variant identifiers, reference protein names, CIGAR strings describing how the variant affects the coding sequence, and high-level functional annotations.

These annotations help researchers interpret the biological relevance of each variant, determine whether a mutation alters coding sequence structure, and assess how genomic alterations translate into observed neoantigen peptides. This table also supports downstream reporting, visualization, and filtering of variant-driven neopeptides.

> <hands-on-title> Variant_sqlite-to-tabular </hands-on-title>
>
> 1. {% tool [SQLite to tabular](toolshed.g2.bx.psu.edu/repos/galaxyp/sqlite_to_tabular/sqlite_to_tabular/2.0.0) %} with the following parameters:
>    - {% icon param-file %} *"SQLite Database"* (sqlite): `variant_annotation.sqlite` (Variant SQLite from **CustomProDB** {% icon tool %})
>    - *"SQL query"*:
>      ```
>      SELECT var_pro_name,
>             pro_name,
>             cigar,
>             annotation
>      FROM variant_annotation
>      ```
>
> 
{: .hands_on}

> <question-title></question-title>
>
> 1. What information does the `variant_annotation` table provide for downstream neoantigen analysis?  
> 2. How can CIGAR strings help interpret the effect of a variant on protein structure?
>
> > <solution-title></solution-title>
> >
> > 1. The table provides variant-specific protein identifiers, reference protein IDs, functional annotations, and CIGAR strings showing how the variant alters the coding sequence. This information helps connect peptide-level findings to genomic alterations.  
> >
> > 2. CIGAR strings summarize how insertions, deletions, or mismatches occur relative to the reference sequence. Understanding these structural changes helps determine whether a mutation may create a novel peptide sequence that could function as a neoantigen.  
> >
> {: .solution}
>
{: .question}


## Cleaning Genomic Annotation Fields Using Column Regex Find and Replace

The **Column Regex Find and Replace** tool is used to normalize and clean genomic annotation fields extracted from the SQLite database. This step ensures that variant identifiers, amino acid changes, and Ensembl protein IDs follow a consistent formatting structure required for downstream integration and matching. Regular expressions (regex) are applied sequentially to correct formatting artifacts, extract relevant mutation information, and harmonize naming conventions.

These transformations are particularly important when variant annotations come from diverse sources and contain mixed notation styles (e.g., Ensembl identifiers, nucleotide substitutions, or amino acid changes). Applying standardized regex cleaning ensures that variant identifiers can be matched correctly across tables, peptide lists, and annotation files later in the workflow.

> <hands-on-title> Column Regex Find and Replace </hands-on-title>
>
> 1. {% tool [Column Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/bgruening/column_regex_find_replace/column_regex_find_and_replace/1.0.3) %} with the following parameters:
>    - {% icon param-file %} *"Select cells from"*: `CustomProDB_Genomic_SQLlite` (Genomic SQLite from **CustomProDB** {% icon tool %})
>    - *"using column"*: `1`
>    - In *"Check"*:
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(ENS[^_]+_\d+:)([ACGTacgt]+)>([ACGTacgt]+)\s*`
>            - *"Replacement"*: `\1\2_\3`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `,([A-Y]\d+[A-Y]?)\s*`
>            - *"Replacement"*: `.\1`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(ENS[^ |]*)\s*`
>            - *"Replacement"*: `\1`
>  
{: .hands_on}

> <question-title></question-title>
>
> 1. Why do we need to apply multiple regex rules to the genomic annotation column?  
> 2. What types of inconsistencies do these regex transformations correct?
>
> > <solution-title></solution-title>
> >
> > 1. Genomic annotations often contain multiple layers of information—nucleotide substitutions, amino acid changes, and various Ensembl IDs—which may appear in inconsistent formats. Applying sequential regex rules ensures a standardized format for matching and downstream processing.  
> >
> > 2. These regex rules correct formatting issues such as removing trailing whitespace, converting nucleotide substitutions into a consistent notation, normalizing amino acid change strings, and ensuring Ensembl protein identifiers are properly extracted. This normalization supports accurate linking between variant, peptide, and genomic data.  
> >
> {: .solution}
>
{: .question}

## Cleaning Variant Annotation Fields Using Column Regex Find and Replace

The **Column Regex Find and Replace** tool is used here to standardize variant-level annotations extracted from the variant SQLite database. Variant identifiers often contain multiple components—such as Ensembl protein IDs, amino acid substitutions, and annotation tags—that may appear in inconsistent formats. This step uses a series of regex rules to normalize these fields, ensuring they can be accurately merged with neopeptide tables and downstream variant annotation outputs.

By harmonizing mutation notation and cleaning extraneous characters or formatting artifacts, this step enables reliable matching across the variant annotation table, validated peptides, and genomic mapping results. Standardized variant IDs are essential for constructing unified annotations and ensuring consistency throughout the neoantigen identification pipeline.

> <hands-on-title> Column Regex Find and Replace (Variant SQLite)</hands-on-title>
>
> 1. {% tool [Column Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/bgruening/column_regex_find_replace/column_regex_find_and_replace/1.0.3) %} with the following parameters:
>    - {% icon param-file %} *"Select cells from"*: `CustomProDB_VARIANT_SQLite` (Variant SQLite from **CustomProDB** {% icon tool %})
>    - *"using column"*: `1`
>    - In *"Check"*:
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(ENS[^_]+_\d+:)([ACGTacgt]+)>([ACGTacgt]+)\s*`
>            - *"Replacement"*: `\1\2_\3`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `,([A-Y]\d+[A-Y]?)\s*`
>            - *"Replacement"*: `.\1`
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `^(ENS[^ |]*)\s*`
>            - *"Replacement"*: `\1`
> 
{: .hands_on}

> <question-title></question-title>
>
> 1. Why is it important to standardize variant identifiers before merging datasets?  
> 2. What types of format inconsistencies do these regex rules help resolve?
>
> > <solution-title></s


topic: proteomics
tutorial: protein-id-oms
title: Peptide and Protein ID using OpenMS tools
difficulty: advanced
time_estimation: 45m
tools_json: []
requirements_json: []
tags_json: ["[DDA, HeLa]"]
content_hash: b9d3e73e13eb01eb7fb8e7537333c650
last_modified: 2026-02-26T21:12:17.506653
zenodo_link: https://zenodo.org/record/546301
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/protein-id-oms/tutorial.html

content:
Identifying the proteins contained in a sample is an important step in any proteomic experiment. However, in most settings, proteins are digested to peptides prior to LC-MS/MS analysis. In this so-called "bottom-up" procedure, only peptide masses are measured. Therefore, protein identification cannot be performed directly from raw data, but is a multi-step process:

1. Raw data preparations
2. Peptide-to-Spectrum matching
3. Peptide inference
4. Protein inference

A plethora of different software solutions exists for each step. In this tutorial, we will use ***msconvert*** {% icon tool %}  for raw data conversion and tools from the [OpenMS software suite](https://openms.de) for all other steps. We will use one peptide search engine at first and later on show how to expand the workflow for using multiple search engines. Protein inference will be performed with the Fido algorithm ([Serang et al, JPR, (2010)](https://www.ncbi.nlm.nih.gov/pubmed/20712337)).
This tutorial covers peptide and protein **identification** only, but you may use the output of this tutorial for the [tutorial on protein quantitation]({{site.baseurl}}/topics/proteomics/tutorials/protein-quant-sil/tutorial.html).

For an alternative protein ID workflow using the [Compomics](https://compomics.com/) tools [SearchGUI](https://compomics.github.io/projects/searchgui.html) and [PeptideShaker](https://compomics.github.io/projects/peptide-shaker.html), please consult [this tutorial]({{site.baseurl}}/topics/proteomics/tutorials/protein-id-sg-ps/tutorial.html).
The latter tutorial does not allow to continue with the tutorial on protein quantitation.

# Input data


As an example dataset, we will use an LC-MS/MS analysis of HeLa cell lysate published
in [Vaudel et al., 2014, Proteomics](https://www.ncbi.nlm.nih.gov/pubmed/24678044). Detailed information
about the dataset can be found on [PRIDE](https://www.ebi.ac.uk/pride/archive/projects/PXD000674).
For step 2 we will use a validated human Uniprot FASTA database with appended decoys.

If you already completed the tutorial on [Database Handling]({{site.baseurl}}/topics/proteomics/tutorials/database-handling/tutorial.html)
you can use the constructed database including decoys.
You can find a prepared database, as well as the input LC-MS/MS data in different file formats on [Zenodo](https://zenodo.org/record/796184).

> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Preparing Raw Data

Raw data conversion is the first step of any proteomic data analysis. The most common converter is MSConvert from the [ProteoWizard software suite](http://proteowizard.sourceforge.net/), the format to convert to is mzML.
Due to licensing reasons, MSConvert runs only on windows systems and will not work on most Galaxy servers.

Depending on your machine settings, raw data will be generated either in profile mode or centroid mode. For most peptide search engines, the MS2 data have to be converted to centroid mode, a process called "peak picking" or "centroiding".
Machine vendors offer algorithms to extract peaks from profile raw data. Those are integrated in ***msconvert*** {% icon tool %} and can be run in parallel to the mzML conversion.
However, the OpenMS tool ***PeakPickerHiRes*** {% icon tool %} is reported to generate slightly better results ([Lange et al., 2006, Pac Symp Biocomput](https://www.ncbi.nlm.nih.gov/pubmed/17094243)) and is therefore recommended for quantitative studies ([Vaudel et al., 2010, Proteomics](https://www.ncbi.nlm.nih.gov/pubmed/19953549)).
If your data were generated on a low resolution mass spectrometer, use **PeakPickerWavelet** {% icon tool %} instead.

> <hands-on-title>File Conversion and Peak Picking</hands-on-title>
>
> We provide the [input data](https://zenodo.org/record/796184) in the original `raw` format and also already converted to `mzML`. If **msconvert** {% icon tool %} does not run on your Galaxy instance, please download the preconverted `mzML` as an input and continue with step 5 of the following hands-on training.
>
> 1. Create a new history for this Peptide and Protein ID exercise.
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
> 2. Load one of the example datasets into your history from Zenodo
>
>    ```
>    https://zenodo.org/record/892005/files/qExactive01819.raw
>    https://zenodo.org/record/892005/files/qExactive01819_profile.mzml
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
> 3. Rename the dataset to something meaningful
>
> 4. Run {% tool [msconvert](toolshed.g2.bx.psu.edu/repos/galaxyp/msconvert/msconvert/3.0.19052.1) %} on the training `raw` file to convert to the `mzML` format
>    - {% icon param-file %} *"Input unrefined MS data"*: imported `raw` file
>    - *"Do you agree to the vendor licenses"*: set to `Yes`
>    - *"Output Type"*: set to `mzML`
>
> 5. Run {% tool [PeakPickerHiRes](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_peakpickerhires/PeakPickerHiRes/2.3.0) %} on the resulting file
>    - {% icon param-file %} *"input profile data file": output of **msconvert** or `mzML` file
>    - In *"param_algorithm_ms_levels"*
>      - {% icon param-repeat %} Click on *"Insert param_algorithm_ms_levels"*
>      - In *"1: param_algorithm_ms_levels"*
>         - *"List of MS levels for which the peak picking is applied"*: `2`
>
>           Peak picking will only be performed on MS2 level.
>
>   > <comment-title>Local Use of MSConvert</comment-title>
>   > The vendor libraries used by MSConvert are only licensed for Windows systems and are therefore rarely implemented in Galaxy instances. If **msconvert** {% icon tool %} is not available in your Galaxy instance, please install the software on a Windows computer and run the conversion locally. You can find a [detailed description of the necessary steps on this site](https://compomics.com/bioinformatics-for-proteomics/identification/) ("Peak List Generation"). Afterwards, upload the resulting mzML file to your Galaxy history.
>   {: .comment}
>
{: .hands_on}

> <comment-title>MS2 peak picking during data acquisition</comment-title>
> MS2 peaks are often acquired in centroided mode in first place. The profile data are converted to centroided mode already during data acquisition, resulting in MS2-centroided `raw` files. If your MS2 data are already centroided, simply omit the peak picking step.
{: .comment}

# Peptide Identification

MS/MS experiments identify peptides by isolating them and subsequently colliding them with a gas for fragmentation. This method generates a spectrum of peptide fragment masses for each isolated peptide - an MS2 spectrum.
To find out the peptide sequences, the MS2 spectrum is compared to a theoretical spectrum generated from a protein database. This step is called peptide-to-spectrum (also: spectrum-to-sequence) matching. Accordingly, a peptide that is successfully matched to a sequence is termed PSM (Peptide-Spectrum-Match). There can be multiple PSMs per peptide, if the peptide was fragmented several times.

Different peptide search engines have been developed to fulfill the matching procedure. Here, we will use the search engine [X!Tandem](https://www.ncbi.nlm.nih.gov/pubmed/14976030). OpenMS provides "adapters" (wrappers) for several other peptide search engines, like MSGF+ or OMSSA. You may replace the XTandemAdapter by another search engine of your choice.

> <hands-on-title>Peptide Identification</hands-on-title>
>
> 1. Copy the prepared protein database (Human database including cRAP contaminants and decoys) from the tutorial [Database Handling]({% link topics/proteomics/tutorials/database-handling/tutorial.md %}) into your current history by using the multiple history view
>
>   > <comment-title>You did not run the Database Handling first?</comment-title>
>   > You can upload the ready-made database from Zenodo
>   >
>   > ```
>   > https://zenodo.org/record/892005/files/Human_database_including_decoys_%28cRAP_added%29.fasta
>   > ```
>   {: .comment}
>
> 2. Run {% tool [XTandemAdapter](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_xtandemadapter/XTandemAdapter/2.6+galaxy0) %} with:
>   - {% icon param-file %} *"Input file containing MS2 spectra"*: MS2-centroided mzML file
>   - {% icon param-file %} *"FASTA file"*: FASTA protein database
>   - *"Fragment mass error"*: `10`
>   - *"Fragment monoisotopic mass error units"*: `ppm`
>   - *"Optional Outputs"* select `out (Output file containing search results)`
>
> 3. Run {% tool [FileInfo](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_fileinfo/FileInfo/2.6+galaxy0) %}
>    - {% icon param-file %} *"input file"*: **XTandem** output
{: .hands_on}

> <comment-title>Settings for labelled data</comment-title>
> Several common quantitation methods are based on labels e.g. SILAC, TMT, iTRAQ, Dimethyl. Those labels must be specified in the search engine as a (variable) modification. See also [Peptide and Protein Quantification via Stable Isotope Labelling]({% link topics/proteomics/tutorials/protein-quant-sil/tutorial.md %}).
{: .comment}

> <comment-title>Advanced Search Engine Parameters</comment-title>
> The OpenMS adapters do not always allow to set every option of the underlying search engine. If an option is missing, you may also run the search engine locally or by using a Galaxy wrapper. Afterwards, convert the search engine output to the OpenMS format `idXML` by running **IDFileConverter** {% icon tool %}.
>
> The search engine X!Tandem features some more advanced options than the ones reflected in the **XTandemAdapter** {% icon tool %}. If you need those advanced options, the **XTandemAdapter** {% icon tool %} allows for the optional input of a classic X!Tandem parameter file. Upload your parameter file to the history and use it as an input in the field `Default X!Tandem configuration file`. You may also set the option *"ignore_adapter_param"* to `Yes` to overwrite all options set by the GUI.
{: .comment}

# Peptide FDR filtering

The next step of peptide identification is to decide which PSMs will be used for protein inference. Measured MS2 spectra never perfectly fit the theoretical spectra. Therefore, peptide search engines calculate a score which indicates how well the measured MS2 spectrum was fitting the theoretical spectrum. How do we decide which PSMs are likely true and which are false?

In proteomics, this decision is typically done by calculating false discovery rates (FDRs). Remember that the database we were using for peptide-to-spectrum matching consisted not only of true proteins, but also the same number of "fake entries", the so-called decoys. Those decoys can now be used to estimate the number of false identifications in the list of PSMs.
The calculation is based on a simple assumption: for every decoy peptide identified with a given score, we expect one false positive with at least the same score.
The false discovery rate is therefore defined as the number of false discoveries (decoy hits) divided by the number of false and correct discoveries (both target and decoy hits) at a given score threshold.

To calculate FDRs, we first have to annotate the identified peptides to determine which of them are decoys. This is done with the tool **PeptideIndexer** {% icon tool %}. This tool allows the annotation of protein names and calculates the protein coverage based on the identified peptides. Then we calculate peptide posterior error probabilities (PEPs), because they are needed for the protein inference algorithm Fido, which is used by OpenMS. We will then filter PSMs for 1 % FDR with **FalseDiscoveryRate** {% icon tool %}. and set the score back to PEP with **IDScoreSwitcher** {% icon tool %}.

> <hands-on-title>Peptide FDR filtering</hands-on-title>
>
> 1. Run {% tool [PeptideIndexer](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_peptideindexer/PeptideIndexer/2.6+galaxy0) %} with
>    - {% icon param-file %} *"Input idXML file containing the identifications"*: output of **XTandemAdapter**
>    - {% icon param-file %} *"Input sequence database in FASTA format"*: FASTA protein database
>    - *"If set, the protein sequences are stored as well"*: `Yes`
>    - *"If set, the protein description is stored as well"*: `Yes`
>    - In the *"enzyme"* section set *"Specificity of the enzyme"*: `none`
>
> 2. Run {% tool [IDPosteriorErrorProbability](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_idposteriorerrorprobability/IDPosteriorErrorProbability/2.6+galaxy) %} with
>    - {% icon param-file %} *"input file"*: **PeptideIndexer** output
>    - *"If set scores will be calculated as '1 - ErrorProbabilities' and can be interpreted as probabilities for correct identifications"*: `Yes`
>
> 3. Run {% tool [FalseDiscoveryRate](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_falsediscoveryrate/FalseDiscoveryRate/2.6+galaxy0) %} with
>    - {% icon param-file %} *"Identifications from searching a target-decoy database"*: output of **IDPosteriorErrorProbability**
>    - *"Perform FDR calculation on protein level"*: `false`
>    - In the *"FDR control"* section select *"Filter PSMs based on q-value"*: `0.01`
>    - In the "Parameter section for the FDR calculation algorithm"* section select *"If 'true' decoy peptides will be written to output file, too"*: `Yes`
>
> 4. Run {% tool [IDScoreSwitcher](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_idscoreswitcher/IDScoreSwitcher/2.6+galaxy0) %} with
>    - {% icon param-file %} *"Input file"*: output of **FalseDiscoveryRate**
>    - *"Name of the meta value to use as the new score"*: `Posterior Probability_score`
>    - *"Orientation of the new score"*: `higher_better`
>
> 5. Run {% tool [FileInfo](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_fileinfo/FileInfo/2.6+galaxy0) %} to get basic information about the identified peptides
>    - {% icon param-file %} *"input file"*: **IDScoreSwitcher** output
>
{: .hands_on}

> <question-title></question-title>
> 1. How many peptides were identified?
> 2. How many peptides with oxidized methionine were identified?
>
> > <solution-title></solution-title>
> > 1. You should have identified 3378 non-redundant peptide hits.
> > 2. 804 peptides contain an oxidized methionine.
> > Numbers may slightly vary depending on the versions of the tools and the used FASTA file.
> {: .solution }
{: .question}


# Protein Inference
In bottom-up proteomics, it is necessary to combine the identified peptides to proteins. This is not a trivial task, as proteins are redundant to some degree. Thus, not every peptide can be assigned to only one protein.
The OpenMS suite implemented the [Fido](https://www.ncbi.nlm.nih.gov/pubmed/20712337) algorithm for protein inference. Fido uses a Bayesian probabilistic model to group and score proteins based on peptide-spectrum matches. Afterwards, we keep only proteins with 1% FDR.

> <hands-on-title>Protein inference</hands-on-title>
>
> 1. Run {% tool [FidoAdapter](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_fidoadapter/FidoAdapter/2.6+galaxy0) %}
>    - {% icon param-file %} *"Input: identification results"*: output of **IDScoreSwitcher**
>    - *"Post-process Fido output with greedy resolution of shared peptides based on the protein probabilities"*: `Yes`
>
> 2. Run {% tool [FalseDiscoveryRate](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_falsediscoveryrate/FalseDiscoveryRate/2.6+galaxy0) %}
>    - {% icon param-file %} *"Identifications from searching a target-decoy database"*: output of **FidoAdapter**
>    - *"Perform FDR calculation on PSM level"*: `false`
>    - In the *"FDR control"* section select *"Filter proteins based on q-value"*: `0.01`
>
> 3. Run {% tool [FileInfo](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_fileinfo/FileInfo/2.6+galaxy0) %} to get basic information about the identified proteins
>    - {% icon param-file %} *"input file"*: output of **FalseDiscoveryRate**
{: .hands_on}

> <comment-title>"Greedy" Group Resolution</comment-title>
> Protein groups are reported, when an identified peptide maps to multiple proteins in the used database [Nesvizhskii and Aebersold (2005)](https://www.ncbi.nlm.nih.gov/pubmed/16009968). Some peptides may map to different protein groups and can therefore not be used for protein quantitation. The option `-greedy_group_resolution` solves this problem by assigning peptides only to the one most probable protein group, thus enabling to quantify proteins based not only on unique, but also on shared peptides. This usually leads to a much higher number of quantified proteins. However it will introduce noise in the FCs when a peptide was indeed shared by different proteins and the quantity of this peptide was a weighted sum of contributions. The greedy group resolution is similar to Occam's razor.
{: .comment}

> <question-title></question-title>
> How many proteins were finally identified?
>
> > <solution-title></solution-title>
> > You should have identified 1252 proteins.
> > Numbers may slightly vary depending on the versions of the tools and the used FASTA file.
> {: .solution }
{: .question}

# Analysis of Contaminants

The FASTA database used for the peptide to spectrum matching contained some entries that were not expected to stem from the HeLa cell lysate, but are common contaminations in LC-MS/MS samples. The main reason to add those is to avoid false assignment of contaminant spectra to other proteins.
It also enables you to check for contaminations in your samples.

**CAVE:** When analyzing human samples, many proteins that are common contaminants may also stem from the sample. Therefore, human contaminants do not have to be excluded from further analysis, but you should keep in mind that the source of these proteins is unclear.

> <hands-on-title>Analysis of Contaminants</hands-on-title>
>
> 1. Run {% tool [TextExporter](toolshed.g2.bx.psu.edu/repos/galaxyp/openms_textexporter/TextExporter/2.6+galaxy0) %} to convert the idXML output to a human-readable tabular file.
>    - {% icon param-file %} *"Input file"*: **FalseDiscoveryRate** output
>
> 2. Run {% tool [Select lines that match an expression](Grep1) %}
>    - {% icon param-file %} *"Select lines from"*: **TextExporter**
>    - *"that"*: `Matching`
>    - *"the pattern"*: `CONTAMINANT`
>
> 2. Run {% tool [Select lines that match an expression](Grep1) %} to remove all non human proteins (e.g. bovine)
>    - {% icon param-file %} *"Select lines from"*: **TextExporter**
>    - *"that"*: `Matching`
>    - *"the pattern"*: `HUMAN`
>
{: .hands_on}

> <question-title></question-title>
> 1. Which contaminants did you identify? Where do these contaminations likely come from?
> 2. What other sources of contaminants exist?
> 3. How many false positives do we expect in our list?
>
> > <solution-title></solution-title>
> > 1.  TRY1_BOVIN is bovine trypsin. It was used to degrade the proteins to peptides. ALBU_BOVIN is bovine serum albumin. It is added to cell culture medium in high amounts. Also, eight human proteins are listed, these are commonly introduced during sample preparation. As we were analyzing a human sample, it is not neccessary to remove these proteins, as they may as well originate from the HeLa cells.
> > 2.  Contaminants often stem from the experimenter, these are typically keratins or other high-abundant human proteins. Basically any protein present in the room of the mass spectrometer might get into the ion source, if it is airborne. As an example, sheep keratins are sometimes found in proteomic samples, stemming from clothing made of sheep wool.
> > 3.  As we were allowing for a false discovery rate of 1 %, we would expect 13 (1255/0.01) false positive proteins in our list.
> {: .solution }
{: .question}


# Using multiple search engines

It is generally recommended to use more than one peptide search engine and use the combined results for peptide inference ([Shteynberg et al., 2013, Mol. Cell. Proteomics](https://www.ncbi.nlm.nih.gov/pubmed/23720762)).
By comparing results of multiple search engines, you may improve the *sensitivity* (when accepting peptides that were found by only one of the engines), the *specificity* (when accepting only peptides that were found by all of the search engines) or *both* (when using n>2 search engines and accept peptides found by a fraction of the (e.g. n-1) search engines).

Here, we will use the OpenMS tool [ConsensusID](https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/release/latest/html/TOPP_ConsensusID.html) to combine the search engine results.

> <hands-on-title>Multiple search engines</hands-on-title>
>
> 1. Run **MSGFPlusAdapter** {% icon tool %} with
>    - {% icon param-file %} *"Input file"*: the MS2-centroided mzML
>    - {% icon param-file %} *"Protein sequence database*": the FASTA protein database
>    - *"Precursor monoisotopic mass tolerance"*: `10.0`
>    - *"Instrument that generated the data"*: `Q_Exactive`
>    - In *"param_fixed_modifications"*
>      - {% icon param-repeat %} Click on *"Insert param_fixed_modifications"*
>      - In *"1: param_fixed_modifications"*
>         - *"Fixed modifications"*: `Carbamidomethyl (C)`
>    - In *"param_variable_modifications"*
>      - {% icon param-repeat %} Click on *"Insert param_variable_modifications"*
>      - In *"1: param_variable_modifications"*
>         - *"Variable modifications"*: `Oxidation (M)`
>
> 2. Run **FileInfo** {% icon tool %} to get basic information about the identified proteins
>    - {% icon param-file %} *"input file"*: **MSGFPlusAdapter** output
>
> 3. Run **IDPosteriorErrorProbability** {% icon tool %} with
>    - {% icon param-file %} *"input file"*: **MSGFPlusAdapter** output
>    - *"If set scores will be calculated as '1 - ErrorProbabilities' and can be interpreted as probabilities for correct identifications"*: `No`
>
> 4. Run **IDMerger** {% icon tool %}
>    - *"Reduce collectons"*: `reduce collections by aggregating single files of multiple collections`
>      - In *"inputs"*
>        - In *"1: Repeat"*
>          - {% icon param-file %} *"Input files separated by blanks"*: output of **IDScoreSwitcher** based on **XTandemAdapter**
>        - In *"2: Repeat"*
>          - {% icon param-file %} *"Input files separated by blanks"*: output of **IDScoreSwitcher** based on **MSGFPlusAdapter**
>
> 5. Run **ConsensusID** {% icon tool %}
>    - {% icon param-file %} *"input file"*: **IDMerger** output
>
> 6. Run **PeptideIndexer** {% icon tool %} with
>    - {% icon param-file %} *"Input idXML file containing the identifications"*: **ConsensusID** output
>    - {% icon param-file %} *"Input sequence database in FASTA format"*: FASTA protein database
>    - *"Specificity of the enzyme"*: `none`
>
> 7. Run **FalseDiscoveryRate** {% icon tool %} with
>    - {% icon param-file %} *"Identifications from searching a target-decoy database"*: output of **PeptideIndexer**
>    - *"Perform FDR calculation on protein level"*: `false`
>    - *"Filter PSMs based on q-value"*: `0.01`
>    - *"If 'true' decoy peptides will be written to output file, too"*: `Yes`
>
> 8. Run **IDScoreSwitcher** {% icon tool %} with
>    - {% icon param-file %} *"Input file"*: **FalseDiscoveryRate** output
>    - *"Name of the meta value to use as the new score"*: `Posterior Error Probability_score`
>    - *"Orientation of the new score"*: `lower_better`
>    - *"Name to use as the type of the new score"*: `Posterior Error Probability`
>
> 9. Run **FileInfo** {% icon tool %} to get basic information about the identified peptides
>    - {% icon param-file %} *"input file"*: **IDScoreSwitcher** output
>
> 10. Proceed with the protein inference as described [above](#protein-inference)
>
{: .hands_on}

> <question-title></question-title>
> 1. How many PSMs could be matched with XTandem and MSGFPlus alone? How many peptides were identified?
> 2. How many PSMs could be matched after combining the results with ConsensusID? How many peptides were identified?
>
> > <solution-title></solution-title>
> > 1.  After FDR-filtering, XTandem matched 3,552 PSMs (2,616 unique peptides) and MSGFPlus matched 4,292 PSMs (2,991 peptides).
> > 2.  Combining the results with ConsensusID leads to matching of 4,299 PSMs (3,041 unique peptides).
> {: .solution }
{: .question}


# Premade Workflow

- [A premade workflow for this tutorial]({% link topics/proteomics/tutorials/protein-id-oms/workflows/workflow.ga %}).
- [A premade workflow using the search engines XTandem and MSGF+]({% link topics/proteomics/tutorials/protein-id-oms/workflows/workflow_two-search-engines.ga %}).

# Further Reading

- [Protein inference](https://www.ncbi.nlm.nih.gov/pubmed/16009968)
- [Fido publication](https://www.ncbi.nlm.nih.gov/pubmed/20712337)
- [Evaluation of protein inference algorithms](https://www.ncbi.nlm.nih.gov/pubmed/27498275)


topic: proteomics
tutorial: metaproteomics
title: Metaproteomics tutorial
difficulty: intermediate
time_estimation: 2h
tools_json: []
requirements_json: []
tags_json: ["[microgalaxy]"]
content_hash: 46b08a8fb5e746a870594a0b747d745b
last_modified: 2026-02-26T21:12:17.495441
zenodo_link: https://doi.org/10.5281/zenodo.839701
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/metaproteomics/tutorial.html

content:
In this metaproteomics tutorial we will identify expressed proteins from a complex bacterial community sample.
For this MS/MS data will be matched to peptide sequences provided through a FASTA file.

Metaproteomics is the large-scale characterization of the entire protein complement of environmental microbiota
at a given point in time. It has the potential to unravel the mechanistic details of microbial interactions with
the host / environment by analyzing the functional dynamics of the microbiome.

In this tutorial, we will analyze a sample of sea water that was collected in August of 2013 from the Bering
Strait chlorophyll maximum layer (7m depth, 65° 43.44″ N, 168° 57.42″ W). The data were originally published in {% cite May_2016 %}.

> <agenda-title></agenda-title>
>
> In this tutorial, we will deal with:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Pretreatments

## Data upload

There are three ways to upload your data.

*   Upload/Import the files from your computer
*   Using a direct link
*   Import from the data library if your instance provides the files

In this tutorial, we will get the data from Zenodo: [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.839701.svg)](https://doi.org/10.5281/zenodo.839701).

> <hands-on-title>Data upload and organization</hands-on-title>
>
> 1. Create a new history and name it something meaningful (e.g. *Metaproteomics tutorial*)
>
>    {% snippet faqs/galaxy/histories_create_new.md %}
>
>    {% snippet faqs/galaxy/histories_rename.md %}
>
> 2. Import the three MGF MS/MS files and the FASTA sequence file from Zenodo.
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>    ```
>    https://zenodo.org/record/839701/files/2016_Jan_12_QE2_45.mgf
>    https://zenodo.org/record/839701/files/2016_Jan_12_QE2_46.mgf
>    https://zenodo.org/record/839701/files/2016_Jan_12_QE2_47.mgf
>    https://zenodo.org/record/839701/files/FASTA_Bering_Strait_Trimmed_metapeptides_cRAP.fasta
>    https://zenodo.org/record/839701/files/Gene_Ontology_Terms.tabular
>    ```
>
>    As default, Galaxy takes the link as name.
>
>    > <comment-title></comment-title>
>    > - Rename the datasets to a more descriptive name
>    {: .comment}
>
> 3. Build a **Dataset list** for the three MGF files
>
>    {% snippet faqs/galaxy/collections_build_list.md %}
>
{: .hands_on}

We have a choice to run all these steps using a single workflow, then discuss each step and the results in more detail.

> <hands-on-title>Pretreatments</hands-on-title>
>
> 1. **Import the workflow** into Galaxy:
>
>    {% snippet faqs/galaxy/workflows_run_trs.md path="topics/proteomics/tutorials/metaproteomics/workflows/workflow.ga" title="Pretreatments" %}
>
> 2. Run **Workflow** {% icon workflow %} using the following parameters:
>    - *"Send results to a new history"*: `No`
>    - {% icon param-file %} *"1: SixGill generated protein fasta file"*: `FASTA_Bering_Strait_Trimmed_metapeptides_cRAP.fasta`
>    - {% icon param-file %} *"2: Dataset collection of Bering Strait MGF files"*: `Dataset collection of bering MGF`
>    - {% icon param-file %} *"3: GeneOntology terms (selected)"*: `Gene_Ontology_terms.tabular`
>
>    {% snippet faqs/galaxy/workflows_run.md %}
>
{: .hands_on}

# Analysis

## Match peptide sequences

The search database labelled `FASTA_Bering_Strait_Trimmed_metapeptides_cRAP.FASTA` is the input database that
will be used to match MS/MS to peptide sequences via a sequence database search. It is a small excerpt of the original database, which was constructed based on a metagenomic screening of the sea water samples (see {% cite May_2016 %}). The full original database [is available online](https://noble.gs.washington.edu/proj/metapeptide/data/metapeptides_BSt.fasta). The contaminant database (cRAP) was merged with the original database.

For this, the sequence database-searching program called [SearchGUI](https://compomics.github.io/projects/searchgui.html) will be used.
The created dataset collection of the three *MGF files* in the history is used as the MS/MS input.

### SearchGUI

> <hands-on-title>SearchGUI</hands-on-title>
>
> 1. {% tool [Search GUI](toolshed.g2.bx.psu.edu/repos/galaxyp/peptideshaker/search_gui/3.3.10.1) %} with the following parameters:
>    - **Protein Database**: `FASTA_Bering_Strait_Trimmed_metapeptides_cRAP.FASTA`(or however you named the `FASTA` file)
>    - **Input Peak lists (mgf)**: `MGF files` dataset collection.
>
>    > <tip-title>Select dataset collections as input</tip-title>
>    >
>    > * Click the **Dataset collection** icon on the left of the input field:
>    >
>    >      ![Dataset collection button](../../images/dataset_button.png)
>    > * Select the appropriate dataset collection from the list
>    {: .tip}
>
>    Section **Search Engine Options**:
>
>    - **Search Engines**: `X!Tandem`
>
>    > <comment-title></comment-title>
>    >
>    > The section **Search Engine Options** contains a selection of sequence database searching
>    > algorithms that are available in SearchGUI. Any combination of these programs can be used for
>    > generating PSMs from MS/MS data. For the purpose of this tutorial, **X!Tandem** we will be used.
>    {: .comment}
>
>    Section **Precursor Options**:
>
>    - **Fragment Tolerance Units**: `Daltons`
>    - **Fragment Tolerance**: `0.2`- this is high resolution MS/MS data
>    - **Maximum Charge**: `6`
>
>    Section **Protein Modification Options**:
>
>    - **Fixed Modifications**: `Carbamidomethylation of C`
>    - **Variable modifications**: `Oxidation of M`
>
>    > <tip-title>Search for options</tip-title>
>    >
>    > * For selection lists, typing the first few letters in the window will filter the available options.
>    {: .tip}
>
>    Section **Advanced Options**:
>    - **X!Tandem Options**: `Advanced`
>    - **X!Tandem: Quick Acetyl**: `No`
>    - **X!Tandem: Quick Pyrolidone**: `No`
>    - **X!Tandem: Protein stP Bias**: `No`
>    - **X!Tandem: Maximum Valid Expectation Value**: `100`
>
>    - leave everything else as default
>
> 2. Click **Run Tool**.
>
{: .hands_on}

Once the database search is completed, the SearchGUI tool will output a file (called a
SearchGUI archive file) that will serve as an input for the next section, PeptideShaker.

> <comment-title></comment-title>
> Note that sequence databases used for metaproteomics are usually much larger than the excerpt used in this tutorial. When using large databases, the peptide identification step can take much more time for computation. In metaproteomics, choosing the optimal database is a crucial step of your workflow, for further reading see [Timmins-Schiffman et al (2017)](https://www.ncbi.nlm.nih.gov/pubmed/27824341).
>
> To learn more about database construction in general, like integrating contaminant databases or using a decoy strategy for FDR searching, please consult our tutorial on [Database Handling]({{site.baseurl}}/topics/proteomics/tutorials/database-handling/tutorial.html).
>
{: .comment}

### PeptideShaker

[PeptideShaker](https://compomics.github.io/projects/peptide-shaker.html) is a post-processing software tool that
processes data from the SearchGUI software tool. It serves to organize the Peptide-Spectral
Matches (PSMs) generated from SearchGUI processing and is contained in the SearchGUI archive.
It provides an assessment of confidence of the data, inferring proteins identified from the
matched peptide sequences and generates outputs that can be visualized by users to interpret
results. PeptideShaker has been wrapped in Galaxy to work in combination with SearchGUI
outputs.

> <comment-title></comment-title>
> There are a number of choices for different data files that can be generated using
> PeptideShaker. A compressed file can be made containing all information needed to view the
> results in the standalone PeptideShaker viewer. A `mzidentML` file can be created that contains
> all peptide sequence matching information and can be utilized by compatible downstream
> software. Other outputs are focused on the inferred proteins identified from the PSMs, as well
> as phosphorylation reports, relevant if a phosphoproteomics experiment has been undertaken.
> More detailed information on peptide inference using SearchGUI and PeptideShaker can be found in our tutorial on [Peptide and Protein ID]({{site.baseurl}}/topics/proteomics/tutorials/protein-id-sg-ps/tutorial.html).
{: .comment}

> <hands-on-title>PeptideShaker</hands-on-title>
>
> 1. {% tool [Peptide Shaker](toolshed.g2.bx.psu.edu/repos/galaxyp/peptideshaker/peptide_shaker/1.16.36.3) %} with the following parameters:
>   - **Compressed SearchGUI results**: The SearchGUI archive file
>   - **Specify Advanced PeptideShaker Processing Options**: `Default Processing Options`
>   - **Specify Advanced Filtering Options**: `Advanced Filtering Options`
>   - **Maximum Precursor Error Type**: `Daltons`
>   - **Specify Contact Information for mzIdendML**: You can leave the default dummy options for now, but feel free to enter custom contact information.
>   - **Include the protein sequences in mzIdentML**: `No`
>   - **Output options**: Select the `PSM Report` (Peptide-Spectral Match) and the `Certificate of Analysis`
>
>       > <comment-title></comment-title>
>       >
>       > The **Certificate of Analysis** provides details on all the parameters
>       > used by both SearchGUI and PeptideShaker in the analysis. This can be downloaded from the
>       > Galaxy instance to your local computer in a text file if desired.
>       {: .comment}
>
> 2. Click **Run Tool** and inspect the resulting files after they turned green with the **View data** icon:
>     ![View data button](../../images/view_data_icon.png)
>
{: .hands_on}


A number of new items will appear in your history, each corresponding to the outputs selected
in the PeptideShaker parameters. Most relevant for this tutorial is the PSM report:

![Display of the PSM report tabular file](../../images/psm_report.png "The PSM report")

Scrolling towards left will show the sequence for the PSM that matched to these
metapeptide entries. Column 3 is the sequence matched for each PSM entry. Every identified PSM is a
new row in the tabular output.

In the following steps of this tutorial, selected portions of this output will be extracted and used for
analysis of the taxonomic make-up of the sample as well as the biochemical functions
represented by the proteins identified.

## Taxonomy analysis

In the previous section, the genome sequencing and mass spectrometry data from
processing of biological samples was used to identify peptides present in those samples.
Now those peptides are used as evidence to infer which organisms are represented in the sample,
and what biological functions those peptides and associated proteins suggest are occurring.

The UniProt organization collects and annotates all known proteins for organisms. A UniProt
entry includes the protein amino acid sequence, the NCBI taxonomy, and any annotations
about structure and function of the protein. The UniPept web resource developed
by Ghent University will be used to match the sample peptides to proteins. UniPept indexes all Uniprot
proteins and provides a fast matching algorithm for peptides.

> <comment-title>Unipept</comment-title>
>
> Users can access UniPept via a [web page](https://unipept.ugent.be) and paste peptide
> sequences into the search form to retrieve protein information. But we'll use the Galaxy
> *Unipept* tool to automate the process. The *Unipept* tool sends the peptide list to the
> UniPept REST API service, then transforms the results into datasets that can be further analyzed
> or operated on within Galaxy.
{: .comment}

### Recieving the list of peptides: Query Tabular

In order to use *Unipept*, a list containing the peptide sequences has to be generated.
The tool **Query Tabular** can load tabular data (the PSM report in this case) into a SQLite data base.
As a tabular file is being read, line filters may be applied and an SQL query can be performed.

> <hands-on-title>Query Tabular</hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.0.0) %} with the following parameters:
>
>    - **Database Table**: Click on `+ Insert Database Table`:
>    - **Tabular Dataset for Table**: The PSM report
>
>    Section **Filter Dataset Input**:
>
>    - **Filter Tabular Input Lines**: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `by regex expression matching`
>        - **regex pattern**: `^\d`
>        - **action for regex match**: `include line on pattern match`
>
>    Section **Table Options**:
>
>    - **Specify Name for Table**: `psm`
>    - **Specify Column Names (comma-separated list)**: `id,,sequence,,,,,,,,,,,,,,,,,,,,confidence,validation`
>
>        > <comment-title></comment-title>
>        >
>        > By default, table columns will be named: c1,c2,c3,...,cn (column names for a table must be unique).
>        > You can override the default names by entering a comma separated list of names, e.g. `,name1,,,name2`
>        > would rename the second and fifth columns.
>        >
>        > Check your input file to find the settings which best fits your needs.
>        {: .comment}
>
>    - **Only load the columns you have named into database**: `Yes`
>
>    - **Save the sqlite database in your history**: `Yes`
>
>        > <comment-title>Querying SQLite Databases</comment-title>
>        >
>        > * **Query Tabular** can also use an existing SQLite database. Activating `Save the sqlite database in your history`
>        > will store the created database in the history, allowing to reuse it directly.
>        >
>        {: .comment}
>
>    - **SQL Query to generate tabular output**:
>
>          SELECT distinct sequence
>
>          FROM psm
>
>          WHERE confidence >= 95
>
>          ORDER BY sequence
>
>    > <question-title></question-title>
>    >
>    > The SQL query might look confusing at first, but having a closer look should clarify a lot.
>    >
>    > 1. What does `FROM psm` mean?
>    > 2. What need to be changed if we only want peptides with a confidence higher then 98%?
>    >
>    > > <solution-title></solution-title>
>    > > 1. We want to read from table "psm". We defined the name before in the "Specify Name for Table" option.
>    > > 2. We need to change the value in line 3: "WHERE validation IS NOT 'Confident' AND confidence >= 98"
>    > {: .solution }
>    {: .question}
>
>    - **include query result column headers**: `No`
>
> 2. Click **Run Tool** and inspect the query results file after it turned green. If everything went well, it should look similiar:
>
>     ![Query Tabular output showing the peptides](../../images/query_tabular_1.png "Query Tabular output")
>
{: .hands_on}

While we can proceed with this list of peptides, let's practice using the created SQLite database for further queries.
We might not only be interested in all the distinct peptides, but also on how many PSMs a single peptide had.
Therefore we can search the database for the peptides and count the occurrence without configuring the tables and columns again:

> <hands-on-title>SQLite to tabular</hands-on-title>
>
> 1. {% tool [SQLite to tabular](toolshed.g2.bx.psu.edu/repos/iuc/sqlite_to_tabular/sqlite_to_tabular/2.0.0) %} with the following parameters:
>
>    - **SQL Query**:
>
>          SELECT sequence as "peptide", count(id) as "PSMs"
>
>          FROM psm
>
>          WHERE confidence >= 95
>
>          GROUP BY sequence
>
>          ORDER BY sequence
>
>
> 2. Click **Run Tool**. The resulting file should have two columns, one with the distinct peptides, the other with the count number of PSMs.
>
{: .hands_on}


### Retrieve taxonomy for peptides: Unipept

The generated list of peptides can now be used to search via *Unipept*.
We do a taxonomy analysis using the UniPept pept2lca function to return the taxonomic lowest common ancestor for each peptide:

> <hands-on-title>Unipept</hands-on-title>
>
> 1. {% tool [Unipept](toolshed.g2.bx.psu.edu/repos/galaxyp/unipept/unipept/4.3.0) %} with the following parameters:
>
>    - **Unipept application**: `pept2lca: lowest common ancestor`
>    - **Peptides input format**: `tabular`
>    - **Tabular Input Containing Peptide column**: The query results file.
>    - **Select column with peptides**: `Column 1`
>    - **Choose outputs**: Select `tabular` and `JSON taxonomy tree`
>
> 2. Click **Run Tool**. The history should grow by two files. View each to see the difference.
>
>       > <comment-title></comment-title>
>       >
>       > The JSON (JavaScript Object Notation) file contains the same information as the tabular file but is not comfortably human readable.
>       > Instead, we can use it to use JavaScript libraries to visualize this data.
>       {: .comment}
>
> 3. Visualize the data:
>
>    - Click on the JSON output file from the *Unipept* tool to expand it. Click on the **Visualize** button and select **Unipept Tree viewer**:
>
>       ![Visualize button](../../images/visualize_button.png)
>
>    - A new window should appear with a visualization of the taxonomy tree of your data. Use the mouse wheel to scroll in and out and click on nodes to expand or collapse them:
>
>       ![Unipept Tree viewer visual output](../../images/unipept_tree_viewer.png "Interactive visualization from the Unipept Tree viever plugin")
>
{: .hands_on}

## Genus taxonomy level summary

The tabular *Unipept* output lists the taxonomy assignments for each peptide. To create a meaningful summary, the **Query Tabular** tool is
once again used, aggregating the number of peptides and PSMs for each genus level taxonomy assignment:

> <hands-on-title>Query Tabular</hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.0.0) %} with the following parameters:
>
>    - **Database Table**: Click on `+ Insert Database Table`
>    - **Tabular Dataset for Table**: The PSM report
>
>    Section **Filter Dataset Input**:
>
>    - **Filter Tabular Input Lines**: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `by regex expression matching`
>        - **regex pattern**: `^\d`
>        - **action for regex match**: `include line on pattern match`
>
>    Section **Table Options**:
>
>    - **Specify Name for Table**: `psm`
>    - **Specify Column Names (comma-separated list)**: `,,sequence,,,,,,,,,,,,,,,,,,,,confidence,validation`
>
>    - **Only load the columns you have named into database**: `Yes`
>
> 2. Repeat this step to have a second **Database Table**:
>
>    - **Database Table**: Click on `+ Insert Database Table`
>    - **Tabular Dataset for Table**: The **Unipept** `tabular`/`tsv` output
>
>    Section **Filter Dataset Input**:
>
>    - **Filter Tabular Input Lines**: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `by regex expression matching`
>        - **regex pattern**: `#peptide`
>        - **action for regex match**: `exclude line on pattern match`
>
>    Section **Table Options**:
>
>    - **Specify Name for Table**: `lca`
>    - **Specify Column Names (comma-separated list)**: `peptide,,,,,,,,,,,,,,,,,,,,,genus`
>
>    - **Only load the columns you have named into database**: `Yes`
>
>    - **Save the sqlite database in your history**: `Yes`
>
>    - **SQL Query to generate tabular output**:
>
>          SELECT lca.genus,count(psm.sequence) as "PSMs",count(distinct psm.sequence) as "DISTINCT PEPTIDES"
>
>          FROM psm LEFT JOIN lca ON psm.sequence = lca.peptide
>
>          WHERE confidence >= 95
>
>          GROUP BY lca.genus
>
>          ORDER BY PSMs desc, 'DISTINCT PEPTIDES' desc
>
>
> 2. Click **Run Tool** and inspect the query results file after it turned green:
>
>     ![Query Tabular output showing gene, PSMs and distinct peptides](../../images/metaproteomics_summary.png "Query Tabular output")
>
{: .hands_on}

## Functional Analysis

Recent advances in microbiome research indicate that functional characterization via metaproteomics analysis has the potential to accurately
measure the microbial response to perturbations. In particular, metaproteomics enables the estimation of the function of the microbial
community based on expressed microbial proteome.

In the following chapter, a functional analysis will be performed using the **UniPept** application `pept2prot` in order to match the list of peptides with the correlated Gene Ontology terms.
This allows to get an insight of the **biological process**, the **molecular function** and the **cellular component** related to the sample data.

> <comment-title>Gene Ontology (GO) Consortium</comment-title>
>
> The [Gene Ontology Consortium](http://www.geneontology.org/) provides with its Ontology a framework for the model of biology.
> The GO defines concepts/classes used to describe gene function, and relationships between these concepts. It classifies functions along three aspects:
>
>
> - **molecular function**
>
>   - molecular activities of gene products
>
> - **cellular component**
>
>   - where gene products are active
>
> - **biological process**
>
>   - pathways and larger processes made up of the activities of multiple gene products.
>
> [more information](http://geneontology.org/page/ontology-documentation)
>
{: .comment}

### Data upload

For this tutorial, a tabular file containing the relevant GO terms has been created. It contains the GO aspect, the ID and the name.
It is available at Zenodo: [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.839701.svg)](https://doi.org/10.5281/zenodo.839701).

> <hands-on-title>Data upload</hands-on-title>
>
> 1. Import the file `Gene_Ontology_Terms.tabular` from Zenodo.
>
>    > <tip-title>Setting file metadata on upload</tip-title>
>    >
>    > In the upload window of Galaxy you can set the filetype and related genome of the file you're uploading in the corresponding columns beforehand.
>    > This might be handy if the automatic detection of the filetype didn't work out perfectly or if you want to avoid setting the genome later on, especially for multiple files.
>    >
>    {: .tip}
>
>    As default, Galaxy takes the link as name.
>
>    > <comment-title></comment-title>
>    > - Rename the datasets to a more descriptive name, e.g. `Gene Ontology Terms`
>    {: .comment}
>
>
{: .hands_on}

> <details-title>Creating your own Gene Ontology list</details-title>
>
> The latest Gene Ontology can be downloaded [the GO website](http://geneontology.org/page/download-ontology) as a text file in the `OBO` format.
> `OBO` files are human-readable (in addition to machine-readable) and can be opened in any text editor. They contain more information than just the name and aspect.
>
> In order to receive a file like we use in the tutorial for your own analysis, different tools are available to extract information from `OBO` files,
> one of them being ONTO-PERL ({% cite Antezana_2008 %}).
> An example file with all GO terms from 08.07.2017 named `Gene_Ontology_Terms_full_07.08.2017.tabular` can be found on the [Zenodo repository](https://doi.org/10.5281/zenodo.839701) of this tutorial as well.
> You could also upload the Gene Ontology Terms by copying this link on to the Upload Data - Paste/Fetch data `https://zenodo.org/record/839701/files/Gene_Ontology_Terms_full_07.08.2017.tabular`
>
{: .details}

### Retrieve GO IDs for peptides: Unipept

The **UniPept** application `pept2prot` can be used to return the list of proteins containing each peptide.
The option `retrieve extra information` option is set to `yes` so that we retrieve Gene Ontology assignments (`go_references`)
for each protein.

> <hands-on-title>Unipept</hands-on-title>
>
> 1. {% tool [Unipept](toolshed.g2.bx.psu.edu/repos/galaxyp/unipept/unipept/4.3.0) %} with the following parameters:
>
>    - **Unipept application**: `pept2prot: UniProt entries containing a given tryptic peptide`
>    - **retrieve extra information**: `Yes`
>    - **Peptides input format**: `tabular`
>    - **Tabular Input Containing Peptide column**: The first query results file.
>    - **Select column with peptides**: `Column 1`
>    - **Choose outputs**: Select `tabular`
>
> 2. Click **Run Tool**.
>
> 3. inspect the result:
>
>    - The output should be a tabular file containing a column labeled `go_references`. This is what we're looking for.
>
{: .hands_on}


### Combine all information to quantify the GO results

As a final step we will use **Query Tabular** in a more sophisticated way to combine all information to quantify the GO analysis. The three used file and the extracted information are:

- **Gene Ontology Terms**:
    - `go_id` to match with **Normalized UniPept output**
    - The GO `aspect` to group the results in three separate files
    - The GO `description` to annotate the results
- **Normalized UniPept output**:
    - `peptide` to match with **PSM Report** and to count distinct peptides per GO term
    - `go_reference` to match with **Gene Ontology Terms**
- **PSM Report**:
    - `sequence` to match with **Normalized UniPept output**
    - `id` to count distinct PSM's per GO term

> <hands-on-title>Query Tabular</hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.0.0) %} with the following parameters:
>
>    - **Database Table**: Click on `+ Insert Database Table`
>    - **Tabular Dataset for Table**: The `Gene Ontology Terms` file
>
>    Section **Filter Dataset Input**:
>
>    - **Filter Tabular Input Lines**: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `skip leading lines`
>        - **Skip lines**: `1`
>
>    Section **Table Options**:
>
>    - **Specify Name for Table**: `go`
>    - **Specify Column Names (comma-separated list)**: `aspect,go_id,description`
>    - **Table Index**: Click on `+ Insert Table Index`:
>        - **This is a unique index**: `No`
>        - **Index on Columns**: `aspect,go_id`
>
>
> 2. Repeat this step to have a second **Database Table**:
>
>    - **Database Table**: Click on `+ Insert Database Table`
>    - **Tabular Dataset for Table**: The **Unipept** `tabluar`/`tsv` output
>
>    Section **Filter Dataset Input**:
>
>    - **Filter Tabular Input Lines**: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `skip leading lines`
>        - **Skip lines**: `1`
>    - Add another Filter: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `prepend a line number column`
>
>    Section **Table Options**:
>
>    - **Specify Name for Table**: `bering_prot`
>    - **Specify Column Names (comma-separated list)**: `id,peptide,uniprot_id,taxon_id,taxon_name,ec_references,go_references,refseq_ids,refseq_protein_ids,insdc_ids,insdc_protein_ids`
>    - **Table Index**: Click on `+ Insert Table Index`:
>        - **This is a unique index**: `No`
>        - **Index on Columns**: `id,peptide`
>
> 3. Repeat this step to have another **Database Table**:
>
>    - **Database Table**: Click on `+ Insert Database Table`
>    - **Tabular Dataset for Table**: The same **Unipept** `tabluar`/`tsv` output
>
>    Section **Filter Dataset Input**:
>
>    - **Filter Tabular Input Lines**: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `skip leading lines`
>        - **Skip lines**: `1`
>    - Add another Filter: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `prepend a line number column`
>    - Add another Filter: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `select columns`
>        - **enter column numbers to keep**: `1,7`
>    - Add another Filter: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `normalize list columns, replicates row for each item in list`
>        - **enter column numbers to normalize**: `2`
>        - **List item delimiter in column**: ` ` (a single blank character)
>
>    > <comment-title></comment-title>
>    > - The UniPept result file can contain multiple GO IDs in a single row. In order to create a normalized table of this data, these rows will be split so each record contains only one GO ID.
>    {: .comment}
>
>    Section **Table Options**:
>
>    - **Specify Name for Table**: `bering_prot_go`
>    - **Specify Column Names (comma-separated list)**: `id,go_reference`
>    - **Table Index**: Click on `+ Insert Table Index`:
>        - **This is a unique index**: `No`
>        - **Index on Columns**: `go_reference,id`
>
> 4. Repeat this step to have another **Database Table**:
>
>    - **Database Table**: Click on `+ Insert Database Table`
>    - **Tabular Dataset for Table**: The `PSM Report`
>
>    Section **Filter Dataset Input**:
>
>    - **Filter Tabular Input Lines**: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `by regex expression matching`
>        - **regex pattern**: `^\d`
>        - **action for regex match**: `include line on pattern match`
>    - Add another Filter: Click on `+ Insert Filter Tabular Input Lines`:
>    - **Filter By**: Select `select columns`
>        - **enter column numbers to keep**: `1,3,23,24`
>
>    Section **Table Options**:
>
>    - **Specify Name for Table**: `bering_psms`
>    - **Specify Column Names (comma-separated list)**: `id,sequence,confidence,validation`
>    - **Only load the columns you have named into database**: `Yes`
>    - **Table Index**: Click on `+ Insert Table Index`:
>        - **This is a unique index**: `No`
>        - **Index on Columns**: `sequence,id`
>
>    - **Save the sqlite database in your history**: `Yes`
>
>    - **SQL Query to generate tabular output**:
>
>          SELECT sequence as "peptide", count(id) as "PSMs"
>
>          FROM bering_psms
>
>          WHERE confidence >= 95
>
>          GROUP BY sequence
>
>          ORDER BY sequence
>
>
> 5. Click **Run Tool**.
>
{: .hands_on}

With this we have combined all the data into a single database which we can now query to extract the desired information with **SQLite to tabular**:

> <hands-on-title>SQLite to tabular</hands-on-title>
>
> 1. {% tool [SQLite to tabular](toolshed.g2.bx.psu.edu/repos/iuc/sqlite_to_tabular/sqlite_to_tabular/2.0.0) %} with the following parameters:
>
>    - **SQLite Database**: The created SQLite database from the former step
>    - **SQL Query**:
>
>          SELECT go.description,
>
>          count(distinct bering_psms.sequence) as "bering_peptides", count(distinct bering_psms.id) as "bering_psms"
>
>          FROM go JOIN bering_prot_go ON go.go_id = bering_prot_go.go_reference JOIN bering_prot on bering_prot_go.id = bering_prot.id JOIN
>
>          bering_psms ON bering_prot.peptide = bering_psms.sequence
>
>          WHERE go.aspect = 'molecular_function'
>
>          GROUP BY go.description
>
>          ORDER BY  bering_peptides desc,bering_psms desc
>
> 2. Click **Run Tool**.
> 3. Repeat these steps two times by replacing `molecular_function` in the fifth row of the SQL query by `biological_process` and `cellular_component`.
>
{: .hands_on}

With these three output files the functional analysis of this tutorial is finished. Each record contains the name of a GO term, the amount of peptides related to it and the amount of PSMs for these peptides.

> <comment-title>References</comment-title>
>
> - [Dataset](https://www.ncbi.nlm.nih.gov/pubmed/27824341) and [SixGill software](https://www.ncbi.nlm.nih.gov/pubmed/27396978)
>
> - [Galaxy workflows for metaproteomics](https://www.ncbi.nlm.nih.gov/pubmed/26058579)
>
> - [Metaproteomics community effort](https://z.umn.edu/gcc2017mporal)
>
> - [Unipept](https://www.ncbi.nlm.nih.gov/pubmed/28552653)
>
> - [Metaproteomics video](http://z.umn.edu/mpvideo2018)
{: .comment}


topic: proteomics
tutorial: neoantigen-variant-annotation
title: Neoantigen 4: Variant Annotation
difficulty: intermediate
time_estimation: 3H
tools_json: []
requirements_json: []
tags_json: ["[label-free]"]
content_hash: 2ee3d65f2b64f3fa83b991361c9c526c
last_modified: 2026-02-26T21:12:17.504566
zenodo_link: https://zenodo.org/records/14375693
workflows_json: []
url:
https://training.galaxyproject.org/training-material/topics/proteomics/tutorials/neoantigen-variant-annotation/tutorial.html

content:
Neoantigens are tumor-specific antigens that arise from somatic mutations in cancer cells. These mutations result in the generation of abnormal peptides that can be presented by the immune system’s Major Histocompatibility Complex (MHC) molecules. Neoantigens are gaining significant attention in cancer immunotherapy, as they hold the potential to be used in personalized vaccines and therapies aimed at stimulating the immune system to target and destroy tumor cells.

The process of identifying neoantigens begins with the analysis of genomic data to detect somatic mutations, followed by the prediction of the corresponding peptide sequences that can bind to MHC molecules. This process involves a combination of bioinformatics tools and techniques, including variant calling, peptide prediction, and MHC binding affinity analysis. Successful identification of neoantigens can pave the way for personalized treatment strategies that improve patient outcomes in cancer therapy.

This tutorial focuses on the Neoantigen Annotation pipeline, which is designed to assist researchers and clinicians in annotating and predicting neoantigens from high-throughput genomic data. By the end of this tutorial, you will be equipped with the skills to extract meaningful insights from genomic datasets, predict immunogenic peptides, and understand how these peptides interact with the immune system. These insights are essential for advancing the field of personalized immunotherapy and improving cancer treatment outcomes.

![Variant-annotation-overview-workflow]({% link topics/proteomics/images/neoantigen/PepPointer_Characterization_1.PNG %})

> <agenda-title></agenda-title>
>
> In this tutorial, we will cover:
>
> 1. TOC
> {:toc}
>
{: .agenda}

# Neoantigen Variant Annotation

This tutorial introduces the process of annotating neoantigens using the PepPointer tool. The aim is to identify potential neoantigens by predicting peptide sequences derived from somatic mutations in cancer genomes, followed by their binding affinity to MHC molecules. The workflow integrates several key bioinformatics steps, including variant analysis, peptide prediction, and immune system interaction analysis. Below is an overview of each major step involved in the tutorial:It is divided into two parts: A, which is the variant annotation, and B, which focuses on the database generation for IEDB.

- **A: Variant Annotation**
   1. **Get Data** The first step in the process is to gather the necessary data for analysis, which typically includes genomic data in VCF (Variant Call Format) or other formats containing information about somatic mutations. The data is then uploaded into the analysis environment. A well-organized dataset is crucial for smooth and efficient analysis, so it’s important to ensure proper file structure and metadata tagging before beginning.
   2. **Mutation to Peptide Mapping** The next step is to map these mutations to peptide sequences. This involves creating a list of peptides that contain the mutated residue, representing potential neoantigens. Peptide prediction tools are used to generate the corresponding peptide sequences from the mutated positions.
   3. **Annotation and Filtering** At this stage, the predicted peptides are annotated with relevant biological and immunological information, such as their predicted MHC class, binding affinity, and potential for being recognized by T-cells. Filtering is performed to retain only the most promising candidates, based on binding affinity thresholds and relevance to the tumor type being studied.
   4. **Mapping Peptide sequences with PepPointer** PepPointer is used to map the peptide sequences to their corresponding genomic coordinates. This tool helps align peptide sequences (often derived from proteomic data) to the genomic context, providing useful insights into where these peptides are located in the genome. It allows researchers to determine which genomic regions are associated with the peptides of interest, facilitating the study of their potential functional roles.
   5. **Visualization and Interpretation** The final step involves visualizing the results of the annotation and filtering steps. Various bioinformatics tools can be used to present the data in a way that is easy to interpret, such as visualizing peptide binding affinity scores or generating summary plots that highlight the most immunogenic neoantigens. This step helps in drawing meaningful conclusions about the potential of the identified peptides for cancer immunotherapy.

- **B: Database Generation for IEDB**
   1. **Generating FASTA for MHC binding tool** In this step, we prepare the peptide sequences in FASTA format to be used with an MHC binding prediction tool. MHC (Major Histocompatibility Complex) binding tools are often used in immunology research to predict which peptides can bind to specific MHC molecules and present them to T-cells.


# Variant annotation

![Variant-annotation-overview-workflow]({% link topics/proteomics/images/neoantigen/PepPointer_Characterization_2.PNG %})

## Get data

> <hands-on-title> Data Upload </hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo]({{ page.zenodo_link }}) or from
>    the shared data library (`GTN - Material` -> `{{ page.topic_name }}`
>     -> `{{ page.title }}`):
>
>    ```
>    https://zenodo.org/records/14375693/files/Novel_Peptides_from_PepQuery.tabular
>    https://zenodo.org/records/14375693/files/Fragpipe-Peptide-Report.tabular
>    https://zenodo.org/records/14375693/files/GffCompare_Annotated_GTF_to_BED.bed
>    https://zenodo.org/records/14375693/files/Homo_sapiens.GRCh38_canon.106.gtf
>    ```
>
>    {% snippet faqs/galaxy/datasets_import_via_link.md %}
>
>    {% snippet faqs/galaxy/datasets_import_from_data_library.md %}
>
> 3. Rename the datasets
> 4. Check that the datatype
>
>    {% snippet faqs/galaxy/datasets_change_datatype.md datatype="datatypes" %}
>
> 5. Add to each database a tag corresponding to ...
>
>    {% snippet faqs/galaxy/datasets_add_tag.md %}
>
{: .hands_on}



## Mutation to Peptide Mapping with Query Tabular

In this step, we will use the Query Tabular tool to perform a SQL query on a tabular dataset. This tool allows you to filter and extract data from a dataset by applying SQL commands, making it ideal for working with structured data such as CSV or TSV files. You will use it to join tables, select relevant columns, and generate a new output based on the criteria specified in the query. This step helps in refining and organizing the data for further analysis.
This step extracts information about novel peptides from FragPipe, which primarily includes protein details and the location of the peptide on the protein.

> <hands-on-title> Query Tabular </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `Novel_Peptides_from_PepQuery.tabular` (Input dataset)
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `Fragpipe-Peptide-Report.tabular` (Input dataset)
>    - *"SQL Query to generate tabular output"*:
> ```
> SELECT t1.c1,t2.c14,t2.c4,t2.c5
> FROM t1
> INNER JOIN t2
> ON t1.c1 = t2.c1
> ```
>    - *"include query result column headers"*: `No`
>
{: .hands_on}



## Annotation and Filtering

Because the annotation structures differ between **assembly-derived peptides** and **SAV (single-amino-acid–variant) peptides**, we apply separate text-formatting procedures for each. We first format the assembly-derived peptide annotations, followed by the SAV annotations, and will be processed separately with **PepPointer**.

### Extracting assembly-derived peptides

The **Select lines that match an expression** tool is used to extract only the rows corresponding to StringTie-assembled transcripts from a mixed text file. StringTie typically labels transcript or gene identifiers with the prefix `STRG`. By filtering for lines containing `STRG`, we keep only the relevant transcript entries and discard header or unrelated lines. This cleaned file is then used for downstream integration with genomic and variant annotations.

> <hands-on-title> Select StringTie transcript lines </hands-on-title>
>
> 1. {% tool [Select lines that match an expression](toolshed.g2.bx.psu.edu/repos/iuc/select/select/1.0.4) %} with the following parameters:
>    - {% icon param-file %} *"Select lines from"* (input): `Peptide_to_Protein_Annotation (Query Tabular output)`
>    - *"that"*: `Matching`
>    - *"the pattern"*:
>      ```
>      (STRG)
>      ```
>
>    This will retain only lines that contain the `STRG` label.
>    
{: .hands_on}

> <question-title></question-title>
>
> 1. Why do we filter for lines containing `STRG` in the StringTie output?  
> 2. What types of lines are removed by this filtering step?
>
> > <solution-title></solution-title>
> >
> > 1. The `STRG` prefix identifies StringTie-assembled genes/transcripts. Filtering for `STRG` ensures that only these biologically relevant entries are carried forward for coordinate annotation.  
> > 2. Header lines, comments, or any non-StringTie records that do not contain `STRG` are removed. This avoids clutter and prevents unrelated entries from interfering with downstream joins and comparisons.  
> >
> {: .solution}
>
{: .question}

### Converting delimiters - Assembly-derived peptides
In this step, we will use the Convert tool to modify characters in a dataset. Specifically, the tool will be used to convert all instances of pipe characters (|) to another format. The tool helps clean and standardize the data for subsequent processing or analysis. This is often a necessary step when preparing data for tools that require specific formats or when performing tasks such as parsing or file importation.


> <hands-on-title> Convert </hands-on-title>
>
> 1. {% tool [Convert - delimiters to TAB](Convert characters1) %} with the following parameters:
>    - *"Convert all"*: `Pipes`
>    - {% icon param-file %} *"in Dataset"*: `output` (output of **Select** {% icon tool %})
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. What is the purpose of using the "Convert" tool in this step?
>
> > <solution-title></solution-title>
> >
> > 1. The "Convert" tool is used to replace or remove specific characters in a dataset, such as pipe characters (`|`). This ensures that the data conforms to the required format for subsequent analysis or tools. Pipe characters are often used as delimiters in tabular data. Replacing them might be necessary if the data needs to be formatted for a different tool or if the pipe character interferes with parsing in downstream steps.
> >
> {: .solution}
>
{: .question}

### Removing Colons and converting into a different tabular
In this step, we will use the Convert tool to remove colons from the dataset. This conversion is useful when you need to standardize the data or prepare it for downstream processing, where colons may interfere with the analysis or formatting. By selecting the "Convert all" option for colons, all occurrences of colons in the dataset will be removed, creating a clean tabular format for further analysis.

> <hands-on-title> Convert </hands-on-title>
>
> 1. {% tool [Convert - delimiters to TAB](Convert characters1) %} with the following parameters:
>    - *"Convert all"*: `Colons`
>    - {% icon param-file %} *"in Dataset"*: `out_file1` (output of **Convert** {% icon tool %})
>
>
{: .hands_on}

### Editing certain "u" characters
In this step, we will use the Column Regex Find And Replace tool to find and replace specific patterns in the data columns. Regular expressions (regex) allow for pattern matching, making it easier to identify and modify data based on specific patterns. In this case, the tool will search for occurrences of the pattern u_ and replace them with u: in column c3 of the dataset. This is commonly done to standardize data formatting or clean data for further analysis. In this step, any protein sequences with a "u_" will be replaced with "u".

> <hands-on-title> Column Regex Find And Replace </hands-on-title>
>
> 1. {% tool [Column Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/galaxyp/regex_find_replace/regexColumn1/1.0.3) %} with the following parameters:
>    - {% icon param-file %} *"Select cells from"*: `out_file1` (output of **Convert** {% icon tool %})
>    - *"using column"*: `3`
>    - In *"Check"*:
>        - {% icon param-repeat %} *"Insert Check"*
>            - *"Find Regex"*: `u_`
>            - *"Replacement"*: `u:`
>
>
>
{: .hands_on}

### Removing Colons and converting into a different tabular

> <hands-on-title> Convert </hands-on-title>
>
> 1. {% tool [Convert - delimiters to TAB](Convert characters1) %} with the following parameters:
>    - *"Convert all"*: `Colons`
>    - {% icon param-file %} *"in Dataset"*: `out_file1` (output of **Column Regex and Replace** {% icon tool %})
>
>
{: .hands_on}


### Extracting bed file information using Query Tabular - Assembly-derived peptides
In this step, we will use the Query Tabular tool to extract specific information from a dataset, such as a BED file containing genomic regions, and match it with novel peptides. This allows for identifying the relevant genomic and peptide information by querying data from two sources and combining them through an SQL query. By using an INNER JOIN operation, we can merge data from two tables based on shared columns, and retrieve the necessary information. This query extracts specific columns from both the BED file (such as genomic coordinates) and the novel peptide dataset (such as peptide sequences or identifiers), enabling the identification of peptides that correspond to specific genomic regions. These are the columns that will be extracted:

- Chrom: Chromosome name (e.g., chr1).
- Start: Starting position of the feature (zero-based index).
- End: Ending position of the feature (one-based index).
- Name: Identifier for the feature (e.g., gene name).
- Score: Numeric score representing feature strength or relevance.
- Strand: Strand orientation (+ or -).
- ThickStart and ThickEnd: Define the start and end of the transcribed or relevant part of the feature, often the coding region.

> <hands-on-title> Extract BED file for Assembly-derived peptides </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `out_file1` (output of **Convert** {% icon tool %})
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `GffCompare_Annotated_GTF_to_BED.bed` (Input dataset)
>    - *"SQL Query to generate tabular output"*:
> ```
> SELECT t1.c1,t1.c8,t1.c9,t1.c11,t1.c12,t2.c5,t2.c6,t1.c7
> FROM t1
> INNER JOIN t2
> ON t1.c3 = t2.c4
> ```
>    - *"include query result column headers"*: `No`
>
> 2. Rename file to "assembly_formatted_table".
> 
{: .hands_on}


### Convert proteomic coordinates to genomic coordinates - Assembly-derived peptides
To convert proteomic coordinates to genomic coordinates, it is essential to account for the relationship between the protein sequence and its corresponding gene or genomic region. In this workflow, the proteomic coordinates have already been extracted at the amino acid level. Since each amino acid in the protein sequence corresponds to a triplet of nucleotides (a codon) in the mRNA, we need to multiply the proteomic coordinate by 3 to obtain the genomic coordinate. This conversion will give us the position of each amino acid within the genomic sequence. The resulting genomic coordinates are stored in a separate column for easy reference. Once this step is completed, we can extract and organize the information in the correct order for further analysis or mapping to the genomic reference.


> <hands-on-title> Text manipulation for Assembly-derived peptides </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `assembly_formatted_table` (output of **Query Tabular - Assembly** {% icon tool %})
>    - *"SQL Query to generate tabular output"*:
>      ```
>      SELECT t1.*,
>      (t1.c4 - 1) * 3 AS c2_multiplied,
>      t1.c5 * 3 AS c3_multiplied
>      FROM t1
>      ```
>    - *"include query result column headers"*: `No`
>
>
{: .hands_on}


### Annotating the genomic coordinates - Assembly-derived peptides

The Query Tabular step in this workflow is used to extract and calculate genomic coordinates based on the proteomic data. The SQL query within the tool defines two calculations for genomic coordinates, start and stop, based on the strand information of the data. For each row in the input dataset (t1), if the strand (t1.c7) is "-" (negative), the genomic coordinates are calculated by subtracting the position from the given end (t1.c3 - t1.c9 for start, and t1.c3 - t1.c10 for stop). If the strand is "+" (positive), the genomic coordinates are calculated by adding the respective positions (t1.c2 + t1.c9 for start, and t1.c2 + t1.c10 for stop). These calculated coordinates are then returned in the query results, where they will be included as new columns (start and stop). This step is essential for transforming the proteomic information into genomic positions for further analysis.

> <hands-on-title> Table processing for Assembly-derived peptides </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `output` (output of **Text manipulation - Assembly** {% icon tool %})
>    - *"SQL Query to generate tabular output"*:
>      ```
>      SELECT t1.*,
>      CASE
>      WHEN t1.c7 = '-' THEN t1.c3 - t1.c9
>      WHEN t1.c7 = '+' THEN t1.c2 + t1.c9
>      END AS start,
>      CASE
>      WHEN t1.c7 = '-' THEN t1.c3 - t1.c10
>      WHEN t1.c7 = '+' THEN t1.c2 + t1.c10
>      END AS stop FROM t1
>      ```
>    - *"include query result column headers"*: `No`
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. Why do I need to differentiate between the positive and negative strands when calculating genomic coordinates?
> 2. What is the significance of the t1.c9 and t1.c10 columns in the SQL query?
>
> > <solution-title></solution-title>
> >
> > 1. The positive and negative strands represent the two directions in which DNA is read, and they affect how genomic coordinates are calculated. For the negative strand, the coordinates are subtracted, whereas for the positive strand, they are added. This ensures the correct mapping of the protein sequence to the genome.
> > 2. The `t1.c9` and `t1.c10` columns represent specific offsets or lengths within the dataset, which are used to adjust the calculated genomic start and stop coordinates. These offsets could correspond to peptide lengths or specific features of the protein that need to be accounted for in the conversion to genomic coordinates.
> >
> {: .solution}
>
{: .question}


### Generating BED file for PepPointer - Assembly-derived peptides
This step is necessary to extract and reorganize relevant genomic information from the dataset. By querying specific columns such as chromosome (chromosome), start (chromStart), end (chromEnd), and strand (strand), we are preparing the data for further analysis. These values are essential for mapping proteomic or peptide data to the genomic coordinates, ensuring accurate alignment and interpretation of the sequence in the context of its genomic location. Additionally, renaming columns enhances clarity and standardizes the format, making it easier to work with the data in subsequent steps.

> <hands-on-title> BED file for Assembly-derived peptides </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `output` (output of **Table processing - Assembly** {% icon tool %})
>    - *"SQL Query to generate tabular output"*:
> ```
> SELECT
> c8 AS `chromosome`,
> c11  AS `chromStart`,
> c12 AS `chromEnd`,
> c1 AS `name`,
> c6 AS `score`,
> c7 AS `strand`
> FROM t1
> ```
>    - *"include query result column headers"*: `No`
>
> 2. Change data type to ".bed".
> 
{: .hands_on}


> <question-title></question-title>
>
> 1. What is the purpose of renaming columns like chromosome, chromStart, and chromEnd in the SQL query?
> 2. Why is the strand column important in this query, and how does it affect genomic coordinate interpretation?
>
> > <solution-title></solution-title>
> >
> > 1. Renaming the columns helps in providing more meaningful names for the data, making it easier to understand the genomic features such as the chromosome and start/end coordinates. These renamed columns align with common genomic nomenclature, improving readability and analysis.
> > 2. The strand column indicates whether the sequence is on the positive or negative strand of the DNA. This is important for correctly interpreting the orientation of the sequence and calculating its genomic coordinates. A positive strand means the coordinates will be calculated from the start, while a negative strand requires reverse calculations to adjust the coordinates properly.
> >
> {: .solution}
>
{: .question}



## Mapping Assembly-derived Peptide sequences with PepPointer

PepPointer is a tool designed to map peptide sequences to their respective genomic locations using data such as GTF and BED files. In this workflow, PepPointer takes the GTF file, which contains gene annotations and genomic coordinates, and combines it with a BED file, which provides chromosomal coordinates of the peptides. By doing this, PepPointer can identify the exact genomic locations of the peptides, linking them to genes and exons. This step is crucial for accurately correlating proteomic data to genomic sequences, enabling a better understanding of the genomic context in which the peptides are found.

> <hands-on-title> PepPointer for Assembly-derived peptides </hands-on-title>
>
> 1. {% tool [PepPointer](toolshed.g2.bx.psu.edu/repos/galaxyp/pep_pointer/pep_pointer/0.1.3+galaxy1) %} with the following parameters:
>    - *"Choose the source of the GTF file"*: `From history`
>        - {% icon param-file %} *"GTF file with the genome of interest"*: `Homo_sapiens.GRCh38_canon.106.gtf` (Input dataset)
>    - {% icon param-file %} *"BED file with chromosomal coordinates of peptide"*: `output` (from **Query Tabular - BED file for Assembly**)
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. How does the GTF file help in identifying the genomic location of peptides?
> 2. What happens if the coordinates in the BED file don't align with the genomic features in the GTF file?
>
> > <solution-title></solution-title>
> >
> > 1. The GTF file contains detailed annotations of the genome, including information about genes, exons, and other genomic features. By providing the chromosomal coordinates of these features, the GTF file allows PepPointer to match peptide sequences to the corresponding genes and exons. This ensures that the peptides are mapped accurately within the genome, linking them to the correct genomic context.
> > 2. If the coordinates in the BED file don't align with the genomic features in the GTF file, PepPointer may not be able to accurately map the peptides to their corresponding genes or exons. This misalignment could result in incorrect annotations or a failure to identify the proper genomic location of the peptides. It is important to ensure that both files are from the same genome assembly to avoid such discrepancies.
> >
> {: .solution}
>
{: .question}


## Visualization and Interpretation of Novel Assembly-derived peptides with annotation

In this step, we are using Query Tabular to extract and format relevant information from the results produced by PepPointer. The SQL query in this tool is designed to structure the data into a more readable format, providing key details such as the peptide ID, chromosome location, start and end positions, strand orientation, and any annotations. Additionally, it generates genome coordinates in a format suitable for viewing in genome browsers like IGV and UCSC Genome Browser. This formatting step helps to visualize and interpret the data more effectively by linking the peptides to their genomic context.

> <hands-on-title> Query Tabular </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `classified` (output of **PepPointer - Assembly** {% icon tool %})
>    - *"SQL Query to generate tabular output"*:
> ```
> SELECT
> c4 AS Peptide,
> c1 AS Chromosome,
> c2 AS Start,
> c3 AS End,
> c6 AS Strand,
> c7 AS Annotation,
> c1||':'||c2||'-'||c3 AS IGV_Genome_Coordinate,
> 'https://genome.ucsc.edu/cgi-bin/hgTracks?db=hg38&position='||c1||'%3A'||c2||'-'||c3 AS UCSC_Genome_Browser
> FROM  t1
> ```
>    - *"include query result column headers"*: `Yes`
>
>
{: .hands_on}


> <question-title></question-title>
>
> 1. Why is it important to include both the "IGV_Genome_Coordinate" and "UCSC_Genome_Browser" columns?
> 2. What is the significance of the "Annotation" column, and what type of data might it contain?
>
> > <solution-title></solution-title>
> >
> > 1. The IGV_Genome_Coordinate column provides a formatted coordinate that is directly usable in genome viewers like IGV, helping users visualize the peptide's genomic location. The UCSC_Genome_Browser column provides a clickable URL to launch UCSC Genome Browser with the coordinates, offering another convenient method for visualizing the peptide's location in a widely used genome browser. Including both makes it easier for users to explore the data in different genomic tools.
> > 2. The Annotation column provides additional context about the genomic feature associated with the peptide, such as whether the peptide is part of a gene, exon, or other regulatory element. This column can help to interpret the biological relevance of the peptide in the genomic region and is important for understanding the functional role of the peptide within the genome. The data might include annotations like "gene", "exon", "intron", or other functional genomic elements.
> >
> {: .solution}
>
{: .question}


## Extracting SAV-derived peptides

The **Select lines that match an expression** tool is used to extract only the rows corresponding to SAV-assembled transcripts from a mixed text file.

> <hands-on-title> Select SAV transcript lines </hands-on-title>
>
> 1. {% tool [Select lines that match an expression](toolshed.g2.bx.psu.edu/repos/iuc/select/select/1.0.4) %} with the following parameters:
>    - {% icon param-file %} *"Select lines from"* (input): `Peptide_to_Protein_Annotation (Query Tabular output)`
>    - *"that"*: `NOT Matching`
>    - *"the pattern"*:
>      ```
>      (STRG)
>      ```
>
>    This will retain only lines that do not contain the `STRG` label.
>    
{: .hands_on}


### Converting delimiters
In this step, we will use the Convert tool to modify characters in a dataset. Specifically, the tool will be used to convert all instances of pipe characters (|) to another format. The tool helps clean and standardize the data for subsequent processing or analysis. This is often a necessary step when preparing data for tools that require specific formats or when performing tasks such as parsing or file importation.


> <hands-on-title> Convert </hands-on-title>
>
> 1. {% tool [Convert - delimiters to TAB](Convert characters1) %} with the following parameters:
>    - *"Convert all"*: `Pipes`
>    - {% icon param-file %} *"in Dataset"*: `output` (output of **Select** {% icon tool %})
>
>
{: .hands_on}

### Extracting Relevant Columns for SAV Peptides

The **Cut Columns** tool is used to extract only the fields required for downstream annotation of SAV-derived peptides. Since the structure of SAV output differs from assembly-derived peptide annotations, this step isolates the key identifiers and coordinate fields that will later be used for peptide mapping. By selecting only the columns necessary for variant-driven peptide interpretation, we ensure a clean and consistent table for integration with tools such as PeptidePointer.

> <hands-on-title> Extracting Columns for SAV </hands-on-title>
>
> 1. {% tool [Cut columns from a table](toolshed.g2.bx.psu.edu/repos/iuc/cut/cut/1.0.2) %} with the following parameters:
>    - *“Cut columns”*:  
>      ```
>      c1,c3,c9,c10
>      ```
>    - *“Delimited by”*: `Tab`
>    - {% icon param-file %} *“From”*: (output of **Convert (SAV)** {% icon tool %})
>
> 2. Execute the tool to produce a reduced SAV-specific annotation table containing only the essential fields for downstream processing.
>
{: .hands_on}

> <question-title></question-title>
>
> 1. Why do we extract only selected columns from the SAV output?  
> 2. How will this SAV table be used later in the workflow?
>
> > <solution-title></solution-title>
> >
> > 1. Extracting only the required fields reduces unnecessary complexity, removes irrelevant columns, and ensures consistent formatting.  
> >
> > 2. The SAV table will later be used in downstream tools such as PeptidePointer, enabling consistent mapping of peptides to genomic features.  
> >
> {: .solution}
>
{: .question}

### Editing SAV Annotation Columns Using Column Regex Find and Replace

The **Column Regex Find and Replace** tool is used in this step to clean and standardize SAV (Single Amino-acid Variant) annotation fields. SAV entries often begin with prefixes such as `SAV_` or `INDEL_`, which must be reformatted to a consistent delimiter-based structure (`SAV|`, `INDEL|`) to be compatible with downstream peptide-annotation tools such as PepPointer.

> <hands-on-title> Editing SAV Annotations </hands-on-title>
>
> 1. {% tool [Column Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/bgruening/column_regex_find_replace/column_regex_find_and_replace/1.0.3) %} with the following parameters:
>    - {% icon param-file %} *“Select cells from”* (tabular): (output of **Cut (SAV)** {% icon tool %})
>    - *“using column”*: `2`
>
>    **Regex Rule 1 — Convert INDEL notation**
>    - *Find Regex*: ```INDEL_```
>    - *Replacement*: ```INDEL|```
>
>    **Regex Rule 2 — Convert SNV notation**
>    - *Find Regex*: ```SNV_```
>    - *Replacement*:```SNV|```
>
{: .hands_on}


> <hands-on-title> Convert </hands-on-title>
>
> 1. {% tool [Convert - delimiters to TAB](Convert characters1) %} with the following parameters:
>    - *"Convert all"*: `Pipes`
>    - {% icon param-file %} *"in Dataset"*: `output` (output of **Column Regex Find And Replace (SAV)** {% icon tool %})
>
>
{: .hands_on}


> <hands-on-title> Query Tabular </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `Genomic-Variant-Protein-BED` (output of **Concatenate from Non-Reference workflow** {% icon tool %})
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: (output of **Convert - delimiters to TAB (SAV)** {% icon tool %})
>    - *"SQL Query to generate tabular output"*:
> ```
> SELECT t1.*, t2.*
> FROM t1
> JOIN t2
> ON t1.c1 = t2.c3
> ```
>    - *"include query result column headers"*: `No`
>
>
{: .hands_on}

> <hands-on-title> Extracting Columns for SAV BED </hands-on-title>
>
> 1. {% tool [Cut columns from a table](toolshed.g2.bx.psu.edu/repos/iuc/cut/cut/1.0.2) %} with the following parameters:
>    - *“Cut columns”*:  
>      ```
>      c8,c3,c4,c11,c12,c9,c5,c2
>      ```
>    - *“Delimited by”*: `Tab`
>    - {% icon param-file %} *“From”*: (output of **Query Tabular (SAV)** {% icon tool %})
>
> 2. Execute the tool to produce a reduced SAV-specific annotation table containing only the essential fields for downstream processing.
> 
{: .hands_on}

> <hands-on-title> Editing SAV Annotations </hands-on-title>
>
> 1. {% tool [Column Regex Find And Replace](toolshed.g2.bx.psu.edu/repos/bgruening/column_regex_find_replace/column_regex_find_and_replace/1.0.3) %} with the following parameters:
>    - {% icon param-file %} *“Select cells from”* (tabular): (output of **Cut columns (SAV)** {% icon tool %})
>    - *“using column”*: `6`
>
>    **Regex Rule 1 — Convert INDEL notation**
>    - *Find Regex*: ```INDEL```
>    - *Replacement*: ``` ```
>
>    **Regex Rule 2 — Convert SNV notation**
>    - *Find Regex*: ```SNV```
>    - *Replacement*:``` ```
>
> 2. Rename file to "sav_formatted_table".
>
> 
{: .hands_on}

Because the annotation formats differ between **assembly-derived peptides** and **SAV-derived peptides**, we must process them separately. We first perform text formatting on the assembly peptide annotations, followed by formatting of the SAV annotations, and finally process both sets for separate peptide annotation using **PepPointer**.


### Convert proteomic coordinates to genomic coordinates - SAV-derived peptides
To convert proteomic coordinates to genomic coordinates, it is essential to account for the relationship between the protein sequence and its corresponding gene or genomic region. In this workflow, the proteomic coordinates have already been extracted at the amino acid level. Since each amino acid in the protein sequence corresponds to a triplet of nucleotides (a codon) in the mRNA, we need to multiply the proteomic coordinate by 3 to obtain the genomic coordinate. This conversion will give us the position of each amino acid within the genomic sequence. The resulting genomic coordinates are stored in a separate column for easy reference. Once this step is completed, we can extract and organize the information in the correct order for further analysis or mapping to the genomic reference.


> <hands-on-title> Text manipulation - SAV-derived peptides </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `sav_formatted_table` (output of **Column Regex Find And Replace - SAV** {% icon tool %})
>    - *"SQL Query to generate tabular output"*:
> ```
> SELECT t1.*,
> (t1.c4 - 1) * 3 AS c2_multiplied,
> t1.c5 * 3 AS c3_multiplied
> FROM t1
> ```
>    - *"include query result column headers"*: `No`
>
>
{: .hands_on}


### Annotating the genomic coordinates - SAV-derived peptides

The Query Tabular step in this workflow is used to extract and calculate genomic coordinates based on the proteomic data. The SQL query within the tool defines two calculations for genomic coordinates, start and stop, based on the strand information of the data. For each row in the input dataset (t1), if the strand (t1.c7) is "-" (negative), the genomic coordinates are calculated by subtracting the position from the given end (t1.c3 - t1.c9 for start, and t1.c3 - t1.c10 for stop). If the strand is "+" (positive), the genomic coordinates are calculated by adding the respective positions (t1.c2 + t1.c9 for start, and t1.c2 + t1.c10 for stop). These calculated coordinates are then returned in the query results, where they will be included as new columns (start and stop). This step is essential for transforming the proteomic information into genomic positions for further analysis.

> <hands-on-title> Text processing - SAV </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `output` (output of **Text manipulation - SAV** {% icon tool %})
>    - *"SQL Query to generate tabular output"*:
> ```
> SELECT t1.*,
> CASE
> WHEN t1.c7 = '-' THEN t1.c3 - t1.c9
> WHEN t1.c7 = '+' THEN t1.c2 + t1.c9
> END AS start,
> CASE
> WHEN t1.c7 = '-' THEN t1.c3 - t1.c10
> WHEN t1.c7 = '+' THEN t1.c2 + t1.c10
> END AS stop FROM t1
> ```
>    - *"include query result column headers"*: `No`
>
>
{: .hands_on}



### Generating BED file for PepPointer - SAV-derived peptides
This step is necessary to extract and reorganize relevant genomic information from the dataset. By querying specific columns such as chromosome (chromosome), start (chromStart), end (chromEnd), and strand (strand), we are preparing the data for further analysis. These values are essential for mapping proteomic or peptide data to the genomic coordinates, ensuring accurate alignment and interpretation of the sequence in the context of its genomic location. Additionally, renaming columns enhances clarity and standardizes the format, making it easier to work with the data in subsequent steps.

> <hands-on-title> BED file for SAV-derived peptides </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `output` (output of **Text processing - SAV** {% icon tool %})
>    - *"SQL Query to generate tabular output"*:
> ```
> SELECT
> c8 AS `chromosome`,
> c11  AS `chromStart`,
> c12 AS `chromEnd`,
> c1 AS `name`,
> c6 AS `score`,
> c7 AS `strand`
> FROM t1
> ```
>    - *"include query result column headers"*: `No`
>
> 2. Change data type to ".bed".
> 
{: .hands_on}


## Mapping SAV-derived Peptide sequences with PepPointer

PepPointer is a tool designed to map peptide sequences to their respective genomic locations using data such as GTF and BED files. In this workflow, PepPointer takes the GTF file, which contains gene annotations and genomic coordinates, and combines it with a BED file, which provides chromosomal coordinates of the peptides. By doing this, PepPointer can identify the exact genomic locations of the peptides, linking them to genes and exons. This step is crucial for accurately correlating proteomic data to genomic sequences, enabling a better understanding of the genomic context in which the peptides are found.

> <hands-on-title> PepPointer for SAV-derived peptides </hands-on-title>
>
> 1. {% tool [PepPointer](toolshed.g2.bx.psu.edu/repos/galaxyp/pep_pointer/pep_pointer/0.1.3+galaxy1) %} with the following parameters:
>    - *"Choose the source of the GTF file"*: `From history`
>        - {% icon param-file %} *"GTF file with the genome of interest"*: `Homo_sapiens.GRCh38_canon.106.gtf` (Input dataset)
>    - {% icon param-file %} *"BED file with chromosomal coordinates of peptide"*: `output` (from **Query Tabular - BED file for Assembly**)
>
>
{: .hands_on}


## Visualization and Interpretation of Novel SAV-derived peptides with annotation

In this step, we are using Query Tabular to extract and format relevant information from the results produced by PepPointer. The SQL query in this tool is designed to structure the data into a more readable format, providing key details such as the peptide ID, chromosome location, start and end positions, strand orientation, and any annotations. Additionally, it generates genome coordinates in a format suitable for viewing in genome browsers like IGV and UCSC Genome Browser. This formatting step helps to visualize and interpret the data more effectively by linking the peptides to their genomic context.

> <hands-on-title> Query Tabular </hands-on-title>
>
> 1. {% tool [Query Tabular](toolshed.g2.bx.psu.edu/repos/iuc/query_tabular/query_tabular/3.3.2) %} with the following parameters:
>    - In *"Database Table"*:
>        - {% icon param-repeat %} *"Insert Database Table"*
>            - {% icon param-file %} *"Tabular Dataset for Table"*: `classified` (output of **PepPointer - SAV** {% icon tool %})
>    - *"SQL Query to generate tabular output"*:
> ```
> SELECT
> c4 AS Peptide,
> c1 AS Chromosome,
> c2 AS Start,
> c3 AS End,
> c6 AS Strand,
> c7 AS Annotation,
> c1||':'||c2||'-'||c3 AS IGV_Genome_Coordinate,
> 'https://genome.ucsc.edu/cgi-bin/hgTracks?db=hg38&position='||c1||'%3A'||c2||'-'||c3 AS UCSC_Genome_Browser
> FROM  t1
> ```
>    - *"include query result column headers"*: `Yes`
>
>
{: .hands_on}


# B: Database for IEDB
![Variant-annotation-overview-workflow]({% link topics/proteomics/images/neoantigen/PepPointer_Characterization_3.PNG %})

## Generating FASTA for MHC binding tool
This output is an input for the next workflow (HLA Binding Novel Peptides).

> <hands-on-title> Tabular-to-FASTA </hands-on-title>
>
> 1. {% tool [Tabular-to-FASTA](toolshed.g2.bx.psu.edu/repos/devteam/tabular_to_fasta/tab2fasta/1.1.1) %} with the following parameters:
>    - {% icon param-file %} *"Tab-delimited file"*: `output` (output of **Query Tabular** {% icon tool %})
>    - *"Title column(s)"*: `2`
>    - *"Sequence column"*: `1`
>
>
{: .hands_on}


# Conclusion

In this tutorial, we have covered the process of annotating neoantigen peptides using the PepPointer tool in a bioinformatics pipeline. The main focus was to guide you through the various steps involved in the analysis, including querying tabular datasets, converting data formats, and using the PepPointer tool to classify peptides based on genomic information.

By the end of this tutorial, you should now be able to:
- Understand the steps involved in preparing and processing peptide data for neoantigen prediction.
- Use tools like Query Tabular, Convert, and PepPointer to manipulate and classify peptide sequences.
- Recognize the importance of annotating neoantigens for immunotherapy research and how bioinformatics tools can facilitate this process.

The key takeaway from this tutorial is the ability to manipulate complex genomic data and generate insights into potential therapeutic targets using bioinformatics techniques. You should now be better equipped to handle similar datasets and apply these techniques to other types of genomic and proteomic research.

# Rerunning on your own data

To rerun this entire analysis at once, you can use our workflow. Below we show how to do this:


> <hands-on-title>Running the Workflow</hands-on-title>
>
> 1. **Import the workflow** into Galaxy:
>    - [Neoantigen Variant Annotation](https://tinyurl.com/ipepgen-pep-annot-wf)
>
>
> 2. Run **Workflow** {% icon workflow %} using the following parameters:
>    - *"Send results to a new history"*: `No`
>    - {% icon param-file %} *"Genomic-Variant-BED-file"*: `Genomic-Variant-BED-file.tabular`
>    - {% icon param-file %} *"Novel Peptides from PepQuery"*: `Novel_Peptides_from_PepQuery.tabular`
>    - {% icon param-file %} *"Fragpipe Peptide Report"*: `Fragpipe-Peptide-Report.tabular`
>    - {% icon param-file %} *"Annotated-GffCompared-GTFtoBED"*: `GffCompare_Annotated_GTF_to_BED.bed`
>    - {% icon param-file %} *"Human Reference Genome Annotation"*: `Homo_sapiens.GRCh38_canon.106.gtf`
>
>    {% snippet faqs/galaxy/workflows_run.md %}
>
> 
> 
> 
>
> <comment-title>DISCLAIMER</comment-title>
>
>  - If any step in this workflow fails, please ensure that the input files have been correctly generated and formatted by the preceding tools. Workflow failures often result from improperly called or incomplete input data rather than errors in the workflow itself. Users are responsible for verifying their input before troubleshooting workflow issues.
>
> {: .comment}
>
>
>
{: .hands_on}

# Are you feeling adventurous? ✨

## One-Click Neoantigen Workflow

This new [One-Click Neoantigen Workflow](https://usegalaxy.eu/u/galaxyp/w/ipepgen-one-click-workflow) brings together all key modules of the neoantigen discovery process into a single, streamlined analysis within Galaxy. Instead of launching each tutorial separately, users can now execute the entire end-to-end pipeline—from database creation to HLA binding prediction—with just one click —without ever leaving Galaxy.

# Disclaimer

Please note that all the software tools used in this workflow are subject to version updates and changes. As a result, the parameters, functionalities, and outcomes may differ with each new version. We recommend that users verify the specific versions of software tools used to ensure the reproducibility and accuracy of results.


