Publication
Title
PEvoLM : Protein Sequence Evolutionary Information Language Model
Author
Abstract
With the exponential increase of the protein sequence databases overtime, multiple-sequence alignment (MSA) methods, like PSI-BLAST, perform exhaustive and time-consuming database search to retrieve evolutionary information. The resulting position-specific scoring matrices (PSSMs) of such search engines represent a crucial input to many machine learning (ML) models in the field of bioinformatics and computational biology. A protein sequence is a collection of contiguous tokens or characters called amino acids (AAs). The analogy to natural language allowed us to exploit the recent advancements in the field of Natural Language Processing (NLP) and therefore transfer NLP state-of-the-art algorithms to bioinformatics. This research presents an Embedding Language Model (ELMo), converting a protein sequence to a numerical vector representation. While the original ELMo trained a 2-layer bidirectional Long Short-Term Memory (LSTMs) network following a two-path architecture, one for the forward and the second for the backward pass, by merging the idea of PSSMs with the concept of transfer-learning, this work introduces a novel bidirectional language model (bi-LM) with four times less free parameters and using rather a single path for both passes. The model was trained not only on predicting the next AA but also on the probability distribution of the next AA derived from similar, yet different sequences as summarized in a PSSM, simultaneously (multi-task learning), hence learning evolutionary information of protein sequences as well. The network architecture and the pre-trained model are made available as open source under the permissive MIT license on GitHub at https://github.com/issararab/PEvoLM.
Language
English
Source (book)
2023 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), 29-31 August 2023, Eindhoven, Netherlands
Publication
IEEE Xplore , 2023
ISBN
979-83-503-1017-7
979-83-503-1018-4
DOI
10.1109/CIBCB56990.2023.10264890
Volume/pages
(2023) , p. 1-8
ISI
001090563700036
Full text (Publisher's DOI)
Full text (open access)
Full text (publisher's version - intranet only)
UAntwerpen
Publication type
Subject
Affiliation
Publications with a UAntwerp address
External links
Web of Science
Record
Identifier
Creation 24.10.2023
Last edited 17.06.2024
To cite this reference