Please use this identifier to cite or link to this item:
https://hdl.handle.net/11147/14815
Title: | Improvements on a Multi-Task Bert Model |
Other Titles: | Çok Görevli Bert Modeli Üzerinde Iyileştirmeler |
Authors: | Agrali, Mahmut Tekir, Selma |
Keywords: | Multi-Task Learning Sentiment Analysis Paraphrase Detection Semantic Textual Similarity |
Publisher: | Ieee |
Series/Report no.: | Signal Processing and Communications Applications Conference |
Abstract: | Pre-trained language models have introduced significant performance boosts in natural language processing. Fine-tuning of these models using downstream tasks' supervised data further improves the acquired results. In the fine-tuning process, combining the learning of tasks is an effective approach. This paper proposes a multi-task learning framework based on BERT. To accomplish the tasks of sentiment analysis, paraphrase detection, and semantic text similarity, we include linear layers, a Siamese network with cosine similarity, and convolutional layers to the appropriate places in the architecture. We conducted an ablation study using Stanford Sentiment Treebank (SST), Quora, and SemEval STS datasets for each task to test the framework and its components' effectiveness. The results demonstrate that the proposed multi-task framework improves the performance of BERT. The best results obtained for sentiment analysis, paraphrase detection, and semantic text similarity are accuracies of 0.534 and 0.697 and a Pearson correlation coefficient of 0.345. |
URI: | https://doi.org/10.1109/SIU61531.2024.10600801 |
ISBN: | 9798350388978 9798350388961 |
ISSN: | 2165-0608 |
Appears in Collections: | Scopus İndeksli Yayınlar Koleksiyonu / Scopus Indexed Publications Collection WoS İndeksli Yayınlar Koleksiyonu / WoS Indexed Publications Collection |
Show full item record
CORE Recommender
Sorry the service is unavailable at the moment. Please try again later.
Items in GCRIS Repository are protected by copyright, with all rights reserved, unless otherwise indicated.