Please use this identifier to cite or link to this item: https://hdl.handle.net/11147/14815
Title: Improvements on a Multi-Task Bert Model
Other Titles: Çok Görevli Bert Modeli Üzerinde Iyileştirmeler
Authors: Agrali, Mahmut
Tekir, Selma
Keywords: Multi-Task Learning
Sentiment Analysis
Paraphrase Detection
Semantic Textual Similarity
Publisher: Ieee
Series/Report no.: Signal Processing and Communications Applications Conference
Abstract: Pre-trained language models have introduced significant performance boosts in natural language processing. Fine-tuning of these models using downstream tasks' supervised data further improves the acquired results. In the fine-tuning process, combining the learning of tasks is an effective approach. This paper proposes a multi-task learning framework based on BERT. To accomplish the tasks of sentiment analysis, paraphrase detection, and semantic text similarity, we include linear layers, a Siamese network with cosine similarity, and convolutional layers to the appropriate places in the architecture. We conducted an ablation study using Stanford Sentiment Treebank (SST), Quora, and SemEval STS datasets for each task to test the framework and its components' effectiveness. The results demonstrate that the proposed multi-task framework improves the performance of BERT. The best results obtained for sentiment analysis, paraphrase detection, and semantic text similarity are accuracies of 0.534 and 0.697 and a Pearson correlation coefficient of 0.345.
URI: https://doi.org/10.1109/SIU61531.2024.10600801
ISBN: 9798350388978
9798350388961
ISSN: 2165-0608
Appears in Collections:Scopus İndeksli Yayınlar Koleksiyonu / Scopus Indexed Publications Collection
WoS İndeksli Yayınlar Koleksiyonu / WoS Indexed Publications Collection

Show full item record



CORE Recommender
Sorry the service is unavailable at the moment. Please try again later.

Page view(s)

88
checked on Mar 31, 2025

Google ScholarTM

Check





Items in GCRIS Repository are protected by copyright, with all rights reserved, unless otherwise indicated.