Please use this identifier to cite or link to this item: https://hdl.handle.net/11147/11863
Title: Author reputation measurement on question and answer sites by the classification of author-generated content
Authors: Sezerer, Erhan
Tenekeci, Samet
Acar, Ali
Baloğlu, Bora
Tekir, Selma
Izmir Institute of Technology
Izmir Institute of Technology
Izmir Institute of Technology
Izmir Institute of Technology
Izmir Institute of Technology
Keywords: Author reputation measurement
Question and answer sites
Stack exchange
Stack overflow
Text classification
Issue Date: Oct-2021
Publisher: World Scientific Publishing
Abstract: In the field of software engineering, practitioners' share in the constructed knowledge cannot be underestimated and is mostly in the form of grey literature (GL). GL is a valuable resource though it is subjective and lacks an objective quality assurance methodology. In this paper, a quality assessment scheme is proposed for question and answer (Q&A) sites. In particular, we target stack overflow (SO) and stack exchange (SE) sites. We model the problem of author reputation measurement as a classification task on the author-provided answers. The authors' mean, median, and total answer scores are used as inputs for class labeling. State-of-the-art language models (BERT and DistilBERT) with a softmax layer on top are utilized as classifiers and compared to SVM and random baselines. Our best model achieves 63.8% accuracy in binary classification in SO design patterns tag and 71.6% accuracy in SE software engineering category. Superior performance in SE software engineering can be explained by its larger dataset size. In addition to quantitative evaluation, we provide qualitative evidence, which supports that the system's predicted reputation labels match the quality of provided answers.
URI: https://doi.org10.1142/S0218194021500479
https://hdl.handle.net/11147/11863
Appears in Collections:Computer Engineering / Bilgisayar Mühendisliği
Scopus İndeksli Yayınlar Koleksiyonu / Scopus Indexed Publications Collection
WoS İndeksli Yayınlar Koleksiyonu / WoS Indexed Publications Collection

Files in This Item:
File Description SizeFormat 
author_reputation.pdf
  Until 2025-01-01
525.37 kBAdobe PDFView/Open    Request a copy
Show full item record

CORE Recommender

Page view(s)

12,918
checked on Jul 4, 2022

Google ScholarTM

Check

Altmetric


Items in GCRIS Repository are protected by copyright, with all rights reserved, unless otherwise indicated.