Suicidal Intention Detection in Tweets Using BERT-Based Transformers

Abstract

Suicidal intention or ideation detection is one of the evolving research fields in social media. People use this platform to share their thoughts, tendencies, opinions, and feelings toward suicide. Therefore, this task becomes a challenging one due to the unstructured and noisy texts. In this paper, we propose five BERT-based pre-trained transformer models, namely, BERT, DistilBERT, ALBERT, RoBERTa, and DistilRoBERTa, for the task of suicidal intention detection. The performance of these models evaluated using the standard classification metrics. Specifically, we use the one-cycle learning rate policy to train all models. Our results show that the RoBERTa model achieves a better performance than other BERT-based models. The model gains 99.23%, 96.35%, and 95.39% accuracy for training, validation, and testing, respectively.