Bilingual emotion recognition of social media texts using deep learning approach

Melkam Enyew Gashaw, Abebaw Alem, Anduamlak Abebe Fenta

On social media platforms, users share their thoughts and opinions through comments in multiple languages, including English and Amharic, resulting in vast amounts of data. Accurately interpreting and understanding these comments has important practical implications, with potential applications in customer sentiment analysis, public opinion monitoring, and mental health awareness. Therefore, emotion classification tasks are crucial. People in Bilingual and multinational societies commonly employ a code mix in both spoken and written communication. Among these, Amharic and English language speakers frequently mix the two languages when conversing and posting on social media. However, analyzing emotions in the bilingual nature is not addressed by other researches and it poses considerable challenges. This research would utilize CNN and BiLSTM, to classify emotions of Amharic and English social media comments. The dataset used for this study was collected from social media and Kaggle repository, and it was annotated by experts into six distinct classes. After cleaning the data, we have 97,728 instances for model development, which we divided into 80% for training, 10% for validation, and the remaining 10% for testing the model. The model’s performance is assessed using metrics including accuracy, precision, recall, F1-score, and AUC-ROC. Using Keras word embeddings, the Convolutional Neural Network model achieved testing accuracies of 89.94%, While BiLSTM model had a testing accuracy of 88.99%. With FastText embeddings, the BiLSTM model achieved testing accuracy of 91.79%, while CNN achieved testing accuracy 91.16%. The results indicate that the BiLSTM model delivers better overall performance than the Convolutional Neural Network model, especially when paired with FastText embeddings.