Validation of Sentiment Markers Extracted by Using Machine Learning: Twitter Mining of COVID-19

Although sentiment analysis for COVID-19 tweets is becoming popular, no study has mined a sentiment other than polarity. This study aims to extract ‘disaster’ sentiment by using various machine learning models, statistically validate sentiment markers extracted by deep learning, and discuss the potentials of ‘disaster’ as valid sentiment marker.

A total of 7,613 disaster tweets from Kaggle site were utilized to train nine machine learning models. A total of 15,619 tweets in English sent from USA were downloaded using streaming API with keywords of Covid, and Omicron, respectively and were classified into disaster/non-disaster categories using the four best performing models: MNB, deep learning, USE and BERT. Principal component analysis, correlation analysis and regression analysis were performed to determine the psychometric properties.

Cronbach Alpha for 13 sentiment markers was 0.71. All 4 machine learning markers were loaded in a factor. A higher level of unfavorable emotions (e.g., fear), a lower level of favorable emotions (e.g., joy), and a higher level of negative polarity were found during surging Omicron variants than early onset of COVID-19. The higher frequency of disaster tweets was found during surging Omicron variants than the early onset of COVID-19.

Our study revealed that disaster tweets were characterized to be a higher level of unfavorable emotions and negative polarity, and a lower level of favorable emotions. Since disaster as a sentiment marker was evidently reliable and valid, it should be a part of the sentiment analysis in describing the global health issues.