Emotion detection of social data: APIs comparative study

Abstract

The development of emotion detection technology has emerged as a highly valuable possibility in the corporate sector due to the nearly limitless uses of this new discipline, particularly with the unceasing propagation of social data. In recent years, the electronic marketplace has witnessed the establishment of a large number of start-up businesses with an almost sole focus on building new commercial and open-source tools and APIs for emotion detection and recognition. Yet, these tools and APIs must be continuously reviewed and evaluated, and their performances should be reported and discussed. There is a lack of research to empirically compare current emotion detection technologies in terms of the results obtained from each model using the same textual dataset. Also, there is a lack of comparative studies that apply benchmark comparison to social data. This study compares eight technologies; IBM Watson NLU, ParallelDots, Symanto-Ekman, Crystalfeel, Text to Emotion, Senpy, Textprobe, and NLP Cloud. The comparison was undertaken using two different datasets. The emotions from the chosen datasets were then derived using the incorporated APIs. The performance of these APIs was assessed using the aggregated scores that they delivered as well as the theoretically proven evaluation metrics such as the micro-average of accuracy, classification error, precision, recall, and f1-score. Lastly, the assessment of these APIs incorporating the evaluation measures is reported and discussed.