A Zero-Shot Aspect-Based Sentiment Analysis of Public Perception Toward AI Chatbots
DOI:
https://doi.org/10.38043/tiers.v6i1.6488Keywords:
Aspect-Based Sentiment Analysis (ABSA), Zero-Shot Learning, AI chatbots, Public Perception, Sentiment AnalysisAbstract
The rapid development of AI chatbots has sparked public discussion on social media regarding their performance, ethical implications, and related concerns. While past studies primarily focused on individual chatbot model using traditional sentiment analysis, this study implements a novel application of Zero-Shot Aspect-Based Sentiment Analysis (ABSA) on 17,562 tweets mentioning AI chatbots such as ChatGPT, Bard (now Gemini), and DeepSeek, utilizing an efficient sentiment extraction method without supervised training. Six aspects were analyzed to understand the sentiment pattern and the results show the discussion was dominated by negative sentiment, with Bard receiving the most positive sentiment, potentially shaped by brand trust and user familiarity. On the other hand, DeepSeek and ChatGPT attracted more criticism, especially related to performance and bias aspects. This study offers data-driven suggestions for developers, including improving response accuracy to shape user trust, reducing biased output, and developing real-time discourse analysis. Future work should incorporate multiple platforms to avoid bias, analyze more AI chatbot models, and include temporal sentiment for broader insights.
Downloads
References
S. Singh Sengar, Affan, B. Hasan, S. Kumar, F. Carroll, and A. Bin Hasan, Multimedia Tools and Applications Generative artificial intelligence: a systematic review and applications, Multimed Tools Appl, 2024, doi: 10.1007/s11042-024-20016-1.
B. Ramdurai and P. Adhithya, The Impact, Advancements and Applications of Generative AI, International Journal of Computer Science and Engineering, vol. 10, no. 6, 2023, doi: 10.14445/23488387/ijcse-v10i6p101.
P. Welsby and B. M. Y. Cheung, ChatGPT, Postgrad Med J, vol. 99, no. 1176, pp. 10471048, Oct. 2023, doi: 10.1093/postmj/qgad056.
R. Koonchanok, Y. Pan, and H. Jang, Public attitudes toward chatgpt on twitter: sentiments, topics, and occupations, Soc Netw Anal Min, vol. 14, no. 1, p. 106, 2024, doi: 10.1007/s13278-024-01260-7.
S. K. Singh, S. Kumar, and P. S. Mehra, ChatGPT & Google Bard AI: A Review, 2023 International Conference on IoT, Communication and Automation Technology, ICICAT 2023, 2023, doi: 10.1109/ICICAT57735.2023.10263706.
D. Milmo, Google trials its own AI chatbot Bard after success of ChatGPT. Accessed: Feb. 18, 2025. [Online]. Available: https://www.theguardian.com/technology/2023/feb/06/google-releases-its-own-ai-chatbot-bard-after-success-of-chatgpt
S. Sutner, ChatGPT bursts into Microsoft Bing as Google Bard rises. Accessed: Feb. 18, 2025. [Online]. Available: https://www.techtarget.com/searchenterpriseai/news/365530303/ChatGPT-bursts-into-Microsoft-Bing-as-Google-Bard-rises
Daniel Newman, Exploring The Ins And Outs Of The Generative AI Boom. Accessed: Feb. 17, 2025. [Online]. Available: https://www.forbes.com/sites/danielnewman/2023/03/14/exploring-the-ins-and-outs-of-the-generative-ai-boom/
A. Rowe, What Is DeepSeek? Chinas New AI Is Now Open-Source. Accessed: Feb. 18, 2025. [Online]. Available: https://tech.co/news/what-is-deepseek
K.-S. Huang, Chinas AI Shock? What DeepSeek Disrupts (and Doesnt) The Diplomat. Accessed: Feb. 18, 2025. [Online]. Available: https://thediplomat.com/2025/01/chinas-ai-shock-what-deepseek-disrupts-and-doesnt/
S. Throne, Putting DeepSeek to the test: how its performance compares against other AI tools. Accessed: Feb. 18, 2025. [Online]. Available: https://theconversation.com/putting-deepseek-to-the-test-how-its-performance-compares-against-other-ai-tools-248368
V. Barassi, Toward a Theory of AI Errors: Making Sense of Hallucinations, Catastrophic Failures, and the Fallacy of Generative AI, Harv Data Sci Rev, vol., no. Special Issue 5, Nov. 2024, doi: 10.1162/99608F92.AD8EBBD4.
V. D. Kirova, C. S. Ku, J. R. Laracy, and T. J. Marlowe, The Ethics of Artificial Intelligence in the Era of Generative AI, J Syst Cybern Inf, vol. 21, no. 4, 2023, doi: 10.54808/jsci.21.04.42.
C. Graham and R. Stough, Consumer perceptions of AI chatbots on Twitter (X) and Reddit: an analysis of social media sentiment and interactive marketing strategies, Journal of Research in Interactive Marketing, vol. ahead-of-print, no. ahead-of-print, Jan. 2025, doi: 10.1108/JRIM-05-2024-0237.
S. Geman and M. Johnson, Probabilistic Grammars and their Applications, in International Encyclopedia of the Social & Behavioral Sciences: Second Edition, 2015. doi: 10.1016/B978-0-08-097086-8.42161-6.
T. Huynh-The, Q. V. Pham, X. Q. Pham, T. T. Nguyen, Z. Han, and D. S. Kim, Artificial intelligence for the metaverse: A survey, Eng Appl Artif Intell, vol. 117, p. 105581, Jan. 2023, doi: 10.1016/J.ENGAPPAI.2022.105581.
X. Fang and J. Zhan, Sentiment analysis using product review data, J Big Data, vol. 2, no. 1, 2015, doi: 10.1186/s40537-015-0015-2.
A. Tanaltay, A. S. Langroudi, R. Akhavan-Tabatabaei, and N. Kasap, Can Social Media Predict Soccer Clubs Stock Prices? The Case of Turkish Teams and Twitter, Sage Open, vol. 11, no. 2, 2021, doi: 10.1177/21582440211004153.
B. Liu, Sentiment analysis and opinion mining, Synthesis Lectures on Human Language Technologies, vol. 5, no. 1, 2012, doi: 10.2200/S00416ED1V01Y201204HLT016.
S. Bhattacharya, D. Sarkar, D. K. Kole, and P. Jana, Chapter 9 - Recent trends in recommendation systems and sentiment analysis, in Advanced Data Mining Tools and Methods for Social Computing, S. De, S. Dey, S. Bhattacharyya, and S. Bhatia, Eds., Academic Press, 2022, pp. 163175. doi: https://doi.org/10.1016/B978-0-32-385708-6.00016-3.
S. Yin, The Current State and Challenges of Aspect-Based Sentiment Analysis, Applied and Computational Engineering, vol. 114, pp. 2531, Dec. 2024, doi: 10.54254/2755-2721/2024.18197.
L. Shu, H. Xu, B. Liu, and J. Chen, Zero-Shot Aspect-Based Sentiment Analysis, Feb. 2022.
D. Van Thin, H. Quoc Ngo, D. Ngoc Hao, and N. Luu-Thuy Nguyen, Exploring zero-shot and joint training cross-lingual strategies for aspect-based sentiment analysis based on contextualized multilingual language models, Journal of Information and Telecommunication, vol. 7, no. 2, 2023, doi: 10.1080/24751839.2023.2173843.
W. Wang, V. W. Zheng, H. Yu, and C. Miao, A survey of zero-shot learning: Settings, methods, and applications, 2019. doi: 10.1145/3293318.
I. Nawawi, K. F. Ilmawan, M. R. Maarif, and M. Syafrudin, Exploring Tourist Experience through Online Reviews Using Aspect-Based Sentiment Analysis with Zero-Shot Learning for Hospitality Service Enhancement, Information, vol. 15, no. 8, 2024, doi: 10.3390/info15080499.
S. R et al., Analyzing Sentiments Regarding ChatGPT Using Novel BERT: A Machine Learning Approach, Information, vol. 14, no. 9, p. 474, Aug. 2023, doi: 10.3390/info14090474.
S. Hussain et al., Investigating public perception on use of ChatGPT in initial consultations prior to healthcare provider consultations, Annals of Medicine and Surgery, Oct. 2024, doi: 10.1097/MS9.0000000000002697.
L. Labadze, M. Grigolia, and L. Machaidze, Role of AI chatbots in education: systematic literature review, 2023. doi: 10.1186/s41239-023-00426-1.
K. Sailunaz and R. Alhajj, Emotion and sentiment analysis from Twitter text, J Comput Sci, vol. 36, 2019, doi: 10.1016/j.jocs.2019.05.009.
S. K. Kumar et al., Stock Price Prediction Using Optimal Network Based Twitter Sentiment Analysis, Intelligent Automation and Soft Computing, vol. 33, no. 2, 2022, doi: 10.32604/iasc.2022.024311.
BwandoWando, Tweets and Reactions on DeepSeek , 2025. [Online]. Available: https://www.kaggle.com/datasets/bwandowando/tweets-and-reaction-on-deepseek-models
sina tavakoli, AI based platforms, 2023. [Online]. Available: https://www.kaggle.com/datasets/sinatavakoli/ai-based-platforms?select=bard.csv
Z. Wang, Y. Pang, and Y. Lin, Large Language Models Are Zero-Shot Text Classifiers, Dec. 2023, doi: 10.48550/arXiv.2312.01044.
P. He, X. Liu, J. Gao, and W. Chen, DeBERTa: Decoding-enhanced BERT with Disentangled Attention, in ICLR 2021 - 9th International Conference on Learning Representations, Jun. 2021. Accessed: Jun. 19, 2025. [Online]. Available: https://arxiv.org/abs/2006.03654
N. D. Cohen, M. Ho, D. McIntire, K. Smith, and K. A. Kho, A comparative analysis of generative artificial intelligence responses from leading chatbots to questions about endometriosis, AJOG Global Reports, vol. 5, no. 1, p. 100405, Feb. 2025, doi: 10.1016/J.XAGR.2024.100405.
A. Williams, Comparison of generative AI performance on undergraduate and postgraduate written assessments in the biomedical sciences, International Journal of Educational Technology in Higher Education, vol. 21, no. 1, p. 52, Sep. 2024, doi: 10.1186/s41239-024-00485-y.
D. Lee, M. Brown, J. Hammond, and M. Zakowski, Readability, quality and accuracy of generative artificial intelligence chatbots for commonly asked questions about labor epidurals: a comparison of ChatGPT and Bard, Int J Obstet Anesth, vol. 61, p. 104317, Feb. 2025, doi: 10.1016/j.ijoa.2024.104317.
R. L. Fleurence et al., Generative Artificial Intelligence for Health Technology Assessment: Opportunities, Challenges, and Policy Considerations: An ISPOR Working Group Report, Value in Health, vol. 28, no. 2, pp. 175183, Feb. 2025, doi: 10.1016/J.JVAL.2024.10.3846.
L. Li, Y. Zhang, J. Sun, and Z. Yang, Generative Artificial Intelligence Application Pricing: Navigating Upstream Strategy, Consumer Behavior, and Market Competition, 2024. doi: 10.2139/ssrn.4956548.
M. Valizadeh, P. Ranjbar-Noiey, C. Caragea, and N. Parde, Identifying Medical Self-Disclosure in Online Communities, in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Stroudsburg, PA, USA: Association for Computational Linguistics, 2021, pp. 43984408. doi: 10.18653/v1/2021.naacl-main.347.
V. Sanh, L. Debut, J. Chaumond, and T. Wolf, DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter, Oct. 2019, Accessed: Jun. 19, 2025. [Online]. Available: https://arxiv.org/pdf/1910.01108
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Naufal Andila Fauzan
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
