Skip to main content
Erschienen in: Social Network Analysis and Mining 1/2024

Open Access 01.12.2024 | Original Article

Analyzing online public opinion on Thailand-China high-speed train and Laos-China railway mega-projects using advanced machine learning for sentiment analysis

verfasst von: Manussawee Nokkaew, Kwankamol Nongpong, Tapanan Yeophantong, Pattravadee Ploykitikoon, Weerachai Arjharn, Apirat Siritaratiwat, Sorawit Narkglom, Wullapa Wongsinlatam, Tawun Remsungnen, Ariya Namvong, Chayada Surawanitkun

Erschienen in: Social Network Analysis and Mining | Ausgabe 1/2024

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Sentiment analysis is becoming a very popular research technique. It can effectively identify hidden emotional trends in social networks to understand people’s opinions and feelings. This research therefore focuses on analyzing the sentiments of the public on the social media platform, YouTube, about the Thailand-China high-speed train project and the Laos-China Railway, a mega-project that is important to the country and a huge investment to develop transportation infrastructure. It affects both the economic and social dimensions of Thai people and is also an important route to connect the rail systems of ASEAN countries as part of the Belt and Road Initiative. We gathered public Thai reviews from YouTube using the Data Application Program Interface. This dataset was used to train six sentiment classifiers using machine learning and deep learning algorithms. The performance of all six models by means of precision, recall, F1-score and accuracy are compared to find the most suitable model architecture for sentiment classification. The results show that the transformer model with the WangchanBERTa language model yields best accuracy, 94.57%. We found that the use of a Thai language-specific model that was trained from a large variety of data sources plays a major role in the model performance and significantly increases the accuracy of sentiment prediction. The promising performance of this sentiment classification model also suggests that it can be used as a tool for government agencies to plan, make strategic decisions, and improve communication with the public for better understanding of their projects. Furthermore, the model can be integrated with any online platform to monitor people's sentiments on other public matters. Regular monitoring of public opinions could help the policy makers in designing public policies to address the citizens’ problems and concerns as well as planning development strategies for the country.
Hinweise

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

1 Introduction

The Belt and Road Initiative (BRI), announced by the Chinese government in 2013, is a massive infrastructure investment by China to connect Asia, Europe, and Africa. The BRI has more than 147 participating countries. Its long-term policies and investment plans present a highly interesting economic and geographic perspective. The BRI focuses on developing transportation infrastructure, through both land and sea routes, to link international markets and promote economic cooperation between various countries (Chirathivat et al. 2022). The railway linking China and ASEAN countries through Laos is a significant route in the BRI. Construction of this route began in 2016 and was completed in December 2021, covering a distance of approximately 417 km. It connects, Vientiane, the capital of Laos with southern China via the Laotian border town, Boten. This has enabled Laos to have railway infrastructure on a scale it has never had before, transforming it from a landlocked country to a land-linked country (Rowedder 2020). Additionally, it will also link up with the Thailand-China high-speed train in the future. The Thailand-China high-speed train project, a collaboration between the Thai and Chinese governments to develop a high-speed railway system linking the two regions, has been jointly agreed upon. A railway line from Bangkok to Nakhon Ratchasima will be built, covering a distance of 250 km, in Phase 1, before opening in 2026. The construction of the second phase of the railway from Nakhon Ratchasima to Nong Khai, covering a distance of 357 km, is scheduled to begin operations between 2029 and 2030. This railway is part of the BRI, aimed at extending a railway from southern China through Laos and Thailand to Malaysia (State Railway of Thailand 2022). Previously, Thailand's export trade relied mainly on sea and land transportation. This railway will serve as a strategic trade and investment connection between Thailand, Laos, and the People’s Republic of China. It is expected to increase the efficiency of product transportation, promote exports of Thai products to other countries, and foster further development of the ASEAN region. Notably, the China-Laos railway has commenced operations, while the Thailand-China railway is still under construction. Our social media analysis revealed that there is a considerable amount of interest in this topic among users, including content, news, with user opinions and criticism (Wei and Sukhotu 2021).
Social media can be regarded as a virtual community that enables users to communicate, exchange information, express positive and negative opinions, and reflect diverse perspectives of people worldwide through the internet (Hou et al. 2020). The use of social media platforms has significantly increased in recent years. According to research, there were more than 3.6 billion social media users in 2020, and this number is expected to increase to 4.41 billion by 2025 (Kemp 2023a). The global digital report by we are social, which surveys digital and social media behavior of consumers worldwide, reveals that in January 2023, the number of social media users reached 4.76 billion globally. This demonstrates a remarkably high growth rate of social media users, increasing by almost 30% since the outbreak of COVID-19. There are over 1 billion new users in the past 3 years (Kemp 2023a). The effectiveness of social media platforms for governments to connect with citizens in the digital world has been demonstrated (Yavetz and Aharony 2022; Wukich 2022). Many governments use social media platforms as tools to disseminate information on government projects, policies, measures to assist, natural disaster warnings, and pandemic situations, as well as to manage disinformation and provide trustworthy information and recommendations (Sesa et al. 2022; Yuan et al. 2023). They have become very important communication and marketing tools for global businesses, enabling entrepreneurs to explore new business opportunities, access advice from experts and a diverse range of knowledge sources, and serve as a marketing channel to promote products and services (Jiawen Chen, Linlin Liu, Social media usage and entrepreneurial investment: An information-based view, Journal of Business Research, Volume 155, Part B 2023). Conversations and interactions with target customers through social media have been found to increase customer engagement. Marketers generally believe that social media marketing is the most effective way for companies to promote their products (Eslami et al. 2022). Academics and researchers are increasingly using social media to share their research findings and disseminate scientific opinions to their followers and networks on various platforms in diverse formats and content. Not only does it enhance the visibility of academics in their fields, but it also opens up a wide range of ideas from diverse groups of people and academics in different fields, which can help improve research and development (Cao 2023).
Utilization of social media platforms for viewing video content has become increasingly prevalent among individuals globally. YouTube, in particular, ranks among the top social media platforms worldwide, with over 2 billion daily active users, making it one of the most widely used applications. Android users spend up to 23 h and 9 min per month on YouTube (Kemp 2023b; Mehta and Deshmukh 2022). YouTube was established in February 2005 and was acquired by Google in November 2006 (Yao et al. 2007). The platform, which offers not only entertainment but also educational, political, and business/marketing videos (Kang and Kim 2023), enables users to upload and share video content as well as express their opinions through comments (Mehta and Deshmukh 2022). In Thailand, there are approximately 52.25 million social media users, of which 43.90 million, or 84.2%, are YouTube users. From 2022 to 2023, the number of active users on YouTube increased to 1.1 million. Marketers are successful at advertising on YouTube Thailand with a 1.9 percent increase in ad spending at the end of 2022 (Kemp 2023a). Therefore, for the purpose of this study, we have selected and analyzed user comments from the YouTube platform to examine the attitudes and opinions of the public.
In the current study, the researchers chose to use public sentiment data about the Thailand-China high-speed train project and Laos-China railway for analysis. Both projects have significant implications for cross-border transportation and other related aspects. Thailand is currently in the process of constructing a railway to connect to Laos, which is a highly specific and important project with massive investment and wide-ranging impacts. However, these data have not been previously utilized in research to the extent that they should be, especially in research in the field of Natural Language Processing (NLP) in the Thai language related to the analysis of government projects. To achieve the research objective of comparing and finding the best model from the popular models used for sentiment classification, the current study can be used as a guideline for applying technology to analyze Thai language data related to government projects for more effectiveness in the future. Therefore, sentiment analysis was performed utilizing various model architectures to analyze and determine the most effective model for predicting outcomes.
Both machine learning (ML) and deep learning (DL) models were employed to determine the most effective model for predicting the sentiment expressed in comments related to the Chinese-Laos and Chinese-Thai high-speed railway systems, with respect to their economic, transportation, and tourism impacts. Data were collected from online activity from October 2014 to May 2022. Six models, including a Logistic Regression (LR), Naive Bayes (NB), Random Forest (RF), Bidirectional Long Short-Term Memory (Bi-LSTM), Bidirectional Encoder Representations from Transformers (BERT) and a Thai Language model Trained specifically for the BERT architecture, BERT-Base-Thai and WangchanBERTa, were compared based on their predictive performance. This approach can provide insights into various aspects of human emotions, including feelings, opinions, and perceptions, and provide useful recommendations for decision-making, strategic planning, communicating news to the public, and promoting entrepreneurship in various industries, such as logistics and tourism. Effective utilization of appropriate technology can enhance the potential and increase an organization’s competitiveness. With automated tools, the government can save time and budget in monitoring public opinion and conducting public opinion surveys. More resources and budget can be allocated for planning development strategies.
Sentiment analysis is a popular research technique. It can be used to determine the hidden trends of emotions and feelings using social network data (Tam et al. 2021). Machine learning is one of the most used tools for analyzing large amounts of data in various fields. Researchers take advantage of the learning capabilities of machines to mine sentiments and opinions (Twitter data sentiment analysis of tourism in Thailand during the COVID-19 pandemic using machine learning 2022). For instance, it has been used to analyze public confidence in COVID-19 vaccines. People extensively share and discuss their opinions about the pandemic, vaccines, and vaccination processes. Users post news and comments, both positive and negative. Negative and false information about vaccines and vaccination processes may create vaccine hesitancy and distrust of the government and pharmaceutical companies, causing people to lose interest in receiving the vaccine and reduce vaccination rates in a country (Sabab Zulfiker et al. 2022; Vishwakarma and Chugh 2023). Therefore, the government can use data analysis as a tool for planning vaccine allocation, budgeting, and problem-solving, as well as to support policy and strategic decision-making (Villavicencio et al. 2021).
The analysis of political sentiments has incorporated public opinions from social media to examine feelings and attitudes toward political parties, their policies, and the popularity of individual candidates vying for the Prime Minister position. This information is then utilized to refine campaigns and vote-seeking strategies, amend party policies, devise communication strategies with the public, disseminate political messages, and also anticipate upcoming election results (Rita et al. 2023).
Analysis of student opinions and perspectives in online social media has been utilized to evaluate their perceptions of new teaching and learning formats. The content of the curriculum, the satisfaction level of students, and their opinions about their educational institution have been considered to assess the effectiveness of the education system. For instance, whether the shift to an online teaching and learning format has affected the quality of education has been examined (Imran et al. 2022). This approach aims to provide university administrators with reliable information on student viewpoints, which can then be utilized for the development and evaluation of new teaching and learning models that are sustainable and adaptable (He et al. 2022).
Investors, economists, and researchers have attempted to find ways to beat the stock market by analyzing data on emotions and their correlation with stock prices. It has been found that confidence data obtained from a large number of individuals can be used to make informed decisions about the stock market. It can even be used to make predictions, especially given that social media serves as a channel for news that can have a direct impact on stock prices. The stock market is greatly impacted when popular individuals share their thoughts about specific stocks (Hasselgren et al. 2023). Thus, the analysis of emotional data for market prediction and correlation with stock prices can significantly benefit investors in making informed decisions on whether to buy or sell stocks (Koukaras et al. 2022).
Utilization of customer feedback and criticism in business practices has had significant economic repercussions, particularly in the transportation and tourism service sectors, following disease outbreaks. Analyzing customer satisfaction, service experiences, factors influencing satisfaction and customer emotions can help evaluate customer experiences and determine areas that require the most improvement, thereby leading to increased revenue generation (Ramos et al. 2023). Research has highlighted the importance of analyzing customer feedback data to understand the feelings about products. This provides in-depth business information that reveals customer attitudes and behaviors that reflect real consumer demands and highlight areas for product development to best meet customer needs. These data can facilitate strategic decision-making and sustainable business growth (Iqbal et al. 2022; Nandwani and Verma 2021). Sentiment analysis is applied in many areas and domains. Various ML algorithms have been used in sentiment analysis to classify polarity in different languages. However, research on Thai sentiment analysis is still limited (Pasupa et al. 2022). In particular, domains related to government projects have not been researched before. The comparison of suitable models in each domain is therefore essential.
Nine different ML algorithms have been implemented, including Support Vector Machine (SVM), Bernoulli Naïve Bayes (BNB), Ridge Regression (RR), Logistic Regression (LR), Random Forest (RF), Stochastic Gradient Decent (SGD), Passive Aggressive (PA), Decision Tree (DT), and AdaBoost (ADA). These were evaluated to find the most suitable model for creating a classification of opinions of hotel service reviews in Thailand from two online agencies, Agoda.com and Booking.com in Thai, comparing the values of recall, precision, F1 score, and accuracy (Khamphakdee and Seresangtakul 2021). Moreover, the research compared the performance of nine DL models (CNN, LSTM, Bi-LSTM, GRU, Bi-GRU, CNN-LSTM, CNN-Bi-LSTM, CNN-GRU, and CNN-Bi-GRU) with different numbers of layers. The underlying architecture for all models is similar, i.e., FastText and BERT pretrained models were used to perform sentiment polarity classification. The model demonstrated the potential of WangchanBERTa to improve accuracy. Factors like hyperparameter tuning, embedding dimensions, and the number of layers in the DL model affect the efficiency of sentiment classification. These research findings can be applied as a tool for Small and Midsized Enterprises (SME) to analyze the sentiment of the Thai language in the hotel domain. Therefore, comparison of suitable models in each domain is crucial (Khamphakdee and Seresangtakul 2023).
Recently, analyzed opinions were presented with agricultural technology (AgriTech) startups in Thailand on Facebook. This work analyzes sentiments using binary classification and categorizes posts and comments as positive and negative. A Naive Bayes classifier was used to determine the sentiments and attitudes of people and investors. The objective of this research is to reflect the awareness rate of agritech startups in Thailand as agricultural startups are one of Thailand’s 4.0 policies. They can be used to develop future agricultural startups and provide interesting points to investors and government sectors (Kewsuwun and Kajornkasirat 2022).

3 Methodology

This section describes collection of public opinion data. The processes and methods used for sentiment analysis are presented. Classification of individual comments on YouTube in Thai was done using machine learning techniques and deep learning models to compare the accuracy of each model. The results of the two highest-precision performance models are compared for each class in a tabular and confusion matrix format, with the highest-precision model presented in a Word Cloud, as demonstrated in Fig. 1.

3.1 Data collection

When analyzing opinion data on social media, most studies use information from Twitter to analyze sentiment. We conducted a survey on both Twitter and YouTube for keywords related to the Thailand-China high-speed train and Laos-China Railway and found that the number of user comments on Twitter is very low; there is not much mention of this. Unlike YouTube, where users make presentations on the platform and many users come to comment, together with the number of YouTube users in Thailand, 43.90 million, which is more than Twitter's 14.60 million users, the researcher therefore chose to use opinions from YouTube for the main analysis. Data collection was conducted via the YouTube Data Application Program Interface (API) based on six keywords that are related to the Thai High-Speed Railway Project and the Laos-China Railway. The keywords used in searching for related video contents in this work includes "รถไฟความเร็วสูง" (the high-speed train), "รถไฟความเร็วสูงกรุงเทพ-นครราชสีมา" (the Bangkok-Nakhon Ratchasima high-speed train), "รถไฟคความเร็วสูงกรุงเทพ-หนองคาย" (the Bangkok-Nong Khai high-speed train), "รถไฟความเร็วสูงไทย-จีน" (the Thailand-China high-speed train), "รถไฟจีนลาว" (the China-Laos railway) and "รถไฟลาวจีน" (the Laos-China railway). After related video content is located, the video information (e.g., title, description, and channel) along with the user comments are collected and exported as an initial dataset. The comments were made from October 2014 to May 2022, thus obtaining a dataset of 29,984 messages for analysis. Figure 2 shows the process of collecting data from YouTube. In part, sample messages and their relevant keywords from the dataset are shown in Table 1.
Table 1
Sample messages from the dataset
Keywords
Messages
รถไฟความเร็วสูง
(The high-speed train)
ทำไมไม่นำเสนอผลประโยชน์ที่จีนได้จากประเทศลาวบ้างละนอกจากโจมตีบ้านเกิดตัวเอง
(Why not present the benefits that China can receive from Laos, apart from attacking their own homeland?)
รถไฟความเร็วสูงกรุงเทพ-นครราชสีมา
(The Bangkok-Nakhon Ratchasima
high-speed train)
อยากให้รีบทำเสียเวลามานานแล้ว
(I wish it could be done sooner.)
รถไฟความเร็วสูงกรุงเทพ-หนองคาย
(The Bangkok-Nong Khai high-speed train)
ดีมากเลยกระจายความเจริญออกไป ใช้จ่ายแบบนี้สนับสนุน
(It's great that it spreads prosperity and is supported by this kind of spending.)
รถไฟความเร็วสูงไทย-จีน
(The Thailand-China high-speed train)
เดินหน้าพัฒนาโครงสร้างพื้นฐานทุกมิติเพื่อกระจายความเจริญ ลดความเหลื่อมล้ำ
(Advancing the development of infrastructure in all dimensions, to distribute prosperity and reduce inequality.)
รถไฟจีนลาว
(The China-Laos railway)
ประเทศไทยเราถนนลูกรังยังไม่หมดจึงยังไม่มีรถไฟความเร็วสูงเหมือนประเทศลาว
(In Thailand, we still have unfinished roads, which is why we don't have high-speed trains like Laos.)
รถไฟลาวจีน
(The Laos-China railway)
ยินดีกับพี่น้อง สปป.ลาว รถไฟสายนี้เปิดโอกาสในการค้าขายระหว่างกันได้สะดวกยิ่งขึ้น
Congratulations to our Lao PDR brothers and sisters. This railway opens up opportunities for more convenient trade between us

3.2 Data labeling

A panel of three individuals is asked to classify each message as positive, neutral, or negative. Each expert independently assigned the label to the messages. In order to ensure the validity of the dataset, messages that were labeled differently were excluded. Only the messages that unanimously received the same label from the panel were considered and used for the model training and evaluation. The process of data labeling is shown in Fig. 3. After filtering out ambiguously labeled data, 10,582 expert-classified messages were used in sentiment analysis. The final dataset contains 3761 (35.54%) neutral messages, followed by 3502 (33.09%) negative messages and 3319 (31.36%) positive messages, which is quite balanced.

3.3 Preprocessing

Data preprocessing is an important step in text processing as it involves eliminating unnecessary and irrelevant data. The main objective of preprocessing is to prepare the data for further processing and analysis by machine learning or deep learning models. This step helps increase accuracy and reduces processing time. One challenge of working with Thai text analysis is that Thai is a low-resource language, and it is very complex. The preprocessing step is generally different than that of English because it is written with no spaces between words and no pauses to set a clear ending of a sentence. Thai text processing requires more complicated word tokenization and sentence boundary segmentation algorithms. There are often slang words, abbreviations, numbers, symbols, and misspellings (Khamphakdee and Seresangtakul 2021). The data preprocessing conducted in this study consists of:
  • removing punctuation
  • removing URL links
  • removing digits
  • removing Unicode characters
  • spelling corrections by replacing "เเ" to "แ". The former is two vowels (เ + เ) while the latter is a single vowel (แ). This is one of the most common Thai spelling mistakes that are found on social media.
  • text normalization such as "มากกกกกกกกก" to "มาก", which is roughly “manyyyy” to “many”. Thai people normally emphasize their expression by adding more ending characters at the end of a word when communicating online.

3.4 Word representation

Computers operate in a numeric feature space; thus, they cannot process and understand textual data directly. We need to transform our text messages into a vector representation so we can use numbers (Khamphakdee and Seresangtakul 2021). We are using the Bag of Words (BoW) technique with machine learning models. As part of deep learning models, we use various pretrained word embedding models which includes Thai2Fit, Bert-Base-Thai, and WangchanBERTa to generate word vectors for processing.

3.4.1 Bag of Words (BoW)

BoW is the simplest and the most straightforward method for extracting features by retrieving them from text. The BoW technique starts by creating a dictionary of specific words from a corpus. This feature is then separated by counting the number of occurrences of each word in the corpus, regardless of word order. When a word in a sentence is not present in the predefined dictionary, it gets a score of 0 and a value equal to or greater than 1 depending on how many times it appears in the corpus (Bengfort et al. 2018; Gogula et al. 2023).

3.4.2 Thai2fit

Thai2Fit, formerly Thai2Vec, is ULMFit language modeling, text feature extraction, and text classification in Thai. Thai2Fit is created as part of PyThaiNLP, which is a Thai natural language processing project developed by Thai researchers. Language model training uses the fast.ai version of the AWD LSTM Language Model, with data from the Thai Wikipedia Dump, which was last updated on February 17, 2019. The pretrained language model of Thai2Fit can be used to convert Thai text into vectors, after which these vectors can be used for various downstream NLP tasks such as text classification, clustering, machine translation, and answering questions (pyThaiNLP 2021).

3.4.3 Bert-base-Thai (BERT-TH).

Google's BERT is now a modern way to display multilingually formatted pre-training messages. BERT-TH is a modified BERT-based Thai language-specific pretrained model based on the original BERT, using training data from a Thai Wikipedia article dump on November 2, 2018. The raw texts are extracted using WikiExtractor. The model was trained for 1 million steps on a Tesla K80 GPU. It took about 20 days to complete. The model used in this study was a snapshot of the training at 0.8 million steps (Google's 2018).

3.4.4 WangchanBERTa

For a language with relatively few resources such as Thai, the choice of a language model is limited to BERT-based models trained on small datasets. However, the multilingual model training on a large-scale data does not consider Thai-specific features either (Lowphansirikul et al. 2021). WangchanBERTa was developed to address these limitations, which is currently the largest Thai language model by the VISTEC-depa Thailand Artificial Intelligence Research Institute (VISTEC-depa Thailand Artificial Intelligence Research Institute 2021). The model was trained on the RoBERTa architecture with over 78.5 GB of datasets gathered from various sources including social media posts, news articles, and other datasets in the public domain (Lowphansirikul et al. 2021). It is a language model designed to classify text, classify words in text, and make linguistic inferences specifically in the Thai language. In English, there have long been processing models. However, for Thai, a quite complicated language, it is necessary to create a specific model (VISTEC-depa Thailand Artificial Intelligence Research Institute 2021).

3.5 Model architectures

3.5.1 Logistic regression (LR)

Logistic regression is a frequently used and well-known multivariate statistical method for predicting the probability of an event occurring based on associations between binary dependent variables and multiple independent variables. It can be used to identify the likelihood of an event in natural language processing by identifying the probability of an event occurring. It is also used to predict variables related to text classification using a set of independent variables that output discrete data, such as 0 or 1, for positive/negative or yes/no predictions (Shahzad et al. 2022). It is worth noting that the inverse regularization strength of 2, L2 penalty and liblinear solver were used in this work.

3.5.2 Naive Bayes (NB)

Naive Bayes is one of the fundamental algorithms used for classification. It is a supervised learning technique that uses Bayes' theorem to calculate pre- and post-target probabilities in the data. It is accurate in predicting the true polarity of a particular phrase. Additionally, this model is a low variance classifier that performs well even with small datasets but works better with large and complex data (Singh and Tiwari 2021).

3.5.3 Random forest (RF)

Random Forest is an algorithm that can accurately classify large amounts of data and is suitable for various classifications and regression tasks. It is based on generation of a number of classification trees. The principle is similar to that of a decision tree, but it randomly selects data to create many trees, like a forest with a large number of trees, each with a different random pattern. When predicting results, each tree makes a prediction by averaging the output of the trees and selecting the results with the most votes as the final result. It reduces overloaded datasets and increases prediction accuracy. When the number of trees increases, the accuracy of the results also increases (Agrawal et al. 2023). The Random Forest classifier was trained with 1000 trees in this research.

3.5.4 Bidirectional long short-term memory (BI-LSTM)

Deep learning has gained popularity in sentiment analysis in recent years due to its success in a wide range of application domains that are based on sequential data. Bi-LSTM is a powerful machine learning technique that analyzes multi-level characterization data models to generate state-of-the-art prediction results. It is a method that allows machines to learn by modeling based on the human nervous system. Data are input into the data receiving layer, processed in a hidden layer, and the results are presented in an output layer. Bi-LSTM was developed using two LSTM layers because traditional LSTMs cannot encrypt data backwards. It computes not only the hidden state of the forward sequence but also the hidden state of the backward sequence. With two layers processing bidirectional information, Bi-LSTM is able to model sequential references of messages from both preceding and succeeding contexts, resulting in higher prediction accuracy (Agarwal et al. 2020). In this study, the Bi-LSTM model was trained with 100 epochs using the ReLU activation function, a kernel size of 5, and a learning rate of 1e-3 with no dropout.

3.5.5 Bidirectional encoder representations from transformers (BERT)

BERT has attracted a lot of attention as a mechanism or representation of the bidirectional encoder from transformers, which is a deep learning model developed by Google in 2018 for natural language processing (NLP) (Mridha et al. 2021). It is widely used in natural language processing tasks, such as named entity recognition or sentiment analysis. Bidirectional context analysis capabilities, both forward and backward, handle long-term references in text (Prottasha et al. 2022). This is important for understanding the context of a sentence. BERT is practiced exclusively in English. To achieve higher accuracy, researchers from different countries have developed their own language-specific BERT models. The accuracy of sentiment analysis was compared with that of the mBERT model, which is a multilingual model. It was found that the obtained results were significantly higher than the mBERT values. This shows that monolingual BERT sentiment analysis generates better results (Mridha et al. 2021).

3.6 Performance evaluation

The performance of our chosen model was assessed according to four evaluation parameters (Bruce and Bruce 2020; Qi and Shabrina 2023; Khan et al. 2021).
  • True Positive (TP) is how many actual true values the model predicted as true.
  • True Negative (TN) is how many actual false values the model predicted as false.
  • False Positive (FP) is how many actual false values are predicted as true.
  • False Negative (FN) is how many actual true values are predicted as false.

3.6.1 Accuracy

Accuracy shows the ratio of correct predictions to total predictions. The formula for calculating accuracy is shown in Eq. (1).
$$\mathrm{Accuracy}=\frac{\mathrm{TP}+\mathrm{TN} }{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$$
(1)

3.6.2 Precision

Precision is used to evaluate the exactness of a classifier and is a display of how accurate the model is at making predictions. The formula for calculating precision is shown as Eq. (2).
$$\mathrm{Precision}=\frac{\mathrm{TP} }{\mathrm{TP}+\mathrm{FP}}$$
(2)

3.6.3 Recall

Recall, also called sensitivity, is a measure of a model's ability to make accurate predictions. The formula for calculating recall is shown as Eq. (3).
$$\mathrm{Recall}=\frac{\mathrm{TP} }{\mathrm{TP}+\mathrm{FN}}$$
(3)

3.6.4 F1-score

Precision and recall alone may not be sufficient to evaluate the model. The F1 score is a measure of performance that uses accuracy and recall to calculate a mean. If the value is high, the model is performing well. The formula for calculating the F1 score is shown as Eq. (4).
$$F1 \mathrm{score}=2*\frac{\mathrm{Precision}*\mathrm{Recall} }{\mathrm{Precision}+\mathrm{Recall}}$$
(4)

4 Results and discussion

4.1 YouTube polarity and percentage

Figure 4 presents a pie chart indicating the percentage distribution of the labeled dataset based on polarity. The dataset comprises 10,582 messages, which were classified by three experts who reached a consensus. The data were divided into an 80:20 ratio for training and testing the model, with each class having the closest proportional representation. The training dataset comprised 8465 messages, with 2647 positive messages, 2818 negative messages, and 3000 neutral messages. The testing dataset comprised 20% of the data and consisted of 672 positive messages, 684 negative messages, and 761 neutral messages.

4.2 Experimental setup

All ML and DL models were developed using Python 3.9. The Tensorflow, sklearn and Keras libraries were utilized in training the models for sentiment classification of the dataset. Other supporting libraries like numpy, pandas, matplotlib and wordcloud were used for the investigation of the dataset, visualization of the confusion matrix and generating the word clouds. All experiments in this work were run on NVIDIA Tesla K80, Intel Xeon CPU with 2 vCPUs, 13 GB of RAM, and the Ubuntu 18.04.6 LTS operating system under Google Colab.

4.3 Model performance

Table 2 shows performance evaluation results of six different models, with WangchanBERTa having the best results compared with other models, with all measures of precision, recall, F1-score, and accuracy being 94.59%, 94.71%, 64%, and 94.57%, respectively. This was followed by LR, where the performance of precision, recall, F1 score, and accuracy measures are 90.64%, 90.26%, 90.39%, and 90.27%, respectively. The third Bi-LSTM are 86.54%, 86.84%, 86.65% and 86.54%, respectively. The fourth NB are 86.38%, 86.54%, 86.37%, and 86.25%, respectively. The fifth BERT: BERT-TH are 84.78%, 85.00%, 84.84%, and 84.79%, respectively. The final ranking RF are 84.24%, 83.20%, 83.54%, and 83.28%, respectively. From the above results, it can be seen that the accuracy performance of all models is relatively high. The DL model is BERT: WangchanBERTa. The ML models LR and NB gave the best results, which were more accurate than BERT: BERT-TH was a DL model, and the least accurate model was RF. Subsequently, we compared the accuracy of the top two models in predicting the results of each class and present this data in Table 3. Both models had high accuracy in predicting the positive class, with LR being second to WangchanBERTa in all classes.
Table 2
Comparisons of measurement results for all models
Model
Precision (%)
Recall (%)
F1-score (%)
Accuracy (%)
LR
90.64
90.26
90.39
90.27
NB
86.38
86.54
86.37
86.25
RF
84.24
83.20
83.54
83.28
Bi-LSTM
86.54
86.84
86.65
86.54
BERT: BERT-TH
84.78
85.00
84.84
84.79
BERT: Wangchan BERTa
94.59
94.71
94.64
94.57
The bold value indicates based on the comparison results, the advanced machine learning model proposed in this work yields the highest accuracy values
Table 3
Comparisons of measurement results of a logistic regression model and BERT: WangchanBERTa that considers all three classes
Model
Polarity
Precision (%)
Recall (%)
F1-score (%)
Accuracy (%)
LR
Negative
90.99
85.67
88.25
 
Neutral
85.47
91.20
88.24
90.27
Positive
95.46
93.90
94.67
 
BERT: Wang chan BERTa
Negative
92.54
94.30
93.41
 
Neutral
94.30
91.33
92.79
94.57
Positive
96.93
98.51
97.71
 

4.4 Confusion matrix

Figure 5 presents a confusion matrix that shows the accuracy of the prediction models, which is depicted by a color intensity. The darker the color, the higher the number of accurate predictions for each class of the model. The researchers selected the two most accurate models for comparison, (a) WangchanBERTa with the highest accuracy and (b) LR with the second highest accuracy. When comparing the two models, it is evident that (a) predicted 6 negative messages as positive, while (b) predicted 10 messages, which is the opposite direction for 4 messages, and (b) predicted 98 negative messages as neutral. This is a high number and significantly affects the assessment value. Variation in prediction affects the performance accuracy, where WangchanBERTa model had a higher performance accuracy than LR.
From the performance comparison results in Tables 1 and 2 and confusion matrix, the best models of ML and DL are compared, with LR having an accuracy of 90.27% and BERT: WangchanBERTa achieving an accuracy of 94.57%, showing that DL models outperform the traditional ML models. It is also evident that the use of a Thai-specific language model, which is trained using a large amount of data from various sources, helps increase the efficiency and accuracy of predicting results. After obtaining the results from the best model, as will be discussed in the next section, we attempt to analyze and interpret the results further by visualizing the most frequent words appearing in each sense class using Word Cloud.

4.5 Data analysis and interpretation

Utilization of visual aids is helpful in understanding the nature of public sentiment and the types of extracted data from these sources. Word clouds, in particular, can provide a more in-depth understanding of the data by depicting frequently occurring words in a larger font size within the cloud. Each visualization comprises a group of sentiment-related words for each of the three classes, positive, negative, and neutral. Specifically, Fig. 6a exhibits the positive class, Fig. 6b represents the negative class, and Fig. 6c displays the neutral class.
Most of the words found are related to the issues of high-speed rail. Positive words were found to express ultimate gratitude and happiness. The words that were found to be neutral were about speed and Japan, which can be interpreted as referring to a comparison of the high-speed rail specs used in the project. Most of the negative words found are related to government. The top 10 most frequent words expressing sentiment are shown in Table 4. Therefore, this study serves as a fundamental framework for analyzing emotions that can lead to identifying important and sustainable issues in the long term of the project.
Table 4
Top 10 most frequent words expressing sentiment
Positive
Neutral
Negative
Word
Frequency
Word
Frequency
Word
Frequency
สุดยอด
(Ultimate)
81
ความเร็ว
(Speed)
153
รัฐบาล
(Government)
65
ขอบคุณ
(Thank You)
46
ญี่ปุ่น
(Japan)
43
คนไทย
(Thai people)
64
ดีใจ
(Happy)
38
คนไทย
(Thai People)
35
ดูถูก
(Look Down Upon)
55
คนไทย
(Thai People)
32
สถานี
(Station)
32
นักข่าว
(Journalists)
40
ดีมาก
(Very Good)
31
กรุงเทพ
(Bangkok)
32
ความเร็ว
(Speed)
40
ยินดี
(Pleased)
26
เมือง
(City)
24
สลิ่ม*
(SLIM)
38
ความเร็ว
(Speed)
22
หนองคาย
(Nong Khai)
22
ผู้นำ
(Leaders)
37
เจริญ
(Prosper)
22
รัฐบาล
(Government)
20
สื่อ
(Media)
30
เยี่ยม
(Excellent)
20
ทางรถไฟ
(Railway)
19
ชาติ
(National)
29
ประชาชน
(Citizens)
16
โครงการ
(Project)
19
อิจฉา
(Envious)
29
*The term "สลิ่ม" (SLIM) within the context of contemporary Thai politics is a label used to refer to a specific group of individuals, designated by their respective names, who exhibit a diverse range of characteristics and may carry derogatory implications
The government can monitor and update information to effectively identify public opinions and emotions related to the project or organization. This information can be used for policy planning, improving communication with the public, providing accurate and comprehensive information, saving time and budget on data collection and can also be applied to government organizations, political parties, or individuals to predict public opinions, and trends.
Furthermore, it is recommended to continually analyze public sentiment about the project, as public opinions on social media platforms are subject to constant change. The use of topic modeling algorithms is also suggested to uncover latent topics within groups of related articles to identify discussion topics or urgent issues that require immediate resolution. Furthermore, it involves amalgamating disparate data sources such as weekly data collection, which could facilitate the creation of multivariate time-series datasets for AI algorithm training, specifically for sentiment analysis. Weekly public opinion forecasting would enable identification of trends and patterns of public sentiment and the results could be utilized for comparative analysis purposes.

5 Conclusions

This research presents a methodology for sentiment analysis of Thai language in the context of the Thailand-China high-speed train project and the Laos-China Railway, utilizing algorithms based on ML and DL techniques. The study collected 10,582 comments from the YouTube Data API in the Thai language, which were classified as positive, neutral, or negative sentiments by experts. Six different algorithms, LR, NB, RF, Bi-LSTM, BERT-Base-Thai, and BERT: WangchanBERTa, were employed to classify the sentiment of the comments. The performance of each algorithm was evaluated and compared based on the accuracy of sentiment classification using various feature extraction techniques. The results indicate that the BERT: WangchanBERTa model achieved the highest accuracy, 94.57%, followed by LR with 90.27% accuracy. This indicates that DL performs better in sentiment classification than traditional ML when used in conjunction with an appropriate language model. The development of sentiment analysis models specifically for the Thai language, trained on large amounts of diverse data sources, significantly improves the accuracy of sentiment prediction. Overall, the feelings of each class can be categorized as moderate. This shows that deep learning algorithms can effectively predict the sentiment of textual data. Due to the robustness of the model, this confirms that the proposed approach can be applied to analyze a much larger scale of data for public opinion analysis.
However, this study has some limitations. While data from YouTube are useful in drawing on public opinion for analysis, some people share their opinions on other social media platforms, and there are limits to how much data can be extracted for analysis. Therefore, in future research endeavors, researchers propose incorporation of additional keywords for information retrieval and utilization of social media platforms from other frameworks to increase the diversity and quantity of available data. Even though the best model obtained from this study is quite stable with high accuracy, much effort is still required in data interpretation. If stories cannot be derived from the analysis, the process is incomplete, as strategic planners usually expect actionable results. We plan to alleviate this issue by integrating sentiment classification with topic extraction techniques to analyze, cluster, and find hidden topics by sentiment. This approach is expected to yield clearer results with greater insights. With the emphasis on delivering actionable results, the framework can be applied to other projects of the Thai government.

Acknowledgements

The authors wholeheartedly thank the Center of Multidisciplinary Innovation Network Talent (MINT Center), Faculty of Interdisciplinary Studies, Department of Technology and Engineering, Khon Kaen University, Nong Khai Campus, Thailand, for providing the equipment, tools, and computer software. We thank the National Science and Technology Development Agency, and who supported us and provided the consultant on the research concept. We thank the Baksters Co., Ltd. and Praram Nine Technology Co., Ltd, who supported us with the data and technical insights. We are grateful to our advisors and co-advisor, for their endless guidance. Last but not least, we would like to extend our gratitude to our institutions for the research-friendly environments.

Declarations

Conflict of interest

The authors declare no competing interests.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://​creativecommons.​org/​licenses/​by/​4.​0/​.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Literatur
Zurück zum Zitat Agarwal B, Nayak R, Mittal N, Patnaik S (eds) (2020) Deep learning-based approaches for sentiment analysis (p. 4). Springer, Singapore Agarwal B, Nayak R, Mittal N, Patnaik S (eds) (2020) Deep learning-based approaches for sentiment analysis (p. 4). Springer, Singapore
Zurück zum Zitat Bengfort B, Bilbro R, Ojeda T (2018) Applied text analysis with python: enabling language-aware data products with machine learning. O’Reilly Media Inc, California Bengfort B, Bilbro R, Ojeda T (2018) Applied text analysis with python: enabling language-aware data products with machine learning. O’Reilly Media Inc, California
Zurück zum Zitat Bruce P, Bruce A (2020) Practical statistics for data scientists. O’Reilly Media Inc, California Bruce P, Bruce A (2020) Practical statistics for data scientists. O’Reilly Media Inc, California
Zurück zum Zitat Chirathivat S, Rutchatorn B, Devendrakumar A (eds) (2022) China's belt and road initiative in ASEAN: growing presence, recent progress and future challenges Chirathivat S, Rutchatorn B, Devendrakumar A (eds) (2022) China's belt and road initiative in ASEAN: growing presence, recent progress and future challenges
Zurück zum Zitat Kewsuwun N, Kajornkasirat S (2022) A sentiment analysis model of agritech startup on facebook comments using naive Bayes classifier. Int J Electr Comput Eng 12(3):2829 Kewsuwun N, Kajornkasirat S (2022) A sentiment analysis model of agritech startup on facebook comments using naive Bayes classifier. Int J Electr Comput Eng 12(3):2829
Zurück zum Zitat Lowphansirikul L, Polpanumas C, Jantrakulchai N, Nutanong S (2021) WangchanBERTa: pretraining transformer-based Thai language models. ArXiv, abs/2101.09635 Lowphansirikul L, Polpanumas C, Jantrakulchai N, Nutanong S (2021) WangchanBERTa: pretraining transformer-based Thai language models. ArXiv, abs/2101.09635
Zurück zum Zitat Mehta T, Deshmukh G (2022) YouTube ad view sentiment analysis using deep learning and machine learning. Int J Comput Appl 184(11):10–14 Mehta T, Deshmukh G (2022) YouTube ad view sentiment analysis using deep learning and machine learning. Int J Comput Appl 184(11):10–14
Zurück zum Zitat Rowedder S (2020) Railroading land-linked Laos: China’s regional profits, Laos’ domestic costs? Eurasian Geogr Econ 61(2):152–161CrossRef Rowedder S (2020) Railroading land-linked Laos: China’s regional profits, Laos’ domestic costs? Eurasian Geogr Econ 61(2):152–161CrossRef
Zurück zum Zitat Singh R, Tiwari A (2021) Youtube comments sentiment analysis. Int J Sci Res Eng Manag 5(5):1–11 Singh R, Tiwari A (2021) Youtube comments sentiment analysis. Int J Sci Res Eng Manag 5(5):1–11
Metadaten
Titel
Analyzing online public opinion on Thailand-China high-speed train and Laos-China railway mega-projects using advanced machine learning for sentiment analysis
verfasst von
Manussawee Nokkaew
Kwankamol Nongpong
Tapanan Yeophantong
Pattravadee Ploykitikoon
Weerachai Arjharn
Apirat Siritaratiwat
Sorawit Narkglom
Wullapa Wongsinlatam
Tawun Remsungnen
Ariya Namvong
Chayada Surawanitkun
Publikationsdatum
01.12.2024
Verlag
Springer Vienna
Erschienen in
Social Network Analysis and Mining / Ausgabe 1/2024
Print ISSN: 1869-5450
Elektronische ISSN: 1869-5469
DOI
https://doi.org/10.1007/s13278-023-01168-8

Weitere Artikel der Ausgabe 1/2024

Social Network Analysis and Mining 1/2024 Zur Ausgabe

Premium Partner