Asahan, September 2026 — Yakub Mas Arbi Sitompul and Marah Dolly Nst from the Information Systems Study Program, Faculty of Computer Science and Information Technology, Universitas Muhammadiyah Sumatera Utara, have developed a web-based sentiment analysis system to process customer reviews of Portos Coffee Shop. Using the Multinomial Naïve Bayes algorithm and a Natural Language Processing (NLP) approach, the system achieved 87.29% accuracy during testing, while cross-validation produced an average accuracy of 90.79%.
An analysis of 586 reviews collected from Google Maps and Instagram found that most customers expressed positive opinions about the coffee shop. These findings provide insights into customer experiences while identifying aspects of service that the business could improve.
Understanding Customer Feedback Through Digital Reviews
Portos Coffee Shop is a local culinary business located in Aek Songsongan District, Asahan Regency, North Sumatra, Indonesia. As digital platforms become increasingly important, customers are sharing their experiences through Google Maps reviews and comments on Instagram.
The two platforms have different characteristics. Google Maps typically contains more structured reviews, often accompanied by star ratings, while Instagram features shorter, more informal comments that frequently include abbreviations, slang, and everyday expressions.
The growing volume of online feedback makes manual evaluation time-consuming and labor-intensive. Manual assessments may also be influenced by subjective judgments. To address these challenges, Sitompul and Nst applied natural language processing technology to transform customer reviews into information that could be analyzed automatically.
Processing 586 Reviews Using Computational Methods
The study employed a quantitative experimental approach and the Waterfall system development model. Data were collected in 2026 from Google Maps and Portos Coffee Shop’s official Instagram account, @portos_idn.
Of the initial 638 reviews, the researchers obtained 586 clean records after removing empty entries, comments consisting solely of emojis, automated promotional comments, and duplicate data. A total of 525 reviews came from Google Maps, while the remaining 61 were collected from Instagram.
Before analysis, the text underwent several preprocessing stages, including text cleaning, conversion to lowercase, tokenization, normalization of non-standard language, stopword removal, and stemming using the Sastrawi library. These steps helped reduce word variations and simplify the text for computational analysis.
The researchers then applied Term Frequency–Inverse Document Frequency (TF-IDF) to assign weights to words according to their importance within the collection of reviews. The Multinomial Naïve Bayes algorithm subsequently classified the reviews into two categories: positive and negative sentiment.
Data labeling combined star ratings with an Indonesian sentiment lexicon. Manual validation was also conducted to reduce errors involving ambiguous or sarcastic statements.
Most Customers Expressed Positive Sentiment
Among the 586 reviews analyzed, 364, or 62.12%, were classified as positive. Meanwhile, 222 reviews, representing 37.88%, expressed negative sentiment.
Positive reviews frequently concerned the taste of food and beverages, the comfort of the premises, and friendly service. Words such as “delicious,” “comfortable,” and “friendly” were among the prominent terms associated with positive sentiment.
In contrast, negative reviews often concerned slow service, prices perceived as expensive, high room temperatures, and customer disappointment. Terms corresponding to “slow,” “expensive,” “hot,” and “disappointed” emerged as important features in the negative sentiment category.
These findings suggest that customer experiences are influenced not only by product quality but also by service speed and the comfort of the facilities.
Text Preprocessing Improves Model Accuracy
In testing that used an 80% training-data and 20% testing-data split, the Multinomial Naïve Bayes model achieved 87.29% accuracy, 86.25% precision, 94.52% recall, and an F1-score of 90.20%.
To evaluate the model’s stability across different data samples, the researchers also conducted five-fold cross-validation. This procedure produced an average accuracy of 90.79%.
The study further examined how text preprocessing affected model performance. Without preprocessing, the model achieved only 71.18% accuracy. After the complete NLP pipeline was applied, accuracy increased to 87.29%, representing an improvement of 16.11 percentage points.
These results demonstrate that text cleaning, normalization of non-standard words, stopword removal, and stemming play important roles in improving the model’s ability to classify customer reviews.
A Monitoring System Helps Management Set Priorities
In addition to developing the classification model, the researchers built a web-based sentiment monitoring system. The system uses FastAPI for the backend, Next.js for the user interface, PostgreSQL as the database, and Docker to support application deployment.
Testing across 14 functional scenarios, including sentiment prediction and data management, achieved a 100% success rate. Unit testing of the backend modules reached 91.04% code coverage, while the average processing time for a single review was 0.18 seconds.
The system allows management to monitor customer feedback more systematically. Based on the findings, the researchers recommend that Portos Coffee Shop prioritize improvements to its air-conditioning facilities and increase service efficiency, particularly during peak hours.
However, sentiment analysis should be treated as a decision-support tool rather than a perfect interpretation of customer opinions. Sarcasm, ambiguous context, and unconventional language may still lead to classification errors. Future research could explore deep learning models such as IndoBERT and Aspect-Based Sentiment Analysis (ABSA) to provide more detailed insights into different aspects of the customer experience.
About the Authors
- Yakub Mas Arbi Sitompul - Universitas Muhammadiyah Sumatera Utara, Medan, Indonesia.
- Marah Dolly Nst - Universitas Muhammadiyah Sumatera Utara, Medan, Indonesia.
Article Source
Journal: International Journal of Educational Technology Research (IJETR), Vol. 4, No. 3, September 2026, pp. 227–236
DOI: https://doi.org/10.59890/ijetr.v4i3.15
Journal URL: https://journalijetr.my.id/index.php/ijetr
0 Komentar