Chatbot Model Development Using BERT for West Sumatera Halal Tourism Information

Abstract

Halal tourism in Indonesia is growing rapidly, highlighting the need for Muslim halal tourism information that gives unique and relevant information for traveller. However, providing timely and reliable information, specifically related to halal tourism remains a challenge. This research aims to address this by developing a chatbot model using BERT for West Sumatra’s halal tourism. A total of 1,125 questions were prepared, divided into nine categories or labels with 125 questions each. Eighty percent (900 questions) was used to fine-tune the BERT-base-multilingual-uncased model, while 20% (225 questions) was used for evaluation. The model was fine-tuned using BertForSequenceClassification for three epochs with a batch size of 32. The chatbot demonstrated high performance, with an overall accuracy of 0.96. However, the lowest precision value was 0.89 for “budaya” (or culture) and “kuliner” (or culinary) labels, and the lowest recall value was 0.64 for the “belanja” (or shopping) label, yielding an F1-score of 0.78. This study describes chatbot model development, from data collection and pre-processing to experimental setup and model training using a fine-tuned BERT-base-multilingual-uncased model. The chatbot model can group user queries into specific purposes and respond to a predefined list. However, one label (e.g “belanja” or shopping) may have the lowest recall due to a poor training dataset and query variation.