CONSTRUCTING A HEALTHCARE CHATBOT USING BART WITH DEEP ATTENTION

Abstract

This thesis provides a method to construct a virtual assistant related to healthcare problems to answer the frequent questions that users usually encounter daily. The BART model, a variant of the Transformer-based model, is used not only to train and construct the chatbot but also to train the translation and summarization tasks to help improve the responses from the virtual assistant further. The virtual assistant consists of many models, each taking one specific task such as generating responses in English or Vietnamese or summarizing the answers generated from the other model. Each model is also trained on different datasets, the core chatbots including two BART models are trained on the EhealthMini, EhealthChat, and EHealthVnChat datasets that we collected from the internet and Kaggle. The other models are trained on the samsum and mt-engvi datasets for summary and translation tasks respectively. We also investigate the reattention used in the vision Transformer-based model and apply it to the BART models. After training the models, the evaluation will take place by using a small part of the datasets (not included in the train datasets), the core model achieves the results of BLEU1: 0.25; BLEU-2: 0,2; BLEU-3: 0,18; and BLEU-4: 0,16 with the BART model and BLEU-1: 0.31; BLEU-2: 0,25; BLEU-3: 0,23; and BLEU-4: 0,21 with the re-attention BART model.