VIETNAMESE ABSTRACTIVE SUMMARIZATION USING TRANSFORMER AND BERT

Abstract

Nowadays, with the strong development of information technology, people are turning to technology to capture information. Therefore, text summarization is becoming more and more essential. As a result of the evolution of machine learning, many Natural Language Processing (NLP) models were introduced. This thesis will apply BERT-Transformer – one of state at the art techniques, and a pre-trained model for implementing a new machine learning model which will receive long sentences as input and take it into shorter ones with the same meaning. To train the model, we used Open-NMT, and the raw material for training the model came from the website vnexpress.net and a few online Vietnamese periodicals. The ROUGE method will be used in this thesis to evaluate the summaries. The results of the model after being evaluated are considered good and acceptable compared to other text summarization models. The result of the thesis introduces an abstractive Vietnamese summarization model. The model enables users to generate a summary that retains the original meaning of the source text at the sentence level.