HARMONIZING LEXICAL AND SEMANTIC ASPECTS IN ABSTRACTIVE TEXT SUMMARIZATION USING BART-BRIO

Abstract

Abstractive text summarization is of interest to many researchers. An extremely important thing in the summary is the use of words and ensuring semantic accuracy. Don't simply extract important pieces of text, but also create new sentences or paragraphs, often using natural language. This requires the model to be able to understand the context, grammar, and meaning of the text to be able to produce quality summaries. This thesis proposes a text summarization model (English and Vietnamese) based on BART using BRIO training technique, called BART-BRIO. BRIO assumes a non-deterministic probability distribution to reduce the model's dependence on the reference summary and improve model performance during inference. We tested it on the Vietnamese-News-Data dataset. The results show that our model outperforms all existing Vietnamese summary models. Specifically, the ROUGE-1, ROUGE-2 and ROUGE-LSum scores of the BART-BRIO model are 60.20, 32.25 and 43.12, respectively. For English (using the CNNDM dataset), the scores were 45.75, 22.36, and 45.46, respectively.