ABSTRACTIVE TEXT SUMMARIZATION USING SEASON WITH DEEP ATTENTIN

Abstract

In the ever-expanding landscape of natural language processing, the demand for effective automatic summarization techniques has intensified, driven by the vast volumes of textual information available. This thesis delves into the realm of abstract summarization, aiming to improve the state-of-the-art through the integration of deep attention mechanisms. The chosen model for investigation is SEASON, renowned for its capabilities in abstractive text summarization tasks. The proposed methodology leverages the inherent strengths of deep attention mechanisms to enhance the comprehension of context and improve the generation of concise and coherent abstracts. The research focuses on the intricate interplay between attention mechanisms and the SEASON model's architecture, exploring how this synergy can be harnessed to capture salient information and produce good abstractions. The experimental phase involves training the model on CNN/Daily Mail dataset and evaluate with ROUGE metrics. With the experiment result, in the first place the reattention modification on SEASON has show a pretty low ROUGE score compare to SEASON with ROUGE-1 38.91, ROUGE-2 16.00, ROUGE-L 25.82, ROUGELSUM 35.52, but the model has some improvement after practicing re-attention modification methods with a random frequent for the re-attention to be happen, the result has some slightly increasement with ROUGE-1 43.93, ROUGE-2 20.85, ROUGE-L 30.23, ROUGE-LSum 40.43.