IMAGE CAPTIONING USING EFFICIENTNETB2 AND DEEP TRANSFORMER

Abstract

Image captioning is a fascinating field that converges computer vision and natural language processing (NLP) within the vast field of artificial intelligence. The fundamental objective is to automatically generate natural descriptive sentences that express the content of an image. Various image captioning models often using integrating convolutional neural networks (CNNs) for image feature extraction and recurrent neural networks (RNNs) or transformers for generating coherent sentences. In this thesis, we employ a combination of EfficientNetB2 and Deep Transformer to generate a natural descriptive sentence from image. We perform experiments on both the English and Vietnamese Flickr8k dataset, then assessing the performance of this through the BLEU metric. The experimental results show that a combination of EfficientNetB2 and Deep Transformer effectively generates captions for images in Vietnamese. The BLEU-1,2,3, and 4 scores of the models on the English and Vietnamese Flickr8k datasets are 0.510, 0.211, 0.083, 0.030; and 0.532, 0.282, 0.155, 0.075, respectively.