IMAGE CAPTIONING USING RESNET AND BI-TRANSFORMER MODELS

Abstract

Image captioning, a topic in the field of machine learning, uses image recognition techniques and natural language processing models to generate captions of photos. In this thesis, we perform experiments with several models to automatically create descriptions for images using the ResNet and Transformer models. In particular, the ResNet-50, ResNet-101, and ResNet-152 models, are used to extract image features, which are later used to feed into the Transform or BiTransformer models to generate image captions. We perform experiments on the Flickr8k dataset in English and Vietnamese, and evaluated it using the BLEU metric. The experimental results show that the combination of ResNet-152 and BiTransformer helps achieve better BLEU scores than another one.