COOKING RECIPE GENERATION FROM FOOD IMAGES USING CONVNEXT V2 AND LLAMA2.

Abstract

Foods play an essential part in our lives. Our everyday experience withfoodsis not just about eating them; behind each meal lies a story encapsulatedinacomplex recipe. With the rise of smartphone usage, people increasinglyenjoyphotographing food. However, images alone do not reveal the preparationprocess.Therefore, in this thesis, we introduce an inverse cooking systemthat recreatescooking recipes given food images. Our system predicts the Vietnamese foodusinga novel architecture that captures inter-ingredient relationships without requiringaspecific arrangement. Then, it generates cooking instructions by jointlyanalyzingboth the image and the identified ingredients. Leveraging the 30VNFoods dataset,comprising 25,000 Vietnamese food images, for ingredient identificationandthecookpad-com dataset for recipe generation, our ConvNeXt V2 model achievesanimpressive accuracy of 87% for ingredient identification. For recipe generation, theLLaMA 2 model attained a BLEU score of 0.41, demonstrating its effectivenessingenerating relevant and accurate recipes. This study, therefore, contributesmeaningfully to the field of machine learning, particularly in advancingtechniquesfor recipe generation from visual food data.