Image Captioning Using Deep Learning

Abstract

In Artificial Intelligence, Caption generation is a challenging task where a textual description must be generated for a given image. It requires both methods from computer vision to understand the objects and actions involved in the image and a natural language processing model to generate a caption. Image captioning has various applications such as usage in virtual assistants, recommendations in editing applications, for image indexing, for social media, for visually impaired persons, and many other natural language processing applications. We propose a hybrid system using multilayer Convolutional Neural Network (CNN) to generate vocabulary describing the images and a Long Short Term Memory (LSTM) to accurately structure meaningful sentences using the generated keywords. The CNN compares the given image to a large dataset of training images, then generates an accurate description using the trained captions.