International Journal of Advanced Innovative Technology in Engineering (IJAITE)

---

Machine Learning Approach to Generate Artistic Captions by Using Neural Network for Visually Impaired People

Manali Modi, Aafrin Deshmukh, Vidya Karmargulwar, Sakshi Deshmukh, Sagar Padiya

Abstract :

Captioning an image involves generating a human readable textual description given an image, such as a photograph. It is an easy task for humans, but very challenging for a machine as it involves both understanding the content of an image and how to translate this understanding into natural language. This project aims to identify the purpose behind a visual depiction of an image captured, analyze the context behind a visual image and generate an artistic caption for the same. The resulting caption will not necessarily be descriptive but rather contextual and creative. There doesnt exist a direct mapping between image and its corresponding description generated but an abstract mapping that denotes the image into sentences which is very much artistic and aiming to exhibit a kind of computational creativity. A user interface is built for users to upload images or paste a remote image URL or image can be captured directly through a real time camera. Further this text can be converted to speech which will help visually impaired people to depict contents in an image which is otherwise impossible for them without any assistance. A neural network is trained using a dataset so that this pre-trained model can be used to predict the context of an image in a text format which is then converted to speech.

Keywords :

Full Text :

Download PDF

DOI :

Cite this paper :

References :

---