An Automated Image Caption Generation and Caption Based Image Retrieval Application.
- Introduction
- Features
- Demo Video
- Dataset and Pre-trained models
- Packages Required
- Future Scope
- Publication
- Contributors
- Contribute
- Acknowledgement
This is a python (Flask Application) based Automated Image Caption and Image Retrieval model which makes use of deep learning image caption generator. It uses a merge model comprising of Convolutional Neural Network (CNN) and a Long Short Term Memory Network (LSTM) . The dataset used here is Flickr8K dataset.
This model can be used via GUI. In model-
- Automated Caption Generation (Offline) - Upload Image and retrive automated caption based on image features.
- Caption Based Image Search (Similar Images) - Given Text Based Query and it will return similar images based on image caption and similarity.
- Anaconda
- Keras with Tensorflow Backend (Python 3.6)
- Flask
- Make a highly scalable REST API which accepts the image and returns the caption of the image
- Make a dashboard through which the training of the captioner could be done on custom datasets.
- Introduce unit tests and logging to enable smooth debugging.
- Improve the caption based image search part for the more accuracy.
- Make a dashboard through which user can manage their image database.
- Improve the UI part of the application.
- Change the architecture of image captioner in order reduce the memory footprint required by the current pre trained models
- Further development may also include working on improvising with more accurate predictions and search results
- Fork this repository and contribute.
- Feel free to report bugs.
- All types of feedbacks are welcome
- A special thanks to Machine Learning Mastery without which we couldn't have thought about the right approach to tackle this problem.