Memegraph is a search engine and recommendation system for memes using OpenAI CLIP.
Search memes via keywords, explore memes with the same template, and discover memes that are visually similar.
Important
This app uses MLX to run CLIP. To generate embeddings, a device with an Apple Silicon processor is needed. For non-Apple Silicon devices, the app can still run, provided there are already images and embeddings in the database, but it will not be able to search for memes by image similarity.
Install dependencies
pip install -r requirements.txt
Download OpenAI CLIP model (clip-vit-base-patch32) from Hugging Face and convert to Apple's MLX
python clip/convert.py
The model is dowloaded to clip/mlx_model/
directory unless argument --model_dir
is provided.
Modify main.py
to point to the correct model. This contains the functions to generate the embeddings.
Modify script.py
to point to your images directory, blob storage, and Postgres database.
Generate image embeddings, store images in an blob storage bucket, and store embeddings in a database
python script.py
Run the app
python app.py
All memes were retrieved from the top 1000 posts of all time on Reddit's meme subreddits using PRAW on the Reddit's API free tier.
Python is not my primary language and I'm new to machine learning, so I greatly appreciate and welcome constructive feedback!