This pipeline is used to extract the Italian Covid-19 stats from the official Italian Civil Protection website in order to analyze it and predict the future trend of the virus.
You can run this project in two modes:
- Simulation: data will load from beginning of dataset to end.
- Production: data will load every Saturday at midnight.
You can choose the mode by comment / uncomment Logstash and Python Server images in docker-compose.yml.
Make sure you select all production / non-production images.
All process is Dockerized, so you need to have Docker installed on your machine.
After it, you can run the following command to build the images and run the pipeline:
docker compose upNow, you have to load Kibana Dashboard, using the following command:
cd kibana_loader
./entrypoint.sh dashboard.ndjsonThen, just simply open Kibana and wait for data to appear