Tangrowth

Python3.6 Async Crawler Example with aiohttp and asyncio

Installation

Installation Python 3.6

sudo add-apt-repository ppa:fkrull/deadsnakes
sudo apt-get update
sudo apt-get install python3.6

In virtualenvwrapper

mkvirtualenv async_crawler --python=/usr/bin/python3.6
python async_crawler.py

or

Replace python3 with python3.6

sudo mv /usr/bin/python3 /usr/bin/python3-backup
sudo ln -s /usr/bin/python3.6 /usr/bin/python3

sudo apt-get install python3-pip
sudo pip3 install aiohttp
python3 async_crawler.py