Database toolkit for aliyundrive share links
Currently, many people share huge pack of files via 阿里云盘. For example, one can find such share links in UP云搜, 万人迷吧, 找资源 and 云盘资源网. Given those abundant resources, we create Alibrary project to index and update shared files. This project SHOULD reach the following goals:
- store index data reliably;
- provide rapid search API;
- update and validate share links efficiently.
We use the following projects:
- aligo,
python3
API of aliyundrive, - PostgreSQL with full-text search extension pg_jieba, database storage with Chinese segemntation,
- psycopg2,
pthon3
API forPostgreSQL
, - python-prompt-toolkit, search prompt interface,
- humanize, show file size,
- aria2, optional download utility.
The following host demo sites are mirrors to each other:
Their datasets are the same, alibrary.sql
in v2.0.
To begin with, you can download alibrary.sql
in the release page as a sample dataset and load it into PostgresSQL
.
To load it, you must have the pg_jieba
extension installed.
This sample includes 552,544 records without duplications.
After this, you can search indexes in our sample database or index your costume share links using aliyun-share, see comments inside it for usage details. As for search syntax, please refer to the tsquery.
- Migrate database to
redis
- Implement search API
- Add cli prompt interface
- Improve prompt interface
- Run as cloud service
- User-friendly front end for public usage
- Migrate database to
PostgresSQL
- Add more resource categories
- Add systemd integration
- Index share folders