-
Notifications
You must be signed in to change notification settings - Fork 5
Deltametadata of repo md files
Jan Šilhan edited this page Mar 24, 2016
·
9 revisions
Goal of this project is to enable delta metadata downloads for DNF/PackageKit users. I.e. to not download whole new metadata files with each change. Zsync tool was identified as a stable tool used in production (Ubuntu images). Metadata files on server will contain generated checksums of the files (.zsync files). Client will download just differences of metadata he has on the system (no matter how old they are) and the up-to-data files on the server. This change would be forward compatible for current Fedora users. Zsync could be used on any file so it could theoretically generate .zsync files for .solv files from libsolv.
- get the server (free hosting, publicly available)
-
setup cron job
- to automatically sync changes from Fedora-updates repository
- store expired files in archive for later research purposes
-
create passive DNF plugin that will
- run zsync from client on metadata from our server
- disable normal fetching of metadata in DNF
- figure out how much bandwidth will be saved on client with day, 2 days, ..., week, ... 2 weeks old metadata
- fork zsync-curl version
- apply patches from other forks of no longer maintained zsync forks on github
- separate zsync to library and client interface
- design zsync library API so it could be used from librepo and createrepo_c
- get rid of zlib bundling (optional, we can bundle it or include version from rsync)
- it should be supported on Fedora infrastructure
- createrepo_c should be used for generating of metadata
- changes in createrepo_c
- use by default gzip
--rsyncable
conversion - it should use zsync library to generate the .zsync files when
--zsync
option is set
- use by default gzip
- changes in librepo
- it should use zsync library API to set curl request
- add into repomd.xml tag with zsync files with checksums and filesize