Built-in Compression

[matheus23]

Summary

Implementation of compression built-in to WNFS.

Problem / Need / Opportunity

Compressing private data after encryption is ineffective, so this puts the burden of dealing with compression on users instead of services that store user WNFS data.
There is an opportunity to support compression on the protocol level, similar to ZFS compression.

Success Scenario

Implementations of WNFS can decide to transparently compress user data without them needing to know, leading to faster writes, faster reads and less memory and bandwidth usage.

Notes

To be considered is the security impact of compression. E.g. if an attacker effectively has an encryption oracle for the first half of a file and the data is compressed before it's encrypted, it may be possible for an attacker to "guess" words from the second half of the file and thus populate a compression dictionary and observe the impact that has on the ciphertext length.

Banyan.Computer is interested in this, please feel free to reach out if you are too.

ETA: 2024-06-30