-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Stream based blob API #9
Comments
I think the problem is that the sha1 checksum needs to be calculated in advance. |
I had a talk with Jodok yesterday & it is indeed the case as far as I understood. think it would still be possible to implement it using a file-system step in-between. function createWriteStream() {
var shasum = crypto.createHash('sha1')
var input = new stream.Duplex();
var fsStream = fs.createWriteStream('tmpfile')
input.on('data', function (data) {
shasum.update(data);
});
input.pipe(fsStream);
fsStream.on('end', function () {
fs.createReadStream('tmpfile').pipe(createHttpBlobRequest(shasum.digest('hex')))
})
return input;
} This would still be better than trying to implement it by the user themselves. |
+0.5 it solves the problem to upload files larger than heap limit and it could reduce memory ussage.
Well, we could start with a simple version - but I would recommend to Crate to accept streams without sha hash in the URL (or allow simply user defined ID's). Crate could calculate the hash during reception to return it in the http response. The client driver could calculate the sha hash during upload to verify it with the server value. Is there an issue open @crate to support streaming of blobs (without sha upfront)? |
testing the graphql |
The current blob API is not stream based. It would be good if it could be similar to
fs.createWriteStream
orfs.createReadStream
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
The text was updated successfully, but these errors were encountered: