forked from bazelbuild/remote-apis
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add BLAKE3CONCAT: a version of BLAKE3 that supports chunking
In PR bazelbuild#233 I proposed the addition of two new ContentAddressableStorage methods (ConcatenateBlobs and SplitBlobs) that allow one to gain random access it large CAS objects, while still providing a way to very data integrity. As part of that change, I added a new digest function to help with that, named BLAKE3CONCAT. This PR adds just this digest function, without bringing in any support for chunking. This will be done separately, as it was requested that both these features landed independently. I have also included test vectors for the BLAKE3CONCAT digest function. I have derived these by modifying the BLAKE3 reference implementation written in Rust, and rerunning the tool that emits the official test vectors: https://github.com/BLAKE3-team/BLAKE3/blob/master/test_vectors/test_vectors.json Furthermore, I have been able to validate the newly obtained test vectors using a custom BLAKE3CONCAT implementation that I have written in Go, which will become part of Buildbarn.
- Loading branch information
1 parent
3a21dee
commit 55dd91f
Showing
3 changed files
with
453 additions
and
252 deletions.
There are no files selected for viewing
145 changes: 145 additions & 0 deletions
145
build/bazel/remote/execution/v2/blake3concat_test_vectors.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,145 @@ | ||
{ | ||
"_comment": "Each test is an input length and one hash output. The input in each case is filled with a repeating sequence of 251 bytes: 0, 1, 2, ..., 249, 250, 0, 1, ..., and so on.", | ||
"cases": [ | ||
{ | ||
"input_len": 0, | ||
"hash": "b4686b3c8d953f4d185d51bc9cd7bce6d9782d7659f8c06068d4c3ffa6e56841" | ||
}, | ||
{ | ||
"input_len": 1, | ||
"hash": "0df1a7881d71d287d0afc2fcd7d23d28f12e401ab858ca267f11c5f1710df315" | ||
}, | ||
{ | ||
"input_len": 2, | ||
"hash": "b2530fd8d3fa7ca531d3e03d3b44f434354af89b59c14123d1d1ecc664b243a6" | ||
}, | ||
{ | ||
"input_len": 3, | ||
"hash": "8e2451ec9f2ae6282e10a52c09d3bf7856236cfa990b5f1aaf0b3943d5b72d18" | ||
}, | ||
{ | ||
"input_len": 4, | ||
"hash": "826e2530317a718add9a012ad6cf1c2cd298c6f7fa09022dd7f1847cfd03ddb8" | ||
}, | ||
{ | ||
"input_len": 5, | ||
"hash": "c6e98dc8c0e86be0a25d8e4a4ac951f5bf4efa111d8a332843aabe723fab3452" | ||
}, | ||
{ | ||
"input_len": 6, | ||
"hash": "9532a581f853cd63594625ddd0bdcac2edaf9cf436b0b42af77938ed5cfc5181" | ||
}, | ||
{ | ||
"input_len": 7, | ||
"hash": "1d2cfb6af10502c1b7067f6e4515dac354cb4f6c9def44704e9a79e32f41688b" | ||
}, | ||
{ | ||
"input_len": 8, | ||
"hash": "ffa72fdba626d7aa0d1ad38bb7f289c91386ae4d2929fa093ccc5d188433e906" | ||
}, | ||
{ | ||
"input_len": 63, | ||
"hash": "eefdb8f62d0cb234d94b16a2837eb72665018861396a89ef9f28d1fb190fca24" | ||
}, | ||
{ | ||
"input_len": 64, | ||
"hash": "6dde41c9d05a39b09b486620f2c3cf7652fde7f3eb412353d95734294c5d348e" | ||
}, | ||
{ | ||
"input_len": 65, | ||
"hash": "5bf4b8a5dba62b01ebd11264f3a724638e3f0ef274f253dfa41c0a657db3ab3e" | ||
}, | ||
{ | ||
"input_len": 127, | ||
"hash": "cdfdc645bd59bb2415df258b0d97a1f07e68710f67e6e61e78aa15f4703d53a2" | ||
}, | ||
{ | ||
"input_len": 128, | ||
"hash": "d41a4a8d07d439dc3892f45a296b934f2fbbd966f29f8640e88f15dde20015c7" | ||
}, | ||
{ | ||
"input_len": 129, | ||
"hash": "7a0235d0b622ed10c03a5c9f7fcfb36b2fc8aefb50e3a95ded8add3e06126214" | ||
}, | ||
{ | ||
"input_len": 1023, | ||
"hash": "ad61f1f5c9f2a65c7b96152b47321c7d4320e2c2719e14bee6640815e4182ddb" | ||
}, | ||
{ | ||
"input_len": 1024, | ||
"hash": "5c9e654411e393d1f4bec710ccd5bc5669ab177d610a0eb691fcfee92fb4e8b1" | ||
}, | ||
{ | ||
"input_len": 1025, | ||
"hash": "ee286548954cbdc4bf9b5fef0555f1b7d813edbe2bc7f3693126d5825999139d" | ||
}, | ||
{ | ||
"input_len": 2048, | ||
"hash": "80c9b515da8b7fcabfefcc4b049b7cbcf16983c2cd1448e7717168f279b4d7ad" | ||
}, | ||
{ | ||
"input_len": 2049, | ||
"hash": "051bad3acf63a2d27429524232b86b8230234e0091f38e0686faf79df77e1b30" | ||
}, | ||
{ | ||
"input_len": 3072, | ||
"hash": "cd28dcf373e40cc36aa4a7349687d3805babb92229c0cf28d903ee2d08e1194c" | ||
}, | ||
{ | ||
"input_len": 3073, | ||
"hash": "fe07c25f6c1bc98d104ed1ca7971177e8515b28f6287e35569098b448f31f39a" | ||
}, | ||
{ | ||
"input_len": 4096, | ||
"hash": "65e4ec2f94073d6eb00a9bf5c085cde958ec73ed3baaf2b80d5e6f90dc868e94" | ||
}, | ||
{ | ||
"input_len": 4097, | ||
"hash": "7f59a96c5b78c65db13b981c327ebc170bcdd155123857dfb8720349b0ae7699" | ||
}, | ||
{ | ||
"input_len": 5120, | ||
"hash": "88cf814865f6b6f1488e3c8dd14ebab858659459e8c1552a26bcfd81f75465c7" | ||
}, | ||
{ | ||
"input_len": 5121, | ||
"hash": "01f181c74b1a8284101fa35c8082734f167fcc7f6ac75c2547ecfb0c765a1073" | ||
}, | ||
{ | ||
"input_len": 6144, | ||
"hash": "b739390236994f666fd8c0a6d57e3a6192034a43ca990906f2abbad1509c719f" | ||
}, | ||
{ | ||
"input_len": 6145, | ||
"hash": "a0e15c6796ed7da911d75d545ee14c960ff1e9dd37ecfbd9101c25f9e4d073f9" | ||
}, | ||
{ | ||
"input_len": 7168, | ||
"hash": "b55ab1c0b44dab6c0e25839c33198f3f76a688f31ec4a8d109f190befb3634f2" | ||
}, | ||
{ | ||
"input_len": 7169, | ||
"hash": "23cb0c89436728d339dae6771e9078109df0fdbdee617d6e49d615c5d5212ef2" | ||
}, | ||
{ | ||
"input_len": 8192, | ||
"hash": "fda9453f03d5f5080d3c41126fddc294c0d8f48aa3fdbcdd63f9cda4d33fc211" | ||
}, | ||
{ | ||
"input_len": 8193, | ||
"hash": "5da258b0a71add844c953fd1b3c74a99f3af9011baeaedae7962f4dde67e5cf9" | ||
}, | ||
{ | ||
"input_len": 16384, | ||
"hash": "222b977bbd1d5dc6529f424379ac4e987f927a10411f52ffce160e7e1f4c06b8" | ||
}, | ||
{ | ||
"input_len": 31744, | ||
"hash": "f1ae00b44cb183227d4296ad578af3c2b27132dcf3ee94e7731acc198fdeb114" | ||
}, | ||
{ | ||
"input_len": 102400, | ||
"hash": "9dd1fb18c727e9bfc58c9ff5487ac5b08e6b1f32d65348a68dc4131e967cd2ce" | ||
} | ||
] | ||
} |
Oops, something went wrong.