Add a flake check for testing the stdlib #2061
Conversation
|
Bencher Report| Branch | 2061/merge |
| Testbed | ubuntu-latest |
⚠️ WARNING: The following Measure does not have a Threshold. Without a Threshold, no Alerts will ever be generated!Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsCLI flag.
Click to view all benchmark results
| Benchmark | Latency | nanoseconds (ns) |
|---|---|---|
| fibonacci 10 | 📈 view plot | 497,790.00 |
| pidigits 100 | 📈 view plot | 3,241,700.00 |
| product 30 | 📈 view plot | 842,070.00 |
| scalar 10 | 📈 view plot | 1,531,500.00 |
| sum 30 | 📈 view plot | 839,480.00 |
Yeah, it's the same for the actual stdlib code - often functions refer to other function of the same module directly, but use |
flake.nix
Outdated
|
|
||
| stdlibTests = pkgs.runCommandLocal "stdlib-test" { } | ||
| '' | ||
| ${pkgs.lib.getExe self.packages."${system}".default} test ${./core/stdlib/std.ncl}; |
There was a problem hiding this comment.
Isn't the ; at the end going to make the command pass whatever is the actual result? Maybe a sanity check would be to push a commit with a deliberate error in the stdlib somewhere, just to be sure.
There was a problem hiding this comment.
I don't know exactly why (maybe runCommandLocal does a set -xeu or something?) but it seems to fail if I make it fail.
There was a problem hiding this comment.
But why do you need the ; ? Can't we use a &&? Do we also have to necessarily create $out ?
There was a problem hiding this comment.
Sure, I can use && instead; the behavior seems to be the same.
Yes, without making $out then nix flake check fails with
error: builder for '/nix/store/kyr0xs7ncr05i3frnphvvvr1q8p8ja1c-stdlib-test.drv' failed to produce output path for output 'out' at '/nix/store/kyr0xs7ncr05i3frnphvvvr1q8p8ja1c-stdlib-test.drv.chroot/root/nix/store/yil1zc0j0vy5p3sjsp0c1kqi8r0h6krj-stdlib-test'
There was a problem hiding this comment.
Thanks, looks good! I think && is indeed better as it doesn't rely on assumptions about how runCommandLocal works internally 👍
This reverts commit 07c34e5.
An amusing little thing that I think is harmless: when a test in the stdlib uses the
stdname (likestd.contract.not) then it's testing the copy of the stdlib that was compiled into the nickel binary. But when it uses a "local" name (likeHasField) then it's testing the copy of the stdlib that it gets from the path./core/stdlib/std.ncl.