Is nickel evaluation lazy?

[Profpatsch]

A question I could not find answered in the README:

Is nickel evaluation lazy, that is will it only evaluate the minimal subset of the configuration that needs to be evaluated?

This has multiple dimensions:

• Can it typecheck modules in a stand-alone fashion?
• Does it only import files that are actually needed for the result?
• Can evaluation even be lazy in the face of contracts?

I’m asking because in practice, any nontrivial e.g. kubernetes config will depend on thousands of definition files, yet e.g. a single service will only need a tiny subset to be instantiated.
This is a big problem with e.g. dhall configs, where the Haskell implementation is not lazy at all.

Even in nixpkgs we have some problems when it comes to NixOS configs, even though the language implementation is lazy in theory (yet the module option merging implementation in practice is not).

[oneingan]

https://nickel-lang.org/user-manual/contracts/#laziness

[Profpatsch]

The docs says “nickel is designed to be lazy”. That doesn’t super answer my question whether it actually is implemented in a lazy fashion. For example, dhall is a total language, so implementations can go any way they like, strict or lazy, or in between.

What about file loading?

[yannham]

Those are interesting questions, because lazyness indeed comes in many different flavours and definitions.

Nickel is lazy in the usual PLT sense, in that the evaluation is call-by-need, as in Haskell and Nix. If you only need one subfield of a huge configuration, you will only evaluate this field. Let me try to address your other questions:

• Currently, not entirely, because Nickel needs to peek at the top-level annotation of an import to get its type. Because there is currently no notion of module and interface, if you do

  let mylib = import "lib.ncl" in
  

  and use it at some point in a statically typed context, you need to know the type of mylib. How it is currently done is that all imports are resolved before typechecking, to have them available at typechecking time. We then typecheck everything.

  If we focus on just one file, we don't need to parse much of the imports because we just look at the top-level annotation, and if the import doesn't have a top-level type annotations then its type is Dyn (not always exactly the case, but true for all intents of purpose). So we can imagine having a typechecking mode that says "only typecheck the main file" (or should that mode be the default? I don't have a definitive answer) and just parse what's needed from the direct imports. We'll also have an option to eschew typechecking before evaluation altogether, which will probably be what you use in production (typechecking only on commits). Or have a typechecking cache. Or Both.

• Because of the previous point, we currently import everything transitively. But this is not entirely a hard requirement as explained, and the Nix use-case most probably needs at least an alternative primitive for lazy imports.

• This is an interesting question, too. Our design guideline is to indeed make contracts preserve laziness, and this is the case currently. One example is that we backed off from implementing full-fledged union and intersection contracts for this very reason (unions would need to be either eager or to become a complex and effectful feature)

Summing up, the evaluation of model of Nickel seems to tick your boxes with respect to laziness, at the exception of imports. But the latter is not totally inevitable, as the typechecking model doesn't require to process imports (such as performing type inference) and relies on a form of external interface declaration: it's just that this interface is currently just a top-level annotation, which is a bit clunky to separate from the rest of the code.

[Profpatsch]

Have you thought about adding the concept of an interface file? e.g. an import of lib.ncl would search for a file lib.ncli which should contain the type of lib.ncl, as e.g. in OCaml.

[yannham]

Yes indeed, this makes total sense, since they have the same motivation in OCaml (not only, but in part for separate compilation and typechecking). I think at some point the conclusion on this proposition was that for now we don't want or need a module system, as this is yet another complex feature to learn and to understand, and we'll try to do everything with plain recursive records + types & contracts which are already powerful. But in fact we could still have a very simple notion of an interface: it can just be the type of the corresponding .ncl file, which does look like an usual interface definition in the specific case of a record.

[Profpatsch]

Yeah example, I’d propose the .ncli file would just be a type signature.

I have the same problem in dhall, usually what we do there is just attach a type sig to the expr, but that’s a bit hack-ish and leads to bad error messages (plus, you have to parse the whole thing to get to the type signature).

[axelkar]

NixOS/nix#4154 Is there anything like this in Nickel?

Also will the following work in a Nickel representation?

({ a = abort "foo"; } // { a = 1; b = 2; }).a

[yannham]

The corresponding in Nickel does indeed evaluate as expected (on current master, but note that fail_with is not stabilized and may change name before the next release):

nickel> ({a = builtin.fail_with "foo"} & { a | force = 1, b = 2}).a
1

nickel>

So, yes, Nickel is mostly lazy as Nix is today.

NixOS/nix#4154 Is there anything like this in Nickel?

No. I'm not sure we even need it: the merging system and the architectural differences make a number of issues (like overriding) disappear, or at least different. Honestly, from a distance, it sounds like a strange feature, as it becomes really hard to predict - as a programmer - what would be the result of extracting a field from a non-trivial record expression. My first feeling is that it's a XY problem: because Nix is as it is, and needs to remain backward-compatible, this is one possible solution to some of the issues mentioned in NixOS/nix#4090. However, as we started fresh, I would avoid it and rather solve the motivating issues differently to begin with, as much as possible (one random example, the mkIf, we would rather provide a native metadata for that, see eg #459)