eval code with checkpoint_dir is failing, but with checkpoint is succeeding

[nikihowe]

Figure out what's going on

[nikihowe]

For now, the workaround is to just call each evaluation individually, which works but might be slower.

[rhaps0dy]

Is this still an issue @nikihowe ?