GraalVM native image compilation of mal

[borkdude]

Hey @mmcgill, relaying a message from @borkdude who tried to build native with native-image and ran into some issues: https://gist.github.com/borkdude/e0e12997ce3a7b09b44f15769add85a9 (tweet for reference: https://twitter.com/borkdude/status/1400149389654597632)

If either of you are interested in working this further, can one of you open a mal issue for tracking and discussion? Thanks.

Originally posted by @kanaka in #528 (comment)

Reply from @chumer:

https://twitter.com/grashalm_/status/1400156055129055232

Way too many runtime compiled methods. Try running with -H:+TruffleCheckBlackListedMethods. Note that what gets marked as runtime compiled is quite subtle sometimes. It is enough to just include bad call to Function.apply to pull in too much code.

If that does not help. Lookout for missing @TruffleBoundary. If that does not help to get the numbers down, binary search the problem by commenting code. The error you are seeing you should actually not get.

[borkdude]

By request of Thomas Wuerthinger, I also logged an issue here: oracle/graal#3448

[mmcgill]

Hi @borkdude! Thanks for sharing this. I never got around to trying native image compilation, so I'm not surprised it doesn't work yet.

I'll see if I can narrow the problem down. Hopefully it turns out to be fixable without significant rework :)

[mmcgill]

Well, good news and bad news :)

Good news: I've got a native image of step2_eval (though I had to add a missing @TruffleBoundary and then tweak some code to work around a SIGSEGV during compilation that might warrant another bug report if I can manage to provide a smaller reproducing case).

Bad news: Getting past your error required deleting all the other steps. I don't know anything yet about how GraalVM's native compilation works, but hoped that maybe some sort of early reachability analysis would mean that all the duplicate code gets ignored when building from a particular step. That doesn't seem to be the case :(

So, following the Mal implementation pattern with each step in its own file might significantly complicate the production of native images in a straightforward way.

It would be awfully neat, though. I'll think on it a bit, and in the mean time see if I can get stepE_macros to AOT compile. Worst case, maybe I push a branch w/ just step E to my fork.

[borkdude]

@mmcgill Cool! Please keep me posted.

[mmcgill]

Success! Step E AOT compiles now, only took a few minor changes.

~/repos/mal/impls/java-truffle$ time ./run ../tests/fib.mal 30 10
Times (in ms) for (fib 30) on java-truffle:  [186 47 25 21 26 21 24 30 30 24]

real    0m2.886s
user    0m3.126s
sys     0m0.444s
~/repos/mal/impls/java-truffle$ time ./mal ../tests/fib.mal 30 10
Times (in ms) for (fib 30) on java-truffle:  [143 40 17 17 19 16 17 18 18 17]

real    0m0.333s
user    0m0.316s
sys     0m0.123s

I'm hopeful that I've found all the trouble spots. I'll get the rest of the steps in shape. I'm thinking I'll also update the build to (optionally?) produce a native image that gets used to run the tests.

Thanks for trying this out, @borkdude!

[borkdude]

Looking forward to inspect the changes and learn more!

[mmcgill]

@borkdude I've submitted a PR. All steps AOT compile for me now.