Improve mal impl macro no meta

[asarhaddon]

This pull request should trigger tests for the changes discussed at #384.

[kanaka]

@asarhaddon okay, I'm finally get back around to reviewing some of the mal/macro/quasiquote related changes. I'd like to start with getting this one in first. I'm fine with the approach and I think removing the need for metadata is a big plus. There are two minor things that I would like changed:

• Printing of macros is ugly. The prior behavior was that macros would be printed just by printing their underlying functions (however the hosting implementation did it). Now it's printed as a vector with a magic keyword:

$ make MAL_IMPL=js repl^mal
Mal [javascript-mal]
mal-user> cond
[:_I_tell_ya_this_is_a_MAL_macro_mark_my_words function () {
        return Eval(ast, new Env(env, params, arguments));
    }]

I think probably the simplest solution is to just retain the previous behavior by adding a _macro_unwrap check to PRINT (for steps 8-A) and printing that if it's "true" otherwise printing the whole thing.

• sequential? and vector? should also have a _macro_unwrap check and return false on macros.

There are also some other behavior changes (such as being able to use macros with sequence functions first, rest, nth, etc) but I'm not really concerned about those and don't think it's worth adding any additional complexity to make them explicitly check and throw errors.

[kanaka]

Oh, I just realized this is a pull request on the "self-host-test" branch. I should really delete that branch because it's just confusing things. It was an experiment that kind of dead-ended. Instead, I have a script in tests/travis_trigger.sh that I can use to re-trigger a Travis build with custom configuration. Unfortunately, Travis is pretty limited and I think I'm the only one that can use it against my branches. :-( Looks like #401 is also targeted at self-host-test.

Anyways, can you re-target this at master (and merge if needed since master has since moved on)? Once you have done that I'll trigger a custom build in self-host mode. If that passes, then I'll merge. Sorry about the confusion.

[kanaka]

Self-hosted test is running here: https://travis-ci.org/kanaka/mal/builds/556847476 Looking good so far.

[kanaka]

So three implementations failed self-host. I did a self-host build of just those three but without these changes and they passed (https://travis-ci.org/kanaka/mal/builds/556920394), so something about these changes is breaking those three.

Also my earlier suggestion for printing is incomplete because it doesn't catch functions embedded in other things. i.e. [cond]. To catch those we would need a full printer that handles collections (map/vector/map/atom) and mal functions specially and then delegates to the underlying printer for basic types. Probably a bit like clojure or hy. But then it has to do string joins too. I think it would be a non-trivial performance hit for self-hosted on slower host languages.

All that to say, I think I'm reconsidering the special printing and leaning towards just printing the underlying object directly. However, I think I would like to switch to using a map rather than a vector as the container so it appears a bit more object-like. Something like this:

{:__MAL_MACRO__ <function>}

fn? and macro? should obviously return false and true for the macro maps but I'm kind of inclined just to leave map? handling as is (i.e. true for macros). Thoughts?

[asarhaddon]

Adapting map? to report false for macros does not cost much, so I have included this.
The powershell implementation may only miss a ; $null after the Write-Host in println/prn, but I cannot test this locally.
I suspect that the rexx problem is related to duplicate gensym definitions using distinct counters after two indirect load-file trivial.mal. I already have a separate branch introducing load-file-once, but I have failed to confirm that this is the actual problem so far.

[kanaka]

Changes look good. The regular tests pass. Running self-host tests again: https://travis-ci.org/kanaka/mal/builds/558757626

[kanaka]

@asarhaddon There are 7 languages that fail now. I pushed the fix you suggested for powershell and that seems to have addressed that one. I poked at the haxe failure for a few minutes and discovered that this change fixes the the rest of the failures apart from rexx and nasm:

--- a/mal/core.mal
+++ b/mal/core.mal
@@ -1,11 +1,11 @@
 (def! _map? (fn* [x]
   (if (map? x)
-    (not (= (keys x) '(:__MAL_MACRO__)))
+    (if (contains? x :__MAL_MACRO__) false true)
     false)))
 
 (def! _macro? (fn* [x]
   (if (map? x)
-    (= (keys x) '(:__MAL_MACRO__))
+    (contains? x :__MAL_MACRO__)
     false)))
 
 (def! core_ns

I think that indicates that there is probably still some hidden bugs in those implementations around comparison (or the definition of not). However, I won't have time any time soon to track those down and the the change is more efficient that doing = on a collection.

Interestingly, the nasm failure appears to be because the step9_try.mal file is exactly 4096 bytes. Adding or removing a character anywhere in the file fixes the issue (likewise, padding step1 to 4096 makes the problem happen there too).

@bendudson can you take a look at why nasm crashes when trying to do load-file of a file that is exactly 4096 bytes?

make repl^nasm
(load-file "../mal/stepA_mal.mal")
Mal [nasm-mal]
nil
mal-user> (+ 2 3)
5
user> (load-file "../mal/step1_read_print.mal") ;; padded to exactly 4096 bytes
Error: Run out of memory for Array objects. Increase heap_array_limit.

[asarhaddon]

The issue with vala seems different. An assertion fails during the initial traversal of core_ns. It can be reproduced with make vala^stepA MAL_IMPL=vala repl^mal^step4. Moving _map? and _macros? from core.mal to the step file removes the message.
I suggest to open a separate issue referencing the previous Travis build, in order to keep track of the bugs in sequence comparison.

[kanaka]

Good idea. Done: #418

[kanaka]

New self-hosted Travis build running: https://travis-ci.org/kanaka/mal/builds/559063412

[kanaka]

Okay, since the only two failures now are nasm and vala and those appear to be issues discovered by the changes rather than introduced by the changes (and we are tracking those separately in #418), I'm going to do one more quick review of the code and then merge if nothing big sticks out at me.

[kanaka]

Merged. It's a big diff, but the majority of it is simplifying individual implementations by removing inc, gensym, and or definitions. And I think removing the need for metadata from the self-host requirements is a good improvement that remove incidental complexity from necessary steps the process/guide without losing any of the value. Thanks for persevering and pushing this through!