easy partial support for regexp -expanded

[The-Markitecht]

<gush>Been using jimsh every day for many years. It's clearly the ultimate evolution of Tcl. It should just be mainstreamed on tcl.tk instead of tclsh.</gush>

But, I used regexp -expanded all the time in tclsh (such as #310 (comment)).
jimsh never had it, and I can see why.

But every time I use any regexp, I have to start like this:
set re [string map {{ } {}} { ^ \s* (\S+) \s+ (\w+) \s+ (\d+ \. \d+ \. \d+ \. \d+ / \d+)+ \s* }]
That adds a LOT of braces and loss of legibility in order to write an expression that's supposed to have GREATER legibility. One step forward, two steps back.

I propose an easy solution: support regexp -expanded which simply deletes all the spaces, as in my example. Don't bother with the comment feature offered by the same option in tclsh. I've never needed it.

That solution should run fast as well. Much faster than my example. And not add much executable size at all.

And it doesn't have to be called -expanded. There's an argument for avoiding that since the feature isn't identical to tclsh. So it would be fine to call it something else, like -compress. But not having it at all is a real nuisance.

If you can't make time for this, I understand. My life is crazy right now, but I might be able to make time for this small PR.

[The-Markitecht]

Deleting also tabs, newlines, and other whitespace-class characters in the same pass is also a good idea. Then we can have multi-line regex's, which is occasionally useful for legibility.

[msteveb]

Thanks for the feedback and the suggestion. I pushed something to master-next that you can try

[The-Markitecht]

Huzzah!!!

I tested that. It's great!!

Spaces, newlines, tabs, comments all worked the way you intended in regexp (i looked at the source). I didn't test regsub.

And it's even documented!

THANK YOU!!! This has pestered me for YEARS, ever since the first week i used jimsh.

Along the way I also noticed in action several of the other improvements of this year. Excellent!!

One question. In this part:

@@ -993,6 +1014,12 @@ cc_switch:
 					break;
 				}
 
+				/* For REG_EXPANDED, if we hit white space, stop */
+				if ((preg->cflags & REG_EXPANDED) && n == 1 && strchr(" \t\r\n\f\v", ch)) {
+					preg->regparse += n;
+					break;
+				}

What case is that for? It wasn't obvious what its effect is.

[msteveb]

You're welcome. If we don't stop when we hit white space, it will become part of the exact match.

[The-Markitecht]

Here are some -expanded test cases to immortalize in this thread. It's probably also a good idea to add these to the official build-test scripts and maybe the documentation. Most of the behaviors are intuitive enough to accept as-is, as long as everybody understands them. So this can be a little cheat-sheet.

Welcome to Jim version 0.84
. info patchlevel
0.82-134-gdcea42c
. regexp -inline -expanded { bcd } {abcde}  ;# spaces at ends of pattern are correctly ignored.
bcd
. regexp -inline -expanded { b c d } {abcde}  ;# spaces between atoms are correctly ignored.
bcd
. regexp -inline -expanded { ^ a b b c d } {abbcde}  ;# spaces around caret are correctly ignored.
abbcd
. regexp -inline -expanded { ^ c d } {abbcde}  ;# caret is still effective.
. regexp -inline -expanded { ^ a b{1,3} c d } {abbcde}   ;# quantifier works as usual.
abbcd
. regexp -inline -expanded { ^ a b{1,3} c d } {abbbbbbcde}   ;# quantifier works as usual.
. regexp -inline -expanded { ^ a b {1,3} c d } {abbcde}  ;# spaces before the quantifier make it ineffective, and no error is thrown.
. regexp -inline -expanded { ^ a b {1,3} c d } {ab  cde}  ;# quantifier doesn't apply to that space character preceding it either, so it applies to nothing!
. regexp -inline -expanded { b { 1 , 3 } c d } {abcde}  ;# spaces are recognized as part of the quantifier, an illegal part.
couldn't compile regular expression pattern: braces {} not balanced
. regexp -inline -expanded { b [cjk] d } {abcde}  ;# character set works as usual.
bcd
. regexp -inline -expanded { b [c jk] d } {abcde}  ;# character set still works even with a space in it.
bcd
. regexp -inline -expanded { b [c jk] d } {ab de}  ;# space is recognized as part of the character set, and it's effective.
{b d}
. regexp -inline -expanded " b \n d " {abcde}
. regexp -inline -expanded " b \n c \n d " {abcde}  ;# newlines are correctly ignored.
bcd
. regexp -inline -expanded " b \n # c \n d " {abcde}  ;# comment at start of line is correctly ignored.
. regexp -inline -expanded " b \n c # comment \n d " {abcde}    ;# comment at end of line is correctly ignored.
bcd
. regexp -inline -expanded { a (bb) c d } {abbcde}  ;# capturing group works as usual.
abbcd bb
. regexp -inline -expanded { a ( bb ) c d } {abbcde}  ;# spaces around parens are correctly ignored.
abbcd bb
. regexp -inline -expanded { a ( b b ) c d } {abbcde}  ;# spaces between atoms inside parens are correctly ignored.
abbcd bb
. regexp -inline -expanded { a b \s b c d } {abbcde}  ;# spaces can still be specified using a character class (or a bracketed character set with just a space in it) and they're effective.
. regexp -inline -expanded { a b \s b c d } {ab bcde}  ;# space class is effective.
{ab bcd}
. regexp -inline -expanded { a ( b b )+ c d } {abbbbbbcde}  ;# group quantifier is still effective.  note only the final submatch was captured.
abbbbbbcd bb
. regexp -inline -expanded { a (?: b b )+ c d } {abbbbbbcde}  ;# non-capturing group quantifier is still effective.
abbbbbbcd
. regexp -inline -expanded { a (?: b b )+ } {abbbbbbcde}  ;# default greedy quantifier works as expected.
abbbbbb
. regexp -inline -expanded { a (?: b b )+? } {abbbbbbcde}  ;# non-greedy quantifier works as expected.
abb
. regexp -inline -expanded { a ( b b )+ } {abbbbbbcde}  ;# capturing group with greedy quantifier works as expected.  note only the final submatch was captured.
abbbbbb bb
. regexp -inline -expanded { a ( b b ) + } {abbbbbbcde}  ;# space before quantifier SEGFAULTS!!  issue #313
Segmentation fault
$ jimsh
Welcome to Jim version 0.84
. regexp -inline -expanded { a ( b b )+? } {abbbbbbcde}  ;# capturing group with non-greedy quantifier works as expected.  note only the final submatch was captured.
abb bb
. regexp -inline -expanded { a ( b b )+ ? } {abbbbbbcde}  ;# space in the middle of non-greedy quantifier makes it ineffective, and throws no error.
abbbbbb bb
. regexp -inline -expanded { a ( b b )+ ? c d e } {abbbbbbcde}  ;# include those trailing characters in the match, as baseline for the next test.
abbbbbbcde bb
. regexp -inline -expanded { a ( b b )+ ? c d e } {abbbbbb cde}  ;# no, the spurious question mark does not cause the preceding space to be recognized.  good.
. regexp -inline -expanded { a ( b b ( c d )+ b b ) e f } {abbcdcdcdbbef}  ;# spaces in nested parens work as expected.  note only the final submatch was captured.
abbcdcdcdbbef bbcdcdcdbb cd
. regexp -inline -expanded { a ( b b ( c d )+ b b ) e f } {abbcdcdcdef}  ;# absence of trailing bb (part of outer group) correctly rejected.
. regexp -inline -expanded { a ( b b ( c d )+ b b ) e f } {abbbbef}   ;# absence of inner group correctly rejected.

[msteveb]

Looks good. Even better would be a pull request with these tests added :-)

[The-Markitecht]

sure, i'll look in the test suite and see if that change is practical for me...