implement placeholder replacement

[skaylink-AndreasUlm]

This PR implements the argument "--replace-placeholders-file".
It allows to specify a YAML structured file that defines a mapping of strings/regex and their replacements.
Using this file you can define that some text used in markdown files (e.g. Hugo Shortcodes is replaced by text or Confluence Macros.

The added sample_placeholders.yaml show some examples.

The syntax of the file allows specifying text or regex as key.
For regex groups are supported and can be backreferenced (see example 3).

The additions key is a list of strings that are added without modification (e.g. example 3).
For this feature I added a check to ConfluenceTag.render() thus child.render() is only called when child is of type ConfluenceTag.

I did not implement the replacements within ConfluenceRenderer as it seemed impossible to extend the default functions of mistune.Renderer in a way that any custom configurable text will be replaced.
Instead I'm using re.sub() as this implementation allow to store all "magic" for a replacement within the configuration file.

[iamjackg]

I feel very conflicted about this, because I appreciate the use cases this solves but I don't particularly like the way it was implemented.

Mistune (even on the old version I'm using) supports custom lexers to parse custom grammars, but your implementation is sidestepping all of that and going a roundabout way to do the replacements outside of the normal flow.

[skaylink-AndreasUlm]

Mistune (even on the old version I'm using) supports custom lexers to parse custom grammars, but your implementation is sidestepping all of that and going a roundabout way to do the replacements outside of the normal flow.

I was conflicted for the same reason and invested about a day to change the implementation using those custom lexers. The problems I faced were the following:

• there is no easy entrypoint when you require a lot of different lexers
• it is hard to implement custom lexers based on a configuration file as the placeholders are varying a lot between the different markdown specification (hugo vs. github vs. gitlab, etc.)
• I could not figure out how the custom grammars are fully activated

Because of those problems I decided to start with my first implementation as PR and hope for such a discussion. :)

1. Is custom grammar automatically handled by mistune when injected as InlineLexer?
2. Do you have an idea how to create rules dynamically based on the configuration file?
3. Is the function output_... automatically called?

As you are okay with the feature itself, I'll test again an implementation based on custom lexers.
If you have some hints, I would appreciate hearing them.

[iamjackg]

Hello :) Were you able to work on this at all? Just checking.

[skaylink-AndreasUlm]

Hello :) Were you able to work on this at all? Just checking.

I'm still trying to figure out a nice pythonic way to implement the feature by using mistune.
Until now I already know for sure that it is not working with the default flow of mistune but requires overwriting the function https://github.com/lepture/mistune/blob/v0.8.4/mistune.py#L572.
With the current information available in output_%s functions the code cannot generate the replacement content as niether the configuration nor the regex are available.

I had a short look into mistune v2 but the change is to much for me to oversee and integrate thus I'm working on the function overwrite.

[jblotus]

I am looking for exactly this sort of feature - I have gitlab markdown documents (in code) and I am setting up a process to update confluence when those docs change. The docs have a lot of "widgets" like __TOC__ and gitlab specific things. Additionally I would like to perform some transformation on the urls in the markdown document.

I can rewrite the urls in a preprocessing step, but I can not figure out how to directly include a confluence macro in the final document before it's uploaded. I assume the library can help me here but a general purpose solution like this would be nice to have.

[skaylink-AndreasUlm]

Unfortunately I don't have access to systems required to test this implementation anymore. Thus I cannot further work on this feature.
Maybe someone else can take over this PR.

[iamjackg]

@jblotus Have you tried simply including the Confluence tags in your documents and seeing if they make it through untouched?