ags4 and ags3

[ericoporto]

So, been thinking a lot about ags4, and wanting to document it, I guess I could start already with branches in the repo that is in the wiki (I have a mirror here).

But since it seems GitHub never added new features to the wiki (like branching in the wiki itself would be nice), it seems it would be more useful to move to a regular repository. Or should we instead create two directories, ags3 and ags4, and maintain the two directories in this repository? Or should we have an ags4-manual repository?

Basically, just wanted to see the general feelings regarding where to go. Overall there is a lot to document already and I would like to start separating the tasks and working on them because it's a lot of things...

NOTE: on using two directories, the sdlwiki is a complex thing, but it does store sdl2 and sdl3 as separate directories (here), this is where I picked the idea.

[AlanDrake]

Branches or subfolders.
I guess the real question is what makes it easier to fetch the chm for the AGS builds.

In any case we should move the wiki contents into a real folder ASAP. I should be able to add "edit page" links to the web version, so we have easy access to make changes.

[ericoporto]

I guess the real question is what makes it easier to fetch the chm for the AGS builds.

I don't get, those will be the releases, they shouldn't change much, I guess there would be an ags3 package and an ags4 one. The issue happens if they aren't both in the same branch because then we can't have release both in a single release - or perhaps we can it will look a bit weird though.

In any case we should move the wiki contents into a real folder ASAP

You mean in this directory? Yeah, I think we could do that. Perhaps have the markdown files here directly and then have two branches - one for ags4 and another for ags3? Or two folders, one for ags4 and another for ags3?

There is also an issue of what happens to the GitHub Pages - the online version of the manual. Should it only be one for ags4 or should it be two subdirs? It's flat too, so if we want one could be the "default" one - without subdirs.

[AlanDrake]

So, both in the same branch is the least contrived path to generate both version at once.
Otherwise we'd have to git clone the extra versions from their own branches, I guess. Still viable, but extra steps that could be unnecessary unless there's a good reason to keep separate branches.

Or two folders, one for ags4 and another for ags3?

There is also an issue of what happens to the GitHub Pages - the online version of the manual. Should it only be one for ags4 or should it be two subdirs? It's flat too, so if we want one could be the "default" one - without subdirs.

I think we should move all versions to subfolders, because in the future we might have the Editor search terms online, an in that case there should be a deterministic link to the version required.
The root index.html can have an http-equiv to redirect to the current version, and then I can add a dropdown with links to other versions (this is how I've seen it handled in several other versioned doc sites).

Of course, moving the contents in a subfolder will temporarily break the google results, until recrawled.

[ericoporto]

I really don't like MULTIPLE versions besides the two major ones because when looking in Google it's always the wrong (not the latest) that is crawled. This is my experience with python and cmake. Need to think a bit more.

[AlanDrake]

it's always the wrong (not the latest) that is crawled

That's true 🤔... but I could add a noindex on the older versions and that should fix the aggravating outdated results.

[AlanDrake]

Thinking on it again though, multiple copies in a repo means it might grow exponentially if we keep copying the whole thing every minor version.
Either we only keep major versions or the branch approach starts making sense in the long run...

[AlanDrake]

Ok, I guess I'll start by working on a version without wiki.
Next I'll see what needs to be done with templates so that we can have a dropdown to switch between version.

After that we'll see how to set CI scripts so that we build both manuals at once, master branch will likely get ags4 contents, and an ags3 branch will be used to pull v3 contents on the fly. But we can decide later on the details.

[ericoporto]

I kinda like git for copying things between branches and I think the tooling that exists is fairly robust. Would it be too much to have two repositories, one for the content and other for the building of things? Essentially just moving the wiki to its own repository so we can have branches and PRs since the current wiki doesn't give the web functionality - even though it's git repository does support branches they aren't visible online.

[AlanDrake]

That works too. As long as the docs content is in a branch somewhere, I can add a link to it for editing.

[AlanDrake]

Alright, we now have a proof of concept in PR #251 with separate repo for contents (https://github.com/adventuregamestudio/ags-manual-source/).

Moving from wiki to raw md files means the github preview's link are not working, because they should end with .md to be recognized.
Does it matter though? I expect navigation to happen in the github pages, and users landing here only to make specific edits.

We could make a branch for ags 3, and start adding 4.x contents. Or make individual branches for both, and decomission master?
Also, should sources be here as submodules, or shoudl we pull them on the fly with a git command?

[ericoporto]

@morganwillcock do you have a suggestion here regarding handling building two manuals?

[morganwillcock]

Personally, I would try to put the manual content back into the main "ags" repository, where it naturally branches with the AGS versions. This means there is no question of how to associate the manual pages with the correct AGS version and the regular CI build for AGS can be used to build the associated manual (by cloning the "ags-manual" repository and using it).

Note: With the wiki-based editing gone I am imagining that at some point in the future it will make sense to transition the manual content to a format which is directly supported by a traditional documentation build tool, so part of my reasoning for the above is to try and fit the model of the traditional tools and allow changes to AGS to also include the associated documentation changes.

[AlanDrake]

I'm not against moving the doc sources back into the main repo, but the github pages will still need to be generated.
I can imagine that we'd end up with both CIs building manuals, one to build/update the web manual, and another in the ags repo to generate the chm.

But how do we get there from here? Should we halt everything and let someone well versed in CIs makes the necessary adjustments?

Or, in the meanwhile, we go forward with the ags-manual-source repo, so we can start making a branch for those ags4 edits, and then we see what to do?

[morganwillcock]

the github pages will still need to be generated

Assuming the documentation is placed back with the code, there are 4 places a site can be generated from:

• AGS 3 development
• AGS 3 releases
• AGS 4 development
• AGS 4 releases

Do you just want a single site from one of those sources? Or do you want to publish more than one? (Maybe the latest AGS 3 release can be at the root of the hosting and other sites are available through sub-directories, or something like that?)

If needing to trigger a build in or push to another repository, I don't think the default access token will work, so that probably leaves using the CI user again and a personal access token, or running the builds on a schedule. Technically it would be much easier to host a site on the main ags repository which also incorporates the manuals. The downside of that is that the previous URLs which people have used aren't going to work, and there is probably no way to do a redirect that covers direct access to the old pages.

Edit: It does look like you can have a custom 404 page and do a redirect in that using JavaScript.

[ericoporto]

I think moving to the same repository is a bad idea and will make anytime you change a letter to rebuild the entire engine and editor and Everytime you edit the code rebuild the entire manual and webpage and such. If they are kept separate we do better use of the CI resources.

By keeping things separate we don't waste the CI and just fetch things.

[ivan-mogilko]

I am also concerned about access: right now we may have separate access permission for editing the manual source, and it's not hard to reset these changes if someone did overly wrong or malicious edits. If the manual source is inside the main repository, then how do we separate permissions?

We already have a working branch association logic in e.g. templates, where template sources and ready template repo have branches corresponding to the engine/editor branches.

It does not seem to me that having manual inside the main repo would give much benefit, any change to the source format may be done within a dedicated repository as well.

[morganwillcock]

By keeping things separate we don't waste the CI and just fetch things.

I imagine you can filter out tasks within a CI job based on which paths have changed.

If the manual source is inside the main repository, then how do we separate permissions?

You would just take pull requests, but I think the majority of changes will be done by people who already have access because that is what happened when the wiki version was open to anyone.

It does not seem to me that having manual inside the main repo would give much benefit, any change to the source format may be done within a dedicated repository as well.

I gave my reasoning in a previous comment.

(I was asked for my suggestion but I'm not going to argue that it be followed - if you want to do something else then you don't need any approval from me.)

[ericoporto]

I imagine you can filter out tasks within a CI job based on which paths have changed.

Uhm. This may work. I think both Cirrus and GHA have a cache system, so the way this could be done is caching the manual when it triggers a build from the markdown pages and then pulling from the cache in the other case. I think the cache would then require some way to know that nothing changed - we use cache in Cirrus for the macOS building dependencies.

I think the webpage CI can be kept in the ags-manual repository though and be triggered either manually or by some condition - say once per day if some condition changed.

But I would like to see if it's possible to keep the existing tooling instead of changing right away. After all afaict it's pandoc + Lua scripts + templates + config. It's mostly the "packaging" of this to be easy to run.

I think I still like the manual being in a separate repository because I think writing it's text requires a different headstate and I am not sure yet on the effort of merging across branches if it's in the main repository - along many things causing conflict vs here, plus if it will cause a pressure to document things as they are developed - which could hinder additional development, I think it's better to have things working and later document once the dust has settled and some experience with the feature has been gained.

[ivan-mogilko]

I'd really rather have it separate for organizational reasons. It's true that there were not alot of contributors to the manual, but there still were few, and besides, that is a problem of us not being to get more people into this task.
TBH I don't see much logical difference between a manual and game templates: both have to be linked to a particular engine (or rather editor) branch, and both have to be built for the release. In my view it's the similar case.
Having them separate from the main repo means that these repositories may in theory be passed to other responsible sub-teams of people, and do not require to clone whole engine repo to be worked on.

[ericoporto]

I will look into this in the weekend and get it to build from @AlanDrake repository and figure how it should go, my idea is to simply build two things from the top of branches and perhaps just have some file with the name of the branches, then if more branches are required it would be just adding them in the text file and the engine will then fetch the latest release artifact that has the name with a pattern that matches the branch.