Mashups being added to the wrong projects leading to false imports

[Frowne]

Good day,

We've been using the extension for a while and have been super happy with it up until now - the last time we pushed a patch however, I noticed that some files appeared out of date. Checking the repository revealed that there are two folders for each of the projects, let's call them Project A and Project B.

Project A should only consist of three Mashups and no other things, while Project B holds a bulk of our working environment. The problem is, that a huge chunk of Things and Mashups exist on both of them with different last updated times.

Since the latest Thingworx / Extension update, we always import the full path and my best guess is, if a Thing's updated version is only in Project A, then it will be overwritten by the old version in Project B, which leads to the problem mentioned above.

Here's my question then - are we using the Extension wrong or is there an underlying problem? The way our export process works is as follows:

1. Export -> Select Project A -> Include dependents -> Add All + Export
2. Export -> Select Project B -> Include dependents -> Add All + Export
3. Push with Commit Message

When exporting Project B like this, we do see the list of files that were in project A, but I assumed since we selected the project, it wouldn't be exported into the folder the project doesn't belong to, which might be wrong - it might also be the include dependents that's causing issues, but I would love some insight on the matter so we can either do it properly in the future or shed light on an existing issue.

Best regards!
Frowne

[vrosu]

Hi @Frowne,
I believe I see what's going, but I am not very sure - some assumptions follow, please validate.
If you had Entities that in the past were assigned to Project X, then exported, they would exist in the Project X folder. At this time, you commit. If you switch the entities to Project Y, then export, they would exist in the Project Y folder. You need to know that the extension does nothing fancy in the background (like ensuring there is only a version of the entity), so if you commit this change as well, you will get both entities in both folders.
When you change the projects, after you export the entities, you would need to go in the Working folder, and delete the entities you do not need (previous entities existing in the older project folder etc) before you commit.
Again, I'm not entirely sure, but this is my best guess.

[Frowne]

That's basically how it plays out - I would only commit after exporting Project X with all things once, then swapping to Project Y and exporting all of those, which is probably why they are all jumbled up.

So just so I understand you correctly - the only way to correct this now is to delete the files from the repository, push that & then be more careful in the future?

Also, I'm not sure I'm following on the "correct" commit procedure - do I:

1. Export Project X, add all + export, then export Project Y, remove everything that I added from Project X, then export & push
2. Export project X, add all + export, push to remote, then do the same for project Y in a whole separate process

Thank you so much for your help! I genuinely appreciate it.

[Frowne]

If you had Entities that in the past were assigned to Project X, then exported, they would exist in the Project X folder. At this time, you commit. If you switch the entities to Project Y, then export, they would exist in the Project Y folder.

Actually, upon re-reading I think I misunderstood - no, the entities always belonged to Project X or Project Y respectively, that never changed - I only exported them through the Git Main Mashup!

[vrosu]

@Frowne
Summarizing the best I can, after reading both replies:

1. The correct commit procedure is option 1. The basic procedure to follow is that before each commit you need to clean, if needed, your working index (one of the main reasons I added file-based operations in the extension, but you can do it via the ThingWorx repository UI - which was not available years back but anyway it's harder since you switch.)
2. What you can do now, to fix this, is to clean the working index of the latest commit, and do one commit (that will contain effectively only removed files).
3. There is a bug I fixed in 5.0.1 that might trigger this. Specifically the "Include Dependents" did not work (bug) and in 5.0.1, if you look at release notes, you will see that entities from different projects "[...] appear in the Entity Picker screen if the user enables the corresponding checkbox in the Export screen". From what I see, it might be that exporting these entities would be done in the Project folder - but I need to verify. If you don't need the 5.0.1 fixes, can you revert to 5.0.0 while I take a look at this (btw I don't guarantee when I'll do this).

[Frowne]

1. The correct commit procedure is option 1. The basic procedure to follow is that before each commit you need to clean, if needed, your working index

The working index being the export window's right table, right? So the list on the right should be completely empty before I add anything from the next project here?

2. What you can do now, to fix this, is to clean the working index of the latest commit, and do one commit (that will contain effectively only removed files).

Can I do this through your Git Interface (basically, deleting something through the file explorer and then push?

3. There is a bug I fixed in 5.0.1 that might trigger this. Specifically the "Include Dependents" did not work (bug) and in 5.0.1, if you look at release notes, you will see that entities from different projects "[...] appear in the Entity Picker screen if the user enables the corresponding checkbox in the Export screen". From what I see, it might be that exporting these entities would be done in the Project folder - but I need to verify.

That absolutely sounds like exactly what's happening - we're using version 5.0.0 and always used the "include dependents" option when exporting!

If you don't need the 5.0.1 fixes, can you revert to 5.0.0 while I take a look at this (btw I don't guarantee when I'll do this).

Sounds like we might need to upgrade to 5.0.1, haha! And absolutely no worries, maintaining a project like this sounds like a lot of work, take all the time you need! I'll be keeping an eye on the project and am excited for any updates! (:

[vrosu]

@Frowne just to be sure, are you using 5.0.0 or 5.0.1 ?
The working index is basically the current on-disk files snapshot corresponding to the currently checked commit (typically the last), that is not the export window right table. You'll see it in the main tab, under Workspace.

[Frowne]

@vrosu I've been using 5.0.0 as stated above - hence why I mentioned upgrading to 5.0.1 potentially!

So the current procedure would be to clean up the working index within the workspace tab, but now I'm confused about an earlier thing - is it not enough then to just "be careful" with the exports to make sure you're not exporting duplicates? Feels like it could be a bit of a hassle if I have to constantly fix the working index...

Or would that just be necessary if I do export poorly? Ultimately I think the main thing that I didn't realize is that I should be clearing out the window on the right from the screenshot above after selecting the second project to export and that pushed things into the wrong project potentially, does that sound about right?

So before commiting future stuff, I clean up the working index, commit that (probably delete the local repo on the receiving end) and then just export the projects one by one making sure there's no overlap in the right hand window during export.

Again, thank you so much for your help! It's much appreciated.