additions_to_repos_workflow_on_20230424

Testing git

NOTE: I SHOULD MERGE THIS DOCUMENT WITH THE EXISTING ONE AT https://github.com/ncats/platform-capability-understanding/wiki/nidap_dashboards_repo_organization!!!!

Also todo: Adapt generic workflow to one on NIDAP where we don't want the history to be preserved. This should be very straightforward from the following!

Setting working directory to $WORKING_DIR using WORKING_DIR=/home/weismanal/notebook/2023-04-24/testing_git, which is where I developed and testing the following procedures.

Setup

# Only run this if the working directory already exists and you want to start from scratch
cd $HOME
rm -rf $WORKING_DIR

# Create the working directory and subdirectories
mkdir $WORKING_DIR
mkdir $WORKING_DIR/{bare_repos,clones}
mkdir $WORKING_DIR/bare_repos/{with_history,without_history}

# Create the bare repositories
cd $WORKING_DIR/bare_repos/with_history
git init --bare
cd $WORKING_DIR/bare_repos/without_history
git init --bare

# Initialize the "with_history" bare repository with a dummy file
git clone $WORKING_DIR/bare_repos/with_history $WORKING_DIR/clones/populating_local_with_history
cd $WORKING_DIR/clones/populating_local_with_history
echo "this is a test file" > bleh.txt
git add .
git commit -m "Initial commit with bleh.txt"
git push

# Clone the main repository and add remotes to the local repositories
git clone git@github.com:ncats/spatial-interaction-tool.git $WORKING_DIR/clones/main_clone
cd $WORKING_DIR/clones/main_clone
git remote add local_without_history $WORKING_DIR/bare_repos/without_history
git remote add local_with_history $WORKING_DIR/bare_repos/with_history

Generic workflow

Push the app-dev-nidap branch to the main branch of a remote repository that has zero history

This is the less frequent workflow because on NIDAP, unlike on GitHub, a new, "empty" repository is never really empty; it has a history and contains template files.

# Checkout the app-dev-nidap branch in the main repository and try pushing it to the local repository $WORKING_DIR/bare_repos/without_history
cd $WORKING_DIR/clones/main_clone
git checkout app-dev-nidap
# git push local_without_history main  # this pushes the main branch to the main branch of the remote repository
# git push local_without_history  # this pushes the local branch to the same-named branch of the remote repository
git push local_without_history app-dev-nidap:main  # this correctly pushes the local branch to the main branch of the remote repository

# Clone that local repository in order to see what happened
cd $WORKING_DIR/clones
git clone $WORKING_DIR/bare_repos/without_history clone_of_local_without_history
cd clone_of_local_without_history

Push the app-dev-nidap branch to the main branch of a remote repository that has some history

This is the more common workflow on NIDAP.

# Checkout the app-dev-nidap branch in the main repository
cd $WORKING_DIR/clones/main_clone
git checkout app-dev-nidap

# Copy in the template files from the remote (assumption is remote history is unimportant whereas the local histiory is) and overwrite the remote history accordingly
cd $WORKING_DIR/clones
git clone $WORKING_DIR/bare_repos/with_history getting_template_files
cp -rp getting_template_files/* main_clone/
cd $WORKING_DIR/clones/main_clone
git add .
git commit -m "Add template files from remote"
git push -f local_with_history app-dev-nidap:main
rm -rf $WORKING_DIR/clones/getting_template_files

# Clone that local repository in order to see what happened
cd $WORKING_DIR/clones
git clone $WORKING_DIR/bare_repos/with_history clone_of_local_with_history
cd clone_of_local_with_history

Subsequent development for both workflows above

# Push changes from local to remote
git push REMOTE app-dev-nidap:main  # can likely substitute HEAD for app-dev-nidap so you don't have to know what branch you're in per the git push help, but this is more explicit

# Pull changes from remote to local
git checkout app-dev-nidap
git pull REMOTE main

Drop the history to start fresh with the current working files

This could be a viable workflow on NIDAP when you want to start fresh with no history.

# Replace the main clone with history with the templates repo on NIDAP
rm -rf $WORKING_DIR/clones/main_clone
cd $WORKING_DIR/clones
git clone $WORKING_DIR/bare_repos/with_history main_clone
cd $WORKING_DIR/clones/main_clone
git checkout -b app-dev-nidap

# Copy in the current working files (assumption is no history is important and we just want the working files)
cd $WORKING_DIR/clones
git clone git@github.com:ncats/spatial-interaction-tool.git getting_current_working_files
cd getting_current_working_files
git checkout app-dev  # this is the latest development branch
cp -rp * $WORKING_DIR/clones/main_clone

# Commit the template repo, which is soon to be the new main repo, containing the current working files
cd $WORKING_DIR/clones/main_clone
git add .
git commit -m "Add working files to template repo which will be the main repo going forward"
git push origin app-dev-nidap:main
rm -rf $WORKING_DIR/clones/getting_current_working_files

# Clone that remote once more in order to freshly see what happened
cd $WORKING_DIR/clones
git clone $WORKING_DIR/bare_repos/with_history main_clone_just_a_check
cd main_clone_just_a_check

Subsequent development for this workflow:

# Push changes from local to remote
git push origin app-dev-nidap:main  # can likely substitute HEAD for app-dev-nidap so you don't have to know what branch you're in per the git push help, but this is more explicit

# Pull changes from remote to local
git checkout app-dev-nidap
git pull origin main

NIDAP workflow

This is adapted from the generic workflow above.

Push a current codebase to NIDAP preserving history

• Create a new Code Workspaces repository on NIDAP, calling the remote below nidap_remote.
• In a local clone of the codebase (latest branch you want to transfer, such as app-dev), do the following:

# Create a new app-dev-nidap2 branch in the main repository based on the app-dev branch
cd $WORKING_DIR/clones/main_clone
git checkout app-dev
git checkout -b app-dev-nidap2

# Add the NIDAP remote to the main clone
git remote add nidap_remote NIDAP-REMOTE-URL

# Copy in the template files from the remote (assumption is remote history is unimportant whereas the local histiory is) and overwrite the remote history accordingly
cd $WORKING_DIR/clones
git clone NIDAP-REMOTE-URL getting_template_files
for dir in $(find getting_template_files/ -type d | grep -v "\.git" | tail -n +2); do new_dir=$(echo $dir | awk '{sub("getting_template_files/", "repo/"); print}'); mkdir $new_dir; done
for file in $(find getting_template_files/ -type f | grep -v "\.git/"); do new_file=$(echo $file | awk '{sub("getting_template_files/", "repo/"); print}'); cp $file $new_file; done  # note already existing files like .gitignore may have to be merged manually
cd $WORKING_DIR/clones/main_clone
git add .
git commit -m "Add template files from NIDAP's Code Workspaces repository"
git push -f nidap_remote app-dev-nidap2:master  # note this is how you overwrite history fully, e.g., may need this for a new NCATS GitHub repo when adding that remote
rm -rf $WORKING_DIR/clones/getting_template_files

• Explore the repository on NIDAP to ensure the current working files are merged with the template files and all history is preserved.
• Subsequent development for this workflow:

# Push changes from local to remote
git push nidap_remote app-dev-nidap2:master  # can likely substitute HEAD for app-dev-nidap2 so you don't have to know what branch you're in per the git push help, but this is more explicit

# Pull changes from remote to local
git checkout app-dev-nidap2
git pull nidap_remote master