IMSE 8410 Advanced Computational Systems and Data Engineering
Material Copyright 2017-2019 by Timothy Middelkoop. Source code licensed under the Apache License, Version 2.0. Documentation licensed under CC by SA 3.0.
"If it is not in git, it did not happen."
Managing a research effort and scientific or engineering workflows is a lot like managing a project. Many times a research effort will be directly linked to a sponsored (or sponsored) research project, or even a dissertation, with specific deliverables, milestones, and products that must be coordinated, managed, and with input (research, code, or simply feedback) from multiple individuals. Managing the code and the project are so often intertwined it makes a lot sense to use a tool such as GitLab or GitHub. In this course we use GitLab due to it's Open Source background and it's use at the University. Many of the tools are directly mapable to other project management and source code management systems (such as GitLab). By using these tools (and specifically GitLab) in a structured and systematic matter it is possible to manage the project, code, and people all in one location. It takes a bit of discipline to maintain this effort but is well worth it.
For all research and projects the first steps should be the following:
- Create a Git Project.
- Clone the repo on the cluster.
- Do some work
- Gather some collaborators (publish project)
- Create a Git Group (optional)
- Work as a Team
- Profit.
- Pro Git (https://git-scm.com/book/en/v2) Chapters 1 (except 1.5), 2, 3, 5, (6 Optional)
- GitLab Markdown https://docs.gitlab.com/ee/user/markdown.html
- GitLab Issues https://docs.gitlab.com/ee/user/project/issues/
- GitLab Issue Boards https://docs.gitlab.com/ee/user/project/issue_board.html
GitLab (https://gitlab.com) and local GitLab websites (https://gitlab.missouri.edu) contain a number of tools and capabilities, which are as follows:
- Projects: A Project is the basic unit for which most of the real
functionality is contained, divided by project. A project contains
the following elements:
- Git Repository (source and documentation)
- Issues
- Milestones
- Continuous Integration and Continuous Deployment.
- Documentation (Wiki) and Wiki's.
- Metrics
- Groups: A number of projects can be combined together as a group. Good for larger projects that may contain multiple repositories or larger groups such as research labs. Groups contain some of the capabilities as projects but applies to the group as a whole. Groups also allow projects to be abstracted from specific individuals (ownership of repositories).
- Access and Permission: Git allows for sharing of projects with different permissions to facilitate a large number of workflows.
Git is a distributed source control management system. It is used to manage source files, through changes, in a decentralized and distributed manner in a secure way (verifiable and integrity). At it's heart is just a series of "changes" with a tools to manage workflows. A git repository is a standalone database that contains the following concepts (covered in the text):
- Blobs (files)
- Commits (changes)
- Branches (trees)
- Files (checkout)
- Staging area (index)
- Tags (releases)
- Remotes (repositories)
Markdown is a human readable markup language that is easy to use and similar in style to programming. The Markdown format is visually similar to the rendered results and has eclipsed many of it's competitors in adoption due to it's simplicity, visual style, and ease of use. All documentation should be written in Markdown even if it is not going to be rendered. There are many extensions and flavors of Markdown, and in this class we will be using GitLab Markdown due to it's strong integration into the GitLab website. These extensions allow for easy cross referencing to issues, milestones, commits, etc. to allow for easier navigation through a project. The authoritative reference for GitLab flavored markdown can be found at the following website: https://docs.gitlab.com/ee/user/markdown.html
The purpose of this exercise is to familiarize yourself with basics of Git and GitLab and to provide a basic student profile. This exercise will also familiarize the student with how to submit exercises, assessments, and projects and how feedback will be provided.
The goal of this exercise is to write your profile in the ReadMe.md
if your course repository. Please consider the following points.
- Place the profile in the
ReadMe.md
file noting the upper/lower case of the file (notreadme.md
orREADME.md
). The profile should contain the following information:- Name:
- How you wish to be referred by:
- Department:
- Degree/Standing (B.S.+ M.S.; M.S.; Ph.D Student; Ph.D. Candidate; Faculty; Staff; Other)
- Degree focus area:
- About you (optional): something trivial/interesting about yourself.
- Include an image that is
- stored in the repository (uploaded), and
- preferably your profile picture.
- You may use the GitLab editor and edit on the 'master' branch.
- Post the repository URL (the
clone with ssh
URL under clone) and the long commit ID. Use theCopy Commit SHA
on the proper commit in the "commits
section of theRepository
section of your project. Please note that the example commit542720c4aa671f6267ff211d110cd81deb06d177
is a long commit id, and542720c4
is the short commit ID.
- GitLab https://gitlab.com
- GitHub https://github.com
- Git https://git-scm.com/book/en/v2
- GitLab Markdown https://docs.gitlab.com/ee/user/markdown.html
- Formal project management (PMBOK) https://www.pmi.org/pmbok-guide-standards