|
|
|
|
|
|
|
|
|
|
The program it allows you to clone repositories from github in bulk and store them in specific directories from a csv file. Aditionally it saves the data of every student & course into a json with the name of the course.
All this is possible by the use of Pandas library and dataframes to manipulate the data, sorting and filtering the courses, students and repositories to get a list of dataframes, one for each course with all the students data sorted by course, surname and name and then, it will create a Pie Chart, using Matplotlib Pyplot, with the percentages of students categorized according to their course whose repositories have been cloned.
At the end of the execution, the program will download the files of every student and save them in the directory of the course that they belong to.
like this:
| Courses & students directories |
|---|
|
And the JSON generates with the data of the students and courses will be like this:
[
{
"Temporal Sign":"2022\/02\/13 10:26:52 p.\u00a0m. GMT-3",
"Name":"Heracles",
"Surname":"Grecian DemiGod",
"Division":"1A - Professor 1 - Helper 3",
"Student ID":"999999",
"E-Mail":"Heracles@nordicreign.com",
"Link to repository":"https:\/\/github.com\/caidevOficial\/CaidevOficial"
},
{
"Temporal Sign":"2022\/02\/13 10:26:52 p.\u00a0m. GMT-3",
"Name":"Hades",
"Surname":"Grecian God",
"Division":"1A - Professor 1 - Helper 3",
"Student ID":"111111",
"E-Mail":"Hades@underworld.com",
"Link to repository":"https:\/\/github.com\/caidevOficial\/CaidevOficial.git"
},
{
"Temporal Sign":"2022\/02\/13 10:26:52 p.\u00a0m. GMT-3",
"Name":"Odin",
"Surname":"Nordic God",
"Division":"1A - Professor 1 - Helper 3",
"Student ID":"777777",
"E-Mail":"odin@fatherofall.com",
"Link to repository":"https:\/\/github.com\/caidevOficial\/Python_RepositoryCloner.git"
},
{
"Temporal Sign":"2022\/02\/13 10:26:52 p.\u00a0m. GMT-3",
"Name":"Valhalla",
"Surname":"Nordic Reign",
"Division":"1A - Professor 1 - Helper 3",
"Student ID":"999999",
"E-Mail":"valhalla@nordicreign.com",
"Link to repository":"https:\/\/github.com\/caidevOficial\/CaidevOficial"
}
]Meanwhile the program is cloning the repositories, the console will show messages like showns below:
| Console Messages |
|---|
|
When finish, you look a final message (with the elapsed time of the execution) like this:
| Console Final Message |
|---|
|
At the end of the execution, the program will download the files of every student and save them in the directory of the course that they belong to. Additionally, the program will generate a JSON with the data of the students and courses and it will generate a Pie Chart with the percentage of students that have downloaded the repositories...
Like the image below:
| Example Pie Chart |
|---|
|
- 1st Column: Date time. [it isn't used yet.]
- 2nd Column: Student Name
- 3rd Column: Student Surname
- 4th Column: Student Division
- 5th Column: Student ID. [it isn't used yet.]
- 6th Column: Student E-mail. [it isn't used yet.]
- 7th Column: Repository Name to download (It could skip the '.git' part)
Like this:
"Marca temporal","Nombre/s","Apellido/s","División","DNI / Legajo","E-Mail","Link al repositorio"
"2022/02/13 10:26:52 p. m. GMT-3","Neptune","Romane God","1G - Professor 1 - Helper 1","222222","neptune@notplanet.com","https://github.com/caidevOficial/SPD2022_TPS.git"
"2022/02/13 10:26:52 p. m. GMT-3","Poseidon","Grecian God","1F - Professor 2 - Helper 2","333333","poseidon@sea.com","https://github.com/caidevOficial/Python_ITBA_IEEE.git"
"2022/02/13 10:26:52 p. m. GMT-3","Hades","Grecian God","1F - Professor 2 - Helper 2","111111","Hades@underworld.com","https://github.com/caidevOficial/CaidevOficial.git"
In order to use this Cloner, you should configure the file API_Info.json with your Github API's information as shown below.
[
"Github": {
"URL": "https://api.github.com/repos",
"USER": "YOUR_GITHUB_USER",
"REPO": "YOUR_REPOSITORY_NAME",
"BRANCH": "THE_PRINCIPAL_BRANCH_NAME"
},
"DataFrame": {
"Fields": {
"Date": "First_Datetime_Field_To_Delete",
"Name": "Name_For_Column_Of_Names",
"Surname": "Name_For_Column_Of_Surnames",
"Course": "Name_For_Column_Of_Courses",
"ID": "Name_For_Column_Of_Students_ID",
"Email": "Name_For_Column_Of_Emails",
"GitLink": "Name_For_Column_Of_Links_To_Repositories"
}
},
"Files": {
"Dir_Plots_img": "./DIR_FOR_PLOTS_IMAGES",
"Dir_Cloned_Repos": "./DIR_FOR_CLONED_REPOSITORIES",
}
]for example:
[
{
"URL": "https://api.github.com/repos",
"USER": "CaidevOficial",
"REPO": "Python_RepositoryCloner",
"BRANCH": "main"
},
"DataFrame": {
"Fields": {
"Date": "Marca temporal",
"Name": "Nombre/s",
"Surname": "Apellido/s",
"Course": "División",
"ID": "DNI / Legajo",
"Email": "E-Mail",
"GitLink": "Link al repositorio"
}
},
"Files": {
"Dir_Plots_img": "./Plot_Images",
"Dir_Cloned_Repos": "./Repositories"
}
]Then the code will make the link like:
https://api.github.com/repos/CaidevOficial/Python_RepositoryCloner/commits/main
This way the program will take the 'Date' of the last commit of the branch 'main' and will use it to create the folder with the name of the repository. Obviously, the repository MUST BE PUBLIC, otherwise the program won't be able to access its API.
Regarding the 'DataFrame' Key, al the keys inside are configured to use them with a 'csv' file with at least theses columns. [Could have more columns, but it's not necessary for us.]
Finally, respect the 'Files' Key, where you can configure the directory where the plots will be saved and the directory where the cloned repositories will be saved.
For our example, the columns of the csv file are:
| Marca Temporal | Nombre/s | Apellido/s | División | DNI / Legajo | Link al repositorio | |
|---|---|---|---|---|---|---|
|
Python |
|
|
Pandas |
|
|
Numpy |
|
MatPlotLib |
|
VSCode |
| LICENSE | |
|---|---|
| Git Repository Cloner 2022 | |
| License | Author |
| [GNU General Public License V3] | [Facundo Falcone - CaidevOficial] |
| This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see GNU Licenses. | |
|
| 🤴 Facu Falcone - Data Engineer |
|
|
|
|