Merge branch 'main' into setup-keycloak-multifactor-authentication

importer/.gitignore

@@ -1 +1,47 @@
 config.py
+importer.log
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+.python-version
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+.idea/

importer/README.md

@@ -12,18 +12,37 @@ This script is used to setup keycloak roles and groups. It takes in a csv file w
 - `csv_file` : (Required) The csv file with the list of roles
 - `group` : (Not required) This is the actual group name. If not passed then the roles will just be created but not assigned to any group
 - `roles_max` : (Not required) This is the maximum number of roles to pull from the api. The default is set to 500. If the number of roles in your setup is more than this you will need to change this value
-- `defaultgroups` : (Not Required)
+- `default_groups` : (Not Required) This is a boolean value to turn on and off the assignment of default roles. The default value is `true`
 
 
 ### To run script
 1. Create virtualenv
 2. Install requirements.txt - `pip install -r requirements.txt`
-3. Create a `config.py` file. The `sample_config.py` is an example  of what this should look like. Populate it with the right credentials, you can either provide an access token or client credentials. Ensure that the user whose details you provide in this config file has the necessary permissions/privilleges.
-4. Run script - `python3 main.py --setup roles --csv_file csv/setup/roles.csv --group Supervisor --defaultgroups true`
+3. Set up your .env file, see sample below. Populate it with the right credentials, you can either provide an access token or client credentials. Ensure that the user whose details you provide in this config file has the necessary permissions/privileges.
+4. Run script - `python3 main.py --setup roles --csv_file csv/setup/roles.csv --group Supervisor`
 5. If you are running the script without `https` setup e.g locally or a server without https setup, you will need to set the `OAUTHLIB_INSECURE_TRANSPORT` environment variable to 1. For example `export OAUTHLIB_INSECURE_TRANSPORT=1 && python3 main.py --setup roles --csv_file csv/setup/roles.csv --group OpenSRP_Provider --log_level debug`
 6. You can turn on logging by passing a `--log_level` to the command line as `info`, `debug` or `error`. For example `python3 main.py --setup roles --csv_file csv/setup/roles.csv --group Supervisor --log_level debug`
 
 
+#### Sample .env file
+```
+client_id = 'example-client-id'
+client_secret = 'example-client-secret'
+fhir_base_url = 'https://example.smartregister.org/fhir'
+keycloak_url = 'https://keycloak.smartregister.org/auth'
+
+# access token for access to where product images are remotely stored
+product_access_token = 'example-product-access-token'
+
+# if using resource owner credentials (i.e importer handles getting authentication by itself)
+# This has greater precedence over the access and refresh tokens if supplied
+username = 'example-username'
+password = 'example-password'
+
+# if embedding importer into a service that already does the authentication
+access_token = 'example-access-token'
+```
+
 # FHIR Resource CSV Importer
 
 This script takes in a csv file with a list of resources, builds the payloads 
@@ -155,7 +174,7 @@ The coverage report `coverage.html` will be at the working directory
 - You can pass in a `list_resource_id` to be used as the identifier for the (reference) List resource, or you can leave it empty and a random uuid will be generated
 
 ### 12. Import JSON resources from file
-- Run `python3 main.py --bulk_import True --json_file tests/fhir_sample.json --chunk_size 500000 --sync sort --resources_count 100 --log_level info`
+- Run `python3 main.py --bulk_import True --json_file tests/json/sample.json --chunk_size 500000 --sync sort --resources_count 100 --log_level info`
 - This takes in a file with a JSON array, reads the resources from the array in the file and posts them to the FHIR server
 - `bulk_import` (Required) must be set to True
 - `json_file` (Required) points to the file with the json array. The resources in the array need to be separated by a single comma (no spaces) and the **"id"** must always be the first attribute in the resource object. This is what the code uses to identify the beginning and end of resources

importer/csv/users.csv

@@ -1,4 +1,4 @@
 firstName,lastName,username,email,userId,userType,enableUser,keycloakGroupId,keycloakGroupName,appId,password
-Jane,Doe,Janey,jdoe@example.com,,Practitioner,true,a715b562-27f2-432a-b1ba-e57db35e0f93,test,demo,pa$$word
-John,Doe,Johny,jodoe@example.com,,Practitioner,true,a715b562-27f2-432a-b1ba-e57db35e0f93,test,demo,pa$$word
-Jenn,Doe,Jenn,jendoe@example.com,99d54e3c-c26f-4500-a7f9-3f4cb788673f,Supervisor,false,a715b562-27f2-432a-b1ba-e57db35e0f93,test,demo,pa$$word
+Jane,Doe,janey,jdoe@example.com,,Practitioner,true,a715b562-27f2-432a-b1ba-e57db35e0f93,test,demo,pa$$word
+John,Doe,johny,jodoe@example.com,,Practitioner,true,a715b562-27f2-432a-b1ba-e57db35e0f93,test,demo,pa$$word
+Jenn,Doe,jenn,jendoe@example.com,99d54e3c-c26f-4500-a7f9-3f4cb788673f,Supervisor,false,a715b562-27f2-432a-b1ba-e57db35e0f93,test,demo,pa$$word