Testing APIs

API Testing

We can test our APIs with a useful tool called Postman. Postman allows you to organize your API requests in a 'Collection' that can be run either through the Postman interface or using a command line tool called Newman.

The Common Services Postman Collection contains many of the available common service API requests developed by our team.

A collection of API requests can make use of variables (for example GET http://my-endpoint/users/{{user_id}}). Variables can be defined in the collection itself, and a Postman Environment. Environment variables allow you to test your APIs in different environments (for example, dev, test, prod) and can be exported for use by the Newman runner.

Functional Tests

One of the main benefits of Postman/Newman collection running is the ability for us to ensure functional integrity at the API level. By being able to periodically run these tests, we can ensure that the service remain functional.

Designing Tests

All of our common services will include a postman collection located at /app/tests/postman/ in their respective repositories (eg: https://raw.githubusercontent.com/bcgov/common-hosted-email-service/master/app/tests/postman/postman_collection.json). This collection is utilized by our scheduled cron job to ensure that our services remain functional.

These collections should be kept up to date with any potential API features and changes. Should new API changes necessitate new variables, remember to update the newman-environment-secret contents in the appropriate OpenShift projects in addition to the example environment file.

Running Tests

Postman/Newman tests can be run on our OpenShift instances (such as CHES, CDOGS, etc) in a variety of ways. The main methods are to 1) Run directly in Postman application 2) Run via Newman Docker container or 3) Run on OpenShift via cronjob definition.

Postman Application

The Postman app includes a tool called 'Runner' that runs your collection (using environment variables already in postman) with one click. you can also set up a 'Monitor' to schedule runs, view results and get alerts. It takes seconds to set up. This is a good option for developers already familiar with Postman and want to inspect the API calls that are defined.

For more info on using Postman collections, see: https://learning.postman.com/docs/running-collections/intro-to-collection-runs/

Local Docker Container

The Newman run command references a Postman Collection (exported as a JSON file) out of Postman and located either in your local file system or by URL.

Postman provides an alpine-based container which allows you to run Postman collections via Docker command line. An example of how it can be invoked is as follows:

Note: If running on windows, you must either use CMD or Powershell; Mingw64 does not work correctly due to path interpolation issues.

docker run --rm -it postman/newman:alpine run https://raw.githubusercontent.com/bcgov/common-hosted-email-service/master/app/tests/postman/postman_collection.json

Running the above, you will get a mixture of successful and failed tests. This is because in order to run the CHES API, you will need to provide the Keycloak service_client_id and service_client_secret variables to the collection. These can be defined in the Postman environment template. See here for a an example environment file.

To provide an accompanying set of environment variables, reference your exported environment with the --environment flag. This file can also be located on your file system or accessed by URL.

To run the same collection in full including the authenticated requests, we pass in the environment variable file my_environment.json as a volume mounted into the container at /etc/newman.

docker run --rm -v C:\mount:/etc/newman -it postman/newman:alpine run https://raw.githubusercontent.com/bcgov/common-hosted-email-service/master/app/tests/postman/postman_collection.json --environment="my_environment.json"

See official docker page for more details.

Openshift Scheduled Tasks

The following steps will set up a cronjob that runs the Newman test on OpenShift.

Config Maps

The cron job will require a config map with a value named COLLECTION specifying what collection URL the container should run on. Replace the export variables with the appropriate values. For example, COLLECTIONURL could be https://raw.githubusercontent.com/bcgov/common-hosted-email-service/master/app/tests/postman/postman_collection.json.

export NAMESPACE=<YOURNAMESPACE>
export COLLECTIONURL=<COLLECTIONURL>

oc create -n $NAMESPACE configmap newman-config --from-literal=COLLECTION=$COLLECTIONURL

Secrets

The cron job will also require a secret which represents your Postman environment variables which are exported from Postman. An example template file can be seen here. Download the appropriate template and replace the values as necessary. For example, the export ENVFILE could be ./postman_environment.json

export NAMESPACE=<YOURNAMESPACE>
export ENVFILE=<ENVFILE>

oc create -n $NAMESPACE secret generic newman-environment-secret --from-file=ENVIRONMENT=$ENVFILE

Cron Job

Once you have your configmaps and secrets set up, you can run the following to deploy a cronjob to your project namespace. The raw OpenShift cronjob template can be inspected here. The SCHEDULE parameter is a schedule in unix cron job syntax. The example below targets daily execution at 1 AM local machine time.

export NAMESPACE=<YOURNAMESPACE>

oc process -n $NAMESPACE -f "https://raw.githubusercontent.com/wiki/bcgov/common-service-showcase/assets/newman.yaml" -p SCHEDULE="0 1 * * *" -o yaml | oc -n $NAMESPACE apply -f -

If deployed correctly, you will begin seeing pods running at 1 AM (or whatever you scheduled) in the project's pod list.

Manual Run

Using the schedule "0 1 * * *", the cronjob will run with a new pod at 1 AM daily. However, you can also manually run the cronjob controller with the following:

export NAMESPACE=<YOURNAMESPACE>

oc create -n $NAMESPACE job --from=cronjob/newman newman-manual-run

If your Postman tests all pass the pod run will successfully and show as 'Completed'. If the test fails, the pod will show with an 'error' status.

Make sure to clean up your temporary pod when you are done with the following:

export NAMESPACE=<YOURNAMESPACE>

oc delete -n $NAMESPACE job newman-manual-run

Cleanup

All jobs in a cron job can be deleted with:

export NAMESPACE=<YOURNAMESPACE>

oc delete -n $NAMESPACE cronjob/newman