Atelier Products Service API

A complete redesign and optimization of a back end system (comprised of an express server and PostgreSQL database) to meet large quanitities of data and traffic.

This section focuses on modernization of the Atelier Products API.

API Endpoints:

• /products/:product_id
• /products/:product_id/styles
• /products/:product_id/related

Analytics & Testing:

Deployed Stress Testing

Goal: ~100 RPS with <1% Error Rate and Latency Rate of <2000 ms

• /products/:product_id had a latency rate of 1861 ms with 0% error rate at 1007.5 RPS
• /products/:product_id/styles had a latency rate of 1780 ms with 0% error rate at 431.8 RPS
• /products/:product_id/related had a latency rate of 1850 ms with 0% error rate at 1022.57 RPS

How to Run Loader.io Testing

1. Deploy Server and Database
2. Add Loader.io API Token
3. Run Tests on Loader.io Interface

Local Stress Testing

Goal: ~1000 RPS with <1% Error Rate and Latency Rate of <2000 ms

• /products/:product_id had a latency rate of 91.11 ms and 0% error rate at 1189.79 RPS
• /products/:product_id/styles had a latency rate of 1689.5 ms and 0% error rate at 1048.25 RPS
• /products/:product_id/related had a latency rate of 55.72 ms and 0% error rate at 1178.61 RPS
• Simultaneous calls to all API endpoints had a latency rate of 151.65 ms and 0% error rate at 1034.76 RPS

How to Run K6 Local Testing

• Test Each Endpoint

k6 run k6/endpoints/products.js

k6 run k6/endpoints/styles.js

k6 run k6/endpoints/related.js

• Test Combined Endpoints

k6 run k6/endpoints/all.js

API Query Response Times

Goal: <50 ms response time per query

Using indices within the PostgreSQL database, the goal was met and the execution times of each API endpoints decreased as follows:

• /products/:product_id by 99.73% (51.257 ms to 0.14 ms)
• /products/:product_id/styles by 99.99% (6155.382 ms to 0.411 ms)
• /products/:product_id/related by 99.99% (445.562 ms to 0.042 ms)

How to Test Query Times

psql -f server/db/query-times.sql

API functionality

All products API endpoints functionally tested with Postman Request

How to run Jest testing of API functionality

npm run test

Getting Started:

Installation

• Install initial dependencies

  npm install
  

• Create a local file .env using example.env as a template.
  • .env is listed in the .gitignore file and thus will not be added to Git's source control
  • Populate this new .env file with the required values for your local machine
• Configure ESLint rules and check code quality and syntax

  npm run eslint-dev
  

Start Server

npm run server-dev

Extract, Transform, Load (ETL)

• Download CSV files for features, photos, products, related, skus, and styles in server/data
• Transform data to correct format

  npm run etl
  

• Populate/Load database with the CSV files

  psql -f server/db/schema.sql