Skip to content

Commit

Permalink
feat(backend): webhook max retry (#2541)
Browse files Browse the repository at this point in the history 
* feat(backend): webhook max retry

* fix: tighten filter

* Update packages/documentation/src/content/docs/integration/deployment.md

Co-authored-by: Max Kurapov <max@interledger.org>

* test: address review feedback

* chore: add feature requests to contributing.md (#2542)

* chore: enable graphql protection - maxDepth, blockFieldSuggestions, maxCost (#2537)

* chore: add graphql depth restriction

* feat: properly enable graphql armor

* style: remove unnecessary space

* chore: add armor to auth admin server

* fix(deps): update module github.com/aws/aws-sdk-go to v1.50.38 (#2543)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency @interledger/open-payments to v6.7.2 (#2544)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(frontend): add X-Frame-Options header (#2538)

* chore(deps): update dependency @types/lodash to ^4.17.0 (#2546)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update dependency @types/react to ^18.2.66 (#2545)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency @interledger/open-payments to v6.8.0 (#2547)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update module github.com/aws/aws-sdk-go to v1.51.0 (#2548)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency @interledger/docs-design-system to ^0.3.2 (#2549)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency axios to v1.6.8 (#2556)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency isbot to v5 (#2553)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update typescript-eslint monorepo to v7 (#2552)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency koa to ^2.15.1 (#2551)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update dependency postcss to ^8.4.36 (#2559)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update helm release redis to v18.19.3 (#2561)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* feat(bruno): use polling in grant continuation (#2550)

* chore: fetch latest OP schemas

* feat(bruno): use polling for grant continuation

* docs: update

* fix: bruno tests

* chore(deps): update pnpm to v8.15.5 (#2563)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* feat(telemetry): LIVENET now points to new livenet NLB (#2523)

* feat(telemetry): LIVENET now points to new livenet NLB

* Update packages/backend/src/config/app.ts

Co-authored-by: Max Kurapov <max@interledger.org>

* Update packages/backend/src/config/app.ts

Co-authored-by: Max Kurapov <max@interledger.org>

---------

Co-authored-by: Max Kurapov <max@interledger.org>
Co-authored-by: Sabine Schaller <sabine@interledger.org>

* feat(auth): improve GNAP error responses (#2400)

* feat(auth): improve GNAP error responses

* fix: add extra arg to ctx.throw

* feat: middleware error improvements

* feat: add error messages to grant details endpoint

* fix: update error messages in grant routes

* feat: create utility function to generate GNAP errors, create enum for GNAP error codes

* feat: helper function throws instead of generates response

* feat: integration tests (#2380)

* feat: setup basic test env

* feat: seed integration environment

- cmd to start integration environment and run tests
- seeds environment on test run
- extracts common MASE functionality into new mock-account-servicing-lib

* refactor(localenv,integration): move graphql url to config from seed

- moves graphql url to config from seed
- also removes self section which only contained unused properties

* refactor: move testenv configuration into new dir

* refactor: cleanup env vars

* feat: add webhook server, fix env vars

* chore: comment

* feat: start grant request test

* fix: eslint errors

* fix: docker compose env vars

* feat: add --build arg

* fix: incoming-payment grant initiation request

Grant request was failing with invalid signature. This was fixed by directing hte backend to use the same private key being used to create the open payments client.

* fix: ts error

* feat: implement tests through Create Quote

* chore: upgrade op client

* feat: partially implemented grant request outgoing payment test

* feat: rework to host.docker.internal

- switches hostname to host.docker.internal to resolve url mismatch problem
- finishes grant request outgoing payment test
- cleans up some configuration and test variables

* feat: add p2p flow test

* feat: add continuation step with consent mocking

* fix: rm obsolete type cast to any and comment

* feat: add create ougoing payment test

* fix: bad pnpm-lock merge

* feat: build deps in mock ase job

* feat: get non existant wallet address test

* fix: update open payments pkg

* chore: fix lint warnings

* feat: implement continuation polling

* chore: test cleanup

* refactor: generate gql in tests instead of import from lib

* chore: rm old comment

* chore: use latest http-singature-utils to match other deps

* chore: pnpm i

* chore: use latest koa-bodyparser

* chore: rm engine strict

* chore: revert rm engine strict

- fixes netlify ci failure but ultimately not the correct fix

* chore(integration): dont ignore env, rm example env

* refactor: use docker healthcheck for running tests

* chore: move healthcheck to last started docker container

* feat: use latest open payments pkg, no body requird on continuation

* chore: pnpm i, fix broken lockfile (no apollo version)

* refactor: move webhook event enum to types

* refactor: move run integration script to test/integration

* Update packages/mock-account-servicing-lib/package.json

Co-authored-by: Sabine Schaller <sabine@interledger.org>

* refactor: simplify mock-account-servicing-entity ci step

* feat(integration): exit run-tests early if containers fail to start

* feat: use pino logger

* refactor: rename mock account servicing lib

* refactor: rename class files to camel case

* fix: import correct body parser

* refactor: poll instead of delay

* refactor: simplify call to poll condition only

* fix: update filenames

* refactor: change filename name to kebab case

---------

Co-authored-by: Sabine Schaller <sabine@interledger.org>

* fix(deps): update dependency isbot to ^5.1.2 (#2567)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update dependency @types/react to ^18.2.67 (#2566)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update module github.com/aws/aws-sdk-go to v1.51.2 (#2565)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update helm release redis to v18.19.4 (#2570)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* feat(backend): make SPSP optional (#2560)

* feat: flag to enable/disable SPSP

* test: update to cover all cases

* docs: add link to glossary

* chore: fix lockfile

* test: address feedback

* delete lockfile

* add lockfile

* delete lockfile

* fix lockfile this time?

* fix: formatting

---------

Co-authored-by: Max Kurapov <max@interledger.org>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: xplicit <111863110+beniaminmunteanu@users.noreply.github.com>
Co-authored-by: Nathan Lie <lie4nathan@gmail.com>
Co-authored-by: Blair Currey <12960453+BlairCurrey@users.noreply.github.com>
  • Loading branch information
@sabineschaller @mkurapov
@renovate @beniaminmunteanu @njlie @BlairCurrey
6 people authored Mar 19, 2024
1 parent f089930 commit b124abd
Show file tree
Hide file tree
Showing 4 changed files with 32 additions and 2 deletions.
1 change: 1 addition & 0 deletions packages/backend/src/config/app.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ export const Config = {
webhookWorkerIdle: envInt('WEBHOOK_WORKER_IDLE', 200), // milliseconds
webhookUrl: envString('WEBHOOK_URL', 'http://127.0.0.1:4001/webhook'),
webhookTimeout: envInt('WEBHOOK_TIMEOUT', 2000), // milliseconds
webhookMaxRetry: envInt('WEBHOOK_MAX_RETRY', 10),

withdrawalThrottleDelay:
process.env.WITHDRAWAL_THROTTLE_DELAY == null
Expand Down
30 changes: 29 additions & 1 deletion packages/backend/src/webhook/service.test.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ describe('Webhook Service', (): void => {

beforeAll(async (): Promise<void> => {
Config.signatureSecret = WEBHOOK_SECRET
Config.webhookMaxRetry = 10
deps = await initIocContainer(Config)
appContainer = await createTestApp(deps)
knex = appContainer.knex
Expand Down Expand Up @@ -297,7 +298,19 @@ describe('Webhook Service', (): void => {
})
})

describe.skip('processNext', (): void => {
describe('processNext', (): void => {
beforeEach(async (): Promise<void> => {
event = await WebhookEvent.query(knex).insertAndFetch({
id: uuid(),
type: WalletAddressEventType.WalletAddressNotFound,
data: {
account: {
id: uuid()
}
}
})
})

function mockWebhookServer(status = 200): Scope {
return nock(webhookUrl.origin)
.post(webhookUrl.pathname, function (this: Definition, body) {
Expand Down Expand Up @@ -375,5 +388,20 @@ describe('Webhook Service', (): void => {
event.createdAt.getTime() + RETRY_BACKOFF_MS
)
})

test('Does not send event if webhookMaxAttempts is reached', async (): Promise<void> => {
let requests = 0
nock(webhookUrl.origin)
.post('/')
.reply(200, () => {
requests++
})
await event.$query(knex).patch({
attempts: Config.webhookMaxRetry
})
await expect(webhookService.getEvent(event.id)).resolves.toEqual(event)
await expect(webhookService.processNext()).resolves.toBeUndefined()
expect(requests).toBe(0)
})
})
})
2 changes: 1 addition & 1 deletion packages/backend/src/webhook/service.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ async function processNextWebhookEvent(
.where('processAt', '<=', new Date(now).toISOString())

const event = events[0]
if (!event) return
if (!event || event.attempts >= deps_.config.webhookMaxRetry) return

const deps = {
...deps_,
Expand Down
1 change: 1 addition & 0 deletions packages/documentation/src/content/docs/integration/deployment.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ Now, the Admin UI can be found on localhost:3010.
| `WALLET_ADDRESS_URL` | backend.serviceUrls.WALLET_ADDRESS_URL | `http://127.0.0.1:3001/.well-known/pay` | this Rafiki instance's internal wallet address |
| `WALLET_ADDRESS_WORKERS` | backend.workers.walletAddress | `1` | number of workers processing wallet address requests |
| `WALLET_ADDRESS_WORKER_IDLE` | backend.workerIdle | `200` | time in milliseconds that WALLET_ADDRESS_WORKERS will wait until they check an empty wallet address request queue again |
| `WEBHOOK_MAX_RETRY` | backend.webhookMaxRetry | `10` | maximum number of times Rafiki backend retries sending a certain webhook event to the configured WEBHOOK_URL |
| `WEBHOOK_TIMEOUT` | backend.lifetime.webhook | `2000` | milliseconds |
| `WEBHOOK_URL` | backend.serviceUrls.WEBHOOK_URL | `http://127.0.0.1:4001/webhook` | endpoint on the Account Servicing Entity that consumes webhook events |
| `WEBHOOK_WORKERS` | backend.workers.webhook | `1` | number of workers processing webhook events |
Expand Down

0 comments on commit b124abd

Please sign in to comment.