Merge remote-tracking branch 'origin' into website

adr/0006-onschedule-implementation.md

@@ -0,0 +1,56 @@
+# 6: OnSchedule Implementation
+
+Date: 2023-10-16
+
+## Status
+
+Accepted
+
+## Context
+
+Zarf wrote a Pepr module to facilitate the development in [using ECR as an external registry during 'Zarf Init'](https://github.com/defenseunicorns/zarf/issues/1594). They are running into a challenge where the ECR Registry token is expiring every ~12 hours. Currently, they are having to maintain two completely different codebases, one being the Pepr module that communicates with AWS, and another being a Go app that they run as a `CronJob` to refresh the token. They are interested in rewriting the Go and scheduling logic in Pepr.
+
+We considered a few options:
+- setTimeout - Pepr is created with devex being a first class citizen and using a setTimeout doesn't cover persistence and stylistically different than the fluent api.
+- setInterval - " " 
+- using a `CronJob` - Forces the user to create a separate container image and maintain two codebases while they already have Pepr which should be able to handle this.
+- Use the crontab api - Provide a crontab string to run at the given time. Too big, didn't want to implement at this point.
+- Use the `OnSchedule` api - Provide a fluent api to run at the given time, with a completions argument.
+
+## Decision
+
+```typescript
+OnSchedule({
+  every: 5,
+  unit: "minutes",
+  startTime: "2023-10-16T00:00:00Z", // all dates times must be in GMT
+  run: () => {
+     // run some code
+  },
+  completions: 5,
+});
+```
+
+We decided to:
+
+Use `OnSchedule` to provide a fluent api to run at the given time, with a completions argument in the event that it needs to `RunAt` some given time and stop. This allows us to cover the functionality that something needs to be run on startup only once, like an IIFE.
+
+### Missed Jobs
+
+On missed (cluster restart, power outage) jobs we will run the code at the next scheduled interval and not count the failure against completions.
+
+### Why watch controller?
+
+The watch controller runs a single instance, this means we can avoid multiple controllers competing to run the same code.
+
+### Why `OnSchedule`?
+
+1. **Covers CronJob usecase**: We can cover the repeatability of a kubernetes native `CronJob` using `OnSchedule` to allow Zarf team to consoludate this functionlity into Pepr.
+2. **Backed by etcd**: We can mitigate the risk of Pepr being down by using etcd as the backing store for the `OnSchedule` API and warn the user that these `jobs` should be idempotent.
+
+
+## Consequences
+
+1. **Developer Experience**: Improved developer experience by extending the functionality provided in Pepr. Reduce the need to maintain a separate codebase to accomplish scheduled `job` type operations.
+2. **Test Stability**: Need to extend the internal tests to cover `OnSchedule`.
+3. **Documentation**: Existing documentation will need to be updated to reflect these changes.

docs/webassembly.md

@@ -1,10 +1,13 @@
-# WASM Support: Introducing WebAssembly to Pepr Guide
+# WASM Support: Running WebAssembly in Pepr Guide
+
+Pepr fully supports WebAssembly. Depending on the language used to generate the WASM, certain files can be too large to fit into a `Secret` or `ConfigMap`. Due to this limitation, users have the ability to incorporate `*.wasm` and any other essential files during the build phase, which are then embedded into the Pepr Controller container. This is achieved through adding an array of files to the `includedFiles` section under `pepr` in the `package.json`.
+
+> **NOTE -** In order to instantiate the WebAsembly module in TypeScript, you need the WebAssembly type. This is accomplished through add the "DOM" to the `lib` array in the `compilerOptions` section of the `tsconfig.json`. Ex: `"lib": ["ES2022", "DOM"]`. Be aware that adding the DOM will add a lot of extra types to your project and your developer experience will be impacted in terms of the intellisense. 
 
-Pepr now supports embedding `*.wasm` files into your Pepr modules during the build process. This is achieved through adding an array of files to the `includedFiles` section under `pepr` in the `package.json`.
 
 ## High-Level Overview
 
-At its core, WASM support is achieved through adding layers on top of the Pepr controller image accesible by the module. The key components of WASM support are:
+WASM support is achieved through adding files as layers atop the Pepr controller image, these files are then able to be read by the individual capabilities. The key components of WASM support are:
 
 - Add files to the **base** of the Pepr module.
 - Reference the files in the `includedFiles` section of the `pepr` block of the `package.json` 
@@ -68,7 +71,73 @@ if (typeof globalThis.crypto === 'undefined') {
 }
 ```
 
-### Calling WASM from the Capability
+
+### Configure Pepr to use WASM
+
+After adding the files to the root of the Pepr module, reference those files in the `package.json`:
+
+```json
+{
+  "name": "pepr-test-module",
+  "version": "0.0.1",
+  "description": "A test module for Pepr",
+  "keywords": [
+    "pepr",
+    "k8s",
+    "policy-engine",
+    "pepr-module",
+    "security"
+  ],
+  "engines": {
+    "node": ">=18.0.0"
+  },
+  "pepr": {
+    "name": "pepr-test-module",
+    "uuid": "static-test",
+    "onError": "ignore",
+    "alwaysIgnore": {
+      "namespaces": [],
+      "labels": []
+    },
+    "includedFiles":[
+      "main.wasm",
+      "wasm_exec.js"
+    ]
+  },
+  ...
+}
+```
+
+Update the `tsconfig.json` to add "DOM" to the `compilerOptions` lib:
+
+```json
+{
+  "compilerOptions": {
+    "allowSyntheticDefaultImports": true,
+    "declaration": true,
+    "declarationMap": true,
+    "emitDeclarationOnly": true,
+    "esModuleInterop": true,
+    "lib": [
+      "ES2022",
+      "DOM" // <- Add this
+    ],
+    "module": "CommonJS",
+    "moduleResolution": "node",
+    "outDir": "dist",
+    "resolveJsonModule": true,
+    "rootDir": ".",
+    "strict": false,
+    "target": "ES2022",
+    "useUnknownInCatchVariables": false
+  },
+  "include": [
+    "**/*.ts"
+  ]
+}
+```
+
+### Call WASM functions from TypeScript
 
 Import the `wasm_exec.js` in the `pepr.ts`
 
@@ -88,7 +157,7 @@ async function callWASM(a,b) {
   await WebAssembly.instantiate(wasmData, go.importObject).then(wasmModule => {
     go.run(wasmModule.instance);
 
-    concated = global.peprWasm(a,b);
+    concated = global.concats(a,b);
   });
   return concated;
 }
@@ -106,47 +175,10 @@ When(a.Pod)
 });
 ```
 
-### Updating the package.json
-
-After adding the files to the root (adjacent to `pepr.ts`) of the Pepr module, reference those files in the package.json:
-
-```json
-{
-  "name": "pepr-test-module",
-  "version": "0.0.1",
-  "description": "A test module for Pepr",
-  "keywords": [
-    "pepr",
-    "k8s",
-    "policy-engine",
-    "pepr-module",
-    "security"
-  ],
-  "engines": {
-    "node": ">=18.0.0"
-  },
-  "pepr": {
-    "name": "pepr-test-module",
-    "uuid": "static-test",
-    "onError": "ignore",
-    "alwaysIgnore": {
-      "namespaces": [],
-      "labels": []
-    },
-    "includedFiles":[
-      "main.wasm",
-      "wasm_exec.js"
-    ]
-  },
-  ...
-}
-```
-
 ### Run Pepr Build 
 
-Build your Pepr module with registry into specified.
+Build your Pepr module with the registry specified.
 
 ```bash
 npx pepr build -r docker.io/defenseunicorns
 ```
-

package.json

@@ -24,14 +24,14 @@
     "test:journey:build": "npm run build && npm pack",
     "test:journey:image": "docker buildx build --tag pepr:dev . && k3d image import pepr:dev -c pepr-dev",
     "test:journey:k3d": "k3d cluster delete pepr-dev && k3d cluster create pepr-dev --k3s-arg '--debug@server:0'",
-    "test:journey:run": "jest journey/entrypoint.test.ts",
-    "test:journey:run-wasm": "jest journey/entrypoint-wasm.test.ts",
+    "test:journey:run": "jest --detectOpenHandles journey/entrypoint.test.ts",
+    "test:journey:run-wasm": "jest --detectOpenHandles journey/entrypoint-wasm.test.ts",
     "test:unit": "npm run gen-data-json && jest src --coverage"
   },
   "dependencies": {
     "express": "4.18.2",
     "fast-json-patch": "3.1.1",
-    "kubernetes-fluent-client": "1.6.1",
+    "kubernetes-fluent-client": "1.7.0",
     "pino": "8.16.0",
     "pino-pretty": "10.2.3",
     "prom-client": "15.0.0",
@@ -41,13 +41,13 @@
     "@commitlint/cli": "17.8.0",
     "@commitlint/config-conventional": "17.8.0",
     "@jest/globals": "29.7.0",
-    "@types/eslint": "8.44.4",
-    "@types/express": "4.17.19",
+    "@types/eslint": "8.44.6",
+    "@types/express": "4.17.20",
     "@types/node": "18.x.x",
-    "@types/node-forge": "1.3.7",
-    "@types/prompts": "2.4.6",
-    "@types/ramda": "0.29.6",
-    "@types/uuid": "9.0.5",
+    "@types/node-forge": "1.3.8",
+    "@types/prompts": "2.4.7",
+    "@types/ramda": "0.29.7",
+    "@types/uuid": "9.0.6",
     "jest": "29.7.0",
     "nock": "13.3.4",
     "ts-jest": "29.1.1"

src/templates/tsconfig.module.json

@@ -5,7 +5,7 @@
     "declarationMap": true,
     "emitDeclarationOnly": true,
     "esModuleInterop": true,
-    "lib": ["ES2022", "DOM"],
+    "lib": ["ES2022"],
     "module": "CommonJS",
     "moduleResolution": "node",
     "outDir": "dist",