chore: update docs (#620)

* quick start Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com> * config Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com> * nits Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com> * tests Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com> * tests Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com> * schemas Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com> * schemas Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com> * office hour Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com> --------- Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com>
kyverno · Dec 8, 2023 · 7fa3ddc · 7fa3ddc
1 parent ae7e9bf
commit 7fa3ddc
Show file tree

Hide file tree

Showing 21 changed files with 551 additions and 551 deletions.
diff --git a/.release-notes/main.md b/.release-notes/main.md
@@ -6,10 +6,6 @@ Release notes for `TODO`.
 ## ⭐ Examples ⭐
 
 ## ⛵ Tutorials ⛵
-
-## 📚 Docs 📚
-
-## 🎸 Misc 🎸
 -->
 
 ## ‼️ Breaking chages ‼️
@@ -37,3 +33,13 @@ Release notes for `TODO`.
 
 - Use `@setup` and `@cleanup` in the logs to indicate setup/cleanup steps
 - Made resource in log messages easier to read
+
+## 📚 Docs 📚
+
+- Improved the *Writing tests/Configuration* docs
+- Improved the *Writing tests/Tests* docs
+- Added *Writing tests/JSON schemas* docs to use generated JSON schemas for validation and autocomplete in VS code
+
+## 🎸 Misc 🎸
+
+- [Nirmata Office Hours for Kyverno- Episode 9- Demonstrate Kyverno Chainsaw](https://www.youtube.com/watch?v=IrIteTTjlbU)
diff --git a/website/docs/configuration/cleanup-delay.md b/website/docs/configuration/cleanup-delay.md
@@ -0,0 +1,26 @@
+# Delay before cleanup
+
+At the end of each test Chainsaw will delete the resources it created during the test.
+
+When testing operators, it can be useful to wait a little bit before starting the cleanup process to make sure the operator/controller had the necessary time to update internal state.
+
+For this reason, Chainsaw provides the `delayBeforeCleanup` configuration option and the corresponding `--delay-before-cleanup` flag.
+
+## Configuration
+
+```yaml
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Configuration
+metadata:
+  name: custom-config
+spec:
+  # ...
+  delayBeforeCleanup: 5s
+  # ...
+```
+
+## Flag
+
+```bash
+$ chainsaw test --delay-before-cleanup 5s ...
+```
diff --git a/website/docs/configuration/file.md b/website/docs/configuration/file.md
@@ -0,0 +1,69 @@
+# Configuration file
+
+Chainsaw is described as a **_Stronger tool for e2e testing_**.
+
+With its versatile configuration options, you can customize the testing process to fit your needs.
+
+Chainsaw prioritizes its configuration in the following order:
+
+1. **User-specified configuration**
+
+    If you explicitly provide a configuration file using a command-line flag
+
+1. **Default configuration file**
+
+    If no configuration is specified, Chainsaw will look for a default file named `.chainsaw.yaml` in the current working directory
+
+1. **Internal default configuration**
+
+    In the absence of both the above, Chainsaw will use a [default configuration](#default-configuration) file embedded in the Chainsaw binary
+
+## Example
+
+```yaml
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Configuration
+metadata:
+  name: custom-config
+spec:
+  timeouts:
+    apply: 45s
+    assert: 20s
+    cleanup: 45s
+    delete: 25s
+    error: 10s
+    exec: 45s
+  skipDelete: false
+  failFast: true
+  parallel: 4
+  # ...
+```
+
+## How to specify a configuration
+
+To use a custom configuration file:
+
+```bash
+$ chainsaw test --config path/to/your/config.yaml
+```
+
+!!! note "Defaults"
+    If you don't specify any configuration, Chainsaw will look for the default configuration file `.chainsaw.yaml` in the current working directory.
+
+    If that file is not found, it will fall back to its internal [default configuration](#default-configuration).
+
+## Default configuration
+
+The default configuration below is used by Chainsaw when no configuration file was provided and the default file `.chainsaw.yaml` does not exist.
+
+```yaml
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Configuration
+metadata:
+  name: default
+spec: {}
+```
+
+## Reference documentation
+
+See [Configuration API reference](../apis/chainsaw.v1alpha1.md#chainsaw-kyverno-io-v1alpha1-Configuration) for more details.
diff --git a/website/docs/configuration/flags.md b/website/docs/configuration/flags.md
@@ -0,0 +1,30 @@
+# Command line flags
+
+Even after a [configuration file](./file.md) is loaded, you can override specific settings using command-line flags.
+
+## Example
+
+```bash
+$ chainsaw test                         \
+    --config path/to/your/config.yaml   \
+    --test-dir path/to/test/dir         \
+    --assert-timeout 45s                \
+    --skip-delete false                 \
+    --fail-fast true                    \
+    --parallel 4                        \
+    ...
+```
+
+In this example, Chainsaw will load a configuration file but the timeout configuration and other settings will be overridden by the values set in the flags, regardless of the value in the loaded configuration file.
+
+## Usage
+
+The command below will run tests using the configuration from `my-config.yaml`, taking tests from `/path/to/tests`, and running a maximum of `10` tests simultaneously.
+
+```bash
+$ chainsaw test --config my-config.yaml --test-dir /path/to/tests --parallel 10
+```
+
+## Reference documentation
+
+See [Chainsaw test command reference](../commands/chainsaw_test.md#options) for more details.
diff --git a/website/docs/configuration/grace.md b/website/docs/configuration/grace.md
@@ -0,0 +1,31 @@
+# Termination graceful period
+
+Some Kubernetes resources can take time before being stopped. For example, deleting a `Pod` can take time if the underlying container doesn't quit quickly enough.
+
+For this reason, Chainsaw provides the `forceTerminationGracePeriod` configuration option and the corresponding `--force-termination-grace-period` flag. If set, Chainsaw will override the `terminationGracePeriodSeconds` when working with the following resource kinds:
+
+- `Pod`
+- `Deployment`
+- `StatefulSet`
+- `DaemonSet`
+- `Job`
+- `CronJob`
+
+## Configuration
+
+```yaml
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Configuration
+metadata:
+  name: custom-config
+spec:
+  # ...
+  forceTerminationGracePeriod: 5s
+  # ...
+```
+
+## Flag
+
+```bash
+$ chainsaw test --force-termination-grace-period 5s ...
+```
diff --git a/website/docs/configuration/index.md b/website/docs/configuration/index.md
@@ -1,151 +1,21 @@
-# Configuration
+# Configuring Chainsaw
 
 Chainsaw is a comprehensive tool designed to facilitate **end to end testing in [Kubernetes](https://kubernetes.io)**.
 
 This documentation will focus on providing a breakdown of its configuration structure and how to use it.
 
 Chainsaw can be configured in two different and complementary ways:
 
-- [Using a configuration file](#configuration-file)
-- [Overriding configuration with command-line flags](#overriding-with-flags)
+- [Using a configuration file](./file.md)
+- [Overriding configuration with command-line flags](./flags.md)
 
-!!! tip
+!!! note "Precedence"
     If both are specified, **command-line flags will take precedence** over configuration coming from a configuration file.
 
-## Configuration file
+## Specific configuration options
 
-### Overview
+Please pay attention to the configuration options below, they may or may not be relevant in your case but can be useful in certain cases:
 
-Chainsaw is described as a **_Stronger tool for e2e testing_**.
-
-With its versatile configuration options, you can customize the testing process to fit your needs.
-
-!!! info "Configuration loading process"
-
-    Chainsaw prioritizes its configuration in the following order:
-
-    1. **User-specified configuration**: If you explicitly provide a configuration file using a command-line flag.
-    1. **Default configuration file**: If no configuration is specified, Chainsaw will look for a default file named `.chainsaw.yaml` in the current working directory.
-    1. **Internal default configuration**: In the absence of both the above, Chainsaw will use a [default configuration](#default-configuration) file embedded in the Chainsaw binary.
-
-### How to specify a configuration
-
-To use a custom configuration file:
-
-```bash
-chainsaw test --config path/to/your/config.yaml
-```
-
-If you don't specify any configuration, Chainsaw will look for the default configuration file `.chainsaw.yaml` in the current working directory.
-
-If that file is not found, it will fall back to its internal [default configuration](#default-configuration).
-
-### Timeouts
-
-Timeouts are specified per type of operation:
-
-- **Apply**    - when Chainsaw applies manifests in a cluster
-- **Assert**   - when Chainsaw validates resources in a cluster
-- **Error**    - when Chainsaw validates resources in a cluster
-- **Delete**   - when Chainsaw deletes resources from a cluster
-- **Cleanup**  - when Chainsaw removes resources from a cluster created for a test
-- **Exec**     - when Chainsaw executes arbitrary commands or scripts
-
-This is required because the timeout varies greatly depending on the nature of an operation.
-For example, applying a manifest in a cluster is expected to be reasonably fast, while validating a resource can be a long operation.
-
-!!! tip "Overriding timeouts"
-
-    Each timeout can be overridden at the test level, test step level, or individual operation level.
-
-    Timeouts defined in the `Configuration` are used when not overridden.
-
-    See the [chainsaw test command line flags](../commands/chainsaw_test.md#options) for default timeout values.
-
-### Example
-
-```yaml
-apiVersion: chainsaw.kyverno.io/v1alpha1
-kind: Configuration
-metadata:
-  name: custom-config
-spec:
-  timeouts:
-    apply: 45s
-    delete: 25s
-    assert: 20s
-    error: 10s
-    cleanup: 45s
-    exec: 45s
-  skipDelete: false
-  failFast: true
-  parallel: 4
-  // ....
-```
-
-The full structure of the configuration file is documented [here](../apis/chainsaw.v1alpha1.md#chainsaw-kyverno-io-v1alpha1-Configuration).
-
-### Default configuration
-
-The default configuration below is used by Chainsaw when no configuration file was provided and the default file `.chainsaw.yaml` does not exist.
-
-```yaml
-apiVersion: chainsaw.kyverno.io/v1alpha1
-kind: Configuration
-metadata:
-  name: default
-spec: {}
-```
-
-## Overriding with flags
-
-Even after a configuration is loaded, you can override specific settings using command-line flags:
-
-```bash
-chainsaw test                           \
-    --config path/to/your/config.yaml   \
-    --test-dir path/to/test/dir         \
-    --assert-timeout 45s                \
-    --skip-delete false                 \
-    --fail-fast true                    \
-    --parallel 4                        \
-    ...
-```
-
-In this example, Chainsaw will load a configuration file but the timeout configuration and other settings will be overridden by the values set in the flags, regardless of the value in the loaded configuration file.
-
-Supported command line flags are documented [here](../commands/chainsaw_test.md#options).
-
-## Usage example
-
-```bash
-chainsaw test --config my-config.yaml --test-dir /path/to/tests --parallel 10
-```
-
-This command will run tests using the configuration from `my-config.yaml`, taking tests from `/path/to/tests`, and running a maximum of `10` tests simultaneously.
-
-## Termination graceful period
-
-Some Kubernetes resources can take time before being stopped. For example, deleting a `Pod` can take time if the underlying container doesn't quit quickly enough.
-
-Chainsaw provides the `forceTerminationGracePeriod` configuration option to override the `terminationGracePeriodSeconds` when working with the following resource kinds:
-
-- `Pod`
-- `Deployment`
-- `StatefulSet`
-- `DaemonSet`
-- `Job`
-- `CronJob`
-
-## Reports
-
-Chainsaw can generate JUnit reports in `XML` or `JSON` format.
-
-Provide a report format and report name in the configuration or via CLI flags.
-
-## Reference
-
-Refer to the reference documentations for details about supported fields in the configuration file and/or supported flags in the `test` command.
-
-- [Configuration API reference](../apis/chainsaw.v1alpha1.md#chainsaw-kyverno-io-v1alpha1-Configuration)
-- [Chainsaw test command reference](../commands/chainsaw_test.md#options)
+- [Timeouts](./timeouts.md)
+- [Termination graceful period](./grace.md)
+- [Cleanup before delay](./cleanup-delay.md)
diff --git a/website/docs/configuration/reports.md b/website/docs/configuration/reports.md
@@ -0,0 +1,25 @@
+# Reports
+
+Chainsaw can generate JUnit reports in `XML` or `JSON` format.
+
+To produce a test report, configure the report format and report name in the configuration or using CLI flags.
+
+## Configuration
+
+```yaml
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Configuration
+metadata:
+  name: custom-config
+spec:
+  # ...
+  reportFormat: JSON
+  reportName: chainsaw-report.json
+  # ...
+```
+
+## Flag
+
+```bash
+$ chainsaw test --report-format JSON --report-name chainsaw-report.json ...
+```