feat(common): json schema linting for common validation(s) (#473)

* feat(validation): add initiall validation json schema

* feat(common): create schema folder in common with schema functionality and tests for the validation schema

* feat(schema): add the validate method to run validation given a schema and a model

* feat(schema): schema tests passing, still need to fix other repo validations

* fix(schema): fix missing top level field not properly returning top level validation error

* chore(tests): fix validations used by tests, TODO: replace remote link with valid validation

* feat(schemas): worked more defined schema for domain

* feat(schemas): fix validation schema and test files relating

* fix(validation): while the documentation says name required if field and namespace required if name, none of our test validations follow this rule

* feat(schemas): validation schema provider and domain specs linked to their type enum
chore(makefile): add test-unit to makefile
chore(adr): update validation artifact format (resource-rule) required fields to match current usage and functionality
chore: update domains and provider types with comments from adr, docs, and schema

* feat(schemas): add back in the resource-rule constraints

* fix(adr): revert validation-artifact-format

* fix(schemas): resource-rule no longer flags name as required when field is null or empty

* test(compose_test): uncomment remote validations and test case

* fix(component_test): uncomment test case

* docs: fix missing optional comment in kubernetes-domain.md

* fix(validation-composition): reintroduced remote components to component-definition

* test(unit): fix composition compoent-definition-local-and-remote to have the remote validations reintroduced

* docs(common): add annotation about schema linting to UnmarshalYaml func

* refactor(schemas): removed the duplicative ExtractErrors method in favor of updated go-oscal implementation

* docs(reference): update the Validation reference readme

* fix(schemas): validation schema fixed create-resources, semver, and kubernetes-spec constraints
refactor(common): created linting method for validation, update ToLulaValidation to run lint, remove linting from validation.UnmarshalYaml

* docs: update reference/README.md

* chore: remove lula.schema.json

* feat(lint): add composition to lint that can be disabled using the -c flag

* chore(docs): add schema-updates.md

* refactor(cmd): rm composition from tools lint
chore(docs): update docs to reflect changes

* feat(cmd): add dev validate command.

---------

Co-authored-by: Brandt Keller <43887158+brandtkeller@users.noreply.github.com>

.github/pull_request_template.md

@@ -17,4 +17,5 @@ Relates to #
 ## Checklist before merging
 
 - [ ] Test, docs, adr added or updated as needed
+- [ ] [Schema Updates](https://github.com/defenseunicorns/lula/blob/main/docs/community-and-contribution/schema-updates.md) applied
 - [ ] [Contributor Guide Steps](https://github.com/defenseunicorns/lula/blob/main/CONTRIBUTING.md) followed

Makefile

@@ -18,6 +18,7 @@ CLI_VERSION ?= $(if $(shell git describe --tags),$(shell git describe --tags),"u
 
 # Go CLI options
 PKG         := ./...
+UNIT_PKG    := $(shell go list ./... | grep -v 'e2e')
 TAGS        :=
 TESTS       := .
 TESTFLAGS   := -race -v
@@ -63,6 +64,10 @@ $(BINDIR)/$(BINNAME): $(SRC)
 test: 
 	go clean -testcache && go test $(GOFLAGS) -run $(TESTS) $(PKG) $(TESTFLAGS)
 
+.PHONY: test-unit
+test-unit: # Run tests excluding those in the e2e folder.
+	go clean -testcache && go test $(GOFLAGS) -run $(TESTS) $(UNIT_PKG) $(TESTFLAGS)
+
 .PHONY: test-e2e
 test-e2e: 
 	cd src/test/e2e && go clean -testcache && go test $(GOFLAGS) -run $(TESTS) $(PKG) $(TESTFLAGS)
@@ -74,4 +79,3 @@ test-cmd:
 .PHONY: install
 install: ## Install binary to $INSTALL_PATH.
 	@install "$(BINDIR)/$(BINNAME)" "$(INSTALL_PATH)/$(BINNAME)"
-

docs/cli-commands/dev/lint.md

@@ -0,0 +1,30 @@
+# Lint Command
+
+The `lula dev lint` command is used to validate validation files against the schema. It can validate both local files and URLs.
+
+## Usage
+
+```bash
+lula dev lint -f <input-files> [-r <result-file>]
+```
+
+## Options
+
+- `-f, --input-files`: The paths to the validation files (comma-separated).
+- `-r, --result-file`: The path to the result file. If not specified, the validation results will be printed to the console.
+
+## Examples
+
+To lint existing validation files:
+```bash
+lula dev lint -f ./validation-file1.yaml,./validation-file2.yaml,https://example.com/validation-file3.yaml
+```
+
+To specify a result file:
+```bash
+lula dev lint -f ./validation-file1.yaml,./validation-file2.yaml -r validation-results.json
+```
+
+## Notes
+
+The validation results will be written to the specified result file. If there is at least one validation result that is not valid, the command will exit with a fatal error listing the files that failed linting.

docs/cli-commands/tools/compose.md

@@ -1,6 +1,6 @@
 # Compose Command
 
-The `compose` command is used to compose an OSCAL component definition. It is used to compose remote validations within a component definition in order to resolve any references for portability.
+The `lula tools compose` command is used to compose an OSCAL component definition. It is used to compose remote validations within a component definition in order to resolve any references for portability.
 
 ## Usage
 
@@ -11,7 +11,7 @@ lula tools compose -f <input-file> -o <output-file>
 ## Options
 
 - `-f, --input-file`: The path to the target OSCAL component definition.
-- `-o, --output-file`: The path to the output file. If not specified, the output file will be the original filename with `-composed` appended.
+- `-o, --output-file`: The path to the output file. If not specified, the output file will be the original filename with `-composed` appended (ie. `oscal-component.yaml` will be composed to `oscal-component-composed.yaml`).
 
 ## Examples
 

docs/cli-commands/tools/lint.md

@@ -0,0 +1,32 @@
+# Lint Command
+
+The `lula tools lint` command is used to validate OSCAL files against the OSCAL schema. It can validate both composed and non-composed OSCAL models.
+> **Note**: the `lint` command does not compose the OSCAL model.
+> If you want to validate a composed OSCAL model, you should use the [`lula tools compose`](../compose/README.md) command first.
+
+## Usage
+
+```bash
+lula tools lint -f <input-files> [-r <result-file>]
+```
+
+## Options
+
+- `-f, --input-files`: The paths to the tar get OSCAL files (comma-separated).
+- `-r, --result-file`: The path to the result file. If not specified, the validation results will be printed to the console.
+
+## Examples
+
+To lint existing OSCAL files:
+```bash
+lula tools lint -f ./oscal-component1.yaml,./oscal-component2.yaml
+```
+
+To specify a result file:
+```bash
+lula tools lint -f ./oscal-component1.yaml,./oscal-component2.yaml -r validation-results.json
+```
+
+## Notes
+
+If no input files are specified, an error will be returned. The validation results will be written to the specified result file. If no result file is specified, the validation results will be printed to the console. If there is at least one validation result that is not valid, the command will exit with a fatal error listing the files that failed linting.

docs/community-and-contribution/schema-updates.md

@@ -0,0 +1,3 @@
+# Schema Updates
+
+Any changes type changes effecting one of the schemas in `src/pkg/common/schemas` should be reflected in the relevant `types.go` file and vice versa. This will ensure that the schema is kept in sync with the Go type definitions.

docs/reference/README.md

@@ -1 +1,96 @@
-# Validation Reference
+# Validation Reference
+
+### Validation Struct
+
+The `Validation` struct is a data structure used for ingesting validation data. It contains the following fields:
+
+- `LulaVersion` (string): Optional field to maintain backward compatibility.
+- `Metadata` (*Metadata): Optional metadata containing the name and UUID of the validation.
+- `Provider` (*Provider): Required field specifying the provider and its corresponding specification.
+- `Domain` (*Domain): Required field specifying the domain and its corresponding specification.
+
+#### Metadata Struct
+
+The `Metadata` struct contains the following fields:
+
+- `Name` (string): Optional short description to use in the output of validations.
+- `UUID` (string): Optional UUID of the validation.
+
+#### Domain Struct
+
+The `Domain` struct contains the following fields:
+
+- `Type` (string): Required field specifying the type of domain (enum: `kubernetes`, `api`).
+- `KubernetesSpec` (*KubernetesSpec): Optional specification for a Kubernetes domain, required if type is `kubernetes`.
+- `ApiSpec` (*ApiSpec): Optional specification for an API domain, required if type is `api`.
+
+#### Provider Struct
+
+The `Provider` struct contains the following fields:
+
+- `Type` (string): Required field specifying the type of provider (enum: `opa`, `kyverno`).
+- `OpaSpec` (*OpaSpec): Optional specification for an OPA provider.
+- `KyvernoSpec` (*KyvernoSpec): Optional specification for a Kyverno provider.
+
+### Example YAML Document
+
+The following is an example of a YAML document for a validation artifact:
+```yaml
+lula-version: ">=v0.2.0"
+metadata:
+  name: Validate pods with label foo=bar
+  uuid: 123e4567-e89b-12d3-a456-426655440000
+domain:
+  type: kubernetes
+  kubernetes-spec:
+    resources:
+      - name: podsvt
+        resource-rule:
+          version: v1
+          resource: pods
+          namespaces: [validation-test]
+provider:
+  type: opa
+  opa-spec:
+    rego: |
+      package validate
+
+      import future.keywords.every
+
+      validate {
+        every pod in input.podsvt {
+          podLabel := pod.metadata.labels.foo
+          podLabel == "bar"
+        }
+      }
+```
+## Linting
+Linting is done by Lula when a `Validation` object is converted to a `LulaValidation` for evaluation.
+
+The `common.Validation.Lint` method is a convenience method to lint a `Validation` object. It performs the following step:
+
+1. **Marshalling**: The method marshals the `Validation` object into a YAML byte array using the `common.Validation.MarshalYaml` function.
+2. **Linting**: The method runs linting against the marshalled `Validation` object. This is done using the `schemas.Validate` function, which ensures that the YAML data conforms to the expected [schema](../../src/pkg/common/schemas/validation.json).
+
+___
+The `schemas.Validate` function is responsible for validating the provided data against a specified JSON schema using [github.com/santhosh-tekuri/jsonschema/v5](https://github.com/santhosh-tekuri/jsonschema). The process involves the following steps:
+
+1. **Coercion to JSON Map**: The provided data, which can be either an interface or a byte array, is coerced into a JSON map using the `model.CoerceToJsonMap` function.
+2. **Schema Retrieval**: The function retrieves the JSON schema specified by the `schema` parameter using the `GetSchema` function.
+3. **Schema Compilation**: The retrieved schema is compiled into a format that can be used for validation using the `jsonschema.CompileString` function.
+4. **Validation**: The coerced JSON map is validated against the compiled schema. If the validation fails, the function extracts the specific errors and returns them as a formatted string.
+
+## VS Code intellisense:
+1. Ensure that the [YAML (Red Hat)](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) extension is installed.
+2. Add the following to your settings.json:
+```json
+"yaml.schemas": {
+    "${PATH_TO_LULA}/lula/src/pkg/common/schemas/validation.json": "*validation*.yaml"
+},
+```
+
+
+> **Note:**
+> - `${PATH_TO_LULA}` should be replaced with your path.
+> - `*validation*.yaml` may be changed to match your project's validation file naming conventions.
+> - can also be limited to project or workspace settings if desired 

docs/reference/domains/kubernetes-domain.md

@@ -17,10 +17,10 @@ domain:
     - name: podsvt                      # Required - Identifier to be read by the policy
       resource-rule:                    # Required - resource selection criteria, at least one resource rule is required
         name:                           # Optional - Used to retrieve a specific resource in a single namespace
-        group:                          # Required - empty or "" for core group
+        group:                          # Optional - empty or "" for core group
         version: v1                     # Required - Version of resource
         resource: pods                  # Required - Resource type (API-recognized type, not Kind)
-        namespaces: [validation-test]   # Required - Namespaces to validate the above resources in. Empty or "" for all namespace pr non-namespaced resources
+        namespaces: [validation-test]   # Optional - Namespaces to validate the above resources in. Empty or "" for all namespace pr non-namespaced resources
         field:                          # Optional - Field to grab in a resource if it is in an unusable type, e.g., string json data. Must specify named resource to use.
           jsonpath:                     # Required - Jsonpath specifier of where to find the field from the top level object
           type:                         # Optional - Accepts "json" or "yaml". Default is "json".