Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initial lula report #599

Draft
wants to merge 25 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
95568b1
initial command
CloudBeard Aug 5, 2024
3f7bb0c
initial command
CloudBeard Aug 9, 2024
d1543fb
initial report - still wip
CloudBeard Aug 12, 2024
0ea6bde
still wip, need to pull controls from catalog and profile to do math …
CloudBeard Aug 14, 2024
8027410
added report doc
CloudBeard Aug 19, 2024
b10e5ff
refactored to test main function, added test and test data
CloudBeard Sep 18, 2024
bba7581
Merge branch 'main' into initial-lula-report
CloudBeard Sep 18, 2024
e793cd1
Merge branch 'main' into initial-lula-report
CloudBeard Sep 24, 2024
3227f7b
update tests
CloudBeard Sep 25, 2024
6296154
re run doc generation
CloudBeard Sep 25, 2024
d707143
fix e2e test
CloudBeard Sep 27, 2024
9371ff8
removed failed case, its covered in unit test for the function itself
CloudBeard Sep 27, 2024
ee25f9f
add compose to handleComponentDefinittion
CloudBeard Sep 30, 2024
9d58163
update e2e test file to contain validations
CloudBeard Sep 30, 2024
ddd7fa8
Merge branch 'main' into initial-lula-report
CloudBeard Sep 30, 2024
60827db
extra space?
CloudBeard Sep 30, 2024
b4430e5
chore: empty commit to re-run CI
CloudBeard Sep 30, 2024
1862398
update report structure
CloudBeard Oct 2, 2024
07e630a
update go fmt
CloudBeard Oct 3, 2024
79a034b
Merge branch 'main' of https://github.com/defenseunicorns/lula into i…
CloudBeard Oct 8, 2024
09592ac
still wip need to fix to work with new compose
CloudBeard Oct 8, 2024
c363e45
updated compose calls
CloudBeard Oct 10, 2024
df8a3cd
update to table function and I think fixed e2e tests to match
CloudBeard Oct 10, 2024
baf8f91
Merge branch 'main' into initial-lula-report
CloudBeard Oct 10, 2024
5698e54
Merge branch 'main' into initial-lula-report
CloudBeard Oct 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/cli-commands/report/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Report

Lula `report` will display useful information contained within the OSCAL files.

Lula `report` currently only supports Component Definition models but does intend to increase support for all models and information within the report.
17 changes: 17 additions & 0 deletions docs/cli-commands/report/report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Report

The Lula `report` command will ingest OSCAL Component Definition models and will display the number of controls mapped on a per source and per framework. This will allow for uses to quickly see how many controls are mapped per source in the event of multiple `control-implementations`. This also allows for the Lula specific `prop` of `framework` which allows for a more custom mapping.

Use the command as follows:

For the default report

```bash
lula report -f oscal-component-definition.yaml
```

To change the display format to json or yaml

```bash
lula report -f oscal-component-definition.yaml --file-format json
```
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ require (
github.com/spf13/afero v1.11.0 // indirect
github.com/spf13/cast v1.6.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/tchap/go-patricia/v2 v2.3.1 // indirect
github.com/vladimirvivien/gexe v0.2.0 // indirect
Expand Down
314 changes: 314 additions & 0 deletions src/cmd/report/report.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,314 @@
package report
brandtkeller marked this conversation as resolved.
Show resolved Hide resolved

import (
"errors"
"os"
"encoding/json"
"strings"
"fmt"

"github.com/spf13/cobra"
"github.com/defenseunicorns/lula/src/pkg/message"
"github.com/defenseunicorns/lula/src/pkg/common/oscal"
"github.com/defenseunicorns/lula/src/pkg/common/network"
oscalTypes_1_1_2 "github.com/defenseunicorns/go-oscal/src/types/oscal-1-1-2"
"gopkg.in/yaml.v3"
)

type flags struct {
InputFile string // -f --input-file
FileFormat string // --file-format
}

type ReportData struct {
ComponentDefinition *ComponentDefinitionReportData `json:"componentDefinition,omitempty" yaml:"componentDefinition,omitempty"`
}

type ComponentDefinitionReportData struct {
Title string `json:"title" yaml:"title"`
ControlIDBySource map[string]int `json:"control ID mapped" yaml:"control ID mapped"`
ControlIDByFramework map[string]int `json:"controlIDFramework" yaml:"controlIDFramework"`
}

var opts = &flags{}

var reportHelp = `
brandtkeller marked this conversation as resolved.
Show resolved Hide resolved
To create a new report:
lula report -f oscal-component-definition.yaml

To create a new report in json format:
lula report -f oscal-component-definition.yaml --file-format json

To create a new report in yaml format:
lula report -f oscal-component-definition.yaml --file-format yaml
`

var reportCmd = &cobra.Command{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this has been up for some time - Can we align the creation of this command to template such that the same cobra patterns are followed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Believe ive restructured this

Use: "report",
Hidden: false,
Aliases: []string{"r"},
Short: "Build a compliance report",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's challenge this a bit more -> what function does or can this command serve?

Report on the current state of compliance
or....

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe not build a compliance report as in a pdf but I can see it pulling various statics and data out of the OSCAL to be easier to read and understand. I am picturing a thousands of lines SSP and I want to know what controls am I missing, what controls do I have, do I have multiple frameworks and can I see that per framework. For POAMs Im thinking how do I see my critical or highs specifically, how do I see time left on moderates to patch.

For components I would like to have a quick count of controls per framework, how many have validations, how many controls are in this component-definition that aren't in a profile/catalog.

I can also see parts of it being reused similar to lula evaluate -f assessment-result.yaml --sumnmary doing the summary part.

Maybe the part that gets the data for UDS Runtime to present too. Thought there is it would be similar questions/data points around the OSCAL to show.

Example: reportHelp,
Run: func(_ *cobra.Command, args []string) {
// Call the core logic for generating the report
err := GenerateReport(opts.InputFile, opts.FileFormat)
if err != nil {
message.Fatal(err, "error running report")
}
},
}

// Runs the logic of report generation
func GenerateReport(inputFile string, fileFormat string) error {
spinner := message.NewProgressSpinner("Fetching or reading file %s", inputFile)
getOSCALModelsFile, err := fetchOrReadFile(inputFile)
if err != nil {
spinner.Fatalf(fmt.Errorf("failed to get OSCAL file: %v", err), "failed to get OSCAL file")
}
spinner.Success()

spinner = message.NewProgressSpinner("Reading OSCAL model from file")
oscalModels, err := readOSCALModel(getOSCALModelsFile)
if err != nil {
spinner.Fatalf(fmt.Errorf("failed to read OSCAL Model data: %v", err), "failed to read OSCAL Model")
}
spinner.Success()

spinner = message.NewProgressSpinner("Determining OSCAL model type")
modelType, err := determineOSCALModel(oscalModels)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The oscal pkg under src/pkg/common/oscal contains a GetOscalModel() that returns the same values.

I would consider removing this -> removing the required parameter from the handleOSCALModel() function and call it directly inside handleOSCALModel().

Unless you have some unique requirement.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up using a few more functions from common where I found redundancy.

if err != nil {
spinner.Fatalf(fmt.Errorf("unable to determine OSCAL model type: %v", err), "unable to determine OSCAL model type")
}
spinner.Success()

err = handleOSCALModel(oscalModels, modelType, fileFormat)
if err != nil {
return err
}
return nil
}

func ReportCommand() *cobra.Command {
reportFlags()
return reportCmd
}

func reportFlags() {
reportFlags := reportCmd.PersistentFlags()
reportFlags.StringVarP(&opts.InputFile, "input-file", "f", "", "Path to an OSCAL file")
reportFlags.StringVar(&opts.FileFormat, "file-format", "table", "File format of the report")
reportCmd.MarkPersistentFlagRequired("input-file")
}

func fetchOrReadFile(source string) ([]byte, error) {
if isURL(source) {
spinner := message.NewProgressSpinner("Fetching data from URL: %s", source)
defer spinner.Stop()
data, err := network.Fetch(source)
if err != nil {
spinner.Fatalf(err, "failed to fetch data from URL")
}
spinner.Success()
return data, nil
}
spinner := message.NewProgressSpinner("Reading file: %s", source)
defer spinner.Stop()
data, err := os.ReadFile(source)
if err != nil {
spinner.Fatalf(err, "failed to read file")
}
spinner.Success()
return data, nil
}

// Reads OSCAL file in either YAML or JSON
func readOSCALModel(data []byte) (oscalTypes_1_1_2.OscalModels, error) {
var oscalModels oscalTypes_1_1_2.OscalModels
err := yaml.Unmarshal(data, &oscalModels)
if err == nil {
return oscalModels, nil
}
err = json.Unmarshal(data, &oscalModels)
if err == nil {
return oscalModels, nil
}
return oscalModels, errors.New("data is neither valid YAML nor JSON")
}

// Checks the OSCAL file to determine what model the file is
func determineOSCALModel(oscalModels oscalTypes_1_1_2.OscalModels) (string, error) {
switch {
case oscalModels.AssessmentPlan != nil:
return "assessment-plan", nil
case oscalModels.AssessmentResults != nil:
return "assessment-results", nil
case oscalModels.Catalog != nil:
return "catalog", nil
case oscalModels.ComponentDefinition != nil:
return "component-definition", nil
case oscalModels.PlanOfActionAndMilestones != nil:
return "plan-of-action-and-milestones", nil
case oscalModels.Profile != nil:
return "profile", nil
case oscalModels.SystemSecurityPlan != nil:
return "system-security-plan", nil
default:
return "", fmt.Errorf("unable to determine OSCAL model type")
}
}

// Processes an OSCAL Model based on the model type
func handleOSCALModel(oscalModels oscalTypes_1_1_2.OscalModels, modelType string, format string) error {
// Start a new spinner for the report generation process
spinner := message.NewProgressSpinner("Processing OSCAL model type: %s", modelType)
defer spinner.Stop() // Ensure the spinner stops even if an error occurs

switch modelType {
case "catalog", "profile", "assessment-plan", "assessment-results", "system-security-plan", "plan-of-action-and-milestones":
// If the model type is not supported, stop the spinner with a warning
spinner.Warnf("reporting does not create reports for %s at this time", modelType)
return fmt.Errorf("reporting does not create reports for %s at this time", modelType)

case "component-definition":
// Process the component-definition model
err := handleComponentDefinition(oscalModels.ComponentDefinition, format)
if err != nil {
// If an error occurs, stop the spinner and display the error
spinner.Fatalf(err, "failed to process component-definition model")
return err
}

default:
// For unknown model types, stop the spinner with a failure
spinner.Fatalf(fmt.Errorf("unknown OSCAL model type: %s", modelType), "failed to process OSCAL file")
return fmt.Errorf("unknown OSCAL model type: %s", modelType)
}

spinner.Success()
message.Info(fmt.Sprintf("Successfully processed OSCAL model: %s", modelType))
return nil
}

// Handler for Component Definition OSCAL files to create the report
func handleComponentDefinition(componentDefinition *oscalTypes_1_1_2.ComponentDefinition, format string) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we consider composing comp-defs here prior to processing?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call! added the compose pieces to the beginning there.

controlMap := oscal.FilterControlImplementations(componentDefinition)
extractedData := ExtractControlIDs(controlMap)
extractedData.Title = componentDefinition.Metadata.Title

report := ReportData{
ComponentDefinition: extractedData,
}

message.Info("Generating report...")
return PrintReport(report, format)
}

// Gets the unique Control IDs from each source and framework in the OSCAL Component Definition
func ExtractControlIDs(controlMap map[string][]oscalTypes_1_1_2.ControlImplementationSet) *ComponentDefinitionReportData {
sourceMap, frameworkMap := SplitControlMap(controlMap)

sourceControlIDs := make(map[string]int)
for source, controlMap := range sourceMap {
total := 0
for _, count := range controlMap {
total += count
}
sourceControlIDs[source] = total
}

aggregatedFrameworkCounts := make(map[string]int)
for framework, controlCounts := range frameworkMap {
total := 0
for _, count := range controlCounts {
total += count
}
aggregatedFrameworkCounts[framework] = total
}

return &ComponentDefinitionReportData{
ControlIDBySource: sourceControlIDs,
ControlIDByFramework: aggregatedFrameworkCounts,
}
}

// Split the default controlMap into framework and source maps for further processing
func SplitControlMap(controlMap map[string][]oscalTypes_1_1_2.ControlImplementationSet) (sourceMap map[string]map[string]int, frameworkMap map[string]map[string]int) {
sourceMap = make(map[string]map[string]int)
frameworkMap = make(map[string]map[string]int)

for key, implementations := range controlMap {
if isURL(key) {
if _, exists := sourceMap[key]; !exists {
sourceMap[key] = make(map[string]int)
}
for _, controlImplementation := range implementations {
for _, implementedReq := range controlImplementation.ImplementedRequirements {
controlID := implementedReq.ControlId
sourceMap[key][controlID]++
}
}
} else {
if _, exists := frameworkMap[key]; !exists {
frameworkMap[key] = make(map[string]int)
}
for _, controlImplementation := range implementations {
for _, implementedReq := range controlImplementation.ImplementedRequirements {
controlID := implementedReq.ControlId
frameworkMap[key][controlID]++
}
}
}
}

return sourceMap, frameworkMap
}

// Helper to determine if the controlMap source is a URL
func isURL(str string) bool {
return strings.HasPrefix(str, "http://") || strings.HasPrefix(str, "https://")
}

func PrintReport(data ReportData, format string) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't picked through this, so this function might not be relevant, but the Table under messages might save some of this functionality

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added that table functionality, I like the way that looks.

if format == "table" {
// Use the message package for printing table data
message.Infof("Title: %s", data.ComponentDefinition.Title)

// Print the Control ID By Source as a table
message.Info("\nControl Source | Number of Controls")
message.Info(strings.Repeat("-", 60))

for source, count := range data.ComponentDefinition.ControlIDBySource {
message.Infof("%-40s | %-15d", source, count)
}

// Print the Control ID By Framework as a table
message.Info("\nFramework | Number of Controls")
message.Info(strings.Repeat("-", 40))

for framework, count := range data.ComponentDefinition.ControlIDByFramework {
message.Infof("%-20s | %-15d", framework, count)
}

} else {
var err error
var fileData []byte

if format == "yaml" {
message.Info("Generating report in YAML format...")
fileData, err = yaml.Marshal(data)
if err != nil {
message.Fatal(err, "Failed to marshal data to YAML")
}
} else {
message.Info("Generating report in JSON format...")
fileData, err = json.MarshalIndent(data, "", " ")
if err != nil {
message.Fatal(err, "Failed to marshal data to JSON")
}
}

message.Info(string(fileData))
}

return nil
}
2 changes: 2 additions & 0 deletions src/cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"github.com/defenseunicorns/lula/src/cmd/dev"
"github.com/defenseunicorns/lula/src/cmd/evaluate"
"github.com/defenseunicorns/lula/src/cmd/generate"
"github.com/defenseunicorns/lula/src/cmd/report"
"github.com/defenseunicorns/lula/src/cmd/tools"
"github.com/defenseunicorns/lula/src/cmd/validate"
"github.com/defenseunicorns/lula/src/cmd/version"
Expand Down Expand Up @@ -44,6 +45,7 @@ func init() {
validate.ValidateCommand(),
evaluate.EvaluateCommand(),
generate.GenerateCommand(),
report.ReportCommand(),
console.ConsoleCommand(),
}

Expand Down
Loading