Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initial lula report #599

Draft
wants to merge 25 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
95568b1
initial command
CloudBeard Aug 5, 2024
3f7bb0c
initial command
CloudBeard Aug 9, 2024
d1543fb
initial report - still wip
CloudBeard Aug 12, 2024
0ea6bde
still wip, need to pull controls from catalog and profile to do math …
CloudBeard Aug 14, 2024
8027410
added report doc
CloudBeard Aug 19, 2024
b10e5ff
refactored to test main function, added test and test data
CloudBeard Sep 18, 2024
bba7581
Merge branch 'main' into initial-lula-report
CloudBeard Sep 18, 2024
e793cd1
Merge branch 'main' into initial-lula-report
CloudBeard Sep 24, 2024
3227f7b
update tests
CloudBeard Sep 25, 2024
6296154
re run doc generation
CloudBeard Sep 25, 2024
d707143
fix e2e test
CloudBeard Sep 27, 2024
9371ff8
removed failed case, its covered in unit test for the function itself
CloudBeard Sep 27, 2024
ee25f9f
add compose to handleComponentDefinittion
CloudBeard Sep 30, 2024
9d58163
update e2e test file to contain validations
CloudBeard Sep 30, 2024
ddd7fa8
Merge branch 'main' into initial-lula-report
CloudBeard Sep 30, 2024
60827db
extra space?
CloudBeard Sep 30, 2024
b4430e5
chore: empty commit to re-run CI
CloudBeard Sep 30, 2024
1862398
update report structure
CloudBeard Oct 2, 2024
07e630a
update go fmt
CloudBeard Oct 3, 2024
79a034b
Merge branch 'main' of https://github.com/defenseunicorns/lula into i…
CloudBeard Oct 8, 2024
09592ac
still wip need to fix to work with new compose
CloudBeard Oct 8, 2024
c363e45
updated compose calls
CloudBeard Oct 10, 2024
df8a3cd
update to table function and I think fixed e2e tests to match
CloudBeard Oct 10, 2024
baf8f91
Merge branch 'main' into initial-lula-report
CloudBeard Oct 10, 2024
5698e54
Merge branch 'main' into initial-lula-report
CloudBeard Oct 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/cli-commands/report/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Report

Lula `report` will display useful information contained within the OSCAL files.

Lula `report` currently only supports Component Definition models but does intend to increase support for all models and information within the report.
17 changes: 17 additions & 0 deletions docs/cli-commands/report/report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Report

The Lula `report` command will ingest OSCAL Component Definition models and will display the number of controls mapped on a per source and per framework. This will allow for uses to quickly see how many controls are mapped per source in the event of multiple `control-implementations`. This also allows for the Lula specific `prop` of `framework` which allows for a more custom mapping.

Use the command as follows:

For the default report

```bash
lula report -f oscal-component-definition.yaml
```

To change the display format to json or yaml

```bash
lula report -f oscal-component-definition.yaml --file-format json
```
281 changes: 281 additions & 0 deletions src/cmd/report/report.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,281 @@
package report
brandtkeller marked this conversation as resolved.
Show resolved Hide resolved

import (
"errors"
"os"
"encoding/json"
"strings"
"fmt"

"github.com/spf13/cobra"
"github.com/defenseunicorns/lula/src/pkg/message"
"github.com/defenseunicorns/lula/src/pkg/common/oscal"
"github.com/defenseunicorns/lula/src/pkg/common/network"
oscalTypes_1_1_2 "github.com/defenseunicorns/go-oscal/src/types/oscal-1-1-2"
"gopkg.in/yaml.v3"
)

type flags struct {
InputFile string // -f --input-file
FileFormat string // --file-format
}

type ReportData struct {
ComponentDefinition *ComponentDefinitionReportData `json:"componentDefinition,omitempty" yaml:"componentDefinition,omitempty"`
}

type ComponentDefinitionReportData struct {
Title string `json:"title" yaml:"title"`
ControlIDBySource map[string]int `json:"control ID mapped" yaml:"control ID mapped"`
ControlIDByFramework map[string]int `json:"controlIDFramework" yaml:"controlIDFramework"`
}

var opts = &flags{}

var reportHelp = `
brandtkeller marked this conversation as resolved.
Show resolved Hide resolved
To create a new report:
lula report -f oscal-component-definition.yaml -o report.yaml
`

var reportCmd = &cobra.Command{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this has been up for some time - Can we align the creation of this command to template such that the same cobra patterns are followed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Believe ive restructured this

Use: "report",
Hidden: false,
Aliases: []string{"r"},
Short: "Build a compliance report",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's challenge this a bit more -> what function does or can this command serve?

Report on the current state of compliance
or....

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe not build a compliance report as in a pdf but I can see it pulling various statics and data out of the OSCAL to be easier to read and understand. I am picturing a thousands of lines SSP and I want to know what controls am I missing, what controls do I have, do I have multiple frameworks and can I see that per framework. For POAMs Im thinking how do I see my critical or highs specifically, how do I see time left on moderates to patch.

For components I would like to have a quick count of controls per framework, how many have validations, how many controls are in this component-definition that aren't in a profile/catalog.

I can also see parts of it being reused similar to lula evaluate -f assessment-result.yaml --sumnmary doing the summary part.

Maybe the part that gets the data for UDS Runtime to present too. Thought there is it would be similar questions/data points around the OSCAL to show.

Example: reportHelp,
Run: func(_ *cobra.Command, args []string) {
if opts.InputFile == "" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we mark this flag as required then we can remove this check

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Swapped that to required removed the check.

message.Fatal(errors.New("flag input-file is not set"),
"Please specify an input file with the -f flag")
}

spinner := message.NewProgressSpinner("Fetching or reading file %s", opts.InputFile)
getOSCALModelsFile, err := fetchOrReadFile(opts.InputFile)
if err != nil {
spinner.Fatalf(err, "Failed to get OSCAL file")
}
spinner.Success()

spinner = message.NewProgressSpinner("Reading OSCAL model from file")
oscalModels, err := readOSCALModel(getOSCALModelsFile)
if err != nil {
spinner.Fatalf(err, "Failed to get OSCAL Model data")
}
spinner.Success()

spinner = message.NewProgressSpinner("Determining OSCAL model type")
modelType, err := determineOSCALModel(oscalModels)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may be a good spot to establish a separation of duties -> We've performed the essential operation of retrieving/parsing the data -> now let's create a function that handles the next steps.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up splitting that up twice, the first time just to separate duties but ended up adding 2 new functions. 1 that is public that will run the actual logic and the second that is a handler for model handlers.

Ran into an issue testing so I came back to add the function that runs the logic.

if err != nil {
spinner.Fatalf(err, "Unable to determine OSCAL model type")
}
spinner.Success()

spinner = message.NewProgressSpinner("Generating report")
brandtkeller marked this conversation as resolved.
Show resolved Hide resolved
var reportModelErr error
switch modelType {
case "catalog", "profile", "assessment-plan", "assessment-results", "system-security-plan", "plan-of-action-and-milestones":
message.Warnf("Reporting does not create reports for %s at this time", modelType)
case "component-definition":
reportModelErr = handleComponentDefinition(oscalModels.ComponentDefinition, opts.FileFormat)
default:
spinner.Fatalf(fmt.Errorf("unknown OSCAL model type: %s", modelType), "Failed to process OSCAL file")
}
spinner.Success()

if reportModelErr != nil {
message.Fatal(reportModelErr, "Failed to create report")
}
},
}

func ReportCommand() *cobra.Command {
reportFlags()
return reportCmd
}

func reportFlags() {
reportFlags := reportCmd.PersistentFlags()
reportFlags.StringVarP(&opts.InputFile, "input-file", "f", "", "Path to an OSCAL file")
reportFlags.StringVar(&opts.FileFormat, "file-format", "table", "File format of output file")
}

func fetchOrReadFile(source string) ([]byte, error) {
if isURL(source) {
spinner := message.NewProgressSpinner("Fetching data from URL: %s", source)
defer spinner.Stop()
data, err := network.Fetch(source)
if err != nil {
spinner.Fatalf(err, "Failed to fetch data from URL")
}
spinner.Success()
return data, nil
}
spinner := message.NewProgressSpinner("Reading file: %s", source)
defer spinner.Stop()
data, err := os.ReadFile(source)
if err != nil {
spinner.Fatalf(err, "Failed to read file")
}
spinner.Success()
return data, nil
}

// Reads OSCAL file in either YAML or JSON
func readOSCALModel(data []byte) (oscalTypes_1_1_2.OscalModels, error) {
var oscalModels oscalTypes_1_1_2.OscalModels
err := yaml.Unmarshal(data, &oscalModels)
if err == nil {
return oscalModels, nil
}
err = json.Unmarshal(data, &oscalModels)
if err == nil {
return oscalModels, nil
}
return oscalModels, errors.New("data is neither valid YAML nor JSON")
}

// Checks the OSCAL file to determine what model the file is
func determineOSCALModel(oscalModels oscalTypes_1_1_2.OscalModels) (string, error) {
switch {
case oscalModels.AssessmentPlan != nil:
return "assessment-plan", nil
case oscalModels.AssessmentResults != nil:
return "assessment-results", nil
case oscalModels.Catalog != nil:
return "catalog", nil
case oscalModels.ComponentDefinition != nil:
return "component-definition", nil
case oscalModels.PlanOfActionAndMilestones != nil:
return "plan-of-action-and-milestones", nil
case oscalModels.Profile != nil:
return "profile", nil
case oscalModels.SystemSecurityPlan != nil:
return "system-security-plan", nil
default:
return "", fmt.Errorf("unable to determine OSCAL model type")
}
}

// Handler for Component Definition OSCAL files to create the report
func handleComponentDefinition(componentDefinition *oscalTypes_1_1_2.ComponentDefinition, format string) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we consider composing comp-defs here prior to processing?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call! added the compose pieces to the beginning there.

controlMap := oscal.FilterControlImplementations(componentDefinition)
extractedData := ExtractControlIDs(controlMap)
extractedData.Title = componentDefinition.Metadata.Title

report := ReportData{
ComponentDefinition: extractedData,
}

message.Info("Generating report...")
return PrintReport(report, format)
}

// Gets the unique Control IDs from each source and framework in the OSCAL Component Definition
func ExtractControlIDs(controlMap map[string][]oscalTypes_1_1_2.ControlImplementationSet) *ComponentDefinitionReportData {
sourceMap, frameworkMap := SplitControlMap(controlMap)

sourceControlIDs := make(map[string]int)
for source, controlMap := range sourceMap {
total := 0
for _, count := range controlMap {
total += count
}
sourceControlIDs[source] = total
}

aggregatedFrameworkCounts := make(map[string]int)
for framework, controlCounts := range frameworkMap {
total := 0
for _, count := range controlCounts {
total += count
}
aggregatedFrameworkCounts[framework] = total
}

return &ComponentDefinitionReportData{
ControlIDBySource: sourceControlIDs,
ControlIDByFramework: aggregatedFrameworkCounts,
}
}

// Split the default controlMap into framework and source maps for further processing
func SplitControlMap(controlMap map[string][]oscalTypes_1_1_2.ControlImplementationSet) (sourceMap map[string]map[string]int, frameworkMap map[string]map[string]int) {
sourceMap = make(map[string]map[string]int)
frameworkMap = make(map[string]map[string]int)

for key, implementations := range controlMap {
if isURL(key) {
if _, exists := sourceMap[key]; !exists {
sourceMap[key] = make(map[string]int)
}
for _, controlImplementation := range implementations {
for _, implementedReq := range controlImplementation.ImplementedRequirements {
controlID := implementedReq.ControlId
sourceMap[key][controlID]++
}
}
} else {
if _, exists := frameworkMap[key]; !exists {
frameworkMap[key] = make(map[string]int)
}
for _, controlImplementation := range implementations {
for _, implementedReq := range controlImplementation.ImplementedRequirements {
controlID := implementedReq.ControlId
frameworkMap[key][controlID]++
}
}
}
}

return sourceMap, frameworkMap
}

// Helper to determine if the controlMap source is a URL
func isURL(str string) bool {
return strings.HasPrefix(str, "http://") || strings.HasPrefix(str, "https://")
}

func PrintReport(data ReportData, format string) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't picked through this, so this function might not be relevant, but the Table under messages might save some of this functionality

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added that table functionality, I like the way that looks.

if format == "table" {
// Use the message package for printing table data
message.Infof("Title: %s", data.ComponentDefinition.Title)

// Print the Control ID By Source as a table
message.Info("\nControl Source | Number of Controls")
message.Info(strings.Repeat("-", 60))

for source, count := range data.ComponentDefinition.ControlIDBySource {
message.Infof("%-40s | %-15d", source, count)
}

// Print the Control ID By Framework as a table
message.Info("\nFramework | Number of Controls")
message.Info(strings.Repeat("-", 40))

for framework, count := range data.ComponentDefinition.ControlIDByFramework {
message.Infof("%-20s | %-15d", framework, count)
}

} else {
var err error
var fileData []byte

if format == "yaml" {
message.Info("Generating report in YAML format...")
fileData, err = yaml.Marshal(data)
if err != nil {
message.Fatal(err, "Failed to marshal data to YAML")
}
} else {
message.Info("Generating report in JSON format...")
fileData, err = json.MarshalIndent(data, "", " ")
if err != nil {
message.Fatal(err, "Failed to marshal data to JSON")
}
}

message.Info(string(fileData))
}

return nil
}
2 changes: 2 additions & 0 deletions src/cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"github.com/defenseunicorns/lula/src/cmd/dev"
"github.com/defenseunicorns/lula/src/cmd/evaluate"
"github.com/defenseunicorns/lula/src/cmd/generate"
"github.com/defenseunicorns/lula/src/cmd/report"
"github.com/defenseunicorns/lula/src/cmd/tools"
"github.com/defenseunicorns/lula/src/cmd/validate"
"github.com/defenseunicorns/lula/src/cmd/version"
Expand Down Expand Up @@ -62,6 +63,7 @@ func init() {
validate.ValidateCommand(),
evaluate.EvaluateCommand(),
generate.GenerateCommand(),
report.ReportCommand(),
}

rootCmd.AddCommand(commands...)
Expand Down
Loading