Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: big bang generate #3002

Closed
wants to merge 55 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
1a81554
WIP
AustinAbro321 Sep 11, 2024
a474130
deprecate big bang
AustinAbro321 Sep 16, 2024
55a9a0b
deprecate big bang
AustinAbro321 Sep 16, 2024
d79554b
big bang package is creating
AustinAbro321 Sep 16, 2024
1d6df4c
CLI flags
AustinAbro321 Sep 16, 2024
d1141e2
unit test passing
AustinAbro321 Sep 17, 2024
d2019d5
tests passing
AustinAbro321 Sep 17, 2024
4a37ff3
test actually working
AustinAbro321 Sep 17, 2024
8587340
Tests passing
AustinAbro321 Sep 17, 2024
13c8ee5
skip flux working
AustinAbro321 Sep 17, 2024
a609142
adding bb-generate example
AustinAbro321 Sep 17, 2024
68f662b
change how we get git repo and helm repo so it's direct from bb
AustinAbro321 Sep 17, 2024
00fb308
remove unused test
AustinAbro321 Sep 17, 2024
7c3c6be
add bb manifests
AustinAbro321 Sep 17, 2024
1af3cf0
delete old examples
AustinAbro321 Sep 17, 2024
b63d986
edit test
AustinAbro321 Sep 17, 2024
4c4d017
test manifests
AustinAbro321 Sep 17, 2024
9fd9bd9
cleanup tests
AustinAbro321 Sep 17, 2024
cae8b6c
cleanup tests
AustinAbro321 Sep 17, 2024
0b59633
cleanup tests
AustinAbro321 Sep 17, 2024
810a86f
delete comments
AustinAbro321 Sep 17, 2024
3487a9a
bb generate example
AustinAbro321 Sep 17, 2024
aaaf288
sort helm releases
AustinAbro321 Sep 17, 2024
6ec763a
different file if zarf.yaml already exists
AustinAbro321 Sep 18, 2024
bbb6b2a
credentials as a file now
AustinAbro321 Sep 18, 2024
237557c
allow secrets with data or string data
AustinAbro321 Sep 18, 2024
275313e
remove stutter
AustinAbro321 Sep 18, 2024
1e7716a
wording
AustinAbro321 Sep 18, 2024
7b0ee2e
update docs
AustinAbro321 Sep 18, 2024
b0ba0e5
values files
AustinAbro321 Sep 18, 2024
b0f79ba
update tests
AustinAbro321 Sep 18, 2024
52e1989
taking away metrics server health checks and failure debug
AustinAbro321 Sep 18, 2024
4a26e5c
create tests
AustinAbro321 Sep 18, 2024
d0646d5
add skip flux test
AustinAbro321 Sep 18, 2024
55e761d
skip flux
AustinAbro321 Sep 18, 2024
db0b291
remove spinner
AustinAbro321 Sep 18, 2024
301780c
mock http
AustinAbro321 Sep 18, 2024
80dded7
tests passing
AustinAbro321 Sep 18, 2024
cc060d5
docs and schema
AustinAbro321 Sep 18, 2024
7be99b9
add type meta
AustinAbro321 Sep 19, 2024
d93eeda
fix health checks
AustinAbro321 Sep 19, 2024
0e0b7e6
change health checks
AustinAbro321 Sep 19, 2024
ba910ec
update to newer kubernetes package
AustinAbro321 Sep 19, 2024
f4c1bf7
controller runtime indrect
AustinAbro321 Sep 19, 2024
54ad03c
better error messages
AustinAbro321 Sep 19, 2024
3a047bc
tests passing
AustinAbro321 Sep 20, 2024
e4ed64c
change tests
AustinAbro321 Sep 20, 2024
bd27215
filepath not sprint
AustinAbro321 Sep 20, 2024
ab080f5
Merge branch 'main' into big-bang-generate
AustinAbro321 Sep 20, 2024
54a97d1
make docs and schema
AustinAbro321 Sep 20, 2024
21ffb72
go mod tidy
AustinAbro321 Sep 20, 2024
ef22af4
Merge branch 'main' into big-bang-generate
AustinAbro321 Sep 20, 2024
0f9d1c5
revert ingress.yaml
AustinAbro321 Sep 20, 2024
a5ccbb2
remove ingress from example
AustinAbro321 Sep 20, 2024
feec104
multiple manifests
AustinAbro321 Sep 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12,409 changes: 12,409 additions & 0 deletions examples/bb-generate/flux/bb-flux.yaml

Large diffs are not rendered by default.

14 changes: 14 additions & 0 deletions examples/bb-generate/manifests/bb-gitrepository.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
name: bigbang
spec:
ignore: |
# exclude file extensions
/**/*.md
/**/*.txt
/**/*.sh
interval: 10m
url: https://repo1.dso.mil/big-bang/bigbang.git
ref:
tag: 2.35.0
36 changes: 36 additions & 0 deletions examples/bb-generate/manifests/bb-helmrelease.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: bigbang
spec:
chart:
spec:
chart: chart
sourceRef:
kind: GitRepository
name: bigbang
install:
remediation:
retries: -1
interval: 10m
releaseName: bigbang
rollback:
cleanupOnFail: false
timeout: 10m
targetNamespace: bigbang
test:
enable: false
upgrade:
cleanupOnFail: true
remediation:
remediateLastFailure: true
retries: 5
valuesFrom:
- kind: Secret
name: zarf-credentials
- kind: ConfigMap
name: kyverno-config
- kind: ConfigMap
name: loki-config
- kind: Secret
name: bb-neuvector-vals
23 changes: 23 additions & 0 deletions examples/bb-generate/manifests/bb-zarf-credentials.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
apiVersion: v1
kind: Secret
metadata:
name: zarf-credentials
namespace: bigbang
stringData:
values.yaml: |
registryCredentials:
registry: "###ZARF_REGISTRY###"
username: "zarf-pull"
password: "###ZARF_REGISTRY_AUTH_PULL###"
git:
existingSecret: "private-git-server" # -- Chart created secrets with user defined values
credentials:
username: "###ZARF_GIT_PUSH###" # -- HTTP git credentials, both username and password must be provided
password: "###ZARF_GIT_AUTH_PUSH###"
kyvernoPolicies:
values:
exclude:
any:
- resources:
namespaces:
- zarf # don't have Kyverno prevent Zarf from doing zarf things
32 changes: 32 additions & 0 deletions examples/bb-generate/values-files/kyverno.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: kyverno-config
data:
values.yaml: |
gatekeeper:
enabled: false
clusterAuditor:
enabled: false
kyverno:
enabled: true
kyvernoPolicies:
enabled: true
values:
policies:
disallow-shared-subpath-volume-writes:
validationFailureAction: audit
restrict-host-ports:
validationFailureAction: audit
restrict-capabilities:
validationFailureAction: audit
restrict-image-registries:
validationFailureAction: audit
disallow-host-namespaces:
validationFailureAction: audit
disallow-privileged-containers:
validationFailureAction: audit
require-non-root-user:
validationFailureAction: audit
restrict-host-path-mount-pv:
validationFailureAction: audit
20 changes: 20 additions & 0 deletions examples/bb-generate/values-files/loki.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: loki-config
data:
values.yaml: |
elasticsearchKibana:
enabled: false

eckOperator:
enabled: false

fluentbit:
enabled: false

loki:
enabled: true

promtail:
enabled: true
12 changes: 12 additions & 0 deletions examples/bb-generate/values-files/neuvector.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: v1
kind: Secret
metadata:
name: bb-neuvector-vals
namespace: bigbang
stringData:
values.yaml: |
# If running in k3s, this is needed for Neuvector to start properly
neuvector:
values:
k3s:
enabled: true
142 changes: 142 additions & 0 deletions examples/bb-generate/zarf.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
apiVersion: zarf.dev/v1alpha1
kind: ZarfPackageConfig
metadata:
name: bigbang
components:
- name: flux
required: true
manifests:
- name: flux-system
namespace: flux-system
files:
- flux/bb-flux.yaml
images:
- registry1.dso.mil/ironbank/fluxcd/source-controller:v1.3.0
- registry1.dso.mil/ironbank/fluxcd/kustomize-controller:v1.3.0
- registry1.dso.mil/ironbank/fluxcd/helm-controller:v1.0.1
- registry1.dso.mil/ironbank/fluxcd/notification-controller:v1.3.0
- name: bigbang
required: true
manifests:
- name: bigbang
namespace: bigbang
files:
- manifests/bb-gitrepository.yaml
- manifests/bb-zarf-credentials.yaml
- values-files/kyverno.yaml
- values-files/loki.yaml
- values-files/neuvector.yaml
- manifests/bb-helmrelease.yaml
images:
- registry1.dso.mil/ironbank/big-bang/grafana/grafana-plugins:11.1.4
- registry1.dso.mil/ironbank/kiwigrid/k8s-sidecar:1.27.5
- registry1.dso.mil/ironbank/big-bang/base:2.1.0
- registry1.dso.mil/ironbank/opensource/istio/pilot:1.22.4
- registry1.dso.mil/ironbank/opensource/istio/proxyv2:1.22.4
- registry1.dso.mil/ironbank/opensource/istio/operator:1.22.4
- registry1.dso.mil/ironbank/opensource/kiali/kiali:v1.89.0
- registry1.dso.mil/ironbank/opensource/kiali/kiali-operator:v1.89.1
- registry1.dso.mil/ironbank/opensource/kyverno:v1.12.5
- registry1.dso.mil/ironbank/opensource/kyverno/kyvernopre:v1.12.5
- registry1.dso.mil/ironbank/opensource/kubernetes/kubectl:v1.29.7
- registry1.dso.mil/ironbank/redhat/ubi/ubi9-minimal:9.4
- registry1.dso.mil/ironbank/opensource/kyverno/kyverno/reports-controller:v1.12.5
- registry1.dso.mil/ironbank/opensource/kyverno/kyverno/background-controller:v1.12.5
- registry1.dso.mil/ironbank/opensource/kyverno/kyverno/cleanup-controller:v1.12.5
- registry1.dso.mil/ironbank/opensource/kyverno/kyvernocli:v1.12.5
- registry1.dso.mil/ironbank/opensource/kyverno/policy-reporter:2.20.1
- registry1.dso.mil/ironbank/opensource/grafana/loki:3.1.1
- registry1.dso.mil/ironbank/opensource/kubernetes-sigs/metrics-server:v0.7.1
- registry1.dso.mil/ironbank/opensource/prometheus/alertmanager:v0.27.0
- registry1.dso.mil/ironbank/opensource/kubernetes/kubectl:v1.29.6
- registry1.dso.mil/ironbank/opensource/kubernetes/kube-state-metrics:v2.12.0
- registry1.dso.mil/ironbank/opensource/ingress-nginx/kube-webhook-certgen:v1.3.0
- registry1.dso.mil/ironbank/opensource/prometheus/prometheus:v2.53.0
- registry1.dso.mil/ironbank/opensource/prometheus-operator/prometheus-config-reloader:v0.75.0
- registry1.dso.mil/ironbank/opensource/prometheus-operator/prometheus-operator:v0.75.0
- registry1.dso.mil/ironbank/opensource/prometheus/node-exporter:v1.8.1
- registry1.dso.mil/ironbank/opensource/thanos/thanos:v0.35.1
- registry1.dso.mil/ironbank/neuvector/neuvector/controller:5.3.4
- registry1.dso.mil/ironbank/neuvector/neuvector/enforcer:5.3.4
- registry1.dso.mil/ironbank/neuvector/neuvector/manager:5.3.4
- registry1.dso.mil/ironbank/neuvector/neuvector/scanner:5
- registry1.dso.mil/ironbank/neuvector/neuvector/prometheus-exporter:5.3.2
- registry1.dso.mil/ironbank/opensource/grafana/promtail:v3.0.0
- registry1.dso.mil/ironbank/opensource/grafana/tempo:2.5.0
- registry1.dso.mil/ironbank/opensource/grafana/tempo-query:2.5.0
repos:
- https://repo1.dso.mil/big-bang/bigbang@2.35.0
- https://repo1.dso.mil/big-bang/product/packages/grafana.git@8.4.6-bb.1
- https://repo1.dso.mil/big-bang/product/packages/istio-controlplane.git@1.22.4-bb.1
- https://repo1.dso.mil/big-bang/product/packages/istio-operator.git@1.22.4-bb.0
- https://repo1.dso.mil/big-bang/product/packages/kiali.git@1.89.0-bb.0
- https://repo1.dso.mil/big-bang/product/packages/kyverno-policies.git@3.2.5-bb.3
- https://repo1.dso.mil/big-bang/product/packages/kyverno-reporter.git@2.24.1-bb.0
- https://repo1.dso.mil/big-bang/product/packages/kyverno.git@3.2.6-bb.0
- https://repo1.dso.mil/big-bang/product/packages/loki.git@6.10.0-bb.0
- https://repo1.dso.mil/big-bang/product/packages/metrics-server.git@3.12.1-bb.4
- https://repo1.dso.mil/big-bang/product/packages/monitoring.git@62.1.0-bb.0
- https://repo1.dso.mil/big-bang/product/packages/neuvector.git@2.7.8-bb.1
- https://repo1.dso.mil/big-bang/product/packages/promtail.git@6.16.2-bb.3
- https://repo1.dso.mil/big-bang/product/packages/tempo.git@1.10.3-bb.0
actions:
onRemove:
before:
- cmd: ./zarf tools kubectl patch helmrelease -n bigbang bigbang --type=merge -p '{"spec":{"suspend":true}}'
description: Suspend Big Bang HelmReleases to prevent reconciliation during removal.
healthChecks:
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: grafana
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: istio
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: istio-operator
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: kiali
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: kyverno
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: kyverno-policies
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: kyverno-reporter
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: loki
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: monitoring
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: neuvector
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: promtail
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
namespace: bigbang
name: tempo

# YAML keys starting with `x-` are custom keys that are ignored by the Zarf CLI
# The `x-mdx` key is used to render the markdown content for https://docs.zarf.dev/ref/examples
x-mdx: |
This package deploys [Big Bang](https://repo1.dso.mil/platform-one/big-bang/bigbang)

It was generated using the command `zarf dev generate big-bang 2.35.0 --values-file-manifests=values-files/kyverno.yaml,values-files/loki.yaml,values-files/neuvector.yaml`
58 changes: 0 additions & 58 deletions examples/big-bang/config/disable-all.yaml

This file was deleted.

Loading