From f6c315e48a6c06b785ce3eed29024486c84887bf Mon Sep 17 00:00:00 2001 From: "T.J. Telan" Date: Tue, 31 Oct 2023 15:32:20 -0700 Subject: [PATCH] chore: Syncing `master` to stable (#713) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat: add deduplication doc (#655) Added small document that explains how deduplication works * feat: graphite sink connector documentation (#654) Documents the Graphite Sink Connector image * [create-pull-request] automated change (#660) Co-authored-by: ozgrakkurt * docs: Update connector reference docs [bot] (#660) Updating connector docs Auto-generated by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action * add section on authentication and auth (#663) * Twif 57 (#664) * docs: Update connector reference docs [bot] (#667) Updating connector docs Auto-generated by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action * docs: Update connector reference docs [bot] (#668) Updating connector docs Auto-generated by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action * mention `age` lookback configuration in docs (#669) * chore(deps): bump @types/node from 20.3.2 to 20.5.3 in /functions/newsletter (#676) Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 20.3.2 to 20.5.3.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=@types/node&package-manager=npm_and_yarn&previous-version=20.3.2&new-version=20.5.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
* feat: smdk docs for `SmartModuleRecord` (#678) Introduces SmartModuleRecord to the documentation, and updates examples to encourage update. * Typo fix (#682) * chore(deps): bump actions/checkout from 3 to 4 (#685) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
Release notes

Sourced from actions/checkout's releases.

v4.0.0

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v3...v4.0.0

v3.6.0

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v3.5.3...v3.6.0

v3.5.3

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v3...v3.5.3

v3.5.2

What's Changed

Full Changelog: https://github.com/actions/checkout/compare/v3.5.1...v3.5.2

v3.5.1

What's Changed

New Contributors

... (truncated)

Changelog

Sourced from actions/checkout's changelog.

Changelog

v4.0.0

v3.6.0

v3.5.3

v3.5.2

v3.5.1

v3.5.0

v3.4.0

v3.3.0

v3.2.0

v3.1.0

v3.0.2

v3.0.1

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/checkout&package-manager=github_actions&previous-version=3&new-version=4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
* update connectors docs * update connectors docs (#684) Updated docs of released connectors Co-authored-by: A.J. Hunyady * docs: Update connector reference docs [bot] (#683) Updating connector docs Auto-generated by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action * chore(deps): Bump @types/node from 20.5.3 to 20.6.0 in /functions/newsletter (#686) Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 20.5.3 to 20.6.0.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=@types/node&package-manager=npm_and_yarn&previous-version=20.5.3&new-version=20.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
* chore(deps): Bump @types/node from 20.6.0 to 20.7.0 in /functions/newsletter (#687) Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 20.6.0 to 20.7.0.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=@types/node&package-manager=npm_and_yarn&previous-version=20.6.0&new-version=20.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
* feat: Add updates for `fvm` (#706) fluvio-cms now supports updating all the CLI docs as well * (website): correct usage of topic create (#704) This PR corrects the example usages of `$ fluvio topic create`. The `$ fluvio topic create` subcommand takes in the the topic name as a positional argument; `--topic` is not valid. Please refer to the following screenshot for more information: Screen Shot 2023-10-18 at 9 42 18 PM * fixup docs before fvm release (#712) * fixup docs before fvm release (#712) change hub-dev to hub (hub-dev changed the fvm endpoint in prep for next release) Co-authored-by: Alan Chen --------- Co-authored-by: Özgür Akkurt Co-authored-by: Esteban Borai Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: ozgrakkurt Co-authored-by: Sehyo Chang Co-authored-by: Deb Co-authored-by: github-actions[bot] Co-authored-by: github-actions[bot] Co-authored-by: Alexander Galibey Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: John Landahl Co-authored-by: A.J. Hunyady Co-authored-by: Benji Nguyen Co-authored-by: Alan Chen Co-authored-by: Alan Chen --- .github/workflows/check_examples.yml | 4 +- .github/workflows/htmltest.yml | 2 +- .github/workflows/screenshots.yml | 4 +- .github/workflows/skip-test-check.yml | 4 +- .github/workflows/update-connector-docs.yml | 15 +- Cargo.lock | 37 ++++- bors.toml | 3 +- content/cli/advanced/cli_channels.md | 73 ++++----- content/cli/cloud/webhook.md | 94 ++++++++++++ content/cli/smartmodules/hub.md | 43 +++--- .../cli/utilities/fluvio-version-manager.md | 32 ++++ content/cli/utilities/install.md | 5 + content/connectors-old/examples/github.md | 8 +- .../how-to/connectors-with-smartmodules.md | 4 +- content/connectors/outbound/graphite.md | 5 + content/docs/architecture/auth.md | 133 ++++++++++++++++ .../docs/architecture/replica-assignment.md | 4 +- content/docs/concepts/topics.md | 2 +- content/docs/kubernetes/external-access.md | 2 +- content/news/this-week-in-fluvio-0057.md | 45 ++++++ content/smartmodules/analytics/aggregate.md | 4 +- content/smartmodules/deduplication.md | 74 +++++++++ content/smartmodules/lookback.md | 10 +- content/smartmodules/smdk/build-test.md | 2 +- content/smartmodules/smdk/generate.md | 4 +- content/smartmodules/smdk/load.md | 2 +- content/smartmodules/smdk/publish.md | 2 +- .../smartmodules/smdk/smartmodulerecord.md | 58 +++++++ content/smartmodules/transform/array-map.md | 4 +- content/smartmodules/transform/filter-map.md | 4 +- content/smartmodules/transform/filter.md | 12 +- content/smartmodules/transform/map.md | 4 +- data/cli-commands.yml | 19 ++- embeds/cli/help/cdk-build.md | 5 + embeds/cli/help/cdk-deploy-list.md | 2 + embeds/cli/help/cdk-deploy-log.md | 11 +- embeds/cli/help/cdk-deploy-shutdown.md | 1 + embeds/cli/help/cdk-deploy-start.md | 13 +- embeds/cli/help/cdk-deploy.md | 6 + embeds/cli/help/cdk-generate.md | 20 ++- embeds/cli/help/cdk-publish.md | 48 +++++- embeds/cli/help/cdk-set-public.md | 12 +- embeds/cli/help/cdk-test.md | 5 + embeds/cli/help/cdk.md | 3 + .../cli/help/fluvio-cloud-cluster-create.md | 14 +- .../cli/help/fluvio-cloud-cluster-delete.md | 15 +- embeds/cli/help/fluvio-cloud-cluster-list.md | 10 +- embeds/cli/help/fluvio-cloud-cluster.md | 21 +-- .../cli/help/fluvio-cloud-connector-config.md | 15 +- .../cli/help/fluvio-cloud-connector-create.md | 7 +- .../cli/help/fluvio-cloud-connector-delete.md | 15 +- .../cli/help/fluvio-cloud-connector-list.md | 10 +- .../cli/help/fluvio-cloud-connector-logs.md | 15 +- .../cli/help/fluvio-cloud-connector-update.md | 1 - embeds/cli/help/fluvio-cloud-connector.md | 27 ++-- embeds/cli/help/fluvio-cloud-login.md | 34 +++-- embeds/cli/help/fluvio-cloud-logout.md | 10 +- embeds/cli/help/fluvio-cloud-secret-delete.md | 11 ++ embeds/cli/help/fluvio-cloud-secret-list.md | 8 + embeds/cli/help/fluvio-cloud-secret-set.md | 12 ++ embeds/cli/help/fluvio-cloud-usage.md | 10 +- embeds/cli/help/fluvio-cloud.md | 30 ++-- embeds/cli/help/fluvio-cluster-check.md | 15 +- embeds/cli/help/fluvio-cluster-delete.md | 2 + embeds/cli/help/fluvio-cluster-diagnostics.md | 2 + embeds/cli/help/fluvio-cluster-spg-create.md | 3 + embeds/cli/help/fluvio-cluster-spg-delete.md | 3 + embeds/cli/help/fluvio-cluster-spg-list.md | 2 + embeds/cli/help/fluvio-cluster-spg.md | 8 +- embeds/cli/help/fluvio-cluster-spu-list.md | 2 + .../cli/help/fluvio-cluster-spu-register.md | 2 + .../cli/help/fluvio-cluster-spu-unregister.md | 2 + embeds/cli/help/fluvio-cluster-spu.md | 8 +- embeds/cli/help/fluvio-cluster-start.md | 14 +- embeds/cli/help/fluvio-cluster-upgrade.md | 14 +- embeds/cli/help/fluvio-cluster.md | 3 + embeds/cli/help/fluvio-completions.md | 13 +- embeds/cli/help/fluvio-consume.md | 49 +++++- .../cli/help/fluvio-derived-stream-create.md | 3 - .../cli/help/fluvio-derived-stream-delete.md | 3 - embeds/cli/help/fluvio-derived-stream-list.md | 3 - embeds/cli/help/fluvio-derived-stream.md | 3 - .../cli/help/fluvio-hub-connector-download.md | 22 ++- embeds/cli/help/fluvio-hub-connector-list.md | 15 +- embeds/cli/help/fluvio-hub-connector.md | 3 + embeds/cli/help/fluvio-hub-download.md | 18 --- embeds/cli/help/fluvio-hub-list.md | 7 - .../help/fluvio-hub-smartmodule-download.md | 46 ++++++ .../cli/help/fluvio-hub-smartmodule-list.md | 18 +++ embeds/cli/help/fluvio-hub-smartmodule.md | 12 ++ embeds/cli/help/fluvio-hub.md | 8 +- embeds/cli/help/fluvio-install.md | 31 +++- embeds/cli/help/fluvio-partition-list.md | 2 + embeds/cli/help/fluvio-partition.md | 8 +- embeds/cli/help/fluvio-produce.md | 5 +- embeds/cli/help/fluvio-profile-current.md | 8 +- .../cli/help/fluvio-profile-delete-cluster.md | 3 + embeds/cli/help/fluvio-profile-delete.md | 3 + embeds/cli/help/fluvio-profile-export.md | 3 + embeds/cli/help/fluvio-profile-list.md | 2 + embeds/cli/help/fluvio-profile-rename.md | 3 + embeds/cli/help/fluvio-profile-switch.md | 3 + embeds/cli/help/fluvio-profile-sync.md | 3 + embeds/cli/help/fluvio-profile.md | 9 +- embeds/cli/help/fluvio-run-metadata.md | 8 +- embeds/cli/help/fluvio-run-sc.md | 5 + embeds/cli/help/fluvio-run-spu.md | 4 + embeds/cli/help/fluvio-run-version.md | 8 +- embeds/cli/help/fluvio-run.md | 2 + embeds/cli/help/fluvio-smartmodule-create.md | 3 + embeds/cli/help/fluvio-smartmodule-delete.md | 3 + embeds/cli/help/fluvio-smartmodule-list.md | 2 + embeds/cli/help/fluvio-smartmodule-watch.md | 2 + embeds/cli/help/fluvio-smartmodule.md | 8 +- embeds/cli/help/fluvio-table-format-create.md | 2 + embeds/cli/help/fluvio-table-format-delete.md | 5 +- embeds/cli/help/fluvio-table-format-list.md | 2 + embeds/cli/help/fluvio-table-format.md | 8 +- embeds/cli/help/fluvio-topic-create.md | 62 ++++++-- embeds/cli/help/fluvio-topic-delete.md | 3 + embeds/cli/help/fluvio-topic-describe.md | 3 + embeds/cli/help/fluvio-topic-list.md | 2 + embeds/cli/help/fluvio-topic.md | 8 +- embeds/cli/help/fluvio-update.md | 20 ++- embeds/cli/help/fluvio-version.md | 12 +- embeds/cli/help/fluvio.md | 6 +- embeds/cli/help/fvm-current.md | 8 + embeds/cli/help/fvm-install.md | 12 ++ embeds/cli/help/fvm-self-install.md | 8 + embeds/cli/help/fvm-self-uninstall.md | 9 ++ embeds/cli/help/fvm-self.md | 13 ++ embeds/cli/help/fvm-show.md | 8 + embeds/cli/help/fvm-switch.md | 11 ++ embeds/cli/help/fvm.md | 18 +++ embeds/cli/help/smdk-build.md | 3 + embeds/cli/help/smdk-generate.md | 127 ++++++++++++++-- embeds/cli/help/smdk-load.md | 2 + embeds/cli/help/smdk-publish.md | 33 +++- embeds/cli/help/smdk-set-public.md | 10 +- embeds/cli/help/smdk-test.md | 14 ++ embeds/cli/help/smdk.md | 3 + embeds/connectors/inbound/http.md | 6 +- embeds/connectors/inbound/kafka.md | 2 +- embeds/connectors/inbound/mqtt.md | 4 +- embeds/connectors/outbound/graphite.md | 143 ++++++++++++++++++ embeds/connectors/outbound/http.md | 6 +- embeds/connectors/outbound/kafka.md | 9 +- embeds/connectors/outbound/sql.md | 58 ++++++- embeds/download-cli/curl-bash-copy.md | 9 ++ embeds/smartmodules/lookback-example.md | 6 +- .../sm-development/regex-filter/src/lib.rs | 4 +- functions/newsletter/package-lock.json | 6 +- functions/newsletter/package.json | 2 +- scripts/fluvio-cms/Cargo.toml | 1 + scripts/fluvio-cms/src/cli/mod.rs | 94 ++++++++++++ scripts/fluvio-cms/src/connectors/mod.rs | 15 +- scripts/fluvio-cms/src/main.rs | 6 +- 157 files changed, 1919 insertions(+), 404 deletions(-) create mode 100644 content/cli/cloud/webhook.md create mode 100644 content/cli/utilities/fluvio-version-manager.md create mode 100644 content/connectors/outbound/graphite.md create mode 100644 content/docs/architecture/auth.md create mode 100644 content/news/this-week-in-fluvio-0057.md create mode 100644 content/smartmodules/deduplication.md create mode 100644 content/smartmodules/smdk/smartmodulerecord.md create mode 100644 embeds/cli/help/fluvio-cloud-secret-delete.md create mode 100644 embeds/cli/help/fluvio-cloud-secret-list.md create mode 100644 embeds/cli/help/fluvio-cloud-secret-set.md delete mode 100644 embeds/cli/help/fluvio-derived-stream-create.md delete mode 100644 embeds/cli/help/fluvio-derived-stream-delete.md delete mode 100644 embeds/cli/help/fluvio-derived-stream-list.md delete mode 100644 embeds/cli/help/fluvio-derived-stream.md delete mode 100644 embeds/cli/help/fluvio-hub-download.md delete mode 100644 embeds/cli/help/fluvio-hub-list.md create mode 100644 embeds/cli/help/fluvio-hub-smartmodule-download.md create mode 100644 embeds/cli/help/fluvio-hub-smartmodule-list.md create mode 100644 embeds/cli/help/fluvio-hub-smartmodule.md create mode 100644 embeds/cli/help/fvm-current.md create mode 100644 embeds/cli/help/fvm-install.md create mode 100644 embeds/cli/help/fvm-self-install.md create mode 100644 embeds/cli/help/fvm-self-uninstall.md create mode 100644 embeds/cli/help/fvm-self.md create mode 100644 embeds/cli/help/fvm-show.md create mode 100644 embeds/cli/help/fvm-switch.md create mode 100644 embeds/cli/help/fvm.md create mode 100644 embeds/connectors/outbound/graphite.md create mode 100644 scripts/fluvio-cms/src/cli/mod.rs diff --git a/.github/workflows/check_examples.yml b/.github/workflows/check_examples.yml index 2b8488eda..5fb92bab3 100644 --- a/.github/workflows/check_examples.yml +++ b/.github/workflows/check_examples.yml @@ -43,10 +43,10 @@ jobs: steps: - name: Dump context uses: crazy-max/ghaction-dump-context@v2.0.0 - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 if: ${{ needs.test-skip-check.outputs.no-skip }} - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 if: ${{ needs.test-skip-check.outputs.no-skip }} with: repository: infinyon/fluvio diff --git a/.github/workflows/htmltest.yml b/.github/workflows/htmltest.yml index 1110a8046..cf1d27df7 100644 --- a/.github/workflows/htmltest.yml +++ b/.github/workflows/htmltest.yml @@ -35,7 +35,7 @@ jobs: - name: Checkout code if: ${{ needs.test-skip-check.outputs.no-skip }} - uses: actions/checkout@v3 + uses: actions/checkout@v4 with: fetch-depth: 1 - name: Setup Hugo diff --git a/.github/workflows/screenshots.yml b/.github/workflows/screenshots.yml index a0b29d20a..5548b4439 100644 --- a/.github/workflows/screenshots.yml +++ b/.github/workflows/screenshots.yml @@ -35,12 +35,12 @@ jobs: run: npm install --global capture-website-cli # Checkout and build files of a base ref. - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 with: ref: ${{ github.event.pull_request.base.sha }} path: current_repo # Checkout and build files of a head ref. - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 with: ref: ${{ github.event.pull_request.head.sha }} clean: false # Prevent removing files in 'path/to/base' folder. diff --git a/.github/workflows/skip-test-check.yml b/.github/workflows/skip-test-check.yml index d97628d39..21247644f 100644 --- a/.github/workflows/skip-test-check.yml +++ b/.github/workflows/skip-test-check.yml @@ -24,14 +24,14 @@ jobs: no-skip: ${{ ! steps.outputs.skip_label_check.result }} steps: - name: Checkout code - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Dump context uses: crazy-max/ghaction-dump-context@v2.0.0 - name: Checkout code for push if: github.event_name == 'push' && github.ref == 'refs/heads/staging' - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Get PR Number from event if: github.event_name == 'pull_request' diff --git a/.github/workflows/update-connector-docs.yml b/.github/workflows/update-connector-docs.yml index 288e8ec54..bdcb337c2 100644 --- a/.github/workflows/update-connector-docs.yml +++ b/.github/workflows/update-connector-docs.yml @@ -1,10 +1,19 @@ name: Connector docs PR +permissions: + contents: read + concurrency: group: connector-docs-${{ github.ref }} cancel-in-progress: true on: + #push: + # branches: + # - staging + # - trying + #pull_request: + # branches: [master] workflow_call: workflow_dispatch: schedule: @@ -17,19 +26,19 @@ jobs: pull-requests: write runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install Fluvio run: | curl -fsS https://hub.infinyon.cloud/install/install.sh | bash echo "$HOME/.fluvio/bin" >> $GITHUB_PATH - - name: Fluvio Login run: fluvio cloud login --email ${{ secrets.CLOUD_USER_EMAIL }} --password ${{ secrets.CLOUD_USER_PASSWORD }} - uses: Swatinem/rust-cache@v2 # Run fluvio cms + - name: Run connector docs update - run: cargo run -- connector --prod + run: cargo run -- connector # Create/Update PR - name: Create Pull Request diff --git a/Cargo.lock b/Cargo.lock index 05cd3f208..31a16ad4f 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -651,7 +651,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "907076dfda823b0b36d2a1bb5f90c96660a5bbcd7729e10727f07858f22c4edc" dependencies = [ "cfg-if", - "hashbrown", + "hashbrown 0.12.3", "lock_api", "once_cell", "parking_lot_core 0.9.7", @@ -795,6 +795,12 @@ dependencies = [ "cfg-if", ] +[[package]] +name = "equivalent" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5443807d6dff69373d433ab9ef5378ad8df50ca6298caf15de6e52e24aaf54d5" + [[package]] name = "errno" version = "0.2.8" @@ -884,6 +890,7 @@ dependencies = [ "lazy_static", "serde", "serde_json", + "serde_yaml", "strum", "tracing", "tracing-subscriber", @@ -1250,6 +1257,12 @@ version = "0.12.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888" +[[package]] +name = "hashbrown" +version = "0.14.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f93e7192158dbcda357bdec5fb5788eebf8bbac027f3f33e719d29135ae84156" + [[package]] name = "heck" version = "0.4.1" @@ -1370,7 +1383,17 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "bd070e393353796e801d209ad339e89596eb4c8d430d18ede6a1cced8fafbd99" dependencies = [ "autocfg", - "hashbrown", + "hashbrown 0.12.3", +] + +[[package]] +name = "indexmap" +version = "2.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8adf3ddd720272c6ea8bf59463c04e0f93d0bbf7c5439b691bca2987e0270897" +dependencies = [ + "equivalent", + "hashbrown 0.14.2", ] [[package]] @@ -2323,11 +2346,11 @@ dependencies = [ [[package]] name = "serde_yaml" -version = "0.9.21" +version = "0.9.25" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d9d684e3ec7de3bf5466b32bd75303ac16f0736426e5a4e0d6e489559ce1249c" +checksum = "1a49e178e4452f45cb61d0cd8cebc1b0fafd3e41929e996cef79aa3aca91f574" dependencies = [ - "indexmap", + "indexmap 2.0.2", "itoa", "ryu", "serde", @@ -2793,7 +2816,7 @@ version = "0.19.10" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2380d56e8670370eee6566b0bfd4265f65b3f432e8c6d85623f728d4fa31f739" dependencies = [ - "indexmap", + "indexmap 1.9.3", "serde", "serde_spanned", "toml_datetime", @@ -3060,7 +3083,7 @@ version = "0.104.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6a396af81a7c56ad976131d6a35e4b693b78a1ea0357843bd436b4577e254a7d" dependencies = [ - "indexmap", + "indexmap 1.9.3", "url", ] diff --git a/bors.toml b/bors.toml index 0665dc987..a2ab7a99f 100644 --- a/bors.toml +++ b/bors.toml @@ -1,5 +1,6 @@ status = [ - "Pass client examples", "Pass html lint" + #"Pass client examples", + "Pass html lint" ] use_squash_merge = true delete_merged_branches = true diff --git a/content/cli/advanced/cli_channels.md b/content/cli/advanced/cli_channels.md index 07911f839..69f36128d 100644 --- a/content/cli/advanced/cli_channels.md +++ b/content/cli/advanced/cli_channels.md @@ -4,6 +4,20 @@ menu: Release Channels weight: 10 --- +The current version of release channels was introduced in 0.10.16 with the introduction of [`fvm`]({{}}) + + +{{}} +This is currently in BETA + +%copy first-line% +```shell +$ https://hub.infinyon.cloud/install_fvm/install_fvm.sh | bash +``` +{{}} + + + ## What is this for? Occasionally users report issues in our Discord. After a fix was found and committed, users wanted a way to verify their issue had been resolved before a release. @@ -16,12 +30,20 @@ Inspired by [Rust's concept of channels](https://rust-lang.github.io/rustup/conc ## Installation -Release channel support is provided by the [official installer script]({{}}). The installer uses the `stable` channel by default. +Release channel support is provided by `fvm`. The installer uses the `stable` channel by default. + +%copy first-line% +```shell +$ https://hub.infinyon.cloud/install_fvm/install_fvm.sh | bash +``` + {{}} -Fluvio release channels were introduced for the CLI in `0.9.16`. +Fluvio release channels were introduced for the CLI in `0.9.16`, and the most recent implementation was introduced in `0.10.16`. +

+If you have an installation from a release earlier than `0.10.16`, you should delete `~/.fluvio` directory and re-install with the [official installer script]({{}}) to install `fvm`, the Fluvio Version Manager CLI.

-If you have an installation from a release earlier than `0.9.16`, you should re-install with the [official installer script]({{}}) to install the Fluvio channel frontend. +You can also run `fvm install` to migrate your existing installation {{
}} ## Quick start: The Channels @@ -35,7 +57,7 @@ To switch to the `stable` channel: %copy first-line% ```shell -$ fluvio version switch stable +$ fvm switch stable ``` ### Latest @@ -45,7 +67,7 @@ To switch to the `latest` channel: %copy first-line% ```shell -$ fluvio version switch latest +$ fvm switch latest ``` The first time you switch to this channel, the binary will be downloaded. @@ -59,49 +81,14 @@ At this step the binary will get downloaded. %copy first-line% ```shell -$ fluvio version create X.Y.Z +$ fvm install X.Y.Z ``` Then you can switch to the version channel %copy first-line% ```shell -$ fluvio version switch X.Y.Z +$ fvm switch X.Y.Z ``` -Where `X.Y.Z` is the version of a release you want to switch to (e.g. `0.9.18`) - -## How it works? - -Release channel support is set up at install time through the [installer script]({{}}). - -2 binaries are installed -* The Fluvio CLI -* A frontend binary to support switching channels - -By default, the `stable` channel is selected. - -This channel is registered in `~/.fluvio/channel` and is marked as the active channel. - -Example channel config: - -```toml -current_channel = "stable" -[channel.stable] -binary_location = "/home/username/.fluvio/bin/fluvio-stable" -extensions = "/home/username/.fluvio/extensions" -image_tag_strategy = "Version" - -[channel.latest] -binary_location = "/home/username/.fluvio/bin/fluvio-latest" -extensions = "/home/username/.fluvio/extensions-latest" -image_tag_strategy = "VersionGit" -``` - -When you run `fluvio update`, the Fluvio binary of your current channel will update to the newest version of the channel. - -{{}} -Only the `stable` and `latest` channels can be updated. Version channels don't support `fluvio update` because those binaries will always be pinned to its version. -

-Follow the [version channel]({{}}) steps to switch to a different version. -{{
}} +Where `X.Y.Z` is the version of a release you want to switch to (e.g. `0.9.18`) \ No newline at end of file diff --git a/content/cli/cloud/webhook.md b/content/cli/cloud/webhook.md new file mode 100644 index 000000000..16c06a8af --- /dev/null +++ b/content/cli/cloud/webhook.md @@ -0,0 +1,94 @@ +--- +title: Cloud Webhook +menu: Webhook +weight: 30 +--- + +The `fluvio cloud webhook` family of commands is used to create, delete, and manage webhook urls in cloud. + +%copy first-line% +```bash +$ fluvio cloud webhook -h +``` + +{{% inline-embed file="embeds/cli/help/fluvio-cloud-webhook.md" %}} + +--- + +## `fluvio cloud webhook create` + +This command is used to create a new webhook url that maps to the current user's topic + +%copy first-line% +```bash +$ fluvio cloud webhook create -h +``` + +{{% inline-embed file="embeds/cli/help/fluvio-cloud-webhook-create.md" %}} + +Example usage: + +%copy first-line% +```bash +$ fluvio cloud webhook create my-webhook-1 --topic my-topic +``` + +--- + +## `fluvio cloud webhook delete` + +This command deletes a webhook url for the current user + +%copy first-line% +```bash +$ fluvio cloud webhook delete -h +``` + +{{% inline-embed file="embeds/cli/help/fluvio-cloud-webhook-delete.md" %}} + +Example usage: + +%copy first-line% +```bash +$ fluvio cloud webhook delete my-webhook-1 +``` + +--- + +## `fluvio cloud webhook list` + +Command to show the webhooks associated with current user. + +%copy first-line% +```bash +$ fluvio cloud webhook list -h +``` + +{{% inline-embed file="embeds/cli/help/fluvio-cloud-webhook-list.md" %}} + +Example usage: + +%copy first-line% +```bash +$ fluvio cloud webhook list +``` + +--- + +## `fluvio cloud webhook update` + +Command to show update mapping details of a webhook url + +%copy first-line% +```bash +$ fluvio cloud webhook update -h +``` + +{{% inline-embed file="embeds/cli/help/fluvio-cloud-webhook-update.md" %}} + +Example usage: + +%copy first-line% +```bash +$ fluvio cloud webhook update --topic different-topic my-webhook +``` \ No newline at end of file diff --git a/content/cli/smartmodules/hub.md b/content/cli/smartmodules/hub.md index 342dd7d9a..b495bd150 100644 --- a/content/cli/smartmodules/hub.md +++ b/content/cli/smartmodules/hub.md @@ -6,32 +6,32 @@ weight: 20 ## Introduction to the SmartModule Hub Service -The SmartModule Hub allows users to start building their data streaming pipeline with SmartModules which have been uploaded into the Hub as ready to use packages. The Hub allows Fluvio pipeline construction focusing on the business logic by providing already built, reliable and tested modules. The modules contain high level configurable inputs which can be applied without needing to rebuild the module. Pipelines can employ a mix of Hub downloaded packages as well as custom developed SmartModules. +The SmartModule Hub offers ready-to-use packages for easy Fluvio pipeline construction, letting users focus on business logic. It provides tested modules with high-level configurable inputs, supporting a mix of Hub packages and custom SmartModules without requiring module rebuilds. -A view of available and installed SmartModules is accessible from both the `fluvio` cli as well as on the Fluvio cloud dashboard. +A view of available and installed SmartModules is accessible from both the `fluvio` cli. -## Fluvio Hub Commands +## Fluvio Hub Smartmodule Commands -Commands for users to interact with the SmartModule hub are under the **fluvio hub** set of commands. +Commands for users to interact with the SmartModule hub are under the **fluvio hub smartmodule** set of commands. -- `fluvio hub download group/package@version` -- `fluvio hub list` +- `fluvio hub smartmodule download group/package@version` +- `fluvio hub smartmodule list` -### `fluvio hub download` +### `fluvio smartmodule hub download` -{{% inline-embed file="embeds/cli/help/fluvio-hub-download.md" %}} +{{% inline-embed file="embeds/cli/help/fluvio-hub-smartmodule-download.md" %}} -The command **fluvio hub download ** will download a hub smartmodule into your cluster so it can be configured and used in your pipeline. +The command **fluvio hub smartmodule download ** will download a hub smartmodule into your cluster so it can be configured and used in your pipeline. %copy first-line% ```bash -$ fluvio hub download infinyon/egex-filter2@0.1.0 +$ fluvio hub smartmodule download infinyon/regex-filter2@0.1.0 downloading infinyon/regex-filter2@0.1.0 to infinyon-regex-filter2-0.1.0.ipkg ... downloading complete ... checking package -trying connection to fluvio router.dev.infinyon.cloud:9003 +trying connection to fluvio router.infinyon.cloud:9003 ... cluster smartmodule install complete ``` @@ -45,28 +45,27 @@ $ flvio sm list infinyon/regex-filter@0.1.0 316.1 KB ``` -Once a SmartModule is download, it can be configured and applied in a pipleine. See the SmartModule page for more details on how work with SmartModules once they are downloaded in your cluster. +Once a SmartModule is download, it can be configured and applied in a pipeline. See the SmartModule page for more details on how work with SmartModules once they are downloaded in your cluster. -### `fluvio hub list` +### `fluvio hub smartmodule list` -The command **fluvio hub list** will list SmartModules and versions available for download +The command **fluvio hub smartmodule list** will list SmartModules and versions available for download -{{% inline-embed file="embeds/cli/help/fluvio-hub-list.md" %}} +{{% inline-embed file="embeds/cli/help/fluvio-hub-smartmodule-list.md" %}} The list command shows Smartmodules by **group/package@version** in the SMARTMODULE column. The Visibility column describes if the package is accessible to public or private views. If package visibility is **private** only your user login can list or download the package. Packages uploaded by InfinyOn will be in the infinyon group. %copy first-line% ```bash -$ fluvio hub list +$ fluvio hub smartmodule list SMARTMODULE Visibility mypriv/foo-priv@0.1.1 private infinyon/jolt@0.1.0 public infinyon/json-sql@0.1.0 public infinyon/regex-filter@0.1.0 public -... ``` @@ -75,19 +74,15 @@ $ fluvio hub list SmartModule Packages are listed in the hub as in the form of `/@version`. -Group `infinyon` packages are published by InfinyOn, the maker of Fluvio. Third-party developers may also upload public packages which appear in other groups. The Hub service enforces signing and publishing restrictions on uploaded packages. Once a signed package of given group has been published, only the group owner may update that package or add new packages within that group. +Packages in the infinyon group are published by InfinyOn, the creators of Fluvio. Third-party developers can upload public packages under different groups. The Hub enforces signing and publishing restrictions on uploaded packages. Once a signed package is published in a group, only the group owner can update it or add new packages within that group. ### Private and Public Packages -Published packages are by default private. In order to make a published package publicly available, the package should be set to public. The cli method to do this is below: +By default, published packages are private. To make a package public, set it to public using the following CLI commands: ```bash cd smartmodule-dev-dir smdk set-public ``` -After the package is set to public, any publish will ask to verify the publish operation. Once a package version is publicly published, the package cannot revoked. - - - - +Once set to public, you'll be prompted for verification on every publish. Note that once a package version is publicly published, it cannot be revoked. \ No newline at end of file diff --git a/content/cli/utilities/fluvio-version-manager.md b/content/cli/utilities/fluvio-version-manager.md new file mode 100644 index 000000000..2581d39c5 --- /dev/null +++ b/content/cli/utilities/fluvio-version-manager.md @@ -0,0 +1,32 @@ +--- +title: Fluvio Version Manager +menu: FVM +weight: 05 +toc: false +--- + +`fvm` manages multiple installations of the fluvio toolchain. Users select a [release channel]({{}}) of fluvio/cdk/smdk, and `fvm` takes care of the rest. + +## `fvm` +{{% inline-embed file="embeds/cli/help/fvm.md" %}} + +## `fvm current` +{{% inline-embed file="embeds/cli/help/fvm-current.md" %}} + +## `fvm self` +{{% inline-embed file="embeds/cli/help/fvm-self.md" %}} + +## `fvm self install` +{{% inline-embed file="embeds/cli/help/fvm-self-install.md" %}} + +## `fvm self uninstall` +{{% inline-embed file="embeds/cli/help/fvm-self-uninstall.md" %}} + +## `fvm install` +{{% inline-embed file="embeds/cli/help/fvm-install.md" %}} + +## `fvm show` +{{% inline-embed file="embeds/cli/help/fvm-show.md" %}} + +## `fvm switch` +{{% inline-embed file="embeds/cli/help/fvm-switch.md" %}} diff --git a/content/cli/utilities/install.md b/content/cli/utilities/install.md index d84cc105b..f9736d912 100644 --- a/content/cli/utilities/install.md +++ b/content/cli/utilities/install.md @@ -1,9 +1,14 @@ --- title: Install +menu: Install (deprecated) weight: 10 toc: false --- +{{}} +This command is deprecated since 0.10.16, and has been replaced by [`fvm install`]({{}}) +{{}} + Some Fluvio CLI commands are distributed as separate executables that we call "extensions". This command installs and updates extensions by name, placing them in the `~/.fluvio/extensions/` directory. diff --git a/content/connectors-old/examples/github.md b/content/connectors-old/examples/github.md index 6885ae334..00f28e735 100644 --- a/content/connectors-old/examples/github.md +++ b/content/connectors-old/examples/github.md @@ -262,11 +262,11 @@ Copy and paste this code into the `src/lib.rs` file of the project. %copy% ```rust -use fluvio_smartmodule::{smartmodule, Record, RecordData, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, RecordData, Result}; use serde_json::{json, Value as JsonValue}; #[smartmodule(map)] -fn shaper(record: &Record) -> Result<(Option, RecordData)> { +fn shaper(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { let repo: JsonValue = serde_json::from_slice(record.value.as_ref())?; let shaped = json!({ @@ -569,11 +569,11 @@ Copy and paste this code into the `src/lib.rs` file of the project. %copy% ```rust -use fluvio_smartmodule::{smartmodule, Record, RecordData, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, RecordData, Result}; use serde_json::{json, Value as JsonValue}; #[smartmodule(map)] -fn shaper(record: &Record) -> Result<(Option, RecordData)> { +fn shaper(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { let repo: JsonValue = serde_json::from_slice(record.value.as_ref())?; let shaped = json!({ diff --git a/content/connectors-old/how-to/connectors-with-smartmodules.md b/content/connectors-old/how-to/connectors-with-smartmodules.md index 651abdb57..64a022921 100644 --- a/content/connectors-old/how-to/connectors-with-smartmodules.md +++ b/content/connectors-old/how-to/connectors-with-smartmodules.md @@ -77,11 +77,11 @@ to have the following contents: %copy% ```rust -use fluvio_smartmodule::{smartmodule, Result, Record, RecordData}; +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord, RecordData}; use serde_json::Value; #[smartmodule(map)] -pub fn map(record: &Record) -> Result<(Option, RecordData)> { +pub fn map(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { let input: Value = serde_json::from_slice(record.value.as_ref())?; let fact = &input["fact"]; let output = serde_json::to_string(fact)?; diff --git a/content/connectors/outbound/graphite.md b/content/connectors/outbound/graphite.md new file mode 100644 index 000000000..41ba7b10c --- /dev/null +++ b/content/connectors/outbound/graphite.md @@ -0,0 +1,5 @@ +--- +menu: Graphite +--- + +{{% inline-embed file="embeds/connectors/outbound/graphite.md" %}} \ No newline at end of file diff --git a/content/docs/architecture/auth.md b/content/docs/architecture/auth.md new file mode 100644 index 000000000..54609786b --- /dev/null +++ b/content/docs/architecture/auth.md @@ -0,0 +1,133 @@ +--- +title: Authentication and Authorization +weight: 60 +--- + + +When the user logs in, the client downloads a profile and extracts the user identification from the client certificate issued by the server. The authorization is driven by simple RBAC (role-based access control), which can be extended for other more advanced authorization schemes in the future. + +# Cluster setup TLS + +Fluvio cluster must have TLS enabled for the authorization mechanism to kick in. With TLS enabled, both SC and SPU will ensure all subsequent traffic to and from the client is encrypted via mTLS. As part of TLS configuration, the cluster administrator must also define an authorization policy file (described below). + +To create a Fluvio cluster with TLS enabled, you must provide the following certs and keys: +- CA cert +- Client cert and key +- Server cert and key +- Authorization policy file + +For development and testing, you can use self-signed certs and keys. For production, you should use certs and keys signed by a trusted CA. It is important to note that the client cert and key must be signed by the same CA as the server cert and key. Otherwise, the client will not be able to connect to the server. In addition, the system admin must specify domain name (DNS) that cluster is bound to. This is required for TLS to work properly. + +## Self signed certs + +For self-signed certs, you can use sample certs templates in [fluvio](https://github.com/infinyon/fluvio/tree/master/tls). + +## Authorization policy file + +Fluvio currently implements a simple type level permission model. The authorization policy file is a JSON file that contains a list of permission entries for the role. Each entry contains a type and a list of permissions. Tye type can be one of the following object types. Current list of types are: +- CustomSpu +- Spu +- Topic +- Partition +- SpuGroup +- SmartModule +- TableFormat + +Each object type can have one or following permissions: +- All +- Read +- Write + +The `All` permission means the user has all permissions for the object type. For example, following is a policy file. It contains a two roles: `Root` and `User`. The `Root` role has all permissions for all object types. The `TopicUser` role only has read permission for `Topic` object type. + +```json +{ + "Root": { + "CustomSpu": [ + "All" + ], + "Partition": [ + "All" + ], + "Spu": [ + "All" + ], + "Topic": [ + "All" + ], + "SpuGroup": [ + "All" + ], + "ManagedConnector": [ + "All" + ], + "SmartModule": [ + "All" + ], + "TableFormat": [ + "All" + ] + }, + "TopicUser": { + "Topic": [ + "Read" + ] + } + +} +``` + +Note that, In order to set up and operator cluster as an adminstrator, "Root" role with full access to all objects is required. + +## Authorization Scope file + +The scope file contains mapping of users to roles. For example, here is a sample scope file that maps user `user1` and `user2` to `TopicUser` role. The `Root` role is mapped to user `root`. + +```json +{ + "Root": [ + "root" + ], + "TopicUser": [ + "user1", + "user2" + ] +} +``` + +The user id comes from the X509 client certs. It is mapped to `CN`. For example, this is a simple command to generate a client cert with `CN` set to `user1`. + +```bash +openssl req -new -key -out -subj "/C=US/ST=CA/O=MyOrg, Inc./CN=user1" +``` + +The client cert with `CN` set to `user1` can be used to connect to Fluvio cluster. The Fluvio cluster will map the `user1` to `TopicUser` role. + +## Provisioning Fluvio cluster with TLS + +Once all the certs and keys are ready, you can provision a Fluvio cluster with TLS enabled. Note that client certs must have user "root" (default policy mapping) in order to provision the cluster. Here is syntax to provision a Fluvio cluster with TLS enabled. + +```bash +fluvio cluster start --tls --domain --ca-cert ca_cert --server-cert --server-key --client-cert --client-key --authorization-policy --authorization-scope +``` + +The domain must match DNS suffice in order to TLS to work properly. For example, if you are using self-signed certs, you can use `local` or `localhost` as domain. If you are using certs signed by a trusted CA, you can use the domain name that is registered with the CA. + +# Connecting to Fluvio cluster with TLS + +In order to connect to Fluvio cluster with TLS enabled, you must provide client certs and keys. They must be signed by the same CA as the server cert and key. In addition, it must have CN(user) that is in the authorization scope file. The client TLS certs and key can be either provided in the fluvio profile file or specified in the env variables. Here is an section of fluvio profile file that contains client certs and keys. Currently TLS certs must be manually copied to config file. + + +```toml +[cluster.mycluster.tls.certs] +domain = "fluvio-cluster.mydomain.com" +key = """ + +-----BEGIN RSA PRIVATE KEY----- +MIIJKQIBAAKCAgEAtLx58BuGSbwWW6AlNb38965IWQv+IJZGPBzMs9GExohuHOai +... +``` + + + + diff --git a/content/docs/architecture/replica-assignment.md b/content/docs/architecture/replica-assignment.md index c78cb81fb..47890f06b 100644 --- a/content/docs/architecture/replica-assignment.md +++ b/content/docs/architecture/replica-assignment.md @@ -209,7 +209,7 @@ The following command creates a topic from a **replica assignment file**: %copy first-line% ```bash -$ fluvio topic create --topic custom-topic --replica-assignment ./my-assignment +$ fluvio topic create custom-topic --replica-assignment ./my-assignment ``` _Validate-only_ flag is available to verify a replica assignment file without applying any changes. @@ -244,4 +244,4 @@ The **replica map** definition meet the following criteria: For additional information on how to check the result of a replica assignment file, checkout [Topics CLI]. -[Topics CLI]: {{< ref "/cli/commands/topic" >}} \ No newline at end of file +[Topics CLI]: {{< ref "/cli/commands/topic" >}} diff --git a/content/docs/concepts/topics.md b/content/docs/concepts/topics.md index 6e634c1ba..6551fc4b0 100644 --- a/content/docs/concepts/topics.md +++ b/content/docs/concepts/topics.md @@ -20,7 +20,7 @@ Replicas have a leader and one or more followers and distributed across all avai For example, when provisioning a topic with **2** partitions and **3** replicas: ```bash -$ fluvio topic create --topic topic-a --partitions 2 --replication 3 +$ fluvio topic create topic-a --partitions 2 --replication 3 ``` **Leaders** maintain the primary data set and **followers** store a copy of the data. Leader and follower replications are assigned to independent **SPUs**: diff --git a/content/docs/kubernetes/external-access.md b/content/docs/kubernetes/external-access.md index dfa181bec..e7bb1fd1d 100644 --- a/content/docs/kubernetes/external-access.md +++ b/content/docs/kubernetes/external-access.md @@ -13,7 +13,7 @@ Port numbers do not change based on whether TLS is enabled, the same ports are u Service behavior can be configured at installation by overriding values in the helm charts. -### Service Enpoint Details +### Service Endpoint Details Run `kubectl describe svc fluvio-sc-public` to view address and port information for accessing the SC public endpoint. diff --git a/content/news/this-week-in-fluvio-0057.md b/content/news/this-week-in-fluvio-0057.md new file mode 100644 index 000000000..dc36a06af --- /dev/null +++ b/content/news/this-week-in-fluvio-0057.md @@ -0,0 +1,45 @@ +--- +title: "This Week in Fluvio #57" +date: 2023-08-08 +weight: 20 +--- +Fluvio is a distributed, programmable streaming platform written in Rust. + +--- +Last week we released Fluvio 0.10.14 with exciting updates! + +We had 4 conversations with lead data engineers and architects to discuss their data pipelines. We have talked to over 14 data engineers and architects in the last couple of weeks and these have been awesome. If you'd like to a have a conversation about your data pipelines and discuss problems, validate ideas - email me at drc@infinyon.com + +## Latest release +* We released topic level deduplication which enables us to implement exactly once semantics on topics and deduplicate based on keys. +* We have also released timestamp injection in smartmodule context. +* Full [changelog is available here](https://github.com/infinyon/fluvio/blob/v0.10.14/CHANGELOG.md) + +## Upcoming features +* We are produtizing a lean binary that runs on top of edge devices with minimum memory and storage with caching and mirroring to ensure data delivery from edge to cloud without losing data. +* We are also building out the foundations for multi region deployment to support users in the EU +* This is a way for us to get hybrid deployments and a single binary install of our stream processing engine. +* Upcoming blog on deduplication is in progress + +## Open positions +We are hiring! +* [Sr. Rust Cloud Software Engineer](https://infinyon.com/careers/cloud-engineer-senior-level/) + +## New blog post +* [Deduplicate data streaming events with SQL Upsert]('https://infinyon.com/blog/2023/07/sql-upsert/') + +## New video +* [Collecting Hackernews RSS Feeds using InfinyOn Cloud]('https://youtu.be/fVzLoaIHQfM') + +That's all folks. until next week. + +--- + +Get in touch with us on [Github Discussions] or join [our Discord channel] and come say hello! + +For the full list of changes this week, be sure to check out [our CHANGELOG]. + +[Fluvio open source]: https://github.com/infinyon/fluvio +[our CHANGELOG]: https://github.com/infinyon/fluvio/blob/master/CHANGELOG.md +[our Discord channel]: https://discordapp.com/invite/bBG2dTz +[Github Discussions]: https://github.com/infinyon/fluvio/discussions \ No newline at end of file diff --git a/content/smartmodules/analytics/aggregate.md b/content/smartmodules/analytics/aggregate.md index 077e96eff..fe695b4e2 100644 --- a/content/smartmodules/analytics/aggregate.md +++ b/content/smartmodules/analytics/aggregate.md @@ -59,10 +59,10 @@ $ cd example-aggregate && cat ./src/lib.rs ``` ```rust -use fluvio_smartmodule::{smartmodule, Result, Record, RecordData}; +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord, RecordData}; #[smartmodule(aggregate)] -pub fn aggregate(accumulator: RecordData, current: &Record) -> Result { +pub fn aggregate(accumulator: RecordData, current: &SmartModuleRecord) -> Result { // Parse the accumulator and current record as strings let accumulator_string = std::str::from_utf8(accumulator.as_ref())?; let current_string = std::str::from_utf8(current.value.as_ref())?; diff --git a/content/smartmodules/deduplication.md b/content/smartmodules/deduplication.md new file mode 100644 index 000000000..85c2b00c7 --- /dev/null +++ b/content/smartmodules/deduplication.md @@ -0,0 +1,74 @@ +--- +title: Deduplication +--- + +The `Deduplication` feature in Fluvio is the way to deduplicate records based on their keys. + +Deduplication can be used on a topic. +It will drop the duplicate records in a window which is defined by the configured `bounds`. + +The current supported `bounds` are `age` and `count`, they are explained in the [bounds section](#bounds). + +## Behavior + +The deduplication function is deterministic and survives restarts. During restart the deduplication algorithm scans the data stream and rebuilds the memory object. + +## Example topic config + +Example configuration on topic: + +%copy% +```yaml +# topic.yaml +version: 0.1.0 +meta: + name: topic-with-dedup +deduplication: + bounds: + count: 5 # remember at least 5 last records + age: 5s # remember records for at least 5 seconds + filter: + transform: + uses: infinyon-labs/dedup-filter@0.0.2 +``` +A topic can be created using this config file like so: + +%copy first-line% +```bash +$ fluvio topic create -c topic.yaml +``` + +After creating the topic, it can be tested like so: + +%copy first-line% +```bash +$ fluvio produce topic-with-dedup --key-separator : +1:2 +1:2 +2:5 +``` + +%copy first-line% +```bash +$ fluvio consume -B topic-with-dedup +2 +5 +``` + +## Bounds + +| Parameter | default | type | optional | description | +|:-------------|:--------| :--- | :--- |:------------------------------------------------------| +| count | - | Integer | false | Minimum number of records the filter will remember. It doesn't guarantee to remember records that came `count` records before now. | +| age | - | Integer | true | Minimum amount of time this filter will remember a record for. It can be specified using this format: `15days 2min 2s`, or `2min 5s`, or `15ms` | | + +## Implementation + +The specific algorithm used for deduplication is defined by a smartmodule. Right now the only smartmodule for this is the `dedup-filter` smartmodule. + +The `dedup-filter` splits the data into smaller chunks and keeps these chunks in memory. Each chunk has an age so it knows if it can +drop a chunk based on it's age. + +When count of total records in memory exceeds the configured `bounds.count`, it will check if it can delete the oldest record chunk using the configured `bounds.age` and delete it if it can. This allows for fast deletion of old data with minimum amount of bookkeeping. + +The chunking of data introduces a small memory usage overhead but allows the filter to have no spikes in execution time and memory usage. diff --git a/content/smartmodules/lookback.md b/content/smartmodules/lookback.md index 71d1145ce..b5763abd0 100644 --- a/content/smartmodules/lookback.md +++ b/content/smartmodules/lookback.md @@ -10,16 +10,20 @@ To activate Lookback SmartModule must fit the following criteria: - A function annotated with `#[smartmodule(look_back)]` macro is present in the code. This function is used to pass requested records to SmartModule: ```rust #[smartmodule(look_back)] -pub fn look_back(record: &Record) -> Result<()> { +pub fn look_back(record: &SmartModuleRecord) -> Result<()> { ... } ``` -- `lookback` parameter is specified in transform configuration. It defines how many lookback records SmartModule receives: +- `lookback` parameter is specified in transform configuration. It defines a set of lookback records that SmartModule receives. Supported ways to define the set: + - by size (last N records existing in the topic) + - by age (records that are younger than the specified age) + - by size and age (max N records younger than the specified age) ```yaml transforms: - uses: local/filter-with-lookback@0.1.0 lookback: - last: 10 # we want last 10 records + age: 30m # we want only records that are younger than 30 minutes + last: 10 # we want maximum of 10 records (last) ``` --- If Fluvio topic is empty, `look_back` is never called. diff --git a/content/smartmodules/smdk/build-test.md b/content/smartmodules/smdk/build-test.md index e407cd520..b7288ef00 100644 --- a/content/smartmodules/smdk/build-test.md +++ b/content/smartmodules/smdk/build-test.md @@ -1,7 +1,7 @@ --- title: SMDK - Build & Test a SmartModule menu: Build & Test -weight: 40 +weight: 50 toc: false --- diff --git a/content/smartmodules/smdk/generate.md b/content/smartmodules/smdk/generate.md index 82c69f74e..f26fdc9f8 100644 --- a/content/smartmodules/smdk/generate.md +++ b/content/smartmodules/smdk/generate.md @@ -53,10 +53,10 @@ $ tree This is a simple SmartModule `filter` matching for all data records that contains letter `a`: ```bash -use fluvio_smartmodule::{smartmodule, Result, Record}; +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord}; #[smartmodule(filter)] -pub fn filter(record: &Record) -> Result { +pub fn filter(record: &SmartModuleRecord) -> Result { let string = std::str::from_utf8(record.value.as_ref())?; Ok(string.contains('a')) } diff --git a/content/smartmodules/smdk/load.md b/content/smartmodules/smdk/load.md index 0d49b000b..53a2fbe53 100644 --- a/content/smartmodules/smdk/load.md +++ b/content/smartmodules/smdk/load.md @@ -1,7 +1,7 @@ --- title: SMDK - Load to Cluster menu: Load -weight: 50 +weight: 60 toc: false --- diff --git a/content/smartmodules/smdk/publish.md b/content/smartmodules/smdk/publish.md index ac04a4819..be685f454 100644 --- a/content/smartmodules/smdk/publish.md +++ b/content/smartmodules/smdk/publish.md @@ -1,7 +1,7 @@ --- title: SMDK - Publish to Hub menu: Publish -weight: 60 +weight: 70 toc: false --- diff --git a/content/smartmodules/smdk/smartmodulerecord.md b/content/smartmodules/smdk/smartmodulerecord.md new file mode 100644 index 000000000..f972f4ba5 --- /dev/null +++ b/content/smartmodules/smdk/smartmodulerecord.md @@ -0,0 +1,58 @@ +--- +title: SMDK - SmartModuleRecord +menu: SmartModuleRecord +weight: 40 +toc: false +--- + +Inside the SmartModule Context, Fluvio `Record` becomes `SmartModuleRecord`. + +`SmartModuleRecord` is a wrapper on the standard `Record`, which provides extra +details on the `Record` such as the moment when the `Record` was produced. + +-> `SmartModuleRecord` was introduced on [Fluvio v0.10.14][1], we recommend upgrading to `SmartModuleRecord` as future versions will completely deprecate `Record` in the SmartModule Context. + +## Usage + +When building any type of SmartModules you will have access to each record, in +the SmartModule function these records are represented with the +`SmartModuleRecord` struct. + +Consider the following SmartModule of type `Map`, which processes weather data: + +```rust +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord, RecordData}; + +#[smartmodule(map)] +pub fn map(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { + let string = std::str::from_utf8(record.value.as_ref())?; // parses UTF-8 string from bytes + let feels_like = string.parse::()?; // attemps to parse the string value into a `i32` + let feels_like = feels_like.to_string(); // turs the integer into a string + + Ok((key, feels_like.into())) // returs to the stream +} +``` + +For weather casting it would be helpful to also have access to the moment when +the data was captured. We could have our wather value mapped to a string which +also holds a timestamp. + +```diff +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord, RecordData}; + +#[smartmodule(map)] +pub fn map(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { + let string = std::str::from_utf8(record.value.as_ref())?; // parses UTF-8 string from bytes + let feels_like = string.parse::()?; // attemps to parse the string value into a `i32` +- let feels_like = feels_like.to_string(); // turs the integer into a string ++ let feels_like_with_timestamp = format!("{}_{}", record.timestamp(), feels_like); + + Ok((key, feels_like_with_timestamp.into())) // returs to the stream +} +``` + +With this small change we are injecting insightful data to our record. This is +a simple use case but we can have more based on the implementation and business +requirements. + +[1]: https://github.com/infinyon/fluvio/blob/master/CHANGELOG.md#platform-version-01014---2023-07-28 diff --git a/content/smartmodules/transform/array-map.md b/content/smartmodules/transform/array-map.md index e7380b322..c0bbd123e 100644 --- a/content/smartmodules/transform/array-map.md +++ b/content/smartmodules/transform/array-map.md @@ -70,10 +70,10 @@ $ cd array-map && cat src/lib.rs ``` ```rust -use fluvio_smartmodule::{smartmodule, Record, RecordData, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, RecordData, Result}; #[smartmodule(array_map)] -pub fn array_map(record: &Record) -> Result, RecordData)>> { +pub fn array_map(record: &SmartModuleRecord) -> Result, RecordData)>> { // Deserialize a JSON array with any kind of values inside let array: Vec = serde_json::from_slice(record.value.as_ref())?; diff --git a/content/smartmodules/transform/filter-map.md b/content/smartmodules/transform/filter-map.md index aebb188f2..a6b18f274 100644 --- a/content/smartmodules/transform/filter-map.md +++ b/content/smartmodules/transform/filter-map.md @@ -83,7 +83,7 @@ Now, let's jump right into the code. Copy and paste the following block into the %copy% ```rust -use fluvio_smartmodule::{smartmodule, Record, RecordData, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, RecordData, Result}; use serde::{Deserialize, Serialize}; /// Events that may take place in an online grocery service @@ -104,7 +104,7 @@ enum GroceryEvent { } #[smartmodule(filter_map)] -fn filter_map(record: &Record) -> Result, RecordData)>> { +fn filter_map(record: &SmartModuleRecord) -> Result, RecordData)>> { let event: GroceryEvent = match serde_json::from_slice(record.value.as_ref()) { Ok(event) => event, Err(_) => return Ok(None), // Skip if we fail to parse JSON diff --git a/content/smartmodules/transform/filter.md b/content/smartmodules/transform/filter.md index 08394e012..7e903fe14 100644 --- a/content/smartmodules/transform/filter.md +++ b/content/smartmodules/transform/filter.md @@ -95,10 +95,10 @@ $ cd json-filter && cat ./src/lib.rs ``` ```rust -use fluvio_smartmodule::{smartmodule, Result, Record}; +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord}; #[smartmodule(filter)] -pub fn filter(record: &Record) -> Result { +pub fn filter(record: &SmartModuleRecord) -> Result { let string = std::str::from_utf8(record.value.as_ref())?; Ok(string.contains('a')) } @@ -146,10 +146,10 @@ Now, let's write the logic for our filter. We'll start by parsing our raw data i instances of `StructuredLog`. ```rust -use fluvio_smartmodule::{smartmodule, Record, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, Result}; #[smartmodule(filter)] -fn filter(record: &Record) -> Result { +fn filter(record: &SmartModuleRecord) -> Result { let log: StructuredLog = serde_json::from_slice(record.value.as_ref())?; todo!() @@ -165,7 +165,7 @@ will be a piece of cake! Let's look at all the code for the finished filter. %copy% ```rust -use fluvio_smartmodule::{smartmodule, Record, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, Result}; #[derive(PartialEq, Eq, PartialOrd, Ord, serde::Deserialize)] #[serde(rename_all = "lowercase")] @@ -184,7 +184,7 @@ struct StructuredLog { } #[smartmodule(filter)] -fn filter(record: &Record) -> Result { +fn filter(record: &SmartModuleRecord) -> Result { let log: StructuredLog = serde_json::from_slice(record.value.as_ref())?; // We keep records that are "greater than" debug diff --git a/content/smartmodules/transform/map.md b/content/smartmodules/transform/map.md index 37165fce0..7b78a7c04 100644 --- a/content/smartmodules/transform/map.md +++ b/content/smartmodules/transform/map.md @@ -56,10 +56,10 @@ $ cd map-example && cat ./src/lib.rs %copy% ```rust -use fluvio_smartmodule::{smartmodule, Result, Record, RecordData}; +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord, RecordData}; #[smartmodule(map)] -pub fn map(record: &Record) -> Result<(Option, RecordData)> { +pub fn map(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { let key = record.key.clone(); let string = std::str::from_utf8(record.value.as_ref())?; diff --git a/data/cli-commands.yml b/data/cli-commands.yml index dfa4ffd17..9d6a923c7 100644 --- a/data/cli-commands.yml +++ b/data/cli-commands.yml @@ -25,14 +25,10 @@ cli-commands: - fluvio table-format list - fluvio table-format delete - - fluvio derived-stream - - fluvio derived-stream create - - fluvio derived-stream list - - fluvio derived-stream delete - - fluvio hub - - fluvio hub download - - fluvio hub list + - fluvio hub smartmodule + - fluvio hub smartmodule list + - fluvio hub smartmodule download - fluvio hub connector - fluvio hub connector list - fluvio hub connector download @@ -111,3 +107,12 @@ cli-commands: - cdk deploy log - cdk publish - cdk set-public + + - fvm + - fvm current + - fvm self + - fvm self install + - fvm self uninstall + - fvm install + - fvm show + - fvm switch \ No newline at end of file diff --git a/embeds/cli/help/cdk-build.md b/embeds/cli/help/cdk-build.md index c44423ca5..106cdfe2a 100644 --- a/embeds/cli/help/cdk-build.md +++ b/embeds/cli/help/cdk-build.md @@ -1,10 +1,15 @@ ``` Build the Connector in the current working directory + Usage: cdk build [OPTIONS] [-- ...] + Arguments: [EXTRA_ARGUMENTS]... Extra arguments to be passed to cargo + Options: --release Release profile name [default: release] + --target Provide target platform for the package. Optional. By default + the host's one is used [default: aarch64-apple-darwin] -p, --package-name Optional package/project name -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-deploy-list.md b/embeds/cli/help/cdk-deploy-list.md index 34037e085..716f9c2bf 100644 --- a/embeds/cli/help/cdk-deploy-list.md +++ b/embeds/cli/help/cdk-deploy-list.md @@ -1,6 +1,8 @@ ``` Print the list of all deployed connectors + Usage: cdk deploy list + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-deploy-log.md b/embeds/cli/help/cdk-deploy-log.md index 4633a0192..bf8bb0f25 100644 --- a/embeds/cli/help/cdk-deploy-log.md +++ b/embeds/cli/help/cdk-deploy-log.md @@ -1,5 +1,10 @@ ``` -Print the connector's logs -Usage: cdk deploy log Options: -Arguments: -h, --help Print help +Print the connector's logs + +Usage: cdk deploy log <--config |--name > + +Options: + -c, --config Path to configuration file in YAML format + -n, --name Name of the running connector + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-deploy-shutdown.md b/embeds/cli/help/cdk-deploy-shutdown.md index 786e6044b..693380ea5 100644 --- a/embeds/cli/help/cdk-deploy-shutdown.md +++ b/embeds/cli/help/cdk-deploy-shutdown.md @@ -1,5 +1,6 @@ ``` Shutdown the Connector's deployment + Usage: cdk deploy shutdown <--config |--name > Options: diff --git a/embeds/cli/help/cdk-deploy-start.md b/embeds/cli/help/cdk-deploy-start.md index 5356e0f14..1fe6c57cb 100644 --- a/embeds/cli/help/cdk-deploy-start.md +++ b/embeds/cli/help/cdk-deploy-start.md @@ -1,10 +1,13 @@ ``` Start new deployment for the given connector config + Usage: cdk deploy start [OPTIONS] --config + Options: - -c, --config Path to configuration file in YAML format - -s, --secrets Path to file with secrets. Secrets are 'key=value' pairs separated by the - new line character. Optional - --ipkg Deploy from local package file - -h, --help Print help + -c, --config Path to configuration file in YAML format + -s, --secrets Path to file with secrets. Secrets are 'key=value' pairs separated by + the new line character. Optional + --ipkg Deploy from local package file + --log-level Log level for the connector process [default: ] + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-deploy.md b/embeds/cli/help/cdk-deploy.md index eaa6caa35..1a7862105 100644 --- a/embeds/cli/help/cdk-deploy.md +++ b/embeds/cli/help/cdk-deploy.md @@ -1,16 +1,22 @@ ``` Deploy the Connector from the current working directory + Usage: cdk deploy [OPTIONS] [-- ...] + Commands: start Start new deployment for the given connector config shutdown Shutdown the Connector's deployment list Print the list of all deployed connectors log Print the connector's logs help Print this message or the help of the given subcommand(s) + Arguments: [EXTRA_ARGUMENTS]... Extra arguments to be passed to cargo + Options: --release Release profile name [default: release] + --target Provide target platform for the package. Optional. By default + the host's one is used [default: aarch64-apple-darwin] -p, --package-name Optional package/project name -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-generate.md b/embeds/cli/help/cdk-generate.md index caa84c784..2c38b115a 100644 --- a/embeds/cli/help/cdk-generate.md +++ b/embeds/cli/help/cdk-generate.md @@ -1,10 +1,22 @@ ``` Generate new SmartConnector project + Usage: cdk generate [OPTIONS] [NAME] + Arguments: - [NAME] SmartConnector Project Name + [NAME] + SmartConnector Project Name + Options: - --destination Local path to generate the SmartConnector project. Default to directory - with project name, created in current directory [env: CDK_DESTINATION=] - -h, --help Print help (see more with '--help') + --destination + Local path to generate the SmartConnector project. Default to directory with project name, + created in current directory + + [env: CDK_DESTINATION=] + + --silent + Disable interactive prompt. Take all values from CLI flags. Fail if a value is missing + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-publish.md b/embeds/cli/help/cdk-publish.md index 3f413393f..967ad7758 100644 --- a/embeds/cli/help/cdk-publish.md +++ b/embeds/cli/help/cdk-publish.md @@ -1,11 +1,49 @@ ``` Publish Connector package to the Hub + Usage: cdk publish [OPTIONS] [PACKAGE_META] + Arguments: - [PACKAGE_META] + [PACKAGE_META] + + Options: - --public-yes don't ask for confirmation of public package publish - --target provide target platform for the package. Optional. By default the host's - one is used [default: aarch64-apple-darwin] - -h, --help Print help (see more with '--help') + --release + Release profile name + + [default: release] + + --target + Provide target platform for the package. Optional. By default the host's one is used + + [default: aarch64-apple-darwin] + + -p, --package-name + Optional package/project name + + --ipkg + path to the ipkg file, used when --push is specified + + --public-yes + don't ask for confirmation of public package publish + + --no-build + + + --pack + do only the pack portion + + --push + given a packed file do only the push + + --remote + + + --readme + Relative path to this connector package README + + [default: ./README.md] + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-set-public.md b/embeds/cli/help/cdk-set-public.md index 252d93f32..adfb58375 100644 --- a/embeds/cli/help/cdk-set-public.md +++ b/embeds/cli/help/cdk-set-public.md @@ -1,4 +1,12 @@ ``` -Set connector visibility to public Options: -Usage: cdk set-public -h, --help Print help +Set connector visibility to public + +Usage: cdk set-public [OPTIONS] + +Options: + --release Release profile name [default: release] + --target Provide target platform for the package. Optional. By default + the host's one is used [default: aarch64-apple-darwin] + -p, --package-name Optional package/project name + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-test.md b/embeds/cli/help/cdk-test.md index 4bedd451e..852c0c1f5 100644 --- a/embeds/cli/help/cdk-test.md +++ b/embeds/cli/help/cdk-test.md @@ -1,10 +1,15 @@ ``` Build and run the Connector in the current working directory + Usage: cdk test [OPTIONS] [-- ...] + Arguments: [EXTRA_ARGUMENTS]... Extra arguments to be passed to cargo + Options: --release Release profile name [default: release] + --target Provide target platform for the package. Optional. By default + the host's one is used [default: aarch64-apple-darwin] -p, --package-name Optional package/project name -c, --config Path to configuration file in YAML format [default: sample-config.yaml] diff --git a/embeds/cli/help/cdk.md b/embeds/cli/help/cdk.md index 54bfe97fc..20ad34b17 100644 --- a/embeds/cli/help/cdk.md +++ b/embeds/cli/help/cdk.md @@ -1,6 +1,8 @@ ``` Connector Development Kit + Usage: cdk + Commands: build Build the Connector in the current working directory test Build and run the Connector in the current working directory @@ -9,6 +11,7 @@ Commands: publish Publish Connector package to the Hub set-public Set connector visibility to public help Print this message or the help of the given subcommand(s) + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-cluster-create.md b/embeds/cli/help/fluvio-cloud-cluster-create.md index 66eb5d4ff..848befd5d 100644 --- a/embeds/cli/help/fluvio-cloud-cluster-create.md +++ b/embeds/cli/help/fluvio-cloud-cluster-create.md @@ -1,9 +1,11 @@ ``` -fluvio-cloud-cluster-create Create a new Fluvio cluster -USAGE: - fluvio-cloud cluster create [OPTIONS] -OPTIONS: - -h, --help Print help information - --profile The name of the Profile to save + +Usage: fluvio-cloud cluster create [OPTIONS] + +Options: + --profile The name of the Profile to save + --region + --version + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-cluster-delete.md b/embeds/cli/help/fluvio-cloud-cluster-delete.md index e1704be8c..0dd8c934f 100644 --- a/embeds/cli/help/fluvio-cloud-cluster-delete.md +++ b/embeds/cli/help/fluvio-cloud-cluster-delete.md @@ -1,10 +1,11 @@ ``` -fluvio-cloud-cluster-delete Delete an existing Fluvio cluster -USAGE: - fluvio-cloud cluster delete -ARGS: - The email for the cluster to be deleted -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud cluster delete + +Arguments: + The email for the cluster to be deleted + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-cluster-list.md b/embeds/cli/help/fluvio-cloud-cluster-list.md index 4951aff65..a194ca5ef 100644 --- a/embeds/cli/help/fluvio-cloud-cluster-list.md +++ b/embeds/cli/help/fluvio-cloud-cluster-list.md @@ -1,8 +1,8 @@ ``` -fluvio-cloud-cluster-list List all Fluvio clusters -USAGE: - fluvio-cloud cluster list -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud cluster list + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-cluster.md b/embeds/cli/help/fluvio-cloud-cluster.md index cf0bb1503..298bec422 100644 --- a/embeds/cli/help/fluvio-cloud-cluster.md +++ b/embeds/cli/help/fluvio-cloud-cluster.md @@ -1,13 +1,14 @@ ``` -fluvio-cloud-cluster View Cluster information -USAGE: - fluvio-cloud cluster -OPTIONS: - -h, --help Print help information -SUBCOMMANDS: - create Create a new Fluvio cluster - delete Delete an existing Fluvio cluster - help Print this message or the help of the given subcommand(s) - list List all Fluvio clusters + +Usage: fluvio-cloud cluster + +Commands: + create Create a new Fluvio cluster + delete Delete an existing Fluvio cluster + list List all Fluvio clusters + help Print this message or the help of the given subcommand(s) + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-config.md b/embeds/cli/help/fluvio-cloud-connector-config.md index 3a858a363..390703d20 100644 --- a/embeds/cli/help/fluvio-cloud-connector-config.md +++ b/embeds/cli/help/fluvio-cloud-connector-config.md @@ -1,10 +1,11 @@ ``` -fluvio-cloud-connector-config Show the connector configuration details -USAGE: - fluvio-cloud connector config [OPTIONS] -ARGS: - Name of connector -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud connector config + +Arguments: + Name of connector + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-create.md b/embeds/cli/help/fluvio-cloud-connector-create.md index e79554919..9b5dd5a50 100644 --- a/embeds/cli/help/fluvio-cloud-connector-create.md +++ b/embeds/cli/help/fluvio-cloud-connector-create.md @@ -1,11 +1,10 @@ ``` -fluvio cloud connector create Create a new Connector -Usage: fluvio cloud connector create [OPTIONS] --config +Usage: fluvio-cloud connector create [OPTIONS] --config Options: - -c, --config Name of connector - --log-level Sets the log level [default: info] + -c, --config Connector config + --log-level Sets the log level [default: LogLevel::default()] -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-delete.md b/embeds/cli/help/fluvio-cloud-connector-delete.md index fe39e84b5..8c6daeede 100644 --- a/embeds/cli/help/fluvio-cloud-connector-delete.md +++ b/embeds/cli/help/fluvio-cloud-connector-delete.md @@ -1,10 +1,11 @@ ``` -fluvio-cloud-connector-delete Delete a Connector -USAGE: - fluvio-cloud connector delete [OPTIONS] ... -ARGS: - ... One or more name(s) of the connector(s) to be deleted -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud connector delete ... + +Arguments: + ... One or more name(s) of the connector(s) to be deleted + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-list.md b/embeds/cli/help/fluvio-cloud-connector-list.md index 68ee9ce73..da3f8083b 100644 --- a/embeds/cli/help/fluvio-cloud-connector-list.md +++ b/embeds/cli/help/fluvio-cloud-connector-list.md @@ -1,8 +1,8 @@ ``` -fluvio-cloud-connector-list List all Connectors -USAGE: - fluvio-cloud connector list [OPTIONS] -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud connector list + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-logs.md b/embeds/cli/help/fluvio-cloud-connector-logs.md index 81b40f19e..305be4243 100644 --- a/embeds/cli/help/fluvio-cloud-connector-logs.md +++ b/embeds/cli/help/fluvio-cloud-connector-logs.md @@ -1,10 +1,11 @@ ``` -fluvio-cloud-connector-logs View connector logs -USAGE: - fluvio-cloud connector logs [OPTIONS] -ARGS: - Name of connector -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud connector logs + +Arguments: + Name of connector + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-update.md b/embeds/cli/help/fluvio-cloud-connector-update.md index 346d0782a..1b876068d 100644 --- a/embeds/cli/help/fluvio-cloud-connector-update.md +++ b/embeds/cli/help/fluvio-cloud-connector-update.md @@ -1,5 +1,4 @@ ``` -fluvio cloud connector update Update and restart a Connector Usage: fluvio-cloud connector update [OPTIONS] --config diff --git a/embeds/cli/help/fluvio-cloud-connector.md b/embeds/cli/help/fluvio-cloud-connector.md index 2bf7e3a7b..b012ed77c 100644 --- a/embeds/cli/help/fluvio-cloud-connector.md +++ b/embeds/cli/help/fluvio-cloud-connector.md @@ -1,16 +1,17 @@ ``` -fluvio-cloud-connector View Fluvio Connector information -USAGE: - fluvio-cloud connector -OPTIONS: - -h, --help Print help information -SUBCOMMANDS: - config Show the connector configuration details - create Create a new Connector - delete Delete a Connector - help Print this message or the help of the given subcommand(s) - list List all Connectors - logs View connector logs - update Update and restart a Connector + +Usage: fluvio-cloud connector + +Commands: + logs View connector logs + create Create a new Connector + delete Delete a Connector + list List all Connectors + update Update and restart a Connector + config Show the connector configuration details + help Print this message or the help of the given subcommand(s) + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-login.md b/embeds/cli/help/fluvio-cloud-login.md index 7edf1b06d..fc82c1ac3 100644 --- a/embeds/cli/help/fluvio-cloud-login.md +++ b/embeds/cli/help/fluvio-cloud-login.md @@ -1,14 +1,26 @@ ``` -fluvio-cloud-login Log into Infinyon Cloud with Oauth2 or username/password -USAGE: - fluvio-cloud login [OPTIONS] -OPTIONS: - --email Infinyon Cloud email to use for logging in - -h, --help Print help information - --password Password to use when logging in (not recommended) - --profile The name of the Profile to save - --use-oauth2 Authenticate using OAuth 2.0 Device Code Flow. CLI will try to - launch a web browser to log in interactively. If a web browser is - not available, CLI will print URL for device code login + +Usage: fluvio-cloud login [OPTIONS] + +Options: + --email + Infinyon Cloud email to use for logging in + + --password + Password to use when logging in (not recommended). + + If not provided, fluvio-cloud will prompt on stdin unless using third party + authentication. + + --profile + The name of the Profile to save + + --use-oauth2 + Authenticate using OAuth 2.0 Device Code Flow. CLI will try to launch a web browser to log + in interactively. If a web browser is not available, CLI will print URL for device code + login + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-logout.md b/embeds/cli/help/fluvio-cloud-logout.md index e88415978..a33803aac 100644 --- a/embeds/cli/help/fluvio-cloud-logout.md +++ b/embeds/cli/help/fluvio-cloud-logout.md @@ -1,8 +1,8 @@ ``` -fluvio-cloud-logout Log out of a Infinyon Cloud account -USAGE: - fluvio-cloud logout -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud logout + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-secret-delete.md b/embeds/cli/help/fluvio-cloud-secret-delete.md new file mode 100644 index 000000000..a90bd43d7 --- /dev/null +++ b/embeds/cli/help/fluvio-cloud-secret-delete.md @@ -0,0 +1,11 @@ +``` +delete a cloud secret + +Usage: fluvio-cloud secret delete + +Arguments: + Name of secret to delete + +Options: + -h, --help Print help +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-secret-list.md b/embeds/cli/help/fluvio-cloud-secret-list.md new file mode 100644 index 000000000..cf405b353 --- /dev/null +++ b/embeds/cli/help/fluvio-cloud-secret-list.md @@ -0,0 +1,8 @@ +``` +list cloud secret names + +Usage: fluvio-cloud secret list + +Options: + -h, --help Print help +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-secret-set.md b/embeds/cli/help/fluvio-cloud-secret-set.md new file mode 100644 index 000000000..feb26206b --- /dev/null +++ b/embeds/cli/help/fluvio-cloud-secret-set.md @@ -0,0 +1,12 @@ +``` +set a cloud secret by name and value + +Usage: fluvio-cloud secret set + +Arguments: + Name of secret to set + Value of secret to associate with name + +Options: + -h, --help Print help +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-usage.md b/embeds/cli/help/fluvio-cloud-usage.md index a236af084..ea6728a99 100644 --- a/embeds/cli/help/fluvio-cloud-usage.md +++ b/embeds/cli/help/fluvio-cloud-usage.md @@ -1,8 +1,8 @@ ``` -fluvio-cloud-usage Print cluster stats -USAGE: - fluvio-cloud usage -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud usage + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud.md b/embeds/cli/help/fluvio-cloud.md index 5cda4b2b2..abf179fac 100644 --- a/embeds/cli/help/fluvio-cloud.md +++ b/embeds/cli/help/fluvio-cloud.md @@ -1,15 +1,19 @@ ``` -fluvio-cloud -USAGE: - fluvio-cloud -OPTIONS: - -h, --help Print help information -SUBCOMMANDS: - cluster View Cluster information - connector View Fluvio Connector information - help Print this message or the help of the given subcommand(s) - login Log into Infinyon Cloud with Oauth2 or username/password - logout Log out of a Infinyon Cloud account - usage Print cluster stats - version Print the current version of fluvio-cloud +Usage: fluvio-cloud + +Commands: + login Log into Infinyon Cloud with Oauth2 or username/password + logout Log out of a Infinyon Cloud account + version Print the current version of fluvio-cloud + usage Print cluster stats + connector View Fluvio Connector information + cluster View Cluster information + secret Manage cloud secrets + webhook Manage Webhooks + metadata Generate metadata for Fluvio base CLI + edge Edge + help Print this message or the help of the given subcommand(s) + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-check.md b/embeds/cli/help/fluvio-cluster-check.md index b834e0dd3..96bf8a63c 100644 --- a/embeds/cli/help/fluvio-cluster-check.md +++ b/embeds/cli/help/fluvio-cluster-check.md @@ -1,7 +1,16 @@ ``` -Check that all requirements for cluster startup are met +Check that all requirements for cluster startup are met. + +This command is useful to check if user has all the required dependencies and permissions to run fluvio on the current Kubernetes context. + +It is not intended to be used in scenarios where user does not have access to Kubernetes resources (eg. Cloud) + Usage: fluvio cluster check [OPTIONS] + Options: - --fix Attempt to fix recoverable errors - -h, --help Print help (see more with '--help') + --fix + Attempt to fix recoverable errors + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-delete.md b/embeds/cli/help/fluvio-cluster-delete.md index 60c9495aa..a6ddce1c7 100644 --- a/embeds/cli/help/fluvio-cluster-delete.md +++ b/embeds/cli/help/fluvio-cluster-delete.md @@ -1,6 +1,8 @@ ``` Uninstall a Fluvio cluster + Usage: fluvio cluster delete [OPTIONS] + Options: --namespace --local Remove only local spu/sc(custom) fluvio installation diff --git a/embeds/cli/help/fluvio-cluster-diagnostics.md b/embeds/cli/help/fluvio-cluster-diagnostics.md index b7ecdd3b1..02a1c0543 100644 --- a/embeds/cli/help/fluvio-cluster-diagnostics.md +++ b/embeds/cli/help/fluvio-cluster-diagnostics.md @@ -1,6 +1,8 @@ ``` Collect anonymous diagnostic information to help with debugging + Usage: fluvio cluster diagnostics [OPTIONS] + Options: --quiet --k8 diff --git a/embeds/cli/help/fluvio-cluster-spg-create.md b/embeds/cli/help/fluvio-cluster-spg-create.md index f6540aafa..29c813f44 100644 --- a/embeds/cli/help/fluvio-cluster-spg-create.md +++ b/embeds/cli/help/fluvio-cluster-spg-create.md @@ -1,8 +1,11 @@ ``` Create a new managed SPU Group + fluvio cluster spg create [OPTIONS] + Arguments: The name for the new SPU Group + Options: -r, --replicas The number of SPUs to create in this SPG [default: 1] --min-id Minimum SPU ID [default: 1] diff --git a/embeds/cli/help/fluvio-cluster-spg-delete.md b/embeds/cli/help/fluvio-cluster-spg-delete.md index d0aae1f0b..04728ad06 100644 --- a/embeds/cli/help/fluvio-cluster-spg-delete.md +++ b/embeds/cli/help/fluvio-cluster-spg-delete.md @@ -1,8 +1,11 @@ ``` Delete a managed SPU Group + fluvio cluster spg delete + Arguments: The name of the SPU Group to delete + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-spg-list.md b/embeds/cli/help/fluvio-cluster-spg-list.md index 759d1e211..1f4f46502 100644 --- a/embeds/cli/help/fluvio-cluster-spg-list.md +++ b/embeds/cli/help/fluvio-cluster-spg-list.md @@ -1,6 +1,8 @@ ``` List all SPU Groups + fluvio cluster spg list [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-cluster-spg.md b/embeds/cli/help/fluvio-cluster-spg.md index 04c2e469d..a093bdaa0 100644 --- a/embeds/cli/help/fluvio-cluster-spg.md +++ b/embeds/cli/help/fluvio-cluster-spg.md @@ -1,10 +1,16 @@ ``` Manage and view SPU Groups (SPGs) + +SPGs are groups of SPUs in a cluster which are managed together. + Usage: fluvio cluster spg + Commands: create Create a new managed SPU Group delete Delete a managed SPU Group list List all SPU Groups + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-spu-list.md b/embeds/cli/help/fluvio-cluster-spu-list.md index 6662aca47..3ff45935b 100644 --- a/embeds/cli/help/fluvio-cluster-spu-list.md +++ b/embeds/cli/help/fluvio-cluster-spu-list.md @@ -1,6 +1,8 @@ ``` List all SPUs known by this cluster (managed AND custom) + fluvio cluster spu list [OPTIONS] + Options: --custom Whether to list only custom SPUs -O, --output Output [default: table] [possible values: table, yaml, json] diff --git a/embeds/cli/help/fluvio-cluster-spu-register.md b/embeds/cli/help/fluvio-cluster-spu-register.md index 5036c2048..2a3ad7b2e 100644 --- a/embeds/cli/help/fluvio-cluster-spu-register.md +++ b/embeds/cli/help/fluvio-cluster-spu-register.md @@ -1,6 +1,8 @@ ``` Register a new custom SPU with the cluster + fluvio cluster spu register [OPTIONS] --id --public-server --private-server + Options: -i, --id SPU id -n, --name SPU name diff --git a/embeds/cli/help/fluvio-cluster-spu-unregister.md b/embeds/cli/help/fluvio-cluster-spu-unregister.md index 925c22606..84a206ff3 100644 --- a/embeds/cli/help/fluvio-cluster-spu-unregister.md +++ b/embeds/cli/help/fluvio-cluster-spu-unregister.md @@ -1,6 +1,8 @@ ``` Unregister a custom SPU from the cluster + fluvio cluster spu unregister [OPTIONS] + Options: -i, --id SPU id -n, --name SPU name diff --git a/embeds/cli/help/fluvio-cluster-spu.md b/embeds/cli/help/fluvio-cluster-spu.md index de5e3b27d..20b6220cf 100644 --- a/embeds/cli/help/fluvio-cluster-spu.md +++ b/embeds/cli/help/fluvio-cluster-spu.md @@ -1,10 +1,16 @@ ``` Manage and view Streaming Processing Units (SPUs) + +SPUs make up the part of a Fluvio cluster which is in charge of receiving messages from producers, storing those messages, and relaying them to consumers. This command lets you see the status of SPUs in your cluster. + Usage: fluvio cluster spu + Commands: register Register a new custom SPU with the cluster unregister Unregister a custom SPU from the cluster list List all SPUs known by this cluster (managed AND custom) + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-start.md b/embeds/cli/help/fluvio-cluster-start.md index c9ef8bb93..92feea0d6 100644 --- a/embeds/cli/help/fluvio-cluster-start.md +++ b/embeds/cli/help/fluvio-cluster-start.md @@ -1,6 +1,8 @@ ``` Install Fluvio cluster + Usage: fluvio cluster start [OPTIONS] + Options: --develop use local image @@ -11,7 +13,7 @@ Options: --registry k8: use custom docker registry --namespace - k8 [default: default] + k8 namespace [default: default] --group-name k8 [default: main] --install-name @@ -22,6 +24,8 @@ Options: chart values --use-k8-port-forwarding Uses port forwarding for connecting to SC during install + --use-cluster-ip + Uses port forwarding for connecting to SC during install --tls-client-secret-name TLS: Client secret name while adding to Kubernetes [default: fluvio-client-tls] --tls-server-secret-name @@ -29,12 +33,13 @@ Options: --spu-storage-size set spu storage size [default: 10] --skip-profile-creation + --spu number of SPU [default: 1] --rust-log RUST_LOG options --log-dir - log dir [default: /tmp] + log dir [default: /usr/local/var/log/fluvio] --sys-only installing/upgrade sys only --local @@ -54,6 +59,7 @@ Options: --server-key TLS: path to server private key --authorization-config-map + --skip-checks Whether to skip pre-install checks, defaults to false --setup @@ -62,8 +68,8 @@ Options: Proxy address --service-type Service Type - --connector-prefix - Connector Prefix + --read-only + Start SC in read only mode -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-upgrade.md b/embeds/cli/help/fluvio-cluster-upgrade.md index 4efe1d1ee..1dce03f7b 100644 --- a/embeds/cli/help/fluvio-cluster-upgrade.md +++ b/embeds/cli/help/fluvio-cluster-upgrade.md @@ -1,6 +1,8 @@ ``` Upgrades an already-started Fluvio cluster + Usage: fluvio cluster upgrade [OPTIONS] + Options: --develop use local image @@ -11,7 +13,7 @@ Options: --registry k8: use custom docker registry --namespace - k8 [default: default] + k8 namespace [default: default] --group-name k8 [default: main] --install-name @@ -22,6 +24,8 @@ Options: chart values --use-k8-port-forwarding Uses port forwarding for connecting to SC during install + --use-cluster-ip + Uses port forwarding for connecting to SC during install --tls-client-secret-name TLS: Client secret name while adding to Kubernetes [default: fluvio-client-tls] --tls-server-secret-name @@ -29,12 +33,13 @@ Options: --spu-storage-size set spu storage size [default: 10] --skip-profile-creation + --spu number of SPU [default: 1] --rust-log RUST_LOG options --log-dir - log dir [default: /tmp] + log dir [default: /usr/local/var/log/fluvio] --sys-only installing/upgrade sys only --local @@ -54,6 +59,7 @@ Options: --server-key TLS: path to server private key --authorization-config-map + --skip-checks Whether to skip pre-install checks, defaults to false --setup @@ -62,8 +68,8 @@ Options: Proxy address --service-type Service Type - --connector-prefix - Connector Prefix + --read-only + Start SC in read only mode -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster.md b/embeds/cli/help/fluvio-cluster.md index 4be24d425..f68689955 100644 --- a/embeds/cli/help/fluvio-cluster.md +++ b/embeds/cli/help/fluvio-cluster.md @@ -1,6 +1,8 @@ ``` Install or uninstall Fluvio cluster + Usage: fluvio cluster + Commands: start Install Fluvio cluster upgrade Upgrades an already-started Fluvio cluster @@ -11,6 +13,7 @@ Commands: diagnostics Collect anonymous diagnostic information to help with debugging status Check the status of a Fluvio cluster shutdown Shutdown cluster processes without deleting data + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-completions.md b/embeds/cli/help/fluvio-completions.md index d95ddcd86..fc7fae51d 100644 --- a/embeds/cli/help/fluvio-completions.md +++ b/embeds/cli/help/fluvio-completions.md @@ -1,10 +1,21 @@ ``` Generate command-line completions for Fluvio + +Run the following two commands to enable fluvio command completions. + +Open a new terminal for the changes to take effect. + +$ fluvio completions bash > ~/fluvio_completions.sh +$ echo "source ~/fluvio_completions.sh" >> ~/.bashrc + Usage: fluvio completions + Commands: bash Generate CLI completions for bash zsh Generate CLI completions for zsh fish Generate CLI completions for fish + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-consume.md b/embeds/cli/help/fluvio-consume.md index dae7184e3..b0cb5a656 100644 --- a/embeds/cli/help/fluvio-consume.md +++ b/embeds/cli/help/fluvio-consume.md @@ -1,53 +1,94 @@ ``` Read messages from a topic/partition + Usage: fluvio consume [OPTIONS] + Arguments: - Topic name + + Topic name + Options: -p, --partition - Partition id [default: 0] + Partition id + -A, --all-partitions Consume records from all partitions + -d, --disable-continuous Disable continuous processing of messages + --disable-progressbar Disable the progress bar and wait spinner + -k, --key-value Print records in "[key] value" format, with "[null]" for no key + -F, --format - Provide a template string to print records with a custom format. See --help for details + Provide a template string to print records with a custom format. See --help for details. + + Template strings may include the variables {{key}}, {{value}}, {{offset}}, {{partition}} and {{time}} which will have each record's contents substituted in their place. Note that timestamp is displayed using RFC3339, is always UTC and ignores system timezone. + + For example, the following template string: + + Offset {{offset}} has key {{key}} and value {{value}} + + Would produce a printout where records might look like this: + + Offset 0 has key A and value Apple + --table-format Consume records using the formatting rules defined by TableFormat name + -B, --beginning Consume records from the beginning of the log + -H, --head Consume records starting from the beginning of the log + -T, --tail Consume records starting from the end of the log + --start The absolute offset of the first record to begin consuming from + --end Consume records until end offset (inclusive) + -b, --maxbytes Maximum number of bytes to be retrieved + --isolation Isolation level that consumer must respect. Supported values: read_committed (ReadCommitted) - consume only committed records, read_uncommitted (ReadUncommitted) - consume all records accepted by leader + --suppress-unknown Suppress items items that have an unknown output type + -O, --output - Output [possible values: dynamic, text, binary, json, raw, table, full-table] + Output + + [possible values: dynamic, text, binary, json, raw, table, full-table] + --smartmodule Name of the smartmodule + --smartmodule-path Path to the smart module + --aggregate-initial (Optional) Value to use as an initial accumulator for aggregate SmartModules + -e, --params (Optional) Extra input parameters passed to the smartmodule. They should be passed using key=value format Eg. fluvio consume topic-name --smartmodule my_filter -e foo=bar -e key=value -e one=1 + --transforms-file (Optional) Path to a file with transformation specification + -t, --transform (Optional) Transformation specification as JSON formatted string. E.g. fluvio consume topic-name --transform='{"uses":"infinyon/jolt@0.1.0","with":{"spec":"[{\"operation\":\"default\",\"spec\":{\"source\":\"test\"}}]"}}' + + --truncate + Truncate the output to one line + -h, --help Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-derived-stream-create.md b/embeds/cli/help/fluvio-derived-stream-create.md deleted file mode 100644 index f1c3fffbc..000000000 --- a/embeds/cli/help/fluvio-derived-stream-create.md +++ /dev/null @@ -1,3 +0,0 @@ -``` -Unable to find plugin 'fluvio-derived-stream'. Make sure it is installed in "/Users/telant/.fluvio/extensions-latest". -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-derived-stream-delete.md b/embeds/cli/help/fluvio-derived-stream-delete.md deleted file mode 100644 index f1c3fffbc..000000000 --- a/embeds/cli/help/fluvio-derived-stream-delete.md +++ /dev/null @@ -1,3 +0,0 @@ -``` -Unable to find plugin 'fluvio-derived-stream'. Make sure it is installed in "/Users/telant/.fluvio/extensions-latest". -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-derived-stream-list.md b/embeds/cli/help/fluvio-derived-stream-list.md deleted file mode 100644 index f1c3fffbc..000000000 --- a/embeds/cli/help/fluvio-derived-stream-list.md +++ /dev/null @@ -1,3 +0,0 @@ -``` -Unable to find plugin 'fluvio-derived-stream'. Make sure it is installed in "/Users/telant/.fluvio/extensions-latest". -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-derived-stream.md b/embeds/cli/help/fluvio-derived-stream.md deleted file mode 100644 index f1c3fffbc..000000000 --- a/embeds/cli/help/fluvio-derived-stream.md +++ /dev/null @@ -1,3 +0,0 @@ -``` -Unable to find plugin 'fluvio-derived-stream'. Make sure it is installed in "/Users/telant/.fluvio/extensions-latest". -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-connector-download.md b/embeds/cli/help/fluvio-hub-connector-download.md index f9b28fb1b..4ec870d85 100644 --- a/embeds/cli/help/fluvio-hub-connector-download.md +++ b/embeds/cli/help/fluvio-hub-connector-download.md @@ -1,10 +1,24 @@ ``` Download SmartConnector to the local folder + Usage: fluvio hub connector download [OPTIONS] + Arguments: - SmartConnector name: e.g. infinyon/salesforce-sink@v0.0.1 + + SmartConnector name: e.g. infinyon/http-sink@vX.Y.Z + Options: - -o, --output Target local folder or file name - --target Target platform for the package. Optional. By default the host's one is used [default: aarch64-apple-darwin] - -h, --help Print help (see more with '--help') + -o, --output + Target local folder or file name + + --target + Target platform for the package. Optional. By default the host's one is used + + [default: aarch64-apple-darwin] + + --remote + + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-connector-list.md b/embeds/cli/help/fluvio-hub-connector-list.md index 39f539d94..fdd25d9d9 100644 --- a/embeds/cli/help/fluvio-hub-connector-list.md +++ b/embeds/cli/help/fluvio-hub-connector-list.md @@ -1,7 +1,18 @@ ``` List all available SmartConnectors + Usage: fluvio hub connector list [OPTIONS] + Options: - -O, --output Output [default: table] [possible values: table, yaml, json] - -h, --help Print help (see more with '--help') + -O, --output + Output + + [default: table] + [possible values: table, yaml, json] + + --remote + + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-connector.md b/embeds/cli/help/fluvio-hub-connector.md index 9b5fc53b0..a09c1a2ef 100644 --- a/embeds/cli/help/fluvio-hub-connector.md +++ b/embeds/cli/help/fluvio-hub-connector.md @@ -1,9 +1,12 @@ ``` List available Connectors in the hub + Usage: fluvio hub connector + Commands: list List all available SmartConnectors download Download SmartConnector to the local folder + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-download.md b/embeds/cli/help/fluvio-hub-download.md deleted file mode 100644 index f52af6c20..000000000 --- a/embeds/cli/help/fluvio-hub-download.md +++ /dev/null @@ -1,18 +0,0 @@ -``` -Download a SmartModule from the hub -Usage: fluvio hub download [OPTIONS] -Arguments: - SmartModule name: e.g. infinyon/jolt@v0.0.1 -Options: - -c, --cluster Address of cluster - --tls Enable TLS - --enable-client-cert TLS: use client cert - --domain Required if client cert is used - --ca-cert Path to TLS ca cert, required when client cert is enabled - --client-cert Path to TLS client certificate - --client-key Path to TLS client private key - -P, --profile - --local just download package to local filesystem - --ipkg given local package file, download to cluster - -h, --help Print help (see more with '--help') -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-list.md b/embeds/cli/help/fluvio-hub-list.md deleted file mode 100644 index f0d14d310..000000000 --- a/embeds/cli/help/fluvio-hub-list.md +++ /dev/null @@ -1,7 +0,0 @@ -``` -List available SmartModules in the hub -Usage: fluvio hub list [OPTIONS] -Options: - -O, --output Output [default: table] [possible values: table, yaml, json] - -h, --help Print help (see more with '--help') -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-smartmodule-download.md b/embeds/cli/help/fluvio-hub-smartmodule-download.md new file mode 100644 index 000000000..f346c34b0 --- /dev/null +++ b/embeds/cli/help/fluvio-hub-smartmodule-download.md @@ -0,0 +1,46 @@ +``` +Download SmartModules - locally or to cluster (default) + +Usage: fluvio hub smartmodule download [OPTIONS] + +Arguments: + + SmartModule name: e.g. infinyon/jolt@v0.0.1 + +Options: + -c, --cluster + Address of cluster + + --tls + Enable TLS + + --enable-client-cert + TLS: use client cert + + --domain + Required if client cert is used + + --ca-cert + Path to TLS ca cert, required when client cert is enabled + + --client-cert + Path to TLS client certificate + + --client-key + Path to TLS client private key + + -P, --profile + + + -o, --output + Download package to local filesystem + + --ipkg + given local package file, download to cluster + + --remote + + + -h, --help + Print help (see a summary with '-h') +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-smartmodule-list.md b/embeds/cli/help/fluvio-hub-smartmodule-list.md new file mode 100644 index 000000000..3572ae4df --- /dev/null +++ b/embeds/cli/help/fluvio-hub-smartmodule-list.md @@ -0,0 +1,18 @@ +``` +List all available SmartModules + +Usage: fluvio hub smartmodule list [OPTIONS] + +Options: + -O, --output + Output + + [default: table] + [possible values: table, yaml, json] + + --remote + + + -h, --help + Print help (see a summary with '-h') +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-smartmodule.md b/embeds/cli/help/fluvio-hub-smartmodule.md new file mode 100644 index 000000000..28a7a5065 --- /dev/null +++ b/embeds/cli/help/fluvio-hub-smartmodule.md @@ -0,0 +1,12 @@ +``` +List available SmartModules in the hub + +Usage: fluvio hub smartmodule + +Commands: + list List all available SmartModules + download Download SmartModules - locally or to cluster (default) + +Options: + -h, --help Print help +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub.md b/embeds/cli/help/fluvio-hub.md index cc9fa5c95..b033d3b6c 100644 --- a/embeds/cli/help/fluvio-hub.md +++ b/embeds/cli/help/fluvio-hub.md @@ -1,10 +1,12 @@ ``` Work with the SmartModule Hub + Usage: fluvio hub + Commands: - download Download a SmartModule from the hub - list List available SmartModules in the hub - connector List available Connectors in the hub + smartmodule List available SmartModules in the hub [aliases: sm] + connector List available Connectors in the hub [aliases: conn] + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-install.md b/embeds/cli/help/fluvio-install.md index 8d4aa3c97..7a8ef78b4 100644 --- a/embeds/cli/help/fluvio-install.md +++ b/embeds/cli/help/fluvio-install.md @@ -1,9 +1,34 @@ ``` Install Fluvio plugins + +The Fluvio CLI considers any executable with the prefix `fluvio-` to be a CLI plugin. For example, an executable named `fluvio-foo` in your PATH may be invoked by running `fluvio foo`. + +This command allows you to install plugins from Fluvio's package registry. + Usage: fluvio install [OPTIONS] [PACKAGE] + Arguments: - [PACKAGE] The ID of a package to install, e.g. "fluvio/fluvio-cloud" + [PACKAGE] + The ID of a package to install, e.g. "fluvio/fluvio-cloud" + Options: - --develop Install the latest prerelease rather than the latest release - -h, --help Print help (see more with '--help') + --develop + Install the latest prerelease rather than the latest release + + If the package ID contains a version (e.g. `fluvio/fluvio:0.6.0`), this is ignored + + --hub + When this flag is provided, use the hub. Dev-only + + --use-hub-defaults + Use local hub defaults. Implied if INFINYON_HUB_REMOTE or FLUVIO_CLOUD_PROFILE env vars are set - Dev-only + + --channel + When this flag is provided, use the hub. Dev-only + + --target + override default target arch determination + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-partition-list.md b/embeds/cli/help/fluvio-partition-list.md index 2b5a7ea9a..32f57b465 100644 --- a/embeds/cli/help/fluvio-partition-list.md +++ b/embeds/cli/help/fluvio-partition-list.md @@ -1,6 +1,8 @@ ``` List all of the Partitions in this cluster + fluvio partition list [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-partition.md b/embeds/cli/help/fluvio-partition.md index 7c6cf27db..b53dffe11 100644 --- a/embeds/cli/help/fluvio-partition.md +++ b/embeds/cli/help/fluvio-partition.md @@ -1,8 +1,14 @@ ``` Manage and view Partitions + +Partitions are a way to divide the total traffic of a single Topic into separate streams which may be processed independently. Data sent to different partitions may be processed by separate SPUs on different computers. By dividing the load of a Topic evenly among partitions, you can increase the total throughput of the Topic. + Usage: fluvio partition + Commands: list List all of the Partitions in this cluster + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-produce.md b/embeds/cli/help/fluvio-produce.md index 20f1dfd29..d6e781de8 100644 --- a/embeds/cli/help/fluvio-produce.md +++ b/embeds/cli/help/fluvio-produce.md @@ -1,8 +1,11 @@ ``` Write messages to a topic/partition + Usage: fluvio produce [OPTIONS] + Arguments: The name of the Topic to produce to + Options: -v, --verbose Print progress output when sending records @@ -13,7 +16,7 @@ Options: --raw Send all input as one record. Use this when producing binary files --compression - Compression algorithm to use when sending records. Supported values: none, gzip, snappy and lz4 + Compression algorithm to use when sending records. Supported values: none, gzip, snappy, zstd and lz4 -f, --file Path to a file to produce to the topic. Default: Each line treated as single record unless `--raw` specified. If absent, producer will read stdin --linger diff --git a/embeds/cli/help/fluvio-profile-current.md b/embeds/cli/help/fluvio-profile-current.md index 1f92f1cd3..316faec1d 100644 --- a/embeds/cli/help/fluvio-profile-current.md +++ b/embeds/cli/help/fluvio-profile-current.md @@ -1,4 +1,8 @@ ``` -Print the name of the current context Options: -Usage: fluvio profile current -h, --help Print help +Print the name of the current context + +Usage: fluvio profile current + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-profile-delete-cluster.md b/embeds/cli/help/fluvio-profile-delete-cluster.md index ce6b29e6f..da52b3ac8 100644 --- a/embeds/cli/help/fluvio-profile-delete-cluster.md +++ b/embeds/cli/help/fluvio-profile-delete-cluster.md @@ -1,8 +1,11 @@ ``` Delete the named cluster + Usage: fluvio profile delete-cluster [OPTIONS] + Arguments: The name of a cluster connection to delete + Options: -f, --force Deletes a cluster even if its active -h, --help Print help diff --git a/embeds/cli/help/fluvio-profile-delete.md b/embeds/cli/help/fluvio-profile-delete.md index e3a226c3c..f1251b528 100644 --- a/embeds/cli/help/fluvio-profile-delete.md +++ b/embeds/cli/help/fluvio-profile-delete.md @@ -1,8 +1,11 @@ ``` Delete the named profile + Usage: fluvio profile delete + Arguments: + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-profile-export.md b/embeds/cli/help/fluvio-profile-export.md index a53ef8096..162a646cc 100644 --- a/embeds/cli/help/fluvio-profile-export.md +++ b/embeds/cli/help/fluvio-profile-export.md @@ -1,8 +1,11 @@ ``` Export a profile for use in other applications + Usage: fluvio profile export [OPTIONS] [PROFILE_NAME] + Arguments: [PROFILE_NAME] + Options: -O, --output [default: json] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-profile-list.md b/embeds/cli/help/fluvio-profile-list.md index 90f6e8293..ad0d1a10a 100644 --- a/embeds/cli/help/fluvio-profile-list.md +++ b/embeds/cli/help/fluvio-profile-list.md @@ -1,6 +1,8 @@ ``` Display the entire Fluvio configuration + Usage: fluvio profile list [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-profile-rename.md b/embeds/cli/help/fluvio-profile-rename.md index 30c55c348..bc150259a 100644 --- a/embeds/cli/help/fluvio-profile-rename.md +++ b/embeds/cli/help/fluvio-profile-rename.md @@ -1,9 +1,12 @@ ``` Rename a profile + Usage: fluvio profile rename + Arguments: The name of the profile to rename The new name to give the profile + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-profile-switch.md b/embeds/cli/help/fluvio-profile-switch.md index 09c5c6171..b6c2479bc 100644 --- a/embeds/cli/help/fluvio-profile-switch.md +++ b/embeds/cli/help/fluvio-profile-switch.md @@ -1,8 +1,11 @@ ``` Switch to the named profile + Usage: fluvio profile switch + Arguments: + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-profile-sync.md b/embeds/cli/help/fluvio-profile-sync.md index b7d4a678c..a0ccfad39 100644 --- a/embeds/cli/help/fluvio-profile-sync.md +++ b/embeds/cli/help/fluvio-profile-sync.md @@ -1,9 +1,12 @@ ``` Sync a profile from a cluster + fluvio profile sync + Commands: k8 Sync a profile from a Kubernetes cluster local Sync a profile from a local cluster + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-profile.md b/embeds/cli/help/fluvio-profile.md index e41acec12..678ec0c98 100644 --- a/embeds/cli/help/fluvio-profile.md +++ b/embeds/cli/help/fluvio-profile.md @@ -1,6 +1,10 @@ ``` Manage Profiles, which describe linked clusters + +Each Profile describes a particular Fluvio cluster you may be connected to. This might correspond to Fluvio running on Minikube or in the Cloud. There is one "active" profile, which determines which cluster all of the Fluvio CLI commands interact with. + Usage: fluvio profile [COMMAND] + Commands: current Print the name of the current context delete Delete the named profile @@ -10,6 +14,9 @@ Commands: switch Switch to the named profile sync Sync a profile from a cluster export Export a profile for use in other applications + add Manually add a profile (advanced) + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-run-metadata.md b/embeds/cli/help/fluvio-run-metadata.md index 5972e2678..74f27f301 100644 --- a/embeds/cli/help/fluvio-run-metadata.md +++ b/embeds/cli/help/fluvio-run-metadata.md @@ -1,4 +1,8 @@ ``` -Return plugin metadata as JSON Options: -Usage: fluvio-run metadata -h, --help Print help +Return plugin metadata as JSON + +Usage: fluvio-run metadata + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-run-sc.md b/embeds/cli/help/fluvio-run-sc.md index d7160f5ec..4aecb277b 100644 --- a/embeds/cli/help/fluvio-run-sc.md +++ b/embeds/cli/help/fluvio-run-sc.md @@ -1,14 +1,19 @@ ``` Run a new Streaming Controller (SC) + Usage: fluvio-run sc [OPTIONS] + Options: --local running in local mode only + --no-k8 + run on k8 --bind-public Address for external service --bind-private Address for internal service -n, --namespace + --tls enable tls --server-cert diff --git a/embeds/cli/help/fluvio-run-spu.md b/embeds/cli/help/fluvio-run-spu.md index 52d6db8b9..99e622cdc 100644 --- a/embeds/cli/help/fluvio-run-spu.md +++ b/embeds/cli/help/fluvio-run-spu.md @@ -1,6 +1,8 @@ ``` Run a new Streaming Processing Unit (SPU) + Usage: fluvio-run spu [OPTIONS] + Options: -i, --id SPU unique identifier @@ -20,6 +22,8 @@ Options: [env: FLV_LOG_INDEX_MAX_INTERVAL_BYTES=] --peer-max-bytes max bytes to transfer between leader and follower [env: FLV_PEER_MAX_BYTES=] [default: 1000000] + --smart-engine-max-memory + [env: FLV_SMART_ENGINE_MAX_MEMORY_BYTES=] --tls enable tls --server-cert diff --git a/embeds/cli/help/fluvio-run-version.md b/embeds/cli/help/fluvio-run-version.md index 1f90188c0..631e9d2ef 100644 --- a/embeds/cli/help/fluvio-run-version.md +++ b/embeds/cli/help/fluvio-run-version.md @@ -1,4 +1,8 @@ ``` -Print version information Options: -Usage: fluvio-run version -h, --help Print help +Print version information + +Usage: fluvio-run version + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-run.md b/embeds/cli/help/fluvio-run.md index 3bdc4e2fd..7a4b32bc8 100644 --- a/embeds/cli/help/fluvio-run.md +++ b/embeds/cli/help/fluvio-run.md @@ -1,11 +1,13 @@ ``` Usage: fluvio-run + Commands: spu Run a new Streaming Processing Unit (SPU) sc Run a new Streaming Controller (SC) metadata Return plugin metadata as JSON version Print version information help Print this message or the help of the given subcommand(s) + Options: -h, --help Print help -V, --version Print version diff --git a/embeds/cli/help/fluvio-smartmodule-create.md b/embeds/cli/help/fluvio-smartmodule-create.md index 75548f6ab..4a8c1fe63 100644 --- a/embeds/cli/help/fluvio-smartmodule-create.md +++ b/embeds/cli/help/fluvio-smartmodule-create.md @@ -1,8 +1,11 @@ ``` Create a new SmartModule with a given name + Usage: fluvio smartmodule create [OPTIONS] --wasm-file + Arguments: The name of the SmartModule to create + Options: --wasm-file The path to a WASM binary to create the SmartModule from --package The path to the SmartModule package (experimental) diff --git a/embeds/cli/help/fluvio-smartmodule-delete.md b/embeds/cli/help/fluvio-smartmodule-delete.md index e31e21ba8..9593aa3c2 100644 --- a/embeds/cli/help/fluvio-smartmodule-delete.md +++ b/embeds/cli/help/fluvio-smartmodule-delete.md @@ -1,8 +1,11 @@ ``` Delete one or more SmartModules with the given name(s) + Usage: fluvio smartmodule delete [OPTIONS] ... + Arguments: ... One or more name(s) of the smartmodule(s) to be deleted + Options: -c, --continue-on-error Continue deleting in case of an error -h, --help Print help diff --git a/embeds/cli/help/fluvio-smartmodule-list.md b/embeds/cli/help/fluvio-smartmodule-list.md index 9b0209c21..238ad5efa 100644 --- a/embeds/cli/help/fluvio-smartmodule-list.md +++ b/embeds/cli/help/fluvio-smartmodule-list.md @@ -1,6 +1,8 @@ ``` List all existing SmartModules + Usage: fluvio smartmodule list [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] --filter diff --git a/embeds/cli/help/fluvio-smartmodule-watch.md b/embeds/cli/help/fluvio-smartmodule-watch.md index 2d5cbcdc9..9bf38a11e 100644 --- a/embeds/cli/help/fluvio-smartmodule-watch.md +++ b/embeds/cli/help/fluvio-smartmodule-watch.md @@ -1,6 +1,8 @@ ``` Watch for changes to SmartModules + Usage: fluvio smartmodule watch [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-smartmodule.md b/embeds/cli/help/fluvio-smartmodule.md index 792e7d574..51f0affb6 100644 --- a/embeds/cli/help/fluvio-smartmodule.md +++ b/embeds/cli/help/fluvio-smartmodule.md @@ -1,11 +1,17 @@ ``` Create and manage SmartModules + +SmartModules are compiled WASM modules used to create SmartModules. + Usage: fluvio smartmodule + Commands: create Create a new SmartModule with a given name list List all existing SmartModules watch Watch for changes to SmartModules delete Delete one or more SmartModules with the given name(s) + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-table-format-create.md b/embeds/cli/help/fluvio-table-format-create.md index eaf392528..fa5b242b2 100644 --- a/embeds/cli/help/fluvio-table-format-create.md +++ b/embeds/cli/help/fluvio-table-format-create.md @@ -1,6 +1,8 @@ ``` Create a new TableFormat display + fluvio table-format create --config + Options: -c, --config The path to the TableFormat config -h, --help Print help diff --git a/embeds/cli/help/fluvio-table-format-delete.md b/embeds/cli/help/fluvio-table-format-delete.md index bf656cb01..1987f3f59 100644 --- a/embeds/cli/help/fluvio-table-format-delete.md +++ b/embeds/cli/help/fluvio-table-format-delete.md @@ -1,8 +1,11 @@ ``` Delete a TableFormat display + fluvio table-format delete + Arguments: - The name of the connector to delete + The name of the table format to delete + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-table-format-list.md b/embeds/cli/help/fluvio-table-format-list.md index 9f1b26ec9..31ba4e7bf 100644 --- a/embeds/cli/help/fluvio-table-format-list.md +++ b/embeds/cli/help/fluvio-table-format-list.md @@ -1,6 +1,8 @@ ``` List all TableFormat display + fluvio table-format list [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-table-format.md b/embeds/cli/help/fluvio-table-format.md index 2d768bba5..7f8e49702 100644 --- a/embeds/cli/help/fluvio-table-format.md +++ b/embeds/cli/help/fluvio-table-format.md @@ -1,10 +1,16 @@ ``` Create a TableFormat display specification + +Used with the consumer output type `full_table` to describe how to render JSON data in a tabular form + Usage: fluvio table-format + Commands: create Create a new TableFormat display delete Delete a TableFormat display list List all TableFormat display + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-topic-create.md b/embeds/cli/help/fluvio-topic-create.md index 8ef496706..70c0bdc03 100644 --- a/embeds/cli/help/fluvio-topic-create.md +++ b/embeds/cli/help/fluvio-topic-create.md @@ -1,17 +1,55 @@ ``` Create a Topic with the given name -fluvio topic create [OPTIONS] + +fluvio topic create [OPTIONS] [name] + Arguments: - The name of the Topic to create + [name] + The name of the Topic to create + Options: - -p, --partitions The number of Partitions to give the Topic [default: 1] - -r, --replication The number of full replicas of the Topic to keep [default: 1] - -i, --ignore-rack-assignment Ignore racks while computing replica assignment - -f, --replica-assignment Replica assignment file - -d, --dry-run Validates configuration, does not provision - --retention-time