From 42723937c840460d5fddfbb40426d2e939ed3617 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Thu, 10 Aug 2023 19:18:59 +0000 Subject: [PATCH 01/19] docs: Update connector reference docs [bot] (#667) Updating connector docs Auto-generated by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action --- content/connectors/outbound/graphite.md | 5 ++ embeds/connectors/outbound/graphite.md | 97 +++++++++++++++++++++++++ 2 files changed, 102 insertions(+) create mode 100644 content/connectors/outbound/graphite.md create mode 100644 embeds/connectors/outbound/graphite.md diff --git a/content/connectors/outbound/graphite.md b/content/connectors/outbound/graphite.md new file mode 100644 index 00000000..41ba7b10 --- /dev/null +++ b/content/connectors/outbound/graphite.md @@ -0,0 +1,5 @@ +--- +menu: Graphite +--- + +{{% inline-embed file="embeds/connectors/outbound/graphite.md" %}} \ No newline at end of file diff --git a/embeds/connectors/outbound/graphite.md b/embeds/connectors/outbound/graphite.md new file mode 100644 index 00000000..c4ac9d1e --- /dev/null +++ b/embeds/connectors/outbound/graphite.md @@ -0,0 +1,97 @@ +# InfinyOn Graphite Sink Connector +The [Graphite][4] Sink connector reads records from Fluvio topic and sends them to +the configured Graphite's Metric using the PlainText approach. + +# Configuration + +This connectors establishes a TCP Stream against the specified host on Graphite, +records are sent as UTF-8 encoded strings following [Graphite's PlainText][5] format. + +The following example connector configuration can be used to send records to +the Graphite's Metric `weather.temperature.ca.sandiego`, the Graphite's TCP +server address is specified on the `addr` field. + +```yaml +# sample-config.yaml +apiVersion: 0.1.0 +meta: + version: 0.1.1 + name: my-graphite-connector-test-connector + type: graphite-sink + topic: test-graphite-connector-topic +graphite: + # https://graphite.readthedocs.io/en/latest/feeding-carbon.html#step-1-plan-a-naming-hierarchy + metric-path: "weather.temperature.ca.sandiego" + addr: "localhost:2003" +``` + +## Configuration Fields + +| Model | Data Type | Description | +|:----------------|:----------|:------------------------------------| +| `metric-path` | `String` | Graphite Metric to send records to | +| `addr` | `String` | Graphite TCP Adddress to stream out | + +## Usage + +In this section we are going to setup a Graphite instance on Docker and then +we will use the CDK to spin up the Graphite Sink Connector to send metrics from +Fluvio Records to the Graphite instance. + +Setup a Graphite instance for local development, you can follow the recipe used +to contribute to the official InfinyOn connector [here][3]. + +Reduce the Graphite's data retention interval by updating `storage-schemas.conf` +file for [Carbon][6]: + +```conf +[all] +pattern = .* +retentions = 10s:12h +``` + +Make sure the Connector Development Key is setup in your system by issuing +the following command in your terminal. + +```bash +fluvio install cdk +``` + +> If you dont have the Fluvio CLI installed already visit the [CLI][2] section + +Create a YAML file with the name `weather-monitor-config.yaml` and specify connector settings: + +```yaml +apiVersion: 0.1.0 +meta: + version: 0.1.1 + name: weather-monitor-sandiego + type: graphite-sink + topic: weather-ca-sandiego +graphite: + # https://graphite.readthedocs.io/en/latest/feeding-carbon.html#step-1-plan-a-naming-hierarchy + metric-path: "weather.temperature.ca.sandiego" + addr: "localhost:2003" +``` + +> Make sure your Graphite instance is running on `localhost:2003`, use the +> `cdk log` subcommand to read logs from the connector instance. + +Then produce records as usual: + +```bash +echo 120 | fluvio produce weather-ca-sandiego +``` + +You can check on Graphite's API for your data points: + +```bash +curl -o ./data.json http://localhost:12345/render\?target\=weather.temperature.ca.sandiego\&format\=json\&noNullPoints +``` + +[1]: https://infinyon.cloud/login +[2]: https://www.fluvio.io/cli/ +[3]: https://github.com/infinyon/graphite-sink-connector/blob/main/CONTRIBUTING.md +[4]: https://graphiteapp.org/ +[5]: https://graphite.readthedocs.io/en/latest/feeding-carbon.html#the-plaintext-protocol +[6]: https://graphite.readthedocs.io/en/latest/config-carbon.html#storage-schemas-conf From d733f9cf2d290eff389101f522896a1e138806b5 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sat, 12 Aug 2023 00:24:14 +0000 Subject: [PATCH 02/19] docs: Update connector reference docs [bot] (#668) Updating connector docs Auto-generated by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action --- embeds/connectors/outbound/graphite.md | 68 +++++++++++++++++++++----- 1 file changed, 57 insertions(+), 11 deletions(-) diff --git a/embeds/connectors/outbound/graphite.md b/embeds/connectors/outbound/graphite.md index c4ac9d1e..877aaa2e 100644 --- a/embeds/connectors/outbound/graphite.md +++ b/embeds/connectors/outbound/graphite.md @@ -15,7 +15,7 @@ server address is specified on the `addr` field. # sample-config.yaml apiVersion: 0.1.0 meta: - version: 0.1.1 + version: 0.1.2 name: my-graphite-connector-test-connector type: graphite-sink topic: test-graphite-connector-topic @@ -34,15 +34,31 @@ graphite: ## Usage -In this section we are going to setup a Graphite instance on Docker and then -we will use the CDK to spin up the Graphite Sink Connector to send metrics from -Fluvio Records to the Graphite instance. +This section will walk you through the process of setting up a Graphite +instance and using Fluvio to send metrics to this Graphite instance. -Setup a Graphite instance for local development, you can follow the recipe used -to contribute to the official InfinyOn connector [here][3]. +> This section assumes you have Docker and Fluvio installed in your system. -Reduce the Graphite's data retention interval by updating `storage-schemas.conf` -file for [Carbon][6]: +### Setting Up Graphite + +We will run our Graphite instance on Docker using the `docker compose` command +for simplicity. + +The Graphite container will setup [Carbon Configuration][6] files in your +working directory, we need to update these files to reduce Carbon's persisntance +intervals, making it more frecuent. + +Create a copy of our [`docker-compose.yml`][7] file and execute the container: + +```bash +docker compose up --build -d +``` + +This will generate a directory with the name `.graphite`, which contains +configuration files. + +Replace the contents of `.graphite/conf/storage-schemas.conf` to record on an +interval of 10 seconds and persist the last 12 hours of data. ```conf [all] @@ -50,7 +66,26 @@ pattern = .* retentions = 10s:12h ``` -Make sure the Connector Development Key is setup in your system by issuing +Now we need to re run the Graphite container so Carbon uses the new +configuration. + +```bash +docker compose down +docker compose up --build -d +``` + +You can visit `http://localhost:12345` in your browser to access the Dashboard. + +> Credentials for the Dashbord are User: `root` and Password: `root` + +With the Graphite instance set, we can move into [Setting Up Fluvio with Graphite Sink Connector][8]. + +### Setting Up Fluvio with Graphite Sink Connector + +In this section we are going use the CDK to spin up the Graphite Sink Connector +to send metrics from Fluvio Records to the Graphite instance. + +Make sure the Connector Development Kit is setup in your system by issuing the following command in your terminal. ```bash @@ -64,7 +99,7 @@ Create a YAML file with the name `weather-monitor-config.yaml` and specify conne ```yaml apiVersion: 0.1.0 meta: - version: 0.1.1 + version: 0.1.2 name: weather-monitor-sandiego type: graphite-sink topic: weather-ca-sandiego @@ -74,6 +109,12 @@ graphite: addr: "localhost:2003" ``` +Deploy the Connector using the CDK + +```bash +cdk deploy start --config weather-monitor-config.yaml +``` + > Make sure your Graphite instance is running on `localhost:2003`, use the > `cdk log` subcommand to read logs from the connector instance. @@ -83,7 +124,10 @@ Then produce records as usual: echo 120 | fluvio produce weather-ca-sandiego ``` -You can check on Graphite's API for your data points: +> Remember that Carbon's retention is set to `10s:12h`, this means that if will +> write metrics every 10s. + +Use Graphite's REST API to check on the stored data. ```bash curl -o ./data.json http://localhost:12345/render\?target\=weather.temperature.ca.sandiego\&format\=json\&noNullPoints @@ -95,3 +139,5 @@ curl -o ./data.json http://localhost:12345/render\?target\=weather.temperature.c [4]: https://graphiteapp.org/ [5]: https://graphite.readthedocs.io/en/latest/feeding-carbon.html#the-plaintext-protocol [6]: https://graphite.readthedocs.io/en/latest/config-carbon.html#storage-schemas-conf +[7]: https://github.com/infinyon/graphite-sink-connector/blob/main/docker-compose.yml +[8]: #setting-up-fluvio-with-graphite-sink-connector From a6d8774ccd383003c756453b14875ca840497ce9 Mon Sep 17 00:00:00 2001 From: Alexander Galibey Date: Tue, 15 Aug 2023 09:18:56 +0000 Subject: [PATCH 03/19] mention `age` lookback configuration in docs (#669) --- content/smartmodules/lookback.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/content/smartmodules/lookback.md b/content/smartmodules/lookback.md index 71d1145c..3790438e 100644 --- a/content/smartmodules/lookback.md +++ b/content/smartmodules/lookback.md @@ -14,12 +14,16 @@ pub fn look_back(record: &Record) -> Result<()> { ... } ``` -- `lookback` parameter is specified in transform configuration. It defines how many lookback records SmartModule receives: +- `lookback` parameter is specified in transform configuration. It defines a set of lookback records that SmartModule receives. Supported ways to define the set: + - by size (last N records existing in the topic) + - by age (records that are younger than the specified age) + - by size and age (max N records younger than the specified age) ```yaml transforms: - uses: local/filter-with-lookback@0.1.0 lookback: - last: 10 # we want last 10 records + age: 30m # we want only records that are younger than 30 minutes + last: 10 # we want maximum of 10 records (last) ``` --- If Fluvio topic is empty, `look_back` is never called. From 36da68d8e5a79e2bc8cdcc736ffbc1864b6afcf5 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 23 Aug 2023 14:01:22 +0000 Subject: [PATCH 04/19] chore(deps): bump @types/node from 20.3.2 to 20.5.3 in /functions/newsletter (#676) Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 20.3.2 to 20.5.3.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=@types/node&package-manager=npm_and_yarn&previous-version=20.3.2&new-version=20.5.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
--- functions/newsletter/package-lock.json | 6 +++--- functions/newsletter/package.json | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/functions/newsletter/package-lock.json b/functions/newsletter/package-lock.json index 1fde8437..995d045a 100644 --- a/functions/newsletter/package-lock.json +++ b/functions/newsletter/package-lock.json @@ -13,9 +13,9 @@ } }, "@types/node": { - "version": "20.3.2", - "resolved": "https://registry.npmjs.org/@types/node/-/node-20.3.2.tgz", - "integrity": "sha512-vOBLVQeCQfIcF/2Y7eKFTqrMnizK5lRNQ7ykML/5RuwVXVWxYkgwS7xbt4B6fKCUPgbSL5FSsjHQpaGQP/dQmw==" + "version": "20.5.3", + "resolved": "https://registry.npmjs.org/@types/node/-/node-20.5.3.tgz", + "integrity": "sha512-ITI7rbWczR8a/S6qjAW7DMqxqFMjjTo61qZVWJ1ubPvbIQsL5D/TvwjYEalM8Kthpe3hTzOGrF2TGbAu2uyqeA==" }, "data-uri-to-buffer": { "version": "4.0.0", diff --git a/functions/newsletter/package.json b/functions/newsletter/package.json index 4b599d94..b6e5fe03 100644 --- a/functions/newsletter/package.json +++ b/functions/newsletter/package.json @@ -15,7 +15,7 @@ "license": "MIT", "dependencies": { "@netlify/functions": "^1.6.0", - "@types/node": "^20.3.2", + "@types/node": "^20.5.3", "node-fetch": "^3.3.0", "typescript": "^5.1.3" } From 33c43bbc6dd00cdd5b9bcda005c4182e67f09aef Mon Sep 17 00:00:00 2001 From: Esteban Borai Date: Wed, 23 Aug 2023 19:58:15 +0000 Subject: [PATCH 05/19] feat: smdk docs for `SmartModuleRecord` (#678) Introduces SmartModuleRecord to the documentation, and updates examples to encourage update. --- content/connectors-old/examples/github.md | 8 +-- .../how-to/connectors-with-smartmodules.md | 4 +- content/smartmodules/analytics/aggregate.md | 4 +- content/smartmodules/lookback.md | 2 +- content/smartmodules/smdk/build-test.md | 2 +- content/smartmodules/smdk/generate.md | 4 +- content/smartmodules/smdk/load.md | 2 +- content/smartmodules/smdk/publish.md | 2 +- .../smartmodules/smdk/smartmodulerecord.md | 58 +++++++++++++++++++ content/smartmodules/transform/array-map.md | 4 +- content/smartmodules/transform/filter-map.md | 4 +- content/smartmodules/transform/filter.md | 12 ++-- content/smartmodules/transform/map.md | 4 +- embeds/smartmodules/lookback-example.md | 6 +- .../sm-development/regex-filter/src/lib.rs | 4 +- 15 files changed, 89 insertions(+), 31 deletions(-) create mode 100644 content/smartmodules/smdk/smartmodulerecord.md diff --git a/content/connectors-old/examples/github.md b/content/connectors-old/examples/github.md index 6885ae33..00f28e73 100644 --- a/content/connectors-old/examples/github.md +++ b/content/connectors-old/examples/github.md @@ -262,11 +262,11 @@ Copy and paste this code into the `src/lib.rs` file of the project. %copy% ```rust -use fluvio_smartmodule::{smartmodule, Record, RecordData, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, RecordData, Result}; use serde_json::{json, Value as JsonValue}; #[smartmodule(map)] -fn shaper(record: &Record) -> Result<(Option, RecordData)> { +fn shaper(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { let repo: JsonValue = serde_json::from_slice(record.value.as_ref())?; let shaped = json!({ @@ -569,11 +569,11 @@ Copy and paste this code into the `src/lib.rs` file of the project. %copy% ```rust -use fluvio_smartmodule::{smartmodule, Record, RecordData, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, RecordData, Result}; use serde_json::{json, Value as JsonValue}; #[smartmodule(map)] -fn shaper(record: &Record) -> Result<(Option, RecordData)> { +fn shaper(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { let repo: JsonValue = serde_json::from_slice(record.value.as_ref())?; let shaped = json!({ diff --git a/content/connectors-old/how-to/connectors-with-smartmodules.md b/content/connectors-old/how-to/connectors-with-smartmodules.md index 651abdb5..64a02292 100644 --- a/content/connectors-old/how-to/connectors-with-smartmodules.md +++ b/content/connectors-old/how-to/connectors-with-smartmodules.md @@ -77,11 +77,11 @@ to have the following contents: %copy% ```rust -use fluvio_smartmodule::{smartmodule, Result, Record, RecordData}; +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord, RecordData}; use serde_json::Value; #[smartmodule(map)] -pub fn map(record: &Record) -> Result<(Option, RecordData)> { +pub fn map(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { let input: Value = serde_json::from_slice(record.value.as_ref())?; let fact = &input["fact"]; let output = serde_json::to_string(fact)?; diff --git a/content/smartmodules/analytics/aggregate.md b/content/smartmodules/analytics/aggregate.md index 077e96ef..fe695b4e 100644 --- a/content/smartmodules/analytics/aggregate.md +++ b/content/smartmodules/analytics/aggregate.md @@ -59,10 +59,10 @@ $ cd example-aggregate && cat ./src/lib.rs ``` ```rust -use fluvio_smartmodule::{smartmodule, Result, Record, RecordData}; +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord, RecordData}; #[smartmodule(aggregate)] -pub fn aggregate(accumulator: RecordData, current: &Record) -> Result { +pub fn aggregate(accumulator: RecordData, current: &SmartModuleRecord) -> Result { // Parse the accumulator and current record as strings let accumulator_string = std::str::from_utf8(accumulator.as_ref())?; let current_string = std::str::from_utf8(current.value.as_ref())?; diff --git a/content/smartmodules/lookback.md b/content/smartmodules/lookback.md index 3790438e..b5763abd 100644 --- a/content/smartmodules/lookback.md +++ b/content/smartmodules/lookback.md @@ -10,7 +10,7 @@ To activate Lookback SmartModule must fit the following criteria: - A function annotated with `#[smartmodule(look_back)]` macro is present in the code. This function is used to pass requested records to SmartModule: ```rust #[smartmodule(look_back)] -pub fn look_back(record: &Record) -> Result<()> { +pub fn look_back(record: &SmartModuleRecord) -> Result<()> { ... } ``` diff --git a/content/smartmodules/smdk/build-test.md b/content/smartmodules/smdk/build-test.md index e407cd52..b7288ef0 100644 --- a/content/smartmodules/smdk/build-test.md +++ b/content/smartmodules/smdk/build-test.md @@ -1,7 +1,7 @@ --- title: SMDK - Build & Test a SmartModule menu: Build & Test -weight: 40 +weight: 50 toc: false --- diff --git a/content/smartmodules/smdk/generate.md b/content/smartmodules/smdk/generate.md index 82c69f74..f26fdc9f 100644 --- a/content/smartmodules/smdk/generate.md +++ b/content/smartmodules/smdk/generate.md @@ -53,10 +53,10 @@ $ tree This is a simple SmartModule `filter` matching for all data records that contains letter `a`: ```bash -use fluvio_smartmodule::{smartmodule, Result, Record}; +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord}; #[smartmodule(filter)] -pub fn filter(record: &Record) -> Result { +pub fn filter(record: &SmartModuleRecord) -> Result { let string = std::str::from_utf8(record.value.as_ref())?; Ok(string.contains('a')) } diff --git a/content/smartmodules/smdk/load.md b/content/smartmodules/smdk/load.md index 0d49b000..53a2fbe5 100644 --- a/content/smartmodules/smdk/load.md +++ b/content/smartmodules/smdk/load.md @@ -1,7 +1,7 @@ --- title: SMDK - Load to Cluster menu: Load -weight: 50 +weight: 60 toc: false --- diff --git a/content/smartmodules/smdk/publish.md b/content/smartmodules/smdk/publish.md index ac04a481..be685f45 100644 --- a/content/smartmodules/smdk/publish.md +++ b/content/smartmodules/smdk/publish.md @@ -1,7 +1,7 @@ --- title: SMDK - Publish to Hub menu: Publish -weight: 60 +weight: 70 toc: false --- diff --git a/content/smartmodules/smdk/smartmodulerecord.md b/content/smartmodules/smdk/smartmodulerecord.md new file mode 100644 index 00000000..f972f4ba --- /dev/null +++ b/content/smartmodules/smdk/smartmodulerecord.md @@ -0,0 +1,58 @@ +--- +title: SMDK - SmartModuleRecord +menu: SmartModuleRecord +weight: 40 +toc: false +--- + +Inside the SmartModule Context, Fluvio `Record` becomes `SmartModuleRecord`. + +`SmartModuleRecord` is a wrapper on the standard `Record`, which provides extra +details on the `Record` such as the moment when the `Record` was produced. + +-> `SmartModuleRecord` was introduced on [Fluvio v0.10.14][1], we recommend upgrading to `SmartModuleRecord` as future versions will completely deprecate `Record` in the SmartModule Context. + +## Usage + +When building any type of SmartModules you will have access to each record, in +the SmartModule function these records are represented with the +`SmartModuleRecord` struct. + +Consider the following SmartModule of type `Map`, which processes weather data: + +```rust +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord, RecordData}; + +#[smartmodule(map)] +pub fn map(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { + let string = std::str::from_utf8(record.value.as_ref())?; // parses UTF-8 string from bytes + let feels_like = string.parse::()?; // attemps to parse the string value into a `i32` + let feels_like = feels_like.to_string(); // turs the integer into a string + + Ok((key, feels_like.into())) // returs to the stream +} +``` + +For weather casting it would be helpful to also have access to the moment when +the data was captured. We could have our wather value mapped to a string which +also holds a timestamp. + +```diff +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord, RecordData}; + +#[smartmodule(map)] +pub fn map(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { + let string = std::str::from_utf8(record.value.as_ref())?; // parses UTF-8 string from bytes + let feels_like = string.parse::()?; // attemps to parse the string value into a `i32` +- let feels_like = feels_like.to_string(); // turs the integer into a string ++ let feels_like_with_timestamp = format!("{}_{}", record.timestamp(), feels_like); + + Ok((key, feels_like_with_timestamp.into())) // returs to the stream +} +``` + +With this small change we are injecting insightful data to our record. This is +a simple use case but we can have more based on the implementation and business +requirements. + +[1]: https://github.com/infinyon/fluvio/blob/master/CHANGELOG.md#platform-version-01014---2023-07-28 diff --git a/content/smartmodules/transform/array-map.md b/content/smartmodules/transform/array-map.md index e7380b32..c0bbd123 100644 --- a/content/smartmodules/transform/array-map.md +++ b/content/smartmodules/transform/array-map.md @@ -70,10 +70,10 @@ $ cd array-map && cat src/lib.rs ``` ```rust -use fluvio_smartmodule::{smartmodule, Record, RecordData, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, RecordData, Result}; #[smartmodule(array_map)] -pub fn array_map(record: &Record) -> Result, RecordData)>> { +pub fn array_map(record: &SmartModuleRecord) -> Result, RecordData)>> { // Deserialize a JSON array with any kind of values inside let array: Vec = serde_json::from_slice(record.value.as_ref())?; diff --git a/content/smartmodules/transform/filter-map.md b/content/smartmodules/transform/filter-map.md index aebb188f..a6b18f27 100644 --- a/content/smartmodules/transform/filter-map.md +++ b/content/smartmodules/transform/filter-map.md @@ -83,7 +83,7 @@ Now, let's jump right into the code. Copy and paste the following block into the %copy% ```rust -use fluvio_smartmodule::{smartmodule, Record, RecordData, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, RecordData, Result}; use serde::{Deserialize, Serialize}; /// Events that may take place in an online grocery service @@ -104,7 +104,7 @@ enum GroceryEvent { } #[smartmodule(filter_map)] -fn filter_map(record: &Record) -> Result, RecordData)>> { +fn filter_map(record: &SmartModuleRecord) -> Result, RecordData)>> { let event: GroceryEvent = match serde_json::from_slice(record.value.as_ref()) { Ok(event) => event, Err(_) => return Ok(None), // Skip if we fail to parse JSON diff --git a/content/smartmodules/transform/filter.md b/content/smartmodules/transform/filter.md index 08394e01..7e903fe1 100644 --- a/content/smartmodules/transform/filter.md +++ b/content/smartmodules/transform/filter.md @@ -95,10 +95,10 @@ $ cd json-filter && cat ./src/lib.rs ``` ```rust -use fluvio_smartmodule::{smartmodule, Result, Record}; +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord}; #[smartmodule(filter)] -pub fn filter(record: &Record) -> Result { +pub fn filter(record: &SmartModuleRecord) -> Result { let string = std::str::from_utf8(record.value.as_ref())?; Ok(string.contains('a')) } @@ -146,10 +146,10 @@ Now, let's write the logic for our filter. We'll start by parsing our raw data i instances of `StructuredLog`. ```rust -use fluvio_smartmodule::{smartmodule, Record, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, Result}; #[smartmodule(filter)] -fn filter(record: &Record) -> Result { +fn filter(record: &SmartModuleRecord) -> Result { let log: StructuredLog = serde_json::from_slice(record.value.as_ref())?; todo!() @@ -165,7 +165,7 @@ will be a piece of cake! Let's look at all the code for the finished filter. %copy% ```rust -use fluvio_smartmodule::{smartmodule, Record, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, Result}; #[derive(PartialEq, Eq, PartialOrd, Ord, serde::Deserialize)] #[serde(rename_all = "lowercase")] @@ -184,7 +184,7 @@ struct StructuredLog { } #[smartmodule(filter)] -fn filter(record: &Record) -> Result { +fn filter(record: &SmartModuleRecord) -> Result { let log: StructuredLog = serde_json::from_slice(record.value.as_ref())?; // We keep records that are "greater than" debug diff --git a/content/smartmodules/transform/map.md b/content/smartmodules/transform/map.md index 37165fce..7b78a7c0 100644 --- a/content/smartmodules/transform/map.md +++ b/content/smartmodules/transform/map.md @@ -56,10 +56,10 @@ $ cd map-example && cat ./src/lib.rs %copy% ```rust -use fluvio_smartmodule::{smartmodule, Result, Record, RecordData}; +use fluvio_smartmodule::{smartmodule, Result, SmartModuleRecord, RecordData}; #[smartmodule(map)] -pub fn map(record: &Record) -> Result<(Option, RecordData)> { +pub fn map(record: &SmartModuleRecord) -> Result<(Option, RecordData)> { let key = record.key.clone(); let string = std::str::from_utf8(record.value.as_ref())?; diff --git a/embeds/smartmodules/lookback-example.md b/embeds/smartmodules/lookback-example.md index b20d788b..c3c6f7ca 100644 --- a/embeds/smartmodules/lookback-example.md +++ b/embeds/smartmodules/lookback-example.md @@ -1,10 +1,10 @@ use std::sync::atomic::{AtomicI32, Ordering::SeqCst}; -use fluvio_smartmodule::{smartmodule, Record, Result}; +use fluvio_smartmodule::{smartmodule, SmartModuleRecord, Result}; static PREV: AtomicI32 = AtomicI32::new(0); #[smartmodule(filter)] -pub fn filter(record: &Record) -> Result { +pub fn filter(record: &SmartModuleRecord) -> Result { let string = std::str::from_utf8(record.value.as_ref())?; let current: i32 = string.parse()?; let last = PREV.load(SeqCst); @@ -17,7 +17,7 @@ pub fn filter(record: &Record) -> Result { } #[smartmodule(look_back)] -pub fn look_back(record: &Record) -> Result<()> { +pub fn look_back(record: &SmartModuleRecord) -> Result<()> { let string = std::str::from_utf8(record.value.as_ref())?; let last: i32 = string.parse()?; PREV.store(last, SeqCst); diff --git a/embeds/tutorials/sm-development/regex-filter/src/lib.rs b/embeds/tutorials/sm-development/regex-filter/src/lib.rs index 0ddd92ff..3934095a 100644 --- a/embeds/tutorials/sm-development/regex-filter/src/lib.rs +++ b/embeds/tutorials/sm-development/regex-filter/src/lib.rs @@ -4,7 +4,7 @@ use regex::Regex; use fluvio_smartmodule::{ dataplane::smartmodule::{SmartModuleExtraParams, SmartModuleInitError}, - eyre, smartmodule, Record, Result, + eyre, smartmodule, SmartModuleRecord, Result, }; static REGEX: OnceCell = OnceCell::new(); @@ -21,7 +21,7 @@ fn init(params: SmartModuleExtraParams) -> Result<()> { } #[smartmodule(filter)] -pub fn filter(record: &Record) -> Result { +pub fn filter(record: &SmartModuleRecord) -> Result { let string = std::str::from_utf8(record.value.as_ref())?; Ok(REGEX.get().unwrap().is_match(string)) } From cb38604ffbaf7b09b828d126cd9b08309c6545ca Mon Sep 17 00:00:00 2001 From: John Landahl Date: Tue, 29 Aug 2023 18:07:13 +0000 Subject: [PATCH 06/19] Typo fix (#682) --- content/docs/kubernetes/external-access.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/kubernetes/external-access.md b/content/docs/kubernetes/external-access.md index dfa181be..e7bb1fd1 100644 --- a/content/docs/kubernetes/external-access.md +++ b/content/docs/kubernetes/external-access.md @@ -13,7 +13,7 @@ Port numbers do not change based on whether TLS is enabled, the same ports are u Service behavior can be configured at installation by overriding values in the helm charts. -### Service Enpoint Details +### Service Endpoint Details Run `kubectl describe svc fluvio-sc-public` to view address and port information for accessing the SC public endpoint. From be9d10a35d4da4153aa3ecbdd30f2d4a453d5649 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 5 Sep 2023 19:38:19 +0000 Subject: [PATCH 07/19] chore(deps): bump actions/checkout from 3 to 4 (#685) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
Release notes

Sourced from actions/checkout's releases.

v4.0.0

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v3...v4.0.0

v3.6.0

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v3.5.3...v3.6.0

v3.5.3

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v3...v3.5.3

v3.5.2

What's Changed

Full Changelog: https://github.com/actions/checkout/compare/v3.5.1...v3.5.2

v3.5.1

What's Changed

New Contributors

... (truncated)

Changelog

Sourced from actions/checkout's changelog.

Changelog

v4.0.0

v3.6.0

v3.5.3

v3.5.2

v3.5.1

v3.5.0

v3.4.0

v3.3.0

v3.2.0

v3.1.0

v3.0.2

v3.0.1

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/checkout&package-manager=github_actions&previous-version=3&new-version=4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
--- .github/workflows/check_examples.yml | 4 ++-- .github/workflows/htmltest.yml | 2 +- .github/workflows/screenshots.yml | 4 ++-- .github/workflows/skip-test-check.yml | 4 ++-- .github/workflows/update-connector-docs.yml | 2 +- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/.github/workflows/check_examples.yml b/.github/workflows/check_examples.yml index 2b8488ed..5fb92bab 100644 --- a/.github/workflows/check_examples.yml +++ b/.github/workflows/check_examples.yml @@ -43,10 +43,10 @@ jobs: steps: - name: Dump context uses: crazy-max/ghaction-dump-context@v2.0.0 - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 if: ${{ needs.test-skip-check.outputs.no-skip }} - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 if: ${{ needs.test-skip-check.outputs.no-skip }} with: repository: infinyon/fluvio diff --git a/.github/workflows/htmltest.yml b/.github/workflows/htmltest.yml index 1110a804..cf1d27df 100644 --- a/.github/workflows/htmltest.yml +++ b/.github/workflows/htmltest.yml @@ -35,7 +35,7 @@ jobs: - name: Checkout code if: ${{ needs.test-skip-check.outputs.no-skip }} - uses: actions/checkout@v3 + uses: actions/checkout@v4 with: fetch-depth: 1 - name: Setup Hugo diff --git a/.github/workflows/screenshots.yml b/.github/workflows/screenshots.yml index a0b29d20..5548b443 100644 --- a/.github/workflows/screenshots.yml +++ b/.github/workflows/screenshots.yml @@ -35,12 +35,12 @@ jobs: run: npm install --global capture-website-cli # Checkout and build files of a base ref. - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 with: ref: ${{ github.event.pull_request.base.sha }} path: current_repo # Checkout and build files of a head ref. - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 with: ref: ${{ github.event.pull_request.head.sha }} clean: false # Prevent removing files in 'path/to/base' folder. diff --git a/.github/workflows/skip-test-check.yml b/.github/workflows/skip-test-check.yml index d97628d3..21247644 100644 --- a/.github/workflows/skip-test-check.yml +++ b/.github/workflows/skip-test-check.yml @@ -24,14 +24,14 @@ jobs: no-skip: ${{ ! steps.outputs.skip_label_check.result }} steps: - name: Checkout code - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Dump context uses: crazy-max/ghaction-dump-context@v2.0.0 - name: Checkout code for push if: github.event_name == 'push' && github.ref == 'refs/heads/staging' - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Get PR Number from event if: github.event_name == 'pull_request' diff --git a/.github/workflows/update-connector-docs.yml b/.github/workflows/update-connector-docs.yml index 288e8ec5..4a8aab2c 100644 --- a/.github/workflows/update-connector-docs.yml +++ b/.github/workflows/update-connector-docs.yml @@ -17,7 +17,7 @@ jobs: pull-requests: write runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install Fluvio run: | curl -fsS https://hub.infinyon.cloud/install/install.sh | bash From e0f63c2eb819f562ffc7e95e30c61eb1562726cc Mon Sep 17 00:00:00 2001 From: Alexander Galibey Date: Fri, 1 Sep 2023 15:28:27 +0400 Subject: [PATCH 08/19] update connectors docs --- embeds/connectors/inbound/http.md | 6 +++--- embeds/connectors/inbound/kafka.md | 2 +- embeds/connectors/inbound/mqtt.md | 4 ++-- embeds/connectors/outbound/http.md | 6 +++--- embeds/connectors/outbound/sql.md | 8 ++++---- 5 files changed, 13 insertions(+), 13 deletions(-) diff --git a/embeds/connectors/inbound/http.md b/embeds/connectors/inbound/http.md index 544d2083..99a54d37 100644 --- a/embeds/connectors/inbound/http.md +++ b/embeds/connectors/inbound/http.md @@ -36,7 +36,7 @@ This is an example of simple connector config file: # config-example.yaml apiVersion: 0.1.0 meta: - version: 0.2.4 + version: 0.2.5 name: cat-facts type: http-source topic: cat-facts @@ -66,7 +66,7 @@ Fluvio HTTP Source Connector supports Secrets in the `endpoint` and in the `head # config-example.yaml apiVersion: 0.1.0 meta: - version: 0.2.4 + version: 0.2.5 name: cat-facts type: http-source topic: cat-facts @@ -92,7 +92,7 @@ The previous example can be extended to add extra transformations to outgoing re # config-example.yaml apiVersion: 0.1.0 meta: - version: 0.2.4 + version: 0.2.5 name: cat-facts type: http-source topic: cat-facts diff --git a/embeds/connectors/inbound/kafka.md b/embeds/connectors/inbound/kafka.md index 8969e3a6..fbd08595 100644 --- a/embeds/connectors/inbound/kafka.md +++ b/embeds/connectors/inbound/kafka.md @@ -17,7 +17,7 @@ Example: ```yaml apiVersion: 0.1.0 meta: - version: 0.2.4 + version: 0.2.5 name: my-kafka-connector type: kafka-source topic: kafka-topic diff --git a/embeds/connectors/inbound/mqtt.md b/embeds/connectors/inbound/mqtt.md index 05f96cfa..129bf596 100644 --- a/embeds/connectors/inbound/mqtt.md +++ b/embeds/connectors/inbound/mqtt.md @@ -50,7 +50,7 @@ This is an example of connector config file: # config-example.yaml apiVersion: 0.1.0 meta: - version: 0.2.4 + version: 0.2.5 name: my-mqtt-connector type: mqtt-source topic: mqtt-topic @@ -105,7 +105,7 @@ The previous example can be extended to add extra transformations to outgoing re # config-example.yaml apiVersion: 0.1.0 meta: - version: 0.2.4 + version: 0.2.5 name: my-mqtt-connector type: mqtt-source topic: mqtt-topic diff --git a/embeds/connectors/outbound/http.md b/embeds/connectors/outbound/http.md index 12087a2a..53d521c7 100644 --- a/embeds/connectors/outbound/http.md +++ b/embeds/connectors/outbound/http.md @@ -16,7 +16,7 @@ HTTP Sink is configured using a YAML file: # config-example.yaml apiVersion: 0.1.0 meta: - version: 0.2.4 + version: 0.2.5 name: my-http-sink type: http-sink topic: http-sink-topic @@ -57,7 +57,7 @@ HTTP request to `http://httpbin.org/post`. # config.yaml apiVersion: 0.1.0 meta: - version: 0.2.4 + version: 0.2.5 name: httpbin type: http-sink topic: httpbin-send-post @@ -133,7 +133,7 @@ The previous example can be extended to add extra transformations to outgoing re # config-example.yaml apiVersion: 0.1.0 meta: - version: 0.2.4 + version: 0.2.5 name: my-http-sink type: http-sink topic: http-sink-topic diff --git a/embeds/connectors/outbound/sql.md b/embeds/connectors/outbound/sql.md index 6107f657..746ab30c 100644 --- a/embeds/connectors/outbound/sql.md +++ b/embeds/connectors/outbound/sql.md @@ -40,7 +40,7 @@ in the config. If a SmartModule requires configuration, it is passed via `with` ```yaml apiVersion: 0.1.0 meta: - version: 0.3.1 + version: 0.3.3 name: my-sql-connector type: sql-sink topic: sql-topic @@ -62,7 +62,7 @@ The connector can use secrets in order to hide sensitive information. ```yaml apiVersion: 0.1.0 meta: - version: 0.3.1 + version: 0.3.3 name: my-sql-connector type: sql-sink topic: sql-topic @@ -95,7 +95,7 @@ Connector configuration file: # connector-config.yaml apiVersion: 0.1.0 meta: - version: 0.3.1 + version: 0.3.3 name: json-sql-connector type: sql-sink topic: sql-topic @@ -156,7 +156,7 @@ Connector configuration file for upsert (assuming `device_id` is a unique column # connector-config.yaml apiVersion: 0.1.0 meta: - version: 0.3.1 + version: 0.3.3 name: json-sql-connector type: sql-sink topic: sql-topic From ff7da1aac23f8ad3db425abc6d1fe6a128c84ee7 Mon Sep 17 00:00:00 2001 From: Alexander Galibey Date: Tue, 5 Sep 2023 23:42:00 +0000 Subject: [PATCH 09/19] update connectors docs (#684) Updated docs of released connectors Co-authored-by: A.J. Hunyady From f0a8b9506ad283e90f2f49dda5f8fd546e0c3e36 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Wed, 6 Sep 2023 04:27:15 +0000 Subject: [PATCH 10/19] docs: Update connector reference docs [bot] (#683) Updating connector docs Auto-generated by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action --- embeds/connectors/outbound/kafka.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/embeds/connectors/outbound/kafka.md b/embeds/connectors/outbound/kafka.md index 690f44b7..ebbaf713 100644 --- a/embeds/connectors/outbound/kafka.md +++ b/embeds/connectors/outbound/kafka.md @@ -29,7 +29,7 @@ Example without security: ```yaml apiVersion: 0.1.0 meta: - version: 0.2.4 + version: 0.2.7 name: my-kafka-connector type: kafka-sink topic: kafka-topic @@ -44,7 +44,7 @@ Example with security enabled: ```yaml apiVersion: 0.1.0 meta: - version: 0.2.4 + version: 0.2.7 name: my-kafka-connector type: kafka-sink topic: kafka-topic @@ -74,5 +74,10 @@ fluvio install cdk cdk deploy -p kafka-sink start --config crates/kafka-sink/config-example.yaml ``` +### Testing with security +[Instructions](https://github.com/galibey/kafka-docker-ssl) of how to deploy local kafka cluster with SSL using docker. +After all steps done, in the `secrets` folder there will be `fluvio.key.pem`, `fluvio.pem` and `fake-ca-1.crt` files that can be used +in the connector config as `ssl_key`, `ssl_cert` and `ssl_ca` correspondingly. + ## Transformations Fluvio Kafka Connectors support [Transformations](https://www.fluvio.io/docs/concepts/transformations-chain/). From 354709d0b91d350feec038238c71587e6e0a1dca Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 12 Sep 2023 00:45:07 +0000 Subject: [PATCH 11/19] chore(deps): Bump @types/node from 20.5.3 to 20.6.0 in /functions/newsletter (#686) Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 20.5.3 to 20.6.0.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=@types/node&package-manager=npm_and_yarn&previous-version=20.5.3&new-version=20.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
--- functions/newsletter/package-lock.json | 6 +++--- functions/newsletter/package.json | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/functions/newsletter/package-lock.json b/functions/newsletter/package-lock.json index 995d045a..ed6f5cda 100644 --- a/functions/newsletter/package-lock.json +++ b/functions/newsletter/package-lock.json @@ -13,9 +13,9 @@ } }, "@types/node": { - "version": "20.5.3", - "resolved": "https://registry.npmjs.org/@types/node/-/node-20.5.3.tgz", - "integrity": "sha512-ITI7rbWczR8a/S6qjAW7DMqxqFMjjTo61qZVWJ1ubPvbIQsL5D/TvwjYEalM8Kthpe3hTzOGrF2TGbAu2uyqeA==" + "version": "20.6.0", + "resolved": "https://registry.npmjs.org/@types/node/-/node-20.6.0.tgz", + "integrity": "sha512-najjVq5KN2vsH2U/xyh2opaSEz6cZMR2SetLIlxlj08nOcmPOemJmUK2o4kUzfLqfrWE0PIrNeE16XhYDd3nqg==" }, "data-uri-to-buffer": { "version": "4.0.0", diff --git a/functions/newsletter/package.json b/functions/newsletter/package.json index b6e5fe03..ed387082 100644 --- a/functions/newsletter/package.json +++ b/functions/newsletter/package.json @@ -15,7 +15,7 @@ "license": "MIT", "dependencies": { "@netlify/functions": "^1.6.0", - "@types/node": "^20.5.3", + "@types/node": "^20.6.0", "node-fetch": "^3.3.0", "typescript": "^5.1.3" } From b2573b392160a1c3290b4da0d9c646eab4993fef Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 27 Sep 2023 01:53:35 +0000 Subject: [PATCH 12/19] chore(deps): Bump @types/node from 20.6.0 to 20.7.0 in /functions/newsletter (#687) Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 20.6.0 to 20.7.0.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=@types/node&package-manager=npm_and_yarn&previous-version=20.6.0&new-version=20.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
--- functions/newsletter/package-lock.json | 6 +++--- functions/newsletter/package.json | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/functions/newsletter/package-lock.json b/functions/newsletter/package-lock.json index ed6f5cda..7964b9d4 100644 --- a/functions/newsletter/package-lock.json +++ b/functions/newsletter/package-lock.json @@ -13,9 +13,9 @@ } }, "@types/node": { - "version": "20.6.0", - "resolved": "https://registry.npmjs.org/@types/node/-/node-20.6.0.tgz", - "integrity": "sha512-najjVq5KN2vsH2U/xyh2opaSEz6cZMR2SetLIlxlj08nOcmPOemJmUK2o4kUzfLqfrWE0PIrNeE16XhYDd3nqg==" + "version": "20.7.0", + "resolved": "https://registry.npmjs.org/@types/node/-/node-20.7.0.tgz", + "integrity": "sha512-zI22/pJW2wUZOVyguFaUL1HABdmSVxpXrzIqkjsHmyUjNhPoWM1CKfvVuXfetHhIok4RY573cqS0mZ1SJEnoTg==" }, "data-uri-to-buffer": { "version": "4.0.0", diff --git a/functions/newsletter/package.json b/functions/newsletter/package.json index ed387082..299e8143 100644 --- a/functions/newsletter/package.json +++ b/functions/newsletter/package.json @@ -15,7 +15,7 @@ "license": "MIT", "dependencies": { "@netlify/functions": "^1.6.0", - "@types/node": "^20.6.0", + "@types/node": "^20.7.0", "node-fetch": "^3.3.0", "typescript": "^5.1.3" } From 2667bed28d7e301be1eee53f80e7180128907d6c Mon Sep 17 00:00:00 2001 From: "T.J. Telan" Date: Sat, 21 Oct 2023 00:30:53 +0000 Subject: [PATCH 13/19] feat: Add updates for `fvm` (#706) fluvio-cms now supports updating all the CLI docs as well --- .github/workflows/update-connector-docs.yml | 13 +- Cargo.lock | 37 ++++- bors.toml | 3 +- content/cli/advanced/cli_channels.md | 73 +++++----- content/cli/cloud/webhook.md | 94 +++++++++++++ content/cli/smartmodules/hub.md | 43 +++--- .../cli/utilities/fluvio-version-manager.md | 32 +++++ content/cli/utilities/install.md | 5 + data/cli-commands.yml | 19 ++- embeds/cli/help/cdk-build.md | 5 + embeds/cli/help/cdk-deploy-list.md | 2 + embeds/cli/help/cdk-deploy-log.md | 11 +- embeds/cli/help/cdk-deploy-shutdown.md | 1 + embeds/cli/help/cdk-deploy-start.md | 13 +- embeds/cli/help/cdk-deploy.md | 6 + embeds/cli/help/cdk-generate.md | 20 ++- embeds/cli/help/cdk-publish.md | 48 ++++++- embeds/cli/help/cdk-set-public.md | 12 +- embeds/cli/help/cdk-test.md | 5 + embeds/cli/help/cdk.md | 3 + .../cli/help/fluvio-cloud-cluster-create.md | 14 +- .../cli/help/fluvio-cloud-cluster-delete.md | 15 ++- embeds/cli/help/fluvio-cloud-cluster-list.md | 10 +- embeds/cli/help/fluvio-cloud-cluster.md | 21 +-- .../cli/help/fluvio-cloud-connector-config.md | 15 ++- .../cli/help/fluvio-cloud-connector-create.md | 7 +- .../cli/help/fluvio-cloud-connector-delete.md | 15 ++- .../cli/help/fluvio-cloud-connector-list.md | 10 +- .../cli/help/fluvio-cloud-connector-logs.md | 15 ++- .../cli/help/fluvio-cloud-connector-update.md | 1 - embeds/cli/help/fluvio-cloud-connector.md | 27 ++-- embeds/cli/help/fluvio-cloud-login.md | 34 +++-- embeds/cli/help/fluvio-cloud-logout.md | 10 +- embeds/cli/help/fluvio-cloud-secret-delete.md | 11 ++ embeds/cli/help/fluvio-cloud-secret-list.md | 8 ++ embeds/cli/help/fluvio-cloud-secret-set.md | 12 ++ embeds/cli/help/fluvio-cloud-usage.md | 10 +- embeds/cli/help/fluvio-cloud.md | 30 +++-- embeds/cli/help/fluvio-cluster-check.md | 15 ++- embeds/cli/help/fluvio-cluster-delete.md | 2 + embeds/cli/help/fluvio-cluster-diagnostics.md | 2 + embeds/cli/help/fluvio-cluster-spg-create.md | 3 + embeds/cli/help/fluvio-cluster-spg-delete.md | 3 + embeds/cli/help/fluvio-cluster-spg-list.md | 2 + embeds/cli/help/fluvio-cluster-spg.md | 8 +- embeds/cli/help/fluvio-cluster-spu-list.md | 2 + .../cli/help/fluvio-cluster-spu-register.md | 2 + .../cli/help/fluvio-cluster-spu-unregister.md | 2 + embeds/cli/help/fluvio-cluster-spu.md | 8 +- embeds/cli/help/fluvio-cluster-start.md | 14 +- embeds/cli/help/fluvio-cluster-upgrade.md | 14 +- embeds/cli/help/fluvio-cluster.md | 3 + embeds/cli/help/fluvio-completions.md | 13 +- embeds/cli/help/fluvio-consume.md | 49 ++++++- .../cli/help/fluvio-derived-stream-create.md | 3 - .../cli/help/fluvio-derived-stream-delete.md | 3 - embeds/cli/help/fluvio-derived-stream-list.md | 3 - embeds/cli/help/fluvio-derived-stream.md | 3 - .../cli/help/fluvio-hub-connector-download.md | 22 ++- embeds/cli/help/fluvio-hub-connector-list.md | 15 ++- embeds/cli/help/fluvio-hub-connector.md | 3 + embeds/cli/help/fluvio-hub-download.md | 18 --- embeds/cli/help/fluvio-hub-list.md | 7 - .../help/fluvio-hub-smartmodule-download.md | 46 +++++++ .../cli/help/fluvio-hub-smartmodule-list.md | 18 +++ embeds/cli/help/fluvio-hub-smartmodule.md | 12 ++ embeds/cli/help/fluvio-hub.md | 8 +- embeds/cli/help/fluvio-install.md | 31 ++++- embeds/cli/help/fluvio-partition-list.md | 2 + embeds/cli/help/fluvio-partition.md | 8 +- embeds/cli/help/fluvio-produce.md | 5 +- embeds/cli/help/fluvio-profile-current.md | 8 +- .../cli/help/fluvio-profile-delete-cluster.md | 3 + embeds/cli/help/fluvio-profile-delete.md | 3 + embeds/cli/help/fluvio-profile-export.md | 3 + embeds/cli/help/fluvio-profile-list.md | 2 + embeds/cli/help/fluvio-profile-rename.md | 3 + embeds/cli/help/fluvio-profile-switch.md | 3 + embeds/cli/help/fluvio-profile-sync.md | 3 + embeds/cli/help/fluvio-profile.md | 9 +- embeds/cli/help/fluvio-run-metadata.md | 8 +- embeds/cli/help/fluvio-run-sc.md | 5 + embeds/cli/help/fluvio-run-spu.md | 4 + embeds/cli/help/fluvio-run-version.md | 8 +- embeds/cli/help/fluvio-run.md | 2 + embeds/cli/help/fluvio-smartmodule-create.md | 3 + embeds/cli/help/fluvio-smartmodule-delete.md | 3 + embeds/cli/help/fluvio-smartmodule-list.md | 2 + embeds/cli/help/fluvio-smartmodule-watch.md | 2 + embeds/cli/help/fluvio-smartmodule.md | 8 +- embeds/cli/help/fluvio-table-format-create.md | 2 + embeds/cli/help/fluvio-table-format-delete.md | 5 +- embeds/cli/help/fluvio-table-format-list.md | 2 + embeds/cli/help/fluvio-table-format.md | 8 +- embeds/cli/help/fluvio-topic-create.md | 62 +++++++-- embeds/cli/help/fluvio-topic-delete.md | 3 + embeds/cli/help/fluvio-topic-describe.md | 3 + embeds/cli/help/fluvio-topic-list.md | 2 + embeds/cli/help/fluvio-topic.md | 8 +- embeds/cli/help/fluvio-update.md | 20 ++- embeds/cli/help/fluvio-version.md | 12 +- embeds/cli/help/fluvio.md | 6 +- embeds/cli/help/fvm-current.md | 8 ++ embeds/cli/help/fvm-install.md | 12 ++ embeds/cli/help/fvm-self-install.md | 8 ++ embeds/cli/help/fvm-self-uninstall.md | 9 ++ embeds/cli/help/fvm-self.md | 13 ++ embeds/cli/help/fvm-show.md | 8 ++ embeds/cli/help/fvm-switch.md | 11 ++ embeds/cli/help/fvm.md | 18 +++ embeds/cli/help/smdk-build.md | 3 + embeds/cli/help/smdk-generate.md | 127 +++++++++++++++--- embeds/cli/help/smdk-load.md | 2 + embeds/cli/help/smdk-publish.md | 33 ++++- embeds/cli/help/smdk-set-public.md | 10 +- embeds/cli/help/smdk-test.md | 14 ++ embeds/cli/help/smdk.md | 3 + embeds/download-cli/curl-bash-copy.md | 9 ++ scripts/fluvio-cms/Cargo.toml | 1 + scripts/fluvio-cms/src/cli/mod.rs | 94 +++++++++++++ scripts/fluvio-cms/src/main.rs | 6 +- 121 files changed, 1325 insertions(+), 338 deletions(-) create mode 100644 content/cli/cloud/webhook.md create mode 100644 content/cli/utilities/fluvio-version-manager.md create mode 100644 embeds/cli/help/fluvio-cloud-secret-delete.md create mode 100644 embeds/cli/help/fluvio-cloud-secret-list.md create mode 100644 embeds/cli/help/fluvio-cloud-secret-set.md delete mode 100644 embeds/cli/help/fluvio-derived-stream-create.md delete mode 100644 embeds/cli/help/fluvio-derived-stream-delete.md delete mode 100644 embeds/cli/help/fluvio-derived-stream-list.md delete mode 100644 embeds/cli/help/fluvio-derived-stream.md delete mode 100644 embeds/cli/help/fluvio-hub-download.md delete mode 100644 embeds/cli/help/fluvio-hub-list.md create mode 100644 embeds/cli/help/fluvio-hub-smartmodule-download.md create mode 100644 embeds/cli/help/fluvio-hub-smartmodule-list.md create mode 100644 embeds/cli/help/fluvio-hub-smartmodule.md create mode 100644 embeds/cli/help/fvm-current.md create mode 100644 embeds/cli/help/fvm-install.md create mode 100644 embeds/cli/help/fvm-self-install.md create mode 100644 embeds/cli/help/fvm-self-uninstall.md create mode 100644 embeds/cli/help/fvm-self.md create mode 100644 embeds/cli/help/fvm-show.md create mode 100644 embeds/cli/help/fvm-switch.md create mode 100644 embeds/cli/help/fvm.md create mode 100644 scripts/fluvio-cms/src/cli/mod.rs diff --git a/.github/workflows/update-connector-docs.yml b/.github/workflows/update-connector-docs.yml index 4a8aab2c..bdcb337c 100644 --- a/.github/workflows/update-connector-docs.yml +++ b/.github/workflows/update-connector-docs.yml @@ -1,10 +1,19 @@ name: Connector docs PR +permissions: + contents: read + concurrency: group: connector-docs-${{ github.ref }} cancel-in-progress: true on: + #push: + # branches: + # - staging + # - trying + #pull_request: + # branches: [master] workflow_call: workflow_dispatch: schedule: @@ -22,14 +31,14 @@ jobs: run: | curl -fsS https://hub.infinyon.cloud/install/install.sh | bash echo "$HOME/.fluvio/bin" >> $GITHUB_PATH - - name: Fluvio Login run: fluvio cloud login --email ${{ secrets.CLOUD_USER_EMAIL }} --password ${{ secrets.CLOUD_USER_PASSWORD }} - uses: Swatinem/rust-cache@v2 # Run fluvio cms + - name: Run connector docs update - run: cargo run -- connector --prod + run: cargo run -- connector # Create/Update PR - name: Create Pull Request diff --git a/Cargo.lock b/Cargo.lock index 05cd3f20..31a16ad4 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -651,7 +651,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "907076dfda823b0b36d2a1bb5f90c96660a5bbcd7729e10727f07858f22c4edc" dependencies = [ "cfg-if", - "hashbrown", + "hashbrown 0.12.3", "lock_api", "once_cell", "parking_lot_core 0.9.7", @@ -795,6 +795,12 @@ dependencies = [ "cfg-if", ] +[[package]] +name = "equivalent" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5443807d6dff69373d433ab9ef5378ad8df50ca6298caf15de6e52e24aaf54d5" + [[package]] name = "errno" version = "0.2.8" @@ -884,6 +890,7 @@ dependencies = [ "lazy_static", "serde", "serde_json", + "serde_yaml", "strum", "tracing", "tracing-subscriber", @@ -1250,6 +1257,12 @@ version = "0.12.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888" +[[package]] +name = "hashbrown" +version = "0.14.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f93e7192158dbcda357bdec5fb5788eebf8bbac027f3f33e719d29135ae84156" + [[package]] name = "heck" version = "0.4.1" @@ -1370,7 +1383,17 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "bd070e393353796e801d209ad339e89596eb4c8d430d18ede6a1cced8fafbd99" dependencies = [ "autocfg", - "hashbrown", + "hashbrown 0.12.3", +] + +[[package]] +name = "indexmap" +version = "2.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8adf3ddd720272c6ea8bf59463c04e0f93d0bbf7c5439b691bca2987e0270897" +dependencies = [ + "equivalent", + "hashbrown 0.14.2", ] [[package]] @@ -2323,11 +2346,11 @@ dependencies = [ [[package]] name = "serde_yaml" -version = "0.9.21" +version = "0.9.25" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d9d684e3ec7de3bf5466b32bd75303ac16f0736426e5a4e0d6e489559ce1249c" +checksum = "1a49e178e4452f45cb61d0cd8cebc1b0fafd3e41929e996cef79aa3aca91f574" dependencies = [ - "indexmap", + "indexmap 2.0.2", "itoa", "ryu", "serde", @@ -2793,7 +2816,7 @@ version = "0.19.10" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2380d56e8670370eee6566b0bfd4265f65b3f432e8c6d85623f728d4fa31f739" dependencies = [ - "indexmap", + "indexmap 1.9.3", "serde", "serde_spanned", "toml_datetime", @@ -3060,7 +3083,7 @@ version = "0.104.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6a396af81a7c56ad976131d6a35e4b693b78a1ea0357843bd436b4577e254a7d" dependencies = [ - "indexmap", + "indexmap 1.9.3", "url", ] diff --git a/bors.toml b/bors.toml index 0665dc98..a2ab7a99 100644 --- a/bors.toml +++ b/bors.toml @@ -1,5 +1,6 @@ status = [ - "Pass client examples", "Pass html lint" + #"Pass client examples", + "Pass html lint" ] use_squash_merge = true delete_merged_branches = true diff --git a/content/cli/advanced/cli_channels.md b/content/cli/advanced/cli_channels.md index 07911f83..075ba87e 100644 --- a/content/cli/advanced/cli_channels.md +++ b/content/cli/advanced/cli_channels.md @@ -4,6 +4,20 @@ menu: Release Channels weight: 10 --- +The current version of release channels was introduced in 0.10.16 with the introduction of [`fvm`]({{}}) + + +{{}} +This is currently in BETA + +%copy first-line% +```shell +$ https://hub-dev.infinyon.cloud/install_fvm/install_fvm.sh | bash +``` +{{}} + + + ## What is this for? Occasionally users report issues in our Discord. After a fix was found and committed, users wanted a way to verify their issue had been resolved before a release. @@ -16,12 +30,20 @@ Inspired by [Rust's concept of channels](https://rust-lang.github.io/rustup/conc ## Installation -Release channel support is provided by the [official installer script]({{}}). The installer uses the `stable` channel by default. +Release channel support is provided by `fvm`. The installer uses the `stable` channel by default. + +%copy first-line% +```shell +$ https://hub-dev.infinyon.cloud/install_fvm/install_fvm.sh | bash +``` + {{}} -Fluvio release channels were introduced for the CLI in `0.9.16`. +Fluvio release channels were introduced for the CLI in `0.9.16`, and the most recent implementation was introduced in `0.10.16`. +

+If you have an installation from a release earlier than `0.10.16`, you should delete `~/.fluvio` directory and re-install with the [official installer script]({{}}) to install `fvm`, the Fluvio Version Manager CLI.

-If you have an installation from a release earlier than `0.9.16`, you should re-install with the [official installer script]({{}}) to install the Fluvio channel frontend. +You can also run `fvm install` to migrate your existing installation {{}} ## Quick start: The Channels @@ -35,7 +57,7 @@ To switch to the `stable` channel: %copy first-line% ```shell -$ fluvio version switch stable +$ fvm switch stable ``` ### Latest @@ -45,7 +67,7 @@ To switch to the `latest` channel: %copy first-line% ```shell -$ fluvio version switch latest +$ fvm switch latest ``` The first time you switch to this channel, the binary will be downloaded. @@ -59,49 +81,14 @@ At this step the binary will get downloaded. %copy first-line% ```shell -$ fluvio version create X.Y.Z +$ fvm install X.Y.Z ``` Then you can switch to the version channel %copy first-line% ```shell -$ fluvio version switch X.Y.Z +$ fvm switch X.Y.Z ``` -Where `X.Y.Z` is the version of a release you want to switch to (e.g. `0.9.18`) - -## How it works? - -Release channel support is set up at install time through the [installer script]({{}}). - -2 binaries are installed -* The Fluvio CLI -* A frontend binary to support switching channels - -By default, the `stable` channel is selected. - -This channel is registered in `~/.fluvio/channel` and is marked as the active channel. - -Example channel config: - -```toml -current_channel = "stable" -[channel.stable] -binary_location = "/home/username/.fluvio/bin/fluvio-stable" -extensions = "/home/username/.fluvio/extensions" -image_tag_strategy = "Version" - -[channel.latest] -binary_location = "/home/username/.fluvio/bin/fluvio-latest" -extensions = "/home/username/.fluvio/extensions-latest" -image_tag_strategy = "VersionGit" -``` - -When you run `fluvio update`, the Fluvio binary of your current channel will update to the newest version of the channel. - -{{}} -Only the `stable` and `latest` channels can be updated. Version channels don't support `fluvio update` because those binaries will always be pinned to its version. -

-Follow the [version channel]({{}}) steps to switch to a different version. -{{}} +Where `X.Y.Z` is the version of a release you want to switch to (e.g. `0.9.18`) \ No newline at end of file diff --git a/content/cli/cloud/webhook.md b/content/cli/cloud/webhook.md new file mode 100644 index 00000000..16c06a8a --- /dev/null +++ b/content/cli/cloud/webhook.md @@ -0,0 +1,94 @@ +--- +title: Cloud Webhook +menu: Webhook +weight: 30 +--- + +The `fluvio cloud webhook` family of commands is used to create, delete, and manage webhook urls in cloud. + +%copy first-line% +```bash +$ fluvio cloud webhook -h +``` + +{{% inline-embed file="embeds/cli/help/fluvio-cloud-webhook.md" %}} + +--- + +## `fluvio cloud webhook create` + +This command is used to create a new webhook url that maps to the current user's topic + +%copy first-line% +```bash +$ fluvio cloud webhook create -h +``` + +{{% inline-embed file="embeds/cli/help/fluvio-cloud-webhook-create.md" %}} + +Example usage: + +%copy first-line% +```bash +$ fluvio cloud webhook create my-webhook-1 --topic my-topic +``` + +--- + +## `fluvio cloud webhook delete` + +This command deletes a webhook url for the current user + +%copy first-line% +```bash +$ fluvio cloud webhook delete -h +``` + +{{% inline-embed file="embeds/cli/help/fluvio-cloud-webhook-delete.md" %}} + +Example usage: + +%copy first-line% +```bash +$ fluvio cloud webhook delete my-webhook-1 +``` + +--- + +## `fluvio cloud webhook list` + +Command to show the webhooks associated with current user. + +%copy first-line% +```bash +$ fluvio cloud webhook list -h +``` + +{{% inline-embed file="embeds/cli/help/fluvio-cloud-webhook-list.md" %}} + +Example usage: + +%copy first-line% +```bash +$ fluvio cloud webhook list +``` + +--- + +## `fluvio cloud webhook update` + +Command to show update mapping details of a webhook url + +%copy first-line% +```bash +$ fluvio cloud webhook update -h +``` + +{{% inline-embed file="embeds/cli/help/fluvio-cloud-webhook-update.md" %}} + +Example usage: + +%copy first-line% +```bash +$ fluvio cloud webhook update --topic different-topic my-webhook +``` \ No newline at end of file diff --git a/content/cli/smartmodules/hub.md b/content/cli/smartmodules/hub.md index 342dd7d9..b495bd15 100644 --- a/content/cli/smartmodules/hub.md +++ b/content/cli/smartmodules/hub.md @@ -6,32 +6,32 @@ weight: 20 ## Introduction to the SmartModule Hub Service -The SmartModule Hub allows users to start building their data streaming pipeline with SmartModules which have been uploaded into the Hub as ready to use packages. The Hub allows Fluvio pipeline construction focusing on the business logic by providing already built, reliable and tested modules. The modules contain high level configurable inputs which can be applied without needing to rebuild the module. Pipelines can employ a mix of Hub downloaded packages as well as custom developed SmartModules. +The SmartModule Hub offers ready-to-use packages for easy Fluvio pipeline construction, letting users focus on business logic. It provides tested modules with high-level configurable inputs, supporting a mix of Hub packages and custom SmartModules without requiring module rebuilds. -A view of available and installed SmartModules is accessible from both the `fluvio` cli as well as on the Fluvio cloud dashboard. +A view of available and installed SmartModules is accessible from both the `fluvio` cli. -## Fluvio Hub Commands +## Fluvio Hub Smartmodule Commands -Commands for users to interact with the SmartModule hub are under the **fluvio hub** set of commands. +Commands for users to interact with the SmartModule hub are under the **fluvio hub smartmodule** set of commands. -- `fluvio hub download group/package@version` -- `fluvio hub list` +- `fluvio hub smartmodule download group/package@version` +- `fluvio hub smartmodule list` -### `fluvio hub download` +### `fluvio smartmodule hub download` -{{% inline-embed file="embeds/cli/help/fluvio-hub-download.md" %}} +{{% inline-embed file="embeds/cli/help/fluvio-hub-smartmodule-download.md" %}} -The command **fluvio hub download ** will download a hub smartmodule into your cluster so it can be configured and used in your pipeline. +The command **fluvio hub smartmodule download ** will download a hub smartmodule into your cluster so it can be configured and used in your pipeline. %copy first-line% ```bash -$ fluvio hub download infinyon/egex-filter2@0.1.0 +$ fluvio hub smartmodule download infinyon/regex-filter2@0.1.0 downloading infinyon/regex-filter2@0.1.0 to infinyon-regex-filter2-0.1.0.ipkg ... downloading complete ... checking package -trying connection to fluvio router.dev.infinyon.cloud:9003 +trying connection to fluvio router.infinyon.cloud:9003 ... cluster smartmodule install complete ``` @@ -45,28 +45,27 @@ $ flvio sm list infinyon/regex-filter@0.1.0 316.1 KB ``` -Once a SmartModule is download, it can be configured and applied in a pipleine. See the SmartModule page for more details on how work with SmartModules once they are downloaded in your cluster. +Once a SmartModule is download, it can be configured and applied in a pipeline. See the SmartModule page for more details on how work with SmartModules once they are downloaded in your cluster. -### `fluvio hub list` +### `fluvio hub smartmodule list` -The command **fluvio hub list** will list SmartModules and versions available for download +The command **fluvio hub smartmodule list** will list SmartModules and versions available for download -{{% inline-embed file="embeds/cli/help/fluvio-hub-list.md" %}} +{{% inline-embed file="embeds/cli/help/fluvio-hub-smartmodule-list.md" %}} The list command shows Smartmodules by **group/package@version** in the SMARTMODULE column. The Visibility column describes if the package is accessible to public or private views. If package visibility is **private** only your user login can list or download the package. Packages uploaded by InfinyOn will be in the infinyon group. %copy first-line% ```bash -$ fluvio hub list +$ fluvio hub smartmodule list SMARTMODULE Visibility mypriv/foo-priv@0.1.1 private infinyon/jolt@0.1.0 public infinyon/json-sql@0.1.0 public infinyon/regex-filter@0.1.0 public -... ``` @@ -75,19 +74,15 @@ $ fluvio hub list SmartModule Packages are listed in the hub as in the form of `/@version`. -Group `infinyon` packages are published by InfinyOn, the maker of Fluvio. Third-party developers may also upload public packages which appear in other groups. The Hub service enforces signing and publishing restrictions on uploaded packages. Once a signed package of given group has been published, only the group owner may update that package or add new packages within that group. +Packages in the infinyon group are published by InfinyOn, the creators of Fluvio. Third-party developers can upload public packages under different groups. The Hub enforces signing and publishing restrictions on uploaded packages. Once a signed package is published in a group, only the group owner can update it or add new packages within that group. ### Private and Public Packages -Published packages are by default private. In order to make a published package publicly available, the package should be set to public. The cli method to do this is below: +By default, published packages are private. To make a package public, set it to public using the following CLI commands: ```bash cd smartmodule-dev-dir smdk set-public ``` -After the package is set to public, any publish will ask to verify the publish operation. Once a package version is publicly published, the package cannot revoked. - - - - +Once set to public, you'll be prompted for verification on every publish. Note that once a package version is publicly published, it cannot be revoked. \ No newline at end of file diff --git a/content/cli/utilities/fluvio-version-manager.md b/content/cli/utilities/fluvio-version-manager.md new file mode 100644 index 00000000..2581d39c --- /dev/null +++ b/content/cli/utilities/fluvio-version-manager.md @@ -0,0 +1,32 @@ +--- +title: Fluvio Version Manager +menu: FVM +weight: 05 +toc: false +--- + +`fvm` manages multiple installations of the fluvio toolchain. Users select a [release channel]({{}}) of fluvio/cdk/smdk, and `fvm` takes care of the rest. + +## `fvm` +{{% inline-embed file="embeds/cli/help/fvm.md" %}} + +## `fvm current` +{{% inline-embed file="embeds/cli/help/fvm-current.md" %}} + +## `fvm self` +{{% inline-embed file="embeds/cli/help/fvm-self.md" %}} + +## `fvm self install` +{{% inline-embed file="embeds/cli/help/fvm-self-install.md" %}} + +## `fvm self uninstall` +{{% inline-embed file="embeds/cli/help/fvm-self-uninstall.md" %}} + +## `fvm install` +{{% inline-embed file="embeds/cli/help/fvm-install.md" %}} + +## `fvm show` +{{% inline-embed file="embeds/cli/help/fvm-show.md" %}} + +## `fvm switch` +{{% inline-embed file="embeds/cli/help/fvm-switch.md" %}} diff --git a/content/cli/utilities/install.md b/content/cli/utilities/install.md index d84cc105..f9736d91 100644 --- a/content/cli/utilities/install.md +++ b/content/cli/utilities/install.md @@ -1,9 +1,14 @@ --- title: Install +menu: Install (deprecated) weight: 10 toc: false --- +{{}} +This command is deprecated since 0.10.16, and has been replaced by [`fvm install`]({{}}) +{{}} + Some Fluvio CLI commands are distributed as separate executables that we call "extensions". This command installs and updates extensions by name, placing them in the `~/.fluvio/extensions/` directory. diff --git a/data/cli-commands.yml b/data/cli-commands.yml index dfa4ffd1..9d6a923c 100644 --- a/data/cli-commands.yml +++ b/data/cli-commands.yml @@ -25,14 +25,10 @@ cli-commands: - fluvio table-format list - fluvio table-format delete - - fluvio derived-stream - - fluvio derived-stream create - - fluvio derived-stream list - - fluvio derived-stream delete - - fluvio hub - - fluvio hub download - - fluvio hub list + - fluvio hub smartmodule + - fluvio hub smartmodule list + - fluvio hub smartmodule download - fluvio hub connector - fluvio hub connector list - fluvio hub connector download @@ -111,3 +107,12 @@ cli-commands: - cdk deploy log - cdk publish - cdk set-public + + - fvm + - fvm current + - fvm self + - fvm self install + - fvm self uninstall + - fvm install + - fvm show + - fvm switch \ No newline at end of file diff --git a/embeds/cli/help/cdk-build.md b/embeds/cli/help/cdk-build.md index c44423ca..106cdfe2 100644 --- a/embeds/cli/help/cdk-build.md +++ b/embeds/cli/help/cdk-build.md @@ -1,10 +1,15 @@ ``` Build the Connector in the current working directory + Usage: cdk build [OPTIONS] [-- ...] + Arguments: [EXTRA_ARGUMENTS]... Extra arguments to be passed to cargo + Options: --release Release profile name [default: release] + --target Provide target platform for the package. Optional. By default + the host's one is used [default: aarch64-apple-darwin] -p, --package-name Optional package/project name -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-deploy-list.md b/embeds/cli/help/cdk-deploy-list.md index 34037e08..716f9c2b 100644 --- a/embeds/cli/help/cdk-deploy-list.md +++ b/embeds/cli/help/cdk-deploy-list.md @@ -1,6 +1,8 @@ ``` Print the list of all deployed connectors + Usage: cdk deploy list + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-deploy-log.md b/embeds/cli/help/cdk-deploy-log.md index 4633a019..bf8bb0f2 100644 --- a/embeds/cli/help/cdk-deploy-log.md +++ b/embeds/cli/help/cdk-deploy-log.md @@ -1,5 +1,10 @@ ``` -Print the connector's logs -Usage: cdk deploy log Options: -Arguments: -h, --help Print help +Print the connector's logs + +Usage: cdk deploy log <--config |--name > + +Options: + -c, --config Path to configuration file in YAML format + -n, --name Name of the running connector + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-deploy-shutdown.md b/embeds/cli/help/cdk-deploy-shutdown.md index 786e6044..693380ea 100644 --- a/embeds/cli/help/cdk-deploy-shutdown.md +++ b/embeds/cli/help/cdk-deploy-shutdown.md @@ -1,5 +1,6 @@ ``` Shutdown the Connector's deployment + Usage: cdk deploy shutdown <--config |--name > Options: diff --git a/embeds/cli/help/cdk-deploy-start.md b/embeds/cli/help/cdk-deploy-start.md index 5356e0f1..1fe6c57c 100644 --- a/embeds/cli/help/cdk-deploy-start.md +++ b/embeds/cli/help/cdk-deploy-start.md @@ -1,10 +1,13 @@ ``` Start new deployment for the given connector config + Usage: cdk deploy start [OPTIONS] --config + Options: - -c, --config Path to configuration file in YAML format - -s, --secrets Path to file with secrets. Secrets are 'key=value' pairs separated by the - new line character. Optional - --ipkg Deploy from local package file - -h, --help Print help + -c, --config Path to configuration file in YAML format + -s, --secrets Path to file with secrets. Secrets are 'key=value' pairs separated by + the new line character. Optional + --ipkg Deploy from local package file + --log-level Log level for the connector process [default: ] + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-deploy.md b/embeds/cli/help/cdk-deploy.md index eaa6caa3..1a786210 100644 --- a/embeds/cli/help/cdk-deploy.md +++ b/embeds/cli/help/cdk-deploy.md @@ -1,16 +1,22 @@ ``` Deploy the Connector from the current working directory + Usage: cdk deploy [OPTIONS] [-- ...] + Commands: start Start new deployment for the given connector config shutdown Shutdown the Connector's deployment list Print the list of all deployed connectors log Print the connector's logs help Print this message or the help of the given subcommand(s) + Arguments: [EXTRA_ARGUMENTS]... Extra arguments to be passed to cargo + Options: --release Release profile name [default: release] + --target Provide target platform for the package. Optional. By default + the host's one is used [default: aarch64-apple-darwin] -p, --package-name Optional package/project name -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-generate.md b/embeds/cli/help/cdk-generate.md index caa84c78..2c38b115 100644 --- a/embeds/cli/help/cdk-generate.md +++ b/embeds/cli/help/cdk-generate.md @@ -1,10 +1,22 @@ ``` Generate new SmartConnector project + Usage: cdk generate [OPTIONS] [NAME] + Arguments: - [NAME] SmartConnector Project Name + [NAME] + SmartConnector Project Name + Options: - --destination Local path to generate the SmartConnector project. Default to directory - with project name, created in current directory [env: CDK_DESTINATION=] - -h, --help Print help (see more with '--help') + --destination + Local path to generate the SmartConnector project. Default to directory with project name, + created in current directory + + [env: CDK_DESTINATION=] + + --silent + Disable interactive prompt. Take all values from CLI flags. Fail if a value is missing + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-publish.md b/embeds/cli/help/cdk-publish.md index 3f413393..967ad775 100644 --- a/embeds/cli/help/cdk-publish.md +++ b/embeds/cli/help/cdk-publish.md @@ -1,11 +1,49 @@ ``` Publish Connector package to the Hub + Usage: cdk publish [OPTIONS] [PACKAGE_META] + Arguments: - [PACKAGE_META] + [PACKAGE_META] + + Options: - --public-yes don't ask for confirmation of public package publish - --target provide target platform for the package. Optional. By default the host's - one is used [default: aarch64-apple-darwin] - -h, --help Print help (see more with '--help') + --release + Release profile name + + [default: release] + + --target + Provide target platform for the package. Optional. By default the host's one is used + + [default: aarch64-apple-darwin] + + -p, --package-name + Optional package/project name + + --ipkg + path to the ipkg file, used when --push is specified + + --public-yes + don't ask for confirmation of public package publish + + --no-build + + + --pack + do only the pack portion + + --push + given a packed file do only the push + + --remote + + + --readme + Relative path to this connector package README + + [default: ./README.md] + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-set-public.md b/embeds/cli/help/cdk-set-public.md index 252d93f3..adfb5837 100644 --- a/embeds/cli/help/cdk-set-public.md +++ b/embeds/cli/help/cdk-set-public.md @@ -1,4 +1,12 @@ ``` -Set connector visibility to public Options: -Usage: cdk set-public -h, --help Print help +Set connector visibility to public + +Usage: cdk set-public [OPTIONS] + +Options: + --release Release profile name [default: release] + --target Provide target platform for the package. Optional. By default + the host's one is used [default: aarch64-apple-darwin] + -p, --package-name Optional package/project name + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/cdk-test.md b/embeds/cli/help/cdk-test.md index 4bedd451..852c0c1f 100644 --- a/embeds/cli/help/cdk-test.md +++ b/embeds/cli/help/cdk-test.md @@ -1,10 +1,15 @@ ``` Build and run the Connector in the current working directory + Usage: cdk test [OPTIONS] [-- ...] + Arguments: [EXTRA_ARGUMENTS]... Extra arguments to be passed to cargo + Options: --release Release profile name [default: release] + --target Provide target platform for the package. Optional. By default + the host's one is used [default: aarch64-apple-darwin] -p, --package-name Optional package/project name -c, --config Path to configuration file in YAML format [default: sample-config.yaml] diff --git a/embeds/cli/help/cdk.md b/embeds/cli/help/cdk.md index 54bfe97f..20ad34b1 100644 --- a/embeds/cli/help/cdk.md +++ b/embeds/cli/help/cdk.md @@ -1,6 +1,8 @@ ``` Connector Development Kit + Usage: cdk + Commands: build Build the Connector in the current working directory test Build and run the Connector in the current working directory @@ -9,6 +11,7 @@ Commands: publish Publish Connector package to the Hub set-public Set connector visibility to public help Print this message or the help of the given subcommand(s) + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-cluster-create.md b/embeds/cli/help/fluvio-cloud-cluster-create.md index 66eb5d4f..848befd5 100644 --- a/embeds/cli/help/fluvio-cloud-cluster-create.md +++ b/embeds/cli/help/fluvio-cloud-cluster-create.md @@ -1,9 +1,11 @@ ``` -fluvio-cloud-cluster-create Create a new Fluvio cluster -USAGE: - fluvio-cloud cluster create [OPTIONS] -OPTIONS: - -h, --help Print help information - --profile The name of the Profile to save + +Usage: fluvio-cloud cluster create [OPTIONS] + +Options: + --profile The name of the Profile to save + --region + --version + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-cluster-delete.md b/embeds/cli/help/fluvio-cloud-cluster-delete.md index e1704be8..0dd8c934 100644 --- a/embeds/cli/help/fluvio-cloud-cluster-delete.md +++ b/embeds/cli/help/fluvio-cloud-cluster-delete.md @@ -1,10 +1,11 @@ ``` -fluvio-cloud-cluster-delete Delete an existing Fluvio cluster -USAGE: - fluvio-cloud cluster delete -ARGS: - The email for the cluster to be deleted -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud cluster delete + +Arguments: + The email for the cluster to be deleted + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-cluster-list.md b/embeds/cli/help/fluvio-cloud-cluster-list.md index 4951aff6..a194ca5e 100644 --- a/embeds/cli/help/fluvio-cloud-cluster-list.md +++ b/embeds/cli/help/fluvio-cloud-cluster-list.md @@ -1,8 +1,8 @@ ``` -fluvio-cloud-cluster-list List all Fluvio clusters -USAGE: - fluvio-cloud cluster list -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud cluster list + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-cluster.md b/embeds/cli/help/fluvio-cloud-cluster.md index cf0bb150..298bec42 100644 --- a/embeds/cli/help/fluvio-cloud-cluster.md +++ b/embeds/cli/help/fluvio-cloud-cluster.md @@ -1,13 +1,14 @@ ``` -fluvio-cloud-cluster View Cluster information -USAGE: - fluvio-cloud cluster -OPTIONS: - -h, --help Print help information -SUBCOMMANDS: - create Create a new Fluvio cluster - delete Delete an existing Fluvio cluster - help Print this message or the help of the given subcommand(s) - list List all Fluvio clusters + +Usage: fluvio-cloud cluster + +Commands: + create Create a new Fluvio cluster + delete Delete an existing Fluvio cluster + list List all Fluvio clusters + help Print this message or the help of the given subcommand(s) + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-config.md b/embeds/cli/help/fluvio-cloud-connector-config.md index 3a858a36..390703d2 100644 --- a/embeds/cli/help/fluvio-cloud-connector-config.md +++ b/embeds/cli/help/fluvio-cloud-connector-config.md @@ -1,10 +1,11 @@ ``` -fluvio-cloud-connector-config Show the connector configuration details -USAGE: - fluvio-cloud connector config [OPTIONS] -ARGS: - Name of connector -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud connector config + +Arguments: + Name of connector + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-create.md b/embeds/cli/help/fluvio-cloud-connector-create.md index e7955491..9b5dd5a5 100644 --- a/embeds/cli/help/fluvio-cloud-connector-create.md +++ b/embeds/cli/help/fluvio-cloud-connector-create.md @@ -1,11 +1,10 @@ ``` -fluvio cloud connector create Create a new Connector -Usage: fluvio cloud connector create [OPTIONS] --config +Usage: fluvio-cloud connector create [OPTIONS] --config Options: - -c, --config Name of connector - --log-level Sets the log level [default: info] + -c, --config Connector config + --log-level Sets the log level [default: LogLevel::default()] -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-delete.md b/embeds/cli/help/fluvio-cloud-connector-delete.md index fe39e84b..8c6daeed 100644 --- a/embeds/cli/help/fluvio-cloud-connector-delete.md +++ b/embeds/cli/help/fluvio-cloud-connector-delete.md @@ -1,10 +1,11 @@ ``` -fluvio-cloud-connector-delete Delete a Connector -USAGE: - fluvio-cloud connector delete [OPTIONS] ... -ARGS: - ... One or more name(s) of the connector(s) to be deleted -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud connector delete ... + +Arguments: + ... One or more name(s) of the connector(s) to be deleted + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-list.md b/embeds/cli/help/fluvio-cloud-connector-list.md index 68ee9ce7..da3f8083 100644 --- a/embeds/cli/help/fluvio-cloud-connector-list.md +++ b/embeds/cli/help/fluvio-cloud-connector-list.md @@ -1,8 +1,8 @@ ``` -fluvio-cloud-connector-list List all Connectors -USAGE: - fluvio-cloud connector list [OPTIONS] -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud connector list + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-logs.md b/embeds/cli/help/fluvio-cloud-connector-logs.md index 81b40f19..305be424 100644 --- a/embeds/cli/help/fluvio-cloud-connector-logs.md +++ b/embeds/cli/help/fluvio-cloud-connector-logs.md @@ -1,10 +1,11 @@ ``` -fluvio-cloud-connector-logs View connector logs -USAGE: - fluvio-cloud connector logs [OPTIONS] -ARGS: - Name of connector -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud connector logs + +Arguments: + Name of connector + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-connector-update.md b/embeds/cli/help/fluvio-cloud-connector-update.md index 346d0782..1b876068 100644 --- a/embeds/cli/help/fluvio-cloud-connector-update.md +++ b/embeds/cli/help/fluvio-cloud-connector-update.md @@ -1,5 +1,4 @@ ``` -fluvio cloud connector update Update and restart a Connector Usage: fluvio-cloud connector update [OPTIONS] --config diff --git a/embeds/cli/help/fluvio-cloud-connector.md b/embeds/cli/help/fluvio-cloud-connector.md index 2bf7e3a7..b012ed77 100644 --- a/embeds/cli/help/fluvio-cloud-connector.md +++ b/embeds/cli/help/fluvio-cloud-connector.md @@ -1,16 +1,17 @@ ``` -fluvio-cloud-connector View Fluvio Connector information -USAGE: - fluvio-cloud connector -OPTIONS: - -h, --help Print help information -SUBCOMMANDS: - config Show the connector configuration details - create Create a new Connector - delete Delete a Connector - help Print this message or the help of the given subcommand(s) - list List all Connectors - logs View connector logs - update Update and restart a Connector + +Usage: fluvio-cloud connector + +Commands: + logs View connector logs + create Create a new Connector + delete Delete a Connector + list List all Connectors + update Update and restart a Connector + config Show the connector configuration details + help Print this message or the help of the given subcommand(s) + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-login.md b/embeds/cli/help/fluvio-cloud-login.md index 7edf1b06..fc82c1ac 100644 --- a/embeds/cli/help/fluvio-cloud-login.md +++ b/embeds/cli/help/fluvio-cloud-login.md @@ -1,14 +1,26 @@ ``` -fluvio-cloud-login Log into Infinyon Cloud with Oauth2 or username/password -USAGE: - fluvio-cloud login [OPTIONS] -OPTIONS: - --email Infinyon Cloud email to use for logging in - -h, --help Print help information - --password Password to use when logging in (not recommended) - --profile The name of the Profile to save - --use-oauth2 Authenticate using OAuth 2.0 Device Code Flow. CLI will try to - launch a web browser to log in interactively. If a web browser is - not available, CLI will print URL for device code login + +Usage: fluvio-cloud login [OPTIONS] + +Options: + --email + Infinyon Cloud email to use for logging in + + --password + Password to use when logging in (not recommended). + + If not provided, fluvio-cloud will prompt on stdin unless using third party + authentication. + + --profile + The name of the Profile to save + + --use-oauth2 + Authenticate using OAuth 2.0 Device Code Flow. CLI will try to launch a web browser to log + in interactively. If a web browser is not available, CLI will print URL for device code + login + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-logout.md b/embeds/cli/help/fluvio-cloud-logout.md index e8841597..a33803aa 100644 --- a/embeds/cli/help/fluvio-cloud-logout.md +++ b/embeds/cli/help/fluvio-cloud-logout.md @@ -1,8 +1,8 @@ ``` -fluvio-cloud-logout Log out of a Infinyon Cloud account -USAGE: - fluvio-cloud logout -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud logout + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-secret-delete.md b/embeds/cli/help/fluvio-cloud-secret-delete.md new file mode 100644 index 00000000..a90bd43d --- /dev/null +++ b/embeds/cli/help/fluvio-cloud-secret-delete.md @@ -0,0 +1,11 @@ +``` +delete a cloud secret + +Usage: fluvio-cloud secret delete + +Arguments: + Name of secret to delete + +Options: + -h, --help Print help +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-secret-list.md b/embeds/cli/help/fluvio-cloud-secret-list.md new file mode 100644 index 00000000..cf405b35 --- /dev/null +++ b/embeds/cli/help/fluvio-cloud-secret-list.md @@ -0,0 +1,8 @@ +``` +list cloud secret names + +Usage: fluvio-cloud secret list + +Options: + -h, --help Print help +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-secret-set.md b/embeds/cli/help/fluvio-cloud-secret-set.md new file mode 100644 index 00000000..feb26206 --- /dev/null +++ b/embeds/cli/help/fluvio-cloud-secret-set.md @@ -0,0 +1,12 @@ +``` +set a cloud secret by name and value + +Usage: fluvio-cloud secret set + +Arguments: + Name of secret to set + Value of secret to associate with name + +Options: + -h, --help Print help +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud-usage.md b/embeds/cli/help/fluvio-cloud-usage.md index a236af08..ea6728a9 100644 --- a/embeds/cli/help/fluvio-cloud-usage.md +++ b/embeds/cli/help/fluvio-cloud-usage.md @@ -1,8 +1,8 @@ ``` -fluvio-cloud-usage Print cluster stats -USAGE: - fluvio-cloud usage -OPTIONS: - -h, --help Print help information + +Usage: fluvio-cloud usage + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cloud.md b/embeds/cli/help/fluvio-cloud.md index 5cda4b2b..abf179fa 100644 --- a/embeds/cli/help/fluvio-cloud.md +++ b/embeds/cli/help/fluvio-cloud.md @@ -1,15 +1,19 @@ ``` -fluvio-cloud -USAGE: - fluvio-cloud -OPTIONS: - -h, --help Print help information -SUBCOMMANDS: - cluster View Cluster information - connector View Fluvio Connector information - help Print this message or the help of the given subcommand(s) - login Log into Infinyon Cloud with Oauth2 or username/password - logout Log out of a Infinyon Cloud account - usage Print cluster stats - version Print the current version of fluvio-cloud +Usage: fluvio-cloud + +Commands: + login Log into Infinyon Cloud with Oauth2 or username/password + logout Log out of a Infinyon Cloud account + version Print the current version of fluvio-cloud + usage Print cluster stats + connector View Fluvio Connector information + cluster View Cluster information + secret Manage cloud secrets + webhook Manage Webhooks + metadata Generate metadata for Fluvio base CLI + edge Edge + help Print this message or the help of the given subcommand(s) + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-check.md b/embeds/cli/help/fluvio-cluster-check.md index b834e0dd..96bf8a63 100644 --- a/embeds/cli/help/fluvio-cluster-check.md +++ b/embeds/cli/help/fluvio-cluster-check.md @@ -1,7 +1,16 @@ ``` -Check that all requirements for cluster startup are met +Check that all requirements for cluster startup are met. + +This command is useful to check if user has all the required dependencies and permissions to run fluvio on the current Kubernetes context. + +It is not intended to be used in scenarios where user does not have access to Kubernetes resources (eg. Cloud) + Usage: fluvio cluster check [OPTIONS] + Options: - --fix Attempt to fix recoverable errors - -h, --help Print help (see more with '--help') + --fix + Attempt to fix recoverable errors + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-delete.md b/embeds/cli/help/fluvio-cluster-delete.md index 60c9495a..a6ddce1c 100644 --- a/embeds/cli/help/fluvio-cluster-delete.md +++ b/embeds/cli/help/fluvio-cluster-delete.md @@ -1,6 +1,8 @@ ``` Uninstall a Fluvio cluster + Usage: fluvio cluster delete [OPTIONS] + Options: --namespace --local Remove only local spu/sc(custom) fluvio installation diff --git a/embeds/cli/help/fluvio-cluster-diagnostics.md b/embeds/cli/help/fluvio-cluster-diagnostics.md index b7ecdd3b..02a1c054 100644 --- a/embeds/cli/help/fluvio-cluster-diagnostics.md +++ b/embeds/cli/help/fluvio-cluster-diagnostics.md @@ -1,6 +1,8 @@ ``` Collect anonymous diagnostic information to help with debugging + Usage: fluvio cluster diagnostics [OPTIONS] + Options: --quiet --k8 diff --git a/embeds/cli/help/fluvio-cluster-spg-create.md b/embeds/cli/help/fluvio-cluster-spg-create.md index f6540aaf..29c813f4 100644 --- a/embeds/cli/help/fluvio-cluster-spg-create.md +++ b/embeds/cli/help/fluvio-cluster-spg-create.md @@ -1,8 +1,11 @@ ``` Create a new managed SPU Group + fluvio cluster spg create [OPTIONS] + Arguments: The name for the new SPU Group + Options: -r, --replicas The number of SPUs to create in this SPG [default: 1] --min-id Minimum SPU ID [default: 1] diff --git a/embeds/cli/help/fluvio-cluster-spg-delete.md b/embeds/cli/help/fluvio-cluster-spg-delete.md index d0aae1f0..04728ad0 100644 --- a/embeds/cli/help/fluvio-cluster-spg-delete.md +++ b/embeds/cli/help/fluvio-cluster-spg-delete.md @@ -1,8 +1,11 @@ ``` Delete a managed SPU Group + fluvio cluster spg delete + Arguments: The name of the SPU Group to delete + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-spg-list.md b/embeds/cli/help/fluvio-cluster-spg-list.md index 759d1e21..1f4f4650 100644 --- a/embeds/cli/help/fluvio-cluster-spg-list.md +++ b/embeds/cli/help/fluvio-cluster-spg-list.md @@ -1,6 +1,8 @@ ``` List all SPU Groups + fluvio cluster spg list [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-cluster-spg.md b/embeds/cli/help/fluvio-cluster-spg.md index 04c2e469..a093bdaa 100644 --- a/embeds/cli/help/fluvio-cluster-spg.md +++ b/embeds/cli/help/fluvio-cluster-spg.md @@ -1,10 +1,16 @@ ``` Manage and view SPU Groups (SPGs) + +SPGs are groups of SPUs in a cluster which are managed together. + Usage: fluvio cluster spg + Commands: create Create a new managed SPU Group delete Delete a managed SPU Group list List all SPU Groups + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-spu-list.md b/embeds/cli/help/fluvio-cluster-spu-list.md index 6662aca4..3ff45935 100644 --- a/embeds/cli/help/fluvio-cluster-spu-list.md +++ b/embeds/cli/help/fluvio-cluster-spu-list.md @@ -1,6 +1,8 @@ ``` List all SPUs known by this cluster (managed AND custom) + fluvio cluster spu list [OPTIONS] + Options: --custom Whether to list only custom SPUs -O, --output Output [default: table] [possible values: table, yaml, json] diff --git a/embeds/cli/help/fluvio-cluster-spu-register.md b/embeds/cli/help/fluvio-cluster-spu-register.md index 5036c204..2a3ad7b2 100644 --- a/embeds/cli/help/fluvio-cluster-spu-register.md +++ b/embeds/cli/help/fluvio-cluster-spu-register.md @@ -1,6 +1,8 @@ ``` Register a new custom SPU with the cluster + fluvio cluster spu register [OPTIONS] --id --public-server --private-server + Options: -i, --id SPU id -n, --name SPU name diff --git a/embeds/cli/help/fluvio-cluster-spu-unregister.md b/embeds/cli/help/fluvio-cluster-spu-unregister.md index 925c2260..84a206ff 100644 --- a/embeds/cli/help/fluvio-cluster-spu-unregister.md +++ b/embeds/cli/help/fluvio-cluster-spu-unregister.md @@ -1,6 +1,8 @@ ``` Unregister a custom SPU from the cluster + fluvio cluster spu unregister [OPTIONS] + Options: -i, --id SPU id -n, --name SPU name diff --git a/embeds/cli/help/fluvio-cluster-spu.md b/embeds/cli/help/fluvio-cluster-spu.md index de5e3b27..20b6220c 100644 --- a/embeds/cli/help/fluvio-cluster-spu.md +++ b/embeds/cli/help/fluvio-cluster-spu.md @@ -1,10 +1,16 @@ ``` Manage and view Streaming Processing Units (SPUs) + +SPUs make up the part of a Fluvio cluster which is in charge of receiving messages from producers, storing those messages, and relaying them to consumers. This command lets you see the status of SPUs in your cluster. + Usage: fluvio cluster spu + Commands: register Register a new custom SPU with the cluster unregister Unregister a custom SPU from the cluster list List all SPUs known by this cluster (managed AND custom) + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-start.md b/embeds/cli/help/fluvio-cluster-start.md index c9ef8bb9..92feea0d 100644 --- a/embeds/cli/help/fluvio-cluster-start.md +++ b/embeds/cli/help/fluvio-cluster-start.md @@ -1,6 +1,8 @@ ``` Install Fluvio cluster + Usage: fluvio cluster start [OPTIONS] + Options: --develop use local image @@ -11,7 +13,7 @@ Options: --registry k8: use custom docker registry --namespace - k8 [default: default] + k8 namespace [default: default] --group-name k8 [default: main] --install-name @@ -22,6 +24,8 @@ Options: chart values --use-k8-port-forwarding Uses port forwarding for connecting to SC during install + --use-cluster-ip + Uses port forwarding for connecting to SC during install --tls-client-secret-name TLS: Client secret name while adding to Kubernetes [default: fluvio-client-tls] --tls-server-secret-name @@ -29,12 +33,13 @@ Options: --spu-storage-size set spu storage size [default: 10] --skip-profile-creation + --spu number of SPU [default: 1] --rust-log RUST_LOG options --log-dir - log dir [default: /tmp] + log dir [default: /usr/local/var/log/fluvio] --sys-only installing/upgrade sys only --local @@ -54,6 +59,7 @@ Options: --server-key TLS: path to server private key --authorization-config-map + --skip-checks Whether to skip pre-install checks, defaults to false --setup @@ -62,8 +68,8 @@ Options: Proxy address --service-type Service Type - --connector-prefix - Connector Prefix + --read-only + Start SC in read only mode -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster-upgrade.md b/embeds/cli/help/fluvio-cluster-upgrade.md index 4efe1d1e..1dce03f7 100644 --- a/embeds/cli/help/fluvio-cluster-upgrade.md +++ b/embeds/cli/help/fluvio-cluster-upgrade.md @@ -1,6 +1,8 @@ ``` Upgrades an already-started Fluvio cluster + Usage: fluvio cluster upgrade [OPTIONS] + Options: --develop use local image @@ -11,7 +13,7 @@ Options: --registry k8: use custom docker registry --namespace - k8 [default: default] + k8 namespace [default: default] --group-name k8 [default: main] --install-name @@ -22,6 +24,8 @@ Options: chart values --use-k8-port-forwarding Uses port forwarding for connecting to SC during install + --use-cluster-ip + Uses port forwarding for connecting to SC during install --tls-client-secret-name TLS: Client secret name while adding to Kubernetes [default: fluvio-client-tls] --tls-server-secret-name @@ -29,12 +33,13 @@ Options: --spu-storage-size set spu storage size [default: 10] --skip-profile-creation + --spu number of SPU [default: 1] --rust-log RUST_LOG options --log-dir - log dir [default: /tmp] + log dir [default: /usr/local/var/log/fluvio] --sys-only installing/upgrade sys only --local @@ -54,6 +59,7 @@ Options: --server-key TLS: path to server private key --authorization-config-map + --skip-checks Whether to skip pre-install checks, defaults to false --setup @@ -62,8 +68,8 @@ Options: Proxy address --service-type Service Type - --connector-prefix - Connector Prefix + --read-only + Start SC in read only mode -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-cluster.md b/embeds/cli/help/fluvio-cluster.md index 4be24d42..f6868995 100644 --- a/embeds/cli/help/fluvio-cluster.md +++ b/embeds/cli/help/fluvio-cluster.md @@ -1,6 +1,8 @@ ``` Install or uninstall Fluvio cluster + Usage: fluvio cluster + Commands: start Install Fluvio cluster upgrade Upgrades an already-started Fluvio cluster @@ -11,6 +13,7 @@ Commands: diagnostics Collect anonymous diagnostic information to help with debugging status Check the status of a Fluvio cluster shutdown Shutdown cluster processes without deleting data + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-completions.md b/embeds/cli/help/fluvio-completions.md index d95ddcd8..fc7fae51 100644 --- a/embeds/cli/help/fluvio-completions.md +++ b/embeds/cli/help/fluvio-completions.md @@ -1,10 +1,21 @@ ``` Generate command-line completions for Fluvio + +Run the following two commands to enable fluvio command completions. + +Open a new terminal for the changes to take effect. + +$ fluvio completions bash > ~/fluvio_completions.sh +$ echo "source ~/fluvio_completions.sh" >> ~/.bashrc + Usage: fluvio completions + Commands: bash Generate CLI completions for bash zsh Generate CLI completions for zsh fish Generate CLI completions for fish + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-consume.md b/embeds/cli/help/fluvio-consume.md index dae7184e..b0cb5a65 100644 --- a/embeds/cli/help/fluvio-consume.md +++ b/embeds/cli/help/fluvio-consume.md @@ -1,53 +1,94 @@ ``` Read messages from a topic/partition + Usage: fluvio consume [OPTIONS] + Arguments: - Topic name + + Topic name + Options: -p, --partition - Partition id [default: 0] + Partition id + -A, --all-partitions Consume records from all partitions + -d, --disable-continuous Disable continuous processing of messages + --disable-progressbar Disable the progress bar and wait spinner + -k, --key-value Print records in "[key] value" format, with "[null]" for no key + -F, --format - Provide a template string to print records with a custom format. See --help for details + Provide a template string to print records with a custom format. See --help for details. + + Template strings may include the variables {{key}}, {{value}}, {{offset}}, {{partition}} and {{time}} which will have each record's contents substituted in their place. Note that timestamp is displayed using RFC3339, is always UTC and ignores system timezone. + + For example, the following template string: + + Offset {{offset}} has key {{key}} and value {{value}} + + Would produce a printout where records might look like this: + + Offset 0 has key A and value Apple + --table-format Consume records using the formatting rules defined by TableFormat name + -B, --beginning Consume records from the beginning of the log + -H, --head Consume records starting from the beginning of the log + -T, --tail Consume records starting from the end of the log + --start The absolute offset of the first record to begin consuming from + --end Consume records until end offset (inclusive) + -b, --maxbytes Maximum number of bytes to be retrieved + --isolation Isolation level that consumer must respect. Supported values: read_committed (ReadCommitted) - consume only committed records, read_uncommitted (ReadUncommitted) - consume all records accepted by leader + --suppress-unknown Suppress items items that have an unknown output type + -O, --output - Output [possible values: dynamic, text, binary, json, raw, table, full-table] + Output + + [possible values: dynamic, text, binary, json, raw, table, full-table] + --smartmodule Name of the smartmodule + --smartmodule-path Path to the smart module + --aggregate-initial (Optional) Value to use as an initial accumulator for aggregate SmartModules + -e, --params (Optional) Extra input parameters passed to the smartmodule. They should be passed using key=value format Eg. fluvio consume topic-name --smartmodule my_filter -e foo=bar -e key=value -e one=1 + --transforms-file (Optional) Path to a file with transformation specification + -t, --transform (Optional) Transformation specification as JSON formatted string. E.g. fluvio consume topic-name --transform='{"uses":"infinyon/jolt@0.1.0","with":{"spec":"[{\"operation\":\"default\",\"spec\":{\"source\":\"test\"}}]"}}' + + --truncate + Truncate the output to one line + -h, --help Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-derived-stream-create.md b/embeds/cli/help/fluvio-derived-stream-create.md deleted file mode 100644 index f1c3fffb..00000000 --- a/embeds/cli/help/fluvio-derived-stream-create.md +++ /dev/null @@ -1,3 +0,0 @@ -``` -Unable to find plugin 'fluvio-derived-stream'. Make sure it is installed in "/Users/telant/.fluvio/extensions-latest". -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-derived-stream-delete.md b/embeds/cli/help/fluvio-derived-stream-delete.md deleted file mode 100644 index f1c3fffb..00000000 --- a/embeds/cli/help/fluvio-derived-stream-delete.md +++ /dev/null @@ -1,3 +0,0 @@ -``` -Unable to find plugin 'fluvio-derived-stream'. Make sure it is installed in "/Users/telant/.fluvio/extensions-latest". -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-derived-stream-list.md b/embeds/cli/help/fluvio-derived-stream-list.md deleted file mode 100644 index f1c3fffb..00000000 --- a/embeds/cli/help/fluvio-derived-stream-list.md +++ /dev/null @@ -1,3 +0,0 @@ -``` -Unable to find plugin 'fluvio-derived-stream'. Make sure it is installed in "/Users/telant/.fluvio/extensions-latest". -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-derived-stream.md b/embeds/cli/help/fluvio-derived-stream.md deleted file mode 100644 index f1c3fffb..00000000 --- a/embeds/cli/help/fluvio-derived-stream.md +++ /dev/null @@ -1,3 +0,0 @@ -``` -Unable to find plugin 'fluvio-derived-stream'. Make sure it is installed in "/Users/telant/.fluvio/extensions-latest". -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-connector-download.md b/embeds/cli/help/fluvio-hub-connector-download.md index f9b28fb1..4ec870d8 100644 --- a/embeds/cli/help/fluvio-hub-connector-download.md +++ b/embeds/cli/help/fluvio-hub-connector-download.md @@ -1,10 +1,24 @@ ``` Download SmartConnector to the local folder + Usage: fluvio hub connector download [OPTIONS] + Arguments: - SmartConnector name: e.g. infinyon/salesforce-sink@v0.0.1 + + SmartConnector name: e.g. infinyon/http-sink@vX.Y.Z + Options: - -o, --output Target local folder or file name - --target Target platform for the package. Optional. By default the host's one is used [default: aarch64-apple-darwin] - -h, --help Print help (see more with '--help') + -o, --output + Target local folder or file name + + --target + Target platform for the package. Optional. By default the host's one is used + + [default: aarch64-apple-darwin] + + --remote + + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-connector-list.md b/embeds/cli/help/fluvio-hub-connector-list.md index 39f539d9..fdd25d9d 100644 --- a/embeds/cli/help/fluvio-hub-connector-list.md +++ b/embeds/cli/help/fluvio-hub-connector-list.md @@ -1,7 +1,18 @@ ``` List all available SmartConnectors + Usage: fluvio hub connector list [OPTIONS] + Options: - -O, --output Output [default: table] [possible values: table, yaml, json] - -h, --help Print help (see more with '--help') + -O, --output + Output + + [default: table] + [possible values: table, yaml, json] + + --remote + + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-connector.md b/embeds/cli/help/fluvio-hub-connector.md index 9b5fc53b..a09c1a2e 100644 --- a/embeds/cli/help/fluvio-hub-connector.md +++ b/embeds/cli/help/fluvio-hub-connector.md @@ -1,9 +1,12 @@ ``` List available Connectors in the hub + Usage: fluvio hub connector + Commands: list List all available SmartConnectors download Download SmartConnector to the local folder + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-download.md b/embeds/cli/help/fluvio-hub-download.md deleted file mode 100644 index f52af6c2..00000000 --- a/embeds/cli/help/fluvio-hub-download.md +++ /dev/null @@ -1,18 +0,0 @@ -``` -Download a SmartModule from the hub -Usage: fluvio hub download [OPTIONS] -Arguments: - SmartModule name: e.g. infinyon/jolt@v0.0.1 -Options: - -c, --cluster Address of cluster - --tls Enable TLS - --enable-client-cert TLS: use client cert - --domain Required if client cert is used - --ca-cert Path to TLS ca cert, required when client cert is enabled - --client-cert Path to TLS client certificate - --client-key Path to TLS client private key - -P, --profile - --local just download package to local filesystem - --ipkg given local package file, download to cluster - -h, --help Print help (see more with '--help') -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-list.md b/embeds/cli/help/fluvio-hub-list.md deleted file mode 100644 index f0d14d31..00000000 --- a/embeds/cli/help/fluvio-hub-list.md +++ /dev/null @@ -1,7 +0,0 @@ -``` -List available SmartModules in the hub -Usage: fluvio hub list [OPTIONS] -Options: - -O, --output Output [default: table] [possible values: table, yaml, json] - -h, --help Print help (see more with '--help') -``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-smartmodule-download.md b/embeds/cli/help/fluvio-hub-smartmodule-download.md new file mode 100644 index 00000000..f346c34b --- /dev/null +++ b/embeds/cli/help/fluvio-hub-smartmodule-download.md @@ -0,0 +1,46 @@ +``` +Download SmartModules - locally or to cluster (default) + +Usage: fluvio hub smartmodule download [OPTIONS] + +Arguments: + + SmartModule name: e.g. infinyon/jolt@v0.0.1 + +Options: + -c, --cluster + Address of cluster + + --tls + Enable TLS + + --enable-client-cert + TLS: use client cert + + --domain + Required if client cert is used + + --ca-cert + Path to TLS ca cert, required when client cert is enabled + + --client-cert + Path to TLS client certificate + + --client-key + Path to TLS client private key + + -P, --profile + + + -o, --output + Download package to local filesystem + + --ipkg + given local package file, download to cluster + + --remote + + + -h, --help + Print help (see a summary with '-h') +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-smartmodule-list.md b/embeds/cli/help/fluvio-hub-smartmodule-list.md new file mode 100644 index 00000000..3572ae4d --- /dev/null +++ b/embeds/cli/help/fluvio-hub-smartmodule-list.md @@ -0,0 +1,18 @@ +``` +List all available SmartModules + +Usage: fluvio hub smartmodule list [OPTIONS] + +Options: + -O, --output + Output + + [default: table] + [possible values: table, yaml, json] + + --remote + + + -h, --help + Print help (see a summary with '-h') +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub-smartmodule.md b/embeds/cli/help/fluvio-hub-smartmodule.md new file mode 100644 index 00000000..28a7a506 --- /dev/null +++ b/embeds/cli/help/fluvio-hub-smartmodule.md @@ -0,0 +1,12 @@ +``` +List available SmartModules in the hub + +Usage: fluvio hub smartmodule + +Commands: + list List all available SmartModules + download Download SmartModules - locally or to cluster (default) + +Options: + -h, --help Print help +``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-hub.md b/embeds/cli/help/fluvio-hub.md index cc9fa5c9..b033d3b6 100644 --- a/embeds/cli/help/fluvio-hub.md +++ b/embeds/cli/help/fluvio-hub.md @@ -1,10 +1,12 @@ ``` Work with the SmartModule Hub + Usage: fluvio hub + Commands: - download Download a SmartModule from the hub - list List available SmartModules in the hub - connector List available Connectors in the hub + smartmodule List available SmartModules in the hub [aliases: sm] + connector List available Connectors in the hub [aliases: conn] + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-install.md b/embeds/cli/help/fluvio-install.md index 8d4aa3c9..7a8ef78b 100644 --- a/embeds/cli/help/fluvio-install.md +++ b/embeds/cli/help/fluvio-install.md @@ -1,9 +1,34 @@ ``` Install Fluvio plugins + +The Fluvio CLI considers any executable with the prefix `fluvio-` to be a CLI plugin. For example, an executable named `fluvio-foo` in your PATH may be invoked by running `fluvio foo`. + +This command allows you to install plugins from Fluvio's package registry. + Usage: fluvio install [OPTIONS] [PACKAGE] + Arguments: - [PACKAGE] The ID of a package to install, e.g. "fluvio/fluvio-cloud" + [PACKAGE] + The ID of a package to install, e.g. "fluvio/fluvio-cloud" + Options: - --develop Install the latest prerelease rather than the latest release - -h, --help Print help (see more with '--help') + --develop + Install the latest prerelease rather than the latest release + + If the package ID contains a version (e.g. `fluvio/fluvio:0.6.0`), this is ignored + + --hub + When this flag is provided, use the hub. Dev-only + + --use-hub-defaults + Use local hub defaults. Implied if INFINYON_HUB_REMOTE or FLUVIO_CLOUD_PROFILE env vars are set - Dev-only + + --channel + When this flag is provided, use the hub. Dev-only + + --target + override default target arch determination + + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-partition-list.md b/embeds/cli/help/fluvio-partition-list.md index 2b5a7ea9..32f57b46 100644 --- a/embeds/cli/help/fluvio-partition-list.md +++ b/embeds/cli/help/fluvio-partition-list.md @@ -1,6 +1,8 @@ ``` List all of the Partitions in this cluster + fluvio partition list [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-partition.md b/embeds/cli/help/fluvio-partition.md index 7c6cf27d..b53dffe1 100644 --- a/embeds/cli/help/fluvio-partition.md +++ b/embeds/cli/help/fluvio-partition.md @@ -1,8 +1,14 @@ ``` Manage and view Partitions + +Partitions are a way to divide the total traffic of a single Topic into separate streams which may be processed independently. Data sent to different partitions may be processed by separate SPUs on different computers. By dividing the load of a Topic evenly among partitions, you can increase the total throughput of the Topic. + Usage: fluvio partition + Commands: list List all of the Partitions in this cluster + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-produce.md b/embeds/cli/help/fluvio-produce.md index 20f1dfd2..d6e781de 100644 --- a/embeds/cli/help/fluvio-produce.md +++ b/embeds/cli/help/fluvio-produce.md @@ -1,8 +1,11 @@ ``` Write messages to a topic/partition + Usage: fluvio produce [OPTIONS] + Arguments: The name of the Topic to produce to + Options: -v, --verbose Print progress output when sending records @@ -13,7 +16,7 @@ Options: --raw Send all input as one record. Use this when producing binary files --compression - Compression algorithm to use when sending records. Supported values: none, gzip, snappy and lz4 + Compression algorithm to use when sending records. Supported values: none, gzip, snappy, zstd and lz4 -f, --file Path to a file to produce to the topic. Default: Each line treated as single record unless `--raw` specified. If absent, producer will read stdin --linger diff --git a/embeds/cli/help/fluvio-profile-current.md b/embeds/cli/help/fluvio-profile-current.md index 1f92f1cd..316faec1 100644 --- a/embeds/cli/help/fluvio-profile-current.md +++ b/embeds/cli/help/fluvio-profile-current.md @@ -1,4 +1,8 @@ ``` -Print the name of the current context Options: -Usage: fluvio profile current -h, --help Print help +Print the name of the current context + +Usage: fluvio profile current + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-profile-delete-cluster.md b/embeds/cli/help/fluvio-profile-delete-cluster.md index ce6b29e6..da52b3ac 100644 --- a/embeds/cli/help/fluvio-profile-delete-cluster.md +++ b/embeds/cli/help/fluvio-profile-delete-cluster.md @@ -1,8 +1,11 @@ ``` Delete the named cluster + Usage: fluvio profile delete-cluster [OPTIONS] + Arguments: The name of a cluster connection to delete + Options: -f, --force Deletes a cluster even if its active -h, --help Print help diff --git a/embeds/cli/help/fluvio-profile-delete.md b/embeds/cli/help/fluvio-profile-delete.md index e3a226c3..f1251b52 100644 --- a/embeds/cli/help/fluvio-profile-delete.md +++ b/embeds/cli/help/fluvio-profile-delete.md @@ -1,8 +1,11 @@ ``` Delete the named profile + Usage: fluvio profile delete + Arguments: + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-profile-export.md b/embeds/cli/help/fluvio-profile-export.md index a53ef809..162a646c 100644 --- a/embeds/cli/help/fluvio-profile-export.md +++ b/embeds/cli/help/fluvio-profile-export.md @@ -1,8 +1,11 @@ ``` Export a profile for use in other applications + Usage: fluvio profile export [OPTIONS] [PROFILE_NAME] + Arguments: [PROFILE_NAME] + Options: -O, --output [default: json] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-profile-list.md b/embeds/cli/help/fluvio-profile-list.md index 90f6e829..ad0d1a10 100644 --- a/embeds/cli/help/fluvio-profile-list.md +++ b/embeds/cli/help/fluvio-profile-list.md @@ -1,6 +1,8 @@ ``` Display the entire Fluvio configuration + Usage: fluvio profile list [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-profile-rename.md b/embeds/cli/help/fluvio-profile-rename.md index 30c55c34..bc150259 100644 --- a/embeds/cli/help/fluvio-profile-rename.md +++ b/embeds/cli/help/fluvio-profile-rename.md @@ -1,9 +1,12 @@ ``` Rename a profile + Usage: fluvio profile rename + Arguments: The name of the profile to rename The new name to give the profile + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-profile-switch.md b/embeds/cli/help/fluvio-profile-switch.md index 09c5c617..b6c2479b 100644 --- a/embeds/cli/help/fluvio-profile-switch.md +++ b/embeds/cli/help/fluvio-profile-switch.md @@ -1,8 +1,11 @@ ``` Switch to the named profile + Usage: fluvio profile switch + Arguments: + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-profile-sync.md b/embeds/cli/help/fluvio-profile-sync.md index b7d4a678..a0ccfad3 100644 --- a/embeds/cli/help/fluvio-profile-sync.md +++ b/embeds/cli/help/fluvio-profile-sync.md @@ -1,9 +1,12 @@ ``` Sync a profile from a cluster + fluvio profile sync + Commands: k8 Sync a profile from a Kubernetes cluster local Sync a profile from a local cluster + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-profile.md b/embeds/cli/help/fluvio-profile.md index e41acec1..678ec0c9 100644 --- a/embeds/cli/help/fluvio-profile.md +++ b/embeds/cli/help/fluvio-profile.md @@ -1,6 +1,10 @@ ``` Manage Profiles, which describe linked clusters + +Each Profile describes a particular Fluvio cluster you may be connected to. This might correspond to Fluvio running on Minikube or in the Cloud. There is one "active" profile, which determines which cluster all of the Fluvio CLI commands interact with. + Usage: fluvio profile [COMMAND] + Commands: current Print the name of the current context delete Delete the named profile @@ -10,6 +14,9 @@ Commands: switch Switch to the named profile sync Sync a profile from a cluster export Export a profile for use in other applications + add Manually add a profile (advanced) + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-run-metadata.md b/embeds/cli/help/fluvio-run-metadata.md index 5972e267..74f27f30 100644 --- a/embeds/cli/help/fluvio-run-metadata.md +++ b/embeds/cli/help/fluvio-run-metadata.md @@ -1,4 +1,8 @@ ``` -Return plugin metadata as JSON Options: -Usage: fluvio-run metadata -h, --help Print help +Return plugin metadata as JSON + +Usage: fluvio-run metadata + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-run-sc.md b/embeds/cli/help/fluvio-run-sc.md index d7160f5e..4aecb277 100644 --- a/embeds/cli/help/fluvio-run-sc.md +++ b/embeds/cli/help/fluvio-run-sc.md @@ -1,14 +1,19 @@ ``` Run a new Streaming Controller (SC) + Usage: fluvio-run sc [OPTIONS] + Options: --local running in local mode only + --no-k8 + run on k8 --bind-public Address for external service --bind-private Address for internal service -n, --namespace + --tls enable tls --server-cert diff --git a/embeds/cli/help/fluvio-run-spu.md b/embeds/cli/help/fluvio-run-spu.md index 52d6db8b..99e622cd 100644 --- a/embeds/cli/help/fluvio-run-spu.md +++ b/embeds/cli/help/fluvio-run-spu.md @@ -1,6 +1,8 @@ ``` Run a new Streaming Processing Unit (SPU) + Usage: fluvio-run spu [OPTIONS] + Options: -i, --id SPU unique identifier @@ -20,6 +22,8 @@ Options: [env: FLV_LOG_INDEX_MAX_INTERVAL_BYTES=] --peer-max-bytes max bytes to transfer between leader and follower [env: FLV_PEER_MAX_BYTES=] [default: 1000000] + --smart-engine-max-memory + [env: FLV_SMART_ENGINE_MAX_MEMORY_BYTES=] --tls enable tls --server-cert diff --git a/embeds/cli/help/fluvio-run-version.md b/embeds/cli/help/fluvio-run-version.md index 1f90188c..631e9d2e 100644 --- a/embeds/cli/help/fluvio-run-version.md +++ b/embeds/cli/help/fluvio-run-version.md @@ -1,4 +1,8 @@ ``` -Print version information Options: -Usage: fluvio-run version -h, --help Print help +Print version information + +Usage: fluvio-run version + +Options: + -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-run.md b/embeds/cli/help/fluvio-run.md index 3bdc4e2f..7a4b32bc 100644 --- a/embeds/cli/help/fluvio-run.md +++ b/embeds/cli/help/fluvio-run.md @@ -1,11 +1,13 @@ ``` Usage: fluvio-run + Commands: spu Run a new Streaming Processing Unit (SPU) sc Run a new Streaming Controller (SC) metadata Return plugin metadata as JSON version Print version information help Print this message or the help of the given subcommand(s) + Options: -h, --help Print help -V, --version Print version diff --git a/embeds/cli/help/fluvio-smartmodule-create.md b/embeds/cli/help/fluvio-smartmodule-create.md index 75548f6a..4a8c1fe6 100644 --- a/embeds/cli/help/fluvio-smartmodule-create.md +++ b/embeds/cli/help/fluvio-smartmodule-create.md @@ -1,8 +1,11 @@ ``` Create a new SmartModule with a given name + Usage: fluvio smartmodule create [OPTIONS] --wasm-file + Arguments: The name of the SmartModule to create + Options: --wasm-file The path to a WASM binary to create the SmartModule from --package The path to the SmartModule package (experimental) diff --git a/embeds/cli/help/fluvio-smartmodule-delete.md b/embeds/cli/help/fluvio-smartmodule-delete.md index e31e21ba..9593aa3c 100644 --- a/embeds/cli/help/fluvio-smartmodule-delete.md +++ b/embeds/cli/help/fluvio-smartmodule-delete.md @@ -1,8 +1,11 @@ ``` Delete one or more SmartModules with the given name(s) + Usage: fluvio smartmodule delete [OPTIONS] ... + Arguments: ... One or more name(s) of the smartmodule(s) to be deleted + Options: -c, --continue-on-error Continue deleting in case of an error -h, --help Print help diff --git a/embeds/cli/help/fluvio-smartmodule-list.md b/embeds/cli/help/fluvio-smartmodule-list.md index 9b0209c2..238ad5ef 100644 --- a/embeds/cli/help/fluvio-smartmodule-list.md +++ b/embeds/cli/help/fluvio-smartmodule-list.md @@ -1,6 +1,8 @@ ``` List all existing SmartModules + Usage: fluvio smartmodule list [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] --filter diff --git a/embeds/cli/help/fluvio-smartmodule-watch.md b/embeds/cli/help/fluvio-smartmodule-watch.md index 2d5cbcdc..9bf38a11 100644 --- a/embeds/cli/help/fluvio-smartmodule-watch.md +++ b/embeds/cli/help/fluvio-smartmodule-watch.md @@ -1,6 +1,8 @@ ``` Watch for changes to SmartModules + Usage: fluvio smartmodule watch [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-smartmodule.md b/embeds/cli/help/fluvio-smartmodule.md index 792e7d57..51f0affb 100644 --- a/embeds/cli/help/fluvio-smartmodule.md +++ b/embeds/cli/help/fluvio-smartmodule.md @@ -1,11 +1,17 @@ ``` Create and manage SmartModules + +SmartModules are compiled WASM modules used to create SmartModules. + Usage: fluvio smartmodule + Commands: create Create a new SmartModule with a given name list List all existing SmartModules watch Watch for changes to SmartModules delete Delete one or more SmartModules with the given name(s) + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-table-format-create.md b/embeds/cli/help/fluvio-table-format-create.md index eaf39252..fa5b242b 100644 --- a/embeds/cli/help/fluvio-table-format-create.md +++ b/embeds/cli/help/fluvio-table-format-create.md @@ -1,6 +1,8 @@ ``` Create a new TableFormat display + fluvio table-format create --config + Options: -c, --config The path to the TableFormat config -h, --help Print help diff --git a/embeds/cli/help/fluvio-table-format-delete.md b/embeds/cli/help/fluvio-table-format-delete.md index bf656cb0..1987f3f5 100644 --- a/embeds/cli/help/fluvio-table-format-delete.md +++ b/embeds/cli/help/fluvio-table-format-delete.md @@ -1,8 +1,11 @@ ``` Delete a TableFormat display + fluvio table-format delete + Arguments: - The name of the connector to delete + The name of the table format to delete + Options: -h, --help Print help ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-table-format-list.md b/embeds/cli/help/fluvio-table-format-list.md index 9f1b26ec..31ba4e7b 100644 --- a/embeds/cli/help/fluvio-table-format-list.md +++ b/embeds/cli/help/fluvio-table-format-list.md @@ -1,6 +1,8 @@ ``` List all TableFormat display + fluvio table-format list [OPTIONS] + Options: -O, --output Output [default: table] [possible values: table, yaml, json] -h, --help Print help diff --git a/embeds/cli/help/fluvio-table-format.md b/embeds/cli/help/fluvio-table-format.md index 2d768bba..7f8e4970 100644 --- a/embeds/cli/help/fluvio-table-format.md +++ b/embeds/cli/help/fluvio-table-format.md @@ -1,10 +1,16 @@ ``` Create a TableFormat display specification + +Used with the consumer output type `full_table` to describe how to render JSON data in a tabular form + Usage: fluvio table-format + Commands: create Create a new TableFormat display delete Delete a TableFormat display list List all TableFormat display + Options: - -h, --help Print help (see more with '--help') + -h, --help + Print help (see a summary with '-h') ``` \ No newline at end of file diff --git a/embeds/cli/help/fluvio-topic-create.md b/embeds/cli/help/fluvio-topic-create.md index 8ef49670..70c0bdc0 100644 --- a/embeds/cli/help/fluvio-topic-create.md +++ b/embeds/cli/help/fluvio-topic-create.md @@ -1,17 +1,55 @@ ``` Create a Topic with the given name -fluvio topic create [OPTIONS] + +fluvio topic create [OPTIONS] [name] + Arguments: - The name of the Topic to create + [name] + The name of the Topic to create + Options: - -p, --partitions The number of Partitions to give the Topic [default: 1] - -r, --replication The number of full replicas of the Topic to keep [default: 1] - -i, --ignore-rack-assignment Ignore racks while computing replica assignment - -f, --replica-assignment Replica assignment file - -d, --dry-run Validates configuration, does not provision - --retention-time