Update Version Number. (#756)

* Update Fluvio live site with master updates (#733) * docs: Cli update (#721) * Update CLI docs * Rearrange nav a bit * Removing the infinyon banner too * Update command * Update labels * First pass updates to docs page (#726) * docs cleanup (#732) * Modify `smdk test` examples and documentation to use `--stdin` (#666) * Remove deprecated script from readme * Modify `smdk test` examples and doc to use `--stdin` --------- Co-authored-by: T.J. Telan <t.telan@gmail.com> --------- Co-authored-by: T.J. Telan <t.telan@gmail.com> Co-authored-by: Consoli <matheus.consoli7@gmail.com> * Merge from master (#747) * docs: Cli update (#721) * Update CLI docs * Rearrange nav a bit * Removing the infinyon banner too * Update command * Update labels * First pass updates to docs page (#726) * docs cleanup (#732) * Modify `smdk test` examples and documentation to use `--stdin` (#666) * Remove deprecated script from readme * Modify `smdk test` examples and doc to use `--stdin` --------- Co-authored-by: T.J. Telan <t.telan@gmail.com> * docs: fix link for go package (#734) Prefers the doc page hosted by fluvio since none is directly offered by package * rename config-example.yml to config-sample.yml, (#744) that's the name of the file generated by `cdk generate` * Clarified the term SPU and linked to appropriate page (#746) * chore: fix lint ci --------- Co-authored-by: T.J. Telan <t.telan@gmail.com> Co-authored-by: Consoli <matheus.consoli7@gmail.com> Co-authored-by: Tom Carrio <tom@carrio.dev> Co-authored-by: mrasputin <150076078+mrasputin@users.noreply.github.com> Co-authored-by: John Landahl <john@landahl.org> * fix: list-log samples (#749) (#750) closes #745 * fix: log-and-list navigation (#751) (#752) * fix: log-and-list navigation fix log nav consistency * embeds/client-examples, use python 3.11 * update connector file to match the the project Readme. --------- Co-authored-by: Deb <122817612+drc-infinyon@users.noreply.github.com> Co-authored-by: T.J. Telan <t.telan@gmail.com> Co-authored-by: Consoli <matheus.consoli7@gmail.com> Co-authored-by: Alan Chen <argot@digikata.com> Co-authored-by: Tom Carrio <tom@carrio.dev> Co-authored-by: mrasputin <150076078+mrasputin@users.noreply.github.com> Co-authored-by: John Landahl <john@landahl.org>
infinyon · Jan 8, 2024 · b97438b · b97438b
1 parent cbfb9c9
commit b97438b
Showing 1 changed file with 33 additions and 7 deletions.
diff --git a/embeds/connectors/inbound/http.md b/embeds/connectors/inbound/http.md
@@ -1,6 +1,11 @@
 # Fluvio HTTP Inbound Connector
 
-Read HTTP Responses given input HTTP request configuration options and `interval` x and produces them to Fluvio topics.
+Read HTTP Responses given input HTTP request configuration options and produce them
+to Fluvio topics.
+
+Supports both polling of atomic endpoints and streaming of HTTP responses when given the `stream` configuration option.
+When polling, an `interval` configuration option is accepted. When streaming a `delimiter` configuration option, which
+defaults to `'\n'`, is accepted.
 
 Supports HTTP/1.0, HTTP/1.1, HTTP/2.0 protocols.
 
@@ -18,6 +23,8 @@ Tutorial for [HTTP to SQL Pipeline](https://www.fluvio.io/docs/tutorials/data-pi
 | user-agent   | "fluvio/http-source 0.1.0" | String  | Request user-agent                                                                         |
 | output_type  | text                       | String  | `text` = UTF-8 String Output, `json` = UTF-8 JSON Serialized String                        |
 | output_parts | body                       | String  | `body` = body only, `full` = all status, header and body parts                             |
+| stream       | false                      | bool    | Flag to indicate HTTP streaming mode                                                       |
+| delimiter    | '\n'                       | String  | Delimiter to separate records when producing from an HTTP streaming endpoint               |
 
 #### Record Type Output
 | Matrix                                                      | Output                                  |
@@ -33,10 +40,10 @@ Tutorial for [HTTP to SQL Pipeline](https://www.fluvio.io/docs/tutorials/data-pi
 This is an example of simple connector config file:
 
 ```yaml
-# sample-config.yaml
+# config-example.yaml
 apiVersion: 0.1.0
 meta:
-  version: 0.2.5
+  version: 0.3.0
   name: cat-facts
   type: http-source
   topic: cat-facts
@@ -63,10 +70,10 @@ The produced record in Fluvio topic will be:
 Fluvio HTTP Source Connector supports Secrets in the `endpoint` and in the `headers` parameters:
 
 ```yaml
-# sample-config.yaml
+# config-example.yaml
 apiVersion: 0.1.0
 meta:
-  version: 0.2.5
+  version: 0.3.0
   name: cat-facts
   type: http-source
   topic: cat-facts
@@ -89,10 +96,10 @@ Fluvio HTTP Source Connector supports [Transformations](https://www.fluvio.io/do
 
 The previous example can be extended to add extra transformations to outgoing records:
 ```yaml
-# sample-config.yaml
+# config-example.yaml
 apiVersion: 0.1.0
 meta:
-  version: 0.2.5
+  version: 0.3.0
   name: cat-facts
   type: http-source
   topic: cat-facts
@@ -123,3 +130,22 @@ Now produced records will have a different shape, for example:
 ```
 
 Read more about [JSON to JSON transformations](https://www.fluvio.io/smartmodules/certified/jolt/).
+
+### Streaming Mode
+
+Provide the `stream` configuration option to enable streaming mode with `delimiter` to determine how the incoming records are separated.
+
+```yaml
+# config-example.yaml
+apiVersion: 0.1.0
+meta:
+  version: 0.3.0
+  name: wiki-updates
+  type: http-source
+  topic: wiki-updates
+http:
+  endpoint: "https://stream.wikimedia.org/v2/stream/recentchange"
+  method: GET
+  stream: true
+  delimiter: "\n\n"
+```