diff --git a/embeds/connectors/outbound/graphite.md b/embeds/connectors/outbound/graphite.md index c4ac9d1e..877aaa2e 100644 --- a/embeds/connectors/outbound/graphite.md +++ b/embeds/connectors/outbound/graphite.md @@ -15,7 +15,7 @@ server address is specified on the `addr` field. # sample-config.yaml apiVersion: 0.1.0 meta: - version: 0.1.1 + version: 0.1.2 name: my-graphite-connector-test-connector type: graphite-sink topic: test-graphite-connector-topic @@ -34,15 +34,31 @@ graphite: ## Usage -In this section we are going to setup a Graphite instance on Docker and then -we will use the CDK to spin up the Graphite Sink Connector to send metrics from -Fluvio Records to the Graphite instance. +This section will walk you through the process of setting up a Graphite +instance and using Fluvio to send metrics to this Graphite instance. -Setup a Graphite instance for local development, you can follow the recipe used -to contribute to the official InfinyOn connector [here][3]. +> This section assumes you have Docker and Fluvio installed in your system. -Reduce the Graphite's data retention interval by updating `storage-schemas.conf` -file for [Carbon][6]: +### Setting Up Graphite + +We will run our Graphite instance on Docker using the `docker compose` command +for simplicity. + +The Graphite container will setup [Carbon Configuration][6] files in your +working directory, we need to update these files to reduce Carbon's persisntance +intervals, making it more frecuent. + +Create a copy of our [`docker-compose.yml`][7] file and execute the container: + +```bash +docker compose up --build -d +``` + +This will generate a directory with the name `.graphite`, which contains +configuration files. + +Replace the contents of `.graphite/conf/storage-schemas.conf` to record on an +interval of 10 seconds and persist the last 12 hours of data. ```conf [all] @@ -50,7 +66,26 @@ pattern = .* retentions = 10s:12h ``` -Make sure the Connector Development Key is setup in your system by issuing +Now we need to re run the Graphite container so Carbon uses the new +configuration. + +```bash +docker compose down +docker compose up --build -d +``` + +You can visit `http://localhost:12345` in your browser to access the Dashboard. + +> Credentials for the Dashbord are User: `root` and Password: `root` + +With the Graphite instance set, we can move into [Setting Up Fluvio with Graphite Sink Connector][8]. + +### Setting Up Fluvio with Graphite Sink Connector + +In this section we are going use the CDK to spin up the Graphite Sink Connector +to send metrics from Fluvio Records to the Graphite instance. + +Make sure the Connector Development Kit is setup in your system by issuing the following command in your terminal. ```bash @@ -64,7 +99,7 @@ Create a YAML file with the name `weather-monitor-config.yaml` and specify conne ```yaml apiVersion: 0.1.0 meta: - version: 0.1.1 + version: 0.1.2 name: weather-monitor-sandiego type: graphite-sink topic: weather-ca-sandiego @@ -74,6 +109,12 @@ graphite: addr: "localhost:2003" ``` +Deploy the Connector using the CDK + +```bash +cdk deploy start --config weather-monitor-config.yaml +``` + > Make sure your Graphite instance is running on `localhost:2003`, use the > `cdk log` subcommand to read logs from the connector instance. @@ -83,7 +124,10 @@ Then produce records as usual: echo 120 | fluvio produce weather-ca-sandiego ``` -You can check on Graphite's API for your data points: +> Remember that Carbon's retention is set to `10s:12h`, this means that if will +> write metrics every 10s. + +Use Graphite's REST API to check on the stored data. ```bash curl -o ./data.json http://localhost:12345/render\?target\=weather.temperature.ca.sandiego\&format\=json\&noNullPoints @@ -95,3 +139,5 @@ curl -o ./data.json http://localhost:12345/render\?target\=weather.temperature.c [4]: https://graphiteapp.org/ [5]: https://graphite.readthedocs.io/en/latest/feeding-carbon.html#the-plaintext-protocol [6]: https://graphite.readthedocs.io/en/latest/config-carbon.html#storage-schemas-conf +[7]: https://github.com/infinyon/graphite-sink-connector/blob/main/docker-compose.yml +[8]: #setting-up-fluvio-with-graphite-sink-connector