Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

updated confluent local commands. And upped CP version to 6.2 #86

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

updated confluent local commands. And upped CP version to 6.2 #86

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ confluent-hub install target/components/packages/confluentinc-kafka-connect-data
Here is an example of how to run the `kafka-connect-datagen` on a local Confluent Platform after it's been installed. [Configuration](#configuration) details are provided below.

```bash
confluent local start connect
confluent local config datagen-pageviews -- -d config/connector_pageviews.config
confluent local status connectors
confluent local consume test1 --value-format avro --max-messages 5 --property print.key=true --property key.deserializer=org.apache.kafka.common.serialization.StringDeserializer --from-beginning
confluent local services connect start
confluent local services connect connector load datagen-pageviews -c config/connector_pageviews.config
confluent local services connect connector status datagen-pageviews
kafka-console-consumer --bootstrap-server localhost:9092 --topic pageviews --property print.key=true --max-messages 5 --from-beginning
```

## Install the connector from Confluent Hub into a [Kafka Connect](https://docs.confluent.io/current/connect/index.html) based Docker image
Expand All @@ -67,27 +67,27 @@ If you want to build a local copy of the Docker image with `kafka-connect-datage

You can create a Docker image packaged with the locally built source by running (for example with the 5.5.0 version of Confluent Platform):
```bash
make build-docker-from-local CP_VERSION=5.5.0
make build-docker-from-local CP_VERSION=6.2.0
```

This will build the connector from source and create a local image with an aggregate version number. The aggregate version number is the kafka-connect-datagen connector version number and the Confluent Platform version number separated with a `-`. The local kafka-connect-datagen version number is defined in the `pom.xml` file, and the Confluent Platform version defined in the [Makefile](Makfile). An example of the aggregate version number might be: `0.4.0-6.1.0`.

Alternatively, you can install the `kafka-connect-datagen` connector from [Confluent Hub](https://www.confluent.io/connector/kafka-connect-datagen/) into a Docker image by running:
```bash
make build-docker-from-released CP_VERSION=5.5.0
make build-docker-from-released CP_VERSION=6.2.0
```

The [Makefile](Makefile) contains some default variables that affect the version numbers of both the installed `kafka-connect-datagen` as well as the base Confluent Platform version. The variables are located near the top of the [Makefile](Makefile) with the following names and current default values:

```bash
CP_VERSION ?= 6.1.0
CP_VERSION ?= 6.2.0

KAFKA_CONNECT_DATAGEN_VERSION ?= 0.4.0
KAFKA_CONNECT_DATAGEN_VERSION ?= 0.5.0
```
These values can be overriden with variable declarations before the `make` command. For example:

```bash
KAFKA_CONNECT_DATAGEN_VERSION=0.3.2 make build-docker-from-released
KAFKA_CONNECT_DATAGEN_VERSION=0.5.0 make build-docker-from-released
```

### Run connector in Docker Compose
Expand Down Expand Up @@ -238,8 +238,8 @@ To release new versions of the Docker images to Dockerhub (https://hub.docker.co
The [Makefile](Makefile) contains some default variables that affect the version numbers of both the installed `kafka-connect-datagen` as well as the base Confluent Platform version. The variables are located near the top of the [Makefile](Makefile) with the following names and current default values:

```bash
CP_VERSION ?= 6.1.0
KAFKA_CONNECT_DATAGEN_VERSION ?= 0.4.0
CP_VERSION ?= 6.2.0
KAFKA_CONNECT_DATAGEN_VERSION ?= 0.5.0
OPERATOR_VERSION ?= 0 # Operator is a 'rev' version appended at the end of the CP version, like so: 5.5.0.0
```

Expand All @@ -250,11 +250,11 @@ make push-from-released

and to override the CP Version of the `kafka-connect-datagen` version you can run something similar to:
```bash
CP_VERSION=5.5.0 KAFKA_CONNECT_DATAGEN_VERSION=0.1.4 make publish-cp-kafka-connect-confluenthub
CP_VERSION=6.2.0 KAFKA_CONNECT_DATAGEN_VERSION=0.1.4 make publish-cp-kafka-connect-confluenthub
```

to override the CP Version and the Operator version, which may happen if Operator releases a patch version, you could run something similar to:
```bash
CP_VERSION=5.5.0 OPERATOR_VERSION=1 KAFKA_CONNECT_DATAGEN_VERSION=0.1.4 make push-cp-server-connect-operator-from-released
CP_VERSION=6.2.0 OPERATOR_VERSION=1 KAFKA_CONNECT_DATAGEN_VERSION=0.1.4 make push-cp-server-connect-operator-from-released
```
which would result in a docker image tagged as: `cp-server-connect-operator-datagen:0.1.4-5.5.0.1` and pushed to DockerHub
which would result in a docker image tagged as: `cp-server-connect-operator-datagen:0.1.4-6.2.0.1` and pushed to DockerHub