Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decodable GH actions example #23

Merged
merged 7 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ _Decodable provides a managed PyFlink service. Learn more [here](https://docs.de

| Example | Description |
|-------------------------------------------------------|-------------|
| [Decodable CI/CD](declarative-cicd) | An example of using Decodable with GitHub Actions|
| [Decodable CLI Docker image](cli-docker) | An example Dockerfile for running the Decodable CLI under Docker.|

## License
Expand Down
5 changes: 5 additions & 0 deletions declarative-cicd/README.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
= Using Decodable with GitHub Actions

The folder shows an example of how you can integrate Decodable with your CICD workflow using GitHub Actions.

Refer to https://docs.decodable.co/cli/integrate-with-github-actions.html[the documentation page] for details.
25 changes: 25 additions & 0 deletions declarative-cicd/connections/datagen-01.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
kind: connection
metadata:
name: datagen-01
description: ""
spec_version: v2
spec:
connector: datagen
type: source
stream_name: envoy
schema_v2:
fields:
- kind: physical
name: message
type: STRING
constraints: {}
properties:
delay: "5000"
format: json
data.type: envoy
execution:
active: true
task_count: 1
task_size: M

48 changes: 48 additions & 0 deletions declarative-cicd/example-workflow.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Deploy Decodable resources

on:
push:
branches:
- main
# pull_request:
workflow_dispatch:

jobs:
setup-and-apply:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v2

- name: Download and unpack the Decodable CLI
run: |
curl -L https://releases.decodable.co/decodable-cli/linux/amd64/decodable-cli-linux-amd64-1.20.0.tar.gz -o /tmp/decodable-cli.tar.gz && \
tar -xzf /tmp/decodable-cli.tar.gz -C /tmp && \
sudo mv /tmp/decodable-cli-linux-amd64-1.20.0/bin/decodable /usr/local/bin && \
rm -r /tmp/decodable-cli.tar.gz /tmp/decodable-cli-linux-amd64-1.20.0

- name: Create Decodable CLI auth file
run: |
mkdir -p ~/.decodable
cat <<EOF > ~/.decodable/auth
version: 1.0.0
tokens:
default:
refresh_token: ${{ secrets.DECODABLE_REFRESH_TOKEN }}
EOF

- name: Create Decodable CLI config file
run: |
mkdir -p ~/.decodable
cat <<EOF > ~/.decodable/config
version: 1.0.0
active-profile: default

profiles:
default:
account: ${{ vars.DECODABLE_ACCOUNT_NAME }}
EOF

- name: Apply all Decodable resources
run: decodable apply $(find declarative-cicd -name '*.yaml')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just out of curiosity: is there any specific reason for splitting up the resources into multiple files and then use find to collect them, over having everything in a single resource file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can imagine in real life people may well use files and folders as ways to organise projects and resources. It'd make git diff etc a lot easier, for one thing.

13 changes: 13 additions & 0 deletions declarative-cicd/pipelines/count-messages.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
kind: pipeline
metadata:
name: count-messages
description: ""
spec_version: v2
spec:
type: SQL
sql: INSERT INTO msg_count SELECT COUNT(*) AS ct FROM envoy
execution:
active: false
task_count: 1
task_size: M