Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

init sql runner app #1018

Merged
merged 11 commits into from
Oct 18, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dataplane-webserver/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ kube = { version = "0.90.0", features = ["runtime", "derive"] }
k8s-openapi = { version = "0.21.0", features = ["v1_25"] }
indexmap = "2.0.0"
regex = "1.9.6"
jsonwebtoken = "8.3.0"
jsonwebtoken = "8.3.0"
1 change: 1 addition & 0 deletions dataplane-webserver/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ impl Default for Config {
"PROMETHEUS_URL",
"http://monitoring-kube-prometheus-prometheus.monitoring.svc.cluster.local:9090",
),

prometheus_timeout_ms: match from_env_default("PROMETHEUS_TIMEOUT_MS", "500")
.parse::<i32>()
{
Expand Down
30 changes: 22 additions & 8 deletions tembo-operator/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,23 @@
# Contributing to the Tembo Kubernetes Operator

Welcome!
And thank you for your interest in contributing to the Tembo Kubernetes Operator.
Here are some quick pointers for orientation:

- Check out the project's [README](https://github.com/tembo-io/tembo/blob/main/tembo-operator/README.md) to learn about the less technical aspects.
- Questions or comments? We'd love to hear from you on our [Tembo Slack Channel](https://join.slack.com/t/tembocommunity/shared_invite/zt-277pu7chi-NHtvHWvLhHwyK0Y5Y6vTPw)!

## Table of Contents

1. [Prerequisites](#prerequisites)
2. [Running locally with Kind](#running-locally)
1. [Initial setup](#1.-initial-setup)
2. [Applying YAML files](#2.-applying-YAML-files)
3. [Loading Docker images](#3.-loading-docker-images)
4. [Connect via psql](#4.-connect-via-psql)
5. [Exec into the pod](#5.-exec-into-the-pod)
1. [Initial setup](#1.-initial-setup)
2. [Applying YAML files](#2.-applying-YAML-files)
3. [Loading Docker images](#3.-loading-docker-images)
4. [Connect via psql](#4.-connect-via-psql)
5. [Exec into the pod](#5.-exec-into-the-pod)
3. [Updating the CRD (CustomResourceDefinition)](#updating-the-crd)
1. [Making and applying changes](#1.-making-and-applying-changes)
1. [Making and applying changes](#1.-making-and-applying-changes)
4. [Testing](#testing)

## Prerequisites
Expand All @@ -25,6 +28,7 @@ Here are some quick pointers for orientation:
- [kind](https://github.com/kubernetes-sigs/kind) — Simplifies creation of local Kubernetes clusters using Docker (_**K**ubernetes **IN** **D**ocker_)
- [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) — Kubernetes primary CLI
- [just](https://github.com/casey/just) — Simplifies running complex, project-specific commands. If you find a new, useful command, consider adding it to the `justfile`
- [helm](https://helm.sh/) - The package manager for Kubernetes

## Running locally

Expand All @@ -35,22 +39,27 @@ If you haven't already, go ahead and clone the tembo repository to your local ma
```bash
git clone https://github.com/tembo-io/tembo.git
```

```bash
cd tembo/tembo-operator
```

From there, initiate a local Kubernetes cluster:

```bash
just start-kind
```

:bulb: Details on this command, as well as others that invoke `just` can be found within the directory's `justfile`.

:wrench: If you encounter an error, confirm that your Docker engine is running.

Once complete, start the Tembo Operator:

```bash
just run
```

:bulb: This operation will be running continuously, so we advise opening a new terminal workspace.

### 2. Applying YAML files
Expand All @@ -62,10 +71,13 @@ You can try out any of the sample YAML files, for example by running the followi
```bash
kubectl apply -f yaml/sample-standard.yaml
```

After some moments, confirm the newly-made kubernetes pod:

```bash
kubectl get pods
```

```text
NAME READY STATUS RESTARTS AGE
sample-standard-1 1/1 Running 0 14s
Expand Down Expand Up @@ -104,6 +116,7 @@ Bear this in mind as you define your own names and tags.
You can confirm that your local Docker registry is running by invoking `docker ps`.

Then you can push the newly-created image and check for it by running the following:

```bash
docker push localhost:5000/my-custom-image:15-0.0.1
```
Expand All @@ -112,7 +125,7 @@ docker push localhost:5000/my-custom-image:15-0.0.1
docker images
```

#### 3.3. Apply custom image and connecting
#### 3.3. Apply custom image and connecting

Say, for example you'd like to update the image of the `sample-standard.yaml` file.
You would begin by replacing the image parameter with the following:
Expand All @@ -136,7 +149,7 @@ kubectl apply -f yaml/sample-standard.yaml
### 4. Connect via psql

Connecting via psql will require a password, which is linked to your current Kubernetes session.
Sections `4.1` and `4.2` will illustrate how to respectively reveal the password, if you're work is more sessions-based, and how to save the password for later use.
Sections `4.1` and `4.2` will illustrate how to respectively reveal the password, if you're work is more sessions-based, and how to save the password for later use.

#### 4.1. Revealing password

Expand All @@ -161,6 +174,7 @@ export PGPASSWORD=$(kubectl get secrets/sample-standard-connection --template={{
```

Add the following line to /etc/hosts

```
127.0.0.1 sample-standard.localhost
```
Expand Down
31 changes: 31 additions & 0 deletions tembo-operator/yaml/sample-sql-runner.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
apiVersion: coredb.io/v1alpha1
kind: CoreDB
metadata:
name: sample-sql
spec:
appServices:
- image: quay.io/tembo/sql-runner:6778cb5
name: sqlrunner
resources:
requests:
cpu: 10m
memory: 100Mi
limits:
cpu: 200m
memory: 100Mi
routing:
- port: 3000
ingressPath: "/v1/sql"
middlewares:
- strip-auth-header
- strip-path-prefix
- replacePathRegex
middlewares:
- replacePathRegex:
name: map-sql
config:
regex: ^\/sql\/?
replacement: /v1/sql
env:
- name: CONNECTION_STRING
valueFromPlatform: ReadWriteConnection
2 changes: 1 addition & 1 deletion tembo-stacks/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion tembo-stacks/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]
name = "tembo-stacks"
description = "Tembo Stacks for Postgres"
version = "0.17.2"
version = "0.17.3"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are adding a completely new feature but still backwards compatible.

Suggested change
version = "0.17.3"
version = "0.18.0"

authors = ["tembo.io"]
edition = "2021"
license = "Apache-2.0"
Expand Down
9 changes: 9 additions & 0 deletions tembo-stacks/src/apps/app.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ lazy_static! {
serde_yaml::from_str(include_str!("embeddings.yaml")).expect("embeddings.yaml not found");
pub static ref PGANALYZE: App =
serde_yaml::from_str(include_str!("pganalyze.yaml")).expect("pganalyze.yaml not found");
pub static ref SQLRUNNER: App =
serde_yaml::from_str(include_str!("sql-runner.yaml")).expect("sql-runner.yaml not found");
}

// handling merging requirements coming from an App into the final
Expand Down Expand Up @@ -135,6 +137,13 @@ pub fn merge_app_reqs(
final_pg_configs.extend(pg_cfg);
}
}
AppType::SqlRunner(_) => {
// there is only 1 app_service in the restAPI
let sqlrunner = SQLRUNNER.clone().app_services.unwrap().clone()[0].clone();
user_app_services.push(sqlrunner);
// sqlrunner only has app_service containers
// no extensions or trunk installs
}
AppType::Custom(custom_app) => {
user_app_services.push(custom_app);
}
Expand Down
35 changes: 35 additions & 0 deletions tembo-stacks/src/apps/sql-runner.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: !sqlrunner
appServices:
- image: quay.io/tembo/sql-runner:6778cb5
name: sqlrunner
resources:
requests:
cpu: 10m
memory: 100Mi
limits:
cpu: 200m
memory: 100Mi
routing:
- port: 3000
ingressPath: "/v1/sql"
middlewares:
- strip-auth-header
- strip-path-prefix
- replacePathRegex
middlewares:
- !customRequestHeaders
name: strip-auth-header
config:
Authorization: ""
- !stripPrefix
name: strip-path-prefix
config:
- /v1/sql
- !replacePathRegex
name: map-sql
config:
regex: ^\/sql\/?
replacement: /v1/sql
env:
- name: CONNECTION_STRING
valueFromPlatform: ReadWriteConnection
3 changes: 3 additions & 0 deletions tembo-stacks/src/apps/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ pub enum AppType {
Embeddings(Option<AppConfig>),
#[serde(rename = "pganalyze")]
PgAnalyze(Option<AppConfig>),
#[serde(rename = "sqlrunner")]
SqlRunner(Option<AppConfig>),
#[serde(rename = "custom")]
Custom(AppService),
}
Expand Down Expand Up @@ -73,6 +75,7 @@ impl TryFrom<AppService> for AppType {
"mq-api" => Ok(AppType::MQ(app_config)),
"embeddings" => Ok(AppType::Embeddings(app_config)),
"pganalyze" => Ok(AppType::PgAnalyze(app_config)),
"sqlrunner" => Ok(AppType::SqlRunner(app_config)),
_ => {
// everything else is a custom app
Ok(AppType::Custom(app_service))
Expand Down
Loading