diff --git a/.github/workflows/pages-deployment.yml b/.github/workflows/pages-deployment.yml new file mode 100644 index 0000000..a959f32 --- /dev/null +++ b/.github/workflows/pages-deployment.yml @@ -0,0 +1,16 @@ +name: github pages deployment +on: + push: + branches: + - main + - master +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + - uses: actions/setup-python@v2 + with: + python-version: 3.x + - run: pip install mkdocs-material + - run: mkdocs gh-deploy --force diff --git a/.gitignore b/.gitignore index 020c65b..f772b88 100644 --- a/.gitignore +++ b/.gitignore @@ -5,4 +5,5 @@ **/*.rs.bk .vscode/ META.json -/prometheus_fdw-* \ No newline at end of file +/prometheus_fdw-* +/site \ No newline at end of file diff --git a/docs/hello_world/basic_setup.md b/docs/hello_world/basic_setup.md new file mode 100644 index 0000000..b0d4144 --- /dev/null +++ b/docs/hello_world/basic_setup.md @@ -0,0 +1,6 @@ +# Prometheus FDW basic demo + +- **Dockerfile**: A Postgres database with prometheus_fdw installed +- **setup-prometheus-fdw.sql**: A script to show how to configure +- **sample-query.sql**: A sample query against Prometheus +- **run-demo.sh**: A script showing the whole process of running the demo diff --git a/docs/hello_world/practical_example/setup.md b/docs/hello_world/practical_example/setup.md new file mode 100644 index 0000000..93b441f --- /dev/null +++ b/docs/hello_world/practical_example/setup.md @@ -0,0 +1,43 @@ +# Practical example + +- **Dockerfile**: A Postgres database with prometheus_fdw installed +- **setup-prometheus-fdw.sql**: A script to show how to configure +- **setup-cache.sql**: Setup tables for local storage of metrics data +- **setup-metrics-sync.sql**: Automatically sync data from Prometheus to Postgres +- **run-demo.sh**: A script showing the whole process of running the demo + +## Data model + +This data model was inspired by the [Crunchy Data postgresql-prometheus-adapter](https://github.com/CrunchyData/postgresql-prometheus-adapter). + +**metric_labels**: Stores the metric name labels. + +``` + id | name | labels +----+------------------------------------+------------------------- + 1 | container_cpu_usage_seconds_total | {"pod": "my-pod-1", ...} + 2 | container_cpu_usage_seconds_total | {"pod": "my-pod-2", ...} + 3 | container_memory_working_set_bytes | {"pod": "my-pod-1", ...} + 4 | container_memory_working_set_bytes | {"pod": "my-pod-2", ...} +``` + +**metric_values**: A partitioned table that stores metric values, when they happened, and the corresponding labels. + +``` + label_id | time | value +----------+------------+---------- + 4320 | 1702678142 | 12214272 + 4320 | 1702678742 | 11923456 + 4320 | 1702679342 | 12230656 + 4320 | 1702679942 | 11804672 + 4320 | 1702677542 | 11870208 + 4331 | 1702679942 | 53743616 + 4331 | 1702678142 | 54022144 + 4331 | 1702678742 | 53903360 + 4331 | 1702679342 | 53288960 + 4331 | 1702677542 | 53514240 +``` + +## Example query + +The query in **sample-query.sql** is an example of showing the current memory utilization of each container in kube-system. diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..b45b4db --- /dev/null +++ b/docs/index.md @@ -0,0 +1,64 @@ +## Prometheus_fdw + +Prometheus_fdw is an integration of Prometheus monitoring data into Postgres. It enables querying for Prometheus metrics directly within Postgres, bridging the gap between Prometheus monitoring and Postgres's robust database capabilities. +[![Static Badge](https://img.shields.io/badge/%40tembo-community?logo=slack&label=slack)](https://join.slack.com/t/tembocommunity/shared_invite/zt-20dtnhcmo-pLNV7_Aobi50TdTLpfQ~EQ) +[![PGXN version](https://badge.fury.io/pg/prometheus_fdw.svg)](https://pgxn.org/dist/prometheus_fdw/) + +### Pre-requisistes + +- Install `prometheus_fdw` +- (Optional) install `pg_partman` and `pg_cron` + +### Quick start + +`create extension prometheus_fdw;` + +Create the foreign data wrapper: + +```sql +create foreign data wrapper prometheus_wrapper + handler prometheus_fdw_handler + validator prometheus_fdw_validator; +``` + +Create the server: + +```sql +create server my_prometheus_server + foreign data wrapper prometheus_wrapper + options ( + base_url ''); +``` + +Create Foreign Table: + +```sql +CREATE FOREIGN TABLE IF NOT EXISTS metrics ( + metric_name TEXT, + metric_labels JSONB, + metric_time BIGINT, + metric_value FLOAT8 + ) +server my_prometheus_server +options ( + object 'metrics', + step '10m' +); +``` + +## Queries + +To simply run the fdw and look at values + +```sql +SELECT + * +FROM metrics +WHERE + metric_name='container_cpu_usage_seconds_total' + AND metric_time > 1696046800 AND metric_time < 1696133000; +``` + +## Examples + +Please see the `examples/` directory to find a basic example and a practical example. In the practical example, metrics are automatically synced into the database using `pg_cron`, and automatically expired using `pg_partman`. Performance is optimized using indexes and partitioning. diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..b583d00 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,39 @@ +site_name: Prometheus FDW +theme: + name: material + palette: + - scheme: default + primary: teal + toggle: + icon: material/weather-night + name: dark mode + - scheme: slate + primary: teal + toggle: + icon: material/weather-sunny + name: light mode +nav: + - Prometheus FDW: 'index.md' + - hello world: + - 'hello_world/basic_setup.md' + - practical example: + - 'practical_example/setup.md' +markdown_extensions: +- toc: + permalink: true +- markdown.extensions.codehilite: + guess_lang: false +- codehilite: +- admonition +- extra +- pymdownx.snippets: + check_paths: true +- pymdownx.highlight: + anchor_linenums: true + line_spans: __span + pygments_lang_class: true +- pymdownx.inlinehilite +- pymdownx.superfences +plugins: + - search + - mkdocstrings \ No newline at end of file