Fix pg_jsonschema benchmark

I had been using a development version with debugging code 🤦🏻. Many thanks to @olirice for the pointers for this Rust newb.
tembo-io · Apr 9, 2024 · cfda397 · cfda397
1 parent 4a803d5
commit cfda397
Show file tree

Hide file tree

Showing 3 changed files with 25 additions and 20 deletions.
diff --git a/README.md b/README.md
@@ -87,53 +87,53 @@ Benchmark
 
 A quick benchmark in [`eg/bench.sql](eg/bench.sql) compares the performance
 for a simple validation a check constraint between the jsonschema and
-[pg_jsonschema]. Example testing `pg_jsonschema` with PostgreSQL 16 on an M3
-Max MacBook Pro with 32G of RAM:
+[pg_jsonschema]. Example testing `jsonschema` with PostgreSQL 16 on an M3 Max
+MacBook Pro with 32G of RAM:
 
 ``` console
-$ psql -Xf eg/bench.sql --set extension=pg_jsonschema
+$ psql -f eg/bench.sql -X --set extension=jsonschema
 
 ######################################################################
-# Test pg_jsonschema JSON validation for 200_000 iterations
+# Test jsonschema JSON validation for 200_000 iterations
 ######################################################################
-Time: 8170.906 ms (00:08.171)
+Time: 2686.546 ms (00:02.687)
 
 ######################################################################
-# Test pg_jsonschema JSONB validation for 200_000 iterations
+# Test jsonschema JSONB validation for 200_000 iterations
 ######################################################################
-Time: 8215.921 ms (00:08.216)
+Time: 2643.178 ms (00:02.643)
 ```
 
-Testing `jsonschema`:
+Testing [pg_jsonschema]:
 
 ``` console
-$ psql -Xf eg/bench.sql --set extension=jsonschema
+$ psql -f eg/bench.sql -X --set extension=pg_jsonschema
 
 ######################################################################
-# Test jsonschema JSON validation for 200_000 iterations
+# Test pg_jsonschema JSON validation for 200_000 iterations
 ######################################################################
-Time: 3356.828 ms (00:03.357)
+Time: 1855.604 ms (00:01.856)
 
 ######################################################################
-# Test jsonschema JSONB validation for 200_000 iterations
+# Test pg_jsonschema JSONB validation for 200_000 iterations
 ######################################################################
-Time: 3428.245 ms (00:03.428)
+Time: 1834.598 ms (00:01.835)
 ```
 
 And a control test with no validation:
 
 ``` console
-❯ psql -Xf eg/bench.sql
+$ psql -f eg/bench.sql -X
 
 ######################################################################
 # Test without JSON validation for 200_000 iterations
 ######################################################################
-Time: 474.688 ms
+Time: 668.716 ms
 
 ######################################################################
 # Test without JSONB validation for 200_000 iterations
 ######################################################################
-Time: 462.265 ms
+Time: 741.202 ms
 ```
 
 Copyright and License

diff --git a/doc/jsonschema.md b/doc/jsonschema.md
@@ -69,9 +69,10 @@ the [prior art](#prior-art)?
     support simple, single-file JSON Schemas, this extension fully supports
     [multi-file schema definition][composition]. This allows multiple smaller
     schemas to be composed into larger, more complicated schemas.
-*   **Performance.** The [boon crate] provides the highest level of schema
-    validation performance. In a [simple test], the jsonschema validates JSON
-    and JSONB objects in a CHECK constraint 2.3x faster than [pg_jsonschema].
+*   **Performance.** The use of Rust provides the highest level of schema
+    validation performance. In a [simple test], the jsonschema extension
+    validates JSON and JSONB objects in a `CHECK` constraint at around 77,000
+    inserts/second.
 
 For many use cases these features aren't necessary. But once schemas become
 more complicated or require more advanced features of the [2020-12 draft],
@@ -413,7 +414,7 @@ Prior Art
 ---------
 
 *   [pg_jsonschema]: JSON Schema Postgres extension written with pgrx +
-    the [jsonschema crate]
+    the [jsonschema crate]; ca. 20-30% faster in a [simple test].
 *   [pgx_json_schema]: Slightly older JSON Schema Postgres extension written
     with pgrx + the [jsonschema crate]
 *   [postgres-json-schema]: JSON Schema Postgres extension written in PL/pgSQL

diff --git a/eg/bench.sql b/eg/bench.sql
@@ -8,6 +8,10 @@
 --
 --     psql -f eg/bench.sql --set extension=pg_jsonschema --iterations 200_000
 --
+-- And without validation (control):
+--
+--     psql -f eg/bench.sql --iterations 200_000
+--
 -- Borrowed from
 -- https://github.com/supabase/pg_jsonschema?tab=readme-ov-file#benchmark