Trino can be configured to load resource groups from a relational database. In order to create resource groups with this setup, an administrator needs to manually insert data in a set of database tables.
This tool exists to provide a CLI which can take a JSON file with resource group definitions and create the necessary records in the tables read by Trino. The configuration is reloaded from the database every second by Trino, and the changes are reflected automatically for incoming queries.
Trino supports MySQL, PostgreSQL, and Oracle as the database for storing resource group configurations.
mvn clean package
This will produce an executable JAR file in the target directory, such as target/trino-db-resource-groups-cli-1.0-SNAPSHOT-executable.jar. This file can be moved and renamed to be whatever you like.
$ cp target/trino-db-resource-groups-cli-1.0-SNAPSHOT-executable.jar /tmp/trino-db-resource-groups-cli
$ /tmp/trino-db-resource-groups-cli
Missing required subcommand
Usage: trino-db-resource-groups-cli [-h] [COMMAND]
-h, --help Show this help message and exit
Commands:
create_resource_groups
list_resource_groups
$
Both commands require the following 2 parameters:
db-config- a properties file with the database configurationenvironment- the Trino cluster environment for the resource groups. This should match the value in thenode.propertiesfile of the cluster you want to change the resource groups for.
The db-config parameter takes a properties file that looks like:
resource-groups.config-db-url=jdbc:mysql://localhost:3306/resource_groups
resource-groups.config-db-user=trino
resource-groups.config-db-password=trino
Any of the properties in this config file can be a Trino secret. For example:
resource-groups.config-db-url=jdbc:mysql://localhost:3306/resource_groups
resource-groups.config-db-user=trino
resource-groups.config-db-password=${ENV:MYSQL_PASS}
trino-db-resource-groups-cli create_resource_groups -h
Usage: trino-db-resource-groups-cli create_resource_groups [-h] --db-config=<configFilename> [--environment=<environment>]
--resource-groups-json=<resourceGroupsSchema>
--db-config=<configFilename>
Properties file with resource groups database config
--environment=<environment>
Environment where resource groups will be used (matches environment in node.properties)
-h, --help Show this help message and exit
--resource-groups-json=<resourceGroupsSchema>
JSON file with resource groups schema to load
The input JSON file will be validated to ensure the resource group definitions are correct before inserting anything into the database.
When running this command and everything is successful, you will see output like:
$ trino-db-resource-groups-cli create_resource_groups --db-config=resource-groups.properties --resource-groups-json=simple.json --environment=test
2022-02-03T15:48:29.847-0500 INFO main io.airlift.log.Logging Logging to stderr
2022-02-03T15:48:29.849-0500 INFO main Bootstrap Loading configuration
2022-02-03T15:48:30.062-0500 INFO main Bootstrap Initializing logging
2022-02-03T15:48:30.325-0500 INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION
2022-02-03T15:48:30.326-0500 INFO main Bootstrap resource-groups.config-db-password [REDACTED] [REDACTED] Database password
2022-02-03T15:48:30.326-0500 INFO main Bootstrap resource-groups.config-db-url ---- jdbc:mysql://localhost:3306/resource_groups
2022-02-03T15:48:30.326-0500 INFO main Bootstrap resource-groups.config-db-user ---- trino Database user name
2022-02-03T15:48:30.326-0500 INFO main Bootstrap resource-groups.exact-match-selector-enabled false false
2022-02-03T15:48:30.326-0500 INFO main Bootstrap resource-groups.max-refresh-interval 1.00h 1.00h Time period for which the cluster will continue to accept queries after refresh failures cause configuration to become stale
2022-02-03T15:48:30.519-0500 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle starting...
2022-02-03T15:48:30.520-0500 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle started
2022-02-03T15:48:30.520-0500 INFO main io.trino.resourcegroups.db.CreateResourceGroupsCommand Environment to update resource groups for: test
2022-02-03T15:48:30.520-0500 INFO main io.trino.resourcegroups.db.CreateResourceGroupsCommand Input JSON file: simple.json
2022-02-03T15:48:31.577-0500 INFO main io.trino.resourcegroups.db.CreateResourceGroupsCommand Resource groups created successfully
2022-02-03T15:48:31.577-0500 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle stopping...
2022-02-03T15:48:31.578-0500 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle stopped
$ trino-db-resource-groups-cli list_resource_groups --help
Usage: trino-db-resource-groups-cli list_resource_groups [-h] --db-config=<configFilename> [--environment=<environment>]
--output-json-file=<outputJsonFile>
--db-config=<configFilename>
Properties file with resource groups database config
--environment=<environment>
Environment where resource groups will be retrieved from (matches environment in node.properties)
-h, --help Show this help message and exit
--output-json-file=<outputJsonFile>
Path to write JSON file with resource groups schema
$
This command will read the current resource groups from the database and write them to a JSON file at the specified path.
This command has no interaction with the resource groups database. It simple validates a JSON file with a resource group schema.
$ trino-db-resource-groups-cli validate_resource_groups_json --help
Usage: trino-db-resource-groups-cli validate_resource_groups_json [-h] --resource-groups-json=<resourceGroupsSchema>
-h, --help Show this help message and exit
--resource-groups-json=<resourceGroupsSchema>
JSON file with resource groups schema to validate
$