This repository contains a Cloud Function that handles GCP billing budget notifications, inspired by the official documentation, an example using the Secret Manager for storing the Slack bot token, and another example describing how to share the same Cloud Function instance across multiple projects.
Whenever the budget threshold for a project is reached, the Cloud Function disables billing for the project and posts a message to a Slack channel. The Cloud Function only needs to be installed once and it will handle Pub/Sub budget notifications for all projects.
-
Create a GCP project named for billing administration, e.g. called
billing-admin-290403
below. -
Enable the Cloud Billing API for the project.
-
Create a Pub/Sub topic called
budget-notifications
. -
Add a service account for running the Cloud Function and grant it Project Billing Manager and Browser roles at the organization level, to allow checking the current billing information and disabling billing for all projects.
-
Create a Slack app called
gcp-cost-control
with achat:write
scope bot token and install the app on your Slack workspace. -
Invite the bot to the channel that you want to receive messages on:
/invite @gcp-cost-control
-
Back in the
billing-admin-290403
GCP project, store the bot user OAuth access token in the Secret Manager as a secret using the nameslack-gcp-cost-control
. -
Grant the previously created service account access to the secret by granting the Secret Manager Secret Accessor role at the project level.
-
Deploy the Cloud Function, replacing
$BILLING_ADMIN_PROJECT
,$REGION
,$SERVICE_ACCOUNT
, and$SLACK_CHANNEL
accordingly:cd gcp_cost_control gcloud config set project $BILLING_ADMIN_PROJECT gcloud functions deploy gcp_cost_control --runtime python37 \ --region=$REGION \ --trigger-topic budget-notifications \ --service-account $SERVICE_ACCOUNT \ --set-env-vars SLACK_CHANNEL=$SLACK_CHANNEL
Create a separate budget for each project that you'd like to cap billing for:
- Go to "Billing".
- Go to "Budgets & Alerts".
- Click "Create Budget".
- Set the name of the budget identical to the project ID (not the project name!) of your project.
- Select your new project from the drop-down "Projects".
- In the "Amount" section, set a non-zero target amount.
- In the "Actions" section, select "Connect a Pub/Sub topic to this budget".
In the dropdown menu, select the topic
projects/billing-admin-290403/topics/budget-notifications
. If you can't see the topic, click on "Switch project" in that dropdown menu and selectbilling-admin-290403
, and you should be able to see the topic. - Click Finish.
To test the full setup, you can publish the following Pub/Sub message to the
budget-notifications
topic in the billing-admin-290403
project, replacing
$TEST_PROJECT
accordingly. However, make sure that it's not a problem to shut
down the whole project when billing gets disabled temporarily. If there are any
issues, check the logs for the gcp-cost-control
Cloud Function.
{
"budgetDisplayName": "$TEST_PROJECT",
"alertThresholdExceeded": 1.0,
"costAmount": 110.01,
"costIntervalStart": "2020-01-01T00:00:00Z",
"budgetAmount": 100.0,
"budgetAmountType": "SPECIFIED_AMOUNT",
"currencyCode": "USD"
}
The gcp_cost_report Cloud Function can be used to get a daily per-project cost report in Slack.
-
Set up Cloud Billing data export to BigQuery in the
billing-admin-290403
project. Replace$BIGQUERY_BILLING_TABLE
below with the corresponding table name, e.g.billing-admin-290403.billing.gcp_billing_export_v1_012345_ABCDEF_123456
. -
Grant the service account BigQuery Job User and BigQuery Data Viewer role permissions.
-
At the organization level, grant the service account Billing Viewer permissions. Replace
$BILLING_ACCOUNT_ID
below with your billing ID, e.g.01D123-234567-CBDEFA
. -
Create a new Pub/Sub topic in the
billing-admin-290403
project, namedcost-report
. -
Create a Cloud Scheduler job that posts a Pub/Sub message to the
cost-report
topic, e.g. using a daily schedule like0 9 * * *
. The payload can be abitrary, as it is ignored in the Cloud Function. -
Install the Cloud Function that gets triggered when a message to the
cost-report
Pub/Sub topic is posted. Set$QUERY_TIME_ZONE
to your local time zone, e.g.Australia/Sydney
.cd gcp_cost_report gcloud config set project $BILLING_ADMIN_PROJECT gcloud functions deploy gcp_cost_report --runtime python37 \ --region=$REGION \ --trigger-topic cost-report \ --service-account $SERVICE_ACCOUNT \ --set-env-vars SLACK_CHANNEL=$SLACK_CHANNEL \ --set-env-vars BIGQUERY_BILLING_TABLE=$BIGQUERY_BILLING_TABLE \ --set-env-vars QUERY_TIME_ZONE=$QUERY_TIME_ZONE \ --set-env-vars BILLING_ACCOUNT_ID=$BILLING_ACCOUNT_ID
To drill down on the recent cost incurred by a particular $PROJECT
, the
following query can be helpful:
SELECT
*
FROM
(
SELECT
FORMAT_TIMESTAMP("%F", export_time, "$QUERY_TIME_ZONE") as day,
service.description as service,
sku.description as sku,
ROUND(sum(cost), 2) as cost,
currency
FROM
`$BIGQUERY_BILLING_TABLE`
WHERE
_PARTITIONTIME >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 8 DAY)
AND project.id = "$PROJECT"
GROUP BY
day,
service,
sku,
currency
)
WHERE
cost > 0.1
ORDER BY
day DESC,
cost DESC;