From da6f9e0c05e9edc14a9265cafa1f5c46dbf3e5c1 Mon Sep 17 00:00:00 2001 From: Abdul Awal Date: Tue, 25 Aug 2026 10:19:42 +0600 Subject: [PATCH 1/2] docs: Make file-upload storage setup the default path for S3, GCP, and R2 Give each provider a single setup path, document s3_compat as a building block, and correct the GCP HMAC region and preventOverwrite behavior. --- .../02-endpoints-and-apis/07-file-uploads.md | 132 ++++++++++-------- 1 file changed, 74 insertions(+), 58 deletions(-) diff --git a/docs/06-concepts/02-endpoints-and-apis/07-file-uploads.md b/docs/06-concepts/02-endpoints-and-apis/07-file-uploads.md index 30944b36..29995dfc 100644 --- a/docs/06-concepts/02-endpoints-and-apis/07-file-uploads.md +++ b/docs/06-concepts/02-endpoints-and-apis/07-file-uploads.md @@ -179,22 +179,27 @@ To delete a stored file, use `deleteFile` with the same `storageId` and `path`. ## Configure a storage provider -Each storage is identified by a `storageId`. Serverpod comes with two default storages, `public` and `private`. You can replace these with a cloud-backed implementation, or add additional storages with custom IDs. Register your cloud storage with `pod.addCloudStorage()` before starting the server. +Each storage is identified by a `storageId`. Serverpod comes with two default storages, `public` and `private`. Replace these with a cloud-backed implementation, or add additional storages with custom IDs. Call `pod.addCloudStorage()` before `pod.start()`. -The sections below give short descriptions on how to set up each provider. Consult the documentation of each cloud provider for more details on bucket creation, access policies, and credential management. +Pick the package that matches your provider. Use [serverpod_cloud_storage_s3](https://pub.dev/packages/serverpod_cloud_storage_s3) for AWS S3, [serverpod_cloud_storage_gcp](https://pub.dev/packages/serverpod_cloud_storage_gcp) for Google Cloud Storage, or [serverpod_cloud_storage_r2](https://pub.dev/packages/serverpod_cloud_storage_r2) for Cloudflare R2. -### GCP (HMAC) +### Configure Google Cloud Storage -Serverpod can use Google Cloud Storage's HMAC interoperability (S3-compatible) to handle file uploads to Google Cloud. To make file uploads work, you must make a few custom configurations in your Google Cloud console: +The default GCP path uses HMAC keys and the S3-compatible API. Create a [service account HMAC key](https://cloud.google.com/storage/docs/authentication/hmackeys), then: -1. Create a service account with the _Storage Admin_ role. -2. Under _Cloud Storage_ > _Settings_ > _Interoperability_, create a new HMAC key for your newly created service account. -3. Add the two keys you received in the previous step to your `config/passwords.yaml` file. The keys should be named `HMACAccessKeyId` and `HMACSecretKey`, respectively. You can also pass them in as environment variables. The environment variable names are `SERVERPOD_HMAC_ACCESS_KEY_ID` and `SERVERPOD_HMAC_SECRET_KEY`. -4. When creating a new bucket, set the _Access control_ to _Fine-grained_ and disable the _Prevent public access_ option. +1. Grant the service account the Storage Admin role. +2. Create an HMAC key for that account under **Cloud Storage** > **Settings** > **Interoperability**. +3. When you create the bucket, set **Access control** to **Fine-grained** and turn off **Prevent public access**. -You may also want to add the bucket as a backend for your load balancer to give it a custom domain name. +Add the HMAC keys to `config/passwords.yaml`, or pass them as `SERVERPOD_HMAC_ACCESS_KEY_ID` and `SERVERPOD_HMAC_SECRET_KEY`. -When you have set up your GCP bucket, you need to configure it in Serverpod. Add the GCP package to your `pubspec.yaml` file and import it in your `server.dart` file. +```yaml title="config/passwords.yaml" +shared: + HMACAccessKeyId: 'XXXXXXXXXXXXXX' + HMACSecretKey: 'XXXXXXXXXXXXXXXXXXXXXXXXXXX' +``` + +Add the package and import it in `server.dart`. ```bash dart pub add serverpod_cloud_storage_gcp @@ -205,7 +210,7 @@ import 'package:serverpod_cloud_storage_gcp/serverpod_cloud_storage_gcp.dart' as gcp; ``` -The `bucket` parameter is the GCP bucket name (find it in the console), and `publicHost` is the domain used to access the bucket over https when it sits behind a load balancer. +The `bucket` parameter is the GCP bucket name. Set `publicHost` if the bucket is reachable on a custom domain behind a load balancer. Use the bucket's GCS region, for example `us-central1`. ```dart pod.addCloudStorage( @@ -213,25 +218,18 @@ pod.addCloudStorage( serverpod: pod, storageId: 'public', public: true, - region: 'auto', + region: 'us-central1', bucket: 'my-bucket-name', publicHost: 'storage.myapp.com', ), ); ``` -### GCP (native) +### Use native Google Cloud Storage -As an alternative to the HMAC approach, you can use Google Cloud Storage's native JSON API with service account credentials. This provides full GCP feature support including custom metadata, conditional writes with `preventOverwrite`, and signed temporary download URLs. +As an alternative to HMAC keys, use Google Cloud Storage's native JSON API with a service account. This path supports custom metadata, conditional writes with `preventOverwrite`, and signed temporary download URLs. It lives in the same `serverpod_cloud_storage_gcp` package. -The native implementation is available from the same `serverpod_cloud_storage_gcp` package: - -```dart -import 'package:serverpod_cloud_storage_gcp/serverpod_cloud_storage_gcp.dart' - as gcp; -``` - -Since the factory constructors are asynchronous, create the storage before starting the pod: +The factory constructors are asynchronous, so create the storage before starting the pod: ```dart pod.addCloudStorage( @@ -245,27 +243,14 @@ pod.addCloudStorage( ); ``` -The `create` factory loads the service account JSON from `passwords.yaml` (key: `gcpServiceAccount`) or the environment variable `SERVERPOD_PASSWORD_gcpServiceAccount`. Add the service account JSON to your `passwords.yaml`: +The `create` factory loads the service account JSON from `passwords.yaml` (key: `gcpServiceAccount`) or the environment variable `SERVERPOD_PASSWORD_gcpServiceAccount`: -```yaml +```yaml title="config/passwords.yaml" shared: gcpServiceAccount: '{"type":"service_account","project_id":"...","private_key":"...",...}' ``` -If you prefer to pass the JSON directly, use `fromServiceAccountJson`: - -```dart -pod.addCloudStorage( - await gcp.NativeGoogleCloudStorage.fromServiceAccountJson( - storageId: 'public', - public: true, - bucket: 'my-bucket-name', - serviceAccountJson: myServiceAccountJson, - ), -); -``` - -For environments that support Application Default Credentials (e.g. Google Compute Engine, Cloud Run), use `fromApplicationDefaultCredentials`: +On Google Compute Engine or Cloud Run, use Application Default Credentials instead: ```dart pod.addCloudStorage( @@ -277,17 +262,23 @@ pod.addCloudStorage( ); ``` -:::info +:::note When using Application Default Credentials, the service account must have the `iam.serviceAccounts.signBlob` IAM permission to generate signed URLs. ::: -### AWS S3 +### Configure AWS S3 -This section shows how to set up a storage using S3. Before you write your Dart code, you need to set up an S3 bucket. Most likely, you will also want to set up a CloudFront for the bucket, where you can use a custom domain and your own SSL certificate. Finally, you will need to get a set of AWS access keys and add them to your Serverpod password file (`AWSAccessKeyId` and `AWSSecretKey`) or pass them in as environment variables (`SERVERPOD_AWS_ACCESS_KEY_ID` and `SERVERPOD_AWS_SECRET_KEY`). +Create an S3 bucket and an IAM user whose access is limited to that bucket. Put CloudFront in front of the bucket if you want a custom domain and TLS certificate. Add the access keys to `config/passwords.yaml`, or pass them as `SERVERPOD_AWS_ACCESS_KEY_ID` and `SERVERPOD_AWS_SECRET_KEY`. -When you are all set with the AWS setup, include the S3 package in your `pubspec.yaml` file and import it in your `server.dart` file. +```yaml title="config/passwords.yaml" +shared: + AWSAccessKeyId: 'XXXXXXXXXXXXXX' + AWSSecretKey: 'XXXXXXXXXXXXXXXXXXXXXXXXXXX' +``` + +Add the package and import it in `server.dart`. ```bash dart pub add serverpod_cloud_storage_s3 @@ -298,7 +289,7 @@ import 'package:serverpod_cloud_storage_s3/serverpod_cloud_storage_s3.dart' as s3; ``` -Set `publicHost` if your S3 bucket is accessible on a custom domain through CloudFront. +Set `publicHost` if the bucket is accessible on a custom domain through CloudFront. ```dart pod.addCloudStorage( @@ -313,19 +304,17 @@ pod.addCloudStorage( ); ``` -For your S3 configuration to work, you will also need to add your AWS credentials to the `passwords.yaml` file. Create the access keys in the AWS console for a dedicated IAM user whose access is limited to the bucket. Avoid root-user access keys. +### Configure Cloudflare R2 + +R2 is S3-compatible and uses presigned PUT uploads. Create [R2 API tokens](https://developers.cloudflare.com/r2/api/tokens/) in the Cloudflare dashboard, then add them to `config/passwords.yaml`, or pass them as `SERVERPOD_R2_ACCESS_KEY_ID` and `SERVERPOD_R2_SECRET_KEY`. -```yaml +```yaml title="config/passwords.yaml" shared: - AWSAccessKeyId: 'XXXXXXXXXXXXXX' - AWSSecretKey: 'XXXXXXXXXXXXXXXXXXXXXXXXXXX' + R2AccessKeyId: 'XXXXXXXXXXXXXX' + R2SecretKey: 'XXXXXXXXXXXXXXXXXXXXXXXXXXX' ``` -### Cloudflare R2 - -Serverpod supports Cloudflare R2 as a cloud storage provider. R2 is S3-compatible and uses presigned PUT uploads. - -Add the R2 package to your `pubspec.yaml` file and import it in your `server.dart` file. +Add the package and import it in `server.dart`. ```bash dart pub add serverpod_cloud_storage_r2 @@ -351,12 +340,39 @@ pod.addCloudStorage( ); ``` -Add your R2 API credentials to the `passwords.yaml` file. You can create access keys from the Cloudflare dashboard under _R2_ > _Manage R2 API Tokens_. +## Use a custom S3-compatible endpoint -```yaml -shared: - R2AccessKeyId: 'XXXXXXXXXXXXXX' - R2SecretKey: 'XXXXXXXXXXXXXXXXXXXXXXXXXXX' +The S3, GCP HMAC, and R2 packages are the ones to add by hand. They sit on [serverpod_cloud_storage_s3_compat](https://pub.dev/packages/serverpod_cloud_storage_s3_compat), which you can also use directly for another S3-compatible service such as LocalStack. + +```bash +dart pub add serverpod_cloud_storage_s3_compat +``` + +```dart +import 'package:serverpod_cloud_storage_s3_compat/serverpod_cloud_storage_s3_compat.dart'; ``` -You can also pass credentials via environment variables: `SERVERPOD_R2_ACCESS_KEY_ID` and `SERVERPOD_R2_SECRET_KEY`. +Pass credentials yourself. `CustomEndpointConfig` sets the base URI. `MultipartPostUploadStrategy` matches AWS-style POST uploads, for example LocalStack. Use `PresignedPutUploadStrategy` for providers that only accept presigned PUT, such as R2. + +```dart +pod.addCloudStorage( + S3CompatCloudStorage( + storageId: 'public', + public: true, + region: 'us-east-1', + bucket: 'my-bucket-name', + accessKey: pod.getPassword('MyAccessKeyId')!, + secretKey: pod.getPassword('MySecretKey')!, + endpoints: CustomEndpointConfig( + baseUri: Uri.http('localhost:4566', '/'), + serviceName: 'LocalStack', + ), + uploadStrategy: MultipartPostUploadStrategy(), + ), +); +``` + +## Related + +- [Configuration](../server-fundamentals/configuration): passwords file and environment variables for storage keys. +- [Sessions](./sessions): the `storage` member used in the examples above. From 7552dfc08729d9d14db0c9b644529454514cee2d Mon Sep 17 00:00:00 2001 From: Abdul Awal Date: Wed, 26 Aug 2026 19:55:05 +0600 Subject: [PATCH 2/2] docs: Address review feedback on file-upload storage setup Label the unrestricted path example, restore fromServiceAccountJson, and warn against root-user AWS keys. --- .../02-endpoints-and-apis/07-file-uploads.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/docs/06-concepts/02-endpoints-and-apis/07-file-uploads.md b/docs/06-concepts/02-endpoints-and-apis/07-file-uploads.md index 29995dfc..30a499cb 100644 --- a/docs/06-concepts/02-endpoints-and-apis/07-file-uploads.md +++ b/docs/06-concepts/02-endpoints-and-apis/07-file-uploads.md @@ -250,6 +250,19 @@ shared: gcpServiceAccount: '{"type":"service_account","project_id":"...","private_key":"...",...}' ``` +To pass the JSON directly instead, use `fromServiceAccountJson`: + +```dart +pod.addCloudStorage( + await gcp.NativeGoogleCloudStorage.fromServiceAccountJson( + storageId: 'public', + public: true, + bucket: 'my-bucket-name', + serviceAccountJson: myServiceAccountJson, + ), +); +``` + On Google Compute Engine or Cloud Run, use Application Default Credentials instead: ```dart @@ -270,7 +283,7 @@ When using Application Default Credentials, the service account must have the `i ### Configure AWS S3 -Create an S3 bucket and an IAM user whose access is limited to that bucket. Put CloudFront in front of the bucket if you want a custom domain and TLS certificate. Add the access keys to `config/passwords.yaml`, or pass them as `SERVERPOD_AWS_ACCESS_KEY_ID` and `SERVERPOD_AWS_SECRET_KEY`. +Create an S3 bucket and an IAM user whose access is limited to that bucket. Avoid root-user access keys. Put CloudFront in front of the bucket if you want a custom domain and TLS certificate. Add the access keys to `config/passwords.yaml`, or pass them as `SERVERPOD_AWS_ACCESS_KEY_ID` and `SERVERPOD_AWS_SECRET_KEY`. ```yaml title="config/passwords.yaml" shared: