Downloading a file with TransferUtility fails if path contains special characters

[olesmartyniuk]

Describe the bug

We download files from S3 buckets with the TransferUtility class. It worked fine until we decided to upgrade the AWSSDK.S3 to version 3.7.101.7. Now in cases when a file path contains special characters (for example '¢' or '€') the downloading fails with the error:

Amazon.S3.AmazonS3Exception: The request signature we calculated does not match the signature you provided. Check your key and signing method.

Expected Behavior

The file should be downloaded successfully.

Current Behavior

Exception is raised:

 ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception, Stream responseStream)    
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext executionContext, HttpErrorResponseException exception)
   at Amazon.Runtime.Internal.ExceptionHandler`1.HandleAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.ProcessExceptionAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Transfer.Internal.DownloadCommand.ExecuteAsync(CancellationToken cancellationToken)
   at Program.<Main>$(String[] args) in C:\Cloud\Work\S3DownloadIssue\Program.cs:line 24
   at Program.<Main>(String[] args)

Reproduction Steps

1. On S3 bucket create a folder that contains pound symbol, for example LongFolder_1234567890£
2. Put some file with name RemoteFile.txt to this folder
3. Create console application with dotnet new console and reference AWSSDK.S3 package dotnet add package AWSSDK.S3
4. Put this code into Program.cs file and run the app.

using Amazon;
using Amazon.Runtime;
using Amazon.S3;
using Amazon.S3.Transfer;

var region = RegionEndpoint.GetBySystemName("us-east-2");

var credentials = new BasicAWSCredentials(
    "***",
    "***");

var client = new AmazonS3Client(credentials, region);

var fileTransferUtility = new TransferUtility(client);

var request = new TransferUtilityDownloadRequest
{
    BucketName = "LongFolder_1234567890£",
    FilePath = @"c:\Temp\LocalFile.txt",
    Key = "RemoteFile.txt"
};

await fileTransferUtility.DownloadAsync(request);

System.Console.WriteLine("File is downloaded successfully");

Possible Solution

No response

Additional Information/Context

Looks like this bug was introduced between versions 3.7.10.1 and 3.7.100

AWS .NET SDK and/or Package version used

AWSSDK.S3 3.7.101.7

Targeted .NET Platform

.NET 6

Operating System and version

Windows 11

[ashishdhingra]

Hi @olesmartyniuk,

Good morning.

I'm curious if you are able to create bucket name with special characters from AWS Console. I tried creating bucket with name TestBucket_1234567890£ and it gave me error Bucket name must not contain uppercase characters Bucket name contains invalid characters: _, £.

I tried creating bucket with this name since you are specifying LongFolder_1234567890£ in BucketName property.

Using your scenario, I was able to create LongFolder_1234567890£ folder though and then uploaded a file to the folder. Then using the below code (used AWSSDK.S3 version 3.7.101.7) downloads the file successfully:

using Amazon;
using Amazon.Runtime;
using Amazon.S3;
using Amazon.S3.Transfer;

var region = RegionEndpoint.GetBySystemName("us-east-2");

var client = new AmazonS3Client(region);

var fileTransferUtility = new TransferUtility(client);

var request = new TransferUtilityDownloadRequest
{
    BucketName = "testbucket-issue1880",
    FilePath = @"/tmp/LocalFile.png",
    Key = "LongFolder_1234567890£/Screen Shot 2022-10-07 at 12.35.04 PM.png"
};

await fileTransferUtility.DownloadAsync(request);

System.Console.WriteLine("File is downloaded successfully");

So you are using wrong bucket name, instead you should be using folder name as part of the Key.

Thanks,
Ashish

[olesmartyniuk]

You are right. I used the directory name as a part of the BucketName property not as a part of the Key property. I moved the directory to the Key property and everything works fine now. It worked before but now I need to change this logic a bit.

Thank you for your help!

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.