Uploads result in CANCELED

[Cyberes]

I've been working to determine why large file uploads fail in Element. I'm not going to re-post everything, but here's the Synapse debugging. matrix-org/synapse#15841

I am attempting to upload two video files (each about 30 MB) to a room at the same time. This always fails with a generic error message popup (The file '[filename]' failed to upload.). There is no error in the console and the network of the debug window shows the fetch request as (canceled). Synapse logs don't show any errors and neither does Openresty. Uploading the files one at a time works, but uploading them together always fails.

• My upload limit in Synapse and Openresty (reverse proxy) is 40M. This has been an issue for a while and I have not discovered the root cause yet.
• My client is desktop and web Element on Ubuntu.
• I have a media worker with an S3 storage provider setup (see config below).

media_store_path:                                 "/mnt/matrix-storage/media"

media_storage_providers:
- module:                                         s3_storage_provider.S3StorageProviderBackend
  store_local:                                    True
  store_remote:                                   True
  store_synchronous:                              True
  config:
      bucket:                                     matrix-synapse
      endpoint_url:                               'https://xxx.r2.cloudflarestorage.com/matrix-synapse'
      access_key_id:                              'xxx'
      secret_access_key:                          'xxx'

When I comment out the media_storage_providers block, uploads are able to complete. I tried setting store_synchronous to False. I'm also using a media worker and disabling that didn't change anything.

The Synapse server's network doesn't have the fastest uplink and I'm suspicious that the upload to the S3 bucket is taking too long and the client gives up. Is there a way to have Synapse do the upload to S3 in the background?

[Cyberes]

Actually it looks like it isn't related to the S3 storage provider because uploads are randomly failing.