Delay before transfer big file with small --transfers

Help and Support
1

What is the problem you are having with rclone?

I have some very big files to s3, due to my s3 service limit, I have to use as small --transfers number as possible, for now I use 1. My obersavation is it takes very long time before any actually network transfer.

By reading Long starting delay (reading data) · Issue #4258 · rclone/rclone · GitHub I understand that this is b/c md5sum before multipart transfer.

My question is, is it possible to compute md5 in checker, so that the md5 computing can be pipeline with the transferring in my scenario?

Run the command 'rclone version' and share the full output of the command.

rclone version
rclone v1.74.3

  • os/version: ubuntu 24.04 (64 bit)
  • os/kernel: 7.0.0-31-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.26.4
  • go/linking: static
  • go/tags: none
    Which cloud storage system are you using? (eg Google Drive)

2

yeah thats the pre-upload md5 pass rclone does on big s3 multipart. transfers=1 doesnt change it, it still hashes the whole file b4 any bytes go out.

for me --s3-disable-checksum kills that stall. upload starts right away. you lose the X-Amz-Meta-Md5chksum on the object tho, per-part checksums still happen.

rclone copy /path s3remote:bucket --transfers 1 --s3-disable-checksum -P

pipelining that md5 into a separate checker isnt a thing yet afaik. if u need a hash record, make ur own sidecar after.