Memory usage S3-alike to Glacier without big directories

[ncw (Nick Craig-Wood)]

5GB 99.61% 99.61% 5GB 99.61% github.com/rclone/rclone/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager.(*uploader.init.func1

This is the AWS SDK upload manager and it looks like this problem

Copy from local to s3 has high memory usage

What is the problem you are having with rclone? When copying a large local file (600+gb) to amazon s3 the rclone process uses a lot of memory, 2gb+. This makes it impossible to have multiple transfers going without hitting the OOM killer (at 3gb). What is your rclone version (output from rclone version) rclone v1.50.2 os/arch: freebsd/386 go version: go1.13.4 Which OS you are using and how many bits (eg Windows 7, 64 bit) FreeBSD 11.1-STABLE, 64-bit Which cloud storage system are you using…

Are you trying to transfer very big files? If so how big are they?

Setting --transfers 1 should help.