Unable to transfer the data due to high memory usage, Getting killed

one option is to split the list of files in parts and run rclone multiple times, using filters.

another option that has been used a number of times.
How to sync S3 with millions of files at root - #2 by asdffdsa
Rclone sync S3 to S3 runs for hours and copy nothing - #23 by ncw

1 Like