Rclone sync parameters

Help and Support
1

Hi,

I’m performing a synchronization of about 500 GB between an FTP and the destination of an S3 bucket.

The configuration of the command is that:

/usr/bin/rclone sync -uv --timeout=600s --ignore-existing --transfers=64 --checkers=64 --ignore-size --progress source: destiny:bucket-s3 --log-file /var/log/rclone/*.log

When I put parameter 64, I felt that the transfer rate increased considerably.

My question is when we are transferring a large number of files, so it is necessary to increase the transfer parameter and checkers for the synchronization to be faster?

Additionally, for a synchronization of approximately 10 TB, between FTP and S3 bucket, what parameters do you think can be synchronized in an accelerated way?

Thank you!

2

Increasing --transfers is likely to make the transfer quicker, yes, up to a certain point when you overload either the network or the FTP server.

How many files is this? If it is lots of small files you might want to consider --fast-list if you have lots of memory.

You might want to consider --size-only also.

1 Like