What is the problem you are having with rclone?
I need to set up synchronization between two object storages in different locations, syncing 7 million objects from one bucket to another, with hash comparison. The synchronization needs to run every 4 hours. I'm struggling to find options that allow the process to complete within 2 hours. I've used transfers and checkers set to 900, as well as the fast-list option. On average, synchronization takes 4 hours, sometimes even longer. During synchronization, less than 10,000 objects are copied at a time, and the volume is under 5GB. Please suggest what options I could try adding to speed up the process.
Run the command 'rclone version' and share the full output of the command.
rclone v1.73.1
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-60-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.25.7
- go/linking: static
- go/tags: none
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone sync src:backet dst:backet -vv --checksum --transfers=900 --checkers=900 --fast-list --log-file /logs/$(date -I).rclone.log
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[dst]
type = s3
provider = AWS
access_key_id = XXX
secret_access_key = XXX
endpoint = dst
acl = private
[src]
type = s3
provider = AWS
access_key_id = XXX
secret_access_key = XXX
endpoint = src
acl = authenticated-read
### Double check the config for sensitive info before posting publicly
A log from the command that you were trying to run with the -vv flag
2026/03/04 12:03:32 INFO :
Transferred: 560.641 MiB / 560.641 MiB, 100%, 405.887 KiB/s, ETA 0s
Errors: 60 (retrying may help)
Checks: 22013856 / 22013856, 100%, Listed 44078563
Transferred: 6142 / 6142, 100%
Elapsed time: 4h52m28.3s
2026/03/04 12:03:32 DEBUG : 657 go routines active