One question about rclone copy becoming slower

See here:500 VMs rclone copy to 1 VM, but the files are not completed

Each 10 minutes, crontab on the 1000 VMs will run rclone to copy one 48MB file to the same target VM. So there will be 1000 files running to the target VM at the same time. But after running maybe 1 hour, 10 rounds, the rclone jobs took more time to finish 1 round. I don't know why. Is it because there're more files on the target. Rclone needs more time during comparing and check before the real copy?

Thanks!

Are the files you are copying overwriting existing files? Are the new files exactly the same size as the old files? If so then rclone will need to checksum the new and old to make sure they are different before transferrring.

What is your rclone command line?

I'm using 'rclone copy -vv $source_file target:$dest_folder'.

If I clean up the target folder, the time is normal. Strange:(

What if you use --no-traverse does that help? It should do.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.