Rclone sync S3 to S3 runs for hours and copy nothing

[kapitainsky]

And similar problem but with different underlying cause:

Rclone copy dry-run of big S3 bucket is taking super long time to start

Good idea. Let us know the outcome. It was worth a try It should be listing both source and destination at the same time. If the destination finishes much sooner maybe you haven't noticed that?

At the minimum you should try:

rclone sync prod-ceph:/docapi-fr01 prod-ovh:/docapi-fr01 --checksum --fast-list --s3-list-version 2 -vv

To see how talking between servers goes add --dump headers - maybe like here one of your servers is not very fast... listing 80 million objects will take time. One HTTP requests gets you 1000 objects - if you see that server does 10 per second it will take 2-3h.

Make sure you have enough RAM.... all listing goes into memory - about 1KB per object. So you need 20GB RAM (source and destination) to absorb it without swapping.