Hello,
We using rclone 1.53.1-1 on centos 7 x64.
We have launch an huge rclone copy between 2 scality s3 rings, 900 000 000 billion objects for a size of 300 TB.
We have used the following command :
rclone copy source:bucket01 dest:bucket01 --progress --no-check-certificate=true --stats-one-line --transfers=160 --checkers=16 --max-backlog=100000 --use-mmap --log-file /root/bucket01.log
The process rclone have crashed one time (network issue) and we have launch another time the command to restart the copy.
For several days, the progress of the copy has been frozen and the rclone process appears to rescan all the copied objects.
218.856T / 218.856 TBytes, 100%, 118.517 MBytes/s, ETA 0s (chk#565409128/565509158)
I have running an rclone size on destination bucket and the size confirm the copy of all the data.
It is normal to have a check of all objects after the copy of data ?
If a missing object is detected what is the normal behaviour of the process ?
It is possible to avoid this final rescan ?
Thanks for your answers.