How does rclone stagger file manifesting and stagger transfers?

Taken from the Documentation

"When doing anything which involves a directory listing (e.g. sync , copy , ls - in fact nearly every command), rclone normally lists a directory and processes it before using more directory lists to process any subdirectories. This can be parallelised and works very quickly using the least amount of memory."

So is my understanding correct that if rclone is currently transferring from sub1:
top/
├── sub1
└── sub2
sub2 (depending on parallelism) may not have been listed yet so a file removal from sub2 may not induce a transfer failure (or be detected)?

But if I have a flat structure:
archive/
├── archive1.tgz
├── archive2.tgz
└── archive3.tgz

If archive has been listed, then any file removal after that but before/during transfer will result in a transfer failure?

Yes you are right. Operations are not instant and there is no locking mechanism in cloud storage. If you run two different rclone operations on the same remote (sync and delete for example) results will be unpredictable.

But rclone does not have any mechanism to prevent you from doing it.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.