I am trying to speedup the rclone back end of my two-way file sync tool PyFiSync.
The transfers are pretty fast because I tell rclone to sync the identified files with --files-from
so it can do many opperations at once. The slow part for me is moving and deleting files.
Before my tool does a sync, it backs up files that will be overwritten (by copy
) or "deletes" them with backup (move
).
I am aware of --backup-dir
but for compatability, simplicty, and control, I want to specify eveything as a move by myself.
The problem is, each move command is its own rclone invocation and rclone has to list a bunch of the files to do it. So it can be slow! I know I am sort-of abusing rclone this way and it's reasonable that it is slow. But I am trying to speed it up if I can. I know the files exist and I do not need to list them. I basically just want to do the copy.
Looking at the documenation, I am not sure if --no-traverse
is what I want. I suspect it isn't. --no-check-dest
seems promising but in my (admitedly quick check with --dump-headers
), it is still doing all of the listings.
Is this possible? If not, it isn't critical. It just uses more API calls and is slower.
Alternativly, is there a way to specify multiple actions at once?
Thanks