Speed up copy/moves and/or multiple actions at once?

I am trying to speedup the rclone back end of my two-way file sync tool PyFiSync.

The transfers are pretty fast because I tell rclone to sync the identified files with --files-from so it can do many opperations at once. The slow part for me is moving and deleting files.

Before my tool does a sync, it backs up files that will be overwritten (by copy) or "deletes" them with backup (move).

I am aware of --backup-dir but for compatability, simplicty, and control, I want to specify eveything as a move by myself.

The problem is, each move command is its own rclone invocation and rclone has to list a bunch of the files to do it. So it can be slow! I know I am sort-of abusing rclone this way and it's reasonable that it is slow. But I am trying to speed it up if I can. I know the files exist and I do not need to list them. I basically just want to do the copy.

Looking at the documenation, I am not sure if --no-traverse is what I want. I suspect it isn't. --no-check-dest seems promising but in my (admitedly quick check with --dump-headers), it is still doing all of the listings.

Is this possible? If not, it isn't critical. It just uses more API calls and is slower.

Alternativly, is there a way to specify multiple actions at once?


This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.