Rclone does not seem to use the move_batch_v2 API command for Dropbox. At least when copying from one crypt to another.
Instead it seems to try to move each individual file, resulting in rate limiting and lots of retries due to too_many_write_operations. Without batch renaming/moving, the rate limit is around 1 operation/second. Enabling it could provide a 100x speed increase.
I'm not sure if it's possible to use this command, but it seems like it should be.
The use case here is re-encrypting existing files to a different format/using a different key.
This happens when using my own app ID too.
- os/version: centos 7.9.2009 (64 bit)
- os/kernel: 3.10.0-1160.42.2.el7.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.6
- go/linking: static
- go/tags: none
Dropbox + Crypt
rclone copy dropbox-crypt-1: dropbox-crypt-2: ----delete-empty-src-dirs --server-side-across-configs --transfers=5 --dropbox-batch-mode async --dropbox-batch-size 1000 --dropbox-batch-timeout 1s --checkers 1 --tpslimit 12 --tpslimit-burst 0
[dropbox] type = dropbox token = *** [dropbox-crypt-1] type = crypt remote = dropbox:foldername password = *** password2 = *** [dropbox-crypt-2] type = crypt remote = dropbox:foldername2 filename_encoding = base32768 password = *** password2 = ***
The only relevant part of the log is a lot of:
2023-07-29 07:16:47 DEBUG : file.ext: Need to transfer - File not found at Destination 2023-07-29 07:16:47 DEBUG : pacer: low level retry 1/10 (error too_many_write_operations/) 2023-07-29 07:16:47 DEBUG : pacer: Rate limited, increasing sleep to 1.062334034s 2023-07-29 07:16:48 DEBUG : pacer: Reducing sleep to 796.750525ms 2023-07-29 07:16:48 INFO : file.ext: Moved (server-side)