I am transferring everything from dropbox into onedrive so for that I spooled up a micro instance of ubuntu on google cloud. It's fast going between 12 and 23MB/s but when it's chunking through small files it tanks severely so I tried --transfers = 10 and rclone crashed.
Gcloud doesn't have a limit to the number of connections that I know of so I figured it was onedrive or dropbox crashing rclone.
Do you folks know that limit? What parameters do you use?
My current working parameters are
rclone --drive-chunk-size=256M --fast-list sync -v drop:/transfer one:/TRANSFER_from_Dropbox
Crashed how? Run out of memory would be my suspicion.
Neither rclone nor drive nor onedrive have a limit on connections as far as I know.
How do I know if it's running out of memory ?
It would crash with an out of memory error 
so the log would say "Error: out of memory"?
It would crash with an out of memory error and it looks like something in this thread:
If you can confirm you are running out of memory then you'd probably fix it easily enough by setting buffers to be reasonable to your current resources. Overly large buffers really aren't needed anyway.
--drive-chunk-size=256M
This probably isn't a great idea (on a resource limited VM), because with that large upload chunks each transfer can eat up to that much memory (for large files). Yes it is faster to use larger upload chunks on large files but it won't do anything on small files. Might want to look at how much memory you actually have available and then set this accordingly to something more reasonable.
I'm not 100% sure if this is actually used in-RAM for remote-to-remote transfers though. It definitely is in local->remote so I would assume so.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.