I'm trying to copy files but it's so slow. What's taking so long? How to decrease elapsed time? What's factors of elapsed time?
rclone -v copy ydisk:/xyz gdrive:/
Output:
Transferred: 2.417 MB
Elapsed Time: 20.4s
May I sugest that you read/follow the thread where I am working on it, and getting some help.
There are some steps that I have taken, and you can see if they help, and maybe together, with outside help, we can all get this working. See especially the section on adding to the config file.
Search is your friend.
1 Like
How big are the files you are transferring? Please be aware that small files will always have problems transferring fast because on Gdrive there is a maximum of 2-3 files you can open pr second - so on small files this will be the limitation and not your bandwidth.
I recommend that when you speed-test you use a large file (at least 50M, or preferably bigger). Then you will get an idea of the real speed that is possible.
Gdrive will be able to ingest about 45MB/sec pr transfer (4 transfers are default), so this is not a limitation you have to worry about 
You can start by adding these general optimizations (the first one is the most important here most likely):
-
--drive-chunk-size 64M (or 128M if you can afford up to 512M RAM total for rclone).
This will help a lot for uploads, but only for larger files. On large files it may as much as double effective transfer speeds. - --max-backlog 999999 if you transfer many file and you are confused by the "remaining files" keeps going up, this will fix that (while using a tiny bit more memory). This will force rclone to count all the files in the transfer at the start instead of only counting up to about 10.000. It not make anything go faster - but it will let you be able to see a better estimate IF you are transferring many thousands of files.
- --fast-list This will make listings (before the transfer) much faster on Gdrive, although it won't really help in the transfer speeds it may still save you time.
have you set up your own API key? If not, then this is recommended for optimal performance, because otherwise you may be competing with API resources to some extent with other rclone users who are using the default API key. See information here:
https://rclone.org/drive/#making-your-own-client-id
Lastly, do you see any ERROR messages from rclone? If you want to be extra thorough and give me the best possible chance of finding the problem, you can make a DEBUG log for me.
You would do that by adding the following flags:
--log-level DEBUG
--log-file="MyDebugLog.txt"
then, go make the problem happen. When you think you have demonstrated the problem - close the mount and remove the flags. Post the debug log here or PM it to me if you prefer. It should not contain much sensitive information - but it can potentially show the names of some of the files and folders.
If the log is very large you may need to post it to pastebin and give a link instead. Debug logs grow very very large quickly...
2 Likes
I tried to copy a video this command(Yandex Disk to DropBox - Size of file = 157 MB):
rclone copy ydisk:/video.mp4 --fast-list --max-backlog=999999 --drive-chunk-size=512M --transfers=45 --checkers=45 --buffer-size=75M --verbose drop:/
Output:
When I execute this command:
rclone copy ydisk:/video.mp4 drop:/
Output :
Elapsed time : 6m32s
That sounds a lot better at least.
It's still not very fast, but maybe you don't have such good upload speed?
What I see here equates to about 3Mbit/sec, so maybe an ADSL connection or something?
I would be very careful about using that many transfers and checkers. Some very high-performance backends can take this, but Dropbox is decidedly not high-performance. I suspect this will just make things go slower. It wouldn't matter in this experiment because it was just 1 file, but with more files the Dropbox API is likely to give you a ton of errors and just bog the process down. I recommend leaveing these two at default values unless you have a really good reason not to and you understand all the ramifications of changing them. Higher numbers are not necessarily better... sometimes it is even counterproductive.
1 Like
Thank you so much for help. I will try again and keep you posted when I go live. Because I was using local thus far. Thx again!
I tried this command:
rclone copy drop:/videolar --drive-chunk-size=512M --max-backlog=999999 --fast-list --transfers=2 --buffer-size=75M --verbose ydisk:/videolar
Output:
Transferred : 1.06 GB
Transferred : 11/11, 100%
Elapsed time : 59m44s
I tried several transfers and checkers value but i can't decrease elapsed time 
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.