rClone speed and Sync feature

Hi, it’s my first time using rClone. currently backing up a local directory to Amazon Cloud Drive (total of 500GB) and I have two questions:

  1. Upload speed started at 6MB/s, kept dropping down slowly (within one hour) and now it’s 700KB/s. Reset my connection, stopped the sync and restarted it, still the same. Any idea why this is happening?

  2. If I quit the sync and one day later run the sync again for the same directories, would it continue where it left off last time, or it would re-sync all over again?

Thank you.

Some sort of throttling? Maybe Amazon or your ISP? It is worth trying to reboot your router too.

It will continue where it left off. Any files partially uploaded will need to be started again though.

Thank you for your reply. I doubt it’s my ISP, doing different upload tests, it’s giving me almost the full speed which is 10MB/s. Also not sure if it’s amazon, using the same file with different clients gives different results:
Uploading using chrome: 300KB/s. (starts at 300 and stays there then go down to 200KB/s)
Uploading using Amazon cloud drive client: between 250 to 350KB/s.
Uploading using GoodSync: between 100KBs to 150KBs.

I need to test it for a day or so and see what happens.

are you running the command with -v ? To see what is going on

1 Like

@habskilla Excuse my ignorance, I am a noob. what does “-v” do?
the current command I am running is: rclone sync [local directory] [online directory]

turns on verbose…might prove to be helpful trying to figure out why your speed is falling

@habskilla what’s verbose? and where should I place the “-v” within the command line I provided?

rclone -v sync [local directory] [online directory]

How are you measuring the speed?

Note that rclone reports speeds in bytes per second nor bits per second.

Rclone has some internal buffering so a quick spike of speed might be expected at the start.

@ncw Looks like it’s an ISP issue, the speed is going up again. One more question, though, how can I speed up the frequency of showing the files being uploaded, instead of every minute, to make it less, like every few seconds?

http://rclone.org/docs/#stats-time

@habskilla thanks for the help :slight_smile: One more question, is there a way to display the amount of size left to be transferred? Let’s say I am transferring 1TB, 700GB is up, and 300GB is left, is there a way to display the size left and the number of files as well?

I don’t know of a way for rclone to display that info…

It would be possible. clone would just have to read the input dir, maybe applying your filters or other flags, and coming up with how much/many there is to move before it starts. In fact, rclone has a size command I think that does just that. It just doesn’t run it before a sync/copy/move. I don’t think it would be too terribly hard to do. I might work on that in my fork down the road, I’ve got to find the time to do my current work first!!!
You can also just look at properties in your OS for the dir, it will tell you how big and how many files are in the dir. Then just do quick math in your head as it’s uploading.

It wouldn’t be 100% reliable. rclone often finds stuff when running it can’t move, skips stuff, has to retry stuff and encounters errors. So size and time could give funky results. But count should be real close. It should error, skip, or transfer each file. I think tracking that could work ok. The other metrics would be best guess, shots in the dark. Depending on what’s in the dir, and number of errors, non-movable/copyiable files. etc.

Nop, rclone does not even know file of the size its transferring until its completed.
But you could calculate how many files you have on dest and how many on source and output it at begining so you know how much left. ( assuming nothing its delete on dest )