Strange ETA reported

Help and Support
1

What is the problem you are having with rclone?

Haven't used rclone for a while, but just resumed as I have a faster internet connection.
To be honest I dont really need an answer, but just thought it was a bit odd that rclone sync reports a a strange ETA sometimes testing my new 1Gbps fibre. Its dropping about a year a second, down to 55y after a few mins so I should be good overall!

The other slightly interesting thing I've found with a fast internet connection is its actually sometimes faster for me to re-upload 300GB of data than let it move things at the remote end if there have been lots of changes. Moving directories etc seems to work at about 1 file/folder per second or even less.

Transferred: 14.919 GiB / 34.283 GiB, 44%, 2 B/s, ETA 251y7w5d3h4s999ms
Checks: 12970 / 12970, 100%
Renamed: 1
Transferred: 1001 / 5577, 18%
Elapsed time: 15m3.7s

Run the command 'rclone version' and share the full output of the command.

% rclone version

rclone v1.61.1

  • os/version: darwin 12.6.3 (64 bit)
  • os/kernel: 21.6.0 (x86_64)
  • os/type: darwin
  • os/arch: amd64
  • go/version: go1.19.4
  • go/linking: dynamic
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Dropbox (crypt)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync -v -PL --stats 2s --transfers 6 --checkers 10 --delete-during --tpslimit=12 --tpslimit-burst=10 --track-renames --track-renames-strategy modtime --backup-dir DB_rclone_crypt:/directory --filter-from rcloneFilters.txt / DB_rclone_crypt:/directory/

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

Paste  log here
2

The log file would give more details.

The ETA is that long because you are getting 2B/s so that's quite some time to move that data.

If you want to share a log file, that would be the best way to assist.

3

Thanks. The ETA isn't reflective of the time the transfer is taking, it finishes much sooner with the number of years dropping right down until near the end of the job (and well before my demise).
It copied 30GB in about 15 minutes, and 300GB in about 2.5hours last night, so definitely not 2B/s...
Will get a log over.

4

I think we may have fixed this problem in the latest beta - we've certainly done work in this areas so worth trying it.

5

Thanks, didn't really affect me, was just curious, but appreciate the update.

I actually zipped up (and then deleted) a load of very small files that I saw going slowly in the output. With those gone the job now completes significantly faster in any event.

I also removed them from the remote, as just moving them on the remote server (to the backup directory) was taking forever. Presumably this is just the remote server limitation? Seems like uploading (but also renaming and moving) lots of small files is very very slow.

6

It is. Dropbox has quite strict rate limits which you are using here --tpslimit=12. You could try increasing that number - that will make things faster, but then dropbox might tell you to slow down by giving short bans of 30s or 5minutes.

7

The logfile would show it slowed down though as it might have been stuck for a period, as the screenshot shows 2B/s which means the ETA would sky rocket. If the speed picked back up, it would return to a normal ETA I'd imagine as well.

The log file would be key as that would show exactly what was happening.