Rclone and b2 chunking - just spinning it's wheels

Help and Support
1

Hi all,

I am wanting to upload a few 100GB files to B2. I am facing a problem though in that it has been over 20 minutes now attempting to copy a file to B2 and “rclone copy” is still spinning at 100% CPU and doing nothing but showing 0% progress output on the screen.

I am assuming it is working on trying to chunk the file for the upload.

What would be the best flags to use to be able to successfully upload these files in a reasonable amount of time?

2

It finished, with a report of:

2017/09/20 12:41:33 DEBUG : cloud.tar: Done sending chunk 40
2017/09/20 12:42:16 INFO :
Transferred: 119.899 GBytes (14.305 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 2h23m2.8s
Transferring:

  •                                 cloud.tar: 142% done, 5.680 MBytes/s, ETA: 0s

2017/09/20 12:43:12 DEBUG : cloud.tar: Done sending chunk 41
2017/09/20 12:43:12 DEBUG : cloud.tar: Finishing large file upload
2017/09/20 12:43:13 INFO : cloud.tar: Copied (new)
2017/09/20 12:43:13 INFO :
Transferred: 120.056 GBytes (14.228 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 2h24m0.4s

142% done, transferred 120GB for an 85GB file. Looking at B2 web browser, it shows 90.3GB.

Please help me understand how it transferred 35GB of extra data. Thanks!

3

That is likely rclone having to retry the chunks, which also explains the weird stats.

You should see the retries in the logs with -vv - they will be low level retries