Rclone newbie can't get files to transfer to B2


#1

I’m new to rclone and CLI programs in general. Think I got the config figured out to do a sync to my B2 acct. I was able to copy files earlier.

However, now I’m trying to set up a sync between my source on my Windows PC and my account on B2.

I’m trying to sync my files and subfolder in this folder:

F:/VeeamBackup/

to my B2 account

so I’m using this line for rclone:

rclone sync F:/VeeamBackup/ B2:[my bucket name] -P

Then it seems to run, and I see that that it’s trying to transfer one of the two files in a subfolder, as it should, but it hangs and the progress stays at 0% and just the time elapses.

When I check on my B2 acct, it just shows that the subfolder name has transferred, but none of the 2 files in that subfolder have transferred.

With the rclone sync command I have listed above, shouldn’t it transfer everything under the VeeamBackup folder? So it should transfer the subfolder and all the files in it, right?

Any ideas on what I’m doing wrong?


#2

Are they big files? You can run with -vv on the options and you can see what it is doing. I think B2 has to generate checksums on files so maybe that it?

The -vv should shed some light on what it’s doing though as your command is right.


#3

Yeah, one of the files is a big backup file: a little over 200GB.

I’ll try what you suggested.

Thanks!


#4

Do I leave the -P in there and just add -vv at the end?


#5

Yep. If it’s a 200Gb file, I’m like 99% sure it’s the checksum that’s happening.


#6

Thx. What does that mean and is there a way to fix it?


#7

Am trying what you suggested. What lines do I look at on the screen? The last INFO line says “waiting for transfers to finish” but it still is staying at 0% transferred.


#8

I personally don’t use B2 but I would think the checksums validate the file is the good on upload.

You can turn them off via:

https://rclone.org/b2/#b2-disable-checksum


#9

Thanks, very much! I added that disable checksum command and now it seems to be transferring. I appreciate the help!


#10

Hmmm, it was working for 12 minutes, then abruptly ended. I restarted and if it doesn’t work, I wonder if I’ll have to add the “upload cutoff” and “chunk size” commands too, which are listed at the end of the B2 web page you linked to?


#11

You sure you want to disable checksums? It’ll work with them on but it’ll need to calculate it and then it’ll store that on b2 so it knows you have a correct and consistent file (s). If you’re transferring lots of files you should just raise the transfer threads.


#12

Not sure about anything. It seemed to be transferring, but slowed down my whole internet (which is usually speedy) over wifi, and was still transferring the file very slowly. Should I re-enable the check sums? How do I increase the transfer threads?


#13

–transfers=

But that will increase your bandwidth usage.

You can limit the bandwidth with --bwlimit but it’ll go slower.


#14

Thanks for the advice, will give it a try. Do you think I should skip the check sums command too?


#15

I wouldn’t but that depends on your level of risk. Without it, uploads aren’t verified. I use b2 but I have smaller files. Images and smaller video.


#16

Yeah, tried increasing that transfer number w/o the checksum command, but that didn’t work – still gets hung up.


#17

Is it hung or is it doing the checksum? Might want to post a debug log -vv

Transfers /checkers help with multiple files. Not just one big one…


#18

Thanks again for the help. How do I post a log with a CLI program, since I can’t copy and paste?


#19

You should be able to just select the contents of the log (you can use the --log-file parameter) and paste it here. Or you can upload that log to some sharing tool like Google drive an share the link here or pm.