rclone copy bb2:image2/ bb2:static/imgs/images2/ --transfers 9999999 --checkers 9999999 -P --ignore-existing
I'm using the above commend for copying more than 9M small files for two B2 buckets.
Will the files not downloaded to local and uploading later ?
hello and welcome to the forum,
based on the rclone documentation, rclone should do a server side copy, not downloading/uploading
the copy should be server side, not downloading then uploading.
this can be tested by copying a single file from source to dest and using
-vv for debug output
rclone copy bb2:image2/file.txt bb2:static/imgs/images2
bb2: the top of a Backblaze remote? So are
static bucket names or "directories" within the same bucket?
If they are in the same bucket (and
bb2: is an alias or crypt) then this should be server side without a problem.
If they are different buckets, it depends. Are they in the same dataceter? I seem to recall they had to be (though now that I look at the docs, I am less certain).
As an aside, given your
--transfers 9999999 --checkers 9999999 settings, I am to believe you have infinite ram. Could you use
--fast-list to speed that up too? Is it possible that you are causing 19999998 threads to spin up on your computer? And very likely getting rate-limited by B2?
Also, since you didn't use the help template, my answer is assuming the latest rclone. It was newer that B2 added the copy so make sure your version is at least relatively up to date.
I did the command and the copying seems to be on server side but the speed is not fact enough like 100mb/s which is not continued to the end.
I checked the status in BB panel and the transfer died today.
And yes the command indeed gave my machine freezing for minutes.
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.