What is the problem you are having with rclone?
I'm trying to transfer files from my small Linux box (Proxmox) up to Backblaze b2 through Rclone. A directory of around 3TB with each file (on average) between 50-100GB.
I have managed to get 3 of the smallest (<10 GB) files up to the cloud with relative ease, but after that I'm finding it impossible to upload further files.
Run the command 'rclone version' and share the full output of the command.
rclone v1.68.2
- os/version: debian 12.8 (64 bit)
- os/kernel: 6.8.12-2-pve (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.3
- go/linking: static
- go/tags: none
I can verify that this is the latest version of rclone.
Which cloud storage system are you using? (eg Google Drive)
Backblaze b2
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone copyto {source_file_name} b2:{my_bucket}/{dest_file_name} --no-check-dest --no-traverse -P -vv --dump headers
I have tried with and without the --no-check-dest
and --no-traverse
flags but from what I can tell, they don't seem to be making any difference.
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[b2]
type = b2
account = XXX
key = XXX
A log from the command that you were trying to run with the -vv
flag
# rclone copyto {source_file_name} b2:{my_bucket}/{dest_file_name} --no-check-dest --no-traverse -P -vv --dump headers
2024/11/28 14:15:02 DEBUG : Setting --fast-list "true" from environment variable RCLONE_FAST_LIST="1"
2024/11/28 14:15:02 NOTICE: Automatically setting -vv as --dump is enabled
2024/11/28 14:15:02 DEBUG : rclone: Version "v1.68.2" starting with parameters ["rclone" "copyto" "{source_file_name}" "b2:{bucket}/{dest_file_name}" "--no-check-dest" "--no-traverse" "-P" "-vv" "--dump" "headers"]
2024/11/28 14:15:02 DEBUG : Creating backend with remote "{dest_file_name}"
2024/11/28 14:15:02 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/11/28 14:15:02 DEBUG : fs cache: adding new entry for parent of "{source_file_name}", "/path/to/folder"
2024/11/28 14:15:02 DEBUG : Creating backend with remote "b2:{bucket}/"
2024/11/28 14:15:02 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/11/28 14:15:02 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/11/28 14:15:02 DEBUG : HTTP REQUEST (req 0xc0001383c0)
2024/11/28 14:15:02 DEBUG : GET /b2api/v1/b2_authorize_account HTTP/1.1
Host: api.backblazeb2.com
User-Agent: rclone/v1.68.2
Authorization: XXXX
Accept-Encoding: gzip
2024/11/28 14:15:02 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/11/28 14:15:03 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/11/28 14:15:03 DEBUG : HTTP RESPONSE (req 0xc0001383c0)
2024/11/28 14:15:03 DEBUG : HTTP/1.1 200
Content-Length: 883
Cache-Control: max-age=0, no-cache, no-store
Connection: keep-alive
Content-Type: application/json;charset=UTF-8
Date: Thu, 28 Nov 2024 14:15:03 GMT
Server: nginx
Strict-Transport-Security: max-age=63072000
2024/11/28 14:15:03 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/11/28 14:15:03 DEBUG : fs cache: renaming cache item "b2:{bucket}/" to be canonical "b2:{bucket}"
2024/11/28 14:15:03 DEBUG : {dest_file_name}: Need to transfer - File not found at Destination
2024/11/28 14:15:03 DEBUG : {dest_file_name}: multi-thread copy: disabling buffering because source is local disk
At the time of writing, I've been trying to upload through rclone
for 15 minutes and still have yet to pass a single byte down the wire. It feels like it's blocked on something.
In playing around with different values, I managed to get rclone
to accidentally create a new file of the one I'm trying to send on that same drive (which made me think it was working) - that started instantly. So I'm wondering if the program is waiting on a response from somewhere before it can kick off?