What is the problem you are having with rclone?
Hey there! I've finally moved on to a more stable platform since the last time I was here (Now running x64 rclone 1.62.2 on an Ubuntu 22.04 Privileged LXC container for Proxmox 8), and have been enjoying it inmensely both with the Union remote and now with mergerfs, but recently it seems I've been running out of space quickly and I'd like to move to Backblaze B2 as a long term solution for my CryptFS remote.
Thing is, most of my information has been saved to several Consumer OneDrive accounts and a single Personal Google Drive, so I'd like to know which would be the fastest way to transfer about 1.7TB of information out of these remotes and into Backblaze.
So far, I've been able to transfer about 700GB with a 1000Mbps down/250Mbps up connection, but I'd like to make sure everything can be double-checked with the --checksum flag at the end and find a way to either pause or restart the transfer whenever one of the remotes gets rate-limited.
Run the command 'rclone version' and share the full output of the command.
rclone v1.62.2
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 6.2.16-3-pve (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.2
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
OneDrive Personal, Google Drive Personal and Backblaze B2
The command you were trying to run (eg rclone copy /tmp remote:tmp)
sudo rclone mount ToasterDEV-OneDrive:/Union /mnt/cloud/ToasterDEV-OneDrive --config=/opt/rclone/rclone.conf --allow-other=true --attr-timeout=8700h --poll-interval=30s --umask=000 --user-agent=unionfs --syslog --checkers=24 --human-readable --low-level-retries=20 --retries=20 --retries-sleep=5s --metadata --multi-thread-streams=1024 -vv --fast-list --transfers=24 &
sudo rclone mount ToasterDEV-GDrive:/ /mnt/cloud/ToasterDEV-GDrive --config=/opt/rclone/rclone.conf --allow-other=true --attr-timeout=8700h --poll-interval=30s --umask=000 --user-agent=unionfs --syslog --checkers=24 --human-readable --low-level-retries=20 --retries=20 --retries-sleep=5s --metadata --multi-thread-streams=1024 -vv --fast-list --transfers=24 &
sudo rclone mount Lizita63-OneDrive:/Union /mnt/cloud/Lizita63-OneDrive --config=/opt/rclone/rclone.conf --allow-other=true --attr-timeout=8700h --poll-interval=30s --umask=000 --user-agent=unionfs --syslog --checkers=24 --human-readable --low-level-retries=20 --retries=20 --retries-sleep=5s --metadata --multi-thread-streams=1024 -vv --fast-list --transfers=24 &
sudo rclone mount Flanch1942-OneDrive:/Union /mnt/cloud/Flanch1942-OneDrive --config=/opt/rclone/rclone.conf --allow-other=true --attr-timeout=8700h --poll-interval=30s --umask=000 --user-agent=unionfs --syslog --checkers=24 --human-readable --low-level-retries=20 --retries=24 --retries-sleep=5s --metadata --multi-thread-streams=1024 -vv --fast-list --transfers=24 &
sudo rclone --config="/opt/rclone/rclone.conf" sync Local:/mnt/mergerfs Backblaze:/RCloneEncFS -vv --fast-list --checkers=100 --transfers=100 --multi-thread-streams=1024 --syslog --rc --rc-enable-metrics --rc-web-gui --rc-allow-origin --rc-web-gui-update --rc-web-gui-no-open-browser --rc-user=rclone --rc-pass=REDACTED --rc-addr=:5572 &
The rclone config contents with secrets removed.
[ToasterDEV-OneDrive]
type = onedrive
token = {"access_token":"REDACTED"}
drive_id = REDACTED
drive_type = personal
chunk_size = 250M
[ToasterDEV-GDrive]
type = drive
client_id = REDACTED
client_secret = REDACTED
scope = drive
root_folder_id = REDACTED
token = {"access_token":"REDACTED"}
chunk_size = 512Mi
team_drive =
use_trash = false
acknowledge_abuse = true
[Local]
type = local
[Union]
type = union
upstreams = ToasterDEV-OneDrive:Union ToasterDEV-GDrive: Lizita63-OneDrive:Union Flanch1942-OneDrive:Union
action_policy = lus
create_policy = lus
search_policy = lus
cache_time = 3600
[Chunker]
type = chunker
remote = Local:/mnt/mergerfs
hash_type = sha1all
chunk_size = 2Gi
[EncFS]
type = crypt
remote = Chunker:/
filename_encryption = obfuscate
directory_name_encryption = true
password = REDACTED
password2 = REDACTED
[Lizita63-OneDrive]
type = onedrive
token = {"access_token":"REDACTED"}
drive_id = REDACTED
drive_type = personal
chunk_size = 250M
[Lizita63-GDrive]
type = drive
client_id = REDACTED
client_secret = REDACTED
scope = drive
root_folder_id = REDACTED
token = {"access_token":"REDACTED"}
chunk_size = 512Mi
team_drive =
use_trash = false
acknowledge_abuse = true
[Flanch1942-OneDrive]
type = onedrive
token = {"access_token":"REDACTED"}
drive_id = 8559f24b32b9889d
drive_type = personal
chunk_size = 250M
[Flanch1942-GDrive]
type = drive
client_id = REDACTED
client_secret = REDACTED
scope = drive
root_folder_id = REDACTED
token = {"access_token":"REDACTED"}
chunk_size = 512Mi
team_drive =
use_trash = false
acknowledge_abuse = true
[MergerFS]
type = local
copy_links = true
links = true
[Backblaze]
type = b2
account = REDACTED
key = REDACTED
download_url = REDACTED
upload_cutoff = 5Gi
chunk_size = 5Gi
memory_pool_use_mmap = true
A log from the command with the -vv flag
Jun 24 16:13:50 UbuntuLXC rclone[5336]: OneDrive root 'Union': Checking for changes on remote
Jun 24 16:13:50 UbuntuLXC rclone[5336]: OneDrive root 'Union': Change notify listener failure: invalidRequest: [token.String] A value is required but was not present in the request.
Jun 24 16:13:53 UbuntuLXC rclone[5477]: OneDrive root 'Union': Checking for changes on remote