Rclone-copy traffic Issue (Google Drive -> Onedrive)

What is the problem you are having with rclone?

Rclone copy resulted in large amount of traffic

Run the command 'rclone version' and share the full output of the command.

rclone v1.67.0
- os/version: alpine 3.20.0 (64 bit)
- os/kernel: 4.15.0-43-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.4
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive, copying to Onedrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy google:TMU3 onedrive-3:TMU3 --config /config/rclone.conf --drive-acknowl
edge-abuse --transfers=16 --checksum --check-first --verbose

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[google]
type = drive
scope = drive
token = XXX

[jottacloud]
type = jottacloud
client_id = XXX
client_secret =
tokenURL = https://id.jottacloud.com/auth/realms/jottacloud/protocol/openid-connect/token
token = XXX
configVersion = 1

[onedrive]
type = onedrive
token = XXX
drive_id = XXX
drive_type = personal

[onedrive-2]
type = onedrive
token = XXX
drive_id = XXX
drive_type = personal

[onedrive-3]
type = onedrive
token = XXX
drive_id = XXX
drive_type = personal
### Double check the config for sensitive info before posting publicly

A log from the command that you were trying to run with the -vv flag

Missing

Rclone Traffic Issue (Google Drive -> Onedrive)

Hello everyone,

I am using rclone copy to back up data from Google Drive to OneDrive. My Google Drive account is provided by an educational institution, which has announced a reduction in storage space, so I need to transfer the data and delete the original files. I was initially using rclone-copy from GitHub - robinostlund/docker-rclone-copy, with the last release dated 2023/07/18.

Recently, I downloaded the Dockerfile and rebuilt the entire image, upgrading rclone to version 1.67. After the update, I noticed a large number of "The resource has changed since the caller last read it; usually an eTag mismatch" errors when syncing to OneDrive and occasional traffic alerts from Linode. However, since the files were still syncing, I didn't pay much attention to it. Until yesterday, when I received a bill from Linode, and my credit balance was exhausted, resulting in a debt. I saw in the management console that my outbound traffic for June was 26.4 TB, while the original data volume I needed to back up was only 2 TB.

My final rclone copy parameters were --drive-acknowledge-abuse --transfers=16 --checksum --check-first --verbose, where --checksum fell back to --size-only due to differences between Google Drive and OneDrive. After confirming that important files were backed up, I prematurely ended the backup and deleted the old data from Google Drive as required by the institution.

I apologize for not retaining further logs before asking on the forum, and I have already deleted the original data, so it may be difficult to reproduce the issue. I just realized that there is a forum where I can ask about this. Sorry for any inconvenience

Could there be any mistakes in my usage process (e.g wrong command line options), and what are the possible solutions? Thanks in advance.

Next to impossible to help without logs.

My hunch is you were really throttled by OneDrive, can't imagine Microsoft keeping up with the 16 transfers, most likely ending in failures and retries that consumed bandwidth. When I migrated from GDrive to OneDrive I was facing throttling with anything over 3 transfers, 3 tps and even then needed babysitting to pause it when Microsoft started to throttle to pause it and resume it after 15-20 minutes and the pacing stopped.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.