What is the problem you are having with rclone?
Copying from Google Workspace to OneDrive business is extremely slow after the first attempt.
- Client ID has been set up for Google Drive and OneDrive
- Tried using the rclone copy command (what I'm calling simple command) and with various flags that others have used in the forum (test and tried a few of them, and ended up with what I'm calling the complex command)
- The first attempt seemed reasonably fast (IIRC ~ 10mb), but now it is super slow (rating from 50 byes to 1-5kb).
- Overall I am attempting to copy ~180gb of data from Google Drive to OneDrive
- To be honest, I am not exactly sure how to check throttling
Run the command 'rclone version' and share the full output of the command.
rclone v1.60.1
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-53-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.19.3
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Source: Google Drive (Workspace)
Destination: OneDrive (M365 E3)
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
simple command
rclone copy jf-gdrive: jf-onedrive:GoogleDriveJF --progress --log-file 'simple1' --log-level DEBUG
complex command
rclone copy jf-gdrive: jf-onedrive:GoogleDriveJF -vv --drive-pacer-min-sleep=10ms --checkers=16 --drive-skip-shortcuts --ignore-existing --drive-chunk-size 256M -vv
The rclone config contents with secrets removed.
[jf-gdrive]
type = drive
client_id = //removed
client_secret = //removed
scope = drive
token = //removed
team_drive =
root_folder_id = 0AO4SmDrVvmYiUk9PVA
[jf-onedrive]
type = onedrive
token = //removed
drive_id = //removed
drive_type = business
client_id = //removed
client_secret = //removed
A log from the command with the -vv
flag
First few lines of log:
2022/11/26 11:46:14 DEBUG : rclone: Version "v1.60.1" starting with parameters ["rclone" "copy" "jf-gdrive:" "jf-onedrive:GoogleDriveJF" "--progress" "--log-file" "simple1" "--log-level" "DEBUG"]
2022/11/26 11:46:14 DEBUG : Creating backend with remote "jf-gdrive:"
2022/11/26 11:46:14 DEBUG : Using config file from "/home/jeffrey/.config/rclone/rclone.conf"
2022/11/26 11:46:14 DEBUG : Creating backend with remote "jf-onedrive:GoogleDriveJF"
2022/11/26 11:46:16 NOTICE: Dangling shortcut "Smart Insights" detected
2022/11/26 11:46:16 DEBUG : Contact Information: No export formats found for "application/vnd.google-apps.form"
2022/11/26 11:46:16 NOTICE: Creativus Design - IT Scope Document updated.docx: Duplicate object found in source - ignoring
2022/11/26 11:46:16 DEBUG : CASTANET - Jeffrey Fazal - My network.xlsx: Size and modification time the same (differ by -804ms, within tolerance 1s)
2022/11/26 11:46:16 DEBUG : CASTANET - Jeffrey Fazal - My network.xlsx: Unchanged skipping
2022/11/26 11:46:16 DEBUG : CASTANET - Jeffrey Fazal - NSW Australian Marketing Institue.xlsx: Size and modification time the same (differ by -179ms, within tolerance 1s)
2022/11/26 11:46:16 DEBUG : CASTANET - Jeffrey Fazal - NSW Australian Marketing Institue.xlsx: Unchanged skipping
First few lines of completed list
2022/11/26 11:58:17 DEBUG : rclone: Version "v1.60.1" starting with parameters ["rclone" "copy" "jf-gdrive:" "jf-onedrive:GoogleDriveJF" "--progress" "--log-file" "complex1" "--log-level" "DEBUG" "--drive-pacer-min-sleep=10ms" "--checkers=16" "--drive-skip-shortcuts" "--ignore-existing" "--drive-chunk-size" "256M"]
2022/11/26 11:58:17 DEBUG : Creating backend with remote "jf-gdrive:"
2022/11/26 11:58:17 DEBUG : Using config file from "/home/jeffrey/.config/rclone/rclone.conf"
2022/11/26 11:58:17 DEBUG : jf-gdrive: detected overridden config - adding "{a0HWK}" suffix to name
2022/11/26 11:58:17 DEBUG : fs cache: renaming cache item "jf-gdrive:" to be canonical "jf-gdrive{a0HWK}:"
2022/11/26 11:58:17 DEBUG : Creating backend with remote "jf-onedrive:GoogleDriveJF"
2022/11/26 11:58:19 DEBUG : Contact Information: No export formats found for "application/vnd.google-apps.form"
2022/11/26 11:58:19 NOTICE: Creativus Design - IT Scope Document updated.docx: Duplicate object found in source - ignoring
2022/11/26 11:58:19 DEBUG : CASTANET - Jeffrey Fazal - My network.xlsx: Destination exists, skipping
2022/11/26 11:58:19 DEBUG : CASTANET - Jeffrey Fazal - NSW Australian Marketing Institue.xlsx: Destination exists, skipping
2022/11/26 11:58:19 DEBUG : CASTANET - Jeffrey Fazal - Resturant Owners ACT.xlsx: Destination exists, skipping