Rclone sync multiple drives to one

Help and Support
1

What is the problem you are having with rclone?

My company gdrive accounts are about to be closed since the company is closing and this is happening at the end of august.
I purchased a new g-drive account and synced about 8TBs of data.

I successfully synced the largest g-drive of about 6TB to the new gdrive using rclone sync then did the second drive and then went back to re-sync the first drive again as a few files were added. Once the sync was complete i was going to be mounting the new drive replacing old drives. After about 2 hours i seen that the drive was still syncing which was strange since it was only about 3GB of data added.

I checked the size of the new drive and it showed only 2TB and not 8TB which is the combined size of drive A and drive B.

Does sync delete everything on the new drive? I thought it would copy everything from drive A and then copy every thing from drive B

so the structure of NEW drive would look like this.

Folder-From-Drive-A
Folder-From-Drive-A
Folder-From-Drive-A
Folder-From-Drive-B
Folder-From-Drive-B

Is there someway to for me to sync multi drives to one drive?

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.1

  • os/version: ubuntu 18.04 (64 bit)
  • os/kernel: 4.15.0-175-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.18.5
  • go/linking: static
  • go/tags: none

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
-->
yes

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync a: new:

The rclone config contents with secrets removed.

[gdrive]
type = drive
client_id = XXXXXXX.apps.googleusercontent.com
client_secret = XXXXXXXXX
scope = drive
token = {"access_token":"XXXXXX","token_type":"Bearer","refresh_token":"XXXXXXX","expiry":"2022-08-21T12:56:51.100078481Z"}
team_drive =

[new]
type = crypt
remote = gdrive:crypt
password = XXXXXXXX
password2 = XXXXXXXX
2

Seems i should of done some more research before using sync as it will delete everything in des drive to match src drive.

I should of used rclone copy --progress a: new: instead as this will only copy.

Hopefully i will have enough time before drives are shut down to get stuff copied over.

3

for each gdrive, can use a different dest dir.
rclone sync a: new:Folder-From-Drive-A --dry-run
rclone sync b: new:Folder-From-Drive-B --dry-run

and read about
--drive-server-side-across-configs

1 Like
4

--drive-server-side-across-configs
Interesting i assumed this will allow to copy far faster but it doesnt seem to be the case.

rclone sync --progress --drive-server-side-across-configs a: new:a

Is there a reason you are suggesting sync over rclone copy?

Also if I stop rclone copy at lets say 50% transfer when i run it again will it pick up where it left off or will it start from 0%?
I tested and it doesnt seem to resume it starts again.

5

well, you were the one using the term sync, multiple times, including in title of your topic.

each time rclone sync/copy is run, the same things happens.
--- compare source to dest.
--- decide what to transfer/delete.
--- transfer/delete files.

1 Like
6

thank you for the prompt response.

That is what i thought happens but i ran copy and transferred about 800GB of data then i stopped it and it started from 0 again not 800GB+

can you clarify what --drive-server-side-across-configs is suppose to do as i from reading the docs here it is suppose to copy the data server side but i see no change in transfer speed.

7

right, when rclone starts, it has not transferred any files yet

google will transfer the file from google server to google server, without using your local bandwidth.

that is documented.

1 Like
8

i left it run for nearly 30 minutes and it didnt seem to make any difference. I would of assumed it detected the already transferred files and incremented the % but it did.

Ill keep testing.

thanks for the help

9

yes, that is what rclone does.

that data transfer counter will only increment as rclone transfers files.
if the the source file and dest file match, then there is not data to transfer.

10

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.