Sync multiple Google drivers

I want to sync multiple google drives using the rclone so I have two question

  1. I want to use Google cloud VPS to sync files, Do I have to pay for bandwidth while syncing file between google drives ?

  2. When I am using the sync command sometimes it fails when it sees the duplicate file and sometimes it just ignore them. how can i resolve this problem?

thanks

You can sync using a server side copy provided the destination account can read from the source account and you use the flag --drive-server-side-across-configs. This won't use bandwidth.

Usually running rclone again will fix up any problems.

1 Like

can you give me an example to how to do it ? I used the following command but it used my bandwidth to copy the files

.\rclone.exe copy --drive-server-side-across-configs=true --progress gdrive1:/myfiles gdrive2:/myfiles

then I edit config file and went to advanced config and set server_side_across_configs = true , again I used above command but nothing happened and result was same.


Edit2:

I notice that it is not possible to use server side copy for crypt drive. So I decided to copy whole encrypted google drive to the new google drive but I received following error:

2020-03-30 21:15:26 ERROR : Media/isv3hullgh1c0s8c36de3e4f2c/tuheug1ca7ujov4poloosuq6eng2g24po7vkj3ov7skk4u6p0f9c0huqlo6d7s50jutah0ffogq9u3gs7cta96l3tsill4mro2kd510: Failed to copy: googleapi: Error 404: File not found: 1hVkJqHHXgtIoe9YqxwAgqWa_KBjwMvs_., notFound

rclone gave this error for all files.

What version of rclone are you using?
What command are you actually running?

Can you run the same command with -vv and share the log of the command?

You mentioned you are running an encrypted remote, you can use the non encrypted remote and server side copies those if you are keeping the same encryption password/salt. If not, you can't use server side.

As I wrote in Edit 2 , I did exactly what you said but I only received the error , I used following command

.\rclone.exe copy gdrive1: gdrive2: --drive-server-side-across-configs -P -v -vv

There are 100000s of not found error I can not see anything else even by putting -vv

2020-03-30 22:06:34 ERROR : Media/isv3hullgh1c0s8c36de3e4f2c/358ottr3g3i010ie79unvj110egl2eg89146jof6cb8evmsg7o1rgc16duvftoaafi49hojev62i5a2kpbjkhd19ojv0j690dn5aongfrs15hclhfoornf3be0bkctvf: Failed to copy: googleapi: Error 404: File not found: 1lkKDAdaEaq8yNu-RUL8TbAbL-6sAD-vo., notFound
2020-03-30 22:06:34 ERROR : Media/isv3hullgh1c0s8c36de3e4f2c/33ogq55sc2dm4djcg2rpj1l9eajf2mf1j95lsqkeu1b0v3cif99dv35vrnalionien0l3uvs9stvdson0cr27a97agm2fvp72s60ur0813hnssraubtd07u5au65vu6ba8b6hsba6u19nrflmfqgq3su1k: Failed to copy: googleapi: Error 404: File not found: 17yYGzQUesDKfNqHm2fdpwsmZ0-Q4ZkMb., notFound
2020-03-30 22:06:34 ERROR : Media/isv3hullgh1c0s8c36de3e4f2c/35fuuhvbdan1n6f3dud5q7hk2ei7m7muu8etnun43lkcqp7vdoabjvqfvqho9ecoadap1ot4ctj2scalftg07i0pjntpsbdjpbpt6ko: Failed to copy: googleapi: Error 404: File not found: 18STYNDReJxGawvdYUKp9OS1Im_C4let5., notFound
2020-03-30 22:06:34 ERROR : Media/isv3hullgh1c0s8c36de3e4f2c/3m56o431g6p71lbmrigkjpt6lnravk1g0gbqv84udba71m244hu7ctnefo5j28mkqi3l1vgp8b9t7etni0i25aqlq8rrcvrer9hrff0: Failed to copy: googleapi: Error 404: File not found: 1XVbRlUganfg0Iias2hlF1vzF9WHiQQzz., notFound
2020-03-30 22:06:34 ERROR : Media/isv3hullgh1c0s8c36de3e4f2c/45hauhl8lf0vnpt61lnhllorgruga62248i8mrgej5qnvg28celj75ja6aqc8moilimv3htp8v6b6aq9el2e255ta47jbq8gdhrdjdp03m6mdckkcvdqo66j1nubmk19: Failed to copy: googleapi: Error 404: File not found: 1WtOvGPgsVx50FsSpeDq6prnk0QNG0_1V., notFound
2020-03-30 22:06:34 ERROR : Media/isv3hullgh1c0s8c36de3e4f2c/3rh9ak823kkujbclpbb9maaq7juq0o3sjd8agdbffcijb0i11han03pjfc30feemqqgjtvkrjea3ma313ba3as3qjbpti5fhu9mp969895at58o9scupgcsnd6u6g3av: Failed to copy: googleapi: Error 404: File not found: 1KQaEKIgnpzKgnQUJqswKuzuwQezVFgMp., notFound

What version of rclone are you running?
Can you share the debug log?

What's your rclone.conf with passwords removed?

That error means that gdrive2: doesn't have permissions to read from gdrive1:. You can share gdrive2 with gdrive1 and I think it will work.

1 Like

Thanks. It's worked but still I am receiving not found error for some files

2020-03-31 21:48:48 ERROR : isv3hullgh1c0s8c36de3e4f2c/0e109jfknh9cq6ioopru8392mqiltsg4ipgv4fljukmirjg6e5ch8ce40rl1t52ul62dr8dm3v5jvhp8jfombge4jfjtr2586gmcudjus2iin1ncu4t3rimklcfjnuip: Failed to copy: googleapi: Error 404: File not found: 1sG_KOSBuwT6EVmK5YglDT-lCizeMOe-d., notFound
2020-03-31 21:48:48 ERROR : isv3hullgh1c0s8c36de3e4f2c/0tnde8kg61gdb1mlprig38pfraqckmc901t3vf8qp6p4rp2drov0ugbg902g8mgh62sans66f5vnfuo12gbm9jam3l36pm7kg64p3jaig11bahjeegfqb6amenm6tkh2kifhmlnj8grldc7r5m18s293vk: Failed to copy: googleapi: Error 404: File not found: 1ObXsYuZzdo2DQq--t9I7gndwVKccI91x., notFound
2020-03-31 21:48:48 ERROR : isv3hullgh1c0s8c36de3e4f2c/1u90is8p0ahgdqlevjjlbf024eoc3dphpd002ilsk439poiouar0: Failed to copy: googleapi: Error 404: File not found: 16U4I5ylknZkrGAUiZ3GhzTWEEttZw5XI., notFound
2020-03-31 21:48:49 DEBUG : pacer: Reducing sleep to 0s
2020-03-31 21:48:49 ERROR : isv3hullgh1c0s8c36de3e4f2c/35fuuhvbdan1n6f3dud5q7hk2ei7m7muu8etnun43lkcqp7vdoabjvqfvqho9ecoadap1ot4ctj2scalftg07i0pjntpsbdjpbpt6ko: Failed to copy: googleapi: Error 404: File not found: 18STYNDReJxGawvdYUKp9OS1Im_C4let5., notFound
2020-03-31 21:48:49 DEBUG : fguojd5ctj3sj34kaau1dftsvc/2b0mnf5f2prcisq5pfup82lq0tngb2qqsqav6q9kts3e6sp9h5lg/ulha1rb42drok8u6ffmjqma5hlqoqlf8hr27vaebo79tbt1gqbjcbfmpj2398bb93da28cg6k5pnc9tufucgfs1vrp9qc6kjaoa9h9238g6l3s01it8atusbcoba7lrm: MD5 = 2025631418c842a6a873eff04dbf6b37 OK
2020-03-31 21:48:49 INFO  : fguojd5ctj3sj34kaau1dftsvc/2b0mnf5f2prcisq5pfup82lq0tngb2qqsqav6q9kts3e6sp9h5lg/ulha1rb42drok8u6ffmjqma5hlqoqlf8hr27vaebo79tbt1gqbjcbfmpj2398bb93da28cg6k5pnc9tufucgfs1vrp9qc6kjaoa9h9238g6l3s01it8atusbcoba7lrm: Copied (server side copy)
2020-03-31 21:48:49 ERROR : isv3hullgh1c0s8c36de3e4f2c/358ottr3g3i010ie79unvj110egl2eg89146jof6cb8evmsg7o1rgc16duvftoaafi49hojev62i5a2kpbjkhd19ojv0j690dn5aongfrs15hclhfoornf3be0bkctvf: Failed to copy: googleapi: Error 404: File not found: 1lkKDAdaEaq8yNu-RUL8TbAbL-6sAD-vo., notFound
2020-03-31 21:48:49 ERROR : isv3hullgh1c0s8c36de3e4f2c/2aiuum8hu6glkn9l2lph9ihlnl1k7idkmm2hdo16q1l0fqh3g7kb4ukiiqm2sq33s3lp4n6ojqtfdo9frua10go2m6os7a572bfi1iq10fe4uf6ovh3nqpkv6ebg0vdl: Failed to copy: googleapi: Error 404: File not found: 1umvMRNOHwR4OCAxBAm8V3WLBE6dtKJFE., notFound
2020-03-31 21:48:49 ERROR : isv3hullgh1c0s8c36de3e4f2c/3m56o431g6p71lbmrigkjpt6lnravk1g0gbqv84udba71m244hu7ctnefo5j28mkqi3l1vgp8b9t7etni0i25aqlq8rrcvrer9hrff0: Failed to copy: googleapi: Error 404: File not found: 1XVbRlUganfg0Iias2hlF1vzF9WHiQQzz., notFound
2020-03-31 21:48:49 ERROR : isv3hullgh1c0s8c36de3e4f2c/4tanl86lkhes0kpdhba3l089b2gdur071cerain6d538hsg9n2g2dt5t444q7g0ca5ufg10e0adv8hboosau82ijt2oe2fkdfsu7560: Failed to copy: googleapi: Error 404: File not found: 1_VbrWQDz1oRRtrRx6gm4WNhb9CHvJQAn., notFound

What version of rclone?

------------ rclone-v1.50.2

There are a few fixes in team drives in 1.51 so you may want to try that. It still sounds like you have a permissions issue though.

What did you you add in terms of permissions?

Do those files have a different owner maybe?

If you want to find out which files they are then you can use rclone cryptdecode on the names.

100% same owner. there is no other owner.

It is not a teamdrive. I shared the main folder in first business google account with the second one which I want to move it. So, It shouldn't be a permission error.

It looks like permission errors. You should try to set the permission again. Or share the files to that specific user.

Anything you can spot about the files it works with vs the files it does not work with?

All files are mp4.

As you told me I used cryptdecode to see the file name , I notice that all not found error are belong to some files in a specific folder. I moved that folder to another place and then rclone successfully copy 742GB. About that particular folder, I don't know what's wrong with it. files inside it are encrypted and there is no permission problem.

Hmm... I wonder what is different about these files. Any ideas?

That is good!

You might have to copy that one folder without --drive-server-size-across-configs

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.