Migrate from TeamDrive to Another TeamDrive

Thanks, i tried without success this command

C:\Users\Robertus\Desktop\UP\RCLONE>rclone copy --drive-server-side-across-configs=true --stats 30s "TEAM_DRIVE1:Varie" "TEAMDRIVE2:Varie" -v
2019/09/21 21:21:05 NOTICE: MYFILE1.mp4: Duplicate object found in source - ignoring
2019/09/21 21:21:05 NOTICE: MYFILE2.mp4: Duplicate object found in source - ignoring
2019/09/21 21:21:05 INFO : Google drive root 'Varie': Waiting for checks to finish
2019/09/21 21:21:05 INFO : Google drive root 'Varie': Waiting for transfers to finish
2019/09/21 21:21:06 ERROR : MYFILE1.mp4: Failed to copy: googleapi: Error 404: File not found: xxxxxxxxxxxxxxxxxd1TOu., notFound
2019/09/21 21:21:06 ERROR : MYFILE2.mp4: Failed to copy: googleapi: Error 404: File not found: xxxxxxxxxxxxxxxxx1tyRh., notFound

Also without =true, so only --drive-server-side-across-configs, but same story.

Can you share your actual rclone.conf without passwords/keys?

[TEAM_DRIVE1]
type = drive
client_id = xxxxx.apps.googleusercontent.com
client_secret = xxxxxxxw6MPAm
scope = drive
token = {"access_token":"xxxxxxkxxxxxxxX2s__VxxxxxxxxxJg","token_type":"Bearer","refresh_token":"1/xxxa2Ixxxzxxxxxxxxxov_p","expiry":"2019-09-21T22:14:23.4246032+02:00"}
team_drive = xxxxxxk9PVA

[TEAMDRIVE2]
type = drive
client_id = xxxxxxxxxxxxx.apps.googleusercontent.com
client_secret = 1xxxxxxxx6gL
scope = drive
token = {"access_token":"xxxxxxxxxxxxxxxxxyVzPxxxxxxxxxxxxxxA","token_type":"Bearer","refresh_token":"1/xxxxxxxxxxxxxO0","expiry":"2019-09-21T22:17:48.3294604+02:00"}
team_drive = xxxxxxxxxxxxxxx

You may want to clean up the duplicates in your source and retry. Not 100% sure that's it or not though.

rclone dedupe the source.

2019/09/21 21:36:59 NOTICE: MYFILE1.mp4: Found 2 duplicates - deleting identical copies
2019/09/21 21:36:59 NOTICE: MYFILE1: Deleting 1/2 identical duplicates (MD5 "32cbcd894a6bf5fc510ae6b5eqc6fa5510")
2019/09/21 21:37:00 NOTICE: MYFILE1.mp4: All duplicates removed
2019/09/21 21:37:00 NOTICE: MYFILE2.mp4: Found 2 duplicates - deleting identical copies
2019/09/21 21:37:00 NOTICE: MYFILE2.mp4: Deleting 1/2 identical duplicates (MD5 "d6ee0cd225fbacwe517dd5662a8c0a166f")
2019/09/21 21:37:01 NOTICE: MYFILE2.mp4: All duplicates removed

done, i tried one more time to copy between drives, but same error

Just did a similar setup:

[felix@gemini rclone]$ rclone copy GD:hosts TEST: -vv
2019/09/21 15:39:33 DEBUG : rclone: Version "v1.49.3" starting with parameters ["rclone" "copy" "GD:hosts" "TEST:" "-vv"]
2019/09/21 15:39:33 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2019/09/21 15:39:34 DEBUG : hosts: Couldn't find file - need to transfer
2019/09/21 15:39:36 DEBUG : hosts: MD5 = 1450f7e2eb39701b01c3d307bf5c6a77 OK
2019/09/21 15:39:36 INFO  : hosts: Copied (server side copy)
2019/09/21 15:39:36 INFO  :
Transferred:   	       254 / 254 Bytes, 100%, 99 Bytes/s, ETA 0s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            1 / 1, 100%
Elapsed time:        2.5s

2019/09/21 15:39:36 DEBUG : 7 go routines active
2019/09/21 15:39:36 DEBUG : rclone: Version "v1.49.3" finishing with parameters ["rclone" "copy" "GD:hosts" "TEST:" "-vv"]

How are you running the command? The conf you shared does not have server-side true in it.

I run ** rclone copy --drive-server-side-across-configs=true --stats 30s "TEAM_DRIVE1:Varie" "TEAMDRIVE2:Varie" -v**

from prompt in windows 10

--drive-server-side-across-configs=true
should work, but just to make sure that this is actually enabled - try setting it in your config instead. Use:

server_side_across_configs = true (note the different formating - copypaste it)
Put this in the bottom of each of your 2 remotes
save the config
remove " --drive-server-side-across-configs=true" from your command
then retry and see if the result is the same.

Are both of these drives accessibe under "shared drives" (previously called teamdrives) in the Google Drive website. In other words - is the user a member of both teamdrives?

I ask because it can be a bit confusing now to talk about shared files and shared drives which are fundamentally different things. Feel free to crop a small image from your Google Drive website that lists them just we can see this looks as expected.

The next thing to check would be permissions. The destination needs to have access to the source on the Share Drive to do the server side copy otherwise, it throws a 404 error as it cannot access the file.

It doesn't matter if you put it in the config or run the command line flag as they both do the same thing.

It shouldn't - no.
I'd probably still test it to be on the safe side since it fast and easy to do. Then at least we know it's not simply the function not being enabled properly.

EDIT: For example I'm not exactly sure if command-line parameters perhaps only apply to the source. I am also not sure if both drives need this feature enabled to work properly (I just always did it that way).
If the answer to both those questions are "yes" then it may not actually work (for both drives) when set via commandline.

I added server_side_across_configs = true on rclone.conf, remove --drive-server-side-across-configs=true from the command, run, but no way. Failed to copy: googleapi: Error 404: File not found: 1koRdiGTlRTspAhIo6TzXDeewJvU8K1tyRh., notFound

about permission, i add second team drive, like Responsable, the first option, i guess the higher option.

this guy on youtube, make it with rclone

In english:
For the source drive you need at least viewer (or any of the levels above).
For the destination drive you need at least "contributor" (or "content manager" or "manager", which are higher)

If you set both to "content manager" (full read/write/delete access) then you will definitely have no problems from these permissions.

You have to translate to your native languge as best you can from that :slight_smile:
I expect the higher permissions are at the top of the list as they are in english.

I have no problems copying between my own teamdrives (on different domains) so it is definitely possible. We just have to identify the problem.

The 404 happens because of permissions. You'd need to turn them up so it works. You've already validated that information as you did a copy without the server side config and it worked.

The destination doesn't have enough permissions to the source.

1 Like

WORKS!

my bad, because i tried to add a member in TeamDrive1, so i can see my shared drive, but needed also to add a member in TeamDrive2.

Now infact works like a charm!

Transferred: 251.000G / 251.000 GBytes, 100%, 6.713 GBytes/s, ETA 0s

Many thanks! I hope that account with this migration, stay on, and don't get a strike from google.

Thanks a lot for this amazing support.

unfortunately in other thread, my upload speed, is very strange, also via browser now, i'll change isp, when is possible ( if is the isp )

Nice! Yes, it's required that the users are all members of the drives they need to access between so that makes sense.

Google shouldn't care if you transfer server-side. That's perfectly allowed.
Do be aware that you still will need to obey 750GB/day limit.
After that rclone will start to get only 403 errors and you will need to wait until next day to continue.
Still ... it beats manual reuploading :slight_smile:

1 Like

When come up 750GB/limit, can i run the same command for the folder that i transfer now, and skip automatically all same files ?

Or i need a flag in command to avoid copy same files and waste limit day ?

rclone will automatically skip any files that are already on the destination (checks file + modtime). So no problem! You can stop and start as much as you want.

You could also use --checksum to compare hash value instead (but this is not possible if both drives are encrypted - with different crypt keys as hashes will not match). This is usually overkill.

You probably will want to add --fast-list to make the drive-listing faster - if you have many files. Google drive supports this. (don't worry if it takes about a minute before you see transfer starting this is normal with --fast-sync as it lists everything first).

I also recommend you add --progress (or -P) so you can see the progress of copying - how much is left ect.

--checksum (probably not needed)
--fast-list
--progress

1 Like

Can you start a new post rather than hijack this thread and use the question template?

Thanks.