Copying folder of Google Documents to other GDrive

[bnr0723]

So I have two Google Drives. One is educational and the other is my personal GSuite. My current task is essentially backing up all the “Shared With Me” files from edu to my GSuite. Most of these are Google Documents.

I want to preserve the Google formats since it is just going to another Drive space, and ideally I would like to be able to keep metadata like original creation date (not sure if this is possible). I haven’t thought of a way to keep track of who the original owner was.

If anyone can give me suggestions on those, please let me know. Here’s what I’ve already been trying, but has been failing. So I was able to successfully copy I think just about everything from my “Shared with Me” to its own folder in the same edu account. I thought it might be better to do this way, but I don’t know.

This is the command I used to copy from “Shared with Me” to a folder in the edu account:

rclone copy edu: --drive-shared-with-me edu:swmbackup -vv --tpslimit 5

What I am failing at doing is getting that folder to copy to my personal Gsuite.

Every time it fails and says it can’t find the files, even while it is displaying the file names. Here are two examples from the endless stream of errors:

2019/04/20 11:18:28 ERROR : Attempt 1/3 failed with 938 errors and: googleapi: Error 404: File not found: [protected]., notFound

and

2019/04/20 11:18:36 ERROR : "[protected].docx: Failed to copy: googleapi: Error 404: File not found: [protected]., notFound

And the command I used:

rclone copy edu:swmbackup "gsuite:Backups\swmbackup" -vv --tpslimit 5

All help is much appreciated!

Other information:

rclone v1.47.0
- os/arch: windows/amd64
- go version: go1.11

[Animosity022]

If I’m following, you copied them into your account without a problem but the latter command is failing?

Does rclone lsd edu:swmbackup -vv return anything?

[bnr0723]

Yes, I managed to copy them into my account, but it fails to copy these from my edu to my personal gsuite.

It returns an accurate listing of the directories in the edu:swmbackup directory.

[roobandev (Rooban)]

I have not used few edu accounts for few days. Tried syncing them today and i am seeing multiple 404 and 403. Seeing errors today only.

Also noticed terms of service updated most for gdrive products, not sure they changed anything

[Animosity022]

I am not sure on the direction of your command.

If you are trying to backup the items shared with you, you’d go the other way.

What’s your rclone.conf of the source GD that have it shared to and what is the destination conf?

[bnr0723]

Here is a diagram of what I’m trying to do:

rclone.conf:

[edu]
type = drive
token =
client_id =
client_secret =

[gsuite]
type = drive
client_id =
client_secret =
token =

Also, I just found this on the forum and at this point it might be the same issue I’m having.

[Animosity022]

Is the item shared with you shared to your edu account or your personal account?

If it is shared with the edu account, you’d want to do:

rclone copy --drive-shared-with-me edu:swmbackup gsuite:Backups -vv

I did a test with a xls file and it kept the format as well when I did a copy.

rclone copy --drive-shared-with-me GD:Test GD:Countries/Test -vv
2019/04/21 16:27:29 DEBUG : rclone: Version "v1.47.0" starting with parameters ["rclone" "copy" "--drive-shared-with-me" "GD:Test" "GD:Countries/Test" "-vv"]
2019/04/21 16:27:29 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2019/04/21 16:27:31 INFO  : Google drive root 'Countries/Test': Waiting for checks to finish
2019/04/21 16:27:31 INFO  : Google drive root 'Countries/Test': Waiting for transfers to finish
2019/04/21 16:27:37 INFO  : (Public) U22.17.0.1 - Mastery Checklist by ／u／Sekwah.xlsx: Copied (server side copy)
2019/04/21 16:27:37 INFO  :
Transferred:   	       off / off, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            1 / 1, 100%
Elapsed time:        7.4s

2019/04/21 16:27:37 DEBUG : 7 go routines active
2019/04/21 16:27:37 DEBUG : rclone: Version "v1.47.0" finishing with parameters ["rclone" "copy" "--drive-shared-with-me" "GD:Test" "GD:Countries/Test" "-vv"]

I removed the disable copy as I’m not getting any 404s.

[bnr0723]

Well I’m mainly dealing with stuff like Google Sheets. If I try the --disable copy flag, it downloads and uploads in Microsoft Excel format instead of remaining a Google Sheet. In most cases, this causes issues with formatting.

Otherwise trying to copy directly just keeps throwing those 404s like in that other post I linked to.

[Animosity022]

Did you try what I suggested?

I made a few Google Sheets and it copies fine as a google sheet.

rclone copy --drive-shared-with-me GD:Test GD:New -vv
2019/04/21 21:49:12 DEBUG : rclone: Version "v1.47.0" starting with parameters ["rclone" "copy" "--drive-shared-with-me" "GD:Test" "GD:New" "-vv"]
2019/04/21 21:49:12 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2019/04/21 21:49:13 INFO  : Google drive root 'New': Waiting for checks to finish
2019/04/21 21:49:13 INFO  : Google drive root 'New': Waiting for transfers to finish
2019/04/21 21:49:17 INFO  : TestSheet.xlsx: Copied (server side copy)
2019/04/21 21:49:18 INFO  : (Public) U22.17.0.1 - Mastery Checklist by ／u／Sekwah.xlsx: Copied (server side copy)
2019/04/21 21:49:18 INFO  :
Transferred:   	       off / off, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            2 / 2, 100%
Elapsed time:        5.6s

2019/04/21 21:49:18 DEBUG : 6 go routines active
2019/04/21 21:49:18 DEBUG : rclone: Version "v1.47.0" finishing with parameters ["rclone" "copy" "--drive-shared-with-me" "GD:Test" "GD:New" "-vv"]

Is there something else you are trying to do?

[bnr0723]

Well I wasn’t sure about two things. The .xlsx threw me off in your log, since that’s Excel but your screenshot shows a Google Sheets. Do you have this checked under settings in the Drive web interface?

Also, I’m trying to understand your command. Are you copying from a source folder that is shared with you called "Test? Or is it a normal folder in your main drive? I’m trying to copy everything under the shared with me tab: all files and all directories.

[Animosity022]

I have a folder called “Test” in one account. I shared that folder Test with my GSuite account.

rclone ls --drive-shared-with-me GD:Test
       -1 Test2.xlsx
       -1 TestSheet.xlsx
       -1 (Public) U22.17.0.1 - Mastery Checklist by ／u／Sekwah.xlsx

Once I have the folder shared, I can copy it to another folder which I called “New” or anything for that matter.

[felix@gemini ~]$ rclone ls --drive-shared-with-me GD:Test
       -1 Test2.xlsx
       -1 TestSheet.xlsx
       -1 (Public) U22.17.0.1 - Mastery Checklist by ／u／Sekwah.xlsx
[felix@gemini ~]$ rclone ls GD:Test
2019/04/21 22:08:18 Failed to ls: directory not found
[felix@gemini ~]$ rclone copy  --drive-shared-with-me GD:Test GD:New
[felix@gemini ~]$ rclone ls GD:New
       -1 Test2.xlsx
       -1 TestSheet.xlsx
       -1 (Public) U22.17.0.1 - Mastery Checklist by ／u／Sekwah.xlsx

I made 2 Google Sheets Documents using the web interface and they get created with a .xlsx extension. If there is a way to change that, I’m not sure as I don’t use the apps.

[bnr0723]

My goal was to copy the folder from one Drive to the other, but it seems at this point that’s just not working properly.

I tried what you did with sharing. The folder in my edu that had the copies of all shared items, I shared that one to the gsuite Drive, then tried the following command

rclone copy gsuite:swmbackup --drive-shared-with-me gsuite:Backups/swmbackup -vv --tpslimit 5

So far, everything is transferring fine. Although this is a workaround to my original goal, I am greatly appreciative of the idea. Thank you!