Can't convert Gsheet to Xlsx

What is the problem you are having with rclone?

I'm attempting to convert all Google format files to Microsoft formats. I'm downloading from Google Shared Drives to local folders on WIndows.

What is your rclone version (output from rclone version)

rclone version
rclone v1.53.1

  • os/arch: windows/amd64
  • go version: go1.15

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows Server 2016 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive and specifically Shared Drives

The command you were trying to run (eg rclone copy /tmp remote:tmp)

\\\rclone --drive-team-drive "$TeamdriveID" --include "{*.gsheet,*.gdoc,*.gslides,*.gdraw,*.gtable,*.gform}" copy gdrives: "$Destination" #--dry-run \\\

The rclone config contents with secrets removed.

[gdrives]
type = drive
client_id = XXXXXXXXXXXX-dddidgdddbdqd2dddhdfddd8ddddd9dk.apps.googleusercontent.com
client_secret = dWdtdOd2dDdddZdVd1d6dzdf
scope = drive
token = {"access_token":"dad9d0dfd6dMdZdbdOd0d9dYdJdKdbdrdSdLdodfdTdUdBdHdEdrdjd--fAfLfzfdf8fGf1fFf3fVfofDfFfdfdftfffYf_fEfnfNfofgf0fJfLfVfgf9fIf5fzfdfffRfvfBfIfJfhfBf8fSfuf9fqfPftf-gWgRgWgAg3gkgg5","token_type":"Bearer","refresh_token":"1//09o9s9p9C9E9N9g9I9R9A9A9S9w9-L99r9j9O9n9s959o9b9Y9h9M9H979_3919X979b9n9j_7909B9K9R9-V9i9o9u9L9f9N9a9W9","expiry":"2020-10-01T13:56:14.227539Z"}
team_drive = 0000000000000000000
root_folder_id =
export_formats = docx,xlsx,pptx,svg

A log from the command with the -vv flag

Since I'm attempting download of entire Team Drive, the log is massive. I tried to bend the command to download just a single file but ran out of time. I may loop around with some more tries and get you a log to review.

Paste  log here

hello and welcome to the forum,

for the files that are downloaded, in what format are they?

Windows shows them as filename.gsheet or whatever format they are originally. I expected him to arrive as filename.xlsx per documentation. Have I misunderstood?

can you download just one file and post the entire log?

Working on it. Stand by.

i am able to create a sheet in gdrive, download it to windows computer.

i do not have this in my config file
export_formats = docx,xlsx,pptx,svg

rclone.exe ls gdrive-a1b2:/x  
       -1 x.xlsx

rclone.exe copy gdrive-a1b2:/x C:\ex\x -vv 
2020/10/01 11:30:34 DEBUG : rclone: Version "v1.53.0" starting with parameters ["c:\\data\\rclone\\scripts\\rclone.exe" "copy" "gdrive-a1b2:/x" "C:\\ex\\x" "-vv"]
2020/10/01 11:30:36 DEBUG : Local file system at //?/C:/ex/x: File to upload is small (4679 bytes), uploading instead of streaming
2020/10/01 11:30:36 DEBUG : x.xlsx: MD5 = 33285d5f64a828cc0dfcfb31aec2cf05 OK
2020/10/01 11:30:36 INFO  : x.xlsx: Copied (new)
2020/10/01 11:30:36 DEBUG : x.xlsx: Updating size of doc after download to 4679
2020/10/01 11:30:36 INFO  : x.xlsx: Copied (Rcat, new)

Interesting. Even your listing of the directory (LS) shows the file as xlsx when you created it as a gsheet. I believe mine would show the file as x.gsheet. Trying to reproduce.

As for the export_formats directive. Mine was originally blank also because that's the default. When export as XLSX didn't work I edited the config to add it incase that was the issue.

I found something interesting.
I created a Shared drive called xxtest and a file "Ducks Fly in Formation" as a Gsheet.

If I run this:

rclone --drive-team-drive "0AFUW00000000Uk9PVA" ls gdrives:
       -1 Ducks Fly in Formation.xlsx

However if I run this I get no results.

rclone --drive-team-drive "0AFUW00000000Uk9PVA" ls gdrives: --include "{*.gsheet,*.gdoc,*.gslides,*.gdraw,*.gtable,*.gform}"

My assumption is that rclone is on the fly interpreting the *.gsheet as XLSX meaning when I filter to include the Google Native formats I actually get nothing. However, I guarantee you I downloaded something with the gsheet extension previously with this:

rclone -vv --drive-team-drive "0AFUW00000000Uk9PVA" --include "{*.gsheet,*.gdoc,*.gslides,*.gdraw,*.gtable,*.gform}" copy gdrives: $Destination

I can't make downloading them with copy work.

So I guess, new question; how can I selectively download only the Google Format docs and have them arrive in Microsoft formats?

Oh and the log.

rclone --drive-team-drive "0AFUW1kpIvcByUk9PVA" copy gdrives: $Destination --include "{*.gsheet,*.gdoc,*.gslides,*.gdraw,*.gtable,*.gform}" -vv
rclone : 2020/10/02 18:26:38 DEBUG : rclone: Version "v1.53.1" starting with parameters 
["F:\\rclone-v1.53.1-windows-amd64\\rclone.exe" "--drive-team-drive" "0AFUW1kpIvcByUk9PVA" "copy" "gdrives:" 
"F:\\GdocFormats\\xxtest\\" "--include" "{*.gsheet,*.gdoc,*.gslides,*.gdraw,*.gtable,*.gform}" "-vv"]
At line:1 char:1
+ rclone --drive-team-drive "0AFUW1kpIvcByUk9PVA" copy gdrives: $Destin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (2020/10/02 18:2....gform}" "-vv"]:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
2020/10/02 18:26:38 DEBUG : Using config file from "C:\\Users\\kla-admin\\.config\\rclone\\rclone.conf"
2020/10/02 18:26:38 DEBUG : Creating backend with remote "gdrives:"
2020/10/02 18:26:38 DEBUG : Creating backend with remote "F:\\GdocFormats\\xxtest\\"
2020/10/02 18:26:38 DEBUG : fs cache: renaming cache item "F:\\GdocFormats\\xxtest\\" to be canonical "//?/F:/GdocFormats/xxtest/"
2020/10/02 18:26:39 DEBUG : Ducks Fly in Formation.xlsx: Excluded
2020/10/02 18:26:39 DEBUG : Local file system at //?/F:/GdocFormats/xxtest/: Waiting for checks to finish
2020/10/02 18:26:39 DEBUG : Local file system at //?/F:/GdocFormats/xxtest/: Waiting for transfers to finish
2020/10/02 18:26:39 INFO  : There was nothing to transfer
2020/10/02 18:26:39 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Elapsed time:         0.7s
2020/10/02 18:26:39 DEBUG : 5 go routines active

i think the docs are clear
"Note that rclone adds the extension to the google doc, so if it is called My Spreadsheet on google docs, it will be exported as My Spreadsheet.xlsx or My Spreadsheet.pdf etc"

"When rclone downloads a Google doc it chooses a format to download depending upon the --drive-export-formats setting"

rclone.exe ls gdrive-a1b2:x  
  8588678 x.log
       -1 x.xlsx

rclone.exe copy gdrive-a1b2:x C:\ex\x -vv --include=*.xlsx 
2020/10/02 15:27:07 DEBUG : rclone: Version "v1.53.0" starting with parameters ["c:\\data\\rclone\\scripts\\rclone.exe" "copy" "gdrive-a1b2:x" "C:\\ex\\x" "-vv" "--include=*.xlsx"]
2020/10/02 15:27:07 DEBUG : x.log: Excluded
2020/10/02 15:27:09 DEBUG : x.xlsx: MD5 = 9359b28c6665db9616ddf7a1707917a6 OK
2020/10/02 15:27:09 INFO  : x.xlsx: Copied (new)
2020/10/02 15:27:09 DEBUG : x.xlsx: Updating size of doc after download to 4679
2020/10/02 15:27:09 INFO  : x.xlsx: Copied (Rcat, new)
2020/10/02 15:27:09 INFO  : 
Transferred:   	    9.139k / 9.139 kBytes, 100%, 6.653 kBytes/s, ETA 0s
Transferred:            3 / 3, 100%
Elapsed time:         2.0s

then once the file is local, i can open it with excel.

but rclone claims to have transferred three files, so something is going on behind the scenes.

if you try to download such a file from gdrive website, gdrive auto-converts it to .xslx and then downloads the file.

Understood. My goal was to download only the Google formatted files but according to this I can't even target them as

--include "{*.gsheet,*.gdoc,*.gslides,*.gdraw,*.gtable,*.gform}"

How can I select just the Google format documents to download?

take a look at my last example.

Right but if I set filter to include *.xlsx I'm going to get all the actual XLSX files, not just the Gsheet files being converted.

Best I can think is to get the full path to "originalfilename.gsheet" file and then individually download them as "originalfilename.xlsx".

as i understand it, .gsheet is a pointer to internal data on google servers.
that you cannot get that raw data file.

here are two files in gdrive, one is a real file x.xlsx and the other is x, a google sheet

image

and this will not download the real .xslx, only download a pointer to the gsheet.

rclone.exe ls gdrive-a1b2:x  
     8559 x.xlsx
       -1 x.xlsx

rclone.exe copy gdrive-a1b2:x C:\ex\x -vv --drive-export-formats=url --exclude=*.xlsx 
x.xlsx: Excluded
x.url: Copied (new)

Hmm. I'm looping through ~260 Google Shared Drives attempting to fetch all the Google formated stuff. I really can't use exclude filtering I think. What you've shown implies that Gsheet documents really don't have an extension since they are just pointers. Would this work to get extensionless files?

rclone copy gdrives: $destination --exclude *.*

I think that might give me more than just Gsheet, Gdocs, and Gslides but maybe I'm ok with a little mess if I can get the files.

hi,
create a test folder, with a mix of files and then you can test your command.

That didn't work. It excluded all files.
I tried the reverse as well

--include *.xlsx

I get the Gsheet and real XLSX files. :frowning:
Because rclone wants to allow you to download as the format you specifiy, it obscures the actual file type from you. I wonder if I can set the format to something I know I don't actually have like Open Office, search for only those but tell the copy command to get them as Microsoft format.

this downloads a file with the needed info inside.
you can parse that file.

That's a thought. I get a shortcut with the URL to the file. Any idea how to tell just from this shortcut format which ones are gsheet vs gdoc vs gslides vs etc...?

yes,

in my testing, i found --drive-export-formats=desktop to be the easiest to parse.
it is in a .ini format and i already use that format with my python script i use to control rclone.

in the example below,
the name of the google document is Name=x
the type of document is from the url spreadsheets, document, presentation and so on

[Desktop Entry]
Encoding=UTF-8
Name=x
URL=https://docs.google.com/spreadsheets/d/1l-gUAA10tp9Ft6Qcj2DMJauwRXi2sbIVnVgY2JwfGq4/edit?usp=drivesdk
Icon=text-html
Type=Link
[Desktop Entry]
Encoding=UTF-8
Name=y
URL=https://docs.google.com/document/d/1K4IBXcGWsdYE5Sk-SjrRtZB4lBzVx8nLjNprRkJd9OY/edit?usp=drivesdk
Icon=text-html
Type=Link
[Desktop Entry]
Encoding=UTF-8
Name=z
URL=https://docs.google.com/presentation/d/1VxY5C2iQHQtiSwCbPnB5DwDDoqkDQ-WhWxjPWbui6iE/edit?usp=drivesdk
Icon=text-html
Type=Link