Google Drive --create-empty-src-dirs confusingly slow

Help and Support
1

rclone v1.57.0

  • os/version: Microsoft Windows Server 2016 Standard 1607 (64 bit)
  • os/kernel: 10.0.14393.4825 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.17.2
  • go/linking: dynamic
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Google Drive Enterprise

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone -i sync path remote: --create-empty-src-dirs -v

The rclone config contents with secrets removed.

[TBG_Products]
type = drive
client_id = 
client_secret = 
scope = drive
token = 
team_drive = 
root_folder_id =

A log from the command with the -vv flag

No logs that indicate anything other than a successful creation of the folder.

We're mirroring our Windows File Server to Google Drive and one of the directory structures involves the creation of an inordinate number of empty folders. The process of instantiating these folders is taking 2 seconds for each folder, one at a time, whereas actual files would transfer comparatively instantaneously and in parallel. I would think this part would be faster, but evidently not?

2

hello and welcome to the forum,

rclone has to wait for gdrive to create the dir and that is where the gdrive slowness comes in.
if you try to delete that same set of dest dirs, you would see that same gdrive slowness.

here is the same command, once for gdrive, once for wasabi, s3 clone known for hot storage

rclone sync d:\source\dir gdrive:zork -vv --create-empty-src-dirs 
2022/03/16 19:54:47 DEBUG : one/two/three/four/four.01: Making directory
2022/03/16 19:54:51 DEBUG : one/two/three/four/four.02: Making directory
2022/03/16 19:54:52 DEBUG : one/two/three/four/four.03: Making directory
2022/03/16 19:54:53 DEBUG : one/two/three/four/four.04: Making directory

rclone sync d:\source\dir wasabi01:zork -vv --create-empty-src-dirs 
2022/03/16 19:54:54 DEBUG : one/two/three/four/four.01: Making directory
2022/03/16 19:54:54 DEBUG : one/two/three/four/four.02: Making directory
2022/03/16 19:54:54 DEBUG : one/two/three/four/four.03: Making directory
2022/03/16 19:54:54 DEBUG : one/two/three/four/four.04: Making directory
3

Whelp. It is what it is I suppose. Thanks for the confirmation.

4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.