Job hangs for many hours at "Failed to Mkdir" due to "Folder Already Exists"

[jansan (Jan Sandorski)]

What is the problem you are having with rclone?

All data gets transferred correctly from Google Workspace Team Drive, but it then hangs for many hours, sometimes indefinitely. It seems like it tries to create folder fails, because "Folder Already Exists" but tries over and over again.

I am perfectly fine with mending the source code a bit to account for folders already being on the destination. Just need some high level guidance. Thank you

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0-DEV
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 6.5.0-1018-azure (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.1
- go/linking: dynamic
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Migrating data from Google Drive using service account and drive impersonate. However, same outcome when using Client ID and Secret and Oauth2 procedure.

Migrating to Egnyte. A custom built rclone version.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync GWUnlock:/ unlockegnyte:/Shared/Destination --fast-list --drive-impersonate user@email.com -P -vv --create-empty-src-dirs --log-file=reports/targetname/transfer.log

The rclone config contents with secrets removed.

[unlockegnyte]
type = egnyte
client_id = <REDACTED>
client_secret = <REDACTED>
domain = unlockhealth.egnyte.com
token = {"access_token":"<REDACTED>","token_type":"bearer","expires_in":-1,"expiry":"2024-06-20T18:13:00.773030Z"}
chunk_size = 274877906944
upload_cutoff = 274877906944

[GWUnlock]
type = drive
scope = drive
token = {"access_token":"<REDACTED>","token_type":"Bearer","refresh_token":"<REDACTED>","expiry":"2024-04-15T22:44:21.755222+02:00"}
service_account_file = sa.json
chunk_size = 274877906944
upload_cutoff = 274877906944
impersonate = <REDACTED>

A log from the command with the -vv flag

2024/04/17 10:32:48 ERROR : 2 - Accounting/Close Folders/2024/06.2024/10 - HealthAware/5 - Key Reports/For Signature: Failed to Mkdir: Folder already exists at this location
2024/04/17 10:32:48 DEBUG : 2 - Accounting/Close Folders/2024/01.2024/12 - Medicom/Revenue Tracker: Making directory
2024/04/17 10:35:10 DEBUG : pacer: low level retry 1/3 (error Folder already exists at this location)
2024/04/17 10:35:10 DEBUG : pacer: Rate limited, increasing sleep to 2m29.731082716s
2024/04/17 10:37:47 DEBUG : pacer: low level retry 2/3 (error Folder already exists at this location)
2024/04/17 10:37:47 DEBUG : pacer: Rate limited, increasing sleep to 27.696750803s
2024/04/17 10:40:16 DEBUG : pacer: low level retry 3/3 (error Folder already exists at this location)
2024/04/17 10:40:16 DEBUG : pacer: Rate limited, increasing sleep to 12.872360258s
2024/04/17 10:40:16 ERROR : 2 - Accounting/Close Folders/2024/01.2024/12 - Medicom/Revenue Tracker: Failed to Mkdir: Folder already exists at this location
2024/04/17 10:40:16 DEBUG : 2 - Accounting/Close Folders/2024/03.2024/7 - Quickbooks Exports: Making directory
2024/04/17 10:40:44 DEBUG : pacer: low level retry 1/3 (error Folder already exists at this location)
2024/04/17 10:40:44 DEBUG : pacer: Rate limited, increasing sleep to 20.312200949s
2024/04/17 10:40:57 DEBUG : pacer: low level retry 2/3 (error Folder already exists at this location)
2024/04/17 10:40:57 DEBUG : pacer: Rate limited, increasing sleep to 5.678771275s
2024/04/17 10:41:17 DEBUG : pacer: low level retry 3/3 (error Folder already exists at this location)
2024/04/17 10:41:17 DEBUG : pacer: Rate limited, increasing sleep to 1m19.573459761s
2024/04/17 10:41:17 ERROR : 2 - Accounting/Close Folders/2024/03.2024/7 - Quickbooks Exports: Failed to Mkdir: Folder already exists at this location
2024/04/17 10:41:17 DEBUG : 2 - Accounting/Close Folders/2024/09.2024/5 - Financial Reporting/Maranon Submission: Making directory
2024/04/17 10:41:23 DEBUG : pacer: low level retry 1/3 (error Folder already exists at this location)
2024/04/17 10:41:23 DEBUG : pacer: Rate limited, increasing sleep to 2m55.450157615s
2024/04/17 10:42:42 DEBUG : pacer: low level retry 2/3 (error Folder already exists at this location)
2024/04/17 10:42:42 DEBUG : pacer: Rate limited, increasing sleep to 33.473212694s
2024/04/17 10:45:38 DEBUG : pacer: low level retry 3/3 (error Folder already exists at this location)
2024/04/17 10:45:38 DEBUG : pacer: Rate limited, increasing sleep to 52.831412022s
2024/04/17 10:45:38 ERROR : 2 - Accounting/Close Folders/2024/09.2024/5 - Financial Reporting/Maranon Submission: Failed to Mkdir: Folder already exists at this location
2024/04/17 10:45:38 DEBUG : 2 - Accounting/Close Folders/2024/08.2024/4 - Expense Analysis: Making directory

[Full Example Log](https://jsandorskidemo.egnyte.com/dl/LGxrVayUsg)

[jansan (Jan Sandorski)]

I was able to work around this by removing the --create-empty-src-dirs flag. Still, the question remains. Can rclone check for the existence of a folder before trying to create it? Or identify the type of error it encountered to move on once a folder cannot be created because it is already in place? Or is this an Egnyte specific issue?

Maybe the http response codes would give clarification? I will try and investigate what error codes I get when trying to create a folder over an existing one.

[jansan (Jan Sandorski)]

Egnyte responds with a 403 Forbidden code http response and the following body:

{
    "errorMessage": "Folder already exists at this location"
}

How do other cloud providers respond when trying to create a folder over an existing one? I will keep investigating with Sharepoint, One Drive and Google and see if there is any consistency.

[jansan (Jan Sandorski)]

Apparently Box, DropBox do not allow for the creation of a folder if there is another one with the same name in the same location.

However, Sharepoint, OneDrive and GoogleDrive do allow for duplicate folders as they are using ids to differentiate them.

This calls for a check before copying/syncing empty folder with --create-empty-src-dirs flag. Thoughts?