Getting this Warning: running mkdir on a remote which can't have empty directories does nothing

Help and Support
1

What is the problem you are having with rclone?

  1. rclone is not able to create an empty folder but other programs can.
    note that the following can create persistent empty folders
  • s3 browser
  • cloudberry explorer
  • aws s3 website
  • wasabi website
  1. based on log file, rclone states that it created the folder, but really it does not create the folder.

What is your rclone version (output from rclone version)

rclone v1.53.1
- os/arch: windows/amd64
- go version: go1.15

Which OS you are using and how many bits (eg Windows 7, 64 bit)

win10.2004.64

Which cloud storage system are you using? (eg Google Drive)

aws s3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone.exe mkdir aws01:vserver03.veeam.br.en07/en07/rclone/archive -vv

The rclone config contents with secrets removed.

[aws01]
type = s3
provider = AWS
access_key_id = redacted
secret_access_key = redacted
region = us-east-1

A log from the command with the -vv flag

2020/09/20 12:19:45 DEBUG : rclone: Version "v1.53.0" starting with parameters ["c:\\data\\rclone\\scripts\\rclone.exe" "mkdir" "aws01:vserver03.veeam.br.en07/en07/rclone/archive" "-vv"]
2020/09/20 12:19:45 DEBUG : Creating backend with remote "aws01:vserver03.veeam.br.en07/en07/rclone/archive"
2020/09/20 12:19:45 DEBUG : Using config file from "c:\\data\\rclone\\scripts\\a.conf"
2020/09/20 12:19:46 NOTICE: S3 bucket vserver03.veeam.br.en07 path en07/rclone/archive: Warning: running mkdir on a remote which can't have empty directories does nothing
2020/09/20 12:19:46 DEBUG : S3 bucket vserver03.veeam.br.en07 path en07/rclone/archive: Making directory
2020/09/20 12:19:46 INFO  : S3 bucket vserver03.veeam.br.en07 path en07/rclone/archive: Bucket "vserver03.veeam.br.en07" created with ACL "private"
2020/09/20 12:19:46 DEBUG : 4 go routines active

tho rclone cannot seem to create an empty folder, it has no problem via lsd to see that archive folder.

rclone.exe lsd -R aws01:vserver03.veeam.br.en07 
           0 2020-09-20 13:59:06        -1 en07
           0 2020-09-20 13:59:06        -1 en07/rclone
           0 2020-09-20 13:59:06        -1 en07/rclone/archive
           0 2020-09-20 13:59:06        -1 en07/rclone/backup
           0 2020-09-20 13:59:06        -1 en07/rclone/backup/VeeamAgentUser3bef6c4c-360f-11b2-a85c-9070b4e81f40
           0 2020-09-20 13:59:06        -1 en07/rclone/backup/VeeamAgentUser3bef6c4c-360f-11b2-a85c-9070b4e81f40/ABP_EN07_-_en07.local
2

Bucket based cloud providers do not really have a concept of "folders". All they have are blobs. On their web UI, they present a sense of a filesystem to the user, but it is just for representational purposes.

To create "empty folders" on such destinations, 0-byte directory markers with the directory name have to be created.

See https://github.com/rclone/rclone/issues/1837 and https://github.com/rclone/rclone/issues/3453

3

thanks i am aware of that 0-byte maker.

given that amazon itself allows for the creation a persistent empty folders, rclone should have that option.

in this On s3, rclone should create persistent empty folders.
@ncw wrote "This should maybe be an option --s3-directory-markers ? Then the user can choose"

4

It should. That's why https://github.com/rclone/rclone/issues/3453 is still open.
It just needs to be taken up and completed by someone. I see activity in that issue, but it seems it was never completed.

5

yes, this is a friendly reminder.

rclone is almost there.

rclone can lsd the folder
now rclone needs to mkdir and sync the folder

1 Like
6

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.