Using Azure Blob with Hierarchical Name Space (HNS/ADLS2) RClone does not support real folders.
With 'minimal' config, creating blobs/files in directories works. Creating empty directories does not.
Enabling "azureblob-directory-markers" does allow for the creation of empty directories though. This shouldn't be required for ADLS as it's creating a blob called / in the virtual directory.
Trying to delete an empty directory from Bash sometimes returns:
rmdir: failed to remove 'test/': Input/output error
Logging shows:
2024/02/16 22:52:04 ERROR : test/: Dir.Remove failed to remove directory: removing directory marker failed: DELETE https://<storageaccount>.blob.core.windows.net/<container>/test/
--------------------------------------------------------------------------------
RESPONSE 400: 400 The requested URI does not represent any resource on the server.
ERROR CODE: InvalidUri
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidUri</Code><Message>The requested URI does not represent any resource on the server.
RequestId:cb99848a-801e-00c3-6f2a-61b155000000
Time:2024-02-16T22:52:04.7683483Z</Message></Error>
--------------------------------------------------------------------------------
2024/02/16 22:52:04 ERROR : IO error: removing directory marker failed: DELETE https://<storageaccount>.blob.core.windows.net/<container>/test/
--------------------------------------------------------------------------------
RESPONSE 400: 400 The requested URI does not represent any resource on the server.
ERROR CODE: InvalidUri
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidUri</Code><Message>The requested URI does not represent any resource on the server.
RequestId:cb99848a-801e-00c3-6f2a-61b155000000
To me, it looks like the appended / that's causing the problem.
Additionally, timestamps are ignored. Blob and directory timestamps are not modifiable by the client. RClone uses the mtime tag to assign a modifiable value to blobs. However, it does not do this for directories and ignores the value if it is present. Therefore all folders show as modified 2000/01/01 00:00:00.
Blobfuse2 uses a dedicated configuration option for ADLS2 which is able to handle creating, deleting and reporting timestamps for folders. However, it simply presents what is in the Container and doesn't have any facility for changing timestamps.
I think this is because of the wide discrepancy between the S3 standard and ADLS2, and realise it's probably a big ask for it to be addressed. But does someone have the inclination to assist in resolving this?
I've most recently tested this on:
rclone v1.65.2
- os/version: almalinux 9.2 (64 bit)
- os/kernel: 5.14.0-284.25.1.el9_2.aarch64 (aarch64)
- os/type: linux
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.21.6
- go/linking: static
- go/tags: none