How to delete undeleted blobs on Azure

What is the problem you are having with rclone?

When running rclone delete on an Azure container that contains previously undeleted blobs, I get the following error:

RESPONSE 409: 409 This operation is not permitted because the blob has snapshots.
ERROR CODE: SnapshotsPresent

The container has soft delete, but no versioning. The snapshots are down to the soft delete process - the blobs can be removed via the portal, but only if 'Also delete blob snapshots' is ticked. Is it possible to replicate this in rclone delete?

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.1

  • os/version: darwin 13.1 (64 bit)
  • os/kernel: 22.2.0 (arm64)
  • os/type: darwin
  • os/arch: arm64 (ARMv8 compatible)
  • go/version: go1.21.5
  • go/linking: dynamic
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Azure

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone delete az:<container_name>

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[az]
type = azureblob
env_auth = false
sas_url = XXX

A log from the command that you were trying to run with the -vv flag

RESPONSE 409: 409 This operation is not permitted because the blob has snapshots.
ERROR CODE: SnapshotsPresent

Thank you!

It looks like from the azure blob docs that this is possible

x-ms-delete-snapshots: {include, only} Required if the blob has associated snapshots. Specify one of the following options:

  • include: Delete the base blob and all of its snapshots.
  • only: Delete only the blob's snapshots, and not the blob itself.

Specify this header only for a request against the base blob resource. If this header is specified on a request to delete an individual snapshot, Blob Storage returns status code 400 (Bad Request).

If this header isn't specified on the request and the blob has associated snapshots, Blob Storage returns status code 409 (Conflict).

This would be easy to implement. There is even a little hint in the source

This should probably be controlled by a flag/config option (say --azureblob-delete-snapshots) this could have 3 potential states:

  1. error: give 409 error like it does now when trying to delete blob with snapshots
  2. only: remove only the blob and not the snapshots
  3. include: remove the blob and the snapshots

Whether we need the 1. option I don't know and what the default should be I don't know.

What do you think?

Is there any prior art from microsoft tools (azcopy or similar) that we can learn from to make the tools consistent?

Thanks for the quick response. Your proposed solution sounds fine to me, default to stay as-is and an option via flag to delete snapshots.

For comparison, azcopy has a flag --delete-snapshots=include

--delete-snapshots (string) By default, the delete operation fails if a blob has snapshots. Specify 'include' to remove the root blob and all its snapshots; alternatively specify 'only' to remove only the snapshots but keep the root blob.

Reference:

I've had a go at implementing this - please give it a go :slight_smile:

v1.66.0-beta.7603.52e4e150d.fix-azure-delete-snapshots on branch fix-azure-delete-snapshots (uploaded in 15-30 mins)

--azureblob-delete-snapshots

Set to specify how to deal with snapshots on blob deletion.

Properties:

  • Config: delete_snapshots
  • Env Var: RCLONE_AZUREBLOB_DELETE_SNAPSHOTS
  • Type: string
  • Required: false
  • Choices:
    • ""
      • By default, the delete operation fails if a blob has snapshots
    • "include"
      • Specify 'include' to remove the root blob and all its snapshots
    • "only"
      • Specify 'only' to remove only the snapshots but keep the root blob.

I tried the beta, and both the "only" and "include" options work exactly as expected (and default fails with the same error as before). This is excellent, thanks so much for the quick fix!

Thanks for testing :slight_smile:

I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.66

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.