Can't delete a folder at the root

rclone v1.55.1
- os/type: linux
- os/arch: amd64
- go/version: go1.16.3
- go/linking: static
- go/tags: none

I can't delete a folder, no matter what command I use.

  1. Check if folder has content. It returns nothing.

rclone lsl Storj:archamps

  1. Issue these commands:

rclone purge Storj:archamps
2021/05/19 12:54:27 ERROR : : Failed to rmdir: directory not empty
2021/05/19 12:54:27 ERROR : Attempt 1/3 failed with 1 errors and: directory not empty
2021/05/19 12:54:27 ERROR : : Failed to rmdir: directory not empty
2021/05/19 12:54:27 ERROR : Attempt 2/3 failed with 1 errors and: directory not empty
2021/05/19 12:54:28 ERROR : : Failed to rmdir: directory not empty
2021/05/19 12:54:28 ERROR : Attempt 3/3 failed with 1 errors and: directory not empty
2021/05/19 12:54:28 Failed to purge: directory not empty

rclone rmdir Storj:archamps
2021/05/19 12:54:41 ERROR : Attempt 1/3 failed with 1 errors and: directory not empty
2021/05/19 12:54:41 ERROR : Attempt 2/3 failed with 1 errors and: directory not empty
2021/05/19 12:54:41 ERROR : Attempt 3/3 failed with 1 errors and: directory not empty
2021/05/19 12:54:41 Failed to rmdir: directory not empty

rclone rmdirs Storj:archamps
2021/05/19 12:54:45 ERROR : : Failed to rmdir: directory not empty
2021/05/19 12:54:45 ERROR : Attempt 1/3 failed with 1 errors and: directory not empty
2021/05/19 12:54:45 ERROR : : Failed to rmdir: directory not empty
2021/05/19 12:54:45 ERROR : Attempt 2/3 failed with 1 errors and: directory not empty
2021/05/19 12:54:45 ERROR : : Failed to rmdir: directory not empty
2021/05/19 12:54:45 ERROR : Attempt 3/3 failed with 1 errors and: directory not empty
2021/05/19 12:54:45 Failed to rmdirs: directory not empty

What's your rclone.conf with keys/passwords removed?

Can you run the command with -vv and share the full output...

rclone rmdirs Storj:archamps -vv
2021/05/19 13:50:43 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2021/05/19 13:50:43 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "rmdirs" "Storj:archamps" "-vv"]
2021/05/19 13:50:43 DEBUG : Creating backend with remote "Storj:archamps"
2021/05/19 13:50:43 DEBUG : FS sj://archamps: connecting...
2021/05/19 13:50:43 DEBUG : FS sj://archamps: connected: <nil>
2021/05/19 13:50:43 DEBUG : FS sj://archamps: ls ./
2021/05/19 13:50:43 DEBUG : FS sj://archamps: OBJ ls ./ ("archamps", "")
2021/05/19 13:50:43 DEBUG : FS sj://archamps: opts &{Prefix: Cursor: Recursive:false System:true Custom:true}
2021/05/19 13:50:43 DEBUG : FS sj://archamps: Removing directory
2021/05/19 13:50:43 DEBUG : FS sj://archamps: rmdir ./
2021/05/19 13:50:44 ERROR : : Failed to rmdir: directory not empty
2021/05/19 13:50:44 ERROR : Attempt 1/3 failed with 1 errors and: directory not empty
2021/05/19 13:50:44 DEBUG : FS sj://archamps: ls ./
2021/05/19 13:50:44 DEBUG : FS sj://archamps: OBJ ls ./ ("archamps", "")
2021/05/19 13:50:44 DEBUG : FS sj://archamps: opts &{Prefix: Cursor: Recursive:false System:true Custom:true}
2021/05/19 13:50:44 DEBUG : FS sj://archamps: Removing directory
2021/05/19 13:50:44 DEBUG : FS sj://archamps: rmdir ./
2021/05/19 13:50:45 ERROR : : Failed to rmdir: directory not empty
2021/05/19 13:50:45 ERROR : Attempt 2/3 failed with 1 errors and: directory not empty
2021/05/19 13:50:45 DEBUG : FS sj://archamps: ls ./
2021/05/19 13:50:45 DEBUG : FS sj://archamps: OBJ ls ./ ("archamps", "")
2021/05/19 13:50:45 DEBUG : FS sj://archamps: opts &{Prefix: Cursor: Recursive:false System:true Custom:true}
2021/05/19 13:50:45 DEBUG : FS sj://archamps: Removing directory
2021/05/19 13:50:45 DEBUG : FS sj://archamps: rmdir ./
2021/05/19 13:50:45 ERROR : : Failed to rmdir: directory not empty
2021/05/19 13:50:45 ERROR : Attempt 3/3 failed with 1 errors and: directory not empty
2021/05/19 13:50:45 DEBUG : 5 go routines active
2021/05/19 13:50:45 Failed to rmdirs: directory not empty

rclone.conf (key is redacted)

[Storj]
type = tardigrade
access_grant = 1cCYAHiNyogB3zhhxnLXXX

This is the right command to remove a directory and its contents recursively in an efficient way...

Can you try this one with some more debug?

rclone purge Storj:archamps -vv --dump bodies --retries 1 --low-level-retries 1
# rclone purge Storj:archamps -vv --dump bodies --retries 1 --low-level-retries 1
2021/05/19 17:06:42 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2021/05/19 17:06:42 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "purge" "Storj:archamps" "-vv" "--dump" "bodies" "--retries" "1" "--low-level-retries" "1"]
2021/05/19 17:06:42 DEBUG : Creating backend with remote "Storj:archamps"
2021/05/19 17:06:42 DEBUG : FS sj://archamps: connecting...
2021/05/19 17:06:42 DEBUG : FS sj://archamps: connected: <nil>
2021/05/19 17:06:42 DEBUG : Waiting for deletions to finish
2021/05/19 17:06:42 DEBUG : FS sj://archamps: ls -R ./
2021/05/19 17:06:42 DEBUG : FS sj://archamps: OBJ ls -R ./ ("archamps", "")
2021/05/19 17:06:43 DEBUG : FS sj://archamps: ls ./
2021/05/19 17:06:43 DEBUG : FS sj://archamps: OBJ ls ./ ("archamps", "")
2021/05/19 17:06:43 DEBUG : FS sj://archamps: opts &{Prefix: Cursor: Recursive:false System:true Custom:true}
2021/05/19 17:06:43 DEBUG : FS sj://archamps: Removing directory
2021/05/19 17:06:43 DEBUG : FS sj://archamps: rmdir ./
2021/05/19 17:06:44 ERROR : : Failed to rmdir: directory not empty
2021/05/19 17:06:44 ERROR : Attempt 1/1 failed with 1 errors and: directory not empty
2021/05/19 17:06:44 DEBUG : 5 go routines active
2021/05/19 17:06:44 Failed to purge: directory not empty

The directory lister really believes there is a file in there...

Can you do this to just check there are no directories

rclone lsf -R Storj:archamps -vv --dump bodies --retries 1 --low-level-retries 1

And also try a non-recursive list

rclone lsf Storj:archamps -vv --dump bodies --retries 1 --low-level-retries 1

There is something odd going on, but I haven't put my finger on it yet!

Here it is:

# rclone lsf -R Storj:archamps -vv --dump bodies --retries 1 --low-level-retries 1
2021/05/19 21:17:02 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2021/05/19 21:17:02 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "lsf" "-R" "Storj:archamps" "-vv" "--dump" "bodies" "--retries" "1" "--low-level-retries" "1"]
2021/05/19 21:17:02 DEBUG : Creating backend with remote "Storj:archamps"
2021/05/19 21:17:02 DEBUG : FS sj://archamps: connecting...
2021/05/19 21:17:02 DEBUG : FS sj://archamps: connected: <nil>
2021/05/19 21:17:02 DEBUG : FS sj://archamps: ls -R ./
2021/05/19 21:17:02 DEBUG : FS sj://archamps: OBJ ls -R ./ ("archamps", "")
2021/05/19 21:17:03 DEBUG : 5 go routines active

and

# rclone lsf Storj:archamps -vv --dump bodies --retries 1 --low-level-retries 1
2021/05/19 21:17:43 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2021/05/19 21:17:43 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "lsf" "Storj:archamps" "-vv" "--dump" "bodies" "--retries" "1" "--low-level-retries" "1"]
2021/05/19 21:17:43 DEBUG : Creating backend with remote "Storj:archamps"
2021/05/19 21:17:43 DEBUG : FS sj://archamps: connecting...
2021/05/19 21:17:43 DEBUG : FS sj://archamps: connected: <nil>
2021/05/19 21:17:43 DEBUG : FS sj://archamps: ls ./
2021/05/19 21:17:43 DEBUG : FS sj://archamps: OBJ ls ./ ("archamps", "")
2021/05/19 21:17:43 DEBUG : FS sj://archamps: opts &{Prefix: Cursor: Recursive:false System:true Custom:true}
2021/05/19 21:17:44 DEBUG : 5 go routines active

Not a sausage!

Did you try the latest beta? That has a later storj/tardigrade library.

I'm suspecting either a bug in the library or a bug in the backend now.

Having undeletable buckets is not unheard of (this happens quite often in QingStor and I have to email support to get them to delete buckets for me).

Nop, same behaviour.

rclone v1.56.0-beta.5519.07f2f3a62 has successfully installed.
Now run "rclone config" for setup. Check https://rclone.org/docs/ for more details.

# rclone purge Storj:archamps -v
2021/05/21 10:55:04 ERROR : : Failed to rmdir: directory not empty
2021/05/21 10:55:04 ERROR : Attempt 1/3 failed with 1 errors and: directory not empty
2021/05/21 10:55:04 ERROR : : Failed to rmdir: directory not empty
2021/05/21 10:55:04 ERROR : Attempt 2/3 failed with 1 errors and: directory not empty
2021/05/21 10:55:04 ERROR : : Failed to rmdir: directory not empty
2021/05/21 10:55:04 ERROR : Attempt 3/3 failed with 1 errors and: directory not empty
2021/05/21 10:55:04 Failed to purge: directory not empty

Are there any other tools you can look into that bucket with and see if there are files in it?

If nothing shows any files then I think you'll have to contact support and ask them.

Storj supplied the initial rclone backend so they should be friendly to rclone help requests.

I've also tried using their Hosted Gateway, same behaviour:

# rclone purge StorjS3:archamps -vv
2021/05/21 11:02:49 DEBUG : rclone: Version "v1.56.0-beta.5519.07f2f3a62" starting with parameters ["rclone" "purge" "StorjS3:archamps" "-vv"]
2021/05/21 11:02:49 DEBUG : Creating backend with remote "StorjS3:archamps"
2021/05/21 11:02:49 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2021/05/21 11:02:49 DEBUG : Waiting for deletions to finish
2021/05/21 11:02:49 DEBUG : S3 bucket archamps: Removing directory
2021/05/21 11:02:50 ERROR : : Failed to rmdir: BucketNotEmpty: The bucket you tried to delete is not empty
        status code: 409, request id: 1681097C250CCAF6, host id:
2021/05/21 11:02:50 ERROR : Attempt 1/3 failed with 1 errors and: BucketNotEmpty: The bucket you tried to delete is not empty
        status code: 409, request id: 1681097C250CCAF6, host id:
2021/05/21 11:02:50 DEBUG : Waiting for deletions to finish
2021/05/21 11:02:50 DEBUG : S3 bucket archamps: Removing directory
2021/05/21 11:02:51 ERROR : : Failed to rmdir: BucketNotEmpty: The bucket you tried to delete is not empty
        status code: 409, request id: 1681097C73EB1C66, host id:
2021/05/21 11:02:51 ERROR : Attempt 2/3 failed with 1 errors and: BucketNotEmpty: The bucket you tried to delete is not empty
        status code: 409, request id: 1681097C73EB1C66, host id:
2021/05/21 11:02:51 DEBUG : Waiting for deletions to finish
2021/05/21 11:02:51 DEBUG : S3 bucket archamps: Removing directory
2021/05/21 11:02:51 ERROR : : Failed to rmdir: BucketNotEmpty: The bucket you tried to delete is not empty
        status code: 409, request id: 1681097C98168555, host id:
2021/05/21 11:02:51 ERROR : Attempt 3/3 failed with 1 errors and: BucketNotEmpty: The bucket you tried to delete is not empty
        status code: 409, request id: 1681097C98168555, host id:
2021/05/21 11:02:51 DEBUG : 3 go routines active
2021/05/21 11:02:51 Failed to purge: BucketNotEmpty: The bucket you tried to delete is not empty
        status code: 409, request id: 1681097C98168555, host id:

# rclone lsf StorjS3:archamps -vv
2021/05/21 11:03:01 DEBUG : rclone: Version "v1.56.0-beta.5519.07f2f3a62" starting with parameters ["rclone" "lsf" "StorjS3:archamps" "-vv"]
2021/05/21 11:03:01 DEBUG : Creating backend with remote "StorjS3:archamps"
2021/05/21 11:03:01 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2021/05/21 11:03:01 DEBUG : 3 go routines active

Will do, and point to this thread. Thanks.

A quick guess without reading all the details:

Did you rule out that the folder contains a folder? (rclone lsd Storj:archamps)

Purge would delete it, no?
But:

# rclone lsd Storj:archamps -vv
2021/05/21 11:07:52 DEBUG : rclone: Version "v1.56.0-beta.5519.07f2f3a62" starting with parameters ["rclone" "lsd" "Storj:archamps" "-vv"]
2021/05/21 11:07:52 DEBUG : Creating backend with remote "Storj:archamps"
2021/05/21 11:07:52 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2021/05/21 11:07:52 DEBUG : FS sj://archamps: connecting...
2021/05/21 11:07:52 DEBUG : FS sj://archamps: connected: <nil>
2021/05/21 11:07:52 DEBUG : FS sj://archamps: ls ./
2021/05/21 11:07:52 DEBUG : FS sj://archamps: OBJ ls ./ ("archamps", "")
2021/05/21 11:07:52 DEBUG : FS sj://archamps: opts &{Prefix: Cursor: Recursive:false System:true Custom:true}
2021/05/21 11:07:53 DEBUG : 5 go routines active

# rclone lsd StorjS3:archamps -vv
2021/05/21 11:07:59 DEBUG : rclone: Version "v1.56.0-beta.5519.07f2f3a62" starting with parameters ["rclone" "lsd" "StorjS3:archamps" "-vv"]
2021/05/21 11:07:59 DEBUG : Creating backend with remote "StorjS3:archamps"
2021/05/21 11:07:59 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2021/05/21 11:07:59 DEBUG : 3 go routines active

Yes it should, but worth trying. Unfortunately no luck.

You could also open a new issue on Github and I can tag the backend author who is active on GitHub.

Done: Bucket can't be deleted at Storj DCS · Issue #5346 · rclone/rclone · GitHub

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.