DELETE issuing two HEAD before issuing delete on S3 (Wasabi)

Help and Support
1

[Re-opened after Akismet wrongly flagged & closed my original post]

What is the problem you are having with rclone?

DELETE issues two HEAD calls before issuing a DELETE to S3 (Wasabi).

Isn't one HEAD call sufficient to determine the presence/absence of object, to determine whether to issue a DELETE or not?

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.1
- os/version: fedora 36 (64 bit)
- os/kernel: 6.2.14-100.fc36.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.6
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

S3 (Wasabi)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone delete xxx-ws:/xxx.org/code/hello_test.txt --dump headers --no-traverse -P

The rclone config contents with secrets removed.

[xxx-ws]
type = s3
provider = Wasabi
env_auth = false
access_key_id = xyz
secret_access_key = xyz
region = us-east-1
endpoint = s3.us-east-2.wasabisys.com
acl = public-read

A log from the command that you were trying to run with the -vv flag

$ ~/rclone-v1.63.1-linux-amd64/rclone delete xxx-ws:/xxx.org/code/hello_test.txt --dump headers --no-traverse -P
2023/09/02 17:20:11 DEBUG : rclone: Version "v1.63.1" starting with parameters ["/home/xxx/rclone-v1.63.1-linux-amd64/rclone" "delete" "xxx-ws:/xxx.org/code/hello_test.txt" "--dump" "headers" "--no-traverse" "-P"]
2023/09/02 17:20:11 DEBUG : Creating backend with remote "xxx-ws:/xxx.org/code/hello_test.txt"
2023/09/02 17:20:11 DEBUG : Using RCLONE_CONFIG_PASS password.
2023/09/02 17:20:11 DEBUG : Using config file from "/home/xxx/.config/rclone/rclone.conf"
2023/09/02 17:20:11 DEBUG : name = "xxx-ws", root = "/xxx.org/code/hello_test.txt", opt = &s3.Options{Provider:"Wasabi", EnvAuth:false, AccessKeyID:"xyz", SecretAccessKey:"xyz", Region:"us-east-1", Endpoint:"s3.us-east-2.wasabisys.com", STSEndpoint:"", LocationConstraint:"", ACL:"public-read", BucketACL:"", RequesterPays:false, ServerSideEncryption:"", SSEKMSKeyID:"", SSECustomerAlgorithm:"", SSECustomerKey:"", SSECustomerKeyBase64:"", SSECustomerKeyMD5:"", StorageClass:"", UploadCutoff:209715200, CopyCutoff:4999341932, ChunkSize:5242880, MaxUploadParts:10000, DisableChecksum:false, SharedCredentialsFile:"", Profile:"", SessionToken:"", UploadConcurrency:4, ForcePathStyle:true, V2Auth:false, UseAccelerateEndpoint:false, LeavePartsOnError:false, ListChunk:1000, ListVersion:0, ListURLEncode:fs.Tristate{Value:false, Valid:false}, NoCheckBucket:false, NoHead:false, NoHeadObject:false, Enc:0x3000002, MemoryPoolFlushTime:60000000000, MemoryPoolUseMmap:false, DisableHTTP2:false, DownloadURL:"", DirectoryMarkers:false, UseMultipartEtag:fs.Tristate{Value:false, Valid:false}, UsePresignedRequest:false, Versions:false, VersionAt:fs.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}, Decompress:false, MightGzip:fs.Tristate{Value:false, Valid:false}, UseAcceptEncodingGzip:fs.Tristate{Value:false, Valid:false}, NoSystemMetadata:false}
2023/09/02 17:20:11 DEBUG : Resolving service "s3" region "us-east-1"
2023/09/02 17:20:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/09/02 17:20:11 DEBUG : HTTP REQUEST (req 0xc000982500)
2023/09/02 17:20:11 DEBUG : HEAD /xxx.org/code/hello_test.txt HTTP/1.1
Host: s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.63.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230903T002011Z

2023/09/02 17:20:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/09/02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/09/02 17:20:12 DEBUG : HTTP RESPONSE (req 0xc000982500)
2023/09/02 17:20:12 DEBUG : HTTP/1.1 200 OK
Content-Length: 71
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Sun, 03 Sep 2023 00:20:12 GMT
Etag: "2478296abbf255b5aeb1a17303f21d48"
Last-Modified: Sun, 03 Sep 2023 00:19:09 GMT
Server: WasabiS3/7.15.2121-2023-07-18-0ee420c377 (head3)
X-Amz-Id-2: UXuOftlzHPJz46Xuw4PTbdWvR2eP4sbmrdwbO7DSZ5X6LHvJGFUDZuUHXPAl6XgCVZLGTn2VoHly
X-Amz-Meta-Mtime: 1693678498.481085693
X-Amz-Request-Id: ED8FCD125544938D:B

2023/09/02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/09/02 17:20:12 DEBUG : fs cache: adding new entry for parent of "xxx-ws:/xxx.org/code/hello_test.txt", "xxx-ws:xxx.org/code"
2023-09-02 17:20:12 DEBUG : Waiting for deletions to finish
2023-09-02 17:20:12 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023-09-02 17:20:12 DEBUG : HTTP REQUEST (req 0xc000982800)
2023-09-02 17:20:12 DEBUG : HEAD /xxx.org/code/hello_test.txt HTTP/1.1
Host: s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.63.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230903T002012Z
2023-09-02 17:20:12 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023-09-02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023-09-02 17:20:12 DEBUG : HTTP RESPONSE (req 0xc000982800)
2023-09-02 17:20:12 DEBUG : HTTP/1.1 200 OK
Content-Length: 71
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Sun, 03 Sep 2023 00:20:12 GMT
Etag: "2478296abbf255b5aeb1a17303f21d48"
Last-Modified: Sun, 03 Sep 2023 00:19:09 GMT
Server: WasabiS3/7.15.2121-2023-07-18-0ee420c377 (head3)
X-Amz-Id-2: yeo5qjx3TaHM8PxqMXNhplB/wIcvUjg3trHLyNnFTPMODVULKuyJgU4HWqSX2lfxTsNM2ot0yEYN
X-Amz-Meta-Mtime: 1693678498.481085693
X-Amz-Request-Id: C66B174930D174C0:A
2023-09-02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023-09-02 17:20:12 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023-09-02 17:20:12 DEBUG : HTTP REQUEST (req 0xc000982d00)
2023-09-02 17:20:12 DEBUG : DELETE /xxx.org/code/hello_test.txt HTTP/1.1
Host: s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.63.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230903T002012Z
Accept-Encoding: gzip
2023-09-02 17:20:12 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023-09-02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023-09-02 17:20:12 DEBUG : HTTP RESPONSE (req 0xc000982d00)
2023-09-02 17:20:12 DEBUG : HTTP/1.1 204 No Content
Date: Sun, 03 Sep 2023 00:20:12 GMT
Server: WasabiS3/7.15.2121-2023-07-18-0ee420c377 (head3)
X-Amz-Id-2: z3p94mIETRgKbvxZWvNNBaNxRSPWQ3tA7eHIBHE8gy1uU5lXa3yzMweBZ2t0SkU7GqO/fVkDEmSx
X-Amz-Request-Id: 282176A99030BF3D:B
2023-09-02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023-09-02 17:20:12 INFO  : hello_test.txt: Deleted
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Deleted:                1 (files), 0 (dirs)
Elapsed time:         0.7s
2023/09/02 17:20:12 DEBUG : 6 go routines active
2

welcome to the forum,

might try --s3-no-head and see what you get

3

@asdffdsa Rclone version - 1.63.1 (latest version).

Regarding --s3-no-head, not sure why it should be required. Without specifying it, it should still be a single HEAD call to inquire on object status. Isn't it?

4

yes, good, i see that you updated between the two topics.

that was a test, to understand what rclone is doing.
in any event, i just did the test myself, and it did not make a difference, still two HEAD
and this has nothing to do with wasabi.

5

I also tried --s3-no-head and observed the same behavior -- no effect.

.. inline with my expectation that for a DELETE, --s3-no-head shouldn't come into picture as there's no integrity check required post-operation for "delete".

6

sorry, confused
--- the two HEAD are per-operation, correct?
--- no post-operation after DELETE, correct?

7

From the docs:

--s3-no-head   If set, don't HEAD uploaded objects to check integrity

In my case (a delete operation), no object uploads are happening. So, s3-no-head option has no bearing.

I'll prefer to keep s3-no-head discussions in a separate thread (if required), to avoid drowning my main question/problem:

-- 2 HEAD calls, before a single DELETE -- why 2 ?

8

good question.

i had meant testing --s3-no-head-object. never mind

just trying to understand what rclone is doing and in the mean time, try to find a workaround.

9

My intention was not to put you away. But to keep the thread lean, so that, in case someone comes along to view, they're not fatigued by a lengthy side discussion.

Regarding - what Rclone is trying to do - I didn't curtail the log/debug file & everything Rclone emitted is in it. And as you were also able to confirm, indeed it fires two HEAD calls before executing the single requested delete.

I'm not stuck or something due to this, only curious why Rclone might be doing this - a bug or some valid scenario escaping me.

10

When you issue an rclone command, rclone doesn't know whether xxx-ws:/xxx.org/code/hello_test.txt refers to a file or a directory. That is where the first HEAD command comes from - figuring that out.

Unfortunately rclone throws the result of the HEAD command away due to the internal architecture (I would like to change this but it is a lot of work!), so the second HEAD is required to see if the object exists or not before issuing the DELETE.

If you are desperate to avoid that extra HEAD use

rclone rc --loopback operations/deletefile fs=s3:rclone/ remote=/test.txt -vv --dump headers

The / on the end of the fs=/ means that rclone knows the remote points to a directory so doesn't need to HEAD it to see if it is a file.

1 Like
11

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.