[Re-opened after Akismet wrongly flagged & closed my original post]
What is the problem you are having with rclone?
DELETE issues two HEAD calls before issuing a DELETE to S3 (Wasabi).
Isn't one HEAD call sufficient to determine the presence/absence of object, to determine whether to issue a DELETE or not?
Run the command 'rclone version' and share the full output of the command.
rclone v1.63.1
- os/version: fedora 36 (64 bit)
- os/kernel: 6.2.14-100.fc36.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.6
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
S3 (Wasabi)
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone delete xxx-ws:/xxx.org/code/hello_test.txt --dump headers --no-traverse -P
The rclone config contents with secrets removed.
[xxx-ws]
type = s3
provider = Wasabi
env_auth = false
access_key_id = xyz
secret_access_key = xyz
region = us-east-1
endpoint = s3.us-east-2.wasabisys.com
acl = public-read
A log from the command that you were trying to run with the -vv
flag
$ ~/rclone-v1.63.1-linux-amd64/rclone delete xxx-ws:/xxx.org/code/hello_test.txt --dump headers --no-traverse -P
2023/09/02 17:20:11 DEBUG : rclone: Version "v1.63.1" starting with parameters ["/home/xxx/rclone-v1.63.1-linux-amd64/rclone" "delete" "xxx-ws:/xxx.org/code/hello_test.txt" "--dump" "headers" "--no-traverse" "-P"]
2023/09/02 17:20:11 DEBUG : Creating backend with remote "xxx-ws:/xxx.org/code/hello_test.txt"
2023/09/02 17:20:11 DEBUG : Using RCLONE_CONFIG_PASS password.
2023/09/02 17:20:11 DEBUG : Using config file from "/home/xxx/.config/rclone/rclone.conf"
2023/09/02 17:20:11 DEBUG : name = "xxx-ws", root = "/xxx.org/code/hello_test.txt", opt = &s3.Options{Provider:"Wasabi", EnvAuth:false, AccessKeyID:"xyz", SecretAccessKey:"xyz", Region:"us-east-1", Endpoint:"s3.us-east-2.wasabisys.com", STSEndpoint:"", LocationConstraint:"", ACL:"public-read", BucketACL:"", RequesterPays:false, ServerSideEncryption:"", SSEKMSKeyID:"", SSECustomerAlgorithm:"", SSECustomerKey:"", SSECustomerKeyBase64:"", SSECustomerKeyMD5:"", StorageClass:"", UploadCutoff:209715200, CopyCutoff:4999341932, ChunkSize:5242880, MaxUploadParts:10000, DisableChecksum:false, SharedCredentialsFile:"", Profile:"", SessionToken:"", UploadConcurrency:4, ForcePathStyle:true, V2Auth:false, UseAccelerateEndpoint:false, LeavePartsOnError:false, ListChunk:1000, ListVersion:0, ListURLEncode:fs.Tristate{Value:false, Valid:false}, NoCheckBucket:false, NoHead:false, NoHeadObject:false, Enc:0x3000002, MemoryPoolFlushTime:60000000000, MemoryPoolUseMmap:false, DisableHTTP2:false, DownloadURL:"", DirectoryMarkers:false, UseMultipartEtag:fs.Tristate{Value:false, Valid:false}, UsePresignedRequest:false, Versions:false, VersionAt:fs.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}, Decompress:false, MightGzip:fs.Tristate{Value:false, Valid:false}, UseAcceptEncodingGzip:fs.Tristate{Value:false, Valid:false}, NoSystemMetadata:false}
2023/09/02 17:20:11 DEBUG : Resolving service "s3" region "us-east-1"
2023/09/02 17:20:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/09/02 17:20:11 DEBUG : HTTP REQUEST (req 0xc000982500)
2023/09/02 17:20:11 DEBUG : HEAD /xxx.org/code/hello_test.txt HTTP/1.1
Host: s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.63.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230903T002011Z
2023/09/02 17:20:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/09/02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/09/02 17:20:12 DEBUG : HTTP RESPONSE (req 0xc000982500)
2023/09/02 17:20:12 DEBUG : HTTP/1.1 200 OK
Content-Length: 71
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Sun, 03 Sep 2023 00:20:12 GMT
Etag: "2478296abbf255b5aeb1a17303f21d48"
Last-Modified: Sun, 03 Sep 2023 00:19:09 GMT
Server: WasabiS3/7.15.2121-2023-07-18-0ee420c377 (head3)
X-Amz-Id-2: UXuOftlzHPJz46Xuw4PTbdWvR2eP4sbmrdwbO7DSZ5X6LHvJGFUDZuUHXPAl6XgCVZLGTn2VoHly
X-Amz-Meta-Mtime: 1693678498.481085693
X-Amz-Request-Id: ED8FCD125544938D:B
2023/09/02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/09/02 17:20:12 DEBUG : fs cache: adding new entry for parent of "xxx-ws:/xxx.org/code/hello_test.txt", "xxx-ws:xxx.org/code"
2023-09-02 17:20:12 DEBUG : Waiting for deletions to finish
2023-09-02 17:20:12 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023-09-02 17:20:12 DEBUG : HTTP REQUEST (req 0xc000982800)
2023-09-02 17:20:12 DEBUG : HEAD /xxx.org/code/hello_test.txt HTTP/1.1
Host: s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.63.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230903T002012Z
2023-09-02 17:20:12 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023-09-02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023-09-02 17:20:12 DEBUG : HTTP RESPONSE (req 0xc000982800)
2023-09-02 17:20:12 DEBUG : HTTP/1.1 200 OK
Content-Length: 71
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Sun, 03 Sep 2023 00:20:12 GMT
Etag: "2478296abbf255b5aeb1a17303f21d48"
Last-Modified: Sun, 03 Sep 2023 00:19:09 GMT
Server: WasabiS3/7.15.2121-2023-07-18-0ee420c377 (head3)
X-Amz-Id-2: yeo5qjx3TaHM8PxqMXNhplB/wIcvUjg3trHLyNnFTPMODVULKuyJgU4HWqSX2lfxTsNM2ot0yEYN
X-Amz-Meta-Mtime: 1693678498.481085693
X-Amz-Request-Id: C66B174930D174C0:A
2023-09-02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023-09-02 17:20:12 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023-09-02 17:20:12 DEBUG : HTTP REQUEST (req 0xc000982d00)
2023-09-02 17:20:12 DEBUG : DELETE /xxx.org/code/hello_test.txt HTTP/1.1
Host: s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.63.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230903T002012Z
Accept-Encoding: gzip
2023-09-02 17:20:12 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023-09-02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023-09-02 17:20:12 DEBUG : HTTP RESPONSE (req 0xc000982d00)
2023-09-02 17:20:12 DEBUG : HTTP/1.1 204 No Content
Date: Sun, 03 Sep 2023 00:20:12 GMT
Server: WasabiS3/7.15.2121-2023-07-18-0ee420c377 (head3)
X-Amz-Id-2: z3p94mIETRgKbvxZWvNNBaNxRSPWQ3tA7eHIBHE8gy1uU5lXa3yzMweBZ2t0SkU7GqO/fVkDEmSx
X-Amz-Request-Id: 282176A99030BF3D:B
2023-09-02 17:20:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023-09-02 17:20:12 INFO : hello_test.txt: Deleted
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 1 / 1, 100%
Deleted: 1 (files), 0 (dirs)
Elapsed time: 0.7s
2023/09/02 17:20:12 DEBUG : 6 go routines active