Too many errors with Blomp, especially in directories with thousands of files

Help and Support
1

What is the problem you are having with rclone?

I'm attempting to sync my whole filesystem which contains about 4.5M files. Too many operations fail with different errors, but for now let's focus on the most annoying and reproducible one.

With large directories, which already contain thousands of files on the remote, checking fails. It waits for 20–50 seconds, than fails with "forbidden" error, which is kinda confusing, but I suppose it's a backend issue. This might happen several times in a row if I attempt more than one retry only on the same large directory, and only around a 4th time a server finally responds with a result, which feels like it "collects" and "caches" the directory content in the meantime.

Which worries me is that:

  1. Is it normal for Blomp or is it related to my configuration or the way rclone works with Swift in general, compared to the official Blomp client? (Especially given that they have a modded Swift.)
  2. Maybe offtopic, but why does rclone have this full retry strategy, which involves full resync and rescan on each retry? Given that the checking process is slow and spawning too many checkers is not a good idea generally and only leads to more errors. I know that cloud storages are expected to be unreliable, but then I also would expect rclone to work with them akin to how tools for recovering data from unreliable media, like ddrescue or NSCopy, do: remember failed blocks and retry only them on next attempts, rather than just spit errors to the log. Maybe there is some flag for that which I missed?

The --low-level-retries flag doesn't really seem to be useful in this case at all, checking fails right after the first request.

Run the command 'rclone version' and share the full output of the command.

rclone v1.67.0
- os/version: debian trixie/sid (64 bit)
- os/kernel: 6.7.12-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.4
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Blomp

The command you were trying to run (eg rclone copy /tmp remote:tmp)

sudo -E rclone sync --log-file /tmp/rclone3.log --retries=1 --low-level-retries=100 --checkers=4 --transfers=96 -xlPM /usr/lib/x86_64-linux-gnu Blomp-compress:usr/lib/x86_64-linux-gnu

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[Blomp]
type = swift
user = XXX
key = XXX
auth = https://authenticate.ain.net
tenant = XXX
auth_version = 2
leave_parts_on_error = true
chunk_size = 1Gi

[Blomp-compress]
type = compress
remote = Blomp-secret:
mode = gzip
level = 6

[Blomp-secret]
type = crypt
remote = Blomp:XXX
password = XXX

[Mega]
type = mega
user = XXX
pass = XXX
debug = true

[Mega-secret]
type = crypt
remote = Mega:data
password = XXX

A log from the command that you were trying to run with the -vv flag

2024/07/10 21:59:36 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "sync" "-vv" "--log-file" "/tmp/rclone3.log" "--retries=1" "--low-level-retries=100" "--checkers=4" "--transfers=96" "-xlPM" "/usr/lib/x86_64-linux-gnu/" "Blomp-compress:usr/lib/x86_64-linux-gnu"]
2024/07/10 21:59:36 DEBUG : Creating backend with remote "/usr/lib/x86_64-linux-gnu/"
2024/07/10 21:59:36 DEBUG : Using RCLONE_CONFIG_PASS password.
2024/07/10 21:59:36 DEBUG : Using config file from "/home/bodqhrohro/.config/rclone/rclone.conf"
2024/07/10 21:59:36 DEBUG : local: detected overridden config - adding "{ljIJR}" suffix to name
2024/07/10 21:59:36 DEBUG : fs cache: renaming cache item "/usr/lib/x86_64-linux-gnu/" to be canonical "local{ljIJR}:/usr/lib/x86_64-linux-gnu"
2024/07/10 21:59:36 DEBUG : Creating backend with remote "Blomp-compress:usr/lib/x86_64-linux-gnu"
2024/07/10 21:59:36 DEBUG : Creating backend with remote "Blomp:XXX/bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/7kojtm9m9ammgb2sncfa0q1dash2gcasr83m7bouood97aftbesg"
2024/07/10 21:59:37 DEBUG : Swift container XXX path bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/7kojtm9m9ammgb2sncfa0q1dash2gcasr83m7bouood97aftbesg: Auto set use_segments_container to false
2024/07/10 21:59:38 DEBUG : Creating backend with remote "Blomp:XXX/bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/p46q52ja97q9obde9mlcjk5lcuvre1sp4roqdh9i4u3aocu36s3g"
2024/07/10 21:59:39 DEBUG : Swift container XXX path bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/p46q52ja97q9obde9mlcjk5lcuvre1sp4roqdh9i4u3aocu36s3g: Auto set use_segments_container to false
2024/07/10 21:59:58 ERROR : Compressed: Blomp-compress:usr/lib/x86_64-linux-gnu: error reading destination root directory: Operation forbidden
2024/07/10 21:59:58 DEBUG : Compressed: Blomp-compress:usr/lib/x86_64-linux-gnu: Waiting for checks to finish
2024/07/10 21:59:58 DEBUG : Compressed: Blomp-compress:usr/lib/x86_64-linux-gnu: Waiting for transfers to finish
2024/07/10 21:59:58 ERROR : Compressed: Blomp-compress:usr/lib/x86_64-linux-gnu: not deleting files as there were IO errors
2024/07/10 21:59:58 ERROR : Compressed: Blomp-compress:usr/lib/x86_64-linux-gnu: not deleting directories as there were IO errors
2024/07/10 21:59:58 ERROR : Attempt 1/1 failed with 1 errors and: Operation forbidden
2024/07/10 21:59:58 INFO  : 
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:        21.7s

2024/07/10 21:59:58 DEBUG : 7 go routines active
2024/07/10 21:59:58 Failed to sync: Operation forbidden
2

welcome to the forum,

each time rclone is run, no idea about what files are in the cloud, so rclone needs to scan the source and dest.
if you have a --magic solution to that, please share.

can use filters, such as --max-age, to reduce the total number of checks

for a deeper look at the api calls, use --dump=headers

3

Yeah, but I mean retries during one run.
Files could change between retries, of course (though I think there should be a flag for an assumption that only rclone syncs them), but they could change between a check and a transfer just as well, there are no transactions.

Yes, I've seen that as a widespread solution, but it makes sense only if a full backup is already made and only new files need to be added. For now, I cannot succeed even with making one complete backup. I need to split the process into smaller directories, repeat retries manually until a success, watch for errors and repeat smaller directories where some files failed (or a complete directory). This is a very tedious process.

2024/07/12 17:46:12 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 17:46:12 DEBUG : Swift container XXXX path bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/7kojtm9m9ammgb2sncfa0q1dash
2gcasr83m7bouood97aftbesg: Auto set use_segments_container to false
2024/07/12 17:46:12 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 17:46:12 DEBUG : HTTP REQUEST (req 0xc000535560)
2024/07/12 17:46:12 DEBUG : HEAD /v1/AUTH_XXXX/XXXX/bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/7
kojtm9m9ammgb2sncfa0q1dash2gcasr83m7bouood97aftbesg HTTP/1.1
Host: swiftproxy.acs.ai.net:8080
User-Agent: rclone/v1.67.0
X-Auth-Token: XXXX

2024/07/12 17:46:12 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 17:46:13 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 17:46:13 DEBUG : HTTP RESPONSE (req 0xc000535560)
2024/07/12 17:46:13 DEBUG : HTTP/1.1 404 Not Found
Content-Type: text/html; charset=UTF-8
Date: Fri, 12 Jul 2024 14:46:13 GMT
X-Openstack-Request-Id: tx5cbfb196ad994de8b30d4-00669141b5
X-Trans-Id: tx5cbfb196ad994de8b30d4-00669141b5
Content-Length: 0

2024/07/12 17:46:13 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 17:46:13 DEBUG : Creating backend with remote "Blomp:XXXX/bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/p46q52ja97q9
obde9mlcjk5lcuvre1sp4roqdh9i4u3aocu36s3g"
2024/07/12 17:46:13 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 17:46:13 DEBUG : HTTP REQUEST (req 0xc0008ec120)
2024/07/12 17:46:13 DEBUG : POST /tokens HTTP/1.1
Host: authenticate.ain.net
User-Agent: rclone/v1.67.0
Content-Length: 118
Content-Type: application/json
Accept-Encoding: gzip

2024/07/12 17:46:13 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 17:46:14 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 17:46:14 DEBUG : HTTP RESPONSE (req 0xc0008ec120)
2024/07/12 17:46:14 DEBUG : HTTP/1.1 200 OK
Content-Length: 4418
Content-Type: application/json
Date: Fri, 12 Jul 2024 14:46:14 GMT
Server: Apache/2.4.7 (Ubuntu)
Vary: X-Auth-Token
X-Distribution: Ubuntu
X-Openstack-Request-Id: req-9bac2620-3702-4fab-bfbf-2fab2d2e2a7e

2024/07/12 17:46:14 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 17:46:14 DEBUG : Swift container XXXX path bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/p46q52ja97q9obde9mlcjk5lcuvre1sp4roqdh9i4u3aocu36s3g: Auto set use_segments_container to false
2024/07/12 17:46:14 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 17:46:14 DEBUG : HTTP REQUEST (req 0xc00095ea20)
2024/07/12 17:46:14 DEBUG : HEAD /v1/AUTH_XXXX/XXXX/bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/p46q52ja97q9obde9mlcjk5lcuvre1sp4roqdh9i4u3aocu36s3g HTTP/1.1
Host: swiftproxy.acs.ai.net:8080
User-Agent: rclone/v1.67.0
X-Auth-Token: XXXX

2024/07/12 17:46:14 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 17:46:15 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 17:46:15 DEBUG : HTTP RESPONSE (req 0xc00095ea20)
2024/07/12 17:46:15 DEBUG : HTTP/1.1 404 Not Found
Content-Type: text/html; charset=UTF-8
Date: Fri, 12 Jul 2024 14:46:15 GMT
X-Openstack-Request-Id: txe1e842cd887f4929b1133-00669141b7
X-Trans-Id: txe1e842cd887f4929b1133-00669141b7
Content-Length: 0

2024/07/12 17:46:15 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 17:46:15 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 17:46:15 DEBUG : HTTP REQUEST (req 0xc000a94480)
2024/07/12 17:46:15 DEBUG : GET /v1/AUTH_XXXX/XXXX?delimiter=%2F&format=json&limit=1000&prefix=bak1%2Frfrf4c3mkdco2cm2junun0jq4s%2F2a7udl32693noa7d4kdegijmas%2Fp46q52ja97q9obde9mlcjk5lcuvre1sp4roqdh9i4u3aocu36s3g%2F HTTP/1.1
Host: swiftproxy.acs.ai.net:8080
User-Agent: rclone/v1.67.0
X-Auth-Token: XXXX
Accept-Encoding: gzip

2024/07/12 17:46:15 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 17:46:26 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 17:46:26 DEBUG : HTTP RESPONSE (req 0xc000a94480)
2024/07/12 17:46:26 DEBUG : HTTP/1.1 403 Forbidden
Content-Length: 73
Content-Type: text/html; charset=UTF-8
Date: Fri, 12 Jul 2024 14:46:26 GMT
X-Openstack-Request-Id: txad1c8930629241478ca9f-00669141b7
X-Trans-Id: txad1c8930629241478ca9f-00669141b7

2024/07/12 17:46:26 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 17:46:26 ERROR : Compressed: Blomp-compress:usr/lib/x86_64-linux-gnu: error reading destination root directory: Operation forbidden
2024/07/12 17:46:26 DEBUG : Compressed: Blomp-compress:usr/lib/x86_64-linux-gnu: Waiting for checks to finish
2024/07/12 17:46:26 DEBUG : Compressed: Blomp-compress:usr/lib/x86_64-linux-gnu: Waiting for transfers to finish
2024/07/12 17:46:26 ERROR : Compressed: Blomp-compress:usr/lib/x86_64-linux-gnu: not deleting files as there were IO errors
2024/07/12 17:46:26 ERROR : Compressed: Blomp-compress:usr/lib/x86_64-linux-gnu: not deleting directories as there were IO errors
2024/07/12 17:46:26 ERROR : Attempt 1/1 failed with 1 errors and: Operation forbidden
2024/07/12 17:46:26 INFO  : 
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:        14.9s

2024/07/12 17:46:26 DEBUG : 6 go routines active
2024/07/12 17:46:26 Failed to sync: Operation forbidden
(END)
4

i would run the command with --dump=headers, get the server to respond with a result.
them compare the GET request with the request from the log you posted.

perhaps contact the provider.

5 
2024/07/12 21:07:41 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 21:07:41 DEBUG : Swift container XXXX path bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/7kojtm9m9ammgb2sncfa0q1dash
2gcasr83m7bouood97aftbesg: Auto set use_segments_container to false
2024/07/12 21:07:41 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 21:07:41 DEBUG : HTTP REQUEST (req 0xc0005eeb40)
2024/07/12 21:07:41 DEBUG : HEAD /v1/AUTH_XXXX/XXXX/bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/7
kojtm9m9ammgb2sncfa0q1dash2gcasr83m7bouood97aftbesg HTTP/1.1
Host: swiftproxy.acs.ai.net:8080
User-Agent: rclone/v1.67.0
X-Auth-Token: XXXX

2024/07/12 21:07:41 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 21:07:42 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 21:07:42 DEBUG : HTTP RESPONSE (req 0xc0005eeb40)
2024/07/12 21:07:42 DEBUG : HTTP/1.1 404 Not Found
Content-Type: text/html; charset=UTF-8
Date: Fri, 12 Jul 2024 18:07:42 GMT
X-Openstack-Request-Id: txf8da4426fe944a68ace15-00669170ed
X-Trans-Id: txf8da4426fe944a68ace15-00669170ed
Content-Length: 0

2024/07/12 21:07:42 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 21:07:42 DEBUG : Creating backend with remote "Blomp:XXXX/bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/p46q52ja97q9obde9mlcjk5lcuvre1sp4roqdh9i4u3aocu36s3g"
2024/07/12 21:07:42 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 21:07:42 DEBUG : HTTP REQUEST (req 0xc0005eefc0)
2024/07/12 21:07:42 DEBUG : POST /tokens HTTP/1.1
Host: authenticate.ain.net
User-Agent: rclone/v1.67.0
Content-Length: 118
Content-Type: application/json
Accept-Encoding: gzip

2024/07/12 21:07:42 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 21:07:43 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 21:07:43 DEBUG : HTTP RESPONSE (req 0xc0005eefc0)
2024/07/12 21:07:43 DEBUG : HTTP/1.1 200 OK
Content-Length: 4418
Content-Type: application/json
Date: Fri, 12 Jul 2024 18:07:43 GMT
Server: Apache/2.4.7 (Ubuntu)
Vary: X-Auth-Token
X-Distribution: Ubuntu
X-Openstack-Request-Id: req-15239eb3-d08e-4547-a997-feae59433527

2024/07/12 21:07:43 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 21:07:43 DEBUG : Swift container XXXX path bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/p46q52ja97q9obde9mlcjk5lcuvre1sp4roqdh9i4u3aocu36s3g: Auto set use_segments_container to false
2024/07/12 21:07:43 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 21:07:43 DEBUG : HTTP REQUEST (req 0xc0006f86c0)
2024/07/12 21:07:43 DEBUG : HEAD /v1/AUTH_XXXX/XXXX/bak1/rfrf4c3mkdco2cm2junun0jq4s/2a7udl32693noa7d4kdegijmas/p46q52ja97q9obde9mlcjk5lcuvre1sp4roqdh9i4u3aocu36s3g HTTP/1.1
Host: swiftproxy.acs.ai.net:8080
User-Agent: rclone/v1.67.0
X-Auth-Token: XXXX

2024/07/12 21:07:43 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 21:07:44 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 21:07:44 DEBUG : HTTP RESPONSE (req 0xc0006f86c0)
2024/07/12 21:07:44 DEBUG : HTTP/1.1 404 Not Found
Content-Type: text/html; charset=UTF-8
Date: Fri, 12 Jul 2024 18:07:44 GMT
X-Openstack-Request-Id: txaddc5e44eda24cad86d1e-00669170ef
X-Trans-Id: txaddc5e44eda24cad86d1e-00669170ef
Content-Length: 0

2024/07/12 21:07:44 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 21:07:44 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 21:07:44 DEBUG : HTTP REQUEST (req 0xc000559560)
2024/07/12 21:07:44 DEBUG : GET /v1/AUTH_XXXX/XXXX?delimiter=%2F&format=json&limit=1000&prefix=bak1%2Frfrf4c3mkdco2cm2junun0jq4s%2F2a7udl32693noa7d4kdegijmas%2Fp46q52ja97q9obde9mlcjk5lcuvre1sp4roqdh9i4u3aocu36s3g%2F HTTP/1.1
Host: swiftproxy.acs.ai.net:8080
User-Agent: rclone/v1.67.0
X-Auth-Token: XXXX
Accept-Encoding: gzip

2024/07/12 21:07:44 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/07/12 21:07:45 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/07/12 21:07:45 DEBUG : HTTP RESPONSE (req 0xc000559560)
2024/07/12 21:07:45 DEBUG : HTTP/1.1 200 OK
Content-Length: 338794
Accept-Ranges: bytes
Content-Type: application/json; charset=utf-8
Date: Fri, 12 Jul 2024 18:07:45 GMT
Last-Modified: Fri, 12 Jul 2024 16:21:56 GMT
X-Container-Bytes-Used: 395803387531
X-Container-Meta-Quota-Bytes: 2199023255552
X-Container-Meta-Read: XXXX:XXXX,.rlistings
X-Container-Meta-Write: XXXX:XXXX
X-Container-Object-Count: 5664096
X-Openstack-Request-Id: tx6fd6ea34ae2944da93f9e-00669170f0
X-Storage-Policy: Policy-0
X-Timestamp: 1719700786.65892
X-Trans-Id: tx6fd6ea34ae2944da93f9e-00669170f0
6

Where does this parameter come from? Should it be lowered maybe?

7

Meh, it seems to be hardcoded rclone/backend/swift/swift.go at master · rclone/rclone · GitHub

And I don't see any pagination and offset parameter there, does it mean that the Swift backend does not support directories with more than 1000 items? It doesn't seem like I had any issues with them though.

I'll probably attempt to build it myself and add 403 to the list of retry codes.

8

i would assume the forum would be full of topics about that.
fwiw, might test without compress remote, as it is experimental.

maybe, as an opt-in flag, something like
--retry-403=true

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.