Rclone doubling 0x201B in S3/Swift (at least) URLs

What is the problem you are having with rclone?

rclone fails to handle any objects (at least via S3 or swift) with 0x201B / SINGLE HIGH-REVERSED-9 QUOTATION MARK / ‛ in their name. It doubles that particular character (a mis-firing attempt at quoting/escaping?) resulting in 404s.

Run the command 'rclone version' and share the full output of the command.

mvernon@ms-backup1003:~$ ./rclone version
rclone v1.75.1
- os/version: debian 13.6 (64 bit)
- os/kernel: 6.12.95+deb13-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.26.8
- go/linking: static
- go/tags: none

I initially observed this with Debian's rclone (v1.60.1), but have reproduced with the latest release, noted above.

Which cloud storage system are you using? (eg Google Drive)

Originally observed with Swift (OpenStack Swift server), reproduced with S3 (Ceph-backed)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Initially observed with our weekly sync (which does rclone sync), but you see it in the output of list etc. too. Simple reproducer (using s3cmd for upload since it doesn't have this problem):

mvernon@ms-backup1003:~$ s3cmd put Post_Boxes_at_the_Isle_of_Man_Motor_Museum.jpg s3://mcv21test/Post‛Box.jpg
upload: 'Post_Boxes_at_the_Isle_of_Man_Motor_Museum.jpg' -> 's3://mcv21test/Post‛Box.jpg'  [1 of 1]
 1484068 of 1484068   100% in    0s    19.83 MB/s  done
mvernon@ms-backup1003:~$ s3cmd ls s3://mcv21test
2024-08-20 13:10      8286485  s3://mcv21test/1-month-old_kittens_32.jpg
2024-08-20 14:21    224878555  s3://mcv21test/Apollo_Butterfly_under_the_microscope,_4k.webm
2024-08-20 13:10       539645  s3://mcv21test/Café_de_Flore.jpg
2024-08-20 12:56        20144  s3://mcv21test/Gnome-edit-delete.svg
2026-09-22 15:09      1484068  s3://mcv21test/Post‛Box.jpg
mvernon@ms-backup1003:~$ ./rclone -vv ls apus:
2026/09/22 15:25:52 DEBUG : rclone: Version "v1.75.1" starting with parameters ["./rclone" "-vv" "ls" "apus:"]
2026/09/22 15:25:52 DEBUG : Creating backend with remote "apus:"
2026/09/22 15:25:52 DEBUG : Using config file from "/home/mvernon/.config/rclone/rclone.conf"
  8286485 mcv21test/1-month-old_kittens_32.jpg
224878555 mcv21test/Apollo_Butterfly_under_the_microscope,_4k.webm
   539645 mcv21test/Café_de_Flore.jpg
    20144 mcv21test/Gnome-edit-delete.svg
  1484068 mcv21test/Post‛‛Box.jpg
2026/09/22 15:25:52 DEBUG : 5 go routines active

See how rclone is doubling the ‛ character? That means that copy / sync / etc all fail with a 404.

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[apus]
type = s3
provider = Ceph
access_key_id = XXX
secret_access_key = XXX
endpoint = apus.discovery.wmnet
acl = private
### Double check the config for sensitive info before posting publicly

A log from the command that you were trying to run with the -vv flag

Here's a simple download failing

mvernon@ms-backup1003:~$ ./rclone copy -vv apus:mcv21test/Post‛Box.jpg .
2026/09/22 15:55:16 DEBUG : rclone: Version "v1.75.1" starting with parameters ["./rclone" "copy" "-vv" "apus:mcv21test/Post‛Box.jpg" "."]
2026/09/22 15:55:16 DEBUG : Creating backend with remote "apus:mcv21test/Post‛Box.jpg"
2026/09/22 15:55:16 DEBUG : Using config file from "/home/mvernon/.config/rclone/rclone.conf"
2026/09/22 15:55:16 DEBUG : Creating backend with remote "."
2026/09/22 15:55:16 DEBUG : fs cache: renaming cache item "." to be canonical "."
2026/09/22 15:55:16 DEBUG : Local file system at .: Waiting for checks to finish
2026/09/22 15:55:16 DEBUG : Local file system at .: Waiting for transfers to finish
2026/09/22 15:55:16 INFO  : There was nothing to transfer
2026/09/22 15:55:16 INFO  : 
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 0 / 0, -, Listed 10
Elapsed time:         0.0s
 
2026/09/22 15:55:16 DEBUG : 5 go routines active

As an example of this character being used "in anger" as it were, here are some examples of it being used for arabic manuscripts on Wikimedia Commons:

That smells like the request path is escaping the key twice. Check the bytes of one bad name first:

rclone lsf apus:mcv21test --recursive --format p | od -An -tx1

Then run one failing read with -vv --dump headers. Compare the key bytes with the GET path. If U+201B is present once in the key and twice in the path, include that small log with the Ceph version in a bug report. Also test the same object after copying it to a plain ASCII name. That tells you fast whether this is the S3 backend path or the object itself.

mvernon@ms-backup1003:~$ ./rclone lsf apus:mcv21test --recursive --format p | od -An -tx1 
 31 2d 6d 6f 6e 74 68 2d 6f 6c 64 5f 6b 69 74 74
 65 6e 73 5f 33 32 2e 6a 70 67 0a 41 70 6f 6c 6c
 6f 5f 42 75 74 74 65 72 66 6c 79 5f 75 6e 64 65
 72 5f 74 68 65 5f 6d 69 63 72 6f 73 63 6f 70 65
 2c 5f 34 6b 2e 77 65 62 6d 0a 43 61 66 c3 a9 5f
 64 65 5f 46 6c 6f 72 65 2e 6a 70 67 0a 47 6e 6f
 6d 65 2d 65 64 69 74 2d 64 65 6c 65 74 65 2e 73
 76 67 0a 50 6f 73 74 e2 80 9b e2 80 9b 42 6f 78
 2e 6a 70 67 0a

which does have the doubled sequence e2 80 9b. Then here's a failing rclone copy invocation with --dump headers (just the relevant HEAD and response):slight_smile:

mvernon@ms-backup1003:~$ ./rclone copy -vv --dump headers apus:mcv21test/Post‛Box.jpg .
2026/09/23 08:42:42 NOTICE: Automatically setting -vv as --dump is enabled
2026/09/23 08:42:42 DEBUG : rclone: Version "v1.75.1" starting with parameters ["./rclone" "copy" "-vv" "--dump" "headers" "apus:mcv21test/Post‛Box.jpg" "."]
2026/09/23 08:42:42 DEBUG : Creating backend with remote "apus:mcv21test/Post‛Box.jpg"
2026/09/23 08:42:42 DEBUG : Using config file from "/home/mvernon/.config/rclone/rclone.conf"
2026/09/23 08:42:42 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2026/09/23 08:42:42 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2026/09/23 08:42:42 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2026/09/23 08:42:42 DEBUG : HTTP REQUEST (req 0xf8f149763c0)
2026/09/23 08:42:42 DEBUG : HEAD /mcv21test/Post%E2%80%9B%E2%80%9BBox.jpg HTTP/1.1
Host: apus.discovery.wmnet
User-Agent: rclone/v1.75.1
Accept-Encoding: identity
Amz-Sdk-Invocation-Id: 4ecb96e4-2678-4129-9b4c-f6d81b99a6eb
Amz-Sdk-Request: attempt=1; max=10
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20260923T084242Z
 
2026/09/23 08:42:42 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2026/09/23 08:42:42 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2026/09/23 08:42:42 DEBUG : HTTP RESPONSE (req 0xf8f149763c0)
2026/09/23 08:42:42 DEBUG : HTTP/1.1 404 Not Found
Content-Length: 233
Accept-Ranges: bytes
Content-Type: application/xml
Date: Wed, 23 Sep 2026 08:42:42 GMT
Server: envoy
X-Amz-Request-Id: tx00000a7994d8264fce853-006ab39102-59156-eqiad
X-Envoy-Upstream-Service-Time: 2

The Ceph server is running Reef (18.2.8). If I upload the same object as PostBox.jpg, rclone copy is able to download it just fine.

This is a known bug in the encoding. If you use the raw encoding then you can disable this. You might need this for the source and/or the destination (can't remember!) See the --s3-e coding flags (there is one for each backend)

Thanks! It looks like I probably need both:

mvernon@ms-backup1003:~/junk$ ../rclone copy --local-encoding Raw apus:mcv21test/Post‛Box.jpg .
mvernon@ms-backup1003:~/junk$ ls
mvernon@ms-backup1003:~/junk$ ../rclone copy --s3-encoding Raw apus:mcv21test/Post‛Box.jpg .
mvernon@ms-backup1003:~/junk$ ls
Post‛‛Box.jpg
mvernon@ms-backup1003:~/junk$ ../rclone copy --s3-encoding Raw --local-encoding Raw apus:mcv21test/Post‛Box.jpg .
mvernon@ms-backup1003:~/junk$ ls
Post‛‛Box.jpg  Post‛Box.jpg

If I just set --local-encoding Raw, then rclone tries to download the incorrect object (and fails); if I just set --s3-encoding Raw, then rclone downloads the correct object, but to the incorrect local path. If I set both, then the right object is downloaded with the right name.

For my production use case, I'm syncing two swift containers, so I think using "Raw" should be safe there, as there should be no invalid-in-swift object names already in my swift store?

[might it be worth documenting this issue in the encoding system?]

Yes I think you'll be fine.

That's a good idea. There is an issue about it which we could point to also.