Leading "/" in S3 object keys causes incorrect MissingOnSrc

Help and Support
1

What is the problem you are having with rclone?

I'm syncing from an S3 remote, where all my object keys begin with a "/". My S3 endpoint is hosted by a ceph cluster running 19.2.0:

ceph version 19.2.0 (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable)

For any files already synced to my destination (from a previous rclone command), rclone reports MissingOnSrc - I think because it strips the leading slash from the object key and then can't find a matching key. That leads to all files that were present in the sync destination prior to the sync action getting deleted after the sync (which seems to be otherwise successful).

This causes an oscillating result: each time I run rclone, any files that were previously present in my sync destination get removed but any files that were missing from destination get correctly synced; so my destination gains the files it was missing and loses the files it already had. When I run rclone again, the same things happens, flipping the destination fileset back to its previous state.

I checked the changelog and existing bugs - found lots of mentions of leading slashes but I couldn't see any config that might fix this behaviour.

Run the command 'rclone version' and share the full output of the command.

$ rclone --version
rclone v1.72.1
- os/version: redhat 9.4 (64 bit)
- os/kernel: 5.14.0-427.61.1.el9_4.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.25.5
- go/linking: static
- go/tags: none
$

Which cloud storage system are you using? (eg Google Drive)

Not sure what this means; my sync source is a Ceph S3 bucket, on squid / 19.2.0.

In the bucket that I'm syncing, all keys begin with a "/". I've focussed debugging on one specific object with key: "/opt/dcl/commp/sipps/ca.crt"

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone --config ~/fusiondevsdc_rclone.conf sync fusiondevsc:fusiondevsc-sipps /home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps --no-check-certificate  --combined sync.tmp -vvvPaste command here

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

$ rclone config redacted --config ~/fusiondevsdc_rclone.conf
[fusiondevsc]
type = s3
provider = Ceph
env_auth = false
access_key_id = XXX
secret_access_key = XXX
endpoint = https://10.60.14.138:7480
sse_customer_algorithm = AES256
sse_customer_key = XXX
sse_customer_key_md5 = XXX
### Double check the config for sensitive info before posting publicly
$

A log from the command that you were trying to run with the -vv flag

I have trimmed the logs to isolate those relating to a single file, which highlights the issue. Can provide the full log if this doesn't provide all the necessary info.

Running the above command when file is already present at the sync destination (/home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps/opt/dcl/commp/sipps/ca.crt)

2026/06/05 14:03:47 DEBUG : rclone: Version "v1.72.1" starting with parameters ["rclone" "--config" "/home/fusiondevsdc2_backup/fusiondevsdc_rclone.conf" "sync" "fusiondevsc:fusiondevsc-sipps" "/home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps" "--no-check-certificate" "--combined" "sync.log" "-vvv"]
2026/06/05 14:03:47 DEBUG : Creating backend with remote "fusiondevsc:fusiondevsc-sipps"
2026/06/05 14:03:47 DEBUG : Using config file from "/home/fusiondevsdc2_backup/fusiondevsdc_rclone.conf"
2026/06/05 14:03:47 DEBUG : Creating backend with remote "/home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps"
2026/06/05 14:03:47 DEBUG : Sync Logger: MissingOnSrc: - opt/dcl/commp/sipps/ca.crt
2026/06/05 14:03:47 DEBUG : Sync Logger: MissingOnDst: + /opt/dcl/commp/sipps/ca.crt
2026/06/05 14:03:47 DEBUG : /opt/dcl/commp/sipps/ca.crt: Need to transfer - File not found at Destination
2026/06/05 14:03:47 DEBUG : /opt/dcl/commp/sipps/ca.crt.8aefe520.partial: size = 8276 OK
2026/06/05 14:03:47 DEBUG : /opt/dcl/commp/sipps/ca.crt: Src hash empty - aborting Dst hash check
2026/06/05 14:03:47 DEBUG : /opt/dcl/commp/sipps/ca.crt.8aefe520.partial: renamed to: /opt/dcl/commp/sipps/ca.crt
2026/06/05 14:03:47 INFO  : /opt/dcl/commp/sipps/ca.crt: Copied (new)
2026/06/05 14:04:28 DEBUG : Waiting for deletions to finish
2026/06/05 14:04:28 INFO  : opt/dcl/commp/sipps/ca.crt: Deleted
...
2026/06/05 14:04:28 INFO  : opt: Removing directory
2026/06/05 14:04:28 DEBUG : opt: Failed to Rmdir: remove /home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps/opt: directory not empty
2026/06/05 14:04:28 DEBUG : Local file system at /home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps: failed to delete 13 directories
2026/06/05 14:04:28 DEBUG : Local file system at /home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps: deleted 40 directories
2026/06/05 14:04:28 INFO  :
Transferred:        4.780 GiB / 4.780 GiB, 100%, 106.625 MiB/s, ETA 0s
Checks:               151 / 151, 100%, Listed 465
Deleted:              151 (files), 53 (dirs), 1.848 GiB (freed)
Transferred:          195 / 195, 100%
Elapsed time:        41.3s

2026/06/05 14:04:28 DEBUG : 30 go routines active

That removes the file from dst.

When I re-run the same command:

2026/06/05 14:05:42 DEBUG : rclone: Version "v1.72.1" starting with parameters ["rclone" "--config" "/home/fusiondevsdc2_backup/fusiondevsdc_rclone.conf" "sync" "fusiondevsc:fusiondevsc-sipps" "/home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps" "--no-check-certificate" "--combined" "sync2.log" "-vvv"]
2026/06/05 14:05:42 DEBUG : Creating backend with remote "fusiondevsc:fusiondevsc-sipps"
2026/06/05 14:05:42 DEBUG : Using config file from "/home/fusiondevsdc2_backup/fusiondevsdc_rclone.conf"
2026/06/05 14:05:42 DEBUG : Creating backend with remote "/home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps"
2026/06/05 14:05:42 DEBUG : Sync Logger: MissingOnDst: + /opt/dcl/commp/sipps/ca.crt
2026/06/05 14:05:42 DEBUG : /opt/dcl/commp/sipps/ca.crt: Need to transfer - File not found at Destination
2026/06/05 14:05:42 DEBUG : /opt/dcl/commp/sipps/ca.crt.8aefe520.partial: size = 8276 OK
2026/06/05 14:05:42 DEBUG : /opt/dcl/commp/sipps/ca.crt: Src hash empty - aborting Dst hash check
2026/06/05 14:05:42 DEBUG : /opt/dcl/commp/sipps/ca.crt.8aefe520.partial: renamed to: /opt/dcl/commp/sipps/ca.crt
2026/06/05 14:05:42 INFO  : /opt/dcl/commp/sipps/ca.crt: Copied (new)
...
2026/06/05 14:06:23 INFO  : opt: Removing directory
2026/06/05 14:06:23 DEBUG : opt: Failed to Rmdir: remove /home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps/opt: directory not empty
2026/06/05 14:06:23 DEBUG : Local file system at /home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps: failed to delete 13 directories
2026/06/05 14:06:23 DEBUG : Local file system at /home/fusiondevsdc2_backup/backup/objectdata/fusiondevsc-sipps: deleted 12 directories
2026/06/05 14:06:23 INFO  :
Transferred:        4.780 GiB / 4.780 GiB, 100%, 104.400 MiB/s, ETA 0s
Checks:                44 / 44, 100%, Listed 330
Deleted:               44 (files), 25 (dirs), 2.932 GiB (freed)
Transferred:          195 / 195, 100%
Elapsed time:        41.0s

2026/06/05 14:06:23 DEBUG : 32 go routines active

And the file is now present on dst again.

2

welcome to the f\orum,

please run rclone selfupdate and test again.
based on what you posted, i doubt that will fix the issue

3

did you read?

S3 (OVHcloud/other): (leading) slash as part of a path not working properly

Leading slash (/) in source/dest paths

4

Thanks for linking those existing posts @asdffdsa - I'd skimmed the first one but the second was new. I don't see a solution to this problem in either, but they did point me at a few extra config options.

I can workaround my specific issue by passing the `--delete-before` arg, so that the sync happens after the delete. That still leads to unnecessary delete-then-sync of existing identical files, but it means the destination is left in the correct state at the end of the rclone command. (This is equivalent to a manual `rm -rf` wipe of the destination before running `rclone sync` but for my usage it's convenient to do all that within a single rclone command.)

So I am able to work around the existing behaviour, but at the cost of a potentially significant perf hit when my rclone destination was already synced.

The underlying issue is that there is not a 1:1 mapping between source and destination when the source key starts with a "/". The fact that a leading "/" is supported in S3 (and other providers) is naturally problematic. In S3 "/name" and "name" are two distinct keys that can validly coexist; treating the "/" as a path separator on a file-based rclone local destination will map those two keys to the same destination path. Ignoring or stripping slashes doesn't change that.

  1. Source "/rest/of/path" and "rest/of/path" both map to dest "/rest/of/path" when working out if files are present or missing in dest.
  2. Dest "/rest/of/path" maps to source "rest/of/path" when working out if files are present in source.

This accurately describes existing rclone behaviour - and is completely reasonable.

An alternative approach to (1) would be to map "/" to a different character. There are different ways that could be done and I think existing rclone config options would allow you to do this (e.g. --name-transform or encoding). This isn't actually my issue though, and I suspect it's a non-issue for most other S3 users too. So - I don't think anything needs to change here.

For (2), in my scenario I want to be able to toggle "always-apply-leading-slash" behaviour. That's understandably not a sensible default but for anyone consistently using leading slashes in S3 as I am, that would produce the desired behaviour here I think? I've not found any way of producing the same effect using the existing config options. (I looked into --name-transform but given how this handle path segments, I think this is not viable.)

Can I submit a fix to add such a config flag? Let's say --s3-leading-slash=true (default false) ?

5

I would open the PR with a small reproducer/test for two keys like /a and a before adding a user-facing option. The tricky bit is not just preserving the leading slash, it is proving rclone can round-trip both names without mapping them onto the same local path. If that cannot be made unambiguous for a local destination, a flag that errors early (or documents the delete-before workaround as lossy/expensive) may be safer than silently changing S3 path handling.