Two different objects treated as same

Help and Support
1

What is the problem you are having with rclone?

Rclone sync complains that there are two identical files on source.
The URL to the two files are different containing two different names that decode to the same character Norwegian å.

Instagram%20Facebook%20Best%C3%A5.mp4
Instagram%20Facebook%20Besta%CC%8A.mp4

Both gives name Instagram Facebook Bestå.mp4 when decoding but they are two diffent objects.

Run the command 'rclone version' and share the full output of the command.

rclone v1.62.2

  • os/version: Microsoft Windows Server 2016 Standard 1607 (64 bit)
  • os/kernel: 10.0.14393.7785 Build 14393.7785.7785 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.20.2
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

IBM S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync source:bucket destination:bucket -P

The rclone config contents with secrets removed.

n/a

A log from the command with the -vv flag

2025/02/25 14:01:23 NOTICE: Instagram Facebook Bestå.mp4: Duplicate object found in source - ignoring
2

this is very old rclone version. Please update and try again.

3

Updated to this:

rclone v1.69.1

  • os/version: Microsoft Windows Server 2016 Standard 1607 (64 bit)
  • os/kernel: 10.0.14393.7785 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.24.0
  • go/linking: static
  • go/tags: cmount

Same result.

4

Try --no-unicode-normalization flag.

1 Like
5

Ok.
So no bug.

--no-unicode-normalization solved the issue.

--no-unicode-normalization should probably be used when mirrorin two S3 buckets. Probably --metadata too.

To ensure identical copy.

Feel free to delete or move thread as this is no bug after all.

Thanks
Torge

6

Yes this is debatable - like many defaults...

7

Can this be detected?
Can the notice below include information about the suggested option that solved the problem?

2025/02/25 14:01:23 NOTICE: Instagram Facebook Bestå.mp4: Duplicate object found in source - ignoring

8

Everything can be done:)

When detected could be logged indeed.

If somebody will feel like doing it then I do not see why not to include it.

9

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.