Rclone creates duplicate files without been asked to

Suspected Bug
1

What is the problem you are having with rclone?

From 1.63.0-beta.7022.e649cf4d5 until

rclone v1.64.0-beta.7120.00512e130

  • os/version: ubuntu 20.04 (64 bit)
  • os/kernel: 5.15.0-101-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.20.5
  • go/linking: static
  • go/tags: none

and also

rclone v1.66.0

  • os/version: ubuntu 20.04 (64 bit)
  • os/kernel: 5.15.0-101-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.22.1
  • go/linking: static
  • go/tags: none

rclone was able to:

  1. Create duplicate files with the same name.
    They don't exist locally - but remote does render them as separate files
  2. Go back-and-forth between saving a file, and then "later" reverting it to its original version
  3. Trigger a number of "file changed, same contents" that Sublime Text (and other editors) detect, resulting in unnecessary churn
    ... but not quite, since you can never know if [2] happened
    And comparing them leads to even more churn :confused:

Focus on the

Char/name/Spells » stuff.html
Char/name/Spells » stuff.pdf

files:

Spells » stuff.html;39198;2024-03-23 13:26:55;9dc7244079393ae604ba2b51ce1d5326;1pY-Y8aBWkXpj_nG8m2g9b5teUTj_749K;1pY-Y8aBWkXpj_nG8m2g9b5teUTj_749K
Spells » stuff.html;39198;2024-03-23 13:25:08;9dc7244079393ae604ba2b51ce1d5326;1-4GITlDAWALvjY3BGpiBW_OcvhQ97SDm;1-4GITlDAWALvjY3BGpiBW_OcvhQ97SDm
Spells » stuff.pdf;316085;2024-03-23 13:59:38;7f3a2475418e1a2f15564cae47743175;1-8NgqsmkuIsOpN4sHNpRk6-o-bickPaO;1-8NgqsmkuIsOpN4sHNpRk6-o-bickPaO
Spells » stuff.pdf;315050;2024-03-23 13:57:34;b781f14723b5f08e251a3b4952266ce6;1USYJVvpBqbiaFHDjF3eHKOUP6XX9TrO1;1USYJVvpBqbiaFHDjF3eHKOUP6XX9TrO1

I have seen Drive duplicate files reported by rclone lsl already, but "it doesn't solve the issue"; also the rclone dedupe is useless to me:

  1. The files are not the same in-content, and
  2. I cannot compare them - and even if I could, idk if that would trigger [3.] above
  3. Even if "everything would work", one of them has history snapshots, and it is shared via link. I'd like to keep "that file".

I would be more interested if dedupe, for text files, would support a merge & delete operation

Run the command 'rclone version' and share the full output of the command.

rclone v1.64.0-beta.7120.00512e130
rclone v1.66.0

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone --vfs-cache-mode full -vv mount remote-stuff:Saved\ Data/A\&A/ /mnt/gdrive/

The rclone config contents with secrets removed.

[remote-stuff]
type = drive
client_id = 102
scope = drive
client_secret = h
token = {...}
team_drive =

Please update the template to say rclone config redacted ...

A log from the command with the -vv flag

rclone-log.html.xz.txt (217.7 KB)
please remember to rename it to .xz and de-compress it ...

2

What do you mean?

3

rclone config redacted is supported nowadays. You can just offer it to the

The rclone config contents with secrets removed.

[remote-stuff]

segment (I cannot edit my post, so I cannot copy/paste the template message)

4

How does a person know that the command exists?

We use this "Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone." now. What would you change it to?

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.