How to find sources files that trigger "Duplicate object found in source - ignoring" without needing to run `rclone copy`?

What is the problem you are having with rclone?

When I run rclone copy, I often see the helpful message

NOTICE:........Duplicate object found in source - ignoring

This only occurs when I am copying files. I am wondering if there is anyway to find these duplicate objects without needing to do rclone copy? I understand using --dry-run could work but it leaves a lot of messages such as saying how it is not actually transferring files due to the dry run. Thanks!

Run the command 'rclone version' and share the full output of the command.

rclone v1.64.0
- os/version: darwin 13.5.2 (64 bit)
- os/kernel: 22.6.0 (x86_64)
- os/type: darwin
- os/arch: amd64
- go/version: go1.21.1
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy -P gd:/ "/Volumes/2 TB/" --include "Backups/**"

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[gd]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
acknowledge_abuse = true
token = XXX
root_folder_id = XXX
team_drive =

A log from the command that you were trying to run with the -vv flag

2023/09/15 18:03:11 DEBUG : Creating backend with remote "gd:/"
2023/09/15 18:03:11 DEBUG : Using config file from "/Users/username/.config/rclone/rclone.conf"
2023/09/15 18:03:11 DEBUG : fs cache: renaming cache item "gd:/" to be canonical "gd:"
2023/09/15 18:03:11 DEBUG : Creating backend with remote "/Volumes/2 TB/"
2023/09/15 18:03:11 DEBUG : fs cache: renaming cache item "/Volumes/2 TB/" to be canonical "/Volumes/2 TB"
2023-09-15 18:03:11 DEBUG : gd: Loaded invalid token from config file - ignoring
2023-09-15 18:03:11 DEBUG : Saving config "token" in section "gd" of the config file
2023-09-15 18:03:11 DEBUG : gd: Saved new token in config file
```#### What is the problem you are having with rclone?
Rclone used to throw an error message if two files with the same name existed in a Google Drive source. For example: `Duplicate directory found in source - ignoring`. Since updating to the latest, I no longer see this and Rclone by default will select the latest file. How can I bring back the error message?


#### Run the command 'rclone version' and share the full output of the command.

rclone v1.64.0

  • os/version: darwin 13.5.2 (64 bit)
  • os/kernel: 22.6.0 (x86_64)
  • os/type: darwin
  • os/arch: amd64
  • go/version: go1.21.1
  • go/linking: dynamic
  • go/tags: cmount

####  Which cloud storage system are you using? (eg Google Drive)

Google Drive

#### The command you were trying to run (eg `rclone copy /tmp remote:tmp`)  
<!--  You should use 3 backticks to begin and end your paste to make it readable.   -->

rclone copy -P gd:/ "/Volumes/2 TB/" --include "Backups/**"



#### Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
<!--  You should use 3 backticks to begin and end your paste to make it readable.   -->

[gd]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
acknowledge_abuse = true
token = XXX
root_folder_id = XXX
team_drive =




#### A log from the command that you were trying to run with the `-vv` flag  
<!-- You should use 3 backticks to begin and end your paste to make it readable.  Or use a service such as https://pastebin.com or https://gist.github.com/   -->

2023/09/15 18:03:11 DEBUG : Creating backend with remote "gd:/"
2023/09/15 18:03:11 DEBUG : Using config file from "/Users/username/.config/rclone/rclone.conf"
2023/09/15 18:03:11 DEBUG : fs cache: renaming cache item "gd:/" to be canonical "gd:"
2023/09/15 18:03:11 DEBUG : Creating backend with remote "/Volumes/2 TB/"
2023/09/15 18:03:11 DEBUG : fs cache: renaming cache item "/Volumes/2 TB/" to be canonical "/Volumes/2 TB"
2023-09-15 18:03:11 DEBUG : gd: Loaded invalid token from config file - ignoring
2023-09-15 18:03:11 DEBUG : Saving config "token" in section "gd" of the config file
2023-09-15 18:03:11 DEBUG : gd: Saved new token in config file

​ ​

rclone dedupe remote: --dry-run

Thank you! I am looking at the documentation and I see

  • --dedupe-mode rename - removes identical files then renames the rest to be different.

What does it exactly mean by "removes"? thanks

remove = delete

If not 100% sure always try with --dry-run first