Posix filesystem to OneDrive: Duplicate object found in source - ignoring

What is the problem you are having with rclone?

rclone sync <local filesystem on linux> onedrive:/Backups yields errors of the format:

NOTICE: <local file path>: Duplicate object found in source - ignoring

What is a duplicate object in source when the source is a Posix filesystem?

Run the command 'rclone version' and share the full output of the command.

rclone 1.60.1

  • os/version: fedora 37 (64 bit)
  • os/kernel: 6.2.7-200.fc37.x86_64 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.3
  • go/linking: dynamic
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

OneDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync <local filesystem path> onedrive:/Backups --links --delete-excluded --exclude-from /tmp/exclude --transfers 10 --progress

The rclone config contents with secrets removed.

[onedrive]
type = onedrive
token = {"access_token":"[redacted]","expiry":"2023-04-03T18:58:48.456070097-04:00"}
drive_id = [redacted]
drive_type = business

A log from the command with the -vv flag

Can't do this at the moment as the rclone sync is still in progress.

hello and welcome to the forum,

rclone sync will not re-copy a file, so ok to kill it.

--- kill the rclone sync
--- run rclone selfupdate to get latest stable.
--- run the command again, with --dry-run -vv
--- post the debug output including the top twenty lines.

Hello and thanks.

--- run rclone selfupdate to get latest stable.

That's not how software is updated on package managed systems such as Linux systems. The package manager (i.e. dnf on RH systems) should only be used to update software. The vendor has tickets open to update it's version of rclone.

Did you really mean I should post the entire debug output including the top twenty lines or did you mean you just want the top twenty lines? The entire debug output is 342116 lines of security sensitive information that I am not allowed to post.

The top 7 lines are:

2023/04/04 08:42:43 DEBUG : rclone: Version "1.60.1" starting with parameters ["rclone" "sync" "[redacted]" "onedrive:/Backups[redacted]" "--links" "--delete-excluded" "--exclude-from" "/tmp/exclude" "--transfers" "10" "--dry-run" "-vv"]
2023/04/04 08:42:43 DEBUG : Creating backend with remote "[redacted]"
2023/04/04 08:42:43 DEBUG : Using config file from "[redacted]/.config/rclone/rclone.conf"
2023/04/04 08:42:43 DEBUG : local: detected overridden config - adding "{b6816}" suffix to name
2023/04/04 08:42:43 DEBUG : fs cache: renaming cache item "[redacted]" to be canonical "local{b6816}:[redacted]"
2023/04/04 08:42:43 DEBUG : Creating backend with remote "onedrive:/Backups[redacted]"
2023/04/04 08:42:45 DEBUG : fs cache: renaming cache item "onedrive:/Backups[redacted]" to be canonical "onedrive:Backups[redacted]"

followed by a series of DEBUG lines regarding local files and whether they are going to be copied to the remote ore skipped, either because they are already there, or because --dry-run is active. So honestly, nothing really interesting.

Where there is a Duplicate object found in source - ignoring message reported, there is no other information about that file printed.

So again, my question is what is the definition of a duplicate object in a Posix filesystem?

Sure it is - many people use self update on Linux as repo or package manager are an awful way for rclone as it's not maintained/hosted or anything rclone so you get whatever the package manager wants which tends to be very old versions and unmaintained versions.

Check out:

The duplicate isn't likely on the source but really it's seeing something like that in the source, which creates a duplicate on Onedrive. The debug logs would probably list out the file I'd imagine but you haven't included a log to check out.

Here's an example as the source isn't the duplicate.

[felix@gemini test]$ ls
one.txt  One.txt
[felix@gemini test]$ rclone copy /home/felix/test DB:test -vv
2023/04/04 08:59:07 DEBUG : Setting --config "/opt/rclone/rclone.conf" from environment variable RCLONE_CONFIG="/opt/rclone/rclone.conf"
2023/04/04 08:59:07 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "copy" "/home/felix/test" "DB:test" "-vv"]
2023/04/04 08:59:07 DEBUG : Creating backend with remote "/home/felix/test"
2023/04/04 08:59:07 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2023/04/04 08:59:07 DEBUG : Creating backend with remote "DB:test"
2023/04/04 08:59:07 NOTICE: one.txt: Duplicate object found in source - ignoring
2023/04/04 08:59:07 DEBUG : Dropbox root 'test': Waiting for checks to finish
2023/04/04 08:59:07 DEBUG : Dropbox root 'test': Waiting for transfers to finish
2023/04/04 08:59:08 DEBUG : One.txt: Uploading chunk 1/0
2023/04/04 08:59:08 DEBUG : One.txt: Uploading chunk 2/0
2023/04/04 08:59:08 DEBUG : Dropbox root 'test': Adding "/test/One.txt" to batch
2023/04/04 08:59:09 DEBUG : Dropbox root 'test': Batch idle for 500ms so committing
2023/04/04 08:59:09 DEBUG : Dropbox root 'test': Committing sync batch length 1 starting with: /test/One.txt
2023/04/04 08:59:09 DEBUG : Dropbox root 'test': Committed sync batch length 1 starting with: /test/One.txt
2023/04/04 08:59:09 DEBUG : One.txt: dropbox = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 OK
2023/04/04 08:59:09 INFO  : One.txt: Copied (new)
2023/04/04 08:59:09 INFO  :
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         2.1s

2023/04/04 08:59:09 DEBUG : 9 go routines active
2023/04/04 08:59:09 INFO  : Dropbox root 'test': Committing uploads - please wait...
[felix@gemini test]$ rclone ls DB:test
        0 One.txt

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.