Union of oneDrive for business upstreams: Getting object not found error

What is the problem you are having with rclone?

Getting object not found when copying a file to a union backend with both upstreams being on OneDrive for Business

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1

  • os/version: arch (64 bit)
  • os/kernel: 6.2.6-arch1-1 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.4
  • go/linking: dynamic
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

OneDrive for Business

The command you were trying to run (eg rclone copy /tmp remote:tmp)

touch x
rclone copy x u:

The rclone config contents with secrets removed.

[main]
type = onedrive
token = <redacted>
drive_id = <redacted>
drive_type = business

[u]
type = union
upstreams = main: main:/xyz


A log from the command with the -vv flag

2023/03/18 17:19:54 DEBUG : rclone: Version "v1.61.1" starting with parameters ["/usr/bin/rclone" "copy" "x" "u:" "-vv" "--log-file" "/tmp/x.log"]
2023/03/18 17:19:54 DEBUG : Creating backend with remote "x"
2023/03/18 17:19:54 DEBUG : Using config file from "/tmp/rclone.conf"
2023/03/18 17:19:54 DEBUG : fs cache: adding new entry for parent of "x", "/home/owner"
2023/03/18 17:19:54 DEBUG : Creating backend with remote "u:"
2023/03/18 17:19:54 DEBUG : Creating backend with remote "main:/xyz"
2023/03/18 17:19:54 DEBUG : Creating backend with remote "main:"
2023/03/18 17:19:54 DEBUG : fs cache: renaming cache item "main:/xyz" to be canonical "main:xyz"
2023/03/18 17:19:54 DEBUG : fs cache: switching user supplied name "main:/xyz" for canonical name "main:xyz"
2023/03/18 17:19:54 DEBUG : union root '': actionPolicy = *policy.EpAll, createPolicy = *policy.EpMfs, searchPolicy = *policy.FF
2023/03/18 17:19:55 DEBUG : x: Need to transfer - File not found at Destination
2023/03/18 17:19:57 ERROR : x: Failed to copy: object not found
2023/03/18 17:19:57 ERROR : Attempt 1/3 failed with 1 errors and: object not found
2023/03/18 17:19:57 DEBUG : x: Need to transfer - File not found at Destination
2023/03/18 17:19:58 ERROR : x: Failed to copy: object not found
2023/03/18 17:19:58 ERROR : Attempt 2/3 failed with 1 errors and: object not found
2023/03/18 17:19:59 DEBUG : x: Need to transfer - File not found at Destination
2023/03/18 17:20:00 ERROR : x: Failed to copy: object not found
2023/03/18 17:20:00 ERROR : Attempt 3/3 failed with 1 errors and: object not found
2023/03/18 17:20:00 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         6.0s

2023/03/18 17:20:00 DEBUG : 5 go routines active
2023/03/18 17:20:00 Failed to copy: object not found

Some notes

  • Running rclone copy x main: and rclone copy x main:/xyz works
  • Adding a local backend as a third upstream and setting create_policy = all resulted in the command succeeding and the file appearing in all three upstreams

hi,

you are using one remote for the union and main:/xyz is a subdir or main
might that be a problem?

Hey there, I tried setting the second upstream to a separate OneDrive for Business account and am getting the same result

Does the destination directory xyz exist?

Yes it does exist

Can you try using main:xyz instead of main:/xyz - I wonder if that is what is confusing things.

In general remotes should not have a / after the :. On a small number of remotes (ftp, sftp, local, dropbox) it means something different.

Same issue unfortunately

As mentioned in my second comment, the command somehow works if I add a third, non-OneDrive for Business upstream, so I doubt it's got to do with my config

Can you run rclone about on both the upstreams?

rclone about main:
rclone about main:/xyz

If it is the MFS policy misbehaving then it is something to do with the sizes reported.

I wonder if it is getting confused because both the sizes reported for your remotes will always be identical?

here's the output I get for both commands

Total:   1 TiB
Used:    1.453 TiB
Free:    0 B
Trashed: 43.033 GiB

I've also tried setting a completely different oneDrive for Business account as the 2nd remote : same behaviour (object not found)

That says it has no bytes free. This means none of the remotes rclone is trying to use has free space to store things on - this will give the error you see.

Yeah that rclone about command seems wrong for 2 reasons

  1. When I login to the storage metrics page on my OneDrive account, I see I have 122GB free, so I'm not sure how rclone calculates free space
  2. If it really was a free space issue, then the commands in my second comment should also have given the same error