Union of oneDrive for business upstreams: Getting object not found error

Suspected Bug
1

What is the problem you are having with rclone?

Getting object not found when copying a file to a union backend with both upstreams being on OneDrive for Business

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1

  • os/version: arch (64 bit)
  • os/kernel: 6.2.6-arch1-1 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.4
  • go/linking: dynamic
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

OneDrive for Business

The command you were trying to run (eg rclone copy /tmp remote:tmp)

touch x
rclone copy x u:

The rclone config contents with secrets removed.

[main]
type = onedrive
token = <redacted>
drive_id = <redacted>
drive_type = business

[u]
type = union
upstreams = main: main:/xyz

A log from the command with the -vv flag

2023/03/18 17:19:54 DEBUG : rclone: Version "v1.61.1" starting with parameters ["/usr/bin/rclone" "copy" "x" "u:" "-vv" "--log-file" "/tmp/x.log"]
2023/03/18 17:19:54 DEBUG : Creating backend with remote "x"
2023/03/18 17:19:54 DEBUG : Using config file from "/tmp/rclone.conf"
2023/03/18 17:19:54 DEBUG : fs cache: adding new entry for parent of "x", "/home/owner"
2023/03/18 17:19:54 DEBUG : Creating backend with remote "u:"
2023/03/18 17:19:54 DEBUG : Creating backend with remote "main:/xyz"
2023/03/18 17:19:54 DEBUG : Creating backend with remote "main:"
2023/03/18 17:19:54 DEBUG : fs cache: renaming cache item "main:/xyz" to be canonical "main:xyz"
2023/03/18 17:19:54 DEBUG : fs cache: switching user supplied name "main:/xyz" for canonical name "main:xyz"
2023/03/18 17:19:54 DEBUG : union root '': actionPolicy = *policy.EpAll, createPolicy = *policy.EpMfs, searchPolicy = *policy.FF
2023/03/18 17:19:55 DEBUG : x: Need to transfer - File not found at Destination
2023/03/18 17:19:57 ERROR : x: Failed to copy: object not found
2023/03/18 17:19:57 ERROR : Attempt 1/3 failed with 1 errors and: object not found
2023/03/18 17:19:57 DEBUG : x: Need to transfer - File not found at Destination
2023/03/18 17:19:58 ERROR : x: Failed to copy: object not found
2023/03/18 17:19:58 ERROR : Attempt 2/3 failed with 1 errors and: object not found
2023/03/18 17:19:59 DEBUG : x: Need to transfer - File not found at Destination
2023/03/18 17:20:00 ERROR : x: Failed to copy: object not found
2023/03/18 17:20:00 ERROR : Attempt 3/3 failed with 1 errors and: object not found
2023/03/18 17:20:00 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         6.0s

2023/03/18 17:20:00 DEBUG : 5 go routines active
2023/03/18 17:20:00 Failed to copy: object not found
2

Some notes

  • Running rclone copy x main: and rclone copy x main:/xyz works
  • Adding a local backend as a third upstream and setting create_policy = all resulted in the command succeeding and the file appearing in all three upstreams
3

hi,

you are using one remote for the union and main:/xyz is a subdir or main
might that be a problem?

4

Hey there, I tried setting the second upstream to a separate OneDrive for Business account and am getting the same result

5

Does the destination directory xyz exist?

6

Yes it does exist

7

Can you try using main:xyz instead of main:/xyz - I wonder if that is what is confusing things.

In general remotes should not have a / after the :. On a small number of remotes (ftp, sftp, local, dropbox) it means something different.

8

Same issue unfortunately

9

As mentioned in my second comment, the command somehow works if I add a third, non-OneDrive for Business upstream, so I doubt it's got to do with my config

10

Can you run rclone about on both the upstreams?

rclone about main:
rclone about main:/xyz

If it is the MFS policy misbehaving then it is something to do with the sizes reported.

I wonder if it is getting confused because both the sizes reported for your remotes will always be identical?

11

here's the output I get for both commands

Total:   1 TiB
Used:    1.453 TiB
Free:    0 B
Trashed: 43.033 GiB
12

I've also tried setting a completely different oneDrive for Business account as the 2nd remote : same behaviour (object not found)

13

That says it has no bytes free. This means none of the remotes rclone is trying to use has free space to store things on - this will give the error you see.

14

Yeah that rclone about command seems wrong for 2 reasons

  1. When I login to the storage metrics page on my OneDrive account, I see I have 122GB free, so I'm not sure how rclone calculates free space
  2. If it really was a free space issue, then the commands in my second comment should also have given the same error