Directory not found in union of OneDrives

Help and Support
1

What is the problem you are having with rclone?

I am having a few manifestations of what I suspect is the same issue, but for now, when listing my union remote, I am getting directory not found on the listing.

For listing, to not find a directory is suspicious. This may be a bug.

Run the command 'rclone version' and share the full output of the command.

rclone v1.69.1
- os/version: darwin 14.6.1 (64 bit)
- os/kernel: 23.6.0 (arm64)
- os/type: darwin
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.24.0
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Union of two OneDrives

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone lsjson \
    --filter '+ /.syncrclone/LOCK/*' \
    --filter '- /.syncrclone/**' \
    --hash \
    --skip-links \
    --filter '- .PyFiSync/**' \
    --filter '- .syncrclone/**' \
    --filter '- .DS_Store' \
    --filter '- /Icon*' \
    --filter '- /Pictures/Camera Roll/**' \
    --exclude-if-present .ignore \
    -R \
    --no-mimetype --files-only \
    onedrive: -vv 1> test.out 2> test.err```

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[onedrive]
type = union
upstreams = onedrive1: onedrive2:
create_policy = epall
action_policy = epall
search_policy = ff

[onedrive1]
type = onedrive
token = XXX
drive_id = XXX
drive_type = personal
client_id = XXX
client_secret = XXX
description = XXX

[onedrive2]
type = onedrive
client_id = XXX
client_secret = XXX
token = XXX
drive_id = XXX
drive_type = personal
description = XXX

A log from the command that you were trying to run with the -vv flag

This runs sometimes nearly to the end--which depending on OneDrive's rate limiting, can be 10 minutes to 2 hours--and sometimes the error comes earlier. This is the main part with DEBUG for exclusions removes

2025/03/23 08:01:40 DEBUG : Creating backend with remote "onedrive:"
2025/03/23 08:01:40 DEBUG : Using config file from "/Users/jwinokur/.config/rclone/rclone.conf"
2025/03/23 08:01:40 DEBUG : Creating backend with remote "onedrive2:"
2025/03/23 08:01:40 DEBUG : Creating backend with remote "onedrive1:"
2025/03/23 08:01:40 DEBUG : union root '': actionPolicy = *policy.EpAll, createPolicy = *policy.EpAll, searchPolicy = *policy.FF
2025/03/23 08:01:42 DEBUG : Justin's Notebook: OneNote file not shown in directory listing
...
2025/03/23 08:05:05 ERROR : personal_files/ipynb: error listing: 1 error: onedrive2: directory not found

I then canceled it at the error but in the logs from prior

NOTICE: Failed to lsjson with 2 errors: last error was: error in ListJSON: 1 error: onedrive2: directory not found

(This was originally run inside my sync tool so it is hard to recreate).

By the way,

$ rclone lsf onedrive:personal_files/ipynb
$ rclone lsf onedrive1:personal_files/ipynb

works and, as expected,

$ rclone lsf onedrive2:personal_files/ipynb

Throws the error. But this is expected since it's not in that OneDrive

2

I wonder if it is related to your union policies and non policy compliant content:

create_policy = epall
action_policy = epall
search_policy = ff

I think that with above config when you create new directory it is created on all remotes. So maybe missing path on onedrive2 is breaking something? As in such case this situation should not happen (some union member without given dir)?

I have never used epall and its definition in docs is not 100% clear for me:

Search category: Given this order configured, act on the first one found where the relative path exists. Action category: apply to all found. Create category: act on all upstreams where the relative path exists.

With Onedrive I use (without any issues):

action_policy = epall
create_policy = rand
search_policy = ff

as I found that all space based policies are useless. rclone about stats are so far from real time that I trust rand more:)

3

I am not sure what you mean by this.

Agreed but that would be a bug, right?

My attempts to isolate the issue(s) haven't worked well as they take a very long time to manifest! I tried turning the cache to a super long value and that didn't help either.

For now, my workaround is to forgo the union since I had a pretty good separation at the top level already but I would like to, eventually, figure out what is going on.

Interesting idea. For now, this union is pretty clean but if I ever need a lot more than what I have, I will look into this.

I've also noticed a huge performance hit with Union on file listing. I thought the cache would have fixed that but again, it is hard to isolate.

Thanks for the engagement. Maybe there will be more ideas but if not, I will have to stick with the workaround until I can manage to create a more testable error case.