Upgraded from 1.45 to 1.54...now cant create new directory within union mount

[Saur]

What is the problem you are having with rclone?

Been using the same command for years, upgraded to 1.54 and am unable to create new files in the mount getting the error Dir.Mkdir failed to create directory: object not found When I downgrade back to 1.45, I do not receive the same issue, and I can both read and write new directories. Not sure why this is happening, "Encrypted 2" is in the root of gdrive.

What is your rclone version?

rclone v1.54.0

• os/arch: windows/amd64
• go version: go1.15.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 Pro 64 bit 20H2

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount union:/ N: --vfs-read-chunk-size 8M --vfs-read-chunk-size-limit 128M --dir-cache-time 8760h --buffer-size 0M --volname "Media Drive" --tpslimit 10 --poll-interval 15s

I use this command to upload using Task Scheduler:

rclone move "E:\PendingDriveUploads" PlexCrypt:/ --tpslimit 10 --delete-empty-src-dirs --min-age 5m --transfers 8 --checkers 10 --buffer-size 0M

The rclone config contents with secrets removed.

[gdrive]
type = drive
client_id = .apps.googleusercontent.com
client_secret = SECRET
scope = drive
token = TOKEN
root_folder_id = ROOT

[PlexCrypt]
type = crypt
remote = gdrive:/Encrypted 2/
filename_encryption = standard
directory_name_encryption = true

[union]
type = union
remotes = PlexCrypt:/ E:\PendingDriveUploads\

A log from the command with the -vv flag

2021/02/12 20:11:49 DEBUG : Creating backend with remote "union:/"
2021/02/12 20:11:49 DEBUG : Using config file from "C:\\Users\\Plex\\.config\\rclone\\rclone.conf"
2021/02/12 20:11:49 INFO  : Starting transaction limiter: max 10 transactions/s with burst 1
2021/02/12 20:11:49 DEBUG : Creating backend with remote "E:/PendingDriveUploads/"
2021/02/12 20:11:49 DEBUG : Creating backend with remote "PlexCrypt:/"
2021/02/12 20:11:49 DEBUG : fs cache: renaming cache item "E:/PendingDriveUploads/" to be canonical "//?/E:/PendingDriveUploads/"
2021/02/12 20:11:49 DEBUG : Creating backend with remote "E:/PendingDriveUploads"
2021/02/12 20:11:49 DEBUG : fs cache: renaming cache item "E:/PendingDriveUploads" to be canonical "//?/E:/PendingDriveUploads"
2021/02/12 20:11:49 DEBUG : Creating backend with remote "gdrive:/Encrypted 2"
2021/02/12 20:11:49 DEBUG : fs cache: renaming cache item "gdrive:/Encrypted 2" to be canonical "gdrive:Encrypted 2"
2021/02/12 20:11:49 DEBUG : fs cache: switching user supplied name "gdrive:/Encrypted 2" for canonical name "gdrive:Encrypted 2"
2021/02/12 20:11:49 DEBUG : Creating backend with remote "PlexCrypt:"
2021/02/12 20:11:50 DEBUG : Creating backend with remote "gdrive:/Encrypted 2/"
2021/02/12 20:11:50 DEBUG : fs cache: renaming cache item "gdrive:/Encrypted 2/" to be canonical "gdrive:Encrypted 2"
2021/02/12 20:11:50 DEBUG : union root '/': actionPolicy = *policy.EpAll, createPolicy = *policy.EpMfs, searchPolicy = *policy.FF
2021/02/12 20:11:50 INFO  : union root '/': poll-interval is not supported by this remote
2021/02/12 20:11:50 DEBUG : Network mode mounting is disabled
2021/02/12 20:11:50 DEBUG : Mounting on "N:" ("Media Drive")
2021/02/12 20:11:50 DEBUG : union root '/': Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=Media Drive"]
2021/02/12 20:11:50 DEBUG : union root '/': Init:
2021/02/12 20:11:50 DEBUG : union root '/': >Init:
2021/02/12 20:11:50 DEBUG : /: Statfs:
2021/02/12 20:11:50 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:278456629688 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/02/12 20:11:50 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:50 DEBUG : /: >Getattr: errc=0
2021/02/12 20:11:50 DEBUG : /: Readlink:
2021/02/12 20:11:50 DEBUG : /: >Readlink: linkPath="", errc=-40
The service rclone has been started.
2021/02/12 20:11:50 DEBUG : /: Statfs:
2021/02/12 20:11:50 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:278456629688 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/02/12 20:11:50 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:50 DEBUG : /: >Getattr: errc=0
2021/02/12 20:11:50 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:50 DEBUG : /: >Getattr: errc=0
2021/02/12 20:11:50 DEBUG : /: Opendir:
2021/02/12 20:11:50 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:11:50 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:11:50 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/02/12 20:11:50 DEBUG : /: Releasedir: fh=0x0
2021/02/12 20:11:50 DEBUG : /: >Releasedir: errc=0
2021/02/12 20:11:50 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:50 DEBUG : /: >Getattr: errc=0
2021/02/12 20:11:50 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:50 DEBUG : /: >Getattr: errc=0
2021/02/12 20:11:50 DEBUG : /: Opendir:
2021/02/12 20:11:50 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:11:50 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:11:50 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/02/12 20:11:50 DEBUG : /: Releasedir: fh=0x0
2021/02/12 20:11:50 DEBUG : /: >Releasedir: errc=0
2021/02/12 20:11:50 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:50 DEBUG : /: >Getattr: errc=0
2021/02/12 20:11:50 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:50 DEBUG : /: >Getattr: errc=0
2021/02/12 20:11:50 DEBUG : /: Opendir:
2021/02/12 20:11:50 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:11:50 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:11:50 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/02/12 20:11:50 DEBUG : /: Releasedir: fh=0x0
2021/02/12 20:11:50 DEBUG : /: >Releasedir: errc=0
2021/02/12 20:11:53 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:53 DEBUG : /autorun.inf: >Getattr: errc=-2
2021/02/12 20:11:53 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:53 DEBUG : /autorun.inf: >Getattr: errc=-2
2021/02/12 20:11:53 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:53 DEBUG : /autorun.inf: >Getattr: errc=-2
2021/02/12 20:11:53 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:53 DEBUG : /: >Getattr: errc=0
2021/02/12 20:11:53 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:53 DEBUG : /: >Getattr: errc=0
2021/02/12 20:11:53 DEBUG : /: Opendir:
2021/02/12 20:11:53 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:11:53 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:11:53 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/02/12 20:11:53 DEBUG : /: Releasedir: fh=0x0
2021/02/12 20:11:53 DEBUG : /: >Releasedir: errc=0
2021/02/12 20:11:53 DEBUG : /AutoRun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:11:53 DEBUG : /AutoRun.inf: >Getattr: errc=-2
2021/02/12 20:12:03 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:03 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:03 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:03 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:03 DEBUG : /: Opendir:
2021/02/12 20:12:03 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:12:03 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:12:03 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/02/12 20:12:03 DEBUG : /: Statfs:
2021/02/12 20:12:03 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:278456629688 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/02/12 20:12:03 DEBUG : /: Releasedir: fh=0x0
2021/02/12 20:12:03 DEBUG : /: >Releasedir: errc=0
2021/02/12 20:12:03 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:03 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:03 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:03 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:03 DEBUG : /: Opendir:
2021/02/12 20:12:03 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:12:03 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:12:03 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/02/12 20:12:04 DEBUG : /: Releasedir: fh=0x0
2021/02/12 20:12:04 DEBUG : /: >Releasedir: errc=0
2021/02/12 20:12:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:04 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:04 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:04 DEBUG : /: Opendir:
2021/02/12 20:12:04 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:12:04 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:12:04 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/02/12 20:12:04 DEBUG : /: Releasedir: fh=0x0
2021/02/12 20:12:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:04 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:04 DEBUG : /: >Releasedir: errc=0
2021/02/12 20:12:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:04 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:04 DEBUG : /: Opendir:
2021/02/12 20:12:04 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:12:04 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:12:04 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/02/12 20:12:04 DEBUG : /: Releasedir: fh=0x0
2021/02/12 20:12:04 DEBUG : /: >Releasedir: errc=0
2021/02/12 20:12:06 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:06 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:06 DEBUG : /desktop.ini: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:06 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:06 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:06 DEBUG : /desktop.ini: >Getattr: errc=-2
2021/02/12 20:12:06 DEBUG : /: Opendir:
2021/02/12 20:12:06 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:06 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:06 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:12:06 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:12:06 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:06 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/02/12 20:12:06 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:06 DEBUG : /: Releasedir: fh=0x0
2021/02/12 20:12:06 DEBUG : /: >Releasedir: errc=0
2021/02/12 20:12:06 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:06 DEBUG : /: Opendir:
2021/02/12 20:12:06 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:12:06 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:12:06 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/02/12 20:12:06 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:06 DEBUG : /: Readdir: ofst=0, fh=0x0
2021/02/12 20:12:06 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:06 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:06 DEBUG : /: Opendir:
2021/02/12 20:12:06 DEBUG : /: >Readdir: items=9, errc=0
2021/02/12 20:12:06 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:12:06 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:12:06 DEBUG : /: Releasedir: fh=0x0
2021/02/12 20:12:06 DEBUG : /: >Opendir: errc=0, fh=0x1
2021/02/12 20:12:06 DEBUG : /: >Releasedir: errc=0
2021/02/12 20:12:06 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:06 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:06 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:06 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:06 DEBUG : /: Opendir:
2021/02/12 20:12:06 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:12:06 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:12:06 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/02/12 20:12:06 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:06 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:06 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/12 20:12:06 DEBUG : /: >Getattr: errc=0
2021/02/12 20:12:06 DEBUG : /: Opendir:
2021/02/12 20:12:06 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/02/12 20:12:06 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/02/12 20:12:06 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/02/12 20:12:06 DEBUG : /: Statfs:
2021/02/12 20:12:06 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:278456629688 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0

[klunky]

Check if it's not related with

Rclone mount :: Fatal error: failed to mount FUSE fs: kernel FUSE version is too old: 7.14 < 7.17

What is the problem you are having with rclone? I have a situation to mount synology webdav on CentOS I've done rclone config and "rclone ls" is working When I try "rclone mount", I get below error message. [root@front rclone]# rclone mount nas:/storage/ /mnt/nas/ --vfs-cache-mode writes 2021/02/12 13:06:38 Fatal error: failed to mount FUSE fs: kernel FUSE version is too old: 7.14 < 7.17 I tried to find FUSE over 7.17, there is only 3.10 Which FUSE version should I use? [root@front rclone…

[ncw (Nick Craig-Wood)]

I think this is probably because of changes in the union backend between those two versions.

I could replicate this problem though.

Can you send a log with -vv which includes the error Dir.Mkdir failed to create directory: object not found as I don't see it in your log above.

If necessary put it on pastebin if it is too long to attach.

[Saur]

Here is the command running 1.54 with -vv

https://pastebin.com/BFbZQCAg

[Saur]

After trying to update to the latest version that will work with my command, the last working version is 1.51, 1.52 results in the same error.

[ncw (Nick Craig-Wood)]

That log is very useful, thanks.

It appeared to be failing to create a directory called New.

Did that directory exist on cloud storage? How about locally?

What did you do to cause the error?

Thanks.

[Saur]

From what I see, I have no existing directories in the root of the remote called "New", neither locally stored, or on the cloud storage.

All I had to do to create the error was click "New Folder", window then displays an error stating:

"Could not find this item. This is no longer located in This PC. Verify the items location and try again."

I use the Union mount with the VFS remote to chunk incoming data, merging it with a local folder to store files to be read, locally cached, then run the rclone move from that folder nightly. No other changes have been in the process of how I've been using rclone except for updating it.

I am including a log from version 1.51, the last update that seems to work. In case it may help.
https://pastebin.com/WU51u5nz

[ncw (Nick Craig-Wood)]

I can't replicate this on my Windows test machine, alas.

Can you try making the directory from the cmd window so open cmd, type "mkdir E:\New7" and see if that works?

I think this might be explorer doing something odd.

[Saur]

So, I have tested making a new directory into the local filesystem using command as you requested, it works flawlessly. Even making a new directory into the folder "E:\PendingDriveUploads" where my union uploads go.

However, using mkdir N:\New in the union mount, I get the error, "The system cannot find the file specified."

Back tested it using 1.51, and there are no errors, and the mount is working as expected.

[ncw (Nick Craig-Wood)]

I finally managed to replicate this

[uniontest2]
type = union
remotes = C:\local2 C:\local

rclone mount uniontest2:/ N: --config Z:\.rclone.conf -vv

And replicated with

C:\Users\Dev>mkdir N:\New
The system cannot find the file specified.

The logs generated when the mkdir is run are

2021/02/16 12:36:52 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/02/16 12:36:52 DEBUG : /: >Getattr: errc=0
2021/02/16 12:36:52 DEBUG : /New: Mkdir: mode=0700
2021/02/16 12:36:52 ERROR : /: Dir.Mkdir failed to create directory: object not found
2021/02/16 12:36:52 DEBUG : /New: >Mkdir: errc=-2

The key to the replication is the trailing / on uniontest2:/

And that turned out to be the bug (after about 2 hours of investigation!)...

Can you give this a go please?

v1.55.0-beta.5196.39c866b08.fix-union-mkdir on branch fix-union-mkdir (uploaded in 15-30 mins)

If this fixes the problem then I'll stick it in a 1.54.1

[Saur]

That fixed it up. It works like a charm!!!

Thank you very much. I appreciate your assistance!!!

[ncw (Nick Craig-Wood)]

Thanks for testing.

I've merged this to master now which means it will be in the latest beta in 15-30 mins and released in v1.54.1