Help with a union mount - IO error and failed to write to cache file

Help and Support
1

What is the problem you are having with rclone?

I am getting various errors on my mount for example:

2024/09/18 23:55:33 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory

2024/09/18 22:32:46 ERROR : TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: lstat /mnt/user/plexdata/TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: no such file or directory

As you can see below in my commands, /mnt/user/plexdata is my local array on my system, /mnt/nvmedl/plexified/mounts/google is my google mount, and the /mnt/nvmedl/plexified/moviesource/Movies/ folder is my movies combined into one folder (they are split due to folder limitations)

I am using my end output to unionize all 3 of the folders (/mnt/user/plexdata/, /mnt/nvmedl/plexified/mounts/google/, /mnt/nvmedl/plexified/moviesource/Movies/) into one end directory /mnt/nvmedl/plexified/secret/ - this is what Plex and Sonarr/Radarr use to import. I have a :nc on both folders except for the plexdata folder.

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.1

  • os/version: slackware 15.0+ (64 bit)
  • os/kernel: 6.8.12-Unraid (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.21.5
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount --config=/mnt/nvme/plexified/mounts/rclone/rclone.conf --allow-other --no-traverse --vfs-cache-mode full --cache-dir /mnt/nvmedl/plexified/mounts/googlecache/ --vfs-cache-max-size 250G --dir-cache-time 96h --vfs-fast-fingerprint --vfs-refresh --drive-impersonate admin@domain.com googledecrypted: /mnt/nvmedl/plexified/mounts/google/

rclone --config=/mnt/nvme/plexified/mounts/rclone/rclone.conf mount --allow-other --allow-non-empty moviesunion: /mnt/nvmedl/plexified/moviesource/Movies/

rclone --config=/mnt/nvme/plexified/mounts/rclone/rclone.conf mount --allow-other --allow-non-empty --no-traverse --vfs-cache-mode full --cache-dir /mnt/nvmedl/plexified/mounts/unioncache/ --vfs-cache-max-size 250G --dir-cache-time 96h --vfs-fast-fingerprint --vfs-refresh --drive-impersonate admin@domain.com unionized: /mnt/nvmedl/plexified/secret/

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[google]
type = drive
scope = drive
service_account_file = /mnt/nvme/plexified/mounts/rclonekeys/humbleproject-825a726988cb.json
team_drive = XXX
root_folder_id = XXX

[googledecrypted]
type = crypt
remote = google:
filename_encryption = standard
password = XXX
password2 = XXX

[moviesunion]
type = union
upstreams = /mnt/nvmedl/plexified/mounts/google/Data/MoviesSrc/0000/ /mnt/nvmedl/plexified/mounts/google/Data/MoviesSrc/0001 /mnt/nvmedl/plexified/mounts/google/Data/MoviesSrc/0002/
action_policy = ff
create_policy = ff
search_policy = all

[unionized]
type = union
upstreams = /mnt/user/plexdata/ /mnt/nvmedl/plexified/mounts/google/Data:nc /mnt/nvmedl/plexified/moviesource:nc
action_policy = ff
create_policy = ff
search_policy = ff

A log from the command that you were trying to run with the -vv flag

2024/09/18 22:32:46 ERROR : TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: lstat /mnt/user/plexdata/TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: no such file or directory
2024/09/18 22:32:46 ERROR : TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: vfs cache: too many errors 13/10: last error: vfs reader: failed to write to cache file: lstat /mnt/user/plexdata/TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: no such file or directory
2024/09/18 22:32:46 ERROR : TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: vfs cache: failed to kick waiters: vfs reader: failed to write to cache file: lstat /mnt/user/plexdata/TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: no such file or directory
2024/09/18 22:32:46 ERROR : TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: vfs cache: failed to _ensure cache vfs reader: failed to write to cache file: lstat /mnt/user/plexdata/TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: no such file or directory
2024/09/18 22:32:46 ERROR : IO error: vfs reader: failed to write to cache file: lstat /mnt/user/plexdata/TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: no such file or directory
2024/09/18 22:32:46 ERROR : TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: lstat /mnt/user/plexdata/TVDocs/Gogglebox Australia (2015)/Season 10/Gogglebox Australia - S10E09 - HDTV-1080p - CCT.mkv: no such file or directory
2024/09/18 23:46:44 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2024/09/18 23:46:50 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2024/09/18 23:47:53 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2024/09/18 23:47:59 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2024/09/18 23:48:01 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2024/09/18 23:48:03 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2024/09/18 23:48:49 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2024/09/18 23:51:51 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2024/09/18 23:52:21 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2024/09/18 23:52:26 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2024/09/18 23:54:31 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2024/09/18 23:55:33 ERROR : IO error: 1 error: local: remove /mnt/user/plexdata/TVDocs/The Repair Shop/Season 13/The Repair Shop - S13E12 - WEBRip-480p - Z-B.mp4: no such file or directory
2

hi,
real complex setup, with a lot of moving parts.

might want to rclone selfupdate and test again.

3

Yes, I agree it is a complex setup. I was previously using mergerfs but there is a lot of commands in regards to that so I switched to rclone union.

I will try updating rclone, once updated will I need to re-run the existing mounts for them to use the new version?

Is there anything obvious that can be done to my 3 rclone commands to improve the setup somewhat?

4

imho, no point discussing that.

this is your second topic about the same basic setup. the first topic went unanswered.
in both topics, you are running an old version of rclone.

5

Alright, I've updated and will leave the commands mounted and see how it goes over the next 24 hours or so!

When I upload from plexdata (local) to google (cloud), how long does the union mount take to update and see the new files from the google mount?

6

I can't quite figure out your workflow.

That is generally bad as it allows for over mounting and hiding things.

What's the goal of the union?

I can't imagine this does anything on a mount.

Plex easily allows for multiple folders in a library and is a much easier/simple setup but I imagine you did this for something.

1 Like
7

The way it is set up now is so that I can have my sonarr/radarr renaming my files and sending them to my array with the :nc on both google mounts (/plexdata) and then I can merge the array with the google mount into one folder using rclone union, which then plex can see one entire structure, and it wouldn't matter if it is hosted on google, or local disks as it would still show in the same location at the end. I send stuff to google occasionally to free up space on my array as and when needed.

Basically the end folder is what Plex/Sonarr/Radarr sees and all 3 tools can do what they need to do as though it was one storage drive and it wouldn't matter if it was on Google or Local drives.

8

really, i was just thinking
i know this expert about transitioning from mergerfs, union and mounts, that this mysterious expert would not appear until the OP updated...

9

I may be wrong but, I did update the other day and I am wondering if the update doesn't persist between rebooting the UNRAID server, I've known this happen to a few things previously because of how UNRAID stores most stuff on boot in RAM.

10

Why do you have 3 mounts though?

I did the same using mergerfs.

If you are using rclone though, you'd only need one union mount, which is why I'm having a tough time following.

You'd union the local and the remote together for one mount.

That error means the cache dir got something removed usually as you have some data on the mount that should be there, but it doesn't find the chunks in the cache area.

Is that cache area persisting on reboots and not getting cleaned up?

11

So I was having issues with a large Movies directory, so I split them up into 4 separate folders, so I merge them into "Movies" first. Hence the 3 directories unionized, plexdata (local), Movies (movies merged) and then Google (cloud)

The cache doesn't seem to be persisting on reboots.

12

@asdffdsa I am still having this issue even after updating. Another extra bit of info. Not seeing any cache issues anymore, although I'm getting a "WriteFileHandle.Write: error: Bad file descriptor" error now.

2024/09/20 22:39:42 ERROR : TVShows/Prisoner: Cell Block H/Season 01/Prisoner - Cell Block H - S01E06 - WEBDL-480p - WH.mp4: WriteFileHandle.Write: error: Bad file descriptor```
14

So where it resides?

Is for example /mnt/nvmedl/plexified/mounts/googlecache/ RAM disk?

15

If the cache area is being removed, you are going to get errors/data loss most likely. You'd want to fix that first.

16

No, its location is on an NVME drive.

17

So how come it is not persistent?

18

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.